[PATCH 0/5] MCPM backend for Exynos5420

[PATCH 0/5] MCPM backend for Exynos5420

[Abhilash Kesavan]

This series adds the MCPM backend support for SMP secondary boot and core
switching on Samsung's Exynos5420. The patches are based on the mcpm
support added for Exynos5420 in the Chromium kernel repository here:
https://chromium.googlesource.com/chromiumos/third_party/kernel-next/+/chromeos-3.8

This patchset depends on:
ARM: EXYNOS5: Add PMU settings for exynos5420
(https://patchwork.kernel.org/patch/3896471/)

The patches have been prepared on Kgene's for-next branch and tested on
SMDK5420 EVT1 using the "/dev/b.L_switcher" user interface.

Abhilash Kesavan (2):
  ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM
  arm: exynos: Add MCPM call-back functions

Andrew Bresticker (3):
  ARM: bL_switcher: Don't enable bL switcher on systems without CCI
  drivers/bus: arm-cci: Add common control interface for ACE ports
  ARM: dts: exynos5420: add CCI node

 arch/arm/boot/dts/exynos5420.dtsi        |   27 ++
 arch/arm/common/bL_switcher.c            |   12 +
 arch/arm/mach-exynos/Kconfig             |    9 +
 arch/arm/mach-exynos/Makefile            |    2 +
 arch/arm/mach-exynos/common.h            |    1 +
 arch/arm/mach-exynos/exynos.c            |   12 +
 arch/arm/mach-exynos/include/mach/map.h  |    1 +
 arch/arm/mach-exynos/mcpm-exynos-setup.S |   35 +++
 arch/arm/mach-exynos/mcpm-exynos.c       |  444 ++++++++++++++++++++++++++++++
 arch/arm/mach-exynos/platsmp.c           |   19 ++
 arch/arm/mach-exynos/regs-pmu.h          |    2 +
 drivers/bus/arm-cci.c                    |   13 +-
 include/linux/arm-cci.h                  |    7 +-
 13 files changed, 576 insertions(+), 8 deletions(-)
 create mode 100644 arch/arm/mach-exynos/mcpm-exynos-setup.S
 create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c

-- 
1.7.9.5

[PATCH 1/5] ARM: bL_switcher: Don't enable bL switcher on systems without CCI

[Abhilash Kesavan]

From: Andrew Bresticker <abrestic@chromium.org>

Do not enable the big.LITTLE switcher on systems that do not have an
ARM CCI (cache-coherent interconnect) present.  Since the CCI is used
to maintain cache coherency between multiple clusters and peripherals,
it is unlikely that a system without CCI would support big.LITTLE.

Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 arch/arm/common/bL_switcher.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/common/bL_switcher.c b/arch/arm/common/bL_switcher.c
index 5774b6e..c4fec1f 100644
--- a/arch/arm/common/bL_switcher.c
+++ b/arch/arm/common/bL_switcher.c
@@ -33,6 +33,7 @@
 #include <linux/sysfs.h>
 #include <linux/irqchip/arm-gic.h>
 #include <linux/moduleparam.h>
+#include <linux/of.h>
 
 #include <asm/smp_plat.h>
 #include <asm/cputype.h>
@@ -796,6 +797,17 @@ core_param(no_bL_switcher, no_bL_switcher, bool, 0644);
 static int __init bL_switcher_init(void)
 {
 	int ret;
+	struct device_node *dn;
+
+	/*
+	 * We don't want to set up the bL switcher if the machine doesn't
+	 * support bL, so use the presence of a CCI to indicate whether or
+	 * not bL is supported on this device.
+	 */
+	dn = of_find_compatible_node(NULL, NULL, "arm,cci-400");
+	if (!dn)
+		return 0;
+	of_node_put(dn);
 
 	if (MAX_NR_CLUSTERS != 2) {
 		pr_err("%s: only dual cluster systems are supported\n", __func__);
-- 
1.7.9.5

[PATCH 1/5] ARM: bL_switcher: Don't enable bL switcher on systems without CCI

[Nicolas Pitre]

On Fri, 11 Apr 2014, Abhilash Kesavan wrote:

> From: Andrew Bresticker <abrestic@chromium.org>
> 
> Do not enable the big.LITTLE switcher on systems that do not have an
> ARM CCI (cache-coherent interconnect) present.  Since the CCI is used
> to maintain cache coherency between multiple clusters and peripherals,
> it is unlikely that a system without CCI would support big.LITTLE.
> 
> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>

The b.L switcher depends on MCPM, and it also expects only 2 clusters 
which is evaluated at run time or it bails out.

There might be in theory other ways than the CCI to enforce coherency 
between clusters.  And that should all be encapsulated by the MCPM 
layer.  The switcher module should not be concerned at all by the 
underlying hardware mechanism.

So if the goal is to determine at run time whether or not the switcher 
should be activated in a multi-config kernel, then the criteria should 
be whether or not MCPM is initialized, and not if there is a CCI.


> ---
>  arch/arm/common/bL_switcher.c |   12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/arch/arm/common/bL_switcher.c b/arch/arm/common/bL_switcher.c
> index 5774b6e..c4fec1f 100644
> --- a/arch/arm/common/bL_switcher.c
> +++ b/arch/arm/common/bL_switcher.c
> @@ -33,6 +33,7 @@
>  #include <linux/sysfs.h>
>  #include <linux/irqchip/arm-gic.h>
>  #include <linux/moduleparam.h>
> +#include <linux/of.h>
>  
>  #include <asm/smp_plat.h>
>  #include <asm/cputype.h>
> @@ -796,6 +797,17 @@ core_param(no_bL_switcher, no_bL_switcher, bool, 0644);
>  static int __init bL_switcher_init(void)
>  {
>  	int ret;
> +	struct device_node *dn;
> +
> +	/*
> +	 * We don't want to set up the bL switcher if the machine doesn't
> +	 * support bL, so use the presence of a CCI to indicate whether or
> +	 * not bL is supported on this device.
> +	 */
> +	dn = of_find_compatible_node(NULL, NULL, "arm,cci-400");
> +	if (!dn)
> +		return 0;
> +	of_node_put(dn);
>  
>  	if (MAX_NR_CLUSTERS != 2) {
>  		pr_err("%s: only dual cluster systems are supported\n", __func__);
> -- 
> 1.7.9.5
> 

[PATCH 1/5] ARM: bL_switcher: Don't enable bL switcher on systems without CCI

[Abhilash Kesavan]

Hi Nicolas,

On Fri, Apr 11, 2014 at 11:44 PM, Nicolas Pitre
<nicolas.pitre@linaro.org> wrote:
> On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>
>> From: Andrew Bresticker <abrestic@chromium.org>
>>
>> Do not enable the big.LITTLE switcher on systems that do not have an
>> ARM CCI (cache-coherent interconnect) present.  Since the CCI is used
>> to maintain cache coherency between multiple clusters and peripherals,
>> it is unlikely that a system without CCI would support big.LITTLE.
>>
>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>
> The b.L switcher depends on MCPM, and it also expects only 2 clusters
> which is evaluated at run time or it bails out.
>
> There might be in theory other ways than the CCI to enforce coherency
> between clusters.  And that should all be encapsulated by the MCPM
> layer.  The switcher module should not be concerned at all by the
> underlying hardware mechanism.
>
> So if the goal is to determine at run time whether or not the switcher
> should be activated in a multi-config kernel, then the criteria should
> be whether or not MCPM is initialized, and not if there is a CCI.

Yes, we have a multi-SoC enabled kernel (with MCPM and BL_SWITCHER
configs enabled); one SoC has a single cluster while the other is a
dual cluster. We wanted a run-time check to prevent bL_switcher from
running on the single cluster one and zeroed in on CCI. But, I get
your point as to why CCI should not be used as a distinguishing factor
for switcher initialization.

For now, I can use the no_bL_switcher parameter to disable it on
certain platforms. However, can you elaborate on the MCPM
initialization check you suggested ?


Regards,
Abhilash
>
>> ---
>>  arch/arm/common/bL_switcher.c |   12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/arch/arm/common/bL_switcher.c b/arch/arm/common/bL_switcher.c
>> index 5774b6e..c4fec1f 100644
>> --- a/arch/arm/common/bL_switcher.c
>> +++ b/arch/arm/common/bL_switcher.c
>> @@ -33,6 +33,7 @@
>>  #include <linux/sysfs.h>
>>  #include <linux/irqchip/arm-gic.h>
>>  #include <linux/moduleparam.h>
>> +#include <linux/of.h>
>>
>>  #include <asm/smp_plat.h>
>>  #include <asm/cputype.h>
>> @@ -796,6 +797,17 @@ core_param(no_bL_switcher, no_bL_switcher, bool, 0644);
>>  static int __init bL_switcher_init(void)
>>  {
>>       int ret;
>> +     struct device_node *dn;
>> +
>> +     /*
>> +      * We don't want to set up the bL switcher if the machine doesn't
>> +      * support bL, so use the presence of a CCI to indicate whether or
>> +      * not bL is supported on this device.
>> +      */
>> +     dn = of_find_compatible_node(NULL, NULL, "arm,cci-400");
>> +     if (!dn)
>> +             return 0;
>> +     of_node_put(dn);
>>
>>       if (MAX_NR_CLUSTERS != 2) {
>>               pr_err("%s: only dual cluster systems are supported\n", __func__);
>> --
>> 1.7.9.5
>>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 1/5] ARM: bL_switcher: Don't enable bL switcher on systems without CCI

[Nicolas Pitre]

On Thu, 17 Apr 2014, Abhilash Kesavan wrote:

> Hi Nicolas,
> 
> On Fri, Apr 11, 2014 at 11:44 PM, Nicolas Pitre
> <nicolas.pitre@linaro.org> wrote:
> > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
> >
> >> From: Andrew Bresticker <abrestic@chromium.org>
> >>
> >> Do not enable the big.LITTLE switcher on systems that do not have an
> >> ARM CCI (cache-coherent interconnect) present.  Since the CCI is used
> >> to maintain cache coherency between multiple clusters and peripherals,
> >> it is unlikely that a system without CCI would support big.LITTLE.
> >>
> >> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >
> > The b.L switcher depends on MCPM, and it also expects only 2 clusters
> > which is evaluated at run time or it bails out.
> >
> > There might be in theory other ways than the CCI to enforce coherency
> > between clusters.  And that should all be encapsulated by the MCPM
> > layer.  The switcher module should not be concerned at all by the
> > underlying hardware mechanism.
> >
> > So if the goal is to determine at run time whether or not the switcher
> > should be activated in a multi-config kernel, then the criteria should
> > be whether or not MCPM is initialized, and not if there is a CCI.
> 
> Yes, we have a multi-SoC enabled kernel (with MCPM and BL_SWITCHER
> configs enabled); one SoC has a single cluster while the other is a
> dual cluster. We wanted a run-time check to prevent bL_switcher from
> running on the single cluster one and zeroed in on CCI. But, I get
> your point as to why CCI should not be used as a distinguishing factor
> for switcher initialization.
> 
> For now, I can use the no_bL_switcher parameter to disable it on
> certain platforms. However, can you elaborate on the MCPM
> initialization check you suggested ?

Here's what I mean:

----- >8

From: Nicolas Pitre <nicolas.pitre@linaro.org>
Date: Wed, 16 Apr 2014 15:43:59 -0400
Subject: [PATCH] ARM: bL_switcher: fix validation check before its activation

The switcher should not depend on MAX_CLUSTER which is a build time
limit to  determine ifit should be activated or not. In a multiplatform
kernel binary it is possible to have dual-cluster and quad-cluster
platforms configured in. In that case MAX_CLUSTER should be 4 and that
shouldn't prevent the switcher from working if the kernel is booted on a
b.L dual-cluster system.

In bL_switcher_halve_cpus() we already have a runtime validation check
to make sure we're dealing with only two clusters, so booting on a quad
cluster system will be caught and switcher activation aborted.

However, the b.L switcher must ensure the MCPM layer is initialized on
the booted hardware before doing anything.  The mcpm_is_available()
function is added to that effect.

Signed-off-by: Nicolas Pitre <nico@linaro.org>

diff --git a/arch/arm/common/bL_switcher.c b/arch/arm/common/bL_switcher.c
index 5774b6ea7a..f01c0ee0c8 100644
--- a/arch/arm/common/bL_switcher.c
+++ b/arch/arm/common/bL_switcher.c
@@ -797,10 +797,8 @@ static int __init bL_switcher_init(void)
 {
 	int ret;
 
-	if (MAX_NR_CLUSTERS != 2) {
-		pr_err("%s: only dual cluster systems are supported\n", __func__);
-		return -EINVAL;
-	}
+	if (!mcpm_is_available())
+		return -ENODEV;
 
 	cpu_notifier(bL_switcher_hotplug_callback, 0);
 
diff --git a/arch/arm/common/mcpm_entry.c b/arch/arm/common/mcpm_entry.c
index 1e361abc29..86fd60fefb 100644
--- a/arch/arm/common/mcpm_entry.c
+++ b/arch/arm/common/mcpm_entry.c
@@ -48,6 +48,11 @@ int __init mcpm_platform_register(const struct mcpm_platform_ops *ops)
 	return 0;
 }
 
+bool mcpm_is_available(void)
+{
+	return (platform_ops) ? true : false;
+}
+
 int mcpm_cpu_power_up(unsigned int cpu, unsigned int cluster)
 {
 	if (!platform_ops)
diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index 608516ebab..a5ff410dcd 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -54,6 +54,13 @@ void mcpm_set_early_poke(unsigned cpu, unsigned cluster,
  */
 
 /**
+ * mcpm_is_available - returns whether MCPM is initialized and available
+ *
+ * This returns true or false accordingly.
+ */
+bool mcpm_is_available(void);
+
+/**
  * mcpm_cpu_power_up - make given CPU in given cluster runable
  *
  * @cpu: CPU number within given cluster

[PATCH 1/5] ARM: bL_switcher: Don't enable bL switcher on systems without CCI

[Abhilash Kesavan]

Hi Nicolas,

On Thu, Apr 17, 2014 at 1:48 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 17 Apr 2014, Abhilash Kesavan wrote:
>
>> Hi Nicolas,
>>
>> On Fri, Apr 11, 2014 at 11:44 PM, Nicolas Pitre
>> <nicolas.pitre@linaro.org> wrote:
>> > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>> >
>> >> From: Andrew Bresticker <abrestic@chromium.org>
>> >>
>> >> Do not enable the big.LITTLE switcher on systems that do not have an
>> >> ARM CCI (cache-coherent interconnect) present.  Since the CCI is used
>> >> to maintain cache coherency between multiple clusters and peripherals,
>> >> it is unlikely that a system without CCI would support big.LITTLE.
>> >>
>> >> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> >
>> > The b.L switcher depends on MCPM, and it also expects only 2 clusters
>> > which is evaluated at run time or it bails out.
>> >
>> > There might be in theory other ways than the CCI to enforce coherency
>> > between clusters.  And that should all be encapsulated by the MCPM
>> > layer.  The switcher module should not be concerned at all by the
>> > underlying hardware mechanism.
>> >
>> > So if the goal is to determine at run time whether or not the switcher
>> > should be activated in a multi-config kernel, then the criteria should
>> > be whether or not MCPM is initialized, and not if there is a CCI.
>>
>> Yes, we have a multi-SoC enabled kernel (with MCPM and BL_SWITCHER
>> configs enabled); one SoC has a single cluster while the other is a
>> dual cluster. We wanted a run-time check to prevent bL_switcher from
>> running on the single cluster one and zeroed in on CCI. But, I get
>> your point as to why CCI should not be used as a distinguishing factor
>> for switcher initialization.
>>
>> For now, I can use the no_bL_switcher parameter to disable it on
>> certain platforms. However, can you elaborate on the MCPM
>> initialization check you suggested ?
>
> Here's what I mean:
>
> ----- >8
>
> From: Nicolas Pitre <nicolas.pitre@linaro.org>
> Date: Wed, 16 Apr 2014 15:43:59 -0400
> Subject: [PATCH] ARM: bL_switcher: fix validation check before its activation
>
> The switcher should not depend on MAX_CLUSTER which is a build time
> limit to  determine ifit should be activated or not. In a multiplatform
> kernel binary it is possible to have dual-cluster and quad-cluster
> platforms configured in. In that case MAX_CLUSTER should be 4 and that
> shouldn't prevent the switcher from working if the kernel is booted on a
> b.L dual-cluster system.
>
> In bL_switcher_halve_cpus() we already have a runtime validation check
> to make sure we're dealing with only two clusters, so booting on a quad
> cluster system will be caught and switcher activation aborted.
>
> However, the b.L switcher must ensure the MCPM layer is initialized on
> the booted hardware before doing anything.  The mcpm_is_available()
> function is added to that effect.
>
> Signed-off-by: Nicolas Pitre <nico@linaro.org>

Thank you for the explanation and patch. I have tested this on our
multi-platform configuration and it works fine - activating the
switcher on one SoC and not on the other.

Even though it is not the case now, could we have a scenario where we
may use mcpm for only secondary core boot-up on one SoC and for
switching on another ? I would then have mcpm ops populated for both
but still want bL switcher activated for only one of them.

Regards,
Abhilash
>
> diff --git a/arch/arm/common/bL_switcher.c b/arch/arm/common/bL_switcher.c
> index 5774b6ea7a..f01c0ee0c8 100644
> --- a/arch/arm/common/bL_switcher.c
> +++ b/arch/arm/common/bL_switcher.c
> @@ -797,10 +797,8 @@ static int __init bL_switcher_init(void)
>  {
>         int ret;
>
> -       if (MAX_NR_CLUSTERS != 2) {
> -               pr_err("%s: only dual cluster systems are supported\n", __func__);
> -               return -EINVAL;
> -       }
> +       if (!mcpm_is_available())
> +               return -ENODEV;
>
>         cpu_notifier(bL_switcher_hotplug_callback, 0);
>
> diff --git a/arch/arm/common/mcpm_entry.c b/arch/arm/common/mcpm_entry.c
> index 1e361abc29..86fd60fefb 100644
> --- a/arch/arm/common/mcpm_entry.c
> +++ b/arch/arm/common/mcpm_entry.c
> @@ -48,6 +48,11 @@ int __init mcpm_platform_register(const struct mcpm_platform_ops *ops)
>         return 0;
>  }
>
> +bool mcpm_is_available(void)
> +{
> +       return (platform_ops) ? true : false;
> +}
> +
>  int mcpm_cpu_power_up(unsigned int cpu, unsigned int cluster)
>  {
>         if (!platform_ops)
> diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
> index 608516ebab..a5ff410dcd 100644
> --- a/arch/arm/include/asm/mcpm.h
> +++ b/arch/arm/include/asm/mcpm.h
> @@ -54,6 +54,13 @@ void mcpm_set_early_poke(unsigned cpu, unsigned cluster,
>   */
>
>  /**
> + * mcpm_is_available - returns whether MCPM is initialized and available
> + *
> + * This returns true or false accordingly.
> + */
> +bool mcpm_is_available(void);
> +
> +/**
>   * mcpm_cpu_power_up - make given CPU in given cluster runable
>   *
>   * @cpu: CPU number within given cluster

[PATCH 1/5] ARM: bL_switcher: Don't enable bL switcher on systems without CCI

[Nicolas Pitre]

On Mon, 21 Apr 2014, Abhilash Kesavan wrote:

> Hi Nicolas,
> 
> On Thu, Apr 17, 2014 at 1:48 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > Here's what I mean:
> >
> > ----- >8
> >
> > From: Nicolas Pitre <nicolas.pitre@linaro.org>
> > Date: Wed, 16 Apr 2014 15:43:59 -0400
> > Subject: [PATCH] ARM: bL_switcher: fix validation check before its activation
> >
> > The switcher should not depend on MAX_CLUSTER which is a build time
> > limit to  determine ifit should be activated or not. In a multiplatform
> > kernel binary it is possible to have dual-cluster and quad-cluster
> > platforms configured in. In that case MAX_CLUSTER should be 4 and that
> > shouldn't prevent the switcher from working if the kernel is booted on a
> > b.L dual-cluster system.
> >
> > In bL_switcher_halve_cpus() we already have a runtime validation check
> > to make sure we're dealing with only two clusters, so booting on a quad
> > cluster system will be caught and switcher activation aborted.
> >
> > However, the b.L switcher must ensure the MCPM layer is initialized on
> > the booted hardware before doing anything.  The mcpm_is_available()
> > function is added to that effect.
> >
> > Signed-off-by: Nicolas Pitre <nico@linaro.org>
> 
> Thank you for the explanation and patch. I have tested this on our
> multi-platform configuration and it works fine - activating the
> switcher on one SoC and not on the other.

Good.  It is in RMK's patch system now.

> Even though it is not the case now, could we have a scenario where we
> may use mcpm for only secondary core boot-up on one SoC and for
> switching on another ? I would then have mcpm ops populated for both
> but still want bL switcher activated for only one of them.

Absolutely.  As soon as you have cluster synchronization issues, you 
must use MCPM.  The switcher is just one amongst a couple users relying 
on MCPM to do the cluster handling.  Secondary CPU boot is an other 
such user, as well as CPU hotplug that you then get for free.  The 
important thing to remember is that MCPM is a separate layer independent 
from the b.L switcher.

The switcher will accept to be started only on a dual cluster system.  
But if you don't want it started on a particular dual-cluster SoC you 
just need to add no_bL_switcher to the kernel cmdline for that SoC.


Nicolas

[PATCH 2/5] drivers/bus: arm-cci: Add common control interface for ACE ports

[Abhilash Kesavan]

From: Andrew Bresticker <abrestic@chromium.org>

cci_disable_port_by_cpu() can be used to disable an arbitrary
ACE port, but there is no C-callable way to enable an ACE port.
Change cci_disable_port_by_cpu() to cci_control_port_by_cpu()
to allow us to disable and enable a CPU's ACE port.

Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 drivers/bus/arm-cci.c   |   13 +++++++------
 include/linux/arm-cci.h |    7 +++++--
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
index 5a86da9..5668b40 100644
--- a/drivers/bus/arm-cci.c
+++ b/drivers/bus/arm-cci.c
@@ -763,10 +763,11 @@ static void notrace cci_port_control(unsigned int port, bool enable)
 }
 
 /**
- * cci_disable_port_by_cpu() - function to disable a CCI port by CPU
+ * cci_control_port_by_cpu() - function to control a CCI port by CPU
  *			       reference
  *
- * @mpidr: mpidr of the CPU whose CCI port should be disabled
+ * @mpidr: mpidr of the CPU whose CCI port should be enabled/disabled
+ * @enable: if true enables the port, if false disables it
  *
  * Disabling a CCI port for a CPU implies disabling the CCI port
  * controlling that CPU cluster. Code disabling CPU CCI ports
@@ -777,20 +778,20 @@ static void notrace cci_port_control(unsigned int port, bool enable)
  *	0 on success
  *	-ENODEV on port look-up failure
  */
-int notrace cci_disable_port_by_cpu(u64 mpidr)
+int notrace cci_control_port_by_cpu(u64 mpidr, bool enable)
 {
 	int cpu;
 	bool is_valid;
 	for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
 		is_valid = cpu_port_is_valid(&cpu_port[cpu]);
 		if (is_valid && cpu_port_match(&cpu_port[cpu], mpidr)) {
-			cci_port_control(cpu_port[cpu].port, false);
+			cci_port_control(cpu_port[cpu].port, enable);
 			return 0;
 		}
 	}
 	return -ENODEV;
 }
-EXPORT_SYMBOL_GPL(cci_disable_port_by_cpu);
+EXPORT_SYMBOL_GPL(cci_control_port_by_cpu);
 
 /**
  * cci_enable_port_for_self() - enable a CCI port for calling CPU
@@ -936,7 +937,7 @@ int notrace __cci_control_port_by_index(u32 port, bool enable)
 	/*
 	 * CCI control for ports connected to CPUS is extremely fragile
 	 * and must be made to go through a specific and controlled
-	 * interface (ie cci_disable_port_by_cpu(); control by general purpose
+	 * interface (ie cci_control_port_by_cpu(); control by general purpose
 	 * indexing is therefore disabled for ACE ports.
 	 */
 	if (ports[port].type == ACE_PORT)
diff --git a/include/linux/arm-cci.h b/include/linux/arm-cci.h
index 79d6edf..c6c3ed0 100644
--- a/include/linux/arm-cci.h
+++ b/include/linux/arm-cci.h
@@ -29,7 +29,7 @@ struct device_node;
 #ifdef CONFIG_ARM_CCI
 extern bool cci_probed(void);
 extern int cci_ace_get_port(struct device_node *dn);
-extern int cci_disable_port_by_cpu(u64 mpidr);
+extern int cci_control_port_by_cpu(u64 mpidr, bool enable);
 extern int __cci_control_port_by_device(struct device_node *dn, bool enable);
 extern int __cci_control_port_by_index(u32 port, bool enable);
 #else
@@ -38,7 +38,10 @@ static inline int cci_ace_get_port(struct device_node *dn)
 {
 	return -ENODEV;
 }
-static inline int cci_disable_port_by_cpu(u64 mpidr) { return -ENODEV; }
+static inline int cci_disable_port_by_cpu(u64 mpidr, bool enable)
+{
+	return -ENODEV;
+}
 static inline int __cci_control_port_by_device(struct device_node *dn,
 					       bool enable)
 {
-- 
1.7.9.5

[PATCH 2/5] drivers/bus: arm-cci: Add common control interface for ACE ports

[Nicolas Pitre]

On Fri, 11 Apr 2014, Abhilash Kesavan wrote:

> From: Andrew Bresticker <abrestic@chromium.org>
> 
> cci_disable_port_by_cpu() can be used to disable an arbitrary
> ACE port, but there is no C-callable way to enable an ACE port.
> Change cci_disable_port_by_cpu() to cci_control_port_by_cpu()
> to allow us to disable and enable a CPU's ACE port.
> 
> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>

As soon as this patch is applied, it'll break compilation for 
arch/arm/mach-vexpress/dcscb.c and arch/arm/mach-vexpress/tc2_pm.c.

I suggest you add a compatibility macro in include/linux/arm-cci.h that 
may look like:

#define cci_disable_port_by_cpu(mpidr) cci_control_port_by_cpu(mpidr, false)

... or at least update existing callers.


> ---
>  drivers/bus/arm-cci.c   |   13 +++++++------
>  include/linux/arm-cci.h |    7 +++++--
>  2 files changed, 12 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
> index 5a86da9..5668b40 100644
> --- a/drivers/bus/arm-cci.c
> +++ b/drivers/bus/arm-cci.c
> @@ -763,10 +763,11 @@ static void notrace cci_port_control(unsigned int port, bool enable)
>  }
>  
>  /**
> - * cci_disable_port_by_cpu() - function to disable a CCI port by CPU
> + * cci_control_port_by_cpu() - function to control a CCI port by CPU
>   *			       reference
>   *
> - * @mpidr: mpidr of the CPU whose CCI port should be disabled
> + * @mpidr: mpidr of the CPU whose CCI port should be enabled/disabled
> + * @enable: if true enables the port, if false disables it
>   *
>   * Disabling a CCI port for a CPU implies disabling the CCI port
>   * controlling that CPU cluster. Code disabling CPU CCI ports
> @@ -777,20 +778,20 @@ static void notrace cci_port_control(unsigned int port, bool enable)
>   *	0 on success
>   *	-ENODEV on port look-up failure
>   */
> -int notrace cci_disable_port_by_cpu(u64 mpidr)
> +int notrace cci_control_port_by_cpu(u64 mpidr, bool enable)
>  {
>  	int cpu;
>  	bool is_valid;
>  	for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
>  		is_valid = cpu_port_is_valid(&cpu_port[cpu]);
>  		if (is_valid && cpu_port_match(&cpu_port[cpu], mpidr)) {
> -			cci_port_control(cpu_port[cpu].port, false);
> +			cci_port_control(cpu_port[cpu].port, enable);
>  			return 0;
>  		}
>  	}
>  	return -ENODEV;
>  }
> -EXPORT_SYMBOL_GPL(cci_disable_port_by_cpu);
> +EXPORT_SYMBOL_GPL(cci_control_port_by_cpu);
>  
>  /**
>   * cci_enable_port_for_self() - enable a CCI port for calling CPU
> @@ -936,7 +937,7 @@ int notrace __cci_control_port_by_index(u32 port, bool enable)
>  	/*
>  	 * CCI control for ports connected to CPUS is extremely fragile
>  	 * and must be made to go through a specific and controlled
> -	 * interface (ie cci_disable_port_by_cpu(); control by general purpose
> +	 * interface (ie cci_control_port_by_cpu(); control by general purpose
>  	 * indexing is therefore disabled for ACE ports.
>  	 */
>  	if (ports[port].type == ACE_PORT)
> diff --git a/include/linux/arm-cci.h b/include/linux/arm-cci.h
> index 79d6edf..c6c3ed0 100644
> --- a/include/linux/arm-cci.h
> +++ b/include/linux/arm-cci.h
> @@ -29,7 +29,7 @@ struct device_node;
>  #ifdef CONFIG_ARM_CCI
>  extern bool cci_probed(void);
>  extern int cci_ace_get_port(struct device_node *dn);
> -extern int cci_disable_port_by_cpu(u64 mpidr);
> +extern int cci_control_port_by_cpu(u64 mpidr, bool enable);
>  extern int __cci_control_port_by_device(struct device_node *dn, bool enable);
>  extern int __cci_control_port_by_index(u32 port, bool enable);
>  #else
> @@ -38,7 +38,10 @@ static inline int cci_ace_get_port(struct device_node *dn)
>  {
>  	return -ENODEV;
>  }
> -static inline int cci_disable_port_by_cpu(u64 mpidr) { return -ENODEV; }
> +static inline int cci_disable_port_by_cpu(u64 mpidr, bool enable)
> +{
> +	return -ENODEV;
> +}
>  static inline int __cci_control_port_by_device(struct device_node *dn,
>  					       bool enable)
>  {
> -- 
> 1.7.9.5
> 

[PATCH 2/5] drivers/bus: arm-cci: Add common control interface for ACE ports

[Abhilash Kesavan]

Hi Nicolas,

On Fri, Apr 11, 2014 at 11:54 PM, Nicolas Pitre
<nicolas.pitre@linaro.org> wrote:
> On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>
>> From: Andrew Bresticker <abrestic@chromium.org>
>>
>> cci_disable_port_by_cpu() can be used to disable an arbitrary
>> ACE port, but there is no C-callable way to enable an ACE port.
>> Change cci_disable_port_by_cpu() to cci_control_port_by_cpu()
>> to allow us to disable and enable a CPU's ACE port.
>>
>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>
> As soon as this patch is applied, it'll break compilation for
> arch/arm/mach-vexpress/dcscb.c and arch/arm/mach-vexpress/tc2_pm.c.
>
> I suggest you add a compatibility macro in include/linux/arm-cci.h that
> may look like:
>
> #define cci_disable_port_by_cpu(mpidr) cci_control_port_by_cpu(mpidr, false)
>
> ... or at least update existing callers.

Thanks for the review.
This was an oversight on my part as our internal 3.8 based tree did
not have any users of this.
I will update the patch so that it does not break any existing callers.


Regards,
Abhilash
>
>
>> ---
>>  drivers/bus/arm-cci.c   |   13 +++++++------
>>  include/linux/arm-cci.h |    7 +++++--
>>  2 files changed, 12 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/bus/arm-cci.c b/drivers/bus/arm-cci.c
>> index 5a86da9..5668b40 100644
>> --- a/drivers/bus/arm-cci.c
>> +++ b/drivers/bus/arm-cci.c
>> @@ -763,10 +763,11 @@ static void notrace cci_port_control(unsigned int port, bool enable)
>>  }
>>
>>  /**
>> - * cci_disable_port_by_cpu() - function to disable a CCI port by CPU
>> + * cci_control_port_by_cpu() - function to control a CCI port by CPU
>>   *                          reference
>>   *
>> - * @mpidr: mpidr of the CPU whose CCI port should be disabled
>> + * @mpidr: mpidr of the CPU whose CCI port should be enabled/disabled
>> + * @enable: if true enables the port, if false disables it
>>   *
>>   * Disabling a CCI port for a CPU implies disabling the CCI port
>>   * controlling that CPU cluster. Code disabling CPU CCI ports
>> @@ -777,20 +778,20 @@ static void notrace cci_port_control(unsigned int port, bool enable)
>>   *   0 on success
>>   *   -ENODEV on port look-up failure
>>   */
>> -int notrace cci_disable_port_by_cpu(u64 mpidr)
>> +int notrace cci_control_port_by_cpu(u64 mpidr, bool enable)
>>  {
>>       int cpu;
>>       bool is_valid;
>>       for (cpu = 0; cpu < nr_cpu_ids; cpu++) {
>>               is_valid = cpu_port_is_valid(&cpu_port[cpu]);
>>               if (is_valid && cpu_port_match(&cpu_port[cpu], mpidr)) {
>> -                     cci_port_control(cpu_port[cpu].port, false);
>> +                     cci_port_control(cpu_port[cpu].port, enable);
>>                       return 0;
>>               }
>>       }
>>       return -ENODEV;
>>  }
>> -EXPORT_SYMBOL_GPL(cci_disable_port_by_cpu);
>> +EXPORT_SYMBOL_GPL(cci_control_port_by_cpu);
>>
>>  /**
>>   * cci_enable_port_for_self() - enable a CCI port for calling CPU
>> @@ -936,7 +937,7 @@ int notrace __cci_control_port_by_index(u32 port, bool enable)
>>       /*
>>        * CCI control for ports connected to CPUS is extremely fragile
>>        * and must be made to go through a specific and controlled
>> -      * interface (ie cci_disable_port_by_cpu(); control by general purpose
>> +      * interface (ie cci_control_port_by_cpu(); control by general purpose
>>        * indexing is therefore disabled for ACE ports.
>>        */
>>       if (ports[port].type == ACE_PORT)
>> diff --git a/include/linux/arm-cci.h b/include/linux/arm-cci.h
>> index 79d6edf..c6c3ed0 100644
>> --- a/include/linux/arm-cci.h
>> +++ b/include/linux/arm-cci.h
>> @@ -29,7 +29,7 @@ struct device_node;
>>  #ifdef CONFIG_ARM_CCI
>>  extern bool cci_probed(void);
>>  extern int cci_ace_get_port(struct device_node *dn);
>> -extern int cci_disable_port_by_cpu(u64 mpidr);
>> +extern int cci_control_port_by_cpu(u64 mpidr, bool enable);
>>  extern int __cci_control_port_by_device(struct device_node *dn, bool enable);
>>  extern int __cci_control_port_by_index(u32 port, bool enable);
>>  #else
>> @@ -38,7 +38,10 @@ static inline int cci_ace_get_port(struct device_node *dn)
>>  {
>>       return -ENODEV;
>>  }
>> -static inline int cci_disable_port_by_cpu(u64 mpidr) { return -ENODEV; }
>> +static inline int cci_disable_port_by_cpu(u64 mpidr, bool enable)
>> +{
>> +     return -ENODEV;
>> +}
>>  static inline int __cci_control_port_by_device(struct device_node *dn,
>>                                              bool enable)
>>  {
>> --
>> 1.7.9.5
>>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 3/5] ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM

[Abhilash Kesavan]

On the Exynos5420 non-secure SYSRAM is used for secondary CPU bring-up,
so add a mapping for it.

Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 arch/arm/mach-exynos/exynos.c           |   11 +++++++++++
 arch/arm/mach-exynos/include/mach/map.h |    1 +
 2 files changed, 12 insertions(+)

diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
index b32a907..b1cf9d5 100644
--- a/arch/arm/mach-exynos/exynos.c
+++ b/arch/arm/mach-exynos/exynos.c
@@ -159,6 +159,15 @@ static struct map_desc exynos5250_iodesc[] __initdata = {
 	},
 };
 
+static struct map_desc exynos5420_iodesc[] __initdata = {
+	{
+		.virtual	= (unsigned long)S5P_VA_SYSRAM_NS,
+		.pfn		= __phys_to_pfn(EXYNOS5420_PA_SYSRAM_NS),
+		.length		= SZ_4K,
+		.type		= MT_DEVICE,
+	},
+};
+
 static struct map_desc exynos5_iodesc[] __initdata = {
 	{
 		.virtual	= (unsigned long)S3C_VA_SYS,
@@ -294,6 +303,8 @@ static void __init exynos_map_io(void)
 		iotable_init(exynos4x12_iodesc, ARRAY_SIZE(exynos4x12_iodesc));
 	if (soc_is_exynos5250())
 		iotable_init(exynos5250_iodesc, ARRAY_SIZE(exynos5250_iodesc));
+	if (soc_is_exynos5420())
+		iotable_init(exynos5420_iodesc, ARRAY_SIZE(exynos5420_iodesc));
 }
 
 void __init exynos_init_io(void)
diff --git a/arch/arm/mach-exynos/include/mach/map.h b/arch/arm/mach-exynos/include/mach/map.h
index 7b046b5..8c5be19 100644
--- a/arch/arm/mach-exynos/include/mach/map.h
+++ b/arch/arm/mach-exynos/include/mach/map.h
@@ -29,6 +29,7 @@
 #define EXYNOS4210_PA_SYSRAM_NS		0x0203F000
 #define EXYNOS4x12_PA_SYSRAM_NS		0x0204F000
 #define EXYNOS5250_PA_SYSRAM_NS		0x0204F000
+#define EXYNOS5420_PA_SYSRAM_NS		0x02073000
 
 #define EXYNOS_PA_CHIPID		0x10000000
 
-- 
1.7.9.5

[PATCH 3/5] ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM

[Dave Martin]

On Fri, Apr 11, 2014 at 11:31:49PM +0530, Abhilash Kesavan wrote:
> On the Exynos5420 non-secure SYSRAM is used for secondary CPU bring-up,
> so add a mapping for it.
> 
> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
>  arch/arm/mach-exynos/exynos.c           |   11 +++++++++++
>  arch/arm/mach-exynos/include/mach/map.h |    1 +
>  2 files changed, 12 insertions(+)
> 
> diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
> index b32a907..b1cf9d5 100644
> --- a/arch/arm/mach-exynos/exynos.c
> +++ b/arch/arm/mach-exynos/exynos.c
> @@ -159,6 +159,15 @@ static struct map_desc exynos5250_iodesc[] __initdata = {
>  	},
>  };
>  
> +static struct map_desc exynos5420_iodesc[] __initdata = {
> +	{
> +		.virtual	= (unsigned long)S5P_VA_SYSRAM_NS,
> +		.pfn		= __phys_to_pfn(EXYNOS5420_PA_SYSRAM_NS),
> +		.length		= SZ_4K,
> +		.type		= MT_DEVICE,
> +	},
> +};
> +

Regarding Arnd's NAK in
http://lists.infradead.org/pipermail/linux-arm-kernel/2014-April/246936.html,
this looks like another instance of the same thing.

Arnd, what was your preferred solution?  Should this be described in DT?

Cheers
---Dave

[PATCH 3/5] ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM

[Arnd Bergmann]

On Monday 14 April 2014 12:25:20 Dave Martin wrote:
> On Fri, Apr 11, 2014 at 11:31:49PM +0530, Abhilash Kesavan wrote:
> > On the Exynos5420 non-secure SYSRAM is used for secondary CPU bring-up,
> > so add a mapping for it.
> > 
> > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> >  arch/arm/mach-exynos/exynos.c           |   11 +++++++++++
> >  arch/arm/mach-exynos/include/mach/map.h |    1 +
> >  2 files changed, 12 insertions(+)
> > 
> > diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
> > index b32a907..b1cf9d5 100644
> > --- a/arch/arm/mach-exynos/exynos.c
> > +++ b/arch/arm/mach-exynos/exynos.c
> > @@ -159,6 +159,15 @@ static struct map_desc exynos5250_iodesc[] __initdata = {
> >       },
> >  };
> >  
> > +static struct map_desc exynos5420_iodesc[] __initdata = {
> > +     {
> > +             .virtual        = (unsigned long)S5P_VA_SYSRAM_NS,
> > +             .pfn            = __phys_to_pfn(EXYNOS5420_PA_SYSRAM_NS),
> > +             .length         = SZ_4K,
> > +             .type           = MT_DEVICE,
> > +     },
> > +};
> > +
> 
> Regarding Arnd's NAK in
> http://lists.infradead.org/pipermail/linux-arm-kernel/2014-April/246936.html,
> this looks like another instance of the same thing.
> 
> Arnd, what was your preferred solution?  Should this be described in DT?

Yes. There are already patches on the list to do this right.

	Arnd

[PATCH 3/5] ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM

[Dave Martin]

On Mon, Apr 14, 2014 at 01:47:56PM +0200, Arnd Bergmann wrote:
> On Monday 14 April 2014 12:25:20 Dave Martin wrote:
> > On Fri, Apr 11, 2014 at 11:31:49PM +0530, Abhilash Kesavan wrote:
> > > On the Exynos5420 non-secure SYSRAM is used for secondary CPU bring-up,
> > > so add a mapping for it.
> > > 
> > > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> > > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > > ---
> > >  arch/arm/mach-exynos/exynos.c           |   11 +++++++++++
> > >  arch/arm/mach-exynos/include/mach/map.h |    1 +
> > >  2 files changed, 12 insertions(+)
> > > 
> > > diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
> > > index b32a907..b1cf9d5 100644
> > > --- a/arch/arm/mach-exynos/exynos.c
> > > +++ b/arch/arm/mach-exynos/exynos.c
> > > @@ -159,6 +159,15 @@ static struct map_desc exynos5250_iodesc[] __initdata = {
> > >       },
> > >  };
> > >  
> > > +static struct map_desc exynos5420_iodesc[] __initdata = {
> > > +     {
> > > +             .virtual        = (unsigned long)S5P_VA_SYSRAM_NS,
> > > +             .pfn            = __phys_to_pfn(EXYNOS5420_PA_SYSRAM_NS),
> > > +             .length         = SZ_4K,
> > > +             .type           = MT_DEVICE,
> > > +     },
> > > +};
> > > +
> > 
> > Regarding Arnd's NAK in
> > http://lists.infradead.org/pipermail/linux-arm-kernel/2014-April/246936.html,
> > this looks like another instance of the same thing.
> > 
> > Arnd, what was your preferred solution?  Should this be described in DT?
> 
> Yes. There are already patches on the list to do this right.

This is http://www.spinics.net/lists/linux-samsung-soc/msg27647.html,
right?

If so, it looks like exynos5420 should definitely follow the same
approach.

Cheers
---Dave

[PATCH 3/5] ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM

[Arnd Bergmann]

On Monday 14 April 2014 12:59:11 Dave Martin wrote:
> On Mon, Apr 14, 2014 at 01:47:56PM +0200, Arnd Bergmann wrote:
> > On Monday 14 April 2014 12:25:20 Dave Martin wrote:
> > > Arnd, what was your preferred solution?  Should this be described in DT?
> > 
> > Yes. There are already patches on the list to do this right.
> 
> This is http://www.spinics.net/lists/linux-samsung-soc/msg27647.html,
> right?
> 
> If so, it looks like exynos5420 should definitely follow the same
> approach.
> 

Yes, exactly.

	Arnd

[PATCH 3/5] ARM: EXYNOS5420: Add IO mapping for non-secure SYSRAM

[Abhilash Kesavan]

Hi Dave and Arnd,

On Mon, Apr 14, 2014 at 6:54 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Monday 14 April 2014 12:59:11 Dave Martin wrote:
>> On Mon, Apr 14, 2014 at 01:47:56PM +0200, Arnd Bergmann wrote:
>> > On Monday 14 April 2014 12:25:20 Dave Martin wrote:
>> > > Arnd, what was your preferred solution?  Should this be described in DT?
>> >
>> > Yes. There are already patches on the list to do this right.
>>
>> This is http://www.spinics.net/lists/linux-samsung-soc/msg27647.html,
>> right?
>>
>> If so, it looks like exynos5420 should definitely follow the same
>> approach.
>>
>
> Yes, exactly.

I'll make use of the SYSRAM dt patch in my next patchset.
>
>         Arnd
Regards,
Abhilash
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 4/5] ARM: dts: exynos5420: add CCI node

[Abhilash Kesavan]

From: Andrew Bresticker <abrestic@chromium.org>

Add device-tree bindings for the ARM CCI-400 on Exynos5420.  There
are two slave interfaces: one for the A15 cluster and one for the
A7 cluster.

Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 arch/arm/boot/dts/exynos5420.dtsi |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/arm/boot/dts/exynos5420.dtsi b/arch/arm/boot/dts/exynos5420.dtsi
index c3a9a66..0498ecf 100644
--- a/arch/arm/boot/dts/exynos5420.dtsi
+++ b/arch/arm/boot/dts/exynos5420.dtsi
@@ -58,6 +58,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x0>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu1: cpu at 1 {
@@ -65,6 +66,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x1>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu2: cpu at 2 {
@@ -72,6 +74,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x2>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu3: cpu at 3 {
@@ -79,6 +82,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x3>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu4: cpu at 100 {
@@ -86,6 +90,7 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x100>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
 		};
 
 		cpu5: cpu at 101 {
@@ -93,6 +98,7 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x101>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
 		};
 
 		cpu6: cpu at 102 {
@@ -100,6 +106,7 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x102>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
 		};
 
 		cpu7: cpu at 103 {
@@ -107,6 +114,26 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x103>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
+		};
+	};
+
+	cci at 10d20000 {
+		compatible = "arm,cci-400";
+		#address-cells = <1>;
+		#size-cells = <1>;
+		reg = <0x10d20000 0x1000>;
+		ranges = <0x0 0x10d20000 0x6000>;
+
+		cci_control0: slave-if at 4000 {
+			compatible = "arm,cci-400-ctrl-if";
+			interface-type = "ace";
+			reg = <0x4000 0x1000>;
+		};
+		cci_control1: slave-if at 5000 {
+			compatible = "arm,cci-400-ctrl-if";
+			interface-type = "ace";
+			reg = <0x5000 0x1000>;
 		};
 	};
 
-- 
1.7.9.5

[PATCH v2 1/3] ARM: dts: exynos5420: add CCI node

[Abhilash Kesavan]

From: Andrew Bresticker <abrestic@chromium.org>

Add device-tree bindings for the ARM CCI-400 on Exynos5420.  There
are two slave interfaces: one for the A15 cluster and one for the
A7 cluster.

Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 arch/arm/boot/dts/exynos5420.dtsi |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/arm/boot/dts/exynos5420.dtsi b/arch/arm/boot/dts/exynos5420.dtsi
index 217376f..43e7b25 100644
--- a/arch/arm/boot/dts/exynos5420.dtsi
+++ b/arch/arm/boot/dts/exynos5420.dtsi
@@ -58,6 +58,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x0>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu1: cpu at 1 {
@@ -65,6 +66,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x1>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu2: cpu at 2 {
@@ -72,6 +74,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x2>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu3: cpu at 3 {
@@ -79,6 +82,7 @@
 			compatible = "arm,cortex-a15";
 			reg = <0x3>;
 			clock-frequency = <1800000000>;
+			cci-control-port = <&cci_control1>;
 		};
 
 		cpu4: cpu at 100 {
@@ -86,6 +90,7 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x100>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
 		};
 
 		cpu5: cpu at 101 {
@@ -93,6 +98,7 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x101>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
 		};
 
 		cpu6: cpu at 102 {
@@ -100,6 +106,7 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x102>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
 		};
 
 		cpu7: cpu at 103 {
@@ -107,6 +114,26 @@
 			compatible = "arm,cortex-a7";
 			reg = <0x103>;
 			clock-frequency = <1000000000>;
+			cci-control-port = <&cci_control0>;
+		};
+	};
+
+	cci at 10d20000 {
+		compatible = "arm,cci-400";
+		#address-cells = <1>;
+		#size-cells = <1>;
+		reg = <0x10d20000 0x1000>;
+		ranges = <0x0 0x10d20000 0x6000>;
+
+		cci_control0: slave-if at 4000 {
+			compatible = "arm,cci-400-ctrl-if";
+			interface-type = "ace";
+			reg = <0x4000 0x1000>;
+		};
+		cci_control1: slave-if at 5000 {
+			compatible = "arm,cci-400-ctrl-if";
+			interface-type = "ace";
+			reg = <0x5000 0x1000>;
 		};
 	};
 
-- 
1.7.9.5

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Add machine-dependent MCPM call-backs for Exynos5420. These are used
to power up/down the secondary CPUs during boot, shutdown, s2r and
switching.

Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 arch/arm/mach-exynos/Kconfig             |    9 +
 arch/arm/mach-exynos/Makefile            |    2 +
 arch/arm/mach-exynos/common.h            |    1 +
 arch/arm/mach-exynos/exynos.c            |    1 +
 arch/arm/mach-exynos/mcpm-exynos-setup.S |   35 +++
 arch/arm/mach-exynos/mcpm-exynos.c       |  444 ++++++++++++++++++++++++++++++
 arch/arm/mach-exynos/platsmp.c           |   19 ++
 arch/arm/mach-exynos/regs-pmu.h          |    2 +
 8 files changed, 513 insertions(+)
 create mode 100644 arch/arm/mach-exynos/mcpm-exynos-setup.S
 create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c

diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
index fc8bf18..a921a80 100644
--- a/arch/arm/mach-exynos/Kconfig
+++ b/arch/arm/mach-exynos/Kconfig
@@ -110,4 +110,13 @@ config SOC_EXYNOS5440
 
 endmenu
 
+config EXYNOS5420_MCPM
+	bool "Exynos5420 Multi-Cluster PM support"
+	depends on MCPM && SOC_EXYNOS5420
+	select ARM_CCI
+	help
+	  Support for Dual Cluster Switching (A15/A7) on Exynos5420.
+	  This is needed to provide CPU and cluster power management
+	  on Exynos5420 implementing big.LITTLE.
+
 endif
diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
index a656dbe..776fcbd 100644
--- a/arch/arm/mach-exynos/Makefile
+++ b/arch/arm/mach-exynos/Makefile
@@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)	+= firmware.o
 
 plus_sec := $(call as-instr,.arch_extension sec,+sec)
 AFLAGS_exynos-smc.o		:=-Wa,-march=armv7-a$(plus_sec)
+
+obj-$(CONFIG_EXYNOS5420_MCPM)	+= mcpm-exynos.o mcpm-exynos-setup.o
diff --git a/arch/arm/mach-exynos/common.h b/arch/arm/mach-exynos/common.h
index 347afc2..a023ccc 100644
--- a/arch/arm/mach-exynos/common.h
+++ b/arch/arm/mach-exynos/common.h
@@ -51,6 +51,7 @@ static inline void exynos_pm_init(void) {}
 extern void exynos_cpu_resume(void);
 
 extern struct smp_operations exynos_smp_ops;
+extern bool exynos_smp_init(void);
 
 extern void exynos_cpu_die(unsigned int cpu);
 
diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
index b1cf9d5..5b72b5e 100644
--- a/arch/arm/mach-exynos/exynos.c
+++ b/arch/arm/mach-exynos/exynos.c
@@ -412,6 +412,7 @@ DT_MACHINE_START(EXYNOS_DT, "SAMSUNG EXYNOS (Flattened Device Tree)")
 	/* Maintainer: Thomas Abraham <thomas.abraham@linaro.org> */
 	/* Maintainer: Kukjin Kim <kgene.kim@samsung.com> */
 	.smp		= smp_ops(exynos_smp_ops),
+	.smp_init	= smp_init_ops(exynos_smp_init),
 	.map_io		= exynos_init_io,
 	.init_early	= exynos_firmware_init,
 	.init_machine	= exynos_dt_machine_init,
diff --git a/arch/arm/mach-exynos/mcpm-exynos-setup.S b/arch/arm/mach-exynos/mcpm-exynos-setup.S
new file mode 100644
index 0000000..990c0d5
--- /dev/null
+++ b/arch/arm/mach-exynos/mcpm-exynos-setup.S
@@ -0,0 +1,35 @@
+/*
+ * Exynos low-level MCPM setup
+ *
+ * Copyright (C) 2013-2014 Google, Inc
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/linkage.h>
+
+ENTRY(exynos_power_up_setup)
+
+	cmp	r0, #0			@ check affinity level
+	beq	1f
+
+/*
+ * Enable cluster-level coherency, in preparation for turning on the MMU.
+ * The ACTLR SMP bit does not need to be set here, because cpu_resume()
+ * already restores that.
+ */
+	b	cci_enable_port_for_self
+
+1:	@ Implementation-specific local CPU setup operations should go here,
+	@ if any.  In this case, there is nothing to do.
+
+	bx	lr
+
+ENDPROC(exynos_power_up_setup)
diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
new file mode 100644
index 0000000..46d4968
--- /dev/null
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -0,0 +1,444 @@
+/*
+ * Copyright (c) 2014 Samsung Electronics Co., Ltd.
+ *		http://www.samsung.com
+ *
+ * arch/arm/mach-exynos/mcpm-exynos.c
+ *
+ * Based on arch/arm/mach-vexpress/dcscb.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/kernel.h>
+#include <linux/io.h>
+#include <linux/spinlock.h>
+#include <linux/uaccess.h>
+#include <linux/miscdevice.h>
+#include <linux/fs.h>
+#include <linux/delay.h>
+#include <linux/arm-cci.h>
+
+#include <asm/mcpm.h>
+#include <asm/cputype.h>
+#include <asm/cp15.h>
+
+#include <plat/cpu.h>
+#include "regs-pmu.h"
+
+#define EXYNOS5420_CPUS_PER_CLUSTER	4
+#define EXYNOS5420_NR_CLUSTERS		2
+
+/* Secondary CPU entry point */
+#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
+
+#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
+#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
+
+#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
+#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
+
+#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
+			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
+#define EXYNOS_ARM_CORE_STATUS(_nr)		\
+			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
+
+#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
+			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
+#define EXYNOS_COMMON_STATUS(_nr)		\
+			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
+
+#define EXYNOS_L2_CONFIGURATION(_nr)		\
+			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
+#define EXYNOS_L2_STATUS(_nr)			\
+			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
+
+/*
+ * We can't use regular spinlocks. In the switcher case, it is possible
+ * for an outbound CPU to call power_down() after its inbound counterpart
+ * is already live using the same logical CPU number which trips lockdep
+ * debugging.
+ */
+static arch_spinlock_t bl_lock = __ARCH_SPIN_LOCK_UNLOCKED;
+static int
+cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
+
+static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
+{
+	unsigned int val;
+	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
+
+	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
+				EXYNOS_CORE_LOCAL_PWR_EN;
+	return !!val;
+}
+
+static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
+						int enable)
+{
+	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
+	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
+
+	if (exynos_core_power_state(cpu, cluster) == enable)
+		return;
+
+	if (enable)
+		val = EXYNOS_CORE_LOCAL_PWR_EN;
+	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
+}
+
+static void exynos_cluster_power_control(unsigned int cluster, int enable)
+{
+	unsigned long timeout = jiffies + msecs_to_jiffies(20);
+	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
+
+	if (enable)
+		val = EXYNOS_CORE_LOCAL_PWR_EN;
+
+	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
+	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
+		return;
+
+	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
+	/* Wait until cluster power control is applied */
+	while (time_before(jiffies, timeout)) {
+		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
+
+		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
+			return;
+
+		cpu_relax();
+	}
+	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
+		enable ? "on" : "off");
+}
+
+static int exynos_power_up(unsigned int cpu, unsigned int cluster)
+{
+	unsigned long mpidr;
+
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
+		cluster >= EXYNOS5420_NR_CLUSTERS)
+		return -EINVAL;
+
+	/*
+	 * Since this is called with IRQs enabled, and no arch_spin_lock_irq
+	 * variant exists, we need to disable IRQs manually here.
+	 */
+	local_irq_disable();
+	arch_spin_lock(&bl_lock);
+
+	cpu_use_count[cpu][cluster]++;
+	if (cpu_use_count[cpu][cluster] == 1) {
+		bool was_cluster_down =
+			__mcpm_cluster_state(cluster) == CLUSTER_DOWN;
+
+		if (was_cluster_down)
+			exynos_cluster_power_control(cluster, 1);
+		exynos_core_power_control(cpu, cluster, 1);
+
+		if (was_cluster_down) {
+			mpidr = read_cpuid_mpidr();
+			udelay(10);
+			cci_control_port_by_cpu(mpidr, true);
+		}
+
+		/* CPU should be powered up there */
+		/* Also setup Cluster Power Sequence here */
+	} else if (cpu_use_count[cpu][cluster] != 2) {
+		/*
+		 * The only possible values are:
+		 * 0 = CPU down
+		 * 1 = CPU (still) up
+		 * 2 = CPU requested to be up before it had a chance
+		 *     to actually make itself down.
+		 * Any other value is a bug.
+		 */
+		BUG();
+	}
+
+	arch_spin_unlock(&bl_lock);
+	local_irq_enable();
+
+	return 0;
+}
+
+static void exynos_power_down(void)
+{
+	unsigned int mpidr, cpu, cluster, cpumask;
+	bool last_man = false, skip_wfi = false;
+
+	mpidr = read_cpuid_mpidr();
+	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	cpumask = (1 << cpu);
+
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
+			cluster >= EXYNOS5420_NR_CLUSTERS);
+
+	__mcpm_cpu_going_down(cpu, cluster);
+
+	arch_spin_lock(&bl_lock);
+	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
+	cpu_use_count[cpu][cluster]--;
+	if (cpu_use_count[cpu][cluster] == 0) {
+		int cnt = 0, i;
+
+		exynos_core_power_control(cpu, cluster, 0);
+		for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
+			cnt += cpu_use_count[i][cluster];
+		if (cnt == 0)
+			last_man = true;
+	} else if (cpu_use_count[cpu][cluster] == 1) {
+		/*
+		 * A power_up request went ahead of us.
+		 * Even if we do not want to shut this CPU down,
+		 * the caller expects a certain state as if the WFI
+		 * was aborted.  So let's continue with cache cleaning.
+		 */
+		skip_wfi = true;
+	} else {
+		BUG();
+	}
+
+	if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
+		arch_spin_unlock(&bl_lock);
+
+		/*
+		 * Flush all cache levels for this cluster.
+		 *
+		 * To do so we do:
+		 * - Clear the SCTLR.C bit to prevent further cache allocations
+		 * - Flush the whole cache
+		 * - Clear the ACTLR "SMP" bit to disable local coherency
+		 *
+		 * Let's do it in the safest possible way i.e. with
+		 * no memory access within the following sequence
+		 * including to the stack.
+		 *
+		 * Note: fp is preserved to the stack explicitly prior doing
+		 * this since adding it to the clobber list is incompatible
+		 * with having CONFIG_FRAME_POINTER=y.
+		 *
+		 * The common v7_exit_coherency_flush API that could not be
+		 * used because of the Erratum 799270 workaround.
+		 */
+		asm volatile(
+		"stmfd	sp!, {fp, ip}\n\t"
+		"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t"
+		"bic	r0, r0, #"__stringify(CR_C)"\n\t"
+		"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t"
+		"isb\n\t"
+		"bl	v7_flush_dcache_all\n\t"
+		"clrex\n\t"
+		"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t"
+		"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t"
+		/* Dummy Load of a device register to avoid Erratum 799270 */
+		"ldr	r4, [%0]\n\t"
+		"and	r4, r4, #0\n\t"
+		"orr	r0, r0, r4\n\t"
+		"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t"
+		"isb\n\t"
+		"dsb\n\t"
+		"ldmfd	sp!, {fp, ip}"
+		:
+		: "Ir" (S5P_INFORM0)
+		: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
+		      "r9", "r10", "lr", "memory");
+
+		/*
+		 * This is a harmless no-op.  On platforms with a real
+		 * outer cache this might either be needed or not,
+		 * depending on where the outer cache sits.
+		 */
+		outer_flush_all();
+
+		/*
+		 * Disable cluster-level coherency by masking
+		 * incoming snoops and DVM messages:
+		 */
+		cci_control_port_by_cpu(mpidr, false);
+		cci_control_port_by_cpu(mpidr ^ (1 << 8), false);
+
+		__mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
+	} else {
+		arch_spin_unlock(&bl_lock);
+
+		/*
+		 * Flush the local CPU cache.
+		 * Let's do it in the safest possible way as above.
+		 */
+		asm volatile(
+		"stmfd	sp!, {fp, ip}\n\t"
+		"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t"
+		"bic	r0, r0, #"__stringify(CR_C)"\n\t"
+		"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t"
+		"isb\n\t"
+		"bl	v7_flush_dcache_louis\n\t"
+		"clrex\n\t"
+		"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t"
+		"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t"
+		/* Dummy Load of a device register to avoid Erratum 799270 */
+		"ldr	r4, [%0]\n\t"
+		"and	r4, r4, #0\n\t"
+		"orr	r0, r0, r4\n\t"
+		"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t"
+		"isb\n\t"
+		"dsb\n\t"
+		"ldmfd	sp!, {fp, ip}"
+		:
+		: "Ir" (S5P_INFORM0)
+		: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
+		      "r9", "r10", "lr", "memory");
+	}
+
+	__mcpm_cpu_down(cpu, cluster);
+
+	/* Now we are prepared for power-down, do it: */
+	dsb();
+	if (!skip_wfi)
+		wfi();
+
+	/* Not dead@this point?  Let our caller cope. */
+}
+
+static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
+{
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
+			cluster >= EXYNOS5420_NR_CLUSTERS);
+
+	/* Wait for the core state to be OFF */
+	while (exynos_core_power_state(cpu, cluster) != 0x0)
+		;
+
+	return 0; /* success: the CPU is halted */
+}
+
+static const struct mcpm_platform_ops exynos_power_ops = {
+	.power_up		= exynos_power_up,
+	.power_down		= exynos_power_down,
+	.power_down_finish	= exynos_power_down_finish,
+};
+
+static void __init exynos_mcpm_usage_count_init(void)
+{
+	unsigned int mpidr, cpu, cluster, i;
+
+	mpidr = read_cpuid_mpidr();
+	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
+			cluster >= EXYNOS5420_NR_CLUSTERS);
+
+	for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++) {
+		cpu_use_count[i][0] = 0;
+		cpu_use_count[i][1] = 0;
+	}
+	cpu_use_count[cpu][cluster] = 1;
+}
+
+static size_t bL_check_status(char *info)
+{
+	size_t len = 0;
+	int i;
+
+	len += sprintf(&info[len], "\t0 1 2 3 L2\n");
+	len += sprintf(&info[len], "[A15]   ");
+	for (i = 0; i < 4; i++) {
+		len += sprintf(&info[len], "%d ",
+			(readl(EXYNOS_ARM_CORE_STATUS(i)) & 0x3) == 3 ? 1 : 0);
+	}
+	len += sprintf(&info[len], "%d\n",
+			(readl(EXYNOS_COMMON_STATUS(0)) & 0x3) == 3 ? 1 : 0);
+
+	len += sprintf(&info[len], "[A7]    ");
+	for (i = 4; i < 8; i++)
+		len += sprintf(&info[len], "%d ",
+			(readl(EXYNOS_ARM_CORE_STATUS(i)) & 0x3) == 3 ? 1 : 0);
+	len += sprintf(&info[len], "%d\n\n",
+			(readl(EXYNOS_COMMON_STATUS(1)) & 0x3) == 3 ? 1 : 0);
+
+	return len;
+}
+
+static ssize_t bL_status_read(struct file *file, char __user *buf,
+			size_t len, loff_t *pos)
+{
+	size_t count = 0;
+	char info[100];
+
+	count = bL_check_status(info);
+	if (count < 0)
+		return -EINVAL;
+
+	return simple_read_from_buffer(buf, len, pos, info, count);
+}
+
+static const struct file_operations bL_status_fops = {
+	.read		= bL_status_read,
+};
+
+static struct miscdevice bL_status_device = {
+	MISC_DYNAMIC_MINOR,
+	"bL_status",
+	&bL_status_fops
+};
+
+extern void exynos_power_up_setup(unsigned int affinity_level);
+
+static int __init exynos_mcpm_init(void)
+{
+	int ret = 0;
+
+	if (!cci_probed())
+		return -ENODEV;
+
+	if (!soc_is_exynos5420())
+		return 0;
+
+	/*
+	 * To increase the stability of KFC reset we need to program
+	 * the PMU SPARE3 register
+	 */
+	__raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
+
+	exynos_mcpm_usage_count_init();
+
+	ret = mcpm_platform_register(&exynos_power_ops);
+	if (!ret)
+		ret = mcpm_sync_init(exynos_power_up_setup);
+
+	pr_info("Exynos MCPM support installed\n");
+
+	/*
+	 * Future entries into the kernel can now go
+	 * through the cluster entry vectors.
+	 */
+	__raw_writel(virt_to_phys(mcpm_entry_point),
+		REG_ENTRY_ADDR);
+
+	return ret;
+}
+
+early_initcall(exynos_mcpm_init);
+
+static int __init exynos_bl_status_init(void)
+{
+	int ret;
+
+	if (!soc_is_exynos5420())
+		return 0;
+
+	ret = misc_register(&bL_status_device);
+	if (ret)
+		pr_info("bl_status not available\n");
+	return 0;
+}
+
+late_initcall(exynos_bl_status_init);
diff --git a/arch/arm/mach-exynos/platsmp.c b/arch/arm/mach-exynos/platsmp.c
index 03e5e9f..4f14457 100644
--- a/arch/arm/mach-exynos/platsmp.c
+++ b/arch/arm/mach-exynos/platsmp.c
@@ -25,6 +25,7 @@
 #include <asm/smp_plat.h>
 #include <asm/smp_scu.h>
 #include <asm/firmware.h>
+#include <asm/mcpm.h>
 
 #include <plat/cpu.h>
 
@@ -226,6 +227,24 @@ static void __init exynos_smp_prepare_cpus(unsigned int max_cpus)
 	}
 }
 
+bool __init exynos_smp_init(void)
+{
+#ifdef CONFIG_MCPM
+	/*
+	 * The best way to detect a multi-cluster configuration at the moment
+	 * is to look for the presence of a CCI in the system.
+	 * Override the default exynos_smp_ops if so.
+	 */
+	struct device_node *node;
+	node = of_find_compatible_node(NULL, NULL, "arm,cci-400");
+	if (node && of_device_is_available(node)) {
+		mcpm_smp_set_ops();
+		return true;
+	}
+#endif
+	return false;
+}
+
 struct smp_operations exynos_smp_ops __initdata = {
 	.smp_init_cpus		= exynos_smp_init_cpus,
 	.smp_prepare_cpus	= exynos_smp_prepare_cpus,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index cfbfc575..43fe7a0 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -38,6 +38,7 @@
 #define S5P_INFORM5				S5P_PMUREG(0x0814)
 #define S5P_INFORM6				S5P_PMUREG(0x0818)
 #define S5P_INFORM7				S5P_PMUREG(0x081C)
+#define S5P_PMU_SPARE3				S5P_PMUREG(0x090C)
 
 #define EXYNOS_IROM_DATA2			S5P_PMUREG(0x0988)
 
@@ -540,5 +541,6 @@
 #define EXYNOS5420_KFC_USE_STANDBY_WFE3				(1 << 23)
 
 #define DUR_WAIT_RESET				0xF
+#define EXYNOS5420_SWRESET_KFC_SEL		0x3
 
 #endif /* __ASM_ARCH_REGS_PMU_H */
-- 
1.7.9.5

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

On Fri, 11 Apr 2014, Abhilash Kesavan wrote:

> Add machine-dependent MCPM call-backs for Exynos5420. These are used
> to power up/down the secondary CPUs during boot, shutdown, s2r and
> switching.
> 
> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>

See comments inline.

> ---
>  arch/arm/mach-exynos/Kconfig             |    9 +
>  arch/arm/mach-exynos/Makefile            |    2 +
>  arch/arm/mach-exynos/common.h            |    1 +
>  arch/arm/mach-exynos/exynos.c            |    1 +
>  arch/arm/mach-exynos/mcpm-exynos-setup.S |   35 +++
>  arch/arm/mach-exynos/mcpm-exynos.c       |  444 ++++++++++++++++++++++++++++++
>  arch/arm/mach-exynos/platsmp.c           |   19 ++
>  arch/arm/mach-exynos/regs-pmu.h          |    2 +
>  8 files changed, 513 insertions(+)
>  create mode 100644 arch/arm/mach-exynos/mcpm-exynos-setup.S
>  create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c
> 
> diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
> index fc8bf18..a921a80 100644
> --- a/arch/arm/mach-exynos/Kconfig
> +++ b/arch/arm/mach-exynos/Kconfig
> @@ -110,4 +110,13 @@ config SOC_EXYNOS5440
>  
>  endmenu
>  
> +config EXYNOS5420_MCPM
> +	bool "Exynos5420 Multi-Cluster PM support"
> +	depends on MCPM && SOC_EXYNOS5420
> +	select ARM_CCI
> +	help
> +	  Support for Dual Cluster Switching (A15/A7) on Exynos5420.
> +	  This is needed to provide CPU and cluster power management
> +	  on Exynos5420 implementing big.LITTLE.

MCPM is not about "cluster switching".  It is about cluster-wide 
power-up/power-down coordination and race avoidance.  MCPM is relied 
upon by the big.LITTLE switcher, but it is also needed by cpuidle, CPU 
hotplug, etc.  Therefore the first line of the help text is wrong and 
could be omitted entirely.

>  endif
> diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
> index a656dbe..776fcbd 100644
> --- a/arch/arm/mach-exynos/Makefile
> +++ b/arch/arm/mach-exynos/Makefile
> @@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)	+= firmware.o
>  
>  plus_sec := $(call as-instr,.arch_extension sec,+sec)
>  AFLAGS_exynos-smc.o		:=-Wa,-march=armv7-a$(plus_sec)
> +
> +obj-$(CONFIG_EXYNOS5420_MCPM)	+= mcpm-exynos.o mcpm-exynos-setup.o
> diff --git a/arch/arm/mach-exynos/common.h b/arch/arm/mach-exynos/common.h
> index 347afc2..a023ccc 100644
> --- a/arch/arm/mach-exynos/common.h
> +++ b/arch/arm/mach-exynos/common.h
> @@ -51,6 +51,7 @@ static inline void exynos_pm_init(void) {}
>  extern void exynos_cpu_resume(void);
>  
>  extern struct smp_operations exynos_smp_ops;
> +extern bool exynos_smp_init(void);
>  
>  extern void exynos_cpu_die(unsigned int cpu);
>  
> diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
> index b1cf9d5..5b72b5e 100644
> --- a/arch/arm/mach-exynos/exynos.c
> +++ b/arch/arm/mach-exynos/exynos.c
> @@ -412,6 +412,7 @@ DT_MACHINE_START(EXYNOS_DT, "SAMSUNG EXYNOS (Flattened Device Tree)")
>  	/* Maintainer: Thomas Abraham <thomas.abraham@linaro.org> */
>  	/* Maintainer: Kukjin Kim <kgene.kim@samsung.com> */
>  	.smp		= smp_ops(exynos_smp_ops),
> +	.smp_init	= smp_init_ops(exynos_smp_init),
>  	.map_io		= exynos_init_io,
>  	.init_early	= exynos_firmware_init,
>  	.init_machine	= exynos_dt_machine_init,
> diff --git a/arch/arm/mach-exynos/mcpm-exynos-setup.S b/arch/arm/mach-exynos/mcpm-exynos-setup.S
> new file mode 100644
> index 0000000..990c0d5
> --- /dev/null
> +++ b/arch/arm/mach-exynos/mcpm-exynos-setup.S
> @@ -0,0 +1,35 @@
> +/*
> + * Exynos low-level MCPM setup
> + *
> + * Copyright (C) 2013-2014 Google, Inc
> + *
> + * This software is licensed under the terms of the GNU General Public
> + * License version 2, as published by the Free Software Foundation, and
> + * may be copied, distributed, and modified under those terms.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <linux/linkage.h>
> +
> +ENTRY(exynos_power_up_setup)
> +
> +	cmp	r0, #0			@ check affinity level
> +	beq	1f
> +
> +/*
> + * Enable cluster-level coherency, in preparation for turning on the MMU.
> + * The ACTLR SMP bit does not need to be set here, because cpu_resume()
> + * already restores that.
> + */
> +	b	cci_enable_port_for_self
> +
> +1:	@ Implementation-specific local CPU setup operations should go here,
> +	@ if any.  In this case, there is nothing to do.
> +
> +	bx	lr
> +
> +ENDPROC(exynos_power_up_setup)

Given this is so simple, I'd suggest you simply copy the TC2 version for 
the above code and dispense with this file altogether.  See 
tc2_pm_power_up_setup() in mach-vexpress/tc2_pm.c.

> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> new file mode 100644
> index 0000000..46d4968
> --- /dev/null
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -0,0 +1,444 @@
> +/*
> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> + *		http://www.samsung.com
> + *
> + * arch/arm/mach-exynos/mcpm-exynos.c
> + *
> + * Based on arch/arm/mach-vexpress/dcscb.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +#include <linux/uaccess.h>
> +#include <linux/miscdevice.h>
> +#include <linux/fs.h>
> +#include <linux/delay.h>
> +#include <linux/arm-cci.h>
> +
> +#include <asm/mcpm.h>
> +#include <asm/cputype.h>
> +#include <asm/cp15.h>
> +
> +#include <plat/cpu.h>
> +#include "regs-pmu.h"
> +
> +#define EXYNOS5420_CPUS_PER_CLUSTER	4
> +#define EXYNOS5420_NR_CLUSTERS		2
> +
> +/* Secondary CPU entry point */
> +#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
> +
> +#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
> +#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
> +
> +#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
> +#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
> +
> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
> +			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_ARM_CORE_STATUS(_nr)		\
> +			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
> +
> +#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
> +			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_COMMON_STATUS(_nr)		\
> +			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
> +
> +#define EXYNOS_L2_CONFIGURATION(_nr)		\
> +			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_L2_STATUS(_nr)			\
> +			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
> +
> +/*
> + * We can't use regular spinlocks. In the switcher case, it is possible
> + * for an outbound CPU to call power_down() after its inbound counterpart
> + * is already live using the same logical CPU number which trips lockdep
> + * debugging.
> + */
> +static arch_spinlock_t bl_lock = __ARCH_SPIN_LOCK_UNLOCKED;

the bl prefix in "bl_lock" might be confusing.  I'd suggest you name 
this "exynos_mcpm_lock" or similar.

> +static int
> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
> +
> +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
> +{
> +	unsigned int val;
> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> +
> +	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
> +				EXYNOS_CORE_LOCAL_PWR_EN;
> +	return !!val;
> +}
> +
> +static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
> +						int enable)
> +{
> +	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> +
> +	if (exynos_core_power_state(cpu, cluster) == enable)
> +		return;
> +
> +	if (enable)
> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> +	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
> +}
> +
> +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> +{
> +	unsigned long timeout = jiffies + msecs_to_jiffies(20);
> +	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> +
> +	if (enable)
> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> +
> +	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> +	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> +		return;
> +
> +	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> +	/* Wait until cluster power control is applied */
> +	while (time_before(jiffies, timeout)) {
> +		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> +
> +		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> +			return;
> +
> +		cpu_relax();
> +	}
> +	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> +		enable ? "on" : "off");

You should not have to wait for the power status to change here.  
Simply signaling the desired state and returning is all that is 
expected.  And because IRQs are turned off, it is likely that 
time_before(jiffies, timeout) will always be true anyway because jiffies 
are not updated if there is no other CPU to service the timer interrupt.

The actual power status should be polled for in the mcpm_finish() 
method only.

> +}
> +
> +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
> +{
> +	unsigned long mpidr;
> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +		cluster >= EXYNOS5420_NR_CLUSTERS)
> +		return -EINVAL;
> +
> +	/*
> +	 * Since this is called with IRQs enabled, and no arch_spin_lock_irq
> +	 * variant exists, we need to disable IRQs manually here.
> +	 */
> +	local_irq_disable();
> +	arch_spin_lock(&bl_lock);
> +
> +	cpu_use_count[cpu][cluster]++;
> +	if (cpu_use_count[cpu][cluster] == 1) {
> +		bool was_cluster_down =
> +			__mcpm_cluster_state(cluster) == CLUSTER_DOWN;
> +
> +		if (was_cluster_down)
> +			exynos_cluster_power_control(cluster, 1);
> +		exynos_core_power_control(cpu, cluster, 1);
> +
> +		if (was_cluster_down) {
> +			mpidr = read_cpuid_mpidr();
> +			udelay(10);
> +			cci_control_port_by_cpu(mpidr, true);
> +		}

This is completely wrong.  Is this why you created the patch to 
introduce cci_control_port_by_cpu()?  If so I'm NAKing that other patch 
as well.

This is going to be completely ineffective with concurrent usage by 
cpuidle where CPUs in the other cluster are awaken by an interrupt and 
not by calling the cpu_up method.  The current cluster will therefore 
not be aware of the other cluster coming online and system memory 
corruption will occur.

I see below that you do turn off the CCI port for the current cluster 
and the other cluster together, hence the need to enable back the CCI 
for the current cluster here.  Please don't do that.  That's not the 
proper way to achieve that and there are many race conditions to take 
care of before this can be done.  And if we were to go that route, I 
want to be convinced it is worth the needed complexity first i.e. I want 
to see evidence this actually does save power or improve performance by 
a non negligible margin.

> +		/* CPU should be powered up there */
> +		/* Also setup Cluster Power Sequence here */
> +	} else if (cpu_use_count[cpu][cluster] != 2) {
> +		/*
> +		 * The only possible values are:
> +		 * 0 = CPU down
> +		 * 1 = CPU (still) up
> +		 * 2 = CPU requested to be up before it had a chance
> +		 *     to actually make itself down.
> +		 * Any other value is a bug.
> +		 */
> +		BUG();
> +	}
> +
> +	arch_spin_unlock(&bl_lock);
> +	local_irq_enable();
> +
> +	return 0;
> +}
> +
> +static void exynos_power_down(void)
> +{
> +	unsigned int mpidr, cpu, cluster, cpumask;
> +	bool last_man = false, skip_wfi = false;
> +
> +	mpidr = read_cpuid_mpidr();
> +	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	cpumask = (1 << cpu);
> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	__mcpm_cpu_going_down(cpu, cluster);
> +
> +	arch_spin_lock(&bl_lock);
> +	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> +	cpu_use_count[cpu][cluster]--;
> +	if (cpu_use_count[cpu][cluster] == 0) {
> +		int cnt = 0, i;
> +
> +		exynos_core_power_control(cpu, cluster, 0);
> +		for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
> +			cnt += cpu_use_count[i][cluster];
> +		if (cnt == 0)
> +			last_man = true;
> +	} else if (cpu_use_count[cpu][cluster] == 1) {
> +		/*
> +		 * A power_up request went ahead of us.
> +		 * Even if we do not want to shut this CPU down,
> +		 * the caller expects a certain state as if the WFI
> +		 * was aborted.  So let's continue with cache cleaning.
> +		 */
> +		skip_wfi = true;
> +	} else {
> +		BUG();
> +	}
> +
> +	if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
> +		arch_spin_unlock(&bl_lock);
> +
> +		/*
> +		 * Flush all cache levels for this cluster.
> +		 *
> +		 * To do so we do:
> +		 * - Clear the SCTLR.C bit to prevent further cache allocations
> +		 * - Flush the whole cache
> +		 * - Clear the ACTLR "SMP" bit to disable local coherency
> +		 *
> +		 * Let's do it in the safest possible way i.e. with
> +		 * no memory access within the following sequence
> +		 * including to the stack.
> +		 *
> +		 * Note: fp is preserved to the stack explicitly prior doing
> +		 * this since adding it to the clobber list is incompatible
> +		 * with having CONFIG_FRAME_POINTER=y.
> +		 *
> +		 * The common v7_exit_coherency_flush API that could not be
> +		 * used because of the Erratum 799270 workaround.
> +		 */

Bummer.  Could you at least create a macro locally, similar to 
v7_exit_coherency_flush, so not to duplicate this code twice please?

> +		asm volatile(
> +		"stmfd	sp!, {fp, ip}\n\t"
> +		"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t"
> +		"bic	r0, r0, #"__stringify(CR_C)"\n\t"
> +		"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t"
> +		"isb\n\t"
> +		"bl	v7_flush_dcache_all\n\t"
> +		"clrex\n\t"
> +		"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t"
> +		"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t"
> +		/* Dummy Load of a device register to avoid Erratum 799270 */
> +		"ldr	r4, [%0]\n\t"
> +		"and	r4, r4, #0\n\t"
> +		"orr	r0, r0, r4\n\t"
> +		"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t"
> +		"isb\n\t"
> +		"dsb\n\t"
> +		"ldmfd	sp!, {fp, ip}"
> +		:
> +		: "Ir" (S5P_INFORM0)
> +		: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
> +		      "r9", "r10", "lr", "memory");
> +
> +		/*
> +		 * This is a harmless no-op.  On platforms with a real
> +		 * outer cache this might either be needed or not,
> +		 * depending on where the outer cache sits.
> +		 */
> +		outer_flush_all();

If you have no actual outer cache, please remove this call.  In most 
cases with existing outer cache controllers this call is wrong anyway.  
I've already sent a patch removing it from dcscb.c to RMK.

> +		/*
> +		 * Disable cluster-level coherency by masking
> +		 * incoming snoops and DVM messages:
> +		 */
> +		cci_control_port_by_cpu(mpidr, false);
> +		cci_control_port_by_cpu(mpidr ^ (1 << 8), false);

See my comment above for not disabling the remote cluster.

> +
> +		__mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
> +	} else {
> +		arch_spin_unlock(&bl_lock);
> +
> +		/*
> +		 * Flush the local CPU cache.
> +		 * Let's do it in the safest possible way as above.
> +		 */
> +		asm volatile(
> +		"stmfd	sp!, {fp, ip}\n\t"
> +		"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t"
> +		"bic	r0, r0, #"__stringify(CR_C)"\n\t"
> +		"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t"
> +		"isb\n\t"
> +		"bl	v7_flush_dcache_louis\n\t"
> +		"clrex\n\t"
> +		"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t"
> +		"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t"
> +		/* Dummy Load of a device register to avoid Erratum 799270 */
> +		"ldr	r4, [%0]\n\t"
> +		"and	r4, r4, #0\n\t"
> +		"orr	r0, r0, r4\n\t"
> +		"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t"
> +		"isb\n\t"
> +		"dsb\n\t"
> +		"ldmfd	sp!, {fp, ip}"
> +		:
> +		: "Ir" (S5P_INFORM0)
> +		: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
> +		      "r9", "r10", "lr", "memory");

See my note above for not duplicating this code.

> +	}
> +
> +	__mcpm_cpu_down(cpu, cluster);
> +
> +	/* Now we are prepared for power-down, do it: */
> +	dsb();

This dsb is redundant.  The cache maintenance implied by 
__mcpm_cpu_down() already contains a dsb.

> +	if (!skip_wfi)
> +		wfi();
> +
> +	/* Not dead at this point?  Let our caller cope. */
> +}
> +
> +static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
> +{
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	/* Wait for the core state to be OFF */
> +	while (exynos_core_power_state(cpu, cluster) != 0x0)
> +		;

Unlike above, here is the proper location to implement a timeout.

> +
> +	return 0; /* success: the CPU is halted */
> +}
> +
> +static const struct mcpm_platform_ops exynos_power_ops = {
> +	.power_up		= exynos_power_up,
> +	.power_down		= exynos_power_down,
> +	.power_down_finish	= exynos_power_down_finish,
> +};
> +
> +static void __init exynos_mcpm_usage_count_init(void)
> +{
> +	unsigned int mpidr, cpu, cluster, i;
> +
> +	mpidr = read_cpuid_mpidr();
> +	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++) {
> +		cpu_use_count[i][0] = 0;
> +		cpu_use_count[i][1] = 0;
> +	}

Global non-initialized variables are already initialized to zero by 
default.  You therefore don't need to do it here.

> +	cpu_use_count[cpu][cluster] = 1;
> +}
> +
> +static size_t bL_check_status(char *info)
> +{
> +	size_t len = 0;
> +	int i;
> +
> +	len += sprintf(&info[len], "\t0 1 2 3 L2\n");
> +	len += sprintf(&info[len], "[A15]   ");
> +	for (i = 0; i < 4; i++) {
> +		len += sprintf(&info[len], "%d ",
> +			(readl(EXYNOS_ARM_CORE_STATUS(i)) & 0x3) == 3 ? 1 : 0);
> +	}
> +	len += sprintf(&info[len], "%d\n",
> +			(readl(EXYNOS_COMMON_STATUS(0)) & 0x3) == 3 ? 1 : 0);
> +
> +	len += sprintf(&info[len], "[A7]    ");
> +	for (i = 4; i < 8; i++)
> +		len += sprintf(&info[len], "%d ",
> +			(readl(EXYNOS_ARM_CORE_STATUS(i)) & 0x3) == 3 ? 1 : 0);
> +	len += sprintf(&info[len], "%d\n\n",
> +			(readl(EXYNOS_COMMON_STATUS(1)) & 0x3) == 3 ? 1 : 0);
> +
> +	return len;
> +}
> +
> +static ssize_t bL_status_read(struct file *file, char __user *buf,
> +			size_t len, loff_t *pos)
> +{
> +	size_t count = 0;
> +	char info[100];
> +
> +	count = bL_check_status(info);
> +	if (count < 0)
> +		return -EINVAL;
> +
> +	return simple_read_from_buffer(buf, len, pos, info, count);
> +}
> +
> +static const struct file_operations bL_status_fops = {
> +	.read		= bL_status_read,
> +};
> +
> +static struct miscdevice bL_status_device = {
> +	MISC_DYNAMIC_MINOR,
> +	"bL_status",
> +	&bL_status_fops
> +};

Please split this debug code out to a separate patch so it can be 
evaluated on its own.

> +extern void exynos_power_up_setup(unsigned int affinity_level);
> +
> +static int __init exynos_mcpm_init(void)
> +{
> +	int ret = 0;
> +
> +	if (!cci_probed())
> +		return -ENODEV;
> +
> +	if (!soc_is_exynos5420())
> +		return 0;

You should return -ENODEV here as well.  Also this would be better to 
test this before calling cci_probed().

> +	 * To increase the stability of KFC reset we need to program
> +	 * the PMU SPARE3 register
> +	 */
> +	__raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
> +
> +	exynos_mcpm_usage_count_init();
> +
> +	ret = mcpm_platform_register(&exynos_power_ops);
> +	if (!ret)
> +		ret = mcpm_sync_init(exynos_power_up_setup);
> +
> +	pr_info("Exynos MCPM support installed\n");
> +
> +	/*
> +	 * Future entries into the kernel can now go
> +	 * through the cluster entry vectors.
> +	 */
> +	__raw_writel(virt_to_phys(mcpm_entry_point),
> +		REG_ENTRY_ADDR);

You should probably do the positive pr_info() and change the vector 
entry point only when ret is equal to 0.


> +
> +	return ret;
> +}
> +
> +early_initcall(exynos_mcpm_init);
> +
> +static int __init exynos_bl_status_init(void)
> +{
> +	int ret;
> +
> +	if (!soc_is_exynos5420())
> +		return 0;
> +
> +	ret = misc_register(&bL_status_device);
> +	if (ret)
> +		pr_info("bl_status not available\n");
> +	return 0;
> +}
> +
> +late_initcall(exynos_bl_status_init);
> diff --git a/arch/arm/mach-exynos/platsmp.c b/arch/arm/mach-exynos/platsmp.c
> index 03e5e9f..4f14457 100644
> --- a/arch/arm/mach-exynos/platsmp.c
> +++ b/arch/arm/mach-exynos/platsmp.c
> @@ -25,6 +25,7 @@
>  #include <asm/smp_plat.h>
>  #include <asm/smp_scu.h>
>  #include <asm/firmware.h>
> +#include <asm/mcpm.h>
>  
>  #include <plat/cpu.h>
>  
> @@ -226,6 +227,24 @@ static void __init exynos_smp_prepare_cpus(unsigned int max_cpus)
>  	}
>  }
>  
> +bool __init exynos_smp_init(void)
> +{
> +#ifdef CONFIG_MCPM
> +	/*
> +	 * The best way to detect a multi-cluster configuration at the moment
> +	 * is to look for the presence of a CCI in the system.
> +	 * Override the default exynos_smp_ops if so.
> +	 */
> +	struct device_node *node;
> +	node = of_find_compatible_node(NULL, NULL, "arm,cci-400");
> +	if (node && of_device_is_available(node)) {
> +		mcpm_smp_set_ops();
> +		return true;
> +	}
> +#endif
> +	return false;
> +}

Unlike on the Versatile Express, it is likely that you can simply call 
mcpm_smp_set_ops() from exynos_mcpm_init() directly (after a successful 
MCPM install of course) and dispense with this.

> +
>  struct smp_operations exynos_smp_ops __initdata = {
>  	.smp_init_cpus		= exynos_smp_init_cpus,
>  	.smp_prepare_cpus	= exynos_smp_prepare_cpus,
> diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
> index cfbfc575..43fe7a0 100644
> --- a/arch/arm/mach-exynos/regs-pmu.h
> +++ b/arch/arm/mach-exynos/regs-pmu.h
> @@ -38,6 +38,7 @@
>  #define S5P_INFORM5				S5P_PMUREG(0x0814)
>  #define S5P_INFORM6				S5P_PMUREG(0x0818)
>  #define S5P_INFORM7				S5P_PMUREG(0x081C)
> +#define S5P_PMU_SPARE3				S5P_PMUREG(0x090C)
>  
>  #define EXYNOS_IROM_DATA2			S5P_PMUREG(0x0988)
>  
> @@ -540,5 +541,6 @@
>  #define EXYNOS5420_KFC_USE_STANDBY_WFE3				(1 << 23)
>  
>  #define DUR_WAIT_RESET				0xF
> +#define EXYNOS5420_SWRESET_KFC_SEL		0x3
>  
>  #endif /* __ASM_ARCH_REGS_PMU_H */
> -- 
> 1.7.9.5
> 

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Dave Martin]

On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
> On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
> 
> > Add machine-dependent MCPM call-backs for Exynos5420. These are used
> > to power up/down the secondary CPUs during boot, shutdown, s2r and
> > switching.
> > 
> > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> 
> See comments inline.

I won't duplicate Nico's review, but I have a couple of extra comments/
questions, below.

[...]

> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > new file mode 100644
> > index 0000000..46d4968
> > --- /dev/null
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -0,0 +1,444 @@

[...]

> > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> > +{
> > +	unsigned long timeout = jiffies + msecs_to_jiffies(20);
> > +	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> > +
> > +	if (enable)
> > +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> > +
> > +	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> > +	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> > +		return;
> > +
> > +	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> > +	/* Wait until cluster power control is applied */
> > +	while (time_before(jiffies, timeout)) {
> > +		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> > +
> > +		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> > +			return;
> > +
> > +		cpu_relax();
> > +	}
> > +	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> > +		enable ? "on" : "off");
> 
> You should not have to wait for the power status to change here.  
> Simply signaling the desired state and returning is all that is 
> expected.  And because IRQs are turned off, it is likely that 
> time_before(jiffies, timeout) will always be true anyway because jiffies 
> are not updated if there is no other CPU to service the timer interrupt.
> 
> The actual power status should be polled for in the mcpm_finish() 
> method only.

Depending on the power controller, it might be possible for writes to
the controller to be lost or not acted upon, if a previous change is
still pending.

Does this issue apply to the exynos power controller?

If this is the case, it might be necessary to ensure before a power-up
request, that the power controller has caught up and reports the
cluster/CPU as down.  Putting this poll before the write to the
power controller maximises the chance of pipelining useful work
in the meantime.  Putting the poll after the write is the worst case.

> 
> > +}
> > +
> > +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
> > +{
> > +	unsigned long mpidr;
> > +
> > +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> > +	if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> > +		cluster >= EXYNOS5420_NR_CLUSTERS)
> > +		return -EINVAL;
> > +
> > +	/*
> > +	 * Since this is called with IRQs enabled, and no arch_spin_lock_irq
> > +	 * variant exists, we need to disable IRQs manually here.
> > +	 */
> > +	local_irq_disable();
> > +	arch_spin_lock(&bl_lock);
> > +
> > +	cpu_use_count[cpu][cluster]++;
> > +	if (cpu_use_count[cpu][cluster] == 1) {
> > +		bool was_cluster_down =
> > +			__mcpm_cluster_state(cluster) == CLUSTER_DOWN;
> > +
> > +		if (was_cluster_down)
> > +			exynos_cluster_power_control(cluster, 1);
> > +		exynos_core_power_control(cpu, cluster, 1);
> > +
> > +		if (was_cluster_down) {
> > +			mpidr = read_cpuid_mpidr();
> > +			udelay(10);
> > +			cci_control_port_by_cpu(mpidr, true);
> > +		}
> 
> This is completely wrong.  Is this why you created the patch to 
> introduce cci_control_port_by_cpu()?  If so I'm NAKing that other patch 
> as well.
> 
> This is going to be completely ineffective with concurrent usage by 
> cpuidle where CPUs in the other cluster are awaken by an interrupt and 
> not by calling the cpu_up method.  The current cluster will therefore 
> not be aware of the other cluster coming online and system memory 
> corruption will occur.
> 
> I see below that you do turn off the CCI port for the current cluster 
> and the other cluster together, hence the need to enable back the CCI 
> for the current cluster here.  Please don't do that.  That's not the 
> proper way to achieve that and there are many race conditions to take 
> care of before this can be done.  And if we were to go that route, I 
> want to be convinced it is worth the needed complexity first i.e. I want 
> to see evidence this actually does save power or improve performance by 
> a non negligible margin.

Agreed.

The fact that there is no C interface for enabling ACE ports is
deliberate.  For CPUs connected to ACE and managed via MCPM,
it is incorrect to enable CCI via C code, since the safe window
is the window during which all outbound CPUs have reached CLUSTER_DOWN
and all inbound CPUs have not turned their MMU on yet (and thus cannot
execute any general Linux C code).

There might be scenarios involving GPUs and other non-CPU devices
connected to ACE ports where the device cannot enable CCI snoops
for itself -- but this would require a holding-pen protocol to enable
the device to wait and signal a CPU to enable CCI snoops on the device's
behalf before the device proceeds.  It is not the correct solution for
CPU clusters attached to ACE, precisely because we can be more efficient
in that case.

In fact, because you implement a power_up_setup method that calls
cci_enable_port_for_self, CCI snoops are actually enabled twice, making
the above code appear redundant.   Have I missed something?

Cheers
---Dave

> 
> > +		/* CPU should be powered up there */
> > +		/* Also setup Cluster Power Sequence here */
> > +	} else if (cpu_use_count[cpu][cluster] != 2) {
> > +		/*
> > +		 * The only possible values are:
> > +		 * 0 = CPU down
> > +		 * 1 = CPU (still) up
> > +		 * 2 = CPU requested to be up before it had a chance
> > +		 *     to actually make itself down.
> > +		 * Any other value is a bug.
> > +		 */
> > +		BUG();
> > +	}
> > +
> > +	arch_spin_unlock(&bl_lock);
> > +	local_irq_enable();
> > +
> > +	return 0;
> > +}

[...]

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

On Mon, 14 Apr 2014, Dave Martin wrote:

> On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
> > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
> > 
> > > Add machine-dependent MCPM call-backs for Exynos5420. These are used
> > > to power up/down the secondary CPUs during boot, shutdown, s2r and
> > > switching.
> > > 
> > > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> > > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> > > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> > > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > 
> > See comments inline.
> 
> I won't duplicate Nico's review, but I have a couple of extra comments/
> questions, below.
> 
> [...]
> 
> > > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > > new file mode 100644
> > > index 0000000..46d4968
> > > --- /dev/null
> > > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > > @@ -0,0 +1,444 @@
> 
> [...]
> 
> > > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> > > +{
> > > +	unsigned long timeout = jiffies + msecs_to_jiffies(20);
> > > +	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> > > +
> > > +	if (enable)
> > > +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> > > +
> > > +	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> > > +	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> > > +		return;
> > > +
> > > +	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> > > +	/* Wait until cluster power control is applied */
> > > +	while (time_before(jiffies, timeout)) {
> > > +		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> > > +
> > > +		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> > > +			return;
> > > +
> > > +		cpu_relax();
> > > +	}
> > > +	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> > > +		enable ? "on" : "off");
> > 
> > You should not have to wait for the power status to change here.  
> > Simply signaling the desired state and returning is all that is 
> > expected.  And because IRQs are turned off, it is likely that 
> > time_before(jiffies, timeout) will always be true anyway because jiffies 
> > are not updated if there is no other CPU to service the timer interrupt.
> > 
> > The actual power status should be polled for in the mcpm_finish() 
> > method only.
> 
> Depending on the power controller, it might be possible for writes to
> the controller to be lost or not acted upon, if a previous change is
> still pending.
> 
> Does this issue apply to the exynos power controller?

Given the way the code is structured now, I suppose that has not been 
the case so far.

> 
> If this is the case, it might be necessary to ensure before a power-up
> request, that the power controller has caught up and reports the
> cluster/CPU as down.  Putting this poll before the write to the
> power controller maximises the chance of pipelining useful work
> in the meantime.  Putting the poll after the write is the worst case.

More useful might be a check on the actual _control_ bit.  If the 
control bits may be read back then I expect the indicated state will 
happen even if delayed.


Nicolas

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Dave Martin]

On Mon, Apr 14, 2014 at 10:01:27AM -0400, Nicolas Pitre wrote:
> On Mon, 14 Apr 2014, Dave Martin wrote:
> 
> > On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
> > > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
> > > 
> > > > Add machine-dependent MCPM call-backs for Exynos5420. These are used
> > > > to power up/down the secondary CPUs during boot, shutdown, s2r and
> > > > switching.
> > > > 
> > > > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> > > > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> > > > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> > > > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > > 
> > > See comments inline.
> > 
> > I won't duplicate Nico's review, but I have a couple of extra comments/
> > questions, below.
> > 
> > [...]
> > 
> > > > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > > > new file mode 100644
> > > > index 0000000..46d4968
> > > > --- /dev/null
> > > > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > > > @@ -0,0 +1,444 @@
> > 
> > [...]
> > 
> > > > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> > > > +{
> > > > +	unsigned long timeout = jiffies + msecs_to_jiffies(20);
> > > > +	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> > > > +
> > > > +	if (enable)
> > > > +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> > > > +
> > > > +	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> > > > +	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> > > > +		return;
> > > > +
> > > > +	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> > > > +	/* Wait until cluster power control is applied */
> > > > +	while (time_before(jiffies, timeout)) {
> > > > +		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> > > > +
> > > > +		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> > > > +			return;
> > > > +
> > > > +		cpu_relax();
> > > > +	}
> > > > +	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> > > > +		enable ? "on" : "off");
> > > 
> > > You should not have to wait for the power status to change here.  
> > > Simply signaling the desired state and returning is all that is 
> > > expected.  And because IRQs are turned off, it is likely that 
> > > time_before(jiffies, timeout) will always be true anyway because jiffies 
> > > are not updated if there is no other CPU to service the timer interrupt.
> > > 
> > > The actual power status should be polled for in the mcpm_finish() 
> > > method only.
> > 
> > Depending on the power controller, it might be possible for writes to
> > the controller to be lost or not acted upon, if a previous change is
> > still pending.
> > 
> > Does this issue apply to the exynos power controller?
> 
> Given the way the code is structured now, I suppose that has not been 
> the case so far.

It depends on the purpose of the polling loop.  If it was added to resolve
a race between a power-up and a power-down that is complete in MCPM but
still pending in the hardware, than that would suggest the power controller
does have this behaviour.

But I'm just guessing.

Abhilash, can you comment?

> > 
> > If this is the case, it might be necessary to ensure before a power-up
> > request, that the power controller has caught up and reports the
> > cluster/CPU as down.  Putting this poll before the write to the
> > power controller maximises the chance of pipelining useful work
> > in the meantime.  Putting the poll after the write is the worst case.
> 
> More useful might be a check on the actual _control_ bit.  If the 
> control bits may be read back then I expect the indicated state will 
> happen even if delayed.

Quite likely, yes.  It just depends on what the hardware specs say.

Cheers
---Dave

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Dave,

On Tue, Apr 15, 2014 at 2:06 PM, Dave Martin <Dave.Martin@arm.com> wrote:
> On Mon, Apr 14, 2014 at 10:01:27AM -0400, Nicolas Pitre wrote:
>> On Mon, 14 Apr 2014, Dave Martin wrote:
>>
>> > On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
>> > > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>> > >
>> > > > Add machine-dependent MCPM call-backs for Exynos5420. These are used
>> > > > to power up/down the secondary CPUs during boot, shutdown, s2r and
>> > > > switching.
>> > > >
>> > > > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>> > > > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>> > > > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> > > > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> > >
>> > > See comments inline.
>> >
>> > I won't duplicate Nico's review, but I have a couple of extra comments/
>> > questions, below.
>> >
>> > [...]
>> >
>> > > > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> > > > new file mode 100644
>> > > > index 0000000..46d4968
>> > > > --- /dev/null
>> > > > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> > > > @@ -0,0 +1,444 @@
>> >
>> > [...]
>> >
>> > > > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
>> > > > +{
>> > > > +       unsigned long timeout = jiffies + msecs_to_jiffies(20);
>> > > > +       unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> > > > +
>> > > > +       if (enable)
>> > > > +               val = EXYNOS_CORE_LOCAL_PWR_EN;
>> > > > +
>> > > > +       status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> > > > +       if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> > > > +               return;
>> > > > +
>> > > > +       __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
>> > > > +       /* Wait until cluster power control is applied */
>> > > > +       while (time_before(jiffies, timeout)) {
>> > > > +               status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> > > > +
>> > > > +               if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> > > > +                       return;
>> > > > +
>> > > > +               cpu_relax();
>> > > > +       }
>> > > > +       pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
>> > > > +               enable ? "on" : "off");
>> > >
>> > > You should not have to wait for the power status to change here.
>> > > Simply signaling the desired state and returning is all that is
>> > > expected.  And because IRQs are turned off, it is likely that
>> > > time_before(jiffies, timeout) will always be true anyway because jiffies
>> > > are not updated if there is no other CPU to service the timer interrupt.
>> > >
>> > > The actual power status should be polled for in the mcpm_finish()
>> > > method only.
>> >

>> > Depending on the power controller, it might be possible for writes to
>> > the controller to be lost or not acted upon, if a previous change is
>> > still pending.
>> >
>> > Does this issue apply to the exynos power controller?
>>
>> Given the way the code is structured now, I suppose that has not been
>> the case so far.
>
> It depends on the purpose of the polling loop.  If it was added to resolve
> a race between a power-up and a power-down that is complete in MCPM but
> still pending in the hardware, than that would suggest the power controller
> does have this behaviour.
>
> But I'm just guessing.
>
> Abhilash, can you comment?
The polling loop in the above function was added to ensure that the
cluster is switched on before we power on the individual cores in
exynos_power_up. This is only required when the first man comes up.

Regards,
Abhilash
>
>> >
>> > If this is the case, it might be necessary to ensure before a power-up
>> > request, that the power controller has caught up and reports the
>> > cluster/CPU as down.  Putting this poll before the write to the
>> > power controller maximises the chance of pipelining useful work
>> > in the meantime.  Putting the poll after the write is the worst case.
>>
>> More useful might be a check on the actual _control_ bit.  If the
>> control bits may be read back then I expect the indicated state will
>> happen even if delayed.
>
> Quite likely, yes.  It just depends on what the hardware specs say.
>
> Cheers
> ---Dave
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Dave Martin]

On Thu, Apr 17, 2014 at 12:41:31AM +0530, Abhilash Kesavan wrote:
> Hi Dave,
> 
> On Tue, Apr 15, 2014 at 2:06 PM, Dave Martin <Dave.Martin@arm.com> wrote:
> > On Mon, Apr 14, 2014 at 10:01:27AM -0400, Nicolas Pitre wrote:
> >> On Mon, 14 Apr 2014, Dave Martin wrote:
> >>
> >> > On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
> >> > > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
> >> > >
> >> > > > Add machine-dependent MCPM call-backs for Exynos5420. These are used
> >> > > > to power up/down the secondary CPUs during boot, shutdown, s2r and
> >> > > > switching.
> >> > > >
> >> > > > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> >> > > > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> >> > > > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> >> > > > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >> > >
> >> > > See comments inline.
> >> >
> >> > I won't duplicate Nico's review, but I have a couple of extra comments/
> >> > questions, below.
> >> >
> >> > [...]
> >> >
> >> > > > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> >> > > > new file mode 100644
> >> > > > index 0000000..46d4968
> >> > > > --- /dev/null
> >> > > > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> >> > > > @@ -0,0 +1,444 @@
> >> >
> >> > [...]
> >> >
> >> > > > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> >> > > > +{
> >> > > > +       unsigned long timeout = jiffies + msecs_to_jiffies(20);
> >> > > > +       unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> >> > > > +
> >> > > > +       if (enable)
> >> > > > +               val = EXYNOS_CORE_LOCAL_PWR_EN;
> >> > > > +
> >> > > > +       status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> >> > > > +       if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> >> > > > +               return;
> >> > > > +
> >> > > > +       __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> >> > > > +       /* Wait until cluster power control is applied */
> >> > > > +       while (time_before(jiffies, timeout)) {
> >> > > > +               status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> >> > > > +
> >> > > > +               if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> >> > > > +                       return;
> >> > > > +
> >> > > > +               cpu_relax();
> >> > > > +       }
> >> > > > +       pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> >> > > > +               enable ? "on" : "off");
> >> > >
> >> > > You should not have to wait for the power status to change here.
> >> > > Simply signaling the desired state and returning is all that is
> >> > > expected.  And because IRQs are turned off, it is likely that
> >> > > time_before(jiffies, timeout) will always be true anyway because jiffies
> >> > > are not updated if there is no other CPU to service the timer interrupt.
> >> > >
> >> > > The actual power status should be polled for in the mcpm_finish()
> >> > > method only.
> >> >
> 
> >> > Depending on the power controller, it might be possible for writes to
> >> > the controller to be lost or not acted upon, if a previous change is
> >> > still pending.
> >> >
> >> > Does this issue apply to the exynos power controller?
> >>
> >> Given the way the code is structured now, I suppose that has not been
> >> the case so far.
> >
> > It depends on the purpose of the polling loop.  If it was added to resolve
> > a race between a power-up and a power-down that is complete in MCPM but
> > still pending in the hardware, than that would suggest the power controller
> > does have this behaviour.
> >
> > But I'm just guessing.
> >
> > Abhilash, can you comment?
> The polling loop in the above function was added to ensure that the
> cluster is switched on before we power on the individual cores in
> exynos_power_up. This is only required when the first man comes up.

Is this only required on power-up, not power-down?  I don't see a call
to exynos_cluster_power_control(..., 0), but maybe I'm looking in the
wrong place.

I wonder whether this is needed, though.  It imposes a requirement on
the OS to waste time polling for the cluster to come online.  Can the
hardware not cope with a CPU poweron request being pending already before
the cluster is powered on?  If that were possible, there would be no need
for an extra poll.

How long do you expect to need to poll for?

Cheers
---Dave

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Dave,

On Thu, Apr 17, 2014 at 3:33 PM, Dave Martin <Dave.Martin@arm.com> wrote:
> On Thu, Apr 17, 2014 at 12:41:31AM +0530, Abhilash Kesavan wrote:
>> Hi Dave,
>>
>> On Tue, Apr 15, 2014 at 2:06 PM, Dave Martin <Dave.Martin@arm.com> wrote:
>> > On Mon, Apr 14, 2014 at 10:01:27AM -0400, Nicolas Pitre wrote:
>> >> On Mon, 14 Apr 2014, Dave Martin wrote:
>> >>
>> >> > On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
>> >> > > On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>> >> > >
>> >> > > > Add machine-dependent MCPM call-backs for Exynos5420. These are used
>> >> > > > to power up/down the secondary CPUs during boot, shutdown, s2r and
>> >> > > > switching.
>> >> > > >
>> >> > > > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>> >> > > > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>> >> > > > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> >> > > > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> >> > >
>> >> > > See comments inline.
>> >> >
>> >> > I won't duplicate Nico's review, but I have a couple of extra comments/
>> >> > questions, below.
>> >> >
>> >> > [...]
>> >> >
>> >> > > > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> >> > > > new file mode 100644
>> >> > > > index 0000000..46d4968
>> >> > > > --- /dev/null
>> >> > > > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> >> > > > @@ -0,0 +1,444 @@
>> >> >
>> >> > [...]
>> >> >
>> >> > > > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
>> >> > > > +{
>> >> > > > +       unsigned long timeout = jiffies + msecs_to_jiffies(20);
>> >> > > > +       unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> >> > > > +
>> >> > > > +       if (enable)
>> >> > > > +               val = EXYNOS_CORE_LOCAL_PWR_EN;
>> >> > > > +
>> >> > > > +       status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> >> > > > +       if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> >> > > > +               return;
>> >> > > > +
>> >> > > > +       __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
>> >> > > > +       /* Wait until cluster power control is applied */
>> >> > > > +       while (time_before(jiffies, timeout)) {
>> >> > > > +               status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> >> > > > +
>> >> > > > +               if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> >> > > > +                       return;
>> >> > > > +
>> >> > > > +               cpu_relax();
>> >> > > > +       }
>> >> > > > +       pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
>> >> > > > +               enable ? "on" : "off");
>> >> > >
>> >> > > You should not have to wait for the power status to change here.
>> >> > > Simply signaling the desired state and returning is all that is
>> >> > > expected.  And because IRQs are turned off, it is likely that
>> >> > > time_before(jiffies, timeout) will always be true anyway because jiffies
>> >> > > are not updated if there is no other CPU to service the timer interrupt.
>> >> > >
>> >> > > The actual power status should be polled for in the mcpm_finish()
>> >> > > method only.
>> >> >
>>
>> >> > Depending on the power controller, it might be possible for writes to
>> >> > the controller to be lost or not acted upon, if a previous change is
>> >> > still pending.
>> >> >
>> >> > Does this issue apply to the exynos power controller?
>> >>
>> >> Given the way the code is structured now, I suppose that has not been
>> >> the case so far.
>> >
>> > It depends on the purpose of the polling loop.  If it was added to resolve
>> > a race between a power-up and a power-down that is complete in MCPM but
>> > still pending in the hardware, than that would suggest the power controller
>> > does have this behaviour.
>> >
>> > But I'm just guessing.
>> >
>> > Abhilash, can you comment?
>> The polling loop in the above function was added to ensure that the
>> cluster is switched on before we power on the individual cores in
>> exynos_power_up. This is only required when the first man comes up.
>
> Is this only required on power-up, not power-down?  I don't see a call
> to exynos_cluster_power_control(..., 0), but maybe I'm looking in the
> wrong place.
No, you have not missed it. As of now, we are turning on both the L2s
for 8 core bring-up and then not turning it off (if all 4 cores in the
clusters are down). Turning the L2 off (A15 L2 mainly) brings in
significant power savings and I will be looking at adding support for
this next.
>
> I wonder whether this is needed, though.  It imposes a requirement on
> the OS to waste time polling for the cluster to come online.  Can the
> hardware not cope with a CPU poweron request being pending already before
> the cluster is powered on?  If that were possible, there would be no need
> for an extra poll.
No, we need to ensure that the cluster L2 is turned On before we can
turn the core on, the hardware does not handle it.
>
> How long do you expect to need to poll for?
In internal testing when we are turning the cluster on from an off
state, I have seen the loop being executed 3-4 times occasionally
during ageing tests. Generally, the status returns as being powered ON
immediately.

Regards,
Abhilash
>
> Cheers
> ---Dave

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

On Mon, 21 Apr 2014, Abhilash Kesavan wrote:

> Hi Dave,
> 
> On Thu, Apr 17, 2014 at 3:33 PM, Dave Martin <Dave.Martin@arm.com> wrote:
> > On Thu, Apr 17, 2014 at 12:41:31AM +0530, Abhilash Kesavan wrote:
> >> The polling loop in the above function was added to ensure that the
> >> cluster is switched on before we power on the individual cores in
> >> exynos_power_up. This is only required when the first man comes up.
> >
> > Is this only required on power-up, not power-down?  I don't see a call
> > to exynos_cluster_power_control(..., 0), but maybe I'm looking in the
> > wrong place.
> No, you have not missed it. As of now, we are turning on both the L2s
> for 8 core bring-up and then not turning it off (if all 4 cores in the
> clusters are down). Turning the L2 off (A15 L2 mainly) brings in
> significant power savings and I will be looking at adding support for
> this next.

Good.  Please add a comment in the code to that effect in the mean time.

> > I wonder whether this is needed, though.  It imposes a requirement on
> > the OS to waste time polling for the cluster to come online.  Can the
> > hardware not cope with a CPU poweron request being pending already before
> > the cluster is powered on?  If that were possible, there would be no need
> > for an extra poll.
> No, we need to ensure that the cluster L2 is turned On before we can
> turn the core on, the hardware does not handle it.
> >
> > How long do you expect to need to poll for?
> In internal testing when we are turning the cluster on from an off
> state, I have seen the loop being executed 3-4 times occasionally
> during ageing tests. Generally, the status returns as being powered ON
> immediately.

Looping 3-4 times is pretty reasonable.  We certainly can live with 
that.


Nicolas

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Dave.
On Mon, Apr 14, 2014 at 4:11 PM, Dave Martin <Dave.Martin@arm.com> wrote:
> On Fri, Apr 11, 2014 at 04:23:04PM -0400, Nicolas Pitre wrote:
>> On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>>
>> > Add machine-dependent MCPM call-backs for Exynos5420. These are used
>> > to power up/down the secondary CPUs during boot, shutdown, s2r and
>> > switching.
>> >
>> > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>> > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>> > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>>
>> See comments inline.
>
> I won't duplicate Nico's review, but I have a couple of extra comments/
> questions, below.
>
> [...]
>
>> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> > new file mode 100644
>> > index 0000000..46d4968
>> > --- /dev/null
>> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> > @@ -0,0 +1,444 @@
>
> [...]
>
>> > +static void exynos_cluster_power_control(unsigned int cluster, int enable)
>> > +{
>> > +   unsigned long timeout = jiffies + msecs_to_jiffies(20);
>> > +   unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> > +
>> > +   if (enable)
>> > +           val = EXYNOS_CORE_LOCAL_PWR_EN;
>> > +
>> > +   status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> > +   if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> > +           return;
>> > +
>> > +   __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
>> > +   /* Wait until cluster power control is applied */
>> > +   while (time_before(jiffies, timeout)) {
>> > +           status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> > +
>> > +           if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> > +                   return;
>> > +
>> > +           cpu_relax();
>> > +   }
>> > +   pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
>> > +           enable ? "on" : "off");
>>
>> You should not have to wait for the power status to change here.
>> Simply signaling the desired state and returning is all that is
>> expected.  And because IRQs are turned off, it is likely that
>> time_before(jiffies, timeout) will always be true anyway because jiffies
>> are not updated if there is no other CPU to service the timer interrupt.
>>
>> The actual power status should be polled for in the mcpm_finish()
>> method only.
>
> Depending on the power controller, it might be possible for writes to
> the controller to be lost or not acted upon, if a previous change is
> still pending.
>
> Does this issue apply to the exynos power controller?
>
> If this is the case, it might be necessary to ensure before a power-up
> request, that the power controller has caught up and reports the
> cluster/CPU as down.  Putting this poll before the write to the
> power controller maximises the chance of pipelining useful work
> in the meantime.  Putting the poll after the write is the worst case.
>
>>
>> > +}
>> > +
>> > +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>> > +{
>> > +   unsigned long mpidr;
>> > +
>> > +   pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> > +   if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> > +           cluster >= EXYNOS5420_NR_CLUSTERS)
>> > +           return -EINVAL;
>> > +
>> > +   /*
>> > +    * Since this is called with IRQs enabled, and no arch_spin_lock_irq
>> > +    * variant exists, we need to disable IRQs manually here.
>> > +    */
>> > +   local_irq_disable();
>> > +   arch_spin_lock(&bl_lock);
>> > +
>> > +   cpu_use_count[cpu][cluster]++;
>> > +   if (cpu_use_count[cpu][cluster] == 1) {
>> > +           bool was_cluster_down =
>> > +                   __mcpm_cluster_state(cluster) == CLUSTER_DOWN;
>> > +
>> > +           if (was_cluster_down)
>> > +                   exynos_cluster_power_control(cluster, 1);
>> > +           exynos_core_power_control(cpu, cluster, 1);
>> > +
>> > +           if (was_cluster_down) {
>> > +                   mpidr = read_cpuid_mpidr();
>> > +                   udelay(10);
>> > +                   cci_control_port_by_cpu(mpidr, true);
>> > +           }
>>
>> This is completely wrong.  Is this why you created the patch to
>> introduce cci_control_port_by_cpu()?  If so I'm NAKing that other patch
>> as well.
>>
>> This is going to be completely ineffective with concurrent usage by
>> cpuidle where CPUs in the other cluster are awaken by an interrupt and
>> not by calling the cpu_up method.  The current cluster will therefore
>> not be aware of the other cluster coming online and system memory
>> corruption will occur.
>>
>> I see below that you do turn off the CCI port for the current cluster
>> and the other cluster together, hence the need to enable back the CCI
>> for the current cluster here.  Please don't do that.  That's not the
>> proper way to achieve that and there are many race conditions to take
>> care of before this can be done.  And if we were to go that route, I
>> want to be convinced it is worth the needed complexity first i.e. I want
>> to see evidence this actually does save power or improve performance by
>> a non negligible margin.
>
> Agreed.
>
> The fact that there is no C interface for enabling ACE ports is
> deliberate.  For CPUs connected to ACE and managed via MCPM,
> it is incorrect to enable CCI via C code, since the safe window
> is the window during which all outbound CPUs have reached CLUSTER_DOWN
> and all inbound CPUs have not turned their MMU on yet (and thus cannot
> execute any general Linux C code).
>
> There might be scenarios involving GPUs and other non-CPU devices
> connected to ACE ports where the device cannot enable CCI snoops
> for itself -- but this would require a holding-pen protocol to enable
> the device to wait and signal a CPU to enable CCI snoops on the device's
> behalf before the device proceeds.  It is not the correct solution for
> CPU clusters attached to ACE, precisely because we can be more efficient
> in that case.
>
> In fact, because you implement a power_up_setup method that calls
> cci_enable_port_for_self, CCI snoops are actually enabled twice, making
> the above code appear redundant.   Have I missed something?
When a cluster is being turned off the snoops for both the clusters
are being turned off. When the other cluster comes back the snoops are
being turned on for the incoming cluster via power_up_setup and here
for the other cluster. As previously mentioned, I will be dropping
this change.

Regards,
Abhilash
>
> Cheers
> ---Dave
>
>>
>> > +           /* CPU should be powered up there */
>> > +           /* Also setup Cluster Power Sequence here */
>> > +   } else if (cpu_use_count[cpu][cluster] != 2) {
>> > +           /*
>> > +            * The only possible values are:
>> > +            * 0 = CPU down
>> > +            * 1 = CPU (still) up
>> > +            * 2 = CPU requested to be up before it had a chance
>> > +            *     to actually make itself down.
>> > +            * Any other value is a bug.
>> > +            */
>> > +           BUG();
>> > +   }
>> > +
>> > +   arch_spin_unlock(&bl_lock);
>> > +   local_irq_enable();
>> > +
>> > +   return 0;
>> > +}
>
> [...]
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Dave Martin]

On Thu, Apr 17, 2014 at 12:41:22AM +0530, Abhilash Kesavan wrote:

[...]

> > The fact that there is no C interface for enabling ACE ports is
> > deliberate.  For CPUs connected to ACE and managed via MCPM,
> > it is incorrect to enable CCI via C code, since the safe window
> > is the window during which all outbound CPUs have reached CLUSTER_DOWN
> > and all inbound CPUs have not turned their MMU on yet (and thus cannot
> > execute any general Linux C code).
> >
> > There might be scenarios involving GPUs and other non-CPU devices
> > connected to ACE ports where the device cannot enable CCI snoops
> > for itself -- but this would require a holding-pen protocol to enable
> > the device to wait and signal a CPU to enable CCI snoops on the device's
> > behalf before the device proceeds.  It is not the correct solution for
> > CPU clusters attached to ACE, precisely because we can be more efficient
> > in that case.
> >
> > In fact, because you implement a power_up_setup method that calls
> > cci_enable_port_for_self, CCI snoops are actually enabled twice, making
> > the above code appear redundant.   Have I missed something?
> When a cluster is being turned off the snoops for both the clusters
> are being turned off. When the other cluster comes back the snoops are
> being turned on for the incoming cluster via power_up_setup and here
> for the other cluster. As previously mentioned, I will be dropping
> this change.

That's a fair point.  If there is only one cluster alive, turning off
snoops for it should be safe, because there is no second cluster for
it to maintain coherency with.

I don't know whether CCI would ever send a snoop back to the same cluster
that requested it, so it is unclear to me whether turning off snoops for
the last cluster would provide significant power/latency benefits.  But
it could be worth investigating.  Have you done any measurements?

If you have a coherent DMA/CLCD controller or GPU connected to an
ACE-Lite support on CCI however, you would need to leave the last
cluster's snoops enabled to that the other coherent masters can contine
to shoop the cluster's caches.

It might be worth thinking about addressing some of these issues as
future generic improvement, but I suggest to leave it out of your
current patches for simplicity.

Cheers
---Dave

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

On Thu, 17 Apr 2014, Dave Martin wrote:

> On Thu, Apr 17, 2014 at 12:41:22AM +0530, Abhilash Kesavan wrote:
> 
> [...]
> 
> > > The fact that there is no C interface for enabling ACE ports is
> > > deliberate.  For CPUs connected to ACE and managed via MCPM,
> > > it is incorrect to enable CCI via C code, since the safe window
> > > is the window during which all outbound CPUs have reached CLUSTER_DOWN
> > > and all inbound CPUs have not turned their MMU on yet (and thus cannot
> > > execute any general Linux C code).
> > >
> > > There might be scenarios involving GPUs and other non-CPU devices
> > > connected to ACE ports where the device cannot enable CCI snoops
> > > for itself -- but this would require a holding-pen protocol to enable
> > > the device to wait and signal a CPU to enable CCI snoops on the device's
> > > behalf before the device proceeds.  It is not the correct solution for
> > > CPU clusters attached to ACE, precisely because we can be more efficient
> > > in that case.
> > >
> > > In fact, because you implement a power_up_setup method that calls
> > > cci_enable_port_for_self, CCI snoops are actually enabled twice, making
> > > the above code appear redundant.   Have I missed something?
> > When a cluster is being turned off the snoops for both the clusters
> > are being turned off. When the other cluster comes back the snoops are
> > being turned on for the incoming cluster via power_up_setup and here
> > for the other cluster. As previously mentioned, I will be dropping
> > this change.
> 
> That's a fair point.  If there is only one cluster alive, turning off
> snoops for it should be safe, because there is no second cluster for
> it to maintain coherency with.

But that's not that simple as I explained in a previous email.  If the 
other cluster has gone down via cpuidle, it may come back up at any 
moment without warning.  We do have the infrastructure in place to cope 
with possible races handling the CCI within a cluster.  We do not have 
anything for cross cluster races.  And before we do, it is necessary to 
know if it is worth it.


Nicolas

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Dave Martin]

On Thu, Apr 17, 2014 at 04:20:26PM +0100, Nicolas Pitre wrote:
> On Thu, 17 Apr 2014, Dave Martin wrote:
> 
> > On Thu, Apr 17, 2014 at 12:41:22AM +0530, Abhilash Kesavan wrote:
> > 
> > [...]
> > 
> > > > The fact that there is no C interface for enabling ACE ports is
> > > > deliberate.  For CPUs connected to ACE and managed via MCPM,
> > > > it is incorrect to enable CCI via C code, since the safe window
> > > > is the window during which all outbound CPUs have reached CLUSTER_DOWN
> > > > and all inbound CPUs have not turned their MMU on yet (and thus cannot
> > > > execute any general Linux C code).
> > > >
> > > > There might be scenarios involving GPUs and other non-CPU devices
> > > > connected to ACE ports where the device cannot enable CCI snoops
> > > > for itself -- but this would require a holding-pen protocol to enable
> > > > the device to wait and signal a CPU to enable CCI snoops on the device's
> > > > behalf before the device proceeds.  It is not the correct solution for
> > > > CPU clusters attached to ACE, precisely because we can be more efficient
> > > > in that case.
> > > >
> > > > In fact, because you implement a power_up_setup method that calls
> > > > cci_enable_port_for_self, CCI snoops are actually enabled twice, making
> > > > the above code appear redundant.   Have I missed something?
> > > When a cluster is being turned off the snoops for both the clusters
> > > are being turned off. When the other cluster comes back the snoops are
> > > being turned on for the incoming cluster via power_up_setup and here
> > > for the other cluster. As previously mentioned, I will be dropping
> > > this change.
> > 
> > That's a fair point.  If there is only one cluster alive, turning off
> > snoops for it should be safe, because there is no second cluster for
> > it to maintain coherency with.
> 
> But that's not that simple as I explained in a previous email.  If the 
> other cluster has gone down via cpuidle, it may come back up at any 
> moment without warning.  We do have the infrastructure in place to cope 
> with possible races handling the CCI within a cluster.  We do not have 
> anything for cross cluster races.  And before we do, it is necessary to 
> know if it is worth it.

Agreed.  It could be done, perhaps by the approach I already considered
for handling multilevel clusters, but it is far from trivial, and I
would like to see some measurement of the potential benefit before
getting into it.

Cheers
---Dave

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Dave,

On Thu, Apr 17, 2014 at 9:08 PM, Dave Martin <Dave.Martin@arm.com> wrote:
> On Thu, Apr 17, 2014 at 04:20:26PM +0100, Nicolas Pitre wrote:
>> On Thu, 17 Apr 2014, Dave Martin wrote:
>>
>> > On Thu, Apr 17, 2014 at 12:41:22AM +0530, Abhilash Kesavan wrote:
>> >
>> > [...]
>> >
>> > > > The fact that there is no C interface for enabling ACE ports is
>> > > > deliberate.  For CPUs connected to ACE and managed via MCPM,
>> > > > it is incorrect to enable CCI via C code, since the safe window
>> > > > is the window during which all outbound CPUs have reached CLUSTER_DOWN
>> > > > and all inbound CPUs have not turned their MMU on yet (and thus cannot
>> > > > execute any general Linux C code).
>> > > >
>> > > > There might be scenarios involving GPUs and other non-CPU devices
>> > > > connected to ACE ports where the device cannot enable CCI snoops
>> > > > for itself -- but this would require a holding-pen protocol to enable
>> > > > the device to wait and signal a CPU to enable CCI snoops on the device's
>> > > > behalf before the device proceeds.  It is not the correct solution for
>> > > > CPU clusters attached to ACE, precisely because we can be more efficient
>> > > > in that case.
>> > > >
>> > > > In fact, because you implement a power_up_setup method that calls
>> > > > cci_enable_port_for_self, CCI snoops are actually enabled twice, making
>> > > > the above code appear redundant.   Have I missed something?
>> > > When a cluster is being turned off the snoops for both the clusters
>> > > are being turned off. When the other cluster comes back the snoops are
>> > > being turned on for the incoming cluster via power_up_setup and here
>> > > for the other cluster. As previously mentioned, I will be dropping
>> > > this change.
>> >
>> > That's a fair point.  If there is only one cluster alive, turning off
>> > snoops for it should be safe, because there is no second cluster for
>> > it to maintain coherency with.
>>
>> But that's not that simple as I explained in a previous email.  If the
>> other cluster has gone down via cpuidle, it may come back up at any
>> moment without warning.  We do have the infrastructure in place to cope
>> with possible races handling the CCI within a cluster.  We do not have
>> anything for cross cluster races.  And before we do, it is necessary to
>> know if it is worth it.
>
> Agreed.  It could be done, perhaps by the approach I already considered
> for handling multilevel clusters, but it is far from trivial, and I
> would like to see some measurement of the potential benefit before
> getting into it.
I am afraid I do not have any power numbers for this change.

Next version of the patches will be posted soon.

Regards,
Abhilash
>
> Cheers
> ---Dave

[PATCH 5/5] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Nicolas and Dave,

Thanks for the review.

On Sat, Apr 12, 2014 at 1:53 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Fri, 11 Apr 2014, Abhilash Kesavan wrote:
>
>> Add machine-dependent MCPM call-backs for Exynos5420. These are used
>> to power up/down the secondary CPUs during boot, shutdown, s2r and
>> switching.
>>
>> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>
> See comments inline.
>
>> ---
>>  arch/arm/mach-exynos/Kconfig             |    9 +
>>  arch/arm/mach-exynos/Makefile            |    2 +
>>  arch/arm/mach-exynos/common.h            |    1 +
>>  arch/arm/mach-exynos/exynos.c            |    1 +
>>  arch/arm/mach-exynos/mcpm-exynos-setup.S |   35 +++
>>  arch/arm/mach-exynos/mcpm-exynos.c       |  444 ++++++++++++++++++++++++++++++
>>  arch/arm/mach-exynos/platsmp.c           |   19 ++
>>  arch/arm/mach-exynos/regs-pmu.h          |    2 +
>>  8 files changed, 513 insertions(+)
>>  create mode 100644 arch/arm/mach-exynos/mcpm-exynos-setup.S
>>  create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c
>>
>> diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
>> index fc8bf18..a921a80 100644
>> --- a/arch/arm/mach-exynos/Kconfig
>> +++ b/arch/arm/mach-exynos/Kconfig
>> @@ -110,4 +110,13 @@ config SOC_EXYNOS5440
>>
>>  endmenu
>>
>> +config EXYNOS5420_MCPM
>> +     bool "Exynos5420 Multi-Cluster PM support"
>> +     depends on MCPM && SOC_EXYNOS5420
>> +     select ARM_CCI
>> +     help
>> +       Support for Dual Cluster Switching (A15/A7) on Exynos5420.
>> +       This is needed to provide CPU and cluster power management
>> +       on Exynos5420 implementing big.LITTLE.
>
> MCPM is not about "cluster switching".  It is about cluster-wide
> power-up/power-down coordination and race avoidance.  MCPM is relied
> upon by the big.LITTLE switcher, but it is also needed by cpuidle, CPU
> hotplug, etc.  Therefore the first line of the help text is wrong and
> could be omitted entirely.
Will modify the help text.
>
>>  endif
>> diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
>> index a656dbe..776fcbd 100644
>> --- a/arch/arm/mach-exynos/Makefile
>> +++ b/arch/arm/mach-exynos/Makefile
>> @@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)   += firmware.o
>>
>>  plus_sec := $(call as-instr,.arch_extension sec,+sec)
>>  AFLAGS_exynos-smc.o          :=-Wa,-march=armv7-a$(plus_sec)
>> +
>> +obj-$(CONFIG_EXYNOS5420_MCPM)        += mcpm-exynos.o mcpm-exynos-setup.o
>> diff --git a/arch/arm/mach-exynos/common.h b/arch/arm/mach-exynos/common.h
>> index 347afc2..a023ccc 100644
>> --- a/arch/arm/mach-exynos/common.h
>> +++ b/arch/arm/mach-exynos/common.h
>> @@ -51,6 +51,7 @@ static inline void exynos_pm_init(void) {}
>>  extern void exynos_cpu_resume(void);
>>
>>  extern struct smp_operations exynos_smp_ops;
>> +extern bool exynos_smp_init(void);
>>
>>  extern void exynos_cpu_die(unsigned int cpu);
>>
>> diff --git a/arch/arm/mach-exynos/exynos.c b/arch/arm/mach-exynos/exynos.c
>> index b1cf9d5..5b72b5e 100644
>> --- a/arch/arm/mach-exynos/exynos.c
>> +++ b/arch/arm/mach-exynos/exynos.c
>> @@ -412,6 +412,7 @@ DT_MACHINE_START(EXYNOS_DT, "SAMSUNG EXYNOS (Flattened Device Tree)")
>>       /* Maintainer: Thomas Abraham <thomas.abraham@linaro.org> */
>>       /* Maintainer: Kukjin Kim <kgene.kim@samsung.com> */
>>       .smp            = smp_ops(exynos_smp_ops),
>> +     .smp_init       = smp_init_ops(exynos_smp_init),
>>       .map_io         = exynos_init_io,
>>       .init_early     = exynos_firmware_init,
>>       .init_machine   = exynos_dt_machine_init,
>> diff --git a/arch/arm/mach-exynos/mcpm-exynos-setup.S b/arch/arm/mach-exynos/mcpm-exynos-setup.S
>> new file mode 100644
>> index 0000000..990c0d5
>> --- /dev/null
>> +++ b/arch/arm/mach-exynos/mcpm-exynos-setup.S
>> @@ -0,0 +1,35 @@
>> +/*
>> + * Exynos low-level MCPM setup
>> + *
>> + * Copyright (C) 2013-2014 Google, Inc
>> + *
>> + * This software is licensed under the terms of the GNU General Public
>> + * License version 2, as published by the Free Software Foundation, and
>> + * may be copied, distributed, and modified under those terms.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +
>> +ENTRY(exynos_power_up_setup)
>> +
>> +     cmp     r0, #0                  @ check affinity level
>> +     beq     1f
>> +
>> +/*
>> + * Enable cluster-level coherency, in preparation for turning on the MMU.
>> + * The ACTLR SMP bit does not need to be set here, because cpu_resume()
>> + * already restores that.
>> + */
>> +     b       cci_enable_port_for_self
>> +
>> +1:   @ Implementation-specific local CPU setup operations should go here,
>> +     @ if any.  In this case, there is nothing to do.
>> +
>> +     bx      lr
>> +
>> +ENDPROC(exynos_power_up_setup)
>
> Given this is so simple, I'd suggest you simply copy the TC2 version for
> the above code and dispense with this file altogether.  See
> tc2_pm_power_up_setup() in mach-vexpress/tc2_pm.c.
Will remove the file.
>
>> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> new file mode 100644
>> index 0000000..46d4968
>> --- /dev/null
>> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> @@ -0,0 +1,444 @@
>> +/*
>> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>> + *           http://www.samsung.com
>> + *
>> + * arch/arm/mach-exynos/mcpm-exynos.c
>> + *
>> + * Based on arch/arm/mach-vexpress/dcscb.c
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/io.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/miscdevice.h>
>> +#include <linux/fs.h>
>> +#include <linux/delay.h>
>> +#include <linux/arm-cci.h>
>> +
>> +#include <asm/mcpm.h>
>> +#include <asm/cputype.h>
>> +#include <asm/cp15.h>
>> +
>> +#include <plat/cpu.h>
>> +#include "regs-pmu.h"
>> +
>> +#define EXYNOS5420_CPUS_PER_CLUSTER  4
>> +#define EXYNOS5420_NR_CLUSTERS               2
>> +
>> +/* Secondary CPU entry point */
>> +#define REG_ENTRY_ADDR               (S5P_VA_SYSRAM_NS + 0x1C)
>> +
>> +#define EXYNOS_CORE_LOCAL_PWR_EN             0x3
>> +#define EXYNOS_CORE_LOCAL_PWR_DIS            0x0
>> +
>> +#define EXYNOS_ARM_COMMON_CONFIGURATION              S5P_PMUREG(0x2500)
>> +#define EXYNOS_ARM_L2_CONFIGURATION          S5P_PMUREG(0x2600)
>> +
>> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)   \
>> +                     (S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_ARM_CORE_STATUS(_nr)          \
>> +                     (S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
>> +
>> +#define EXYNOS_COMMON_CONFIGURATION(_nr)     \
>> +                     (EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_COMMON_STATUS(_nr)            \
>> +                     (EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
>> +
>> +#define EXYNOS_L2_CONFIGURATION(_nr)         \
>> +                     (EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_L2_STATUS(_nr)                        \
>> +                     (EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
>> +
>> +/*
>> + * We can't use regular spinlocks. In the switcher case, it is possible
>> + * for an outbound CPU to call power_down() after its inbound counterpart
>> + * is already live using the same logical CPU number which trips lockdep
>> + * debugging.
>> + */
>> +static arch_spinlock_t bl_lock = __ARCH_SPIN_LOCK_UNLOCKED;
>
> the bl prefix in "bl_lock" might be confusing.  I'd suggest you name
> this "exynos_mcpm_lock" or similar.
OK.
>
>> +static int
>> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
>> +
>> +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
>> +{
>> +     unsigned int val;
>> +     unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
>> +
>> +     val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
>> +                             EXYNOS_CORE_LOCAL_PWR_EN;
>> +     return !!val;
>> +}
>> +
>> +static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
>> +                                             int enable)
>> +{
>> +     unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> +     unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
>> +
>> +     if (exynos_core_power_state(cpu, cluster) == enable)
>> +             return;
>> +
>> +     if (enable)
>> +             val = EXYNOS_CORE_LOCAL_PWR_EN;
>> +     __raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
>> +}
>> +
>> +static void exynos_cluster_power_control(unsigned int cluster, int enable)
>> +{
>> +     unsigned long timeout = jiffies + msecs_to_jiffies(20);
>> +     unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> +
>> +     if (enable)
>> +             val = EXYNOS_CORE_LOCAL_PWR_EN;
>> +
>> +     status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> +     if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> +             return;
>> +
>> +     __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
>> +     /* Wait until cluster power control is applied */
>> +     while (time_before(jiffies, timeout)) {
>> +             status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> +
>> +             if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> +                     return;
>> +
>> +             cpu_relax();
>> +     }
>> +     pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
>> +             enable ? "on" : "off");
>
> You should not have to wait for the power status to change here.
> Simply signaling the desired state and returning is all that is
> expected.  And because IRQs are turned off, it is likely that
> time_before(jiffies, timeout) will always be true anyway because jiffies
> are not updated if there is no other CPU to service the timer interrupt.
>
> The actual power status should be polled for in the mcpm_finish()
> method only.
>
The above code is used to control cluster power; there is a separate
function being used for core power on/off. As the
mcpm_cpu_power_down_finish call-back is being used for core power down
checks, this should be fine ?
>> +}
>> +
>> +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>> +{
>> +     unsigned long mpidr;
>> +
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +             cluster >= EXYNOS5420_NR_CLUSTERS)
>> +             return -EINVAL;
>> +
>> +     /*
>> +      * Since this is called with IRQs enabled, and no arch_spin_lock_irq
>> +      * variant exists, we need to disable IRQs manually here.
>> +      */
>> +     local_irq_disable();
>> +     arch_spin_lock(&bl_lock);
>> +
>> +     cpu_use_count[cpu][cluster]++;
>> +     if (cpu_use_count[cpu][cluster] == 1) {
>> +             bool was_cluster_down =
>> +                     __mcpm_cluster_state(cluster) == CLUSTER_DOWN;
>> +
>> +             if (was_cluster_down)
>> +                     exynos_cluster_power_control(cluster, 1);
>> +             exynos_core_power_control(cpu, cluster, 1);
>> +
>> +             if (was_cluster_down) {
>> +                     mpidr = read_cpuid_mpidr();
>> +                     udelay(10);
>> +                     cci_control_port_by_cpu(mpidr, true);
>> +             }
>
> This is completely wrong.  Is this why you created the patch to
> introduce cci_control_port_by_cpu()?  If so I'm NAKing that other patch
> as well.
Yes, I had introduced the API because of this change.
>
> This is going to be completely ineffective with concurrent usage by
> cpuidle where CPUs in the other cluster are awaken by an interrupt and
> not by calling the cpu_up method.  The current cluster will therefore
> not be aware of the other cluster coming online and system memory
> corruption will occur.
>
> I see below that you do turn off the CCI port for the current cluster
> and the other cluster together, hence the need to enable back the CCI
> for the current cluster here.  Please don't do that.  That's not the
> proper way to achieve that and there are many race conditions to take
> care of before this can be done.  And if we were to go that route, I
> want to be convinced it is worth the needed complexity first i.e. I want
> to see evidence this actually does save power or improve performance by
> a non negligible margin.
Based on Dave's and your comments on this, I will drop the cci port
enable api patch and modify the
above code. I had not considered the CPUIdle scenario.
>
>> +             /* CPU should be powered up there */
>> +             /* Also setup Cluster Power Sequence here */
>> +     } else if (cpu_use_count[cpu][cluster] != 2) {
>> +             /*
>> +              * The only possible values are:
>> +              * 0 = CPU down
>> +              * 1 = CPU (still) up
>> +              * 2 = CPU requested to be up before it had a chance
>> +              *     to actually make itself down.
>> +              * Any other value is a bug.
>> +              */
>> +             BUG();
>> +     }
>> +
>> +     arch_spin_unlock(&bl_lock);
>> +     local_irq_enable();
>> +
>> +     return 0;
>> +}
>> +
>> +static void exynos_power_down(void)
>> +{
>> +     unsigned int mpidr, cpu, cluster, cpumask;
>> +     bool last_man = false, skip_wfi = false;
>> +
>> +     mpidr = read_cpuid_mpidr();
>> +     cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +     cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +     cpumask = (1 << cpu);
>> +
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +                     cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +     __mcpm_cpu_going_down(cpu, cluster);
>> +
>> +     arch_spin_lock(&bl_lock);
>> +     BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> +     cpu_use_count[cpu][cluster]--;
>> +     if (cpu_use_count[cpu][cluster] == 0) {
>> +             int cnt = 0, i;
>> +
>> +             exynos_core_power_control(cpu, cluster, 0);
>> +             for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
>> +                     cnt += cpu_use_count[i][cluster];
>> +             if (cnt == 0)
>> +                     last_man = true;
>> +     } else if (cpu_use_count[cpu][cluster] == 1) {
>> +             /*
>> +              * A power_up request went ahead of us.
>> +              * Even if we do not want to shut this CPU down,
>> +              * the caller expects a certain state as if the WFI
>> +              * was aborted.  So let's continue with cache cleaning.
>> +              */
>> +             skip_wfi = true;
>> +     } else {
>> +             BUG();
>> +     }
>> +
>> +     if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
>> +             arch_spin_unlock(&bl_lock);
>> +
>> +             /*
>> +              * Flush all cache levels for this cluster.
>> +              *
>> +              * To do so we do:
>> +              * - Clear the SCTLR.C bit to prevent further cache allocations
>> +              * - Flush the whole cache
>> +              * - Clear the ACTLR "SMP" bit to disable local coherency
>> +              *
>> +              * Let's do it in the safest possible way i.e. with
>> +              * no memory access within the following sequence
>> +              * including to the stack.
>> +              *
>> +              * Note: fp is preserved to the stack explicitly prior doing
>> +              * this since adding it to the clobber list is incompatible
>> +              * with having CONFIG_FRAME_POINTER=y.
>> +              *
>> +              * The common v7_exit_coherency_flush API that could not be
>> +              * used because of the Erratum 799270 workaround.
>> +              */
>
> Bummer.  Could you at least create a macro locally, similar to
> v7_exit_coherency_flush, so not to duplicate this code twice please?
Will do.
>
>> +             asm volatile(
>> +             "stmfd  sp!, {fp, ip}\n\t"
>> +             "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t"
>> +             "bic    r0, r0, #"__stringify(CR_C)"\n\t"
>> +             "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t"
>> +             "isb\n\t"
>> +             "bl     v7_flush_dcache_all\n\t"
>> +             "clrex\n\t"
>> +             "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t"
>> +             "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t"
>> +             /* Dummy Load of a device register to avoid Erratum 799270 */
>> +             "ldr    r4, [%0]\n\t"
>> +             "and    r4, r4, #0\n\t"
>> +             "orr    r0, r0, r4\n\t"
>> +             "mcr    p15, 0, r0, c1, c0, 1   @ set ACTLR\n\t"
>> +             "isb\n\t"
>> +             "dsb\n\t"
>> +             "ldmfd  sp!, {fp, ip}"
>> +             :
>> +             : "Ir" (S5P_INFORM0)
>> +             : "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
>> +                   "r9", "r10", "lr", "memory");
>> +
>> +             /*
>> +              * This is a harmless no-op.  On platforms with a real
>> +              * outer cache this might either be needed or not,
>> +              * depending on where the outer cache sits.
>> +              */
>> +             outer_flush_all();
>
> If you have no actual outer cache, please remove this call.  In most
> cases with existing outer cache controllers this call is wrong anyway.
> I've already sent a patch removing it from dcscb.c to RMK.
Will remove this call.
>
>> +             /*
>> +              * Disable cluster-level coherency by masking
>> +              * incoming snoops and DVM messages:
>> +              */
>> +             cci_control_port_by_cpu(mpidr, false);
>> +             cci_control_port_by_cpu(mpidr ^ (1 << 8), false);
>
> See my comment above for not disabling the remote cluster.
OK.
>
>> +
>> +             __mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
>> +     } else {
>> +             arch_spin_unlock(&bl_lock);
>> +
>> +             /*
>> +              * Flush the local CPU cache.
>> +              * Let's do it in the safest possible way as above.
>> +              */
>> +             asm volatile(
>> +             "stmfd  sp!, {fp, ip}\n\t"
>> +             "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t"
>> +             "bic    r0, r0, #"__stringify(CR_C)"\n\t"
>> +             "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t"
>> +             "isb\n\t"
>> +             "bl     v7_flush_dcache_louis\n\t"
>> +             "clrex\n\t"
>> +             "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t"
>> +             "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t"
>> +             /* Dummy Load of a device register to avoid Erratum 799270 */
>> +             "ldr    r4, [%0]\n\t"
>> +             "and    r4, r4, #0\n\t"
>> +             "orr    r0, r0, r4\n\t"
>> +             "mcr    p15, 0, r0, c1, c0, 1   @ set ACTLR\n\t"
>> +             "isb\n\t"
>> +             "dsb\n\t"
>> +             "ldmfd  sp!, {fp, ip}"
>> +             :
>> +             : "Ir" (S5P_INFORM0)
>> +             : "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7",
>> +                   "r9", "r10", "lr", "memory");
>
> See my note above for not duplicating this code.
Will fix.
>
>> +     }
>> +
>> +     __mcpm_cpu_down(cpu, cluster);
>> +
>> +     /* Now we are prepared for power-down, do it: */
>> +     dsb();
>
> This dsb is redundant.  The cache maintenance implied by
> __mcpm_cpu_down() already contains a dsb.
OK, will remove the dsb.
>
>> +     if (!skip_wfi)
>> +             wfi();
>> +
>> +     /* Not dead at this point?  Let our caller cope. */
>> +}
>> +
>> +static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
>> +{
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +                     cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +     /* Wait for the core state to be OFF */
>> +     while (exynos_core_power_state(cpu, cluster) != 0x0)
>> +             ;
>
> Unlike above, here is the proper location to implement a timeout.
>
>> +
>> +     return 0; /* success: the CPU is halted */
>> +}
>> +
>> +static const struct mcpm_platform_ops exynos_power_ops = {
>> +     .power_up               = exynos_power_up,
>> +     .power_down             = exynos_power_down,
>> +     .power_down_finish      = exynos_power_down_finish,
>> +};
>> +
>> +static void __init exynos_mcpm_usage_count_init(void)
>> +{
>> +     unsigned int mpidr, cpu, cluster, i;
>> +
>> +     mpidr = read_cpuid_mpidr();
>> +     cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +     cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
>> +                     cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +     for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++) {
>> +             cpu_use_count[i][0] = 0;
>> +             cpu_use_count[i][1] = 0;
>> +     }
>
> Global non-initialized variables are already initialized to zero by
> default.  You therefore don't need to do it here.
OK.
>
>> +     cpu_use_count[cpu][cluster] = 1;
>> +}
>> +
>> +static size_t bL_check_status(char *info)
>> +{
>> +     size_t len = 0;
>> +     int i;
>> +
>> +     len += sprintf(&info[len], "\t0 1 2 3 L2\n");
>> +     len += sprintf(&info[len], "[A15]   ");
>> +     for (i = 0; i < 4; i++) {
>> +             len += sprintf(&info[len], "%d ",
>> +                     (readl(EXYNOS_ARM_CORE_STATUS(i)) & 0x3) == 3 ? 1 : 0);
>> +     }
>> +     len += sprintf(&info[len], "%d\n",
>> +                     (readl(EXYNOS_COMMON_STATUS(0)) & 0x3) == 3 ? 1 : 0);
>> +
>> +     len += sprintf(&info[len], "[A7]    ");
>> +     for (i = 4; i < 8; i++)
>> +             len += sprintf(&info[len], "%d ",
>> +                     (readl(EXYNOS_ARM_CORE_STATUS(i)) & 0x3) == 3 ? 1 : 0);
>> +     len += sprintf(&info[len], "%d\n\n",
>> +                     (readl(EXYNOS_COMMON_STATUS(1)) & 0x3) == 3 ? 1 : 0);
>> +
>> +     return len;
>> +}
>> +
>> +static ssize_t bL_status_read(struct file *file, char __user *buf,
>> +                     size_t len, loff_t *pos)
>> +{
>> +     size_t count = 0;
>> +     char info[100];
>> +
>> +     count = bL_check_status(info);
>> +     if (count < 0)
>> +             return -EINVAL;
>> +
>> +     return simple_read_from_buffer(buf, len, pos, info, count);
>> +}
>> +
>> +static const struct file_operations bL_status_fops = {
>> +     .read           = bL_status_read,
>> +};
>> +
>> +static struct miscdevice bL_status_device = {
>> +     MISC_DYNAMIC_MINOR,
>> +     "bL_status",
>> +     &bL_status_fops
>> +};
>
> Please split this debug code out to a separate patch so it can be
> evaluated on its own.
Will make a separate patch for this.
>
>> +extern void exynos_power_up_setup(unsigned int affinity_level);
>> +
>> +static int __init exynos_mcpm_init(void)
>> +{
>> +     int ret = 0;
>> +
>> +     if (!cci_probed())
>> +             return -ENODEV;
>> +
>> +     if (!soc_is_exynos5420())
>> +             return 0;
>
> You should return -ENODEV here as well.  Also this would be better to
> test this before calling cci_probed().
OK.
>
>> +      * To increase the stability of KFC reset we need to program
>> +      * the PMU SPARE3 register
>> +      */
>> +     __raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
>> +
>> +     exynos_mcpm_usage_count_init();
>> +
>> +     ret = mcpm_platform_register(&exynos_power_ops);
>> +     if (!ret)
>> +             ret = mcpm_sync_init(exynos_power_up_setup);
>> +
>> +     pr_info("Exynos MCPM support installed\n");
>> +
>> +     /*
>> +      * Future entries into the kernel can now go
>> +      * through the cluster entry vectors.
>> +      */
>> +     __raw_writel(virt_to_phys(mcpm_entry_point),
>> +             REG_ENTRY_ADDR);
>
> You should probably do the positive pr_info() and change the vector
> entry point only when ret is equal to 0.
OK.
>
>
>> +
>> +     return ret;
>> +}
>> +
>> +early_initcall(exynos_mcpm_init);
>> +
>> +static int __init exynos_bl_status_init(void)
>> +{
>> +     int ret;
>> +
>> +     if (!soc_is_exynos5420())
>> +             return 0;
>> +
>> +     ret = misc_register(&bL_status_device);
>> +     if (ret)
>> +             pr_info("bl_status not available\n");
>> +     return 0;
>> +}
>> +
>> +late_initcall(exynos_bl_status_init);
>> diff --git a/arch/arm/mach-exynos/platsmp.c b/arch/arm/mach-exynos/platsmp.c
>> index 03e5e9f..4f14457 100644
>> --- a/arch/arm/mach-exynos/platsmp.c
>> +++ b/arch/arm/mach-exynos/platsmp.c
>> @@ -25,6 +25,7 @@
>>  #include <asm/smp_plat.h>
>>  #include <asm/smp_scu.h>
>>  #include <asm/firmware.h>
>> +#include <asm/mcpm.h>
>>
>>  #include <plat/cpu.h>
>>
>> @@ -226,6 +227,24 @@ static void __init exynos_smp_prepare_cpus(unsigned int max_cpus)
>>       }
>>  }
>>
>> +bool __init exynos_smp_init(void)
>> +{
>> +#ifdef CONFIG_MCPM
>> +     /*
>> +      * The best way to detect a multi-cluster configuration at the moment
>> +      * is to look for the presence of a CCI in the system.
>> +      * Override the default exynos_smp_ops if so.
>> +      */
>> +     struct device_node *node;
>> +     node = of_find_compatible_node(NULL, NULL, "arm,cci-400");
>> +     if (node && of_device_is_available(node)) {
>> +             mcpm_smp_set_ops();
>> +             return true;
>> +     }
>> +#endif
>> +     return false;
>> +}
>
> Unlike on the Versatile Express, it is likely that you can simply call
> mcpm_smp_set_ops() from exynos_mcpm_init() directly (after a successful
> MCPM install of course) and dispense with this.
Calling mcpm_set_ops directly from exynos_mcpm_init works fine; will change.
>
>> +
>>  struct smp_operations exynos_smp_ops __initdata = {
>>       .smp_init_cpus          = exynos_smp_init_cpus,
>>       .smp_prepare_cpus       = exynos_smp_prepare_cpus,
>> diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
>> index cfbfc575..43fe7a0 100644
>> --- a/arch/arm/mach-exynos/regs-pmu.h
>> +++ b/arch/arm/mach-exynos/regs-pmu.h
>> @@ -38,6 +38,7 @@
>>  #define S5P_INFORM5                          S5P_PMUREG(0x0814)
>>  #define S5P_INFORM6                          S5P_PMUREG(0x0818)
>>  #define S5P_INFORM7                          S5P_PMUREG(0x081C)
>> +#define S5P_PMU_SPARE3                               S5P_PMUREG(0x090C)
>>
>>  #define EXYNOS_IROM_DATA2                    S5P_PMUREG(0x0988)
>>
>> @@ -540,5 +541,6 @@
>>  #define EXYNOS5420_KFC_USE_STANDBY_WFE3                              (1 << 23)
>>
>>  #define DUR_WAIT_RESET                               0xF
>> +#define EXYNOS5420_SWRESET_KFC_SEL           0x3
>>
>>  #endif /* __ASM_ARCH_REGS_PMU_H */
>> --
>> 1.7.9.5
>>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Add machine-dependent MCPM call-backs for Exynos5420. These are used
to power up/down the secondary CPUs during boot, shutdown, s2r and
switching.

Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
 arch/arm/mach-exynos/Kconfig       |    8 +
 arch/arm/mach-exynos/Makefile      |    2 +
 arch/arm/mach-exynos/mcpm-exynos.c |  345 ++++++++++++++++++++++++++++++++++++
 arch/arm/mach-exynos/regs-pmu.h    |    2 +
 4 files changed, 357 insertions(+)
 create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c

diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
index fc8bf18..1602abc 100644
--- a/arch/arm/mach-exynos/Kconfig
+++ b/arch/arm/mach-exynos/Kconfig
@@ -110,4 +110,12 @@ config SOC_EXYNOS5440
 
 endmenu
 
+config EXYNOS5420_MCPM
+	bool "Exynos5420 Multi-Cluster PM support"
+	depends on MCPM && SOC_EXYNOS5420
+	select ARM_CCI
+	help
+	  This is needed to provide CPU and cluster power management
+	  on Exynos5420 implementing big.LITTLE.
+
 endif
diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
index a656dbe..01bc9b9 100644
--- a/arch/arm/mach-exynos/Makefile
+++ b/arch/arm/mach-exynos/Makefile
@@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)	+= firmware.o
 
 plus_sec := $(call as-instr,.arch_extension sec,+sec)
 AFLAGS_exynos-smc.o		:=-Wa,-march=armv7-a$(plus_sec)
+
+obj-$(CONFIG_EXYNOS5420_MCPM)	+= mcpm-exynos.o
diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
new file mode 100644
index 0000000..49b9031
--- /dev/null
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -0,0 +1,345 @@
+/*
+ * Copyright (c) 2014 Samsung Electronics Co., Ltd.
+ *		http://www.samsung.com
+ *
+ * arch/arm/mach-exynos/mcpm-exynos.c
+ *
+ * Based on arch/arm/mach-vexpress/dcscb.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/kernel.h>
+#include <linux/io.h>
+#include <linux/spinlock.h>
+#include <linux/uaccess.h>
+#include <linux/miscdevice.h>
+#include <linux/fs.h>
+#include <linux/delay.h>
+#include <linux/arm-cci.h>
+
+#include <asm/mcpm.h>
+#include <asm/cputype.h>
+#include <asm/cp15.h>
+
+#include <plat/cpu.h>
+#include "regs-pmu.h"
+
+#define EXYNOS5420_CPUS_PER_CLUSTER	4
+#define EXYNOS5420_NR_CLUSTERS		2
+
+/* Secondary CPU entry point */
+#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
+
+#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
+#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
+
+#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
+#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
+
+#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
+			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
+#define EXYNOS_ARM_CORE_STATUS(_nr)		\
+			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
+
+#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
+			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
+#define EXYNOS_COMMON_STATUS(_nr)		\
+			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
+
+#define EXYNOS_L2_CONFIGURATION(_nr)		\
+			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
+#define EXYNOS_L2_STATUS(_nr)			\
+			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
+
+/*
+ * The common v7_exit_coherency_flush API could not be used because of the
+ * Erratum 799270 workaround. This macro is the same as the common one (in
+ * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
+ */
+#define exynos_v7_exit_coherency_flush(level) \
+	asm volatile( \
+	"stmfd	sp!, {fp, ip}\n\t"\
+	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
+	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
+	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
+	"isb\n\t"\
+	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
+	"clrex\n\t"\
+	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
+	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
+	/* Dummy Load of a device register to avoid Erratum 799270 */ \
+	"ldr	r4, [%0]\n\t" \
+	"and	r4, r4, #0\n\t" \
+	"orr	r0, r0, r4\n\t" \
+	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
+	"isb\n\t" \
+	"dsb\n\t" \
+	"ldmfd	sp!, {fp, ip}" \
+	: \
+	: "Ir" (S5P_INFORM0) \
+	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
+	  "r9", "r10", "lr", "memory")
+
+/*
+ * We can't use regular spinlocks. In the switcher case, it is possible
+ * for an outbound CPU to call power_down() after its inbound counterpart
+ * is already live using the same logical CPU number which trips lockdep
+ * debugging.
+ */
+static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
+static int
+cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
+
+static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
+{
+	unsigned int val;
+	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
+
+	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
+				EXYNOS_CORE_LOCAL_PWR_EN;
+	return !!val;
+}
+
+static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
+						int enable)
+{
+	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
+	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
+
+	if (exynos_core_power_state(cpu, cluster) == enable)
+		return;
+
+	if (enable)
+		val = EXYNOS_CORE_LOCAL_PWR_EN;
+	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
+}
+
+static void exynos_cluster_power_control(unsigned int cluster, int enable)
+{
+	unsigned long timeout = jiffies + msecs_to_jiffies(20);
+	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
+
+	if (enable)
+		val = EXYNOS_CORE_LOCAL_PWR_EN;
+
+	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
+	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
+		return;
+
+	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
+	/* Wait until cluster power control is applied */
+	while (time_before(jiffies, timeout)) {
+		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
+
+		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
+			return;
+
+		cpu_relax();
+	}
+	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
+		enable ? "on" : "off");
+}
+
+static int exynos_power_up(unsigned int cpu, unsigned int cluster)
+{
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
+		cluster >= EXYNOS5420_NR_CLUSTERS)
+		return -EINVAL;
+
+	/*
+	 * Since this is called with IRQs enabled, and no arch_spin_lock_irq
+	 * variant exists, we need to disable IRQs manually here.
+	 */
+	local_irq_disable();
+	arch_spin_lock(&exynos_mcpm_lock);
+
+	cpu_use_count[cpu][cluster]++;
+	if (cpu_use_count[cpu][cluster] == 1) {
+		bool was_cluster_down =
+			__mcpm_cluster_state(cluster) == CLUSTER_DOWN;
+
+		/*
+		 * Turn on the cluster (L2/COMMON) and then power on the cores.
+		 * TODO: Turn off the clusters when all cores in the cluster
+		 * are down to achieve significant power savings.
+		 */
+		if (was_cluster_down)
+			exynos_cluster_power_control(cluster, 1);
+		exynos_core_power_control(cpu, cluster, 1);
+
+		/* CPU should be powered up there */
+		/* Also setup Cluster Power Sequence here */
+	} else if (cpu_use_count[cpu][cluster] != 2) {
+		/*
+		 * The only possible values are:
+		 * 0 = CPU down
+		 * 1 = CPU (still) up
+		 * 2 = CPU requested to be up before it had a chance
+		 *     to actually make itself down.
+		 * Any other value is a bug.
+		 */
+		BUG();
+	}
+
+	arch_spin_unlock(&exynos_mcpm_lock);
+	local_irq_enable();
+
+	return 0;
+}
+
+static void exynos_power_down(void)
+{
+	unsigned int mpidr, cpu, cluster, cpumask;
+	bool last_man = false, skip_wfi = false;
+
+	mpidr = read_cpuid_mpidr();
+	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	cpumask = (1 << cpu);
+
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
+			cluster >= EXYNOS5420_NR_CLUSTERS);
+
+	__mcpm_cpu_going_down(cpu, cluster);
+
+	arch_spin_lock(&exynos_mcpm_lock);
+	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
+	cpu_use_count[cpu][cluster]--;
+	if (cpu_use_count[cpu][cluster] == 0) {
+		int cnt = 0, i;
+
+		exynos_core_power_control(cpu, cluster, 0);
+		for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
+			cnt += cpu_use_count[i][cluster];
+		if (cnt == 0)
+			last_man = true;
+	} else if (cpu_use_count[cpu][cluster] == 1) {
+		/*
+		 * A power_up request went ahead of us.
+		 * Even if we do not want to shut this CPU down,
+		 * the caller expects a certain state as if the WFI
+		 * was aborted.  So let's continue with cache cleaning.
+		 */
+		skip_wfi = true;
+	} else {
+		BUG();
+	}
+
+	if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
+		arch_spin_unlock(&exynos_mcpm_lock);
+
+		/* Flush all cache levels for this cluster. */
+		exynos_v7_exit_coherency_flush(all);
+
+		/*
+		 * Disable cluster-level coherency by masking
+		 * incoming snoops and DVM messages:
+		 */
+		cci_disable_port_by_cpu(mpidr);
+
+		__mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
+	} else {
+		arch_spin_unlock(&exynos_mcpm_lock);
+
+		/* Disable and flush the local CPU cache. */
+		exynos_v7_exit_coherency_flush(louis);
+	}
+
+	__mcpm_cpu_down(cpu, cluster);
+
+	/* Now we are prepared for power-down, do it: */
+	if (!skip_wfi)
+		wfi();
+
+	/* Not dead@this point?  Let our caller cope. */
+}
+
+static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
+{
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
+			cluster >= EXYNOS5420_NR_CLUSTERS);
+
+	/* Wait for the core state to be OFF */
+	while (exynos_core_power_state(cpu, cluster) != 0x0)
+		;
+
+	return 0; /* success: the CPU is halted */
+}
+
+static const struct mcpm_platform_ops exynos_power_ops = {
+	.power_up		= exynos_power_up,
+	.power_down		= exynos_power_down,
+	.power_down_finish	= exynos_power_down_finish,
+};
+
+static void __init exynos_mcpm_usage_count_init(void)
+{
+	unsigned int mpidr, cpu, cluster;
+
+	mpidr = read_cpuid_mpidr();
+	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+
+	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
+	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
+			cluster >= EXYNOS5420_NR_CLUSTERS);
+
+	cpu_use_count[cpu][cluster] = 1;
+}
+
+/*
+ * Enable cluster-level coherency, in preparation for turning on the MMU.
+ */
+static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
+{
+	asm volatile ("\n"
+	"cmp	r0, #1\n"
+	"bxne	lr\n"
+	"b	cci_enable_port_for_self");
+}
+
+static int __init exynos_mcpm_init(void)
+{
+	int ret = 0;
+
+	if (!soc_is_exynos5420())
+		return -ENODEV;
+
+	if (!cci_probed())
+		return -ENODEV;
+
+	/*
+	 * To increase the stability of KFC reset we need to program
+	 * the PMU SPARE3 register
+	 */
+	__raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
+
+	exynos_mcpm_usage_count_init();
+
+	ret = mcpm_platform_register(&exynos_power_ops);
+	if (!ret)
+		ret = mcpm_sync_init(exynos_pm_power_up_setup);
+	if (ret)
+		return ret;
+
+	mcpm_smp_set_ops();
+
+	pr_info("Exynos MCPM support installed\n");
+
+	/*
+	 * Future entries into the kernel can now go
+	 * through the cluster entry vectors.
+	 */
+	__raw_writel(virt_to_phys(mcpm_entry_point),
+		REG_ENTRY_ADDR);
+
+	return ret;
+}
+
+early_initcall(exynos_mcpm_init);
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index cfbfc575..43fe7a0 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -38,6 +38,7 @@
 #define S5P_INFORM5				S5P_PMUREG(0x0814)
 #define S5P_INFORM6				S5P_PMUREG(0x0818)
 #define S5P_INFORM7				S5P_PMUREG(0x081C)
+#define S5P_PMU_SPARE3				S5P_PMUREG(0x090C)
 
 #define EXYNOS_IROM_DATA2			S5P_PMUREG(0x0988)
 
@@ -540,5 +541,6 @@
 #define EXYNOS5420_KFC_USE_STANDBY_WFE3				(1 << 23)
 
 #define DUR_WAIT_RESET				0xF
+#define EXYNOS5420_SWRESET_KFC_SEL		0x3
 
 #endif /* __ASM_ARCH_REGS_PMU_H */
-- 
1.7.9.5

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Daniel Lezcano]

On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
> Add machine-dependent MCPM call-backs for Exynos5420. These are used
> to power up/down the secondary CPUs during boot, shutdown, s2r and
> switching.
>
> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
>   arch/arm/mach-exynos/Kconfig       |    8 +
>   arch/arm/mach-exynos/Makefile      |    2 +
>   arch/arm/mach-exynos/mcpm-exynos.c |  345 ++++++++++++++++++++++++++++++++++++
>   arch/arm/mach-exynos/regs-pmu.h    |    2 +
>   4 files changed, 357 insertions(+)
>   create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c
>
> diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
> index fc8bf18..1602abc 100644
> --- a/arch/arm/mach-exynos/Kconfig
> +++ b/arch/arm/mach-exynos/Kconfig
> @@ -110,4 +110,12 @@ config SOC_EXYNOS5440
>
>   endmenu
>
> +config EXYNOS5420_MCPM
> +	bool "Exynos5420 Multi-Cluster PM support"
> +	depends on MCPM && SOC_EXYNOS5420
> +	select ARM_CCI
> +	help
> +	  This is needed to provide CPU and cluster power management
> +	  on Exynos5420 implementing big.LITTLE.
> +
>   endif
> diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
> index a656dbe..01bc9b9 100644
> --- a/arch/arm/mach-exynos/Makefile
> +++ b/arch/arm/mach-exynos/Makefile
> @@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)	+= firmware.o
>
>   plus_sec := $(call as-instr,.arch_extension sec,+sec)
>   AFLAGS_exynos-smc.o		:=-Wa,-march=armv7-a$(plus_sec)
> +
> +obj-$(CONFIG_EXYNOS5420_MCPM)	+= mcpm-exynos.o
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> new file mode 100644
> index 0000000..49b9031
> --- /dev/null
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -0,0 +1,345 @@
> +/*
> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> + *		http://www.samsung.com
> + *
> + * arch/arm/mach-exynos/mcpm-exynos.c
> + *
> + * Based on arch/arm/mach-vexpress/dcscb.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +#include <linux/uaccess.h>
> +#include <linux/miscdevice.h>
> +#include <linux/fs.h>
> +#include <linux/delay.h>
> +#include <linux/arm-cci.h>
> +
> +#include <asm/mcpm.h>
> +#include <asm/cputype.h>
> +#include <asm/cp15.h>
> +
> +#include <plat/cpu.h>
> +#include "regs-pmu.h"
> +
> +#define EXYNOS5420_CPUS_PER_CLUSTER	4
> +#define EXYNOS5420_NR_CLUSTERS		2
> +
> +/* Secondary CPU entry point */
> +#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
> +
> +#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
> +#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
>
> +#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
> +#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
> +
> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
> +			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_ARM_CORE_STATUS(_nr)		\
> +			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
> +
> +#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
> +			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_COMMON_STATUS(_nr)		\
> +			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
> +
> +#define EXYNOS_L2_CONFIGURATION(_nr)		\
> +			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_L2_STATUS(_nr)			\
> +			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
> +

Is it possible to share the definition of those macros with the rest of 
the code via functions, so they can be re-used for the other sub-systems 
? eg: https://patches.linaro.org/27798/

> +/*
> + * The common v7_exit_coherency_flush API could not be used because of the
> + * Erratum 799270 workaround. This macro is the same as the common one (in
> + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
> + */
> +#define exynos_v7_exit_coherency_flush(level) \
> +	asm volatile( \
> +	"stmfd	sp!, {fp, ip}\n\t"\
> +	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
> +	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
> +	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
> +	"isb\n\t"\
> +	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
> +	"clrex\n\t"\
> +	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
> +	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
> +	/* Dummy Load of a device register to avoid Erratum 799270 */ \

Wouldn't make sense to add the erratum in the Kconfig and re-use it in 
the generic v7_exit_coherency_flush macro instead of redefining it ?

> +	"ldr	r4, [%0]\n\t" \
> +	"and	r4, r4, #0\n\t" \
> +	"orr	r0, r0, r4\n\t" \
> +	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
> +	"isb\n\t" \
> +	"dsb\n\t" \
> +	"ldmfd	sp!, {fp, ip}" \
> +	: \
> +	: "Ir" (S5P_INFORM0) \
> +	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
> +	  "r9", "r10", "lr", "memory")



> +/*
> + * We can't use regular spinlocks. In the switcher case, it is possible
> + * for an outbound CPU to call power_down() after its inbound counterpart
> + * is already live using the same logical CPU number which trips lockdep
> + * debugging.
> + */
> +static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
> +static int
> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
> +
> +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
> +{
> +	unsigned int val;
> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> +
> +	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
> +				EXYNOS_CORE_LOCAL_PWR_EN;
> +	return !!val;
> +}
> +
> +static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
> +						int enable)
> +{
> +	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> +
> +	if (exynos_core_power_state(cpu, cluster) == enable)
> +		return;
> +
> +	if (enable)
> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> +	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
> +}

Same comment as above about sharing these functions.

Shouldn't these functions to be reused by hotplug ?

> +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> +{
> +	unsigned long timeout = jiffies + msecs_to_jiffies(20);
> +	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> +
> +	if (enable)
> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> +
> +	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> +	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> +		return;
> +
> +	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> +	/* Wait until cluster power control is applied */
> +	while (time_before(jiffies, timeout)) {
> +		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> +
> +		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> +			return;
> +
> +		cpu_relax();
> +	}
> +	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> +		enable ? "on" : "off");
> +}
> +
> +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
> +{
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +		cluster >= EXYNOS5420_NR_CLUSTERS)
> +		return -EINVAL;
> +
> +	/*
> +	 * Since this is called with IRQs enabled, and no arch_spin_lock_irq
> +	 * variant exists, we need to disable IRQs manually here.
> +	 */
> +	local_irq_disable();
> +	arch_spin_lock(&exynos_mcpm_lock);
> +
> +	cpu_use_count[cpu][cluster]++;
> +	if (cpu_use_count[cpu][cluster] == 1) {
> +		bool was_cluster_down =
> +			__mcpm_cluster_state(cluster) == CLUSTER_DOWN;
> +
> +		/*
> +		 * Turn on the cluster (L2/COMMON) and then power on the cores.
> +		 * TODO: Turn off the clusters when all cores in the cluster
> +		 * are down to achieve significant power savings.
> +		 */
> +		if (was_cluster_down)
> +			exynos_cluster_power_control(cluster, 1);
> +		exynos_core_power_control(cpu, cluster, 1);
> +
> +		/* CPU should be powered up there */
> +		/* Also setup Cluster Power Sequence here */
> +	} else if (cpu_use_count[cpu][cluster] != 2) {
> +		/*
> +		 * The only possible values are:
> +		 * 0 = CPU down
> +		 * 1 = CPU (still) up
> +		 * 2 = CPU requested to be up before it had a chance
> +		 *     to actually make itself down.
> +		 * Any other value is a bug.
> +		 */
> +		BUG();
> +	}
> +
> +	arch_spin_unlock(&exynos_mcpm_lock);
> +	local_irq_enable();
> +
> +	return 0;
> +}
> +
> +static void exynos_power_down(void)
> +{
> +	unsigned int mpidr, cpu, cluster, cpumask;
> +	bool last_man = false, skip_wfi = false;
> +
> +	mpidr = read_cpuid_mpidr();
> +	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	cpumask = (1 << cpu);

This variable is not used.

> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	__mcpm_cpu_going_down(cpu, cluster);
> +
> +	arch_spin_lock(&exynos_mcpm_lock);
> +	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> +	cpu_use_count[cpu][cluster]--;
> +	if (cpu_use_count[cpu][cluster] == 0) {
> +		int cnt = 0, i;
> +
> +		exynos_core_power_control(cpu, cluster, 0);
> +		for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
> +			cnt += cpu_use_count[i][cluster];
> +		if (cnt == 0)
> +			last_man = true;
> +	} else if (cpu_use_count[cpu][cluster] == 1) {
> +		/*
> +		 * A power_up request went ahead of us.
> +		 * Even if we do not want to shut this CPU down,
> +		 * the caller expects a certain state as if the WFI
> +		 * was aborted.  So let's continue with cache cleaning.
> +		 */
> +		skip_wfi = true;
> +	} else {
> +		BUG();
> +	}
> +
> +	if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
> +		arch_spin_unlock(&exynos_mcpm_lock);
> +
> +		/* Flush all cache levels for this cluster. */
> +		exynos_v7_exit_coherency_flush(all);
> +
> +		/*
> +		 * Disable cluster-level coherency by masking
> +		 * incoming snoops and DVM messages:
> +		 */
> +		cci_disable_port_by_cpu(mpidr);
> +
> +		__mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
> +	} else {
> +		arch_spin_unlock(&exynos_mcpm_lock);
> +
> +		/* Disable and flush the local CPU cache. */
> +		exynos_v7_exit_coherency_flush(louis);
> +	}
> +
> +	__mcpm_cpu_down(cpu, cluster);
> +
> +	/* Now we are prepared for power-down, do it: */
> +	if (!skip_wfi)
> +		wfi();
> +
> +	/* Not dead at this point?  Let our caller cope. */
> +}
> +
> +static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
> +{
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	/* Wait for the core state to be OFF */
> +	while (exynos_core_power_state(cpu, cluster) != 0x0)
> +		;

As this is a potential infinite loop, a timeout may be desirable, no ?

> +
> +	return 0; /* success: the CPU is halted */
> +}
> +
> +static const struct mcpm_platform_ops exynos_power_ops = {
> +	.power_up		= exynos_power_up,
> +	.power_down		= exynos_power_down,
> +	.power_down_finish	= exynos_power_down_finish,
> +};
> +
> +static void __init exynos_mcpm_usage_count_init(void)
> +{
> +	unsigned int mpidr, cpu, cluster;
> +
> +	mpidr = read_cpuid_mpidr();
> +	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	cpu_use_count[cpu][cluster] = 1;
> +}
> +
> +/*
> + * Enable cluster-level coherency, in preparation for turning on the MMU.
> + */
> +static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
> +{
> +	asm volatile ("\n"
> +	"cmp	r0, #1\n"
> +	"bxne	lr\n"
> +	"b	cci_enable_port_for_self");
> +}
> +
> +static int __init exynos_mcpm_init(void)
> +{
> +	int ret = 0;
> +
> +	if (!soc_is_exynos5420())
> +		return -ENODEV;
> +
> +	if (!cci_probed())
> +		return -ENODEV;
> +
> +	/*
> +	 * To increase the stability of KFC reset we need to program
> +	 * the PMU SPARE3 register
> +	 */
> +	__raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
> +
> +	exynos_mcpm_usage_count_init();
> +
> +	ret = mcpm_platform_register(&exynos_power_ops);
> +	if (!ret)
> +		ret = mcpm_sync_init(exynos_pm_power_up_setup);
> +	if (ret)
> +		return ret;
> +
> +	mcpm_smp_set_ops();
> +
> +	pr_info("Exynos MCPM support installed\n");
> +
> +	/*
> +	 * Future entries into the kernel can now go
> +	 * through the cluster entry vectors.
> +	 */
> +	__raw_writel(virt_to_phys(mcpm_entry_point),
> +		REG_ENTRY_ADDR);
> +
> +	return ret;
> +}
> +
> +early_initcall(exynos_mcpm_init);
> diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
> index cfbfc575..43fe7a0 100644
> --- a/arch/arm/mach-exynos/regs-pmu.h
> +++ b/arch/arm/mach-exynos/regs-pmu.h
> @@ -38,6 +38,7 @@
>   #define S5P_INFORM5				S5P_PMUREG(0x0814)
>   #define S5P_INFORM6				S5P_PMUREG(0x0818)
>   #define S5P_INFORM7				S5P_PMUREG(0x081C)
> +#define S5P_PMU_SPARE3				S5P_PMUREG(0x090C)
>
>   #define EXYNOS_IROM_DATA2			S5P_PMUREG(0x0988)
>
> @@ -540,5 +541,6 @@
>   #define EXYNOS5420_KFC_USE_STANDBY_WFE3				(1 << 23)
>
>   #define DUR_WAIT_RESET				0xF
> +#define EXYNOS5420_SWRESET_KFC_SEL		0x3
>
>   #endif /* __ASM_ARCH_REGS_PMU_H */
>


-- 
  <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

On Tue, 22 Apr 2014, Daniel Lezcano wrote:

> On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
> > Add machine-dependent MCPM call-backs for Exynos5420. These are used
> > to power up/down the secondary CPUs during boot, shutdown, s2r and
> > switching.
> >
> > Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> > Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> > Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> >   arch/arm/mach-exynos/Kconfig       |    8 +
> >   arch/arm/mach-exynos/Makefile      |    2 +
> >   arch/arm/mach-exynos/mcpm-exynos.c |  345
> >   ++++++++++++++++++++++++++++++++++++
> >   arch/arm/mach-exynos/regs-pmu.h    |    2 +
> >   4 files changed, 357 insertions(+)
> >   create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c
> >
> > diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
> > index fc8bf18..1602abc 100644
> > --- a/arch/arm/mach-exynos/Kconfig
> > +++ b/arch/arm/mach-exynos/Kconfig
> > @@ -110,4 +110,12 @@ config SOC_EXYNOS5440
> >
> >   endmenu
> >
> > +config EXYNOS5420_MCPM
> > +	bool "Exynos5420 Multi-Cluster PM support"
> > +	depends on MCPM && SOC_EXYNOS5420
> > +	select ARM_CCI
> > +	help
> > +	  This is needed to provide CPU and cluster power management
> > +	  on Exynos5420 implementing big.LITTLE.
> > +
> >   endif
> > diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
> > index a656dbe..01bc9b9 100644
> > --- a/arch/arm/mach-exynos/Makefile
> > +++ b/arch/arm/mach-exynos/Makefile
> > @@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)	+= firmware.o
> >
> >   plus_sec := $(call as-instr,.arch_extension sec,+sec)
> >   AFLAGS_exynos-smc.o		:=-Wa,-march=armv7-a$(plus_sec)
> > +
> > +obj-$(CONFIG_EXYNOS5420_MCPM)	+= mcpm-exynos.o
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c
> > b/arch/arm/mach-exynos/mcpm-exynos.c
> > new file mode 100644
> > index 0000000..49b9031
> > --- /dev/null
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -0,0 +1,345 @@
> > +/*
> > + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> > + *		http://www.samsung.com
> > + *
> > + * arch/arm/mach-exynos/mcpm-exynos.c
> > + *
> > + * Based on arch/arm/mach-vexpress/dcscb.c
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + */
> > +
> > +#include <linux/kernel.h>
> > +#include <linux/io.h>
> > +#include <linux/spinlock.h>
> > +#include <linux/uaccess.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/fs.h>
> > +#include <linux/delay.h>
> > +#include <linux/arm-cci.h>
> > +
> > +#include <asm/mcpm.h>
> > +#include <asm/cputype.h>
> > +#include <asm/cp15.h>
> > +
> > +#include <plat/cpu.h>
> > +#include "regs-pmu.h"
> > +
> > +#define EXYNOS5420_CPUS_PER_CLUSTER	4
> > +#define EXYNOS5420_NR_CLUSTERS		2
> > +
> > +/* Secondary CPU entry point */
> > +#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
> > +
> > +#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
> > +#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
> >
> > +#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
> > +#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
> > +
> > +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
> > +			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
> > +#define EXYNOS_ARM_CORE_STATUS(_nr)		\
> > +			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
> > +
> > +#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
> > +			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
> > +#define EXYNOS_COMMON_STATUS(_nr)		\
> > +			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
> > +
> > +#define EXYNOS_L2_CONFIGURATION(_nr)		\
> > +			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
> > +#define EXYNOS_L2_STATUS(_nr)			\
> > +			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
> > +
> 
> Is it possible to share the definition of those macros with the rest of the
> code via functions, so they can be re-used for the other sub-systems ? eg:
> https://patches.linaro.org/27798/

This patch is incompatible with MCPM.  A proper idle driver on top of 
MCPM is required instead.  That's the role of MCPM i.e. arbitrate power 
requests between different requestors, including hotplug, cpuidle, and 
the b.L switcher in some cases.

> > +/*
> > + * The common v7_exit_coherency_flush API could not be used because of the
> > + * Erratum 799270 workaround. This macro is the same as the common one (in
> > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
> > + */
> > +#define exynos_v7_exit_coherency_flush(level) \
> > +	asm volatile( \
> > +	"stmfd	sp!, {fp, ip}\n\t"\
> > +	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
> > +	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
> > +	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
> > +	"isb\n\t"\
> > +	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
> > +	"clrex\n\t"\
> > +	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
> > +	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
> > +	/* Dummy Load of a device register to avoid Erratum 799270 */ \
> 
> Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
> generic v7_exit_coherency_flush macro instead of redefining it ?

The implementation of the erratum (the dummy device register load) is 
platform specific I'm afraid.

Is TC2 also concerned by this erratum, or is this Samsung specific?

> > +	"ldr	r4, [%0]\n\t" \
> > +	"and	r4, r4, #0\n\t" \
> > +	"orr	r0, r0, r4\n\t" \
> > +	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
> > +	"isb\n\t" \
> > +	"dsb\n\t" \
> > +	"ldmfd	sp!, {fp, ip}" \
> > +	: \
> > +	: "Ir" (S5P_INFORM0) \
> > +	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
> > +	  "r9", "r10", "lr", "memory")
> 
> 
> 
> > +/*
> > + * We can't use regular spinlocks. In the switcher case, it is possible
> > + * for an outbound CPU to call power_down() after its inbound counterpart
> > + * is already live using the same logical CPU number which trips lockdep
> > + * debugging.
> > + */
> > +static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
> > +static int
> > +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
> > +
> > +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
> > +{
> > +	unsigned int val;
> > +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> > +
> > +	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
> > +				EXYNOS_CORE_LOCAL_PWR_EN;
> > +	return !!val;
> > +}
> > +
> > +static void exynos_core_power_control(unsigned int cpu, unsigned int
> > cluster,
> > +						int enable)
> > +{
> > +	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
> > +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> > +
> > +	if (exynos_core_power_state(cpu, cluster) == enable)
> > +		return;
> > +
> > +	if (enable)
> > +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> > +	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
> > +}
> 
> Same comment as above about sharing these functions.
> 
> Shouldn't these functions to be reused by hotplug ?

This _is_ hotplug.  Once this MCPM backend is registered, CPU hotplug is 
automatically available.  See arch/arm/common/mcpm_platsmp.c.


Nicolas

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Daniel Lezcano]

On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
> On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>
>> On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
>>> Add machine-dependent MCPM call-backs for Exynos5420. These are used
>>> to power up/down the secondary CPUs during boot, shutdown, s2r and
>>> switching.
>>>
>>> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>>> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>>> ---
>>>    arch/arm/mach-exynos/Kconfig       |    8 +
>>>    arch/arm/mach-exynos/Makefile      |    2 +
>>>    arch/arm/mach-exynos/mcpm-exynos.c |  345
>>>    ++++++++++++++++++++++++++++++++++++
>>>    arch/arm/mach-exynos/regs-pmu.h    |    2 +
>>>    4 files changed, 357 insertions(+)
>>>    create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c
>>>
>>> diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
>>> index fc8bf18..1602abc 100644
>>> --- a/arch/arm/mach-exynos/Kconfig
>>> +++ b/arch/arm/mach-exynos/Kconfig
>>> @@ -110,4 +110,12 @@ config SOC_EXYNOS5440
>>>
>>>    endmenu
>>>
>>> +config EXYNOS5420_MCPM
>>> +	bool "Exynos5420 Multi-Cluster PM support"
>>> +	depends on MCPM && SOC_EXYNOS5420
>>> +	select ARM_CCI
>>> +	help
>>> +	  This is needed to provide CPU and cluster power management
>>> +	  on Exynos5420 implementing big.LITTLE.
>>> +
>>>    endif
>>> diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
>>> index a656dbe..01bc9b9 100644
>>> --- a/arch/arm/mach-exynos/Makefile
>>> +++ b/arch/arm/mach-exynos/Makefile
>>> @@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)	+= firmware.o
>>>
>>>    plus_sec := $(call as-instr,.arch_extension sec,+sec)
>>>    AFLAGS_exynos-smc.o		:=-Wa,-march=armv7-a$(plus_sec)
>>> +
>>> +obj-$(CONFIG_EXYNOS5420_MCPM)	+= mcpm-exynos.o
>>> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c
>>> b/arch/arm/mach-exynos/mcpm-exynos.c
>>> new file mode 100644
>>> index 0000000..49b9031
>>> --- /dev/null
>>> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>>> @@ -0,0 +1,345 @@
>>> +/*
>>> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>>> + *		http://www.samsung.com
>>> + *
>>> + * arch/arm/mach-exynos/mcpm-exynos.c
>>> + *
>>> + * Based on arch/arm/mach-vexpress/dcscb.c
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License version 2 as
>>> + * published by the Free Software Foundation.
>>> + */
>>> +
>>> +#include <linux/kernel.h>
>>> +#include <linux/io.h>
>>> +#include <linux/spinlock.h>
>>> +#include <linux/uaccess.h>
>>> +#include <linux/miscdevice.h>
>>> +#include <linux/fs.h>
>>> +#include <linux/delay.h>
>>> +#include <linux/arm-cci.h>
>>> +
>>> +#include <asm/mcpm.h>
>>> +#include <asm/cputype.h>
>>> +#include <asm/cp15.h>
>>> +
>>> +#include <plat/cpu.h>
>>> +#include "regs-pmu.h"
>>> +
>>> +#define EXYNOS5420_CPUS_PER_CLUSTER	4
>>> +#define EXYNOS5420_NR_CLUSTERS		2
>>> +
>>> +/* Secondary CPU entry point */
>>> +#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
>>> +
>>> +#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
>>> +#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
>>>
>>> +#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
>>> +#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
>>> +
>>> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
>>> +			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
>>> +#define EXYNOS_ARM_CORE_STATUS(_nr)		\
>>> +			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
>>> +
>>> +#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
>>> +			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
>>> +#define EXYNOS_COMMON_STATUS(_nr)		\
>>> +			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
>>> +
>>> +#define EXYNOS_L2_CONFIGURATION(_nr)		\
>>> +			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
>>> +#define EXYNOS_L2_STATUS(_nr)			\
>>> +			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
>>> +
>>
>> Is it possible to share the definition of those macros with the rest of the
>> code via functions, so they can be re-used for the other sub-systems ? eg:
>> https://patches.linaro.org/27798/
>
> This patch is incompatible with MCPM.  A proper idle driver on top of
> MCPM is required instead.  That's the role of MCPM i.e. arbitrate power
> requests between different requestors, including hotplug, cpuidle, and
> the b.L switcher in some cases.

The driver is resulting from some empirical testing, so I am not sure 
yet MCPM will work. I will give it a try and I will be happy to convert 
to MCPM if possible.

>>> +/*
>>> + * The common v7_exit_coherency_flush API could not be used because of the
>>> + * Erratum 799270 workaround. This macro is the same as the common one (in
>>> + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
>>> + */
>>> +#define exynos_v7_exit_coherency_flush(level) \
>>> +	asm volatile( \
>>> +	"stmfd	sp!, {fp, ip}\n\t"\
>>> +	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
>>> +	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
>>> +	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
>>> +	"isb\n\t"\
>>> +	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
>>> +	"clrex\n\t"\
>>> +	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
>>> +	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
>>> +	/* Dummy Load of a device register to avoid Erratum 799270 */ \
>>
>> Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
>> generic v7_exit_coherency_flush macro instead of redefining it ?
>
> The implementation of the erratum (the dummy device register load) is
> platform specific I'm afraid.
>
> Is TC2 also concerned by this erratum, or is this Samsung specific?

Sounds like it is ARM Cortex-A15MP specific:

http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf

Page 31.

>>> +	"ldr	r4, [%0]\n\t" \
>>> +	"and	r4, r4, #0\n\t" \
>>> +	"orr	r0, r0, r4\n\t" \
>>> +	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
>>> +	"isb\n\t" \
>>> +	"dsb\n\t" \
>>> +	"ldmfd	sp!, {fp, ip}" \
>>> +	: \
>>> +	: "Ir" (S5P_INFORM0) \
>>> +	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
>>> +	  "r9", "r10", "lr", "memory")
>>
>>
>>
>>> +/*
>>> + * We can't use regular spinlocks. In the switcher case, it is possible
>>> + * for an outbound CPU to call power_down() after its inbound counterpart
>>> + * is already live using the same logical CPU number which trips lockdep
>>> + * debugging.
>>> + */
>>> +static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
>>> +static int
>>> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
>>> +
>>> +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
>>> +{
>>> +	unsigned int val;
>>> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
>>> +
>>> +	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
>>> +				EXYNOS_CORE_LOCAL_PWR_EN;
>>> +	return !!val;
>>> +}
>>> +
>>> +static void exynos_core_power_control(unsigned int cpu, unsigned int
>>> cluster,
>>> +						int enable)
>>> +{
>>> +	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
>>> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
>>> +
>>> +	if (exynos_core_power_state(cpu, cluster) == enable)
>>> +		return;
>>> +
>>> +	if (enable)
>>> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
>>> +	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
>>> +}
>>
>> Same comment as above about sharing these functions.
>>
>> Shouldn't these functions to be reused by hotplug ?
>
> This _is_ hotplug.  Once this MCPM backend is registered, CPU hotplug is
> automatically available.  See arch/arm/common/mcpm_platsmp.c.

Sorry, I think I was not clear. What I meant is : couldn't these macros 
be converted to wrapper functions (powerdown, powerup, powerstate, etc 
...) and be generic for the entire EXYNOS arch, so the different sub 
systems can reuse it (eg. hotplug.c) instead of defining macros + 
read_raw + writel_raw all over the place (eg. hotplug.c + smp.c + mcpm [ 
+ exynos4 dual cpus for cpuidle ]).

Thanks

   -- Daniel


-- 
  <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

[ Moved Lorenzo up in the addressee list to get his attention ]

On Tue, 22 Apr 2014, Daniel Lezcano wrote:

> On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
> > On Tue, 22 Apr 2014, Daniel Lezcano wrote:
> >
> > > On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
> > > > +/*
> > > > + * The common v7_exit_coherency_flush API could not be used because of the
> > > > + * Erratum 799270 workaround. This macro is the same as the common one (in
> > > > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
> > > > + */
> > > > +#define exynos_v7_exit_coherency_flush(level) \
> > > > +	asm volatile( \
> > > > +	"stmfd	sp!, {fp, ip}\n\t"\
> > > > +	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
> > > > +	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
> > > > +	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
> > > > +	"isb\n\t"\
> > > > +	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
> > > > +	"clrex\n\t"\
> > > > +	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
> > > > +	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
> > > > +	/* Dummy Load of a device register to avoid Erratum 799270 */ \
> > >
> > > Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
> > > generic v7_exit_coherency_flush macro instead of redefining it ?
> >
> > The implementation of the erratum (the dummy device register load) is
> > platform specific I'm afraid.
> >
> > Is TC2 also concerned by this erratum, or is this Samsung specific?
> 
> Sounds like it is ARM Cortex-A15MP specific:
> 
> http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf
> 
> Page 31.

The condition for hitting the erratum is: "The L2 cache block has been 
idle for 256 or more cycles with no memory requests from any core, no 
external snoops, and no ACP requests".

Given that the operation that almost immediately precedes the 
problematic mcr is a call to v7_flush_dcache_louis or 
v7_flush_dcache_all, is this possible that the condition for the erratum 
could ever be met?

If no then we should document that v7_exit_coherency_flush(() is safe 
against erratum 799270 and use it here.

> > > > +	"ldr	r4, [%0]\n\t" \
> > > > +	"and	r4, r4, #0\n\t" \
> > > > +	"orr	r0, r0, r4\n\t" \
> > > > +	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
> > > > +	"isb\n\t" \
> > > > +	"dsb\n\t" \
> > > > +	"ldmfd	sp!, {fp, ip}" \
> > > > +	: \
> > > > +	: "Ir" (S5P_INFORM0) \
> > > > +	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
> > > > +	  "r9", "r10", "lr", "memory")
> > >

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Nicolas,

On Wed, Apr 23, 2014 at 1:26 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>
> [ Moved Lorenzo up in the addressee list to get his attention ]
>
> On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>
>> On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
>> > On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>> >
>> > > On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
>> > > > +/*
>> > > > + * The common v7_exit_coherency_flush API could not be used because of the
>> > > > + * Erratum 799270 workaround. This macro is the same as the common one (in
>> > > > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
>> > > > + */
>> > > > +#define exynos_v7_exit_coherency_flush(level) \
>> > > > +       asm volatile( \
>> > > > +       "stmfd  sp!, {fp, ip}\n\t"\
>> > > > +       "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t" \
>> > > > +       "bic    r0, r0, #"__stringify(CR_C)"\n\t" \
>> > > > +       "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t" \
>> > > > +       "isb\n\t"\
>> > > > +       "bl     v7_flush_dcache_"__stringify(level)"\n\t" \
>> > > > +       "clrex\n\t"\
>> > > > +       "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t" \
>> > > > +       "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t" \
>> > > > +       /* Dummy Load of a device register to avoid Erratum 799270 */ \
>> > >
>> > > Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
>> > > generic v7_exit_coherency_flush macro instead of redefining it ?
>> >
>> > The implementation of the erratum (the dummy device register load) is
>> > platform specific I'm afraid.
>> >
>> > Is TC2 also concerned by this erratum, or is this Samsung specific?
>>
>> Sounds like it is ARM Cortex-A15MP specific:
>>
>> http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf
>>
>> Page 31.
>
> The condition for hitting the erratum is: "The L2 cache block has been
> idle for 256 or more cycles with no memory requests from any core, no
> external snoops, and no ACP requests".
>
> Given that the operation that almost immediately precedes the
> problematic mcr is a call to v7_flush_dcache_louis or
> v7_flush_dcache_all, is this possible that the condition for the erratum
> could ever be met?
>
> If no then we should document that v7_exit_coherency_flush(() is safe
> against erratum 799270 and use it here.
We have L2 dynamic clock gating enabled  (Force L2 logic clock enable
bit in L2ACTLR disabled) for our SoC. During initial development we
added the workaround to be on the safe side.
However, as you pointed out it takes 256 idle cycles for the L2 clock
gating to kick in and so we might never hit the erratum with
v7_exit_coherency_flush(). I will do some ageing tests without the
dummy load and confirm.

Regards,
Abhilash
>
>> > > > +       "ldr    r4, [%0]\n\t" \
>> > > > +       "and    r4, r4, #0\n\t" \
>> > > > +       "orr    r0, r0, r4\n\t" \
>> > > > +       "mcr    p15, 0, r0, c1, c0, 1   @ set ACTLR\n\t" \
>> > > > +       "isb\n\t" \
>> > > > +       "dsb\n\t" \
>> > > > +       "ldmfd  sp!, {fp, ip}" \
>> > > > +       : \
>> > > > +       : "Ir" (S5P_INFORM0) \
>> > > > +       : "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
>> > > > +         "r9", "r10", "lr", "memory")
>> > >
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Lorenzo Pieralisi]

On Tue, Apr 22, 2014 at 08:56:23PM +0100, Nicolas Pitre wrote:
> 
> [ Moved Lorenzo up in the addressee list to get his attention ]

Sorry for the delay in replying.

> On Tue, 22 Apr 2014, Daniel Lezcano wrote:
> 
> > On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
> > > On Tue, 22 Apr 2014, Daniel Lezcano wrote:
> > >
> > > > On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
> > > > > +/*
> > > > > + * The common v7_exit_coherency_flush API could not be used because of the
> > > > > + * Erratum 799270 workaround. This macro is the same as the common one (in
> > > > > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
> > > > > + */
> > > > > +#define exynos_v7_exit_coherency_flush(level) \
> > > > > +	asm volatile( \
> > > > > +	"stmfd	sp!, {fp, ip}\n\t"\
> > > > > +	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
> > > > > +	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
> > > > > +	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
> > > > > +	"isb\n\t"\
> > > > > +	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
> > > > > +	"clrex\n\t"\
> > > > > +	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
> > > > > +	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
> > > > > +	/* Dummy Load of a device register to avoid Erratum 799270 */ \
> > > >
> > > > Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
> > > > generic v7_exit_coherency_flush macro instead of redefining it ?
> > >
> > > The implementation of the erratum (the dummy device register load) is
> > > platform specific I'm afraid.
> > >
> > > Is TC2 also concerned by this erratum, or is this Samsung specific?
> > 
> > Sounds like it is ARM Cortex-A15MP specific:
> > 
> > http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf
> > 
> > Page 31.
> 
> The condition for hitting the erratum is: "The L2 cache block has been 
> idle for 256 or more cycles with no memory requests from any core, no 
> external snoops, and no ACP requests".
> 
> Given that the operation that almost immediately precedes the 
> problematic mcr is a call to v7_flush_dcache_louis or 
> v7_flush_dcache_all, is this possible that the condition for the erratum 
> could ever be met?

Yes, if we execute out of the I-cache (no-data accesses) I think this might be
triggered.

On a side note, the errata affects also the setting of SMP bit
(__v7_ca15mp_setup and cpu_v7_do_resume) but we do nothing there (it
is a different situation though, MMU there is off and D-cache off...but
I-cache is allowed to be on, so I am actually checking if we should be doing
something there too).

> If no then we should document that v7_exit_coherency_flush(() is safe 
> against erratum 799270 and use it here.

Eh, I wish we could use it safely, I am currently investigating and will get
back to you asap.

Lorenzo

> > > > > +	"ldr	r4, [%0]\n\t" \
> > > > > +	"and	r4, r4, #0\n\t" \
> > > > > +	"orr	r0, r0, r4\n\t" \
> > > > > +	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
> > > > > +	"isb\n\t" \
> > > > > +	"dsb\n\t" \
> > > > > +	"ldmfd	sp!, {fp, ip}" \
> > > > > +	: \
> > > > > +	: "Ir" (S5P_INFORM0) \
> > > > > +	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
> > > > > +	  "r9", "r10", "lr", "memory")
> > > >
> 

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Lorenzo,

On Wed, Apr 23, 2014 at 9:43 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Tue, Apr 22, 2014 at 08:56:23PM +0100, Nicolas Pitre wrote:
>>
>> [ Moved Lorenzo up in the addressee list to get his attention ]
>
> Sorry for the delay in replying.
>
>> On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>>
>> > On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
>> > > On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>> > >
>> > > > On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
>> > > > > +/*
>> > > > > + * The common v7_exit_coherency_flush API could not be used because of the
>> > > > > + * Erratum 799270 workaround. This macro is the same as the common one (in
>> > > > > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
>> > > > > + */
>> > > > > +#define exynos_v7_exit_coherency_flush(level) \
>> > > > > +     asm volatile( \
>> > > > > +     "stmfd  sp!, {fp, ip}\n\t"\
>> > > > > +     "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t" \
>> > > > > +     "bic    r0, r0, #"__stringify(CR_C)"\n\t" \
>> > > > > +     "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t" \
>> > > > > +     "isb\n\t"\
>> > > > > +     "bl     v7_flush_dcache_"__stringify(level)"\n\t" \
>> > > > > +     "clrex\n\t"\
>> > > > > +     "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t" \
>> > > > > +     "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t" \
>> > > > > +     /* Dummy Load of a device register to avoid Erratum 799270 */ \
>> > > >
>> > > > Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
>> > > > generic v7_exit_coherency_flush macro instead of redefining it ?
>> > >
>> > > The implementation of the erratum (the dummy device register load) is
>> > > platform specific I'm afraid.
>> > >
>> > > Is TC2 also concerned by this erratum, or is this Samsung specific?
>> >
>> > Sounds like it is ARM Cortex-A15MP specific:
>> >
>> > http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf
>> >
>> > Page 31.
>>
>> The condition for hitting the erratum is: "The L2 cache block has been
>> idle for 256 or more cycles with no memory requests from any core, no
>> external snoops, and no ACP requests".
>>
>> Given that the operation that almost immediately precedes the
>> problematic mcr is a call to v7_flush_dcache_louis or
>> v7_flush_dcache_all, is this possible that the condition for the erratum
>> could ever be met?
>
> Yes, if we execute out of the I-cache (no-data accesses) I think this might be
> triggered.
>
> On a side note, the errata affects also the setting of SMP bit
> (__v7_ca15mp_setup and cpu_v7_do_resume) but we do nothing there (it
> is a different situation though, MMU there is off and D-cache off...but
> I-cache is allowed to be on, so I am actually checking if we should be doing
> something there too).
>
>> If no then we should document that v7_exit_coherency_flush(() is safe
>> against erratum 799270 and use it here.
>
> Eh, I wish we could use it safely, I am currently investigating and will get
> back to you asap.

Have you been able to come to a conclusion regarding the necessity of
the erratum workaround ? We are currently carrying the workaround in
the merged exynos back-end.

Regards,
Abhilash
>
> Lorenzo
>
>> > > > > +     "ldr    r4, [%0]\n\t" \
>> > > > > +     "and    r4, r4, #0\n\t" \
>> > > > > +     "orr    r0, r0, r4\n\t" \
>> > > > > +     "mcr    p15, 0, r0, c1, c0, 1   @ set ACTLR\n\t" \
>> > > > > +     "isb\n\t" \
>> > > > > +     "dsb\n\t" \
>> > > > > +     "ldmfd  sp!, {fp, ip}" \
>> > > > > +     : \
>> > > > > +     : "Ir" (S5P_INFORM0) \
>> > > > > +     : "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
>> > > > > +       "r9", "r10", "lr", "memory")
>> > > >
>>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Lorenzo Pieralisi]

On Tue, Jun 17, 2014 at 03:38:29AM +0100, Abhilash Kesavan wrote:
> Hi Lorenzo,
> 
> On Wed, Apr 23, 2014 at 9:43 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> > On Tue, Apr 22, 2014 at 08:56:23PM +0100, Nicolas Pitre wrote:
> >>
> >> [ Moved Lorenzo up in the addressee list to get his attention ]
> >
> > Sorry for the delay in replying.
> >
> >> On Tue, 22 Apr 2014, Daniel Lezcano wrote:
> >>
> >> > On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
> >> > > On Tue, 22 Apr 2014, Daniel Lezcano wrote:
> >> > >
> >> > > > On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
> >> > > > > +/*
> >> > > > > + * The common v7_exit_coherency_flush API could not be used because of the
> >> > > > > + * Erratum 799270 workaround. This macro is the same as the common one (in
> >> > > > > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
> >> > > > > + */
> >> > > > > +#define exynos_v7_exit_coherency_flush(level) \
> >> > > > > +     asm volatile( \
> >> > > > > +     "stmfd  sp!, {fp, ip}\n\t"\
> >> > > > > +     "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t" \
> >> > > > > +     "bic    r0, r0, #"__stringify(CR_C)"\n\t" \
> >> > > > > +     "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t" \
> >> > > > > +     "isb\n\t"\
> >> > > > > +     "bl     v7_flush_dcache_"__stringify(level)"\n\t" \
> >> > > > > +     "clrex\n\t"\
> >> > > > > +     "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t" \
> >> > > > > +     "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t" \
> >> > > > > +     /* Dummy Load of a device register to avoid Erratum 799270 */ \
> >> > > >
> >> > > > Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
> >> > > > generic v7_exit_coherency_flush macro instead of redefining it ?
> >> > >
> >> > > The implementation of the erratum (the dummy device register load) is
> >> > > platform specific I'm afraid.
> >> > >
> >> > > Is TC2 also concerned by this erratum, or is this Samsung specific?
> >> >
> >> > Sounds like it is ARM Cortex-A15MP specific:
> >> >
> >> > http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf
> >> >
> >> > Page 31.
> >>
> >> The condition for hitting the erratum is: "The L2 cache block has been
> >> idle for 256 or more cycles with no memory requests from any core, no
> >> external snoops, and no ACP requests".
> >>
> >> Given that the operation that almost immediately precedes the
> >> problematic mcr is a call to v7_flush_dcache_louis or
> >> v7_flush_dcache_all, is this possible that the condition for the erratum
> >> could ever be met?
> >
> > Yes, if we execute out of the I-cache (no-data accesses) I think this might be
> > triggered.
> >
> > On a side note, the errata affects also the setting of SMP bit
> > (__v7_ca15mp_setup and cpu_v7_do_resume) but we do nothing there (it
> > is a different situation though, MMU there is off and D-cache off...but
> > I-cache is allowed to be on, so I am actually checking if we should be doing
> > something there too).
> >
> >> If no then we should document that v7_exit_coherency_flush(() is safe
> >> against erratum 799270 and use it here.
> >
> > Eh, I wish we could use it safely, I am currently investigating and will get
> > back to you asap.
> 
> Have you been able to come to a conclusion regarding the necessity of
> the erratum workaround ? We are currently carrying the workaround in
> the merged exynos back-end.

Yes. There is a way to avoid it, but we would end up deviating from the
standard power down procedure, so I am keener on duplicating the macro and
adding errata code there (current code) rather than rewriting the whole
thing, which has to be retested and we will still have a procedure that
differs from the standard one to work around that issue. At least as it is
it is explicit.

Thanks,
Lorenzo

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Lorenzo,

On Thu, Jun 19, 2014 at 2:26 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Tue, Jun 17, 2014 at 03:38:29AM +0100, Abhilash Kesavan wrote:
>> Hi Lorenzo,
>>
>> On Wed, Apr 23, 2014 at 9:43 PM, Lorenzo Pieralisi
>> <lorenzo.pieralisi@arm.com> wrote:
>> > On Tue, Apr 22, 2014 at 08:56:23PM +0100, Nicolas Pitre wrote:
>> >>
>> >> [ Moved Lorenzo up in the addressee list to get his attention ]
>> >
>> > Sorry for the delay in replying.
>> >
>> >> On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>> >>
>> >> > On 04/22/2014 05:40 PM, Nicolas Pitre wrote:
>> >> > > On Tue, 22 Apr 2014, Daniel Lezcano wrote:
>> >> > >
>> >> > > > On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
>> >> > > > > +/*
>> >> > > > > + * The common v7_exit_coherency_flush API could not be used because of the
>> >> > > > > + * Erratum 799270 workaround. This macro is the same as the common one (in
>> >> > > > > + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
>> >> > > > > + */
>> >> > > > > +#define exynos_v7_exit_coherency_flush(level) \
>> >> > > > > +     asm volatile( \
>> >> > > > > +     "stmfd  sp!, {fp, ip}\n\t"\
>> >> > > > > +     "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t" \
>> >> > > > > +     "bic    r0, r0, #"__stringify(CR_C)"\n\t" \
>> >> > > > > +     "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t" \
>> >> > > > > +     "isb\n\t"\
>> >> > > > > +     "bl     v7_flush_dcache_"__stringify(level)"\n\t" \
>> >> > > > > +     "clrex\n\t"\
>> >> > > > > +     "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t" \
>> >> > > > > +     "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t" \
>> >> > > > > +     /* Dummy Load of a device register to avoid Erratum 799270 */ \
>> >> > > >
>> >> > > > Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
>> >> > > > generic v7_exit_coherency_flush macro instead of redefining it ?
>> >> > >
>> >> > > The implementation of the erratum (the dummy device register load) is
>> >> > > platform specific I'm afraid.
>> >> > >
>> >> > > Is TC2 also concerned by this erratum, or is this Samsung specific?
>> >> >
>> >> > Sounds like it is ARM Cortex-A15MP specific:
>> >> >
>> >> > http://infocenter.arm.com/help/topic/com.arm.doc.epm028090/cortex_a15_mpcore_software_developers_errata_notice_r2_v12.pdf
>> >> >
>> >> > Page 31.
>> >>
>> >> The condition for hitting the erratum is: "The L2 cache block has been
>> >> idle for 256 or more cycles with no memory requests from any core, no
>> >> external snoops, and no ACP requests".
>> >>
>> >> Given that the operation that almost immediately precedes the
>> >> problematic mcr is a call to v7_flush_dcache_louis or
>> >> v7_flush_dcache_all, is this possible that the condition for the erratum
>> >> could ever be met?
>> >
>> > Yes, if we execute out of the I-cache (no-data accesses) I think this might be
>> > triggered.
>> >
>> > On a side note, the errata affects also the setting of SMP bit
>> > (__v7_ca15mp_setup and cpu_v7_do_resume) but we do nothing there (it
>> > is a different situation though, MMU there is off and D-cache off...but
>> > I-cache is allowed to be on, so I am actually checking if we should be doing
>> > something there too).
>> >
>> >> If no then we should document that v7_exit_coherency_flush(() is safe
>> >> against erratum 799270 and use it here.
>> >
>> > Eh, I wish we could use it safely, I am currently investigating and will get
>> > back to you asap.
>>
>> Have you been able to come to a conclusion regarding the necessity of
>> the erratum workaround ? We are currently carrying the workaround in
>> the merged exynos back-end.
>
> Yes. There is a way to avoid it, but we would end up deviating from the
> standard power down procedure, so I am keener on duplicating the macro and
> adding errata code there (current code) rather than rewriting the whole
> thing, which has to be retested and we will still have a procedure that
> differs from the standard one to work around that issue. At least as it is
> it is explicit.
Thanks for checking. I will leave the code as it is then.

Regards,
Abhilash
>
> Thanks,
> Lorenzo
>

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Daniel,
On Tue, Apr 22, 2014 at 4:51 PM, Daniel Lezcano
<daniel.lezcano@linaro.org> wrote:
> On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
>>
>> Add machine-dependent MCPM call-backs for Exynos5420. These are used
>> to power up/down the secondary CPUs during boot, shutdown, s2r and
>> switching.
>>
>> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> ---
>>   arch/arm/mach-exynos/Kconfig       |    8 +
>>   arch/arm/mach-exynos/Makefile      |    2 +
>>   arch/arm/mach-exynos/mcpm-exynos.c |  345
>> ++++++++++++++++++++++++++++++++++++
>>   arch/arm/mach-exynos/regs-pmu.h    |    2 +
>>   4 files changed, 357 insertions(+)
>>   create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c
>>
>> diff --git a/arch/arm/mach-exynos/Kconfig b/arch/arm/mach-exynos/Kconfig
>> index fc8bf18..1602abc 100644
>> --- a/arch/arm/mach-exynos/Kconfig
>> +++ b/arch/arm/mach-exynos/Kconfig
>> @@ -110,4 +110,12 @@ config SOC_EXYNOS5440
>>
>>   endmenu
>>
>> +config EXYNOS5420_MCPM
>> +       bool "Exynos5420 Multi-Cluster PM support"
>> +       depends on MCPM && SOC_EXYNOS5420
>> +       select ARM_CCI
>> +       help
>> +         This is needed to provide CPU and cluster power management
>> +         on Exynos5420 implementing big.LITTLE.
>> +
>>   endif
>> diff --git a/arch/arm/mach-exynos/Makefile b/arch/arm/mach-exynos/Makefile
>> index a656dbe..01bc9b9 100644
>> --- a/arch/arm/mach-exynos/Makefile
>> +++ b/arch/arm/mach-exynos/Makefile
>> @@ -29,3 +29,5 @@ obj-$(CONFIG_ARCH_EXYNOS)     += firmware.o
>>
>>   plus_sec := $(call as-instr,.arch_extension sec,+sec)
>>   AFLAGS_exynos-smc.o           :=-Wa,-march=armv7-a$(plus_sec)
>> +
>> +obj-$(CONFIG_EXYNOS5420_MCPM)  += mcpm-exynos.o
>> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c
>> b/arch/arm/mach-exynos/mcpm-exynos.c
>> new file mode 100644
>> index 0000000..49b9031
>> --- /dev/null
>> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> @@ -0,0 +1,345 @@
>> +/*
>> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>> + *             http://www.samsung.com
>> + *
>> + * arch/arm/mach-exynos/mcpm-exynos.c
>> + *
>> + * Based on arch/arm/mach-vexpress/dcscb.c
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/io.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/miscdevice.h>
>> +#include <linux/fs.h>
>> +#include <linux/delay.h>
>> +#include <linux/arm-cci.h>
>> +
>> +#include <asm/mcpm.h>
>> +#include <asm/cputype.h>
>> +#include <asm/cp15.h>
>> +
>> +#include <plat/cpu.h>
>> +#include "regs-pmu.h"
>> +
>> +#define EXYNOS5420_CPUS_PER_CLUSTER    4
>> +#define EXYNOS5420_NR_CLUSTERS         2
>> +
>> +/* Secondary CPU entry point */
>> +#define REG_ENTRY_ADDR         (S5P_VA_SYSRAM_NS + 0x1C)
>> +
>> +#define EXYNOS_CORE_LOCAL_PWR_EN               0x3
>> +#define EXYNOS_CORE_LOCAL_PWR_DIS              0x0
>>
>> +#define EXYNOS_ARM_COMMON_CONFIGURATION                S5P_PMUREG(0x2500)
>> +#define EXYNOS_ARM_L2_CONFIGURATION            S5P_PMUREG(0x2600)
>> +
>> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)     \
>> +                       (S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_ARM_CORE_STATUS(_nr)            \
>> +                       (S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
>> +
>> +#define EXYNOS_COMMON_CONFIGURATION(_nr)       \
>> +                       (EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_COMMON_STATUS(_nr)              \
>> +                       (EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
>> +
>> +#define EXYNOS_L2_CONFIGURATION(_nr)           \
>> +                       (EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_L2_STATUS(_nr)                  \
>> +                       (EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
>> +
>
>
> Is it possible to share the definition of those macros with the rest of the
> code via functions, so they can be re-used for the other sub-systems ? eg:
> https://patches.linaro.org/27798/
OK..I will work on making wrapper functions for these. Would these new
functions be better placed in the mcpm code or the pm code ?
>
>
>> +/*
>> + * The common v7_exit_coherency_flush API could not be used because of
>> the
>> + * Erratum 799270 workaround. This macro is the same as the common one
>> (in
>> + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
>> + */
>> +#define exynos_v7_exit_coherency_flush(level) \
>> +       asm volatile( \
>> +       "stmfd  sp!, {fp, ip}\n\t"\
>> +       "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t" \
>> +       "bic    r0, r0, #"__stringify(CR_C)"\n\t" \
>> +       "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t" \
>> +       "isb\n\t"\
>> +       "bl     v7_flush_dcache_"__stringify(level)"\n\t" \
>> +       "clrex\n\t"\
>> +       "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t" \
>> +       "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t" \
>> +       /* Dummy Load of a device register to avoid Erratum 799270 */ \
>
>
> Wouldn't make sense to add the erratum in the Kconfig and re-use it in the
> generic v7_exit_coherency_flush macro instead of redefining it ?
I am re-looking at the necessity of this after Nicolas' comments.
>
>
>> +       "ldr    r4, [%0]\n\t" \
>> +       "and    r4, r4, #0\n\t" \
>> +       "orr    r0, r0, r4\n\t" \
>> +       "mcr    p15, 0, r0, c1, c0, 1   @ set ACTLR\n\t" \
>> +       "isb\n\t" \
>> +       "dsb\n\t" \
>> +       "ldmfd  sp!, {fp, ip}" \
>> +       : \
>> +       : "Ir" (S5P_INFORM0) \
>> +       : "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
>> +         "r9", "r10", "lr", "memory")
>
>
>
>
>> +/*
>> + * We can't use regular spinlocks. In the switcher case, it is possible
>> + * for an outbound CPU to call power_down() after its inbound counterpart
>> + * is already live using the same logical CPU number which trips lockdep
>> + * debugging.
>> + */
>> +static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
>> +static int
>> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
>> +
>> +static bool exynos_core_power_state(unsigned int cpu, unsigned int
>> cluster)
>> +{
>> +       unsigned int val;
>> +       unsigned int cpunr = cpu + (cluster *
>> EXYNOS5420_CPUS_PER_CLUSTER);
>> +
>> +       val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
>> +                               EXYNOS_CORE_LOCAL_PWR_EN;
>> +       return !!val;
>> +}
>> +
>> +static void exynos_core_power_control(unsigned int cpu, unsigned int
>> cluster,
>> +                                               int enable)
>> +{
>> +       unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> +       unsigned int cpunr = cpu + (cluster *
>> EXYNOS5420_CPUS_PER_CLUSTER);
>> +
>> +       if (exynos_core_power_state(cpu, cluster) == enable)
>> +               return;
>> +
>> +       if (enable)
>> +               val = EXYNOS_CORE_LOCAL_PWR_EN;
>> +       __raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
>> +}
>
>
> Same comment as above about sharing these functions.
>
> Shouldn't these functions to be reused by hotplug ?
>
>
>> +static void exynos_cluster_power_control(unsigned int cluster, int
>> enable)
>> +{
>> +       unsigned long timeout = jiffies + msecs_to_jiffies(20);
>> +       unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> +
>> +       if (enable)
>> +               val = EXYNOS_CORE_LOCAL_PWR_EN;
>> +
>> +       status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> +       if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> +               return;
>> +
>> +       __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
>> +       /* Wait until cluster power control is applied */
>> +       while (time_before(jiffies, timeout)) {
>> +               status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> +
>> +               if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> +                       return;
>> +
>> +               cpu_relax();
>> +       }
>> +       pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
>> +               enable ? "on" : "off");
>> +}
>> +
>> +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>> +{
>> +       pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +       if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +               cluster >= EXYNOS5420_NR_CLUSTERS)
>> +               return -EINVAL;
>> +
>> +       /*
>> +        * Since this is called with IRQs enabled, and no
>> arch_spin_lock_irq
>> +        * variant exists, we need to disable IRQs manually here.
>> +        */
>> +       local_irq_disable();
>> +       arch_spin_lock(&exynos_mcpm_lock);
>> +
>> +       cpu_use_count[cpu][cluster]++;
>> +       if (cpu_use_count[cpu][cluster] == 1) {
>> +               bool was_cluster_down =
>> +                       __mcpm_cluster_state(cluster) == CLUSTER_DOWN;
>> +
>> +               /*
>> +                * Turn on the cluster (L2/COMMON) and then power on the
>> cores.
>> +                * TODO: Turn off the clusters when all cores in the
>> cluster
>> +                * are down to achieve significant power savings.
>> +                */
>> +               if (was_cluster_down)
>> +                       exynos_cluster_power_control(cluster, 1);
>> +               exynos_core_power_control(cpu, cluster, 1);
>> +
>> +               /* CPU should be powered up there */
>> +               /* Also setup Cluster Power Sequence here */
>> +       } else if (cpu_use_count[cpu][cluster] != 2) {
>> +               /*
>> +                * The only possible values are:
>> +                * 0 = CPU down
>> +                * 1 = CPU (still) up
>> +                * 2 = CPU requested to be up before it had a chance
>> +                *     to actually make itself down.
>> +                * Any other value is a bug.
>> +                */
>> +               BUG();
>> +       }
>> +
>> +       arch_spin_unlock(&exynos_mcpm_lock);
>> +       local_irq_enable();
>> +
>> +       return 0;
>> +}
>> +
>> +static void exynos_power_down(void)
>> +{
>> +       unsigned int mpidr, cpu, cluster, cpumask;
>> +       bool last_man = false, skip_wfi = false;
>> +
>> +       mpidr = read_cpuid_mpidr();
>> +       cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +       cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +       cpumask = (1 << cpu);
>
>
> This variable is not used.
Will remove.
>
>
>> +
>> +       pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +       BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +                       cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +       __mcpm_cpu_going_down(cpu, cluster);
>> +
>> +       arch_spin_lock(&exynos_mcpm_lock);
>> +       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> +       cpu_use_count[cpu][cluster]--;
>> +       if (cpu_use_count[cpu][cluster] == 0) {
>> +               int cnt = 0, i;
>> +
>> +               exynos_core_power_control(cpu, cluster, 0);
>> +               for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
>> +                       cnt += cpu_use_count[i][cluster];
>> +               if (cnt == 0)
>> +                       last_man = true;
>> +       } else if (cpu_use_count[cpu][cluster] == 1) {
>> +               /*
>> +                * A power_up request went ahead of us.
>> +                * Even if we do not want to shut this CPU down,
>> +                * the caller expects a certain state as if the WFI
>> +                * was aborted.  So let's continue with cache cleaning.
>> +                */
>> +               skip_wfi = true;
>> +       } else {
>> +               BUG();
>> +       }
>> +
>> +       if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
>> +               arch_spin_unlock(&exynos_mcpm_lock);
>> +
>> +               /* Flush all cache levels for this cluster. */
>> +               exynos_v7_exit_coherency_flush(all);
>> +
>> +               /*
>> +                * Disable cluster-level coherency by masking
>> +                * incoming snoops and DVM messages:
>> +                */
>> +               cci_disable_port_by_cpu(mpidr);
>> +
>> +               __mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
>> +       } else {
>> +               arch_spin_unlock(&exynos_mcpm_lock);
>> +
>> +               /* Disable and flush the local CPU cache. */
>> +               exynos_v7_exit_coherency_flush(louis);
>> +       }
>> +
>> +       __mcpm_cpu_down(cpu, cluster);
>> +
>> +       /* Now we are prepared for power-down, do it: */
>> +       if (!skip_wfi)
>> +               wfi();
>> +
>> +       /* Not dead at this point?  Let our caller cope. */
>> +}
>> +
>> +static int exynos_power_down_finish(unsigned int cpu, unsigned int
>> cluster)
>> +{
>> +       pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +       BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +                       cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +       /* Wait for the core state to be OFF */
>> +       while (exynos_core_power_state(cpu, cluster) != 0x0)
>> +               ;
>
>
> As this is a potential infinite loop, a timeout may be desirable, no ?
Will be re-working this function.
>
>
>> +
>> +       return 0; /* success: the CPU is halted */
>> +}
>> +
>> +static const struct mcpm_platform_ops exynos_power_ops = {
>> +       .power_up               = exynos_power_up,
>> +       .power_down             = exynos_power_down,
>> +       .power_down_finish      = exynos_power_down_finish,
>> +};
>> +
>> +static void __init exynos_mcpm_usage_count_init(void)
>> +{
>> +       unsigned int mpidr, cpu, cluster;
>> +
>> +       mpidr = read_cpuid_mpidr();
>> +       cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +       cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +
>> +       pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +       BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
>> +                       cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +       cpu_use_count[cpu][cluster] = 1;
>> +}
>> +
>> +/*
>> + * Enable cluster-level coherency, in preparation for turning on the MMU.
>> + */
>> +static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
>> +{
>> +       asm volatile ("\n"
>> +       "cmp    r0, #1\n"
>> +       "bxne   lr\n"
>> +       "b      cci_enable_port_for_self");
>> +}
>> +
>> +static int __init exynos_mcpm_init(void)
>> +{
>> +       int ret = 0;
>> +
>> +       if (!soc_is_exynos5420())
>> +               return -ENODEV;
>> +
>> +       if (!cci_probed())
>> +               return -ENODEV;
>> +
>> +       /*
>> +        * To increase the stability of KFC reset we need to program
>> +        * the PMU SPARE3 register
>> +        */
>> +       __raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
>> +
>> +       exynos_mcpm_usage_count_init();
>> +
>> +       ret = mcpm_platform_register(&exynos_power_ops);
>> +       if (!ret)
>> +               ret = mcpm_sync_init(exynos_pm_power_up_setup);
>> +       if (ret)
>> +               return ret;
>> +
>> +       mcpm_smp_set_ops();
>> +
>> +       pr_info("Exynos MCPM support installed\n");
>> +
>> +       /*
>> +        * Future entries into the kernel can now go
>> +        * through the cluster entry vectors.
>> +        */
>> +       __raw_writel(virt_to_phys(mcpm_entry_point),
>> +               REG_ENTRY_ADDR);
>> +
>> +       return ret;
>> +}
>> +
>> +early_initcall(exynos_mcpm_init);
>> diff --git a/arch/arm/mach-exynos/regs-pmu.h
>> b/arch/arm/mach-exynos/regs-pmu.h
>> index cfbfc575..43fe7a0 100644
>> --- a/arch/arm/mach-exynos/regs-pmu.h
>> +++ b/arch/arm/mach-exynos/regs-pmu.h
>> @@ -38,6 +38,7 @@
>>   #define S5P_INFORM5                           S5P_PMUREG(0x0814)
>>   #define S5P_INFORM6                           S5P_PMUREG(0x0818)
>>   #define S5P_INFORM7                           S5P_PMUREG(0x081C)
>> +#define S5P_PMU_SPARE3                         S5P_PMUREG(0x090C)
>>
>>   #define EXYNOS_IROM_DATA2                     S5P_PMUREG(0x0988)
>>
>> @@ -540,5 +541,6 @@
>>   #define EXYNOS5420_KFC_USE_STANDBY_WFE3                               (1
>> << 23)
>>
>>   #define DUR_WAIT_RESET                                0xF
>> +#define EXYNOS5420_SWRESET_KFC_SEL             0x3
>>
>>   #endif /* __ASM_ARCH_REGS_PMU_H */
>>
>
>
> --
>  <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs
>
> Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
> <http://twitter.com/#!/linaroorg> Twitter |
> <http://www.linaro.org/linaro-blog/> Blog
>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Daniel Lezcano]

On 04/23/2014 05:31 PM, Abhilash Kesavan wrote:
> Hi Daniel,
> On Tue, Apr 22, 2014 at 4:51 PM, Daniel Lezcano
> <daniel.lezcano@linaro.org> wrote:
>> On 04/22/2014 08:17 AM, Abhilash Kesavan wrote:
>>>
>>> Add machine-dependent MCPM call-backs for Exynos5420. These are used
>>> to power up/down the secondary CPUs during boot, shutdown, s2r and
>>> switching.
>>>
>>> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>>> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>>> ---

[ ... ]

>>> +#include <plat/cpu.h>
>>> +#include "regs-pmu.h"
>>> +
>>> +#define EXYNOS5420_CPUS_PER_CLUSTER    4
>>> +#define EXYNOS5420_NR_CLUSTERS         2
>>> +
>>> +/* Secondary CPU entry point */
>>> +#define REG_ENTRY_ADDR         (S5P_VA_SYSRAM_NS + 0x1C)
>>> +
>>> +#define EXYNOS_CORE_LOCAL_PWR_EN               0x3
>>> +#define EXYNOS_CORE_LOCAL_PWR_DIS              0x0
>>>
>>> +#define EXYNOS_ARM_COMMON_CONFIGURATION                S5P_PMUREG(0x2500)
>>> +#define EXYNOS_ARM_L2_CONFIGURATION            S5P_PMUREG(0x2600)
>>> +
>>> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)     \
>>> +                       (S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
>>> +#define EXYNOS_ARM_CORE_STATUS(_nr)            \
>>> +                       (S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
>>> +
>>> +#define EXYNOS_COMMON_CONFIGURATION(_nr)       \
>>> +                       (EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
>>> +#define EXYNOS_COMMON_STATUS(_nr)              \
>>> +                       (EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
>>> +
>>> +#define EXYNOS_L2_CONFIGURATION(_nr)           \
>>> +                       (EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
>>> +#define EXYNOS_L2_STATUS(_nr)                  \
>>> +                       (EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
>>> +
>>
>>
>> Is it possible to share the definition of those macros with the rest of the
>> code via functions, so they can be re-used for the other sub-systems ? eg:
>> https://patches.linaro.org/27798/
> OK..I will work on making wrapper functions for these. Would these new
> functions be better placed in the mcpm code or the pm code ?

Hi Abhilash,

yes, in the pm code. That would make more sense.

Thanks
   -- Daniel


-- 
  <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Nicolas Pitre]

On Tue, 22 Apr 2014, Abhilash Kesavan wrote:

> Add machine-dependent MCPM call-backs for Exynos5420. These are used
> to power up/down the secondary CPUs during boot, shutdown, s2r and
> switching.
> 
> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>

Getting there!  See comments below.

[...]
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> new file mode 100644
> index 0000000..49b9031
> --- /dev/null
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -0,0 +1,345 @@
> +/*
> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
> + *		http://www.samsung.com
> + *
> + * arch/arm/mach-exynos/mcpm-exynos.c
> + *
> + * Based on arch/arm/mach-vexpress/dcscb.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +#include <linux/uaccess.h>
> +#include <linux/miscdevice.h>
> +#include <linux/fs.h>
> +#include <linux/delay.h>
> +#include <linux/arm-cci.h>
> +
> +#include <asm/mcpm.h>
> +#include <asm/cputype.h>
> +#include <asm/cp15.h>
> +
> +#include <plat/cpu.h>
> +#include "regs-pmu.h"
> +
> +#define EXYNOS5420_CPUS_PER_CLUSTER	4
> +#define EXYNOS5420_NR_CLUSTERS		2
> +
> +/* Secondary CPU entry point */
> +#define REG_ENTRY_ADDR		(S5P_VA_SYSRAM_NS + 0x1C)
> +
> +#define EXYNOS_CORE_LOCAL_PWR_EN		0x3
> +#define EXYNOS_CORE_LOCAL_PWR_DIS		0x0
> +
> +#define EXYNOS_ARM_COMMON_CONFIGURATION		S5P_PMUREG(0x2500)
> +#define EXYNOS_ARM_L2_CONFIGURATION		S5P_PMUREG(0x2600)
> +
> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)	\
> +			(S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_ARM_CORE_STATUS(_nr)		\
> +			(S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
> +
> +#define EXYNOS_COMMON_CONFIGURATION(_nr)	\
> +			(EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_COMMON_STATUS(_nr)		\
> +			(EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
> +
> +#define EXYNOS_L2_CONFIGURATION(_nr)		\
> +			(EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
> +#define EXYNOS_L2_STATUS(_nr)			\
> +			(EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
> +
> +/*
> + * The common v7_exit_coherency_flush API could not be used because of the
> + * Erratum 799270 workaround. This macro is the same as the common one (in
> + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
> + */
> +#define exynos_v7_exit_coherency_flush(level) \
> +	asm volatile( \
> +	"stmfd	sp!, {fp, ip}\n\t"\
> +	"mrc	p15, 0, r0, c1, c0, 0	@ get SCTLR\n\t" \
> +	"bic	r0, r0, #"__stringify(CR_C)"\n\t" \
> +	"mcr	p15, 0, r0, c1, c0, 0	@ set SCTLR\n\t" \
> +	"isb\n\t"\
> +	"bl	v7_flush_dcache_"__stringify(level)"\n\t" \
> +	"clrex\n\t"\
> +	"mrc	p15, 0, r0, c1, c0, 1	@ get ACTLR\n\t" \
> +	"bic	r0, r0, #(1 << 6)	@ disable local coherency\n\t" \
> +	/* Dummy Load of a device register to avoid Erratum 799270 */ \
> +	"ldr	r4, [%0]\n\t" \
> +	"and	r4, r4, #0\n\t" \
> +	"orr	r0, r0, r4\n\t" \
> +	"mcr	p15, 0, r0, c1, c0, 1	@ set ACTLR\n\t" \
> +	"isb\n\t" \
> +	"dsb\n\t" \
> +	"ldmfd	sp!, {fp, ip}" \
> +	: \
> +	: "Ir" (S5P_INFORM0) \
> +	: "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
> +	  "r9", "r10", "lr", "memory")
> +
> +/*
> + * We can't use regular spinlocks. In the switcher case, it is possible
> + * for an outbound CPU to call power_down() after its inbound counterpart
> + * is already live using the same logical CPU number which trips lockdep
> + * debugging.
> + */
> +static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
> +static int
> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
> +
> +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
> +{
> +	unsigned int val;
> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> +
> +	val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
> +				EXYNOS_CORE_LOCAL_PWR_EN;
> +	return !!val;
> +}
> +
> +static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
> +						int enable)
> +{
> +	unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
> +	unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
> +
> +	if (exynos_core_power_state(cpu, cluster) == enable)
> +		return;

I think this isn't right.

Normally, the power _status_ and the power _control_ are not always in 
sync.  It is possible for the power control to be set to OFF but the
status won't reflect that until the CPU has executed WFI for example.

Here we don't care about the actual status.  What matters is the 
control.  In the exynos_power_up case, if the control bit says "power 
off" but the status bit says "it isn't off yet" then the code won't turn 
the control bit back on.

For example, it should be possible for CPU1 to call exynos_power_up() 
while CPU0 is still executing code in exynos_power_down() right after 
releasing exynos_mcpm_lock but before WFI is executed.  There is a cache 
flush during that window which might take some time to execute, making 
this scenario more likely that you might think.

What we want here is simply to set the power control bit back on.  No 
need to test its previous status as the value of cpu_use_count already 
reflects it.  And if the affected CPU didn't reach WFI yet, then the 
execution of WFI will simply return and the CPU will be soft rebooted.

And in the exynos_power_down() case the CPU turning the control bit off 
is always doing so for itself, therefore its status bit must obviously 
show that it is running.  Testing it is therefore redundant.

Therefore you should be able to simplify this greatly.

> +	if (enable)
> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> +	__raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
> +}
> +
> +static void exynos_cluster_power_control(unsigned int cluster, int enable)
> +{
> +	unsigned long timeout = jiffies + msecs_to_jiffies(20);
> +	unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
> +
> +	if (enable)
> +		val = EXYNOS_CORE_LOCAL_PWR_EN;
> +
> +	status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> +	if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> +		return;

Same comment as above.  You shouldn't need to check current status 
before adjusting the wanted control.  If you get here that's because the 
control is never what you want it to be.

> +	__raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
> +	/* Wait until cluster power control is applied */
> +	while (time_before(jiffies, timeout)) {
> +		status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
> +
> +		if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
> +			return;
> +
> +		cpu_relax();
> +	}

As mentioned in a previous email, it is possible for the CPU executing 
this code to be the only CPU currently alive.  And in this code path 
IRQs are disabled.  That means nothing will ever increase jiffies in 
that case and the timeout will never expire.

If in practice there is only a few loops to perform here, I'd suggest 
simply looping, say, 100 times and bail out if that fails.

Oh and if the poll loop fails then you must turn the power control back 
off and return an error via exynos_power_up().

> +	pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
> +		enable ? "on" : "off");
> +}
> +
> +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
> +{
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +		cluster >= EXYNOS5420_NR_CLUSTERS)
> +		return -EINVAL;
> +
> +	/*
> +	 * Since this is called with IRQs enabled, and no arch_spin_lock_irq
> +	 * variant exists, we need to disable IRQs manually here.
> +	 */
> +	local_irq_disable();
> +	arch_spin_lock(&exynos_mcpm_lock);
> +
> +	cpu_use_count[cpu][cluster]++;
> +	if (cpu_use_count[cpu][cluster] == 1) {
> +		bool was_cluster_down =
> +			__mcpm_cluster_state(cluster) == CLUSTER_DOWN;
> +
> +		/*
> +		 * Turn on the cluster (L2/COMMON) and then power on the cores.
> +		 * TODO: Turn off the clusters when all cores in the cluster
> +		 * are down to achieve significant power savings.
> +		 */
> +		if (was_cluster_down)
> +			exynos_cluster_power_control(cluster, 1);
> +		exynos_core_power_control(cpu, cluster, 1);
> +
> +		/* CPU should be powered up there */
> +		/* Also setup Cluster Power Sequence here */
> +	} else if (cpu_use_count[cpu][cluster] != 2) {
> +		/*
> +		 * The only possible values are:
> +		 * 0 = CPU down
> +		 * 1 = CPU (still) up
> +		 * 2 = CPU requested to be up before it had a chance
> +		 *     to actually make itself down.
> +		 * Any other value is a bug.
> +		 */
> +		BUG();
> +	}
> +
> +	arch_spin_unlock(&exynos_mcpm_lock);
> +	local_irq_enable();
> +
> +	return 0;
> +}
> +
> +static void exynos_power_down(void)
> +{
> +	unsigned int mpidr, cpu, cluster, cpumask;
> +	bool last_man = false, skip_wfi = false;
> +
> +	mpidr = read_cpuid_mpidr();
> +	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	cpumask = (1 << cpu);
> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	__mcpm_cpu_going_down(cpu, cluster);
> +
> +	arch_spin_lock(&exynos_mcpm_lock);
> +	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> +	cpu_use_count[cpu][cluster]--;
> +	if (cpu_use_count[cpu][cluster] == 0) {
> +		int cnt = 0, i;
> +
> +		exynos_core_power_control(cpu, cluster, 0);
> +		for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
> +			cnt += cpu_use_count[i][cluster];
> +		if (cnt == 0)
> +			last_man = true;
> +	} else if (cpu_use_count[cpu][cluster] == 1) {
> +		/*
> +		 * A power_up request went ahead of us.
> +		 * Even if we do not want to shut this CPU down,
> +		 * the caller expects a certain state as if the WFI
> +		 * was aborted.  So let's continue with cache cleaning.
> +		 */
> +		skip_wfi = true;
> +	} else {
> +		BUG();
> +	}
> +
> +	if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
> +		arch_spin_unlock(&exynos_mcpm_lock);
> +
> +		/* Flush all cache levels for this cluster. */
> +		exynos_v7_exit_coherency_flush(all);
> +
> +		/*
> +		 * Disable cluster-level coherency by masking
> +		 * incoming snoops and DVM messages:
> +		 */
> +		cci_disable_port_by_cpu(mpidr);
> +
> +		__mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
> +	} else {
> +		arch_spin_unlock(&exynos_mcpm_lock);
> +
> +		/* Disable and flush the local CPU cache. */
> +		exynos_v7_exit_coherency_flush(louis);
> +	}
> +
> +	__mcpm_cpu_down(cpu, cluster);
> +
> +	/* Now we are prepared for power-down, do it: */
> +	if (!skip_wfi)
> +		wfi();
> +
> +	/* Not dead at this point?  Let our caller cope. */
> +}
> +
> +static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
> +{
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	/* Wait for the core state to be OFF */
> +	while (exynos_core_power_state(cpu, cluster) != 0x0)
> +		;
> +

*Here* is the place to actually establish a timeout.  Please see 
tc2_pm_power_down_finish() for example.

Also beware that this method is going to be renamed once RMK applies the 
following patch:

http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=8029/1





> +	return 0; /* success: the CPU is halted */
> +}
> +
> +static const struct mcpm_platform_ops exynos_power_ops = {
> +	.power_up		= exynos_power_up,
> +	.power_down		= exynos_power_down,
> +	.power_down_finish	= exynos_power_down_finish,
> +};
> +
> +static void __init exynos_mcpm_usage_count_init(void)
> +{
> +	unsigned int mpidr, cpu, cluster;
> +
> +	mpidr = read_cpuid_mpidr();
> +	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +	cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +
> +	pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
> +	BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
> +			cluster >= EXYNOS5420_NR_CLUSTERS);
> +
> +	cpu_use_count[cpu][cluster] = 1;
> +}
> +
> +/*
> + * Enable cluster-level coherency, in preparation for turning on the MMU.
> + */
> +static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
> +{
> +	asm volatile ("\n"
> +	"cmp	r0, #1\n"
> +	"bxne	lr\n"
> +	"b	cci_enable_port_for_self");
> +}
> +
> +static int __init exynos_mcpm_init(void)
> +{
> +	int ret = 0;
> +
> +	if (!soc_is_exynos5420())
> +		return -ENODEV;
> +
> +	if (!cci_probed())
> +		return -ENODEV;
> +
> +	/*
> +	 * To increase the stability of KFC reset we need to program
> +	 * the PMU SPARE3 register
> +	 */
> +	__raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
> +
> +	exynos_mcpm_usage_count_init();
> +
> +	ret = mcpm_platform_register(&exynos_power_ops);
> +	if (!ret)
> +		ret = mcpm_sync_init(exynos_pm_power_up_setup);
> +	if (ret)
> +		return ret;
> +
> +	mcpm_smp_set_ops();
> +
> +	pr_info("Exynos MCPM support installed\n");
> +
> +	/*
> +	 * Future entries into the kernel can now go
> +	 * through the cluster entry vectors.
> +	 */
> +	__raw_writel(virt_to_phys(mcpm_entry_point),
> +		REG_ENTRY_ADDR);
> +
> +	return ret;
> +}
> +
> +early_initcall(exynos_mcpm_init);
> diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
> index cfbfc575..43fe7a0 100644
> --- a/arch/arm/mach-exynos/regs-pmu.h
> +++ b/arch/arm/mach-exynos/regs-pmu.h
> @@ -38,6 +38,7 @@
>  #define S5P_INFORM5				S5P_PMUREG(0x0814)
>  #define S5P_INFORM6				S5P_PMUREG(0x0818)
>  #define S5P_INFORM7				S5P_PMUREG(0x081C)
> +#define S5P_PMU_SPARE3				S5P_PMUREG(0x090C)
>  
>  #define EXYNOS_IROM_DATA2			S5P_PMUREG(0x0988)
>  
> @@ -540,5 +541,6 @@
>  #define EXYNOS5420_KFC_USE_STANDBY_WFE3				(1 << 23)
>  
>  #define DUR_WAIT_RESET				0xF
> +#define EXYNOS5420_SWRESET_KFC_SEL		0x3
>  
>  #endif /* __ASM_ARCH_REGS_PMU_H */
> -- 
> 1.7.9.5
> 

[PATCH v2 2/3] arm: exynos: Add MCPM call-back functions

[Abhilash Kesavan]

Hi Nicolas,

On Wed, Apr 23, 2014 at 12:48 AM, Nicolas Pitre
<nicolas.pitre@linaro.org> wrote:
> On Tue, 22 Apr 2014, Abhilash Kesavan wrote:
>
>> Add machine-dependent MCPM call-backs for Exynos5420. These are used
>> to power up/down the secondary CPUs during boot, shutdown, s2r and
>> switching.
>>
>> Signed-off-by: Thomas Abraham <thomas.ab@samsung.com>
>> Signed-off-by: Inderpal Singh <inderpal.s@samsung.com>
>> Signed-off-by: Andrew Bresticker <abrestic@chromium.org>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>
> Getting there!  See comments below.
>
> [...]
>> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> new file mode 100644
>> index 0000000..49b9031
>> --- /dev/null
>> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> @@ -0,0 +1,345 @@
>> +/*
>> + * Copyright (c) 2014 Samsung Electronics Co., Ltd.
>> + *           http://www.samsung.com
>> + *
>> + * arch/arm/mach-exynos/mcpm-exynos.c
>> + *
>> + * Based on arch/arm/mach-vexpress/dcscb.c
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + */
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/io.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/miscdevice.h>
>> +#include <linux/fs.h>
>> +#include <linux/delay.h>
>> +#include <linux/arm-cci.h>
>> +
>> +#include <asm/mcpm.h>
>> +#include <asm/cputype.h>
>> +#include <asm/cp15.h>
>> +
>> +#include <plat/cpu.h>
>> +#include "regs-pmu.h"
>> +
>> +#define EXYNOS5420_CPUS_PER_CLUSTER  4
>> +#define EXYNOS5420_NR_CLUSTERS               2
>> +
>> +/* Secondary CPU entry point */
>> +#define REG_ENTRY_ADDR               (S5P_VA_SYSRAM_NS + 0x1C)
>> +
>> +#define EXYNOS_CORE_LOCAL_PWR_EN             0x3
>> +#define EXYNOS_CORE_LOCAL_PWR_DIS            0x0
>> +
>> +#define EXYNOS_ARM_COMMON_CONFIGURATION              S5P_PMUREG(0x2500)
>> +#define EXYNOS_ARM_L2_CONFIGURATION          S5P_PMUREG(0x2600)
>> +
>> +#define EXYNOS_ARM_CORE_CONFIGURATION(_nr)   \
>> +                     (S5P_ARM_CORE0_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_ARM_CORE_STATUS(_nr)          \
>> +                     (S5P_ARM_CORE0_STATUS + ((_nr) * 0x80))
>> +
>> +#define EXYNOS_COMMON_CONFIGURATION(_nr)     \
>> +                     (EXYNOS_ARM_COMMON_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_COMMON_STATUS(_nr)            \
>> +                     (EXYNOS_COMMON_CONFIGURATION(_nr) + 0x4)
>> +
>> +#define EXYNOS_L2_CONFIGURATION(_nr)         \
>> +                     (EXYNOS_ARM_L2_CONFIGURATION + ((_nr) * 0x80))
>> +#define EXYNOS_L2_STATUS(_nr)                        \
>> +                     (EXYNOS_L2_CONFIGURATION(_nr) + 0x4)
>> +
>> +/*
>> + * The common v7_exit_coherency_flush API could not be used because of the
>> + * Erratum 799270 workaround. This macro is the same as the common one (in
>> + * arch/arm/include/asm/cacheflush.h) except for the erratum handling.
>> + */
>> +#define exynos_v7_exit_coherency_flush(level) \
>> +     asm volatile( \
>> +     "stmfd  sp!, {fp, ip}\n\t"\
>> +     "mrc    p15, 0, r0, c1, c0, 0   @ get SCTLR\n\t" \
>> +     "bic    r0, r0, #"__stringify(CR_C)"\n\t" \
>> +     "mcr    p15, 0, r0, c1, c0, 0   @ set SCTLR\n\t" \
>> +     "isb\n\t"\
>> +     "bl     v7_flush_dcache_"__stringify(level)"\n\t" \
>> +     "clrex\n\t"\
>> +     "mrc    p15, 0, r0, c1, c0, 1   @ get ACTLR\n\t" \
>> +     "bic    r0, r0, #(1 << 6)       @ disable local coherency\n\t" \
>> +     /* Dummy Load of a device register to avoid Erratum 799270 */ \
>> +     "ldr    r4, [%0]\n\t" \
>> +     "and    r4, r4, #0\n\t" \
>> +     "orr    r0, r0, r4\n\t" \
>> +     "mcr    p15, 0, r0, c1, c0, 1   @ set ACTLR\n\t" \
>> +     "isb\n\t" \
>> +     "dsb\n\t" \
>> +     "ldmfd  sp!, {fp, ip}" \
>> +     : \
>> +     : "Ir" (S5P_INFORM0) \
>> +     : "r0", "r1", "r2", "r3", "r4", "r5", "r6", "r7", \
>> +       "r9", "r10", "lr", "memory")
>> +
>> +/*
>> + * We can't use regular spinlocks. In the switcher case, it is possible
>> + * for an outbound CPU to call power_down() after its inbound counterpart
>> + * is already live using the same logical CPU number which trips lockdep
>> + * debugging.
>> + */
>> +static arch_spinlock_t exynos_mcpm_lock = __ARCH_SPIN_LOCK_UNLOCKED;
>> +static int
>> +cpu_use_count[EXYNOS5420_CPUS_PER_CLUSTER][EXYNOS5420_NR_CLUSTERS];
>> +
>> +static bool exynos_core_power_state(unsigned int cpu, unsigned int cluster)
>> +{
>> +     unsigned int val;
>> +     unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
>> +
>> +     val = __raw_readl(EXYNOS_ARM_CORE_STATUS(cpunr)) &
>> +                             EXYNOS_CORE_LOCAL_PWR_EN;
>> +     return !!val;
>> +}
>> +
>> +static void exynos_core_power_control(unsigned int cpu, unsigned int cluster,
>> +                                             int enable)
>> +{
>> +     unsigned int val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> +     unsigned int cpunr = cpu + (cluster * EXYNOS5420_CPUS_PER_CLUSTER);
>> +
>> +     if (exynos_core_power_state(cpu, cluster) == enable)
>> +             return;
>
> I think this isn't right.
>
> Normally, the power _status_ and the power _control_ are not always in
> sync.  It is possible for the power control to be set to OFF but the
> status won't reflect that until the CPU has executed WFI for example.
>
> Here we don't care about the actual status.  What matters is the
> control.  In the exynos_power_up case, if the control bit says "power
> off" but the status bit says "it isn't off yet" then the code won't turn
> the control bit back on.
>
> For example, it should be possible for CPU1 to call exynos_power_up()
> while CPU0 is still executing code in exynos_power_down() right after
> releasing exynos_mcpm_lock but before WFI is executed.  There is a cache
> flush during that window which might take some time to execute, making
> this scenario more likely that you might think.
>
> What we want here is simply to set the power control bit back on.  No
> need to test its previous status as the value of cpu_use_count already
> reflects it.  And if the affected CPU didn't reach WFI yet, then the
> execution of WFI will simply return and the CPU will be soft rebooted.
>
> And in the exynos_power_down() case the CPU turning the control bit off
> is always doing so for itself, therefore its status bit must obviously
> show that it is running.  Testing it is therefore redundant.
>
> Therefore you should be able to simplify this greatly.
Thanks for the detailed explanation. I will remove the status checks
and only activate the control bits.
>
>> +     if (enable)
>> +             val = EXYNOS_CORE_LOCAL_PWR_EN;
>> +     __raw_writel(val, EXYNOS_ARM_CORE_CONFIGURATION(cpunr));
>> +}
>> +
>> +static void exynos_cluster_power_control(unsigned int cluster, int enable)
>> +{
>> +     unsigned long timeout = jiffies + msecs_to_jiffies(20);
>> +     unsigned int status, val = EXYNOS_CORE_LOCAL_PWR_DIS;
>> +
>> +     if (enable)
>> +             val = EXYNOS_CORE_LOCAL_PWR_EN;
>> +
>> +     status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> +     if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> +             return;
>
> Same comment as above.  You shouldn't need to check current status
> before adjusting the wanted control.  If you get here that's because the
> control is never what you want it to be.
OK.
>
>> +     __raw_writel(val, EXYNOS_COMMON_CONFIGURATION(cluster));
>> +     /* Wait until cluster power control is applied */
>> +     while (time_before(jiffies, timeout)) {
>> +             status = __raw_readl(EXYNOS_COMMON_STATUS(cluster));
>> +
>> +             if ((status & EXYNOS_CORE_LOCAL_PWR_EN) == val)
>> +                     return;
>> +
>> +             cpu_relax();
>> +     }
>
> As mentioned in a previous email, it is possible for the CPU executing
> this code to be the only CPU currently alive.  And in this code path
> IRQs are disabled.  That means nothing will ever increase jiffies in
> that case and the timeout will never expire.
>
> If in practice there is only a few loops to perform here, I'd suggest
> simply looping, say, 100 times and bail out if that fails.
>
> Oh and if the poll loop fails then you must turn the power control back
> off and return an error via exynos_power_up().
Will change the polling loop and add error handling in case of failure..
>
>> +     pr_warn("timed out waiting for cluster %u to power %s\n", cluster,
>> +             enable ? "on" : "off");
>> +}
>> +
>> +static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>> +{
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     if (cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +             cluster >= EXYNOS5420_NR_CLUSTERS)
>> +             return -EINVAL;
>> +
>> +     /*
>> +      * Since this is called with IRQs enabled, and no arch_spin_lock_irq
>> +      * variant exists, we need to disable IRQs manually here.
>> +      */
>> +     local_irq_disable();
>> +     arch_spin_lock(&exynos_mcpm_lock);
>> +
>> +     cpu_use_count[cpu][cluster]++;
>> +     if (cpu_use_count[cpu][cluster] == 1) {
>> +             bool was_cluster_down =
>> +                     __mcpm_cluster_state(cluster) == CLUSTER_DOWN;
>> +
>> +             /*
>> +              * Turn on the cluster (L2/COMMON) and then power on the cores.
>> +              * TODO: Turn off the clusters when all cores in the cluster
>> +              * are down to achieve significant power savings.
>> +              */
>> +             if (was_cluster_down)
>> +                     exynos_cluster_power_control(cluster, 1);
>> +             exynos_core_power_control(cpu, cluster, 1);
>> +
>> +             /* CPU should be powered up there */
>> +             /* Also setup Cluster Power Sequence here */
>> +     } else if (cpu_use_count[cpu][cluster] != 2) {
>> +             /*
>> +              * The only possible values are:
>> +              * 0 = CPU down
>> +              * 1 = CPU (still) up
>> +              * 2 = CPU requested to be up before it had a chance
>> +              *     to actually make itself down.
>> +              * Any other value is a bug.
>> +              */
>> +             BUG();
>> +     }
>> +
>> +     arch_spin_unlock(&exynos_mcpm_lock);
>> +     local_irq_enable();
>> +
>> +     return 0;
>> +}
>> +
>> +static void exynos_power_down(void)
>> +{
>> +     unsigned int mpidr, cpu, cluster, cpumask;
>> +     bool last_man = false, skip_wfi = false;
>> +
>> +     mpidr = read_cpuid_mpidr();
>> +     cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +     cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +     cpumask = (1 << cpu);
>> +
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +                     cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +     __mcpm_cpu_going_down(cpu, cluster);
>> +
>> +     arch_spin_lock(&exynos_mcpm_lock);
>> +     BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> +     cpu_use_count[cpu][cluster]--;
>> +     if (cpu_use_count[cpu][cluster] == 0) {
>> +             int cnt = 0, i;
>> +
>> +             exynos_core_power_control(cpu, cluster, 0);
>> +             for (i = 0; i < EXYNOS5420_CPUS_PER_CLUSTER; i++)
>> +                     cnt += cpu_use_count[i][cluster];
>> +             if (cnt == 0)
>> +                     last_man = true;
>> +     } else if (cpu_use_count[cpu][cluster] == 1) {
>> +             /*
>> +              * A power_up request went ahead of us.
>> +              * Even if we do not want to shut this CPU down,
>> +              * the caller expects a certain state as if the WFI
>> +              * was aborted.  So let's continue with cache cleaning.
>> +              */
>> +             skip_wfi = true;
>> +     } else {
>> +             BUG();
>> +     }
>> +
>> +     if (last_man && __mcpm_outbound_enter_critical(cpu, cluster)) {
>> +             arch_spin_unlock(&exynos_mcpm_lock);
>> +
>> +             /* Flush all cache levels for this cluster. */
>> +             exynos_v7_exit_coherency_flush(all);
>> +
>> +             /*
>> +              * Disable cluster-level coherency by masking
>> +              * incoming snoops and DVM messages:
>> +              */
>> +             cci_disable_port_by_cpu(mpidr);
>> +
>> +             __mcpm_outbound_leave_critical(cluster, CLUSTER_DOWN);
>> +     } else {
>> +             arch_spin_unlock(&exynos_mcpm_lock);
>> +
>> +             /* Disable and flush the local CPU cache. */
>> +             exynos_v7_exit_coherency_flush(louis);
>> +     }
>> +
>> +     __mcpm_cpu_down(cpu, cluster);
>> +
>> +     /* Now we are prepared for power-down, do it: */
>> +     if (!skip_wfi)
>> +             wfi();
>> +
>> +     /* Not dead at this point?  Let our caller cope. */
>> +}
>> +
>> +static int exynos_power_down_finish(unsigned int cpu, unsigned int cluster)
>> +{
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER ||
>> +                     cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +     /* Wait for the core state to be OFF */
>> +     while (exynos_core_power_state(cpu, cluster) != 0x0)
>> +             ;
>> +
>
> *Here* is the place to actually establish a timeout.  Please see
> tc2_pm_power_down_finish() for example.
>
> Also beware that this method is going to be renamed once RMK applies the
> following patch:
>
> http://www.arm.linux.org.uk/developer/patches/viewpatch.php?id=8029/1
>
I will re-work this function referencing the tc2 code. v3 will be posted soon.

Regards,
Abhilash
>
>
>
>
>> +     return 0; /* success: the CPU is halted */
>> +}
>> +
>> +static const struct mcpm_platform_ops exynos_power_ops = {
>> +     .power_up               = exynos_power_up,
>> +     .power_down             = exynos_power_down,
>> +     .power_down_finish      = exynos_power_down_finish,
>> +};
>> +
>> +static void __init exynos_mcpm_usage_count_init(void)
>> +{
>> +     unsigned int mpidr, cpu, cluster;
>> +
>> +     mpidr = read_cpuid_mpidr();
>> +     cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +     cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +
>> +     pr_debug("%s: cpu %u cluster %u\n", __func__, cpu, cluster);
>> +     BUG_ON(cpu >= EXYNOS5420_CPUS_PER_CLUSTER  ||
>> +                     cluster >= EXYNOS5420_NR_CLUSTERS);
>> +
>> +     cpu_use_count[cpu][cluster] = 1;
>> +}
>> +
>> +/*
>> + * Enable cluster-level coherency, in preparation for turning on the MMU.
>> + */
>> +static void __naked exynos_pm_power_up_setup(unsigned int affinity_level)
>> +{
>> +     asm volatile ("\n"
>> +     "cmp    r0, #1\n"
>> +     "bxne   lr\n"
>> +     "b      cci_enable_port_for_self");
>> +}
>> +
>> +static int __init exynos_mcpm_init(void)
>> +{
>> +     int ret = 0;
>> +
>> +     if (!soc_is_exynos5420())
>> +             return -ENODEV;
>> +
>> +     if (!cci_probed())
>> +             return -ENODEV;
>> +
>> +     /*
>> +      * To increase the stability of KFC reset we need to program
>> +      * the PMU SPARE3 register
>> +      */
>> +     __raw_writel(EXYNOS5420_SWRESET_KFC_SEL, S5P_PMU_SPARE3);
>> +
>> +     exynos_mcpm_usage_count_init();
>> +
>> +     ret = mcpm_platform_register(&exynos_power_ops);
>> +     if (!ret)
>> +             ret = mcpm_sync_init(exynos_pm_power_up_setup);
>> +     if (ret)
>> +             return ret;
>> +
>> +     mcpm_smp_set_ops();
>> +
>> +     pr_info("Exynos MCPM support installed\n");
>> +
>> +     /*
>> +      * Future entries into the kernel can now go
>> +      * through the cluster entry vectors.
>> +      */
>> +     __raw_writel(virt_to_phys(mcpm_entry_point),
>> +             REG_ENTRY_ADDR);
>> +
>> +     return ret;
>> +}
>> +
>> +early_initcall(exynos_mcpm_init);
>> diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
>> index cfbfc575..43fe7a0 100644
>> --- a/arch/arm/mach-exynos/regs-pmu.h
>> +++ b/arch/arm/mach-exynos/regs-pmu.h
>> @@ -38,6 +38,7 @@
>>  #define S5P_INFORM5                          S5P_PMUREG(0x0814)
>>  #define S5P_INFORM6                          S5P_PMUREG(0x0818)
>>  #define S5P_INFORM7                          S5P_PMUREG(0x081C)
>> +#define S5P_PMU_SPARE3                               S5P_PMUREG(0x090C)
>>
>>  #define EXYNOS_IROM_DATA2                    S5P_PMUREG(0x0988)
>>
>> @@ -540,5 +541,6 @@
>>  #define EXYNOS5420_KFC_USE_STANDBY_WFE3                              (1 << 23)
>>
>>  #define DUR_WAIT_RESET                               0xF
>> +#define EXYNOS5420_SWRESET_KFC_SEL           0x3
>>
>>  #endif /* __ASM_ARCH_REGS_PMU_H */
>> --
>> 1.7.9.5
>>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 0/3] MCPM backend for Exynos5420

[Abhilash Kesavan]

This is v2 of the series adding MCPM backend support for SMP secondary boot
and core switching on Samsung's Exynos5420. The patches are based on the mcpm
support added for Exynos5420 in the Chromium kernel repository here:
https://chromium.googlesource.com/chromiumos/third_party/kernel-next/+/chromeos-3.8

This patchset depends on:
- ARM: EXYNOS5: Add PMU settings for exynos5420
(https://patchwork.kernel.org/patch/3896471/)
- ARM: EXYNOS: Map SYSRAM address through DT
(http://www.spinics.net/lists/arm-kernel/msg323011.html)
- ARM: bL_switcher: fix validation check before its activation
(http://www.spinics.net/lists/arm-kernel/msg323274.html)

The patches have been prepared on Kgene's for-next branch and tested on SMDK5420
EVT1 using the "/dev/b.L_switcher" user interface. Patches have also been tested
on linux-next(20140417).

Changes since v1:
Addressed the following comments from Dave Martin and Nicolas Pitre:
	- Fixed help text for EXYNOS5420_MCPM symbol.
	- Removed mcpm-exynos-setup.S file and added similar code from tc2_pm.c.
	- Changed the spinlock name from "bl_lock" to "exynos_mcpm_lock".
	- Removed snoop enable/disable calls due to possible cpuidle issue and
	  not having numbers proving a significant power savings. Dropped the
	  "drivers/bus: arm-cci: Add common control interface for ACE ports"
	  patch from v1 as it was no longer needed.
	- Created a macro for exynos-specific v7_exit_coherency_flush which
	  handles an erratum. This was done to prevent duplication of code.
	- Removed "outer_flush_all" call.
	- Removed redundant dsb in power_down function.
	- Removed unnecessary initialization of global variables to zero.
	- Split the /dev/bL_status debug interface into another patch.
	- Fixed error handling in exynos_mcpm_init().
	- Called mcpm_smp_set_ops directly from exynos_mcpm_init().
	- Added a TODO for supporting cluster down on exynos5420.

There is no change in the patch " ARM: dts: exynos5420: add CCI node" from v1.

Abhilash Kesavan (2):
  arm: exynos: Add MCPM call-back functions
  arm: exynos: Add /dev/bL_status user interface on Exynos5420

Andrew Bresticker (1):
  ARM: dts: exynos5420: add CCI node

 arch/arm/boot/dts/exynos5420.dtsi  |   27 +++
 arch/arm/mach-exynos/Kconfig       |    8 +
 arch/arm/mach-exynos/Makefile      |    2 +
 arch/arm/mach-exynos/mcpm-exynos.c |  409 ++++++++++++++++++++++++++++++++++++
 arch/arm/mach-exynos/regs-pmu.h    |    2 +
 5 files changed, 448 insertions(+)
 create mode 100644 arch/arm/mach-exynos/mcpm-exynos.c

-- 
1.7.9.5