Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Daniel Drake]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

Hi Tomasz,

Thanks for working on this!

I have just tried this, against Linus master
64b2d1fbbfda07765dae3f601862796a61b2c451.
Added patch "ARM: dts: Initial ODROID U2 support" and booted on
ODROID-U2. I believe this board has the security enabled.

Unfortunately, it hangs during early boot. With earlyprintk the last
messages seen are:

L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001
L2C: platform provided aux values permit register corruption.
L2C: DT/platform modifies aux control register: 0x02070000 -> 0x3e470001
L2C-310 enabling early BRESP for Cortex-A9
L2C-310: enabling full line of zeros but not enabled in Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 16 ways, 1024 kB
L2C-310: CACHE_ID 0x4100c4c8, AUX_CTRL 0x7e470001

I then tried to go back to the earlier patch "ARM: EXYNOS: Add secure
firmware support for l2x0 init" (attached, needed a rebase) but that
one also now hangs at:

L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001

It did work on 3.14 though. Looking at the changelogs, many changes
have been made to l2x0 recently. Can you confirm that you have tested
your patches against a kernel with all of Russell King's recent
changes?

Thanks
Daniel

[-- Attachment #2: 0001-ARM-EXYNOS-Add-secure-firmware-support-for-l2x0-init.txt --]
[-- Type: text/plain, Size: 2154 bytes --]

From 2e67231f10ed0b05c2bacfdd05774fe21315d6da Mon Sep 17 00:00:00 2001
From: Gu1 <gu1@aeroxteam.fr>
Date: Mon, 21 Jan 2013 04:13:56 +0100
Subject: [PATCH] ARM: EXYNOS: Add secure firmware support for l2x0 init

Conflicts:
	arch/arm/mm/cache-l2x0.c
---
 arch/arm/mach-exynos/firmware.c | 19 +++++++++++++++++++
 arch/arm/mm/cache-l2x0.c        |  3 +++
 2 files changed, 22 insertions(+)

diff --git a/arch/arm/mach-exynos/firmware.c b/arch/arm/mach-exynos/firmware.c
index eb91d23..c200f77 100644
--- a/arch/arm/mach-exynos/firmware.c
+++ b/arch/arm/mach-exynos/firmware.c
@@ -16,6 +16,7 @@
 
 #include <asm/firmware.h>
 
+#include <asm/hardware/cache-l2x0.h>
 #include <mach/map.h>
 
 #include "common.h"
@@ -64,10 +65,28 @@ static int exynos_set_cpu_boot_addr(int cpu, unsigned long boot_addr)
 	return 0;
 }
 
+extern struct l2x0_regs l2x0_saved_regs;
+#define L2_AUX_VAL 0x7C470001
+#define L2_AUX_MASK 0xC200ffff
+
+static int exynos_l2x0_init(void)
+{
+	exynos_smc(SMC_CMD_L2X0SETUP1, l2x0_saved_regs.tag_latency,
+			l2x0_saved_regs.data_latency,
+			l2x0_saved_regs.prefetch_ctrl);
+	exynos_smc(SMC_CMD_L2X0SETUP2, l2x0_saved_regs.pwr_ctrl,
+			L2_AUX_VAL, L2_AUX_MASK);
+	exynos_smc(SMC_CMD_L2X0INVALL, 0, 0, 0);
+	exynos_smc(SMC_CMD_L2X0CTRL, 1, 0, 0);
+	return 0;
+}
+
+
 static const struct firmware_ops exynos_firmware_ops = {
 	.do_idle		= exynos_do_idle,
 	.set_cpu_boot_addr	= exynos_set_cpu_boot_addr,
 	.cpu_boot		= exynos_cpu_boot,
+	.l2x0_init		= exynos_l2x0_init,
 };
 
 void __init exynos_firmware_init(void)
diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c
index efc5cab..7367bef 100644
--- a/arch/arm/mm/cache-l2x0.c
+++ b/arch/arm/mm/cache-l2x0.c
@@ -29,6 +29,7 @@
 #include <asm/cp15.h>
 #include <asm/cputype.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/firmware.h>
 #include "cache-tauros3.h"
 #include "cache-aurora-l2.h"
 
@@ -1504,6 +1505,8 @@ int __init l2x0_of_init(u32 aux_val, u32 aux_mask)
 		if (data->of_parse)
 			data->of_parse(np, &aux_val, &aux_mask);
 
+	call_firmware_op(l2x0_init);
+
 	if (cache_id_part_number_from_dt)
 		cache_id = cache_id_part_number_from_dt;
 	else
-- 
1.9.1

Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Tomasz Figa]

Hi Daniel,

On 12.06.2014 15:38, Daniel Drake wrote:
> Hi Tomasz,
> 
> Thanks for working on this!
> 
> I have just tried this, against Linus master
> 64b2d1fbbfda07765dae3f601862796a61b2c451.
> Added patch "ARM: dts: Initial ODROID U2 support" and booted on
> ODROID-U2. I believe this board has the security enabled.
> 
> Unfortunately, it hangs during early boot. With earlyprintk the last
> messages seen are:
> 
> L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001
> L2C: platform provided aux values permit register corruption.
> L2C: DT/platform modifies aux control register: 0x02070000 -> 0x3e470001
> L2C-310 enabling early BRESP for Cortex-A9
> L2C-310: enabling full line of zeros but not enabled in Cortex-A9
> L2C-310 ID prefetch enabled, offset 1 lines
> L2C-310 dynamic clock gating enabled, standby mode enabled
> L2C-310 cache controller enabled, 16 ways, 1024 kB
> L2C-310: CACHE_ID 0x4100c4c8, AUX_CTRL 0x7e470001
> 

Thanks for testing. We have some ODROIDs U2 and U3 here too so I'll try
to reproduce and investigate the issue.

> I then tried to go back to the earlier patch "ARM: EXYNOS: Add secure
> firmware support for l2x0 init" (attached, needed a rebase) but that
> one also now hangs at:
> 
> L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001
> 
> It did work on 3.14 though. Looking at the changelogs, many changes
> have been made to l2x0 recently. Can you confirm that you have tested
> your patches against a kernel with all of Russell King's recent
> changes?

I'm usually working on latest linux-next, so I'm pretty sure all the
mentioned patches were already in my tree. The base for this series was
next-20140610 tag of linux-next tree.

Best regards,
Tomasz

Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Russell King - ARM Linux]

On Thu, Jun 12, 2014 at 02:38:49PM +0100, Daniel Drake wrote:
> From 2e67231f10ed0b05c2bacfdd05774fe21315d6da Mon Sep 17 00:00:00 2001
> From: Gu1 <gu1@aeroxteam.fr>
> Date: Mon, 21 Jan 2013 04:13:56 +0100
> Subject: [PATCH] ARM: EXYNOS: Add secure firmware support for l2x0 init
> 
> Conflicts:
> 	arch/arm/mm/cache-l2x0.c

For this patch... it's a big NAK because it's taking us right back down
the route of a totally fucked up cache-l2x0.c driver.

Why can't you people write stuff properly?  There's already another set
of patches on this mailing list which want to pass the virtual base
address of the l2x0 controller to l2c_write_sec() so that various
registers can be read back, because the platform's secure API can
only update several registers at the same time.

This is the same pattern that is revealed in this patch.  So, what
this means is that the l2c_write_sec API is wrong.  We need to come
up with a *replacement* API which allows the platforms to do this
kind of setup in a *clean* way, and stop creating rotten hacks like
this which just makes long term maintanence a nightmare.

So... please start doing stuff properly.  If you don't, you're going
to be getting more flames from me, especially if you start doing this
kind of hackery on code that I've been cleaning up to get rid of such
crap.

-- 
FTTC broadband for 0.8mile line: now at 9.7Mbps down 460kbps up... slowly
improving, and getting towards what was expected from it.

Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Tomasz Figa]

Hi Russell,

On 12.06.2014 18:20, Russell King - ARM Linux wrote:
> On Thu, Jun 12, 2014 at 02:38:49PM +0100, Daniel Drake wrote:
>> From 2e67231f10ed0b05c2bacfdd05774fe21315d6da Mon Sep 17 00:00:00 2001
>> From: Gu1 <gu1@aeroxteam.fr>
>> Date: Mon, 21 Jan 2013 04:13:56 +0100
>> Subject: [PATCH] ARM: EXYNOS: Add secure firmware support for l2x0 init
>>
>> Conflicts:
>> 	arch/arm/mm/cache-l2x0.c
> 
> For this patch... it's a big NAK because it's taking us right back down
> the route of a totally fucked up cache-l2x0.c driver.

This is just an old, internal patch that was not going to be upstreamed.
I just posted another series yesterday[1], hopefully doing it the right
way. Looking forward for review comments.

[1] http://thread.gmane.org/gmane.linux.kernel/1722989

> 
> Why can't you people write stuff properly?  There's already another set
> of patches on this mailing list which want to pass the virtual base
> address of the l2x0 controller to l2c_write_sec() so that various
> registers can be read back, because the platform's secure API can
> only update several registers at the same time.

Probably the series you mention is [1]? :)

> 
> This is the same pattern that is revealed in this patch.  So, what
> this means is that the l2c_write_sec API is wrong.  We need to come
> up with a *replacement* API which allows the platforms to do this
> kind of setup in a *clean* way, and stop creating rotten hacks like
> this which just makes long term maintanence a nightmare.
> 
> So... please start doing stuff properly.  If you don't, you're going
> to be getting more flames from me, especially if you start doing this
> kind of hackery on code that I've been cleaning up to get rid of such
> crap.
> 

Yes, I fully agree. Fortunately that was not supposed to hit upstream.

You've done a lot of great work to refactor this driver (thanks!), which
made it possible to support Exynos secure firmware in a sane way and
this is how it should be done.

Best regards,
Tomasz

Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Tomasz Figa]

[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]

Hi Daniel,

I have attached, three patches which make the kernel boot fine with L2
cache enabled on ODROID-U3. Could you test them on your setup to verify
that they indeed fix the issue?

Best regards,
Tomasz

On 12.06.2014 15:38, Daniel Drake wrote:
> Hi Tomasz,
> 
> Thanks for working on this!
> 
> I have just tried this, against Linus master
> 64b2d1fbbfda07765dae3f601862796a61b2c451.
> Added patch "ARM: dts: Initial ODROID U2 support" and booted on
> ODROID-U2. I believe this board has the security enabled.
> 
> Unfortunately, it hangs during early boot. With earlyprintk the last
> messages seen are:
> 
> L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001
> L2C: platform provided aux values permit register corruption.
> L2C: DT/platform modifies aux control register: 0x02070000 -> 0x3e470001
> L2C-310 enabling early BRESP for Cortex-A9
> L2C-310: enabling full line of zeros but not enabled in Cortex-A9
> L2C-310 ID prefetch enabled, offset 1 lines
> L2C-310 dynamic clock gating enabled, standby mode enabled
> L2C-310 cache controller enabled, 16 ways, 1024 kB
> L2C-310: CACHE_ID 0x4100c4c8, AUX_CTRL 0x7e470001
> 
> I then tried to go back to the earlier patch "ARM: EXYNOS: Add secure
> firmware support for l2x0 init" (attached, needed a rebase) but that
> one also now hangs at:
> 
> L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001
> 
> It did work on 3.14 though. Looking at the changelogs, many changes
> have been made to l2x0 recently. Can you confirm that you have tested
> your patches against a kernel with all of Russell King's recent
> changes?
> 
> Thanks
> Daniel
> 

[-- Attachment #2: 0001-ARM-EXYNOS-Invalidate-L2-cache-with-SMC-command-befo.patch --]
[-- Type: text/x-patch, Size: 818 bytes --]

>From b574212db2c1c226212c74b475acceb7fa507c27 Mon Sep 17 00:00:00 2001
From: Tomasz Figa <t.figa@samsung.com>
Date: Fri, 13 Jun 2014 16:40:29 +0200
Subject: [PATCH 1/3] ARM: EXYNOS: Invalidate L2 cache with SMC command before
 enabling

Signed-off-by: Tomasz Figa <t.figa@samsung.com>
---
 arch/arm/mach-exynos/firmware.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/mach-exynos/firmware.c b/arch/arm/mach-exynos/firmware.c
index d8544537..a688757 100644
--- a/arch/arm/mach-exynos/firmware.c
+++ b/arch/arm/mach-exynos/firmware.c
@@ -110,6 +110,8 @@ static void exynos_l2_write_sec(unsigned long val, void __iomem *base,
 {
 	switch (reg) {
 	case L2X0_CTRL:
+		if (val & L2X0_CTRL_EN)
+			exynos_smc(SMC_CMD_L2X0INVALL, 0, 0, 0);
 		exynos_smc(SMC_CMD_L2X0CTRL, val, 0, 0);
 		break;
 
-- 
1.9.3


[-- Attachment #3: 0002-ARM-mm-l2x0-Add-support-for-overriding-prefetch-sett.patch --]
[-- Type: text/x-patch, Size: 3544 bytes --]

>From 0803df887262849ab8ef905f15fdbe2b34598dde Mon Sep 17 00:00:00 2001
From: Tomasz Figa <t.figa@samsung.com>
Date: Fri, 13 Jun 2014 16:48:47 +0200
Subject: [PATCH 2/3] ARM: mm: l2x0: Add support for overriding prefetch
 settings

Signed-off-by: Tomasz Figa <t.figa@samsung.com>
---
 Documentation/devicetree/bindings/arm/l2cc.txt | 10 ++++++
 arch/arm/mm/cache-l2x0.c                       | 46 ++++++++++++++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/Documentation/devicetree/bindings/arm/l2cc.txt b/Documentation/devicetree/bindings/arm/l2cc.txt
index b513cb8..8096fcd 100644
--- a/Documentation/devicetree/bindings/arm/l2cc.txt
+++ b/Documentation/devicetree/bindings/arm/l2cc.txt
@@ -44,6 +44,16 @@ Optional properties:
 - cache-id-part: cache id part number to be used if it is not present
   on hardware
 - wt-override: If present then L2 is forced to Write through mode
+- arm,double-linefill : Override double linefill enable setting. Enable if
+  non-zero, disable if zero.
+- arm,double-linefill-incr : Override double linefill on INCR read. Enable
+  if non-zero, disable if zero.
+- arm,double-linefill-wrap : Override double linefill on WRAP read. Enable
+  if non-zero, disable if zero.
+- arm,prefetch-drop : Override prefetch drop enable setting. Enable if non-zero,
+  disable if zero.
+- arm,prefetch-offset : Override prefetch offset value. Valid values are
+  0-7, 15, 23, and 31.
 
 Example:
 
diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c
index c25cc13..de39865 100644
--- a/arch/arm/mm/cache-l2x0.c
+++ b/arch/arm/mm/cache-l2x0.c
@@ -1018,9 +1018,12 @@ static const struct l2c_init_data of_l2c220_data __initconst = {
 static void __init l2c310_of_parse(const struct device_node *np,
 	u32 *aux_val, u32 *aux_mask)
 {
+	bool set_prefetch = false;
 	u32 data[3] = { 0, 0, 0 };
 	u32 tag[3] = { 0, 0, 0 };
 	u32 filter[2] = { 0, 0 };
+	u32 prefetch;
+	u32 val;
 
 	of_property_read_u32_array(np, "arm,tag-latency", tag, ARRAY_SIZE(tag));
 	if (tag[0] && tag[1] && tag[2])
@@ -1047,6 +1050,49 @@ static void __init l2c310_of_parse(const struct device_node *np,
 		writel_relaxed((filter[0] & ~(SZ_1M - 1)) | L310_ADDR_FILTER_EN,
 			       l2x0_base + L310_ADDR_FILTER_START);
 	}
+
+	prefetch = readl_relaxed(l2x0_base + L310_PREFETCH_CTRL);
+
+	if (!of_property_read_u32(np, "arm,double-linefill", &val)) {
+		if (val)
+			prefetch |= L310_PREFETCH_CTRL_DBL_LINEFILL;
+		else
+			prefetch &= ~L310_PREFETCH_CTRL_DBL_LINEFILL;
+		set_prefetch = true;
+	}
+
+	if (!of_property_read_u32(np, "arm,double-linefill-incr", &val)) {
+		if (val)
+			prefetch |= L310_PREFETCH_CTRL_DBL_LINEFILL_INCR;
+		else
+			prefetch &= ~L310_PREFETCH_CTRL_DBL_LINEFILL_INCR;
+		set_prefetch = true;
+	}
+
+	if (!of_property_read_u32(np, "arm,double-linefill-wrap", &val)) {
+		if (!val)
+			prefetch |= L310_PREFETCH_CTRL_DBL_LINEFILL_WRAP;
+		else
+			prefetch &= ~L310_PREFETCH_CTRL_DBL_LINEFILL_WRAP;
+		set_prefetch = true;
+	}
+
+	if (!of_property_read_u32(np, "arm,prefetch-drop", &val)) {
+		if (val)
+			prefetch |= L310_PREFETCH_CTRL_PREFETCH_DROP;
+		else
+			prefetch &= ~L310_PREFETCH_CTRL_PREFETCH_DROP;
+		set_prefetch = true;
+	}
+
+	if (!of_property_read_u32(np, "arm,prefetch-offset", &val)) {
+		prefetch &= ~L310_PREFETCH_CTRL_OFFSET_MASK;
+		prefetch |= val & L310_PREFETCH_CTRL_OFFSET_MASK;
+		set_prefetch = true;
+	}
+
+	if (set_prefetch)
+		l2c_write_sec(prefetch, l2x0_base, L310_PREFETCH_CTRL);
 }
 
 static const struct l2c_init_data of_l2c310_data __initconst = {
-- 
1.9.3


[-- Attachment #4: 0003-ARM-dts-exynos4x12-Override-prefetch-settings.patch --]
[-- Type: text/x-patch, Size: 860 bytes --]

>From 032125d7f099d9160ad98371313a829131ebed8c Mon Sep 17 00:00:00 2001
From: Tomasz Figa <t.figa@samsung.com>
Date: Fri, 13 Jun 2014 16:49:09 +0200
Subject: [PATCH 3/3] ARM: dts: exynos4x12: Override prefetch settings.

Signed-off-by: Tomasz Figa <t.figa@samsung.com>
---
 arch/arm/boot/dts/exynos4x12.dtsi | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/arm/boot/dts/exynos4x12.dtsi b/arch/arm/boot/dts/exynos4x12.dtsi
index 9487f9c..ddffefe 100644
--- a/arch/arm/boot/dts/exynos4x12.dtsi
+++ b/arch/arm/boot/dts/exynos4x12.dtsi
@@ -67,6 +67,11 @@
 		cache-level = <2>;
 		arm,tag-latency = <2 2 1>;
 		arm,data-latency = <3 2 1>;
+		arm,double-linefill = <1>;
+		arm,double-linefill-incr = <0>;
+		arm,double-linefill-wrap = <1>;
+		arm,prefetch-drop = <1>;
+		arm,prefetch-offset = <7>;
 	};
 
 	clock: clock-controller@10030000 {
-- 
1.9.3

Re: [PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Daniel Drake]

Hi,

On Fri, Jun 13, 2014 at 3:59 PM, Tomasz Figa <t.figa@samsung.com> wrote:
> I have attached, three patches which make the kernel boot fine with L2
> cache enabled on ODROID-U3. Could you test them on your setup to verify
> that they indeed fix the issue?

Nice work, now my ODROID-U2 boots fine.

L2C: platform modifies aux control register: 0x02070000 -> 0x3e470001
L2C: platform provided aux values permit register corruption.
L2C: DT/platform modifies aux control register: 0x02070000 -> 0x3e470001
L2C-310 enabling early BRESP for Cortex-A9
L2C-310: enabling full line of zeros but not enabled in Cortex-A9
L2C-310 ID prefetch enabled, offset 8 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 16 ways, 1024 kB
L2C-310: CACHE_ID 0x4100c4c8, AUX_CTRL 0x7e470001

Thanks!
Daniel

[PATCH 0/5] Handle non-secure L2C initialization on Exynos4

[Tomasz Figa]

This series intends to add support for L2 cache on Exynos4 SoCs on boards
running under secure firmware, which requires certain initialization steps
to be done with help of firmware, as selected registers are writable only
from secure mode.

First three patches extend existing support for secure write in L2C driver
to account for design of secure firmware running on Exynos. Namely:
 1) direct read access to certain registers is needed on Exynos, because
    secure firmware calls set several registers at once,
 2) not all boards are running secure firmware, so .write_sec callback
    needs to be installed in Exynos firmware ops initialization code,
 3) write access to {DATA,TAG}_LATENCY_CTRL registers fron non-secure world
    is not allowed and so must use l2c_write_sec as well.
Those patches might affect other platforms using .write_sec callback, so
I'd like to kindly ask any interested people for testing.

Further two patches add impelmentation of .write_sec for Exynos secure
firmware and necessary DT nodes to enable L2 cache.

Tested on Exynos4210-based Universal C210 board (without secure firmware)
and Exynos4412-based TRATS2 board (with secure firmware).

Tomasz Figa (5):
  ARM: mm: cache-l2x0: Add base address argument to write_sec callback
  ARM: Get outer cache .write_sec callback from mach_desc only if not
    NULL
  ARM: mm: cache-l2x0: Use l2c_write_sec() for LATENCY_CTRL registers
  ARM: EXYNOS: Add .write_sec outer cache callback for L2C-310
  ARM: dts: exynos4: Add nodes for L2 cache controller

 arch/arm/boot/dts/exynos4210.dtsi  |  9 ++++++
 arch/arm/boot/dts/exynos4x12.dtsi  |  9 ++++++
 arch/arm/include/asm/mach/arch.h   |  3 +-
 arch/arm/include/asm/outercache.h  |  2 +-
 arch/arm/kernel/irq.c              |  3 +-
 arch/arm/mach-exynos/firmware.c    | 61 ++++++++++++++++++++++++++++++++++++++
 arch/arm/mach-highbank/highbank.c  |  3 +-
 arch/arm/mach-omap2/omap4-common.c |  3 +-
 arch/arm/mach-ux500/cache-l2x0.c   |  3 +-
 arch/arm/mm/cache-l2x0.c           | 10 +++----
 10 files changed, 95 insertions(+), 11 deletions(-)

-- 
1.9.3