[PATCH 0/2] Detect and handle I-cache aliases when D-cache is non-aliasing

[PATCH 0/2] Detect and handle I-cache aliases when D-cache is non-aliasing

[Will Deacon]

Last month, I posted some patches to fix ptrace software breakpoints:

http://lists.infradead.org/pipermail/linux-arm-kernel/2010-July/020649.html

One of these patches added a workaround for a hardware erratum, so I
have submitted that to the patch system. The other patch simply called
__flush_icache_all() in flush_ptrace_access to avoid having to worry
about I-cache aliases when cache_is_vipt_nonaliasing() is true. This
isn't ideal if the debugger decides to insert lots of software breakpoints
sequentially (for example, when single-stepping a number of times).

These two patches add proper identification of aliasing I-caches
when the D-cache is nonaliasing. flush_ptrace_access can then
flush only the relevant I-cache lines instead of the entire cache.

Cc: Russell King - ARM Linux <linux@arm.linux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>

Will Deacon (2):
  ARM: setup: detect aliasing I-cache when D-cache is non-aliasing
  ARM: flush_ptrace_access: invalidate correct I-cache alias

 arch/arm/include/asm/cachetype.h |    8 ++++++--
 arch/arm/kernel/setup.c          |   38 +++++++++++++++++++++++++++++++++++---
 arch/arm/mm/flush.c              |   23 ++++++++++++++++++++---
 3 files changed, 61 insertions(+), 8 deletions(-)

[PATCH 1/2] ARM: setup: detect aliasing I-cache when D-cache is non-aliasing

[Will Deacon]

Currently, the Kernel assumes that if a CPU has a non-aliasing D-cache
then the I-cache is also non-aliasing. This may not be true on ARM cores
from v6 onwards, which may have aliasing I-caches but non-aliasing
D-caches.

This patch adds a cpu_has_aliasing_icache function, which is called from
cacheid_init and adds CACHEID_VIPT_I_ALIASING to the cacheid when
appropriate. A utility macro, icache_is_vipt_aliasing(), is also
provided.

Cc: Russell King - ARM Linux <linux@arm.linux.org.uk>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
 arch/arm/include/asm/cachetype.h |    8 ++++++--
 arch/arm/kernel/setup.c          |   38 +++++++++++++++++++++++++++++++++++---
 2 files changed, 41 insertions(+), 5 deletions(-)

diff --git a/arch/arm/include/asm/cachetype.h b/arch/arm/include/asm/cachetype.h
index d3a4c2c..c023db0 100644
--- a/arch/arm/include/asm/cachetype.h
+++ b/arch/arm/include/asm/cachetype.h
@@ -6,6 +6,7 @@
 #define CACHEID_VIPT_ALIASING		(1 << 2)
 #define CACHEID_VIPT			(CACHEID_VIPT_ALIASING|CACHEID_VIPT_NONALIASING)
 #define CACHEID_ASID_TAGGED		(1 << 3)
+#define CACHEID_VIPT_I_ALIASING		(1 << 4)
 
 extern unsigned int cacheid;
 
@@ -14,15 +15,18 @@ extern unsigned int cacheid;
 #define cache_is_vipt_nonaliasing()	cacheid_is(CACHEID_VIPT_NONALIASING)
 #define cache_is_vipt_aliasing()	cacheid_is(CACHEID_VIPT_ALIASING)
 #define icache_is_vivt_asid_tagged()	cacheid_is(CACHEID_ASID_TAGGED)
+#define icache_is_vipt_aliasing()	cacheid_is(CACHEID_VIPT_I_ALIASING)
 
 /*
  * __LINUX_ARM_ARCH__ is the minimum supported CPU architecture
  * Mask out support which will never be present on newer CPUs.
  * - v6+ is never VIVT
- * - v7+ VIPT never aliases
+ * - v7+ VIPT never aliases on D-side
  */
 #if __LINUX_ARM_ARCH__ >= 7
-#define __CACHEID_ARCH_MIN	(CACHEID_VIPT_NONALIASING | CACHEID_ASID_TAGGED)
+#define __CACHEID_ARCH_MIN	(CACHEID_VIPT_NONALIASING |\
+				 CACHEID_ASID_TAGGED |\
+				 CACHEID_VIPT_I_ALIASING)
 #elif __LINUX_ARM_ARCH__ >= 6
 #define	__CACHEID_ARCH_MIN	(~CACHEID_VIVT)
 #else
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 122d999..cf8e590 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -233,6 +233,33 @@ int cpu_architecture(void)
 	return cpu_arch;
 }
 
+static int cpu_has_aliasing_icache(unsigned int arch)
+{
+	int aliasing_icache;
+	unsigned int id_reg, num_sets, line_size;
+
+	/* arch specifies the register format */
+	switch (arch) {
+	case CPU_ARCH_ARMv7:
+		asm("mcr	p15, 2, %1, c0, c0, 0	@ set CSSELR\n"
+		    "mrc	p15, 1, %0, c0, c0, 0	@ read CCSIDR"
+		    : "=r" (id_reg)
+		    : "r" (1));
+		line_size = 4 << ((id_reg & 0x7) + 2);
+		num_sets = ((id_reg >> 13) & 0x7fff) + 1;
+		aliasing_icache = (line_size * num_sets) > PAGE_SIZE;
+		break;
+	case CPU_ARCH_ARMv6:
+		aliasing_icache = read_cpuid_cachetype() & (1 << 11);
+		break;
+	default:
+		/* I-cache aliases will be handled by D-cache aliasing code */
+		aliasing_icache = 0;
+	}
+
+	return aliasing_icache;
+}
+
 static void __init cacheid_init(void)
 {
 	unsigned int cachetype = read_cpuid_cachetype();
@@ -244,10 +271,15 @@ static void __init cacheid_init(void)
 			cacheid = CACHEID_VIPT_NONALIASING;
 			if ((cachetype & (3 << 14)) == 1 << 14)
 				cacheid |= CACHEID_ASID_TAGGED;
-		} else if (cachetype & (1 << 23))
+			else if (cpu_has_aliasing_icache(CPU_ARCH_ARMv7))
+				cacheid |= CACHEID_VIPT_I_ALIASING;
+		} else if (cachetype & (1 << 23)) {
 			cacheid = CACHEID_VIPT_ALIASING;
-		else
+		} else {
 			cacheid = CACHEID_VIPT_NONALIASING;
+			if (cpu_has_aliasing_icache(CPU_ARCH_ARMv6))
+				cacheid |= CACHEID_VIPT_I_ALIASING;
+		}
 	} else {
 		cacheid = CACHEID_VIVT;
 	}
@@ -258,7 +290,7 @@ static void __init cacheid_init(void)
 		cache_is_vipt_nonaliasing() ? "VIPT nonaliasing" : "unknown",
 		cache_is_vivt() ? "VIVT" :
 		icache_is_vivt_asid_tagged() ? "VIVT ASID tagged" :
-		cache_is_vipt_aliasing() ? "VIPT aliasing" :
+		icache_is_vipt_aliasing() ? "VIPT aliasing" :
 		cache_is_vipt_nonaliasing() ? "VIPT nonaliasing" : "unknown");
 }
 
-- 
1.6.3.3

[PATCH 2/2] ARM: flush_ptrace_access: invalidate correct I-cache alias

[Will Deacon]

copy_to_user_page can be used by access_process_vm to write to an
executable page of a process using a mapping acquired by kmap.
For systems with I-cache aliasing, flushing the I-cache using the
Kernel mapping may leave stale data in the I-cache if the user
mapping is of a different colour.

This patch introduces a flush_icache_alias function to flush.c,
which calls flush_icache_range with a mapping of the specified
colour. flush_ptrace_access is then modified to call this new
function instead of coherent_kern_range in the case of an aliasing
I-cache and a non-aliasing D-cache.

Cc: Russell King - ARM Linux <linux@arm.linux.org.uk>
Signed-off-by: Will Deacon <will.deacon@arm.com>
---
 arch/arm/mm/flush.c |   23 ++++++++++++++++++++---
 1 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c
index c6844cb..25c771c 100644
--- a/arch/arm/mm/flush.c
+++ b/arch/arm/mm/flush.c
@@ -39,6 +39,18 @@ static void flush_pfn_alias(unsigned long pfn, unsigned long vaddr)
 	    : "cc");
 }
 
+static void flush_icache_alias(unsigned long pfn, unsigned long vaddr, unsigned long len)
+{
+	unsigned long colour = CACHE_COLOUR(vaddr);
+	unsigned long offset = vaddr & (PAGE_SIZE - 1);
+	unsigned long to;
+
+	set_pte_ext(TOP_PTE(ALIAS_FLUSH_START) + colour, pfn_pte(pfn, PAGE_KERNEL), 0);
+	to = ALIAS_FLUSH_START + (colour << PAGE_SHIFT) + offset;
+	flush_tlb_kernel_page(to);
+	flush_icache_range(to, to + len);
+}
+
 void flush_cache_mm(struct mm_struct *mm)
 {
 	if (cache_is_vivt()) {
@@ -89,8 +101,10 @@ void flush_cache_page(struct vm_area_struct *vma, unsigned long user_addr, unsig
 	if (vma->vm_flags & VM_EXEC && icache_is_vivt_asid_tagged())
 		__flush_icache_all();
 }
+
 #else
-#define flush_pfn_alias(pfn,vaddr)	do { } while (0)
+#define flush_pfn_alias(pfn,vaddr)		do { } while (0)
+#define flush_icache_alias(pfn,vaddr,len)	do { } while (0)
 #endif
 
 #ifdef CONFIG_SMP
@@ -118,10 +132,13 @@ void flush_ptrace_access(struct vm_area_struct *vma, struct page *page,
 		return;
 	}
 
-	/* VIPT non-aliasing cache */
+	/* VIPT non-aliasing D-cache */
 	if (vma->vm_flags & VM_EXEC) {
 		unsigned long addr = (unsigned long)kaddr;
-		__cpuc_coherent_kern_range(addr, addr + len);
+		if (icache_is_vipt_aliasing())
+			flush_icache_alias(page_to_pfn(page), uaddr, len);
+		else
+			__cpuc_coherent_kern_range(addr, addr + len);
 #ifdef CONFIG_SMP
 		if (cache_ops_need_broadcast())
 			smp_call_function(flush_ptrace_access_other,
-- 
1.6.3.3

[PATCH 0/2] Detect and handle I-cache aliases when D-cache is non-aliasing

[Will Deacon]

Hi Russell,

> Last month, I posted some patches to fix ptrace software breakpoints:
> 
> http://lists.infradead.org/pipermail/linux-arm-kernel/2010-July/020649.html
> 
> One of these patches added a workaround for a hardware erratum, so I
> have submitted that to the patch system. The other patch simply called
> __flush_icache_all() in flush_ptrace_access to avoid having to worry
> about I-cache aliases when cache_is_vipt_nonaliasing() is true. This
> isn't ideal if the debugger decides to insert lots of software breakpoints
> sequentially (for example, when single-stepping a number of times).
> 
> These two patches add proper identification of aliasing I-caches
> when the D-cache is nonaliasing. flush_ptrace_access can then
> flush only the relevant I-cache lines instead of the entire cache.
> 
> Cc: Russell King - ARM Linux <linux@arm.linux.org.uk>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> 
> Will Deacon (2):
>   ARM: setup: detect aliasing I-cache when D-cache is non-aliasing
>   ARM: flush_ptrace_access: invalidate correct I-cache alias
> 
>  arch/arm/include/asm/cachetype.h |    8 ++++++--
>  arch/arm/kernel/setup.c          |   38 +++++++++++++++++++++++++++++++++++---
>  arch/arm/mm/flush.c              |   23 ++++++++++++++++++++---
>  3 files changed, 61 insertions(+), 8 deletions(-)

Any thoughts on these patches? I prefer them to my previous approach but
I'd like a second opinion.

Thanks,

Will