linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround
@ 2016-06-28 17:07 Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 1/6] Revert "arm64: alternatives: add enable parameter to conditional asm macros" Andre Przywara
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

According to the errata documentation for the ARM errata 819472, 826319,
827319 and 824069, in addition to the already covered promotion of
"dc cvac" cache maintenance instructions to "dc civac"[1], we also need
to promote "dc cvau" operations.
Also as cache maintenance instructions on ARMv8 can be issued by EL0 as
well, we unfortunately have to promote them too, which is only possible
by means of trap-and-emulate.

These patches cover all in-kernel users of "dc cvau" and make sure
they are using "dc civac" if run on an affected core.
In addition if at least one core in the system has one of the above
mentioned erratas, we set the respective bit in SCTLR to trap cache
maintenance instructions from EL0 to EL1 on all CPUs, where we "emulate"
them by executing the potentially fixed instruction on behalf of userspace.

Apart from the actual patches 2/6 and 6/6, which do the main work, the
other patches are cleanups and do refactoring to make the promotion and
trapping of EL0 cache maintenance easier.

Tested on a Juno R0 with an userspace tool to issue various cache
maintenance instructions (including one with triggers a SIGSEGV) and
verified with some debugfs entries. Also tested on a (non-affected)
Juno R1 to confirm it does not trap or emulate.
At least one LTP test also issues around 100 cache maintenance
instructions, which this code survived happily.

Cheers,
Andre.

Changelog v1 .. v2:
- rebase to current upstream kernel
- revert broken macro patch instead of trying to fix it
- more refactoring for segfault injection
- fixing bug when xzr was used as the address register
- minor nitpicks

[1] commit 301bcfac4289 ("arm64: add Cortex-A53 cache errata workaround")

Andre Przywara (6):
  Revert "arm64: alternatives: add enable parameter to conditional asm
    macros"
  arm64: fix "dc cvau" cache operation on errata-affected core
  arm64: include alternative handling in dcache_by_line_op
  arm64: errata: Calling enable functions for CPU errata too
  arm64: consolidate signal injection on emulation errors
  arm64: trap userspace "dc cvau" cache operation on errata-affected
    core

 arch/arm64/include/asm/alternative.h |  16 +++--
 arch/arm64/include/asm/assembler.h   |  12 +++-
 arch/arm64/include/asm/cpufeature.h  |   2 +
 arch/arm64/include/asm/processor.h   |   1 +
 arch/arm64/include/asm/sysreg.h      |   2 +-
 arch/arm64/include/asm/traps.h       |   2 +
 arch/arm64/kernel/armv8_deprecated.c |  23 +------
 arch/arm64/kernel/cpu_errata.c       |   7 +++
 arch/arm64/kernel/cpufeature.c       |   4 +-
 arch/arm64/kernel/entry.S            |  12 +++-
 arch/arm64/kernel/traps.c            | 119 +++++++++++++++++++++++++++++++----
 arch/arm64/mm/cache.S                |   2 +-
 12 files changed, 154 insertions(+), 48 deletions(-)

-- 
2.9.0

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 1/6] Revert "arm64: alternatives: add enable parameter to conditional asm macros"
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
@ 2016-06-28 17:07 ` Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 2/6] arm64: fix "dc cvau" cache operation on errata-affected core Andre Przywara
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

Commit 77ee306c0aea9 ("arm64: alternatives: add enable parameter to
conditional asm macros") extended the alternative assembly macros.
Unfortunately this does not really work as one would expect, as the
enable parameter in fact correctly protects the alternative section
magic, but not the actual code sequences.
This results in having both the original instruction(s) _and_  the
alternative ones, if enable if false.
Since there is no user of this macros anyway, just revert it.

This reverts commit 77ee306c0aea9a219daec256ad25982944affef8.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm64/include/asm/alternative.h | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
index beccbde..7288071 100644
--- a/arch/arm64/include/asm/alternative.h
+++ b/arch/arm64/include/asm/alternative.h
@@ -95,13 +95,11 @@ void apply_alternatives(void *start, size_t length);
  * The code that follows this macro will be assembled and linked as
  * normal. There are no restrictions on this code.
  */
-.macro alternative_if_not cap, enable = 1
-	.if \enable
+.macro alternative_if_not cap
 	.pushsection .altinstructions, "a"
 	altinstruction_entry 661f, 663f, \cap, 662f-661f, 664f-663f
 	.popsection
 661:
-	.endif
 .endm
 
 /*
@@ -118,22 +116,18 @@ void apply_alternatives(void *start, size_t length);
  *    alternative sequence it is defined in (branches into an
  *    alternative sequence are not fixed up).
  */
-.macro alternative_else, enable = 1
-	.if \enable
+.macro alternative_else
 662:	.pushsection .altinstr_replacement, "ax"
 663:
-	.endif
 .endm
 
 /*
  * Complete an alternative code sequence.
  */
-.macro alternative_endif, enable = 1
-	.if \enable
+.macro alternative_endif
 664:	.popsection
 	.org	. - (664b-663b) + (662b-661b)
 	.org	. - (662b-661b) + (664b-663b)
-	.endif
 .endm
 
 #define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...)	\
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 2/6] arm64: fix "dc cvau" cache operation on errata-affected core
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 1/6] Revert "arm64: alternatives: add enable parameter to conditional asm macros" Andre Przywara
@ 2016-06-28 17:07 ` Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 3/6] arm64: include alternative handling in dcache_by_line_op Andre Przywara
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

The ARM errata 819472, 826319, 827319 and 824069 for affected
Cortex-A53 cores demand to promote "dc cvau" instructions to
"dc civac" as well.
Attribute the usage of the instruction in __flush_cache_user_range
to also be covered by our alternative patching efforts.
For that we introduce an assembly macro which both deals with
alternatives while still tagging the instructions as USER.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm64/include/asm/alternative.h | 4 ++++
 arch/arm64/mm/cache.S                | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h
index 7288071..8746ff6 100644
--- a/arch/arm64/include/asm/alternative.h
+++ b/arch/arm64/include/asm/alternative.h
@@ -133,6 +133,10 @@ void apply_alternatives(void *start, size_t length);
 #define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...)	\
 	alternative_insn insn1, insn2, cap, IS_ENABLED(cfg)
 
+.macro user_alt, label, oldinstr, newinstr, cond
+9999:	alternative_insn "\oldinstr", "\newinstr", \cond
+	_ASM_EXTABLE 9999b, \label
+.endm
 
 /*
  * Generate the assembly for UAO alternatives with exception table entries.
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
index 50ff9ba..07d7352 100644
--- a/arch/arm64/mm/cache.S
+++ b/arch/arm64/mm/cache.S
@@ -52,7 +52,7 @@ ENTRY(__flush_cache_user_range)
 	sub	x3, x2, #1
 	bic	x4, x0, x3
 1:
-USER(9f, dc	cvau, x4	)		// clean D line to PoU
+user_alt 9f, "dc cvau, x4",  "dc civac, x4",  ARM64_WORKAROUND_CLEAN_CACHE
 	add	x4, x4, x2
 	cmp	x4, x1
 	b.lo	1b
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 3/6] arm64: include alternative handling in dcache_by_line_op
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 1/6] Revert "arm64: alternatives: add enable parameter to conditional asm macros" Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 2/6] arm64: fix "dc cvau" cache operation on errata-affected core Andre Przywara
@ 2016-06-28 17:07 ` Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 4/6] arm64: errata: Calling enable functions for CPU errata too Andre Przywara
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

The newly introduced dcache_by_line_op macro is used at least in
one occassion at the moment to issue a "dc cvau" instruction,
which is affected by ARM errata 819472, 826319, 827319 and 824069.
Change the macro to allow for alternative patching in there to
protect affected Cortex-A53 cores.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm64/include/asm/assembler.h | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index 10b017c..1a021f1 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -24,6 +24,7 @@
 #define __ASM_ASSEMBLER_H
 
 #include <asm/asm-offsets.h>
+#include <asm/cpufeature.h>
 #include <asm/page.h>
 #include <asm/pgtable-hwdef.h>
 #include <asm/ptrace.h>
@@ -261,7 +262,16 @@ lr	.req	x30		// link register
 	add	\size, \kaddr, \size
 	sub	\tmp2, \tmp1, #1
 	bic	\kaddr, \kaddr, \tmp2
-9998:	dc	\op, \kaddr
+9998:
+	.if	(\op == cvau || \op == cvac)
+		alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE
+		dc	\op, \kaddr
+		alternative_else
+		dc	civac, \kaddr
+		alternative_endif
+	.else
+		dc	\op, \kaddr
+	.endif
 	add	\kaddr, \kaddr, \tmp1
 	cmp	\kaddr, \size
 	b.lo	9998b
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 4/6] arm64: errata: Calling enable functions for CPU errata too
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
                   ` (2 preceding siblings ...)
  2016-06-28 17:07 ` [PATCH v2 3/6] arm64: include alternative handling in dcache_by_line_op Andre Przywara
@ 2016-06-28 17:07 ` Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 5/6] arm64: consolidate signal injection on emulation errors Andre Przywara
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

Currently we call the (optional) enable function for CPU _features_
only. As CPU _errata_ descriptions share the same data structure and
having an enable function is useful for errata as well (for instance
to set bits in SCTLR), lets call it when enumerating erratas too.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm64/include/asm/cpufeature.h | 2 ++
 arch/arm64/kernel/cpu_errata.c      | 5 +++++
 arch/arm64/kernel/cpufeature.c      | 4 ++--
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 224efe7..49dd1bd 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -191,7 +191,9 @@ void __init setup_cpu_features(void);
 
 void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
 			    const char *info);
+void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps);
 void check_local_cpu_errata(void);
+void __init enable_errata_workarounds(void);
 
 void verify_local_cpu_errata(void);
 void verify_local_cpu_capabilities(void);
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index d427894..c2261a7 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -127,3 +127,8 @@ void check_local_cpu_errata(void)
 {
 	update_cpu_capabilities(arm64_errata, "enabling workaround for");
 }
+
+void __init enable_errata_workarounds(void)
+{
+	enable_cpu_capabilities(arm64_errata);
+}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 811773d..916d27a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -913,8 +913,7 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
  * Run through the enabled capabilities and enable() it on all active
  * CPUs
  */
-static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
 {
 	for (; caps->matches; caps++)
 		if (caps->enable && cpus_have_cap(caps->capability))
@@ -1036,6 +1035,7 @@ void __init setup_cpu_features(void)
 
 	/* Set the CPU feature capabilies */
 	setup_feature_capabilities();
+	enable_errata_workarounds();
 	setup_elf_hwcaps(arm64_elf_hwcaps);
 
 	if (system_supports_32bit_el0())
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 5/6] arm64: consolidate signal injection on emulation errors
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
                   ` (3 preceding siblings ...)
  2016-06-28 17:07 ` [PATCH v2 4/6] arm64: errata: Calling enable functions for CPU errata too Andre Przywara
@ 2016-06-28 17:07 ` Andre Przywara
  2016-06-28 17:07 ` [PATCH v2 6/6] arm64: trap userspace "dc cvau" cache operation on errata-affected core Andre Przywara
  2016-07-01 10:53 ` [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Catalin Marinas
  6 siblings, 0 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

The code for injecting a signal into userland if a trapped instruction
fails emulation due to a _userland_ error (like an illegal address)
will be used more often with the next patch.
Factor out the core functionality into a separate function and use
that both for the existing trap handler and for the deprecated
instructions emulation.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm64/include/asm/traps.h       |  2 ++
 arch/arm64/kernel/armv8_deprecated.c | 23 +------------
 arch/arm64/kernel/traps.c            | 63 ++++++++++++++++++++++++++++--------
 3 files changed, 53 insertions(+), 35 deletions(-)

diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h
index 0cc2f29..816b293 100644
--- a/arch/arm64/include/asm/traps.h
+++ b/arch/arm64/include/asm/traps.h
@@ -34,6 +34,8 @@ struct undef_hook {
 void register_undef_hook(struct undef_hook *hook);
 void unregister_undef_hook(struct undef_hook *hook);
 
+void set_segfault(struct pt_regs *regs, unsigned long addr);
+
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 static inline int __in_irqentry_text(unsigned long ptr)
 {
diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c
index c37202c..b619274 100644
--- a/arch/arm64/kernel/armv8_deprecated.c
+++ b/arch/arm64/kernel/armv8_deprecated.c
@@ -316,28 +316,6 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table)
  */
 #define TYPE_SWPB (1 << 22)
 
-/*
- * Set up process info to signal segmentation fault - called on access error.
- */
-static void set_segfault(struct pt_regs *regs, unsigned long addr)
-{
-	siginfo_t info;
-
-	down_read(&current->mm->mmap_sem);
-	if (find_vma(current->mm, addr) == NULL)
-		info.si_code = SEGV_MAPERR;
-	else
-		info.si_code = SEGV_ACCERR;
-	up_read(&current->mm->mmap_sem);
-
-	info.si_signo = SIGSEGV;
-	info.si_errno = 0;
-	info.si_addr  = (void *) instruction_pointer(regs);
-
-	pr_debug("SWP{B} emulation: access caused memory abort!\n");
-	arm64_notify_die("Illegal memory access", regs, &info, 0);
-}
-
 static int emulate_swpX(unsigned int address, unsigned int *data,
 			unsigned int type)
 {
@@ -430,6 +408,7 @@ ret:
 	return 0;
 
 fault:
+	pr_debug("SWP{B} emulation: access caused memory abort!\n");
 	set_segfault(regs, address);
 
 	return 0;
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 2a43012..613b391 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -373,30 +373,67 @@ exit:
 	return fn ? fn(regs, instr) : 1;
 }
 
-asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
+static void force_signal_inject(int signal, int code, struct pt_regs *regs,
+				unsigned long address)
 {
 	siginfo_t info;
 	void __user *pc = (void __user *)instruction_pointer(regs);
+	const char *desc;
 
-	/* check for AArch32 breakpoint instructions */
-	if (!aarch32_break_handler(regs))
-		return;
-
-	if (call_undef_hook(regs) == 0)
-		return;
+	switch (signal) {
+	case SIGILL:
+		desc = "undefined instruction";
+		break;
+	case SIGSEGV:
+		desc = "illegal memory access";
+		break;
+	default:
+		desc = "bad mode";
+		break;
+	}
 
-	if (unhandled_signal(current, SIGILL) && show_unhandled_signals_ratelimited()) {
-		pr_info("%s[%d]: undefined instruction: pc=%p\n",
-			current->comm, task_pid_nr(current), pc);
+	if (unhandled_signal(current, signal) &&
+	    show_unhandled_signals_ratelimited()) {
+		pr_info("%s[%d]: %s: pc=%p\n",
+			current->comm, task_pid_nr(current), desc, pc);
 		dump_instr(KERN_INFO, regs);
 	}
 
-	info.si_signo = SIGILL;
+	info.si_signo = signal;
 	info.si_errno = 0;
-	info.si_code  = ILL_ILLOPC;
+	info.si_code  = code;
 	info.si_addr  = pc;
 
-	arm64_notify_die("Oops - undefined instruction", regs, &info, 0);
+	arm64_notify_die(desc, regs, &info, 0);
+}
+
+/*
+ * Set up process info to signal segmentation fault - called on access error.
+ */
+void set_segfault(struct pt_regs *regs, unsigned long addr)
+{
+	int code;
+
+	down_read(&current->mm->mmap_sem);
+	if (find_vma(current->mm, addr) == NULL)
+		code = SEGV_MAPERR;
+	else
+		code = SEGV_ACCERR;
+	up_read(&current->mm->mmap_sem);
+
+	force_signal_inject(SIGSEGV, code, regs, addr);
+}
+
+asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
+{
+	/* check for AArch32 breakpoint instructions */
+	if (!aarch32_break_handler(regs))
+		return;
+
+	if (call_undef_hook(regs) == 0)
+		return;
+
+	force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
 }
 
 long compat_arm_syscall(struct pt_regs *regs);
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 6/6] arm64: trap userspace "dc cvau" cache operation on errata-affected core
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
                   ` (4 preceding siblings ...)
  2016-06-28 17:07 ` [PATCH v2 5/6] arm64: consolidate signal injection on emulation errors Andre Przywara
@ 2016-06-28 17:07 ` Andre Przywara
  2016-07-01 10:53 ` [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Catalin Marinas
  6 siblings, 0 replies; 8+ messages in thread
From: Andre Przywara @ 2016-06-28 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

The ARM errata 819472, 826319, 827319 and 824069 for affected
Cortex-A53 cores demand to promote "dc cvau" instructions to
"dc civac". Since we allow userspace to also emit those instructions,
we should make sure that "dc cvau" gets promoted there too.
So lets grasp the nettle here and actually trap every userland cache
maintenance instruction once we detect at least one affected core in
the system.
We then emulate the instruction by executing it on behalf of userland,
promoting "dc cvau" to "dc civac" on the way and injecting access
fault back into userspace.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm64/include/asm/processor.h |  1 +
 arch/arm64/include/asm/sysreg.h    |  2 +-
 arch/arm64/kernel/cpu_errata.c     |  2 ++
 arch/arm64/kernel/entry.S          | 12 +++++++-
 arch/arm64/kernel/traps.c          | 60 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 75 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index cef1cf3..ace0a96 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -192,5 +192,6 @@ static inline void spin_lock_prefetch(const void *ptr)
 
 void cpu_enable_pan(void *__unused);
 void cpu_enable_uao(void *__unused);
+void cpu_enable_cache_maint_trap(void *__unused);
 
 #endif /* __ASM_PROCESSOR_H */
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 751e901..cc06794 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -98,11 +98,11 @@
 			 SCTLR_ELx_SA | SCTLR_ELx_I)
 
 /* SCTLR_EL1 specific flags. */
+#define SCTLR_EL1_UCI		(1 << 26)
 #define SCTLR_EL1_SPAN		(1 << 23)
 #define SCTLR_EL1_SED		(1 << 8)
 #define SCTLR_EL1_CP15BEN	(1 << 5)
 
-
 /* id_aa64isar0 */
 #define ID_AA64ISAR0_RDM_SHIFT		28
 #define ID_AA64ISAR0_ATOMICS_SHIFT	20
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index c2261a7..af647d2 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -46,6 +46,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		.desc = "ARM errata 826319, 827319, 824069",
 		.capability = ARM64_WORKAROUND_CLEAN_CACHE,
 		MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
+		.enable = cpu_enable_cache_maint_trap,
 	},
 #endif
 #ifdef CONFIG_ARM64_ERRATUM_819472
@@ -54,6 +55,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 		.desc = "ARM errata 819472",
 		.capability = ARM64_WORKAROUND_CLEAN_CACHE,
 		MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
+		.enable = cpu_enable_cache_maint_trap,
 	},
 #endif
 #ifdef CONFIG_ARM64_ERRATUM_832075
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index 12e8d2b..86480b4 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -451,7 +451,7 @@ el0_sync:
 	cmp	x24, #ESR_ELx_EC_FP_EXC64	// FP/ASIMD exception
 	b.eq	el0_fpsimd_exc
 	cmp	x24, #ESR_ELx_EC_SYS64		// configurable trap
-	b.eq	el0_undef
+	b.eq	el0_sys
 	cmp	x24, #ESR_ELx_EC_SP_ALIGN	// stack alignment exception
 	b.eq	el0_sp_pc
 	cmp	x24, #ESR_ELx_EC_PC_ALIGN	// pc alignment exception
@@ -579,6 +579,16 @@ el0_undef:
 	mov	x0, sp
 	bl	do_undefinstr
 	b	ret_to_user
+el0_sys:
+	/*
+	 * System instructions, for trapped cache maintenance instructions
+	 */
+	enable_dbg_and_irq
+	ct_user_exit
+	mov	x0, x25
+	mov	x1, sp
+	bl	do_sysinstr
+	b	ret_to_user
 el0_dbg:
 	/*
 	 * Debug exception handling
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 613b391..dc6a1db 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -41,6 +41,7 @@
 #include <asm/stacktrace.h>
 #include <asm/exception.h>
 #include <asm/system_misc.h>
+#include <asm/sysreg.h>
 
 static const char *handler[]= {
 	"Synchronous Abort",
@@ -436,6 +437,65 @@ asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
 	force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
 }
 
+void cpu_enable_cache_maint_trap(void *__unused)
+{
+	config_sctlr_el1(SCTLR_EL1_UCI, 0);
+}
+
+#define __user_cache_maint(insn, address, res)			\
+	asm volatile (						\
+		"1:	" insn ", %1\n"				\
+		"	mov	%w0, #0\n"			\
+		"2:\n"						\
+		"	.pushsection .fixup,\"ax\"\n"		\
+		"	.align	2\n"				\
+		"3:	mov	%w0, %w2\n"			\
+		"	b	2b\n"				\
+		"	.popsection\n"				\
+		_ASM_EXTABLE(1b, 3b)				\
+		: "=r" (res)					\
+		: "r" (address), "i" (-EFAULT) )
+
+asmlinkage void __exception do_sysinstr(unsigned int esr, struct pt_regs *regs)
+{
+	unsigned long address;
+	int ret;
+
+	/* if this is a write with: Op0=1, Op2=1, Op1=3, CRn=7 */
+	if ((esr & 0x01fffc01) == 0x0012dc00) {
+		int rt = (esr >> 5) & 0x1f;
+		int crm = (esr >> 1) & 0x0f;
+
+		address = (rt == 31) ? 0 : regs->regs[rt];
+
+		switch (crm) {
+		case 11:		/* DC CVAU, gets promoted */
+			__user_cache_maint("dc civac", address, ret);
+			break;
+		case 10:		/* DC CVAC, gets promoted */
+			__user_cache_maint("dc civac", address, ret);
+			break;
+		case 14:		/* DC CIVAC */
+			__user_cache_maint("dc civac", address, ret);
+			break;
+		case 5:			/* IC IVAU */
+			__user_cache_maint("ic ivau", address, ret);
+			break;
+		default:
+			force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
+			return;
+		}
+	} else {
+		force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
+		return;
+	}
+
+	if (ret)
+		set_segfault(regs, address);
+	else
+		regs->pc += 4;
+}
+
 long compat_arm_syscall(struct pt_regs *regs);
 
 asmlinkage long do_ni_syscall(struct pt_regs *regs)
-- 
2.9.0

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround
  2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
                   ` (5 preceding siblings ...)
  2016-06-28 17:07 ` [PATCH v2 6/6] arm64: trap userspace "dc cvau" cache operation on errata-affected core Andre Przywara
@ 2016-07-01 10:53 ` Catalin Marinas
  6 siblings, 0 replies; 8+ messages in thread
From: Catalin Marinas @ 2016-07-01 10:53 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Jun 28, 2016 at 06:07:26PM +0100, Andre Przywara wrote:
> Andre Przywara (6):
>   Revert "arm64: alternatives: add enable parameter to conditional asm
>     macros"
>   arm64: fix "dc cvau" cache operation on errata-affected core
>   arm64: include alternative handling in dcache_by_line_op
>   arm64: errata: Calling enable functions for CPU errata too
>   arm64: consolidate signal injection on emulation errors
>   arm64: trap userspace "dc cvau" cache operation on errata-affected
>     core

Patches queued for 4.8. Thanks.

-- 
Catalin

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2016-07-01 10:53 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-06-28 17:07 [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Andre Przywara
2016-06-28 17:07 ` [PATCH v2 1/6] Revert "arm64: alternatives: add enable parameter to conditional asm macros" Andre Przywara
2016-06-28 17:07 ` [PATCH v2 2/6] arm64: fix "dc cvau" cache operation on errata-affected core Andre Przywara
2016-06-28 17:07 ` [PATCH v2 3/6] arm64: include alternative handling in dcache_by_line_op Andre Przywara
2016-06-28 17:07 ` [PATCH v2 4/6] arm64: errata: Calling enable functions for CPU errata too Andre Przywara
2016-06-28 17:07 ` [PATCH v2 5/6] arm64: consolidate signal injection on emulation errors Andre Przywara
2016-06-28 17:07 ` [PATCH v2 6/6] arm64: trap userspace "dc cvau" cache operation on errata-affected core Andre Przywara
2016-07-01 10:53 ` [PATCH v2 0/6] arm64: Extend Cortex-A53 errata workaround Catalin Marinas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).