* [PATCH 0/4] Patches for -next
@ 2010-05-20 14:29 Catalin Marinas
2010-05-20 14:30 ` [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs Catalin Marinas
` (4 more replies)
0 siblings, 5 replies; 23+ messages in thread
From: Catalin Marinas @ 2010-05-20 14:29 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
I would like to get these patches in the linux-next tree for this
release cycle and aim to submit them for upstream 2.6.36-rc1 (next
merging window).
Please let me know if you have any objections.
Thanks.
Catalin Marinas (4):
ARM: Remove the domain switching on ARMv6k/v7 CPUs
ARM: Use lazy cache flushing on ARMv7 SMP systems
ARM: Assume new page cache pages have dirty D-cache
ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP
arch/arm/include/asm/assembler.h | 13 +++---
arch/arm/include/asm/cacheflush.h | 6 +--
arch/arm/include/asm/domain.h | 31 +++++++++++++-
arch/arm/include/asm/futex.h | 9 ++--
arch/arm/include/asm/pgtable.h | 12 +++++
arch/arm/include/asm/smp_plat.h | 4 ++
arch/arm/include/asm/tlbflush.h | 2 -
arch/arm/include/asm/uaccess.h | 16 ++++---
arch/arm/kernel/entry-armv.S | 4 +-
arch/arm/kernel/traps.c | 17 ++++++++
arch/arm/lib/getuser.S | 13 +++---
arch/arm/lib/putuser.S | 29 +++++++------
arch/arm/lib/uaccess.S | 83 +++++++++++++++++++------------------
arch/arm/mm/Kconfig | 8 ++++
arch/arm/mm/copypage-v4mc.c | 2 -
arch/arm/mm/copypage-v6.c | 2 -
arch/arm/mm/copypage-xscale.c | 2 -
arch/arm/mm/dma-mapping.c | 6 +++
arch/arm/mm/fault-armv.c | 21 ++++++---
arch/arm/mm/flush.c | 15 ++-----
arch/arm/mm/proc-macros.S | 7 +++
arch/arm/mm/proc-v7.S | 5 +-
22 files changed, 197 insertions(+), 110 deletions(-)
--
Catalin
^ permalink raw reply [flat|nested] 23+ messages in thread* [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs 2010-05-20 14:29 [PATCH 0/4] Patches for -next Catalin Marinas @ 2010-05-20 14:30 ` Catalin Marinas 2010-05-24 11:45 ` Russell King - ARM Linux 2010-05-20 14:30 ` [PATCH 2/4] ARM: Use lazy cache flushing on ARMv7 SMP systems Catalin Marinas ` (3 subsequent siblings) 4 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-20 14:30 UTC (permalink / raw) To: linux-arm-kernel This patch removes the domain switching functionality via the set_fs and __switch_to functions on cores that have a TLS register. Currently, the ioremap and vmalloc areas share the same level 1 page tables and therefore have the same domain (DOMAIN_KERNEL). When the kernel domain is modified from Client to Manager (via the __set_fs or in the __switch_to function), the XN (eXecute Never) bit is overridden and newer CPUs can speculatively prefetch the ioremap'ed memory. Linux performs the kernel domain switching to allow user-specific functions (copy_to/from_user, get/put_user etc.) to access kernel memory. In order for these functions to work with the kernel domain set to Client, the patch modifies the LDRT/STRT and related instructions to the LDR/STR ones. The user pages access rights are also modified for kernel read-only access rather than read/write so that the copy-on-write mechanism still works. CPU_USE_DOMAINS gets disabled only if HAS_TLS_REG is defined since writing the TLS value to the high vectors page isn't possible. The user addresses passed to the kernel are checked by the access_ok() function so that they do not point to the kernel space. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/assembler.h | 13 +++--- arch/arm/include/asm/domain.h | 31 +++++++++++++- arch/arm/include/asm/futex.h | 9 ++-- arch/arm/include/asm/uaccess.h | 16 ++++--- arch/arm/kernel/entry-armv.S | 4 +- arch/arm/kernel/traps.c | 17 ++++++++ arch/arm/lib/getuser.S | 13 +++--- arch/arm/lib/putuser.S | 29 +++++++------ arch/arm/lib/uaccess.S | 83 +++++++++++++++++++------------------- arch/arm/mm/Kconfig | 8 ++++ arch/arm/mm/proc-macros.S | 7 +++ arch/arm/mm/proc-v7.S | 5 +- 12 files changed, 150 insertions(+), 85 deletions(-) diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h index 6e8f05c..66db132 100644 --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -18,6 +18,7 @@ #endif #include <asm/ptrace.h> +#include <asm/domain.h> /* * Endian independent macros for shifting bytes within registers. @@ -183,12 +184,12 @@ */ #ifdef CONFIG_THUMB2_KERNEL - .macro usraccoff, instr, reg, ptr, inc, off, cond, abort + .macro usraccoff, instr, reg, ptr, inc, off, cond, abort, t=T() 9999: .if \inc == 1 - \instr\cond\()bt \reg, [\ptr, #\off] + \instr\cond\()b\()\t\().w \reg, [\ptr, #\off] .elseif \inc == 4 - \instr\cond\()t \reg, [\ptr, #\off] + \instr\cond\()\t\().w \reg, [\ptr, #\off] .else .error "Unsupported inc macro argument" .endif @@ -223,13 +224,13 @@ #else /* !CONFIG_THUMB2_KERNEL */ - .macro usracc, instr, reg, ptr, inc, cond, rept, abort + .macro usracc, instr, reg, ptr, inc, cond, rept, abort, t=T() .rept \rept 9999: .if \inc == 1 - \instr\cond\()bt \reg, [\ptr], #\inc + \instr\cond\()b\()\t \reg, [\ptr], #\inc .elseif \inc == 4 - \instr\cond\()t \reg, [\ptr], #\inc + \instr\cond\()\t \reg, [\ptr], #\inc .else .error "Unsupported inc macro argument" .endif diff --git a/arch/arm/include/asm/domain.h b/arch/arm/include/asm/domain.h index cc7ef40..af18cea 100644 --- a/arch/arm/include/asm/domain.h +++ b/arch/arm/include/asm/domain.h @@ -45,13 +45,17 @@ */ #define DOMAIN_NOACCESS 0 #define DOMAIN_CLIENT 1 +#ifdef CONFIG_CPU_USE_DOMAINS #define DOMAIN_MANAGER 3 +#else +#define DOMAIN_MANAGER 1 +#endif #define domain_val(dom,type) ((type) << (2*(dom))) #ifndef __ASSEMBLY__ -#ifdef CONFIG_MMU +#ifdef CONFIG_CPU_USE_DOMAINS #define set_domain(x) \ do { \ __asm__ __volatile__( \ @@ -74,5 +78,28 @@ #define modify_domain(dom,type) do { } while (0) #endif +/* + * Generate the T (user) versions of the LDR/STR and related + * instructions (inline assembly) + */ +#ifdef CONFIG_CPU_USE_DOMAINS +#define T(instr) #instr "t" +#else +#define T(instr) #instr #endif -#endif /* !__ASSEMBLY__ */ + +#else /* __ASSEMBLY__ */ + +/* + * Generate the T (user) versions of the LDR/STR and related + * instructions + */ +#ifdef CONFIG_CPU_USE_DOMAINS +#define T(instr) instr ## t +#else +#define T(instr) instr +#endif + +#endif /* __ASSEMBLY__ */ + +#endif /* !__ASM_PROC_DOMAIN_H */ diff --git a/arch/arm/include/asm/futex.h b/arch/arm/include/asm/futex.h index 540a044..b33fe70 100644 --- a/arch/arm/include/asm/futex.h +++ b/arch/arm/include/asm/futex.h @@ -13,12 +13,13 @@ #include <linux/preempt.h> #include <linux/uaccess.h> #include <asm/errno.h> +#include <asm/domain.h> #define __futex_atomic_op(insn, ret, oldval, uaddr, oparg) \ __asm__ __volatile__( \ - "1: ldrt %1, [%2]\n" \ + "1: " T(ldr) " %1, [%2]\n" \ " " insn "\n" \ - "2: strt %0, [%2]\n" \ + "2: " T(str) " %0, [%2]\n" \ " mov %0, #0\n" \ "3:\n" \ " .pushsection __ex_table,\"a\"\n" \ @@ -97,10 +98,10 @@ futex_atomic_cmpxchg_inatomic(int __user *uaddr, int oldval, int newval) pagefault_disable(); /* implies preempt_disable() */ __asm__ __volatile__("@futex_atomic_cmpxchg_inatomic\n" - "1: ldrt %0, [%3]\n" + "1: " T(ldr) " %0, [%3]\n" " teq %0, %1\n" " it eq @ explicit IT needed for the 2b label\n" - "2: streqt %2, [%3]\n" + "2: " T(streq) " %2, [%3]\n" "3:\n" " .pushsection __ex_table,\"a\"\n" " .align 3\n" diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h index 33e4a48..b293616 100644 --- a/arch/arm/include/asm/uaccess.h +++ b/arch/arm/include/asm/uaccess.h @@ -227,7 +227,7 @@ do { \ #define __get_user_asm_byte(x,addr,err) \ __asm__ __volatile__( \ - "1: ldrbt %1,[%2]\n" \ + "1: " T(ldrb) " %1,[%2],#0\n" \ "2:\n" \ " .pushsection .fixup,\"ax\"\n" \ " .align 2\n" \ @@ -263,7 +263,7 @@ do { \ #define __get_user_asm_word(x,addr,err) \ __asm__ __volatile__( \ - "1: ldrt %1,[%2]\n" \ + "1: " T(ldr) " %1,[%2],#0\n" \ "2:\n" \ " .pushsection .fixup,\"ax\"\n" \ " .align 2\n" \ @@ -308,7 +308,7 @@ do { \ #define __put_user_asm_byte(x,__pu_addr,err) \ __asm__ __volatile__( \ - "1: strbt %1,[%2]\n" \ + "1: " T(strb) " %1,[%2],#0\n" \ "2:\n" \ " .pushsection .fixup,\"ax\"\n" \ " .align 2\n" \ @@ -341,7 +341,7 @@ do { \ #define __put_user_asm_word(x,__pu_addr,err) \ __asm__ __volatile__( \ - "1: strt %1,[%2]\n" \ + "1: " T(str) " %1,[%2],#0\n" \ "2:\n" \ " .pushsection .fixup,\"ax\"\n" \ " .align 2\n" \ @@ -366,10 +366,10 @@ do { \ #define __put_user_asm_dword(x,__pu_addr,err) \ __asm__ __volatile__( \ - ARM( "1: strt " __reg_oper1 ", [%1], #4\n" ) \ - ARM( "2: strt " __reg_oper0 ", [%1]\n" ) \ - THUMB( "1: strt " __reg_oper1 ", [%1]\n" ) \ - THUMB( "2: strt " __reg_oper0 ", [%1, #4]\n" ) \ + ARM( "1: " T(str) " " __reg_oper1 ", [%1], #4\n" ) \ + ARM( "2: " T(str) " " __reg_oper0 ", [%1]\n" ) \ + THUMB( "1: " T(str) " " __reg_oper1 ", [%1]\n" ) \ + THUMB( "2: " T(str) " " __reg_oper0 ", [%1, #4]\n" ) \ "3:\n" \ " .pushsection .fixup,\"ax\"\n" \ " .align 2\n" \ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 7ee48e7..ba654fa 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -736,7 +736,7 @@ ENTRY(__switch_to) THUMB( stmia ip!, {r4 - sl, fp} ) @ Store most regs on stack THUMB( str sp, [ip], #4 ) THUMB( str lr, [ip], #4 ) -#ifdef CONFIG_MMU +#ifdef CONFIG_CPU_USE_DOMAINS ldr r6, [r2, #TI_CPU_DOMAIN] #endif #if defined(CONFIG_HAS_TLS_REG) @@ -745,7 +745,7 @@ ENTRY(__switch_to) mov r4, #0xffff0fff str r3, [r4, #-15] @ TLS val at 0xffff0ff0 #endif -#ifdef CONFIG_MMU +#ifdef CONFIG_CPU_USE_DOMAINS mcr p15, 0, r6, c3, c0, 0 @ Set domain register #endif mov r5, r0 diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 1621e53..6571e19 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -30,6 +30,7 @@ #include <asm/unistd.h> #include <asm/traps.h> #include <asm/unwind.h> +#include <asm/tlbflush.h> #include "ptrace.h" #include "signal.h" @@ -750,6 +751,16 @@ void __init early_trap_init(void) extern char __vectors_start[], __vectors_end[]; extern char __kuser_helper_start[], __kuser_helper_end[]; int kuser_sz = __kuser_helper_end - __kuser_helper_start; +#if !defined(CONFIG_CPU_USE_DOMAINS) && defined(CONFIG_MMU) + pgd_t *pgd = pgd_offset_k(vectors); + pmd_t *pmd = pmd_offset(pgd, vectors); + pte_t *pte = pte_offset_kernel(pmd, vectors); + pte_t entry = *pte; + + /* allow writing to the vectors page */ + set_pte_ext(pte, pte_mkwrite(entry), 0); + local_flush_tlb_kernel_page(vectors); +#endif /* * Copy the vectors, stubs and kuser helpers (in entry-armv.S) @@ -769,6 +780,12 @@ void __init early_trap_init(void) memcpy((void *)KERN_RESTART_CODE, syscall_restart_code, sizeof(syscall_restart_code)); +#if !defined(CONFIG_CPU_USE_DOMAINS) && defined(CONFIG_MMU) + /* restore the vectors page permissions */ + set_pte_ext(pte, entry, 0); + local_flush_tlb_kernel_page(vectors); +#endif + flush_icache_range(vectors, vectors + PAGE_SIZE); modify_domain(DOMAIN_USER, DOMAIN_CLIENT); } diff --git a/arch/arm/lib/getuser.S b/arch/arm/lib/getuser.S index b1631a7..1b049cd 100644 --- a/arch/arm/lib/getuser.S +++ b/arch/arm/lib/getuser.S @@ -28,20 +28,21 @@ */ #include <linux/linkage.h> #include <asm/errno.h> +#include <asm/domain.h> ENTRY(__get_user_1) -1: ldrbt r2, [r0] +1: T(ldrb) r2, [r0] mov r0, #0 mov pc, lr ENDPROC(__get_user_1) ENTRY(__get_user_2) #ifdef CONFIG_THUMB2_KERNEL -2: ldrbt r2, [r0] -3: ldrbt r3, [r0, #1] +2: T(ldrb) r2, [r0] +3: T(ldrb) r3, [r0, #1] #else -2: ldrbt r2, [r0], #1 -3: ldrbt r3, [r0] +2: T(ldrb) r2, [r0], #1 +3: T(ldrb) r3, [r0] #endif #ifndef __ARMEB__ orr r2, r2, r3, lsl #8 @@ -53,7 +54,7 @@ ENTRY(__get_user_2) ENDPROC(__get_user_2) ENTRY(__get_user_4) -4: ldrt r2, [r0] +4: T(ldr) r2, [r0] mov r0, #0 mov pc, lr ENDPROC(__get_user_4) diff --git a/arch/arm/lib/putuser.S b/arch/arm/lib/putuser.S index 5a01a23..c023fc1 100644 --- a/arch/arm/lib/putuser.S +++ b/arch/arm/lib/putuser.S @@ -28,9 +28,10 @@ */ #include <linux/linkage.h> #include <asm/errno.h> +#include <asm/domain.h> ENTRY(__put_user_1) -1: strbt r2, [r0] +1: T(strb) r2, [r0] mov r0, #0 mov pc, lr ENDPROC(__put_user_1) @@ -39,19 +40,19 @@ ENTRY(__put_user_2) mov ip, r2, lsr #8 #ifdef CONFIG_THUMB2_KERNEL #ifndef __ARMEB__ -2: strbt r2, [r0] -3: strbt ip, [r0, #1] +2: T(strb) r2, [r0] +3: T(strb) ip, [r0, #1] #else -2: strbt ip, [r0] -3: strbt r2, [r0, #1] +2: T(strb) ip, [r0] +3: T(strb) r2, [r0, #1] #endif #else /* !CONFIG_THUMB2_KERNEL */ #ifndef __ARMEB__ -2: strbt r2, [r0], #1 -3: strbt ip, [r0] +2: T(strb) r2, [r0], #1 +3: T(strb) ip, [r0] #else -2: strbt ip, [r0], #1 -3: strbt r2, [r0] +2: T(strb) ip, [r0], #1 +3: T(strb) r2, [r0] #endif #endif /* CONFIG_THUMB2_KERNEL */ mov r0, #0 @@ -59,18 +60,18 @@ ENTRY(__put_user_2) ENDPROC(__put_user_2) ENTRY(__put_user_4) -4: strt r2, [r0] +4: T(str) r2, [r0] mov r0, #0 mov pc, lr ENDPROC(__put_user_4) ENTRY(__put_user_8) #ifdef CONFIG_THUMB2_KERNEL -5: strt r2, [r0] -6: strt r3, [r0, #4] +5: T(str) r2, [r0] +6: T(str) r3, [r0, #4] #else -5: strt r2, [r0], #4 -6: strt r3, [r0] +5: T(str) r2, [r0], #4 +6: T(str) r3, [r0] #endif mov r0, #0 mov pc, lr diff --git a/arch/arm/lib/uaccess.S b/arch/arm/lib/uaccess.S index fee9f6f..d0ece2a 100644 --- a/arch/arm/lib/uaccess.S +++ b/arch/arm/lib/uaccess.S @@ -14,6 +14,7 @@ #include <linux/linkage.h> #include <asm/assembler.h> #include <asm/errno.h> +#include <asm/domain.h> .text @@ -31,11 +32,11 @@ rsb ip, ip, #4 cmp ip, #2 ldrb r3, [r1], #1 -USER( strbt r3, [r0], #1) @ May fault +USER( T(strb) r3, [r0], #1) @ May fault ldrgeb r3, [r1], #1 -USER( strgebt r3, [r0], #1) @ May fault +USER( T(strgeb) r3, [r0], #1) @ May fault ldrgtb r3, [r1], #1 -USER( strgtbt r3, [r0], #1) @ May fault +USER( T(strgtb) r3, [r0], #1) @ May fault sub r2, r2, ip b .Lc2u_dest_aligned @@ -58,7 +59,7 @@ ENTRY(__copy_to_user) addmi ip, r2, #4 bmi .Lc2u_0nowords ldr r3, [r1], #4 -USER( strt r3, [r0], #4) @ May fault +USER( T(str) r3, [r0], #4) @ May fault mov ip, r0, lsl #32 - PAGE_SHIFT @ On each page, use a ld/st??t instruction rsb ip, ip, #0 movs ip, ip, lsr #32 - PAGE_SHIFT @@ -87,18 +88,18 @@ USER( strt r3, [r0], #4) @ May fault stmneia r0!, {r3 - r4} @ Shouldnt fault tst ip, #4 ldrne r3, [r1], #4 - strnet r3, [r0], #4 @ Shouldnt fault + T(strne) r3, [r0], #4 @ Shouldnt fault ands ip, ip, #3 beq .Lc2u_0fupi .Lc2u_0nowords: teq ip, #0 beq .Lc2u_finished .Lc2u_nowords: cmp ip, #2 ldrb r3, [r1], #1 -USER( strbt r3, [r0], #1) @ May fault +USER( T(strb) r3, [r0], #1) @ May fault ldrgeb r3, [r1], #1 -USER( strgebt r3, [r0], #1) @ May fault +USER( T(strgeb) r3, [r0], #1) @ May fault ldrgtb r3, [r1], #1 -USER( strgtbt r3, [r0], #1) @ May fault +USER( T(strgtb) r3, [r0], #1) @ May fault b .Lc2u_finished .Lc2u_not_enough: @@ -119,7 +120,7 @@ USER( strgtbt r3, [r0], #1) @ May fault mov r3, r7, pull #8 ldr r7, [r1], #4 orr r3, r3, r7, push #24 -USER( strt r3, [r0], #4) @ May fault +USER( T(str) r3, [r0], #4) @ May fault mov ip, r0, lsl #32 - PAGE_SHIFT rsb ip, ip, #0 movs ip, ip, lsr #32 - PAGE_SHIFT @@ -154,18 +155,18 @@ USER( strt r3, [r0], #4) @ May fault movne r3, r7, pull #8 ldrne r7, [r1], #4 orrne r3, r3, r7, push #24 - strnet r3, [r0], #4 @ Shouldnt fault + T(strne) r3, [r0], #4 @ Shouldnt fault ands ip, ip, #3 beq .Lc2u_1fupi .Lc2u_1nowords: mov r3, r7, get_byte_1 teq ip, #0 beq .Lc2u_finished cmp ip, #2 -USER( strbt r3, [r0], #1) @ May fault +USER( T(strb) r3, [r0], #1) @ May fault movge r3, r7, get_byte_2 -USER( strgebt r3, [r0], #1) @ May fault +USER( T(strgeb) r3, [r0], #1) @ May fault movgt r3, r7, get_byte_3 -USER( strgtbt r3, [r0], #1) @ May fault +USER( T(strgtb) r3, [r0], #1) @ May fault b .Lc2u_finished .Lc2u_2fupi: subs r2, r2, #4 @@ -174,7 +175,7 @@ USER( strgtbt r3, [r0], #1) @ May fault mov r3, r7, pull #16 ldr r7, [r1], #4 orr r3, r3, r7, push #16 -USER( strt r3, [r0], #4) @ May fault +USER( T(str) r3, [r0], #4) @ May fault mov ip, r0, lsl #32 - PAGE_SHIFT rsb ip, ip, #0 movs ip, ip, lsr #32 - PAGE_SHIFT @@ -209,18 +210,18 @@ USER( strt r3, [r0], #4) @ May fault movne r3, r7, pull #16 ldrne r7, [r1], #4 orrne r3, r3, r7, push #16 - strnet r3, [r0], #4 @ Shouldnt fault + T(strne) r3, [r0], #4 @ Shouldnt fault ands ip, ip, #3 beq .Lc2u_2fupi .Lc2u_2nowords: mov r3, r7, get_byte_2 teq ip, #0 beq .Lc2u_finished cmp ip, #2 -USER( strbt r3, [r0], #1) @ May fault +USER( T(strb) r3, [r0], #1) @ May fault movge r3, r7, get_byte_3 -USER( strgebt r3, [r0], #1) @ May fault +USER( T(strgeb) r3, [r0], #1) @ May fault ldrgtb r3, [r1], #0 -USER( strgtbt r3, [r0], #1) @ May fault +USER( T(strgtb) r3, [r0], #1) @ May fault b .Lc2u_finished .Lc2u_3fupi: subs r2, r2, #4 @@ -229,7 +230,7 @@ USER( strgtbt r3, [r0], #1) @ May fault mov r3, r7, pull #24 ldr r7, [r1], #4 orr r3, r3, r7, push #8 -USER( strt r3, [r0], #4) @ May fault +USER( T(str) r3, [r0], #4) @ May fault mov ip, r0, lsl #32 - PAGE_SHIFT rsb ip, ip, #0 movs ip, ip, lsr #32 - PAGE_SHIFT @@ -264,18 +265,18 @@ USER( strt r3, [r0], #4) @ May fault movne r3, r7, pull #24 ldrne r7, [r1], #4 orrne r3, r3, r7, push #8 - strnet r3, [r0], #4 @ Shouldnt fault + T(strne) r3, [r0], #4 @ Shouldnt fault ands ip, ip, #3 beq .Lc2u_3fupi .Lc2u_3nowords: mov r3, r7, get_byte_3 teq ip, #0 beq .Lc2u_finished cmp ip, #2 -USER( strbt r3, [r0], #1) @ May fault +USER( T(strb) r3, [r0], #1) @ May fault ldrgeb r3, [r1], #1 -USER( strgebt r3, [r0], #1) @ May fault +USER( T(strgeb) r3, [r0], #1) @ May fault ldrgtb r3, [r1], #0 -USER( strgtbt r3, [r0], #1) @ May fault +USER( T(strgtb) r3, [r0], #1) @ May fault b .Lc2u_finished ENDPROC(__copy_to_user) @@ -294,11 +295,11 @@ ENDPROC(__copy_to_user) .Lcfu_dest_not_aligned: rsb ip, ip, #4 cmp ip, #2 -USER( ldrbt r3, [r1], #1) @ May fault +USER( T(ldrb) r3, [r1], #1) @ May fault strb r3, [r0], #1 -USER( ldrgebt r3, [r1], #1) @ May fault +USER( T(ldrgeb) r3, [r1], #1) @ May fault strgeb r3, [r0], #1 -USER( ldrgtbt r3, [r1], #1) @ May fault +USER( T(ldrgtb) r3, [r1], #1) @ May fault strgtb r3, [r0], #1 sub r2, r2, ip b .Lcfu_dest_aligned @@ -321,7 +322,7 @@ ENTRY(__copy_from_user) .Lcfu_0fupi: subs r2, r2, #4 addmi ip, r2, #4 bmi .Lcfu_0nowords -USER( ldrt r3, [r1], #4) +USER( T(ldr) r3, [r1], #4) str r3, [r0], #4 mov ip, r1, lsl #32 - PAGE_SHIFT @ On each page, use a ld/st??t instruction rsb ip, ip, #0 @@ -350,18 +351,18 @@ USER( ldrt r3, [r1], #4) ldmneia r1!, {r3 - r4} @ Shouldnt fault stmneia r0!, {r3 - r4} tst ip, #4 - ldrnet r3, [r1], #4 @ Shouldnt fault + T(ldrne) r3, [r1], #4 @ Shouldnt fault strne r3, [r0], #4 ands ip, ip, #3 beq .Lcfu_0fupi .Lcfu_0nowords: teq ip, #0 beq .Lcfu_finished .Lcfu_nowords: cmp ip, #2 -USER( ldrbt r3, [r1], #1) @ May fault +USER( T(ldrb) r3, [r1], #1) @ May fault strb r3, [r0], #1 -USER( ldrgebt r3, [r1], #1) @ May fault +USER( T(ldrgeb) r3, [r1], #1) @ May fault strgeb r3, [r0], #1 -USER( ldrgtbt r3, [r1], #1) @ May fault +USER( T(ldrgtb) r3, [r1], #1) @ May fault strgtb r3, [r0], #1 b .Lcfu_finished @@ -374,7 +375,7 @@ USER( ldrgtbt r3, [r1], #1) @ May fault .Lcfu_src_not_aligned: bic r1, r1, #3 -USER( ldrt r7, [r1], #4) @ May fault +USER( T(ldr) r7, [r1], #4) @ May fault cmp ip, #2 bgt .Lcfu_3fupi beq .Lcfu_2fupi @@ -382,7 +383,7 @@ USER( ldrt r7, [r1], #4) @ May fault addmi ip, r2, #4 bmi .Lcfu_1nowords mov r3, r7, pull #8 -USER( ldrt r7, [r1], #4) @ May fault +USER( T(ldr) r7, [r1], #4) @ May fault orr r3, r3, r7, push #24 str r3, [r0], #4 mov ip, r1, lsl #32 - PAGE_SHIFT @@ -417,7 +418,7 @@ USER( ldrt r7, [r1], #4) @ May fault stmneia r0!, {r3 - r4} tst ip, #4 movne r3, r7, pull #8 -USER( ldrnet r7, [r1], #4) @ May fault +USER( T(ldrne) r7, [r1], #4) @ May fault orrne r3, r3, r7, push #24 strne r3, [r0], #4 ands ip, ip, #3 @@ -437,7 +438,7 @@ USER( ldrnet r7, [r1], #4) @ May fault addmi ip, r2, #4 bmi .Lcfu_2nowords mov r3, r7, pull #16 -USER( ldrt r7, [r1], #4) @ May fault +USER( T(ldr) r7, [r1], #4) @ May fault orr r3, r3, r7, push #16 str r3, [r0], #4 mov ip, r1, lsl #32 - PAGE_SHIFT @@ -473,7 +474,7 @@ USER( ldrt r7, [r1], #4) @ May fault stmneia r0!, {r3 - r4} tst ip, #4 movne r3, r7, pull #16 -USER( ldrnet r7, [r1], #4) @ May fault +USER( T(ldrne) r7, [r1], #4) @ May fault orrne r3, r3, r7, push #16 strne r3, [r0], #4 ands ip, ip, #3 @@ -485,7 +486,7 @@ USER( ldrnet r7, [r1], #4) @ May fault strb r3, [r0], #1 movge r3, r7, get_byte_3 strgeb r3, [r0], #1 -USER( ldrgtbt r3, [r1], #0) @ May fault +USER( T(ldrgtb) r3, [r1], #0) @ May fault strgtb r3, [r0], #1 b .Lcfu_finished @@ -493,7 +494,7 @@ USER( ldrgtbt r3, [r1], #0) @ May fault addmi ip, r2, #4 bmi .Lcfu_3nowords mov r3, r7, pull #24 -USER( ldrt r7, [r1], #4) @ May fault +USER( T(ldr) r7, [r1], #4) @ May fault orr r3, r3, r7, push #8 str r3, [r0], #4 mov ip, r1, lsl #32 - PAGE_SHIFT @@ -528,7 +529,7 @@ USER( ldrt r7, [r1], #4) @ May fault stmneia r0!, {r3 - r4} tst ip, #4 movne r3, r7, pull #24 -USER( ldrnet r7, [r1], #4) @ May fault +USER( T(ldrne) r7, [r1], #4) @ May fault orrne r3, r3, r7, push #8 strne r3, [r0], #4 ands ip, ip, #3 @@ -538,9 +539,9 @@ USER( ldrnet r7, [r1], #4) @ May fault beq .Lcfu_finished cmp ip, #2 strb r3, [r0], #1 -USER( ldrgebt r3, [r1], #1) @ May fault +USER( T(ldrgeb) r3, [r1], #1) @ May fault strgeb r3, [r0], #1 -USER( ldrgtbt r3, [r1], #1) @ May fault +USER( T(ldrgtb) r3, [r1], #1) @ May fault strgtb r3, [r0], #1 b .Lcfu_finished ENDPROC(__copy_from_user) diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig index 346ae14..f33c422 100644 --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -599,6 +599,14 @@ config CPU_CP15_MPU help Processor has the CP15 register, which has MPU related registers. +config CPU_USE_DOMAINS + bool + depends on MMU + default y if !HAS_TLS_REG + help + This option enables or disables the use of domain switching + via the set_fs() function. + # # CPU supports 36-bit I/O # diff --git a/arch/arm/mm/proc-macros.S b/arch/arm/mm/proc-macros.S index 7d63bea..337f102 100644 --- a/arch/arm/mm/proc-macros.S +++ b/arch/arm/mm/proc-macros.S @@ -99,6 +99,10 @@ * 110x 0 1 0 r/w r/o * 11x0 0 1 0 r/w r/o * 1111 0 1 1 r/w r/w + * + * If !CONFIG_CPU_USE_DOMAINS, the following permissions are changed: + * 110x 1 1 1 r/o r/o + * 11x0 1 1 1 r/o r/o */ .macro armv6_mt_table pfx \pfx\()_mt_table: @@ -138,8 +142,11 @@ tst r1, #L_PTE_USER orrne r3, r3, #PTE_EXT_AP1 +#ifdef CONFIG_CPU_USE_DOMAINS + @ allow kernel read/write access to read-only user pages tstne r3, #PTE_EXT_APX bicne r3, r3, #PTE_EXT_APX | PTE_EXT_AP0 +#endif tst r1, #L_PTE_EXEC orreq r3, r3, #PTE_EXT_XN diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index 7aaf88a..c1c3fe0 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -152,8 +152,11 @@ ENTRY(cpu_v7_set_pte_ext) tst r1, #L_PTE_USER orrne r3, r3, #PTE_EXT_AP1 +#ifdef CONFIG_CPU_USE_DOMAINS + @ allow kernel read/write access to read-only user pages tstne r3, #PTE_EXT_APX bicne r3, r3, #PTE_EXT_APX | PTE_EXT_AP0 +#endif tst r1, #L_PTE_EXEC orreq r3, r3, #PTE_EXT_XN @@ -240,8 +243,6 @@ __v7_setup: mcr p15, 0, r10, c2, c0, 2 @ TTB control register orr r4, r4, #TTB_FLAGS mcr p15, 0, r4, c2, c0, 1 @ load TTB1 - mov r10, #0x1f @ domains 0, 1 = manager - mcr p15, 0, r10, c3, c0, 0 @ load domain access register /* * Memory region attributes with SCTLR.TRE=1 * ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs 2010-05-20 14:30 ` [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs Catalin Marinas @ 2010-05-24 11:45 ` Russell King - ARM Linux 2010-05-24 13:25 ` Catalin Marinas 0 siblings, 1 reply; 23+ messages in thread From: Russell King - ARM Linux @ 2010-05-24 11:45 UTC (permalink / raw) To: linux-arm-kernel On Thu, May 20, 2010 at 03:30:04PM +0100, Catalin Marinas wrote: > The user pages access rights are also modified for kernel read-only > access rather than read/write so that the copy-on-write mechanism still > works. CPU_USE_DOMAINS gets disabled only if HAS_TLS_REG is defined > since writing the TLS value to the high vectors page isn't possible. When FIQ handlers are installed/removed, the vector page is also written, so the above restriction is incompatible with FIQ support. Do we want to tell people that they can't use FIQs without domain support? ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs 2010-05-24 11:45 ` Russell King - ARM Linux @ 2010-05-24 13:25 ` Catalin Marinas 2010-05-24 13:52 ` Catalin Marinas 0 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-24 13:25 UTC (permalink / raw) To: linux-arm-kernel On Mon, 2010-05-24 at 12:45 +0100, Russell King - ARM Linux wrote: > On Thu, May 20, 2010 at 03:30:04PM +0100, Catalin Marinas wrote: > > The user pages access rights are also modified for kernel read-only > > access rather than read/write so that the copy-on-write mechanism still > > works. CPU_USE_DOMAINS gets disabled only if HAS_TLS_REG is defined > > since writing the TLS value to the high vectors page isn't possible. > > When FIQ handlers are installed/removed, the vector page is also written, > so the above restriction is incompatible with FIQ support. > > Do we want to tell people that they can't use FIQs without domain support? They could still be used (with an additional patch). At first, I thought that we could do something similar to early_trap_init() but set_fiq_handler() could be called from a module at run-time and it may temporarily enable write access to the vectors page (without an additional L_PTE_ bit we can't make the vectors page user RO, kernel R/W and I wouldn't do this anyway, for other reasons). What about making a global vectors_page variable (initialised in devicemaps_init) that can be written directly when !CPU_USE_DOMAINS? On such CPUs, we have a non-aliasing VIPT cache anyway. -- Catalin ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs 2010-05-24 13:25 ` Catalin Marinas @ 2010-05-24 13:52 ` Catalin Marinas 2010-05-24 13:59 ` Russell King - ARM Linux 0 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-24 13:52 UTC (permalink / raw) To: linux-arm-kernel On Mon, 2010-05-24 at 14:25 +0100, Catalin Marinas wrote: > On Mon, 2010-05-24 at 12:45 +0100, Russell King - ARM Linux wrote: > > On Thu, May 20, 2010 at 03:30:04PM +0100, Catalin Marinas wrote: > > > The user pages access rights are also modified for kernel read-only > > > access rather than read/write so that the copy-on-write mechanism still > > > works. CPU_USE_DOMAINS gets disabled only if HAS_TLS_REG is defined > > > since writing the TLS value to the high vectors page isn't possible. > > > > When FIQ handlers are installed/removed, the vector page is also written, > > so the above restriction is incompatible with FIQ support. > > > > Do we want to tell people that they can't use FIQs without domain support? > > They could still be used (with an additional patch). At first, I thought > that we could do something similar to early_trap_init() but > set_fiq_handler() could be called from a module at run-time and it may > temporarily enable write access to the vectors page (without an > additional L_PTE_ bit we can't make the vectors page user RO, kernel R/W > and I wouldn't do this anyway, for other reasons). > > What about making a global vectors_page variable (initialised in > devicemaps_init) that can be written directly when !CPU_USE_DOMAINS? On > such CPUs, we have a non-aliasing VIPT cache anyway. That's the additional diff which allows FIQs when !CPU_USE_DOMAINS (assuming VIPT non-aliasing caches). I also modified the early_trap_init() to make use of the vectors page rather than setting the PTE as in the original patch. diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index 491960b..af5d5d1 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -27,4 +27,6 @@ static inline int in_exception_text(unsigned long ptr) extern void __init early_trap_init(void); extern void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long frame); +extern void *vectors_page; + #endif diff --git a/arch/arm/kernel/fiq.c b/arch/arm/kernel/fiq.c index 6ff7919..fb1a640 100644 --- a/arch/arm/kernel/fiq.c +++ b/arch/arm/kernel/fiq.c @@ -45,6 +45,7 @@ #include <asm/fiq.h> #include <asm/irq.h> #include <asm/system.h> +#include <asm/traps.h> static unsigned long no_fiq_insn; @@ -77,7 +78,11 @@ int show_fiq_list(struct seq_file *p, void *v) void set_fiq_handler(void *start, unsigned int length) { +#if defined(CONFIG_CPU_USE_DOMAINS) memcpy((void *)0xffff001c, start, length); +#else + memcpy(vectors_page + 0x1c, start, length); +#endif flush_icache_range(0xffff001c, 0xffff001c + length); if (!vectors_high()) flush_icache_range(0x1c, 0x1c + length); diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 6571e19..6bc57c5 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -30,13 +30,14 @@ #include <asm/unistd.h> #include <asm/traps.h> #include <asm/unwind.h> -#include <asm/tlbflush.h> #include "ptrace.h" #include "signal.h" static const char *handler[]= { "prefetch abort", "data abort", "address exception", "interrupt" }; +void *vectors_page; + #ifdef CONFIG_DEBUG_USER unsigned int user_debug; @@ -746,21 +747,15 @@ void __init trap_init(void) void __init early_trap_init(void) { +#if defined(CONFIG_CPU_USE_DOMAINS) unsigned long vectors = CONFIG_VECTORS_BASE; +#else + unsigned long vectors = (unsigned long)vectors_page; +#endif extern char __stubs_start[], __stubs_end[]; extern char __vectors_start[], __vectors_end[]; extern char __kuser_helper_start[], __kuser_helper_end[]; int kuser_sz = __kuser_helper_end - __kuser_helper_start; -#if !defined(CONFIG_CPU_USE_DOMAINS) && defined(CONFIG_MMU) - pgd_t *pgd = pgd_offset_k(vectors); - pmd_t *pmd = pmd_offset(pgd, vectors); - pte_t *pte = pte_offset_kernel(pmd, vectors); - pte_t entry = *pte; - - /* allow writing to the vectors page */ - set_pte_ext(pte, pte_mkwrite(entry), 0); - local_flush_tlb_kernel_page(vectors); -#endif /* * Copy the vectors, stubs and kuser helpers (in entry-armv.S) @@ -775,16 +770,10 @@ void __init early_trap_init(void) * Copy signal return handlers into the vector page, and * set sigreturn to be a pointer to these. */ - memcpy((void *)KERN_SIGRETURN_CODE, sigreturn_codes, - sizeof(sigreturn_codes)); - memcpy((void *)KERN_RESTART_CODE, syscall_restart_code, - sizeof(syscall_restart_code)); - -#if !defined(CONFIG_CPU_USE_DOMAINS) && defined(CONFIG_MMU) - /* restore the vectors page permissions */ - set_pte_ext(pte, entry, 0); - local_flush_tlb_kernel_page(vectors); -#endif + memcpy((void *)(vectors + KERN_SIGRETURN_CODE - CONFIG_VECTORS_BASE), + sigreturn_codes, sizeof(sigreturn_codes)); + memcpy((void *)(vectors + KERN_RESTART_CODE - CONFIG_VECTORS_BASE), + syscall_restart_code, sizeof(syscall_restart_code)); flush_icache_range(vectors, vectors + PAGE_SIZE); modify_domain(DOMAIN_USER, DOMAIN_CLIENT); diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c index 2858941..499e22d 100644 --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -25,6 +25,7 @@ #include <asm/smp_plat.h> #include <asm/tlb.h> #include <asm/highmem.h> +#include <asm/traps.h> #include <asm/mach/arch.h> #include <asm/mach/map.h> @@ -935,12 +936,11 @@ static void __init devicemaps_init(struct machine_desc *mdesc) { struct map_desc map; unsigned long addr; - void *vectors; /* * Allocate the vector page early. */ - vectors = alloc_bootmem_low_pages(PAGE_SIZE); + vectors_page = alloc_bootmem_low_pages(PAGE_SIZE); for (addr = VMALLOC_END; addr; addr += PGDIR_SIZE) pmd_clear(pmd_off_k(addr)); @@ -980,7 +980,7 @@ static void __init devicemaps_init(struct machine_desc *mdesc) * location (0xffff0000). If we aren't using high-vectors, also * create a mapping at the low-vectors virtual address. */ - map.pfn = __phys_to_pfn(virt_to_phys(vectors)); + map.pfn = __phys_to_pfn(virt_to_phys(vectors_page)); map.virtual = 0xffff0000; map.length = PAGE_SIZE; map.type = MT_HIGH_VECTORS; -- Catalin ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs 2010-05-24 13:52 ` Catalin Marinas @ 2010-05-24 13:59 ` Russell King - ARM Linux 2010-05-24 14:02 ` Catalin Marinas 0 siblings, 1 reply; 23+ messages in thread From: Russell King - ARM Linux @ 2010-05-24 13:59 UTC (permalink / raw) To: linux-arm-kernel On Mon, May 24, 2010 at 02:52:52PM +0100, Catalin Marinas wrote: > @@ -77,7 +78,11 @@ int show_fiq_list(struct seq_file *p, void *v) > > void set_fiq_handler(void *start, unsigned int length) > { > +#if defined(CONFIG_CPU_USE_DOMAINS) > memcpy((void *)0xffff001c, start, length); > +#else > + memcpy(vectors_page + 0x1c, start, length); > +#endif > flush_icache_range(0xffff001c, 0xffff001c + length); This is clearly buggy from an I/D coherency point of view. ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs 2010-05-24 13:59 ` Russell King - ARM Linux @ 2010-05-24 14:02 ` Catalin Marinas 0 siblings, 0 replies; 23+ messages in thread From: Catalin Marinas @ 2010-05-24 14:02 UTC (permalink / raw) To: linux-arm-kernel On Mon, 2010-05-24 at 14:59 +0100, Russell King - ARM Linux wrote: > On Mon, May 24, 2010 at 02:52:52PM +0100, Catalin Marinas wrote: > > @@ -77,7 +78,11 @@ int show_fiq_list(struct seq_file *p, void *v) > > > > void set_fiq_handler(void *start, unsigned int length) > > { > > +#if defined(CONFIG_CPU_USE_DOMAINS) > > memcpy((void *)0xffff001c, start, length); > > +#else > > + memcpy(vectors_page + 0x1c, start, length); > > +#endif > > flush_icache_range(0xffff001c, 0xffff001c + length); > > This is clearly buggy from an I/D coherency point of view. As I said, it only works on VIPT non-aliasing caches, which would be fine give the CPUs that this patch targets. If you think it's safer, we can restrict this patch for ARMv7 onwards. -- Catalin ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 2/4] ARM: Use lazy cache flushing on ARMv7 SMP systems 2010-05-20 14:29 [PATCH 0/4] Patches for -next Catalin Marinas 2010-05-20 14:30 ` [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs Catalin Marinas @ 2010-05-20 14:30 ` Catalin Marinas 2010-05-20 14:30 ` [PATCH 3/4] ARM: Assume new page cache pages have dirty D-cache Catalin Marinas ` (2 subsequent siblings) 4 siblings, 0 replies; 23+ messages in thread From: Catalin Marinas @ 2010-05-20 14:30 UTC (permalink / raw) To: linux-arm-kernel ARMv7 processors like Cortex-A9 broadcast the cache maintenance operations in hardware. This patch allows the flush_dcache_page/update_mmu_cache pair to work in lazy flushing mode similar to the UP case. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/smp_plat.h | 4 ++++ arch/arm/mm/fault-armv.c | 2 -- arch/arm/mm/flush.c | 13 ++++--------- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/arch/arm/include/asm/smp_plat.h b/arch/arm/include/asm/smp_plat.h index e621530..963a338 100644 --- a/arch/arm/include/asm/smp_plat.h +++ b/arch/arm/include/asm/smp_plat.h @@ -13,9 +13,13 @@ static inline int tlb_ops_need_broadcast(void) return ((read_cpuid_ext(CPUID_EXT_MMFR3) >> 12) & 0xf) < 2; } +#if !defined(CONFIG_SMP) || __LINUX_ARM_ARCH__ >= 7 +#define cache_ops_need_broadcast() 0 +#else static inline int cache_ops_need_broadcast(void) { return ((read_cpuid_ext(CPUID_EXT_MMFR3) >> 12) & 0xf) < 1; } +#endif #endif diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 9b906de..9d77d90 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -168,10 +168,8 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, return; mapping = page_mapping(page); -#ifndef CONFIG_SMP if (test_and_clear_bit(PG_dcache_dirty, &page->flags)) __flush_dcache_page(mapping, page); -#endif if (mapping) { if (cache_is_vivt()) make_coherent(mapping, vma, addr, ptep, pfn); diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c index c6844cb..5ad8711 100644 --- a/arch/arm/mm/flush.c +++ b/arch/arm/mm/flush.c @@ -17,6 +17,7 @@ #include <asm/smp_plat.h> #include <asm/system.h> #include <asm/tlbflush.h> +#include <asm/smp_plat.h> #include "mm.h" @@ -93,12 +94,10 @@ void flush_cache_page(struct vm_area_struct *vma, unsigned long user_addr, unsig #define flush_pfn_alias(pfn,vaddr) do { } while (0) #endif -#ifdef CONFIG_SMP static void flush_ptrace_access_other(void *args) { __flush_icache_all(); } -#endif static void flush_ptrace_access(struct vm_area_struct *vma, struct page *page, @@ -122,11 +121,9 @@ void flush_ptrace_access(struct vm_area_struct *vma, struct page *page, if (vma->vm_flags & VM_EXEC) { unsigned long addr = (unsigned long)kaddr; __cpuc_coherent_kern_range(addr, addr + len); -#ifdef CONFIG_SMP if (cache_ops_need_broadcast()) smp_call_function(flush_ptrace_access_other, NULL, 1); -#endif } } @@ -246,12 +243,10 @@ void flush_dcache_page(struct page *page) mapping = page_mapping(page); -#ifndef CONFIG_SMP - if (!PageHighMem(page) && mapping && !mapping_mapped(mapping)) + if (!cache_ops_need_broadcast() && + !PageHighMem(page) && mapping && !mapping_mapped(mapping)) set_bit(PG_dcache_dirty, &page->flags); - else -#endif - { + else { __flush_dcache_page(mapping, page); if (mapping && cache_is_vivt()) __flush_dcache_aliases(mapping, page); ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 3/4] ARM: Assume new page cache pages have dirty D-cache 2010-05-20 14:29 [PATCH 0/4] Patches for -next Catalin Marinas 2010-05-20 14:30 ` [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs Catalin Marinas 2010-05-20 14:30 ` [PATCH 2/4] ARM: Use lazy cache flushing on ARMv7 SMP systems Catalin Marinas @ 2010-05-20 14:30 ` Catalin Marinas 2010-05-20 14:30 ` [PATCH 4/4] ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP Catalin Marinas 2010-05-21 5:59 ` [PATCH 0/4] Patches for -next Shilimkar, Santosh 4 siblings, 0 replies; 23+ messages in thread From: Catalin Marinas @ 2010-05-20 14:30 UTC (permalink / raw) To: linux-arm-kernel There are places in Linux where writes to newly allocated page cache pages happen without a subsequent call to flush_dcache_page() (several PIO drivers including USB HCD). This patch changes the meaning of PG_arch_1 to be PG_dcache_clean and always flush the D-cache for a newly mapped page in update_mmu_cache(). The patch also sets the PG_arch_1 bit in the DMA cache maintenance function to avoid additional cache flushing in update_mmu_cache(). Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/cacheflush.h | 6 +++--- arch/arm/include/asm/tlbflush.h | 2 +- arch/arm/mm/copypage-v4mc.c | 2 +- arch/arm/mm/copypage-v6.c | 2 +- arch/arm/mm/copypage-xscale.c | 2 +- arch/arm/mm/dma-mapping.c | 6 ++++++ arch/arm/mm/fault-armv.c | 4 ++-- arch/arm/mm/flush.c | 2 +- 8 files changed, 16 insertions(+), 10 deletions(-) diff --git a/arch/arm/include/asm/cacheflush.h b/arch/arm/include/asm/cacheflush.h index 4656a24..d3730f0 100644 --- a/arch/arm/include/asm/cacheflush.h +++ b/arch/arm/include/asm/cacheflush.h @@ -137,10 +137,10 @@ #endif /* - * This flag is used to indicate that the page pointed to by a pte - * is dirty and requires cleaning before returning it to the user. + * This flag is used to indicate that the page pointed to by a pte is clean + * and does not require cleaning before returning it to the user. */ -#define PG_dcache_dirty PG_arch_1 +#define PG_dcache_clean PG_arch_1 /* * MM Cache Management diff --git a/arch/arm/include/asm/tlbflush.h b/arch/arm/include/asm/tlbflush.h index bd863d8..40a7092 100644 --- a/arch/arm/include/asm/tlbflush.h +++ b/arch/arm/include/asm/tlbflush.h @@ -552,7 +552,7 @@ extern void flush_tlb_kernel_range(unsigned long start, unsigned long end); #endif /* - * if PG_dcache_dirty is set for the page, we need to ensure that any + * If PG_dcache_clean is not set for the page, we need to ensure that any * cache entries for the kernels virtual memory range are written * back to the page. */ diff --git a/arch/arm/mm/copypage-v4mc.c b/arch/arm/mm/copypage-v4mc.c index 598c51a..b806151 100644 --- a/arch/arm/mm/copypage-v4mc.c +++ b/arch/arm/mm/copypage-v4mc.c @@ -73,7 +73,7 @@ void v4_mc_copy_user_highpage(struct page *to, struct page *from, { void *kto = kmap_atomic(to, KM_USER1); - if (test_and_clear_bit(PG_dcache_dirty, &from->flags)) + if (!test_and_set_bit(PG_dcache_clean, &from->flags)) __flush_dcache_page(page_mapping(from), from); spin_lock(&minicache_lock); diff --git a/arch/arm/mm/copypage-v6.c b/arch/arm/mm/copypage-v6.c index f55fa10..bdba6c6 100644 --- a/arch/arm/mm/copypage-v6.c +++ b/arch/arm/mm/copypage-v6.c @@ -79,7 +79,7 @@ static void v6_copy_user_highpage_aliasing(struct page *to, unsigned int offset = CACHE_COLOUR(vaddr); unsigned long kfrom, kto; - if (test_and_clear_bit(PG_dcache_dirty, &from->flags)) + if (!test_and_set_bit(PG_dcache_clean, &from->flags)) __flush_dcache_page(page_mapping(from), from); /* FIXME: not highmem safe */ diff --git a/arch/arm/mm/copypage-xscale.c b/arch/arm/mm/copypage-xscale.c index 9920c0a..649bbcd 100644 --- a/arch/arm/mm/copypage-xscale.c +++ b/arch/arm/mm/copypage-xscale.c @@ -95,7 +95,7 @@ void xscale_mc_copy_user_highpage(struct page *to, struct page *from, { void *kto = kmap_atomic(to, KM_USER1); - if (test_and_clear_bit(PG_dcache_dirty, &from->flags)) + if (!test_and_set_bit(PG_dcache_clean, &from->flags)) __flush_dcache_page(page_mapping(from), from); spin_lock(&minicache_lock); diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 13fa536..bb9b612 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -508,6 +508,12 @@ void ___dma_page_dev_to_cpu(struct page *page, unsigned long off, outer_inv_range(paddr, paddr + size); dma_cache_maint_page(page, off, size, dir, dmac_unmap_area); + + /* + * Mark the D-cache clean for this page to avoid extra flushing. + */ + if (dir != DMA_TO_DEVICE) + set_bit(PG_dcache_clean, &page->flags); } EXPORT_SYMBOL(___dma_page_dev_to_cpu); diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 9d77d90..91a691f 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -141,7 +141,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma, * a page table, or changing an existing PTE. Basically, there are two * things that we need to take care of: * - * 1. If PG_dcache_dirty is set for the page, we need to ensure + * 1. If PG_dcache_clean is not set for the page, we need to ensure * that any cache entries for the kernels virtual memory * range are written back to the page. * 2. If we have multiple shared mappings of the same space in @@ -168,7 +168,7 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, return; mapping = page_mapping(page); - if (test_and_clear_bit(PG_dcache_dirty, &page->flags)) + if (!test_and_set_bit(PG_dcache_clean, &page->flags)) __flush_dcache_page(mapping, page); if (mapping) { if (cache_is_vivt()) diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c index 5ad8711..5abab9a 100644 --- a/arch/arm/mm/flush.c +++ b/arch/arm/mm/flush.c @@ -245,7 +245,7 @@ void flush_dcache_page(struct page *page) if (!cache_ops_need_broadcast() && !PageHighMem(page) && mapping && !mapping_mapped(mapping)) - set_bit(PG_dcache_dirty, &page->flags); + clear_bit(PG_dcache_clean, &page->flags); else { __flush_dcache_page(mapping, page); if (mapping && cache_is_vivt()) ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 4/4] ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP 2010-05-20 14:29 [PATCH 0/4] Patches for -next Catalin Marinas ` (2 preceding siblings ...) 2010-05-20 14:30 ` [PATCH 3/4] ARM: Assume new page cache pages have dirty D-cache Catalin Marinas @ 2010-05-20 14:30 ` Catalin Marinas 2010-05-21 5:59 ` [PATCH 0/4] Patches for -next Shilimkar, Santosh 4 siblings, 0 replies; 23+ messages in thread From: Catalin Marinas @ 2010-05-20 14:30 UTC (permalink / raw) To: linux-arm-kernel On SMP systems, there is a small chance of a PTE becoming visible to a different CPU before the cache maintenance operations in update_mmu_cache(). This patch clears the L_PTE_EXEC bit in set_pte_at() but sets it later in update_mmu_cache() if vm_flags & VM_EXEC. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/pgtable.h | 12 ++++++++++++ arch/arm/mm/fault-armv.c | 17 ++++++++++++----- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index ab68cf1..c50691f 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -278,9 +278,21 @@ extern struct page *empty_zero_page; #define set_pte_ext(ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext) +#ifndef CONFIG_SMP #define set_pte_at(mm,addr,ptep,pteval) do { \ set_pte_ext(ptep, pteval, (addr) >= TASK_SIZE ? 0 : PTE_EXT_NG); \ } while (0) +#else +/* + * The L_PTE_EXEC attribute is later be set in update_mmu_cache() to avoid a + * race with SMP systems executing from the new mapping before the cache + * flushing took place. + */ +#define set_pte_at(mm,addr,ptep,pteval) do { \ + set_pte_ext(ptep, __pte(pte_val(pteval) & ~L_PTE_EXEC), \ + (addr) >= TASK_SIZE ? 0 : PTE_EXT_NG); \ + } while (0) +#endif /* * The following only work if pte_present() is true. diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 91a691f..f2b2fa4 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -170,11 +170,18 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, mapping = page_mapping(page); if (!test_and_set_bit(PG_dcache_clean, &page->flags)) __flush_dcache_page(mapping, page); - if (mapping) { - if (cache_is_vivt()) - make_coherent(mapping, vma, addr, ptep, pfn); - else if (vma->vm_flags & VM_EXEC) - __flush_icache_all(); + if (!mapping) + return; + + if (cache_is_vivt()) + make_coherent(mapping, vma, addr, ptep, pfn); + else if (vma->vm_flags & VM_EXEC) { + __flush_icache_all(); +#ifdef CONFIG_SMP + set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), + addr >= TASK_SIZE ? 0 : PTE_EXT_NG); + flush_tlb_page(vma, addr); +#endif } } ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-20 14:29 [PATCH 0/4] Patches for -next Catalin Marinas ` (3 preceding siblings ...) 2010-05-20 14:30 ` [PATCH 4/4] ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP Catalin Marinas @ 2010-05-21 5:59 ` Shilimkar, Santosh 2010-05-21 6:06 ` Shilimkar, Santosh 2010-05-21 9:52 ` Catalin Marinas 4 siblings, 2 replies; 23+ messages in thread From: Shilimkar, Santosh @ 2010-05-21 5:59 UTC (permalink / raw) To: linux-arm-kernel Catalin, > -----Original Message----- > From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel- > bounces at lists.infradead.org] On Behalf Of Catalin Marinas > Sent: Thursday, May 20, 2010 8:00 PM > To: linux-arm-kernel at lists.infradead.org > Subject: [PATCH 0/4] Patches for -next > > Hi, > > I would like to get these patches in the linux-next tree for this > release cycle and aim to submit them for upstream 2.6.36-rc1 (next > merging window). > > Please let me know if you have any objections. > > Thanks. > > > Catalin Marinas (4): > ARM: Remove the domain switching on ARMv6k/v7 CPUs > ARM: Use lazy cache flushing on ARMv7 SMP systems > ARM: Assume new page cache pages have dirty D-cache > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > I just gave quick try with these patches on OMAP4 yesterday. "ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP" patch is creating regression when loading the module. Didn't get time to debug it but though if reporting it. ****************************************************************************** # insmod test_module.ko Unable to handle kernel paging request at virtual address bf005da8 pgd = dfe20000 [bf005da8] *pgd=9fede011, *pte=9fedf4df, *ppte=9fedf45f Internal error: Oops: 8000000f [#1] PREEMPT SMP last sysfs file: Modules linked in: g_zero(+) CPU: 1 Not tainted (2.6.34-00074-g8e3b656 #1) PC is at init+0x0/0x1c [g_zero] LR is at do_one_initcall+0x64/0x1bc pc : [<bf005da8>] lr : [<c002a51c>] psr: 60000013 sp : dfe9ff50 ip : c0356060 fp : dfe9ff7c r10: 00000000 r9 : dfe9e000 r8 : c002b2e8 r7 : 00000000 r6 : bf005da8 r5 : bf002304 r4 : 00000000 r3 : 00000000 r2 : dfe9e000 r1 : 00000000 r0 : bf005da8 Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user Control: 10c53c7d Table: 9fe2004a DAC: 00000015 Process insmod (pid: 585, stack limit = 0xdfe9e2f8) Stack: (0xdfe9ff50 to 0xdfea0000) ff40: 40212008 c002b2e8 dfe9ff7c dfe9ff68 ff60: 00000000 bf002304 00092008 40212008 dfe9ffa4 dfe9ff80 c007a72c c002a4c4 ff80: 00000003 40212000 0008f6b0 00000002 00024c72 00000080 00000000 dfe9ffa8 ffa0: c002b140 c007a67c 0008f6b0 00000002 40212008 00024c72 00092008 00000140 ffc0: 0008f6b0 00000002 00024c72 00000080 0008f6b0 bef73f2b 0008f8a8 00000038 ffe0: bef73c60 bef73c50 00013ee4 401a7740 60000010 40212008 ffffffff 00000000 Backtrace: [<c002a4b8>] (do_one_initcall+0x0/0x1bc) from [<c007a72c>] (sys_init_module+0xbc/0x1ec) r7:40212008 r6:00092008 r5:bf002304 r4:00000000 [<c007a670>] (sys_init_module+0x0/0x1ec) from [<c002b140>] (ret_fast_syscall+0x0/0x30) r7:00000080 r6:00024c72 r5:00000002 r4:0008f6b0 Code: e89da800 bf002428 bf00205c bf001f47 (e1a0c00d) ---[ end trace 11affbcdd9483af2 ]--- Segmentation fault ****************************************************************************** ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 5:59 ` [PATCH 0/4] Patches for -next Shilimkar, Santosh @ 2010-05-21 6:06 ` Shilimkar, Santosh 2010-05-21 9:52 ` Catalin Marinas 1 sibling, 0 replies; 23+ messages in thread From: Shilimkar, Santosh @ 2010-05-21 6:06 UTC (permalink / raw) To: linux-arm-kernel One correction > -----Original Message----- > From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel- > bounces at lists.infradead.org] On Behalf Of Shilimkar, Santosh > Sent: Friday, May 21, 2010 11:30 AM > To: Catalin Marinas > Cc: Russell King - ARM Linux; linux-arm-kernel at lists.infradead.org > Subject: RE: [PATCH 0/4] Patches for -next > > Catalin, > > -----Original Message----- > > From: linux-arm-kernel-bounces at lists.infradead.org [mailto:linux-arm-kernel- > > bounces at lists.infradead.org] On Behalf Of Catalin Marinas > > Sent: Thursday, May 20, 2010 8:00 PM > > To: linux-arm-kernel at lists.infradead.org > > Subject: [PATCH 0/4] Patches for -next > > > > Hi, > > > > I would like to get these patches in the linux-next tree for this > > release cycle and aim to submit them for upstream 2.6.36-rc1 (next > > merging window). > > > > Please let me know if you have any objections. > > > > Thanks. > > > > > > Catalin Marinas (4): > > ARM: Remove the domain switching on ARMv6k/v7 CPUs > > ARM: Use lazy cache flushing on ARMv7 SMP systems > > ARM: Assume new page cache pages have dirty D-cache > > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > > > > I just gave quick try with these patches on OMAP4 yesterday. > "ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP" > patch is creating regression when loading the module. > > Didn't get time to debug it but though if reporting it. > > ****************************************************************************** > # insmod test_module.ko #insmod g_zero.ko > Unable to handle kernel paging request at virtual address bf005da8 > pgd = dfe20000 > [bf005da8] *pgd=9fede011, *pte=9fedf4df, *ppte=9fedf45f > Internal error: Oops: 8000000f [#1] PREEMPT SMP > last sysfs file: > Modules linked in: g_zero(+) > CPU: 1 Not tainted (2.6.34-00074-g8e3b656 #1) > PC is at init+0x0/0x1c [g_zero] > LR is at do_one_initcall+0x64/0x1bc > pc : [<bf005da8>] lr : [<c002a51c>] psr: 60000013 > sp : dfe9ff50 ip : c0356060 fp : dfe9ff7c > r10: 00000000 r9 : dfe9e000 r8 : c002b2e8 > r7 : 00000000 r6 : bf005da8 r5 : bf002304 r4 : 00000000 > r3 : 00000000 r2 : dfe9e000 r1 : 00000000 r0 : bf005da8 > Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user > Control: 10c53c7d Table: 9fe2004a DAC: 00000015 > Process insmod (pid: 585, stack limit = 0xdfe9e2f8) > Stack: (0xdfe9ff50 to 0xdfea0000) > ff40: 40212008 c002b2e8 dfe9ff7c > dfe9ff68 > ff60: 00000000 bf002304 00092008 40212008 dfe9ffa4 dfe9ff80 c007a72c > c002a4c4 > ff80: 00000003 40212000 0008f6b0 00000002 00024c72 00000080 00000000 > dfe9ffa8 > ffa0: c002b140 c007a67c 0008f6b0 00000002 40212008 00024c72 00092008 > 00000140 > ffc0: 0008f6b0 00000002 00024c72 00000080 0008f6b0 bef73f2b 0008f8a8 > 00000038 > ffe0: bef73c60 bef73c50 00013ee4 401a7740 60000010 40212008 ffffffff > 00000000 > Backtrace: > [<c002a4b8>] (do_one_initcall+0x0/0x1bc) from [<c007a72c>] > (sys_init_module+0xbc/0x1ec) > r7:40212008 r6:00092008 r5:bf002304 r4:00000000 > [<c007a670>] (sys_init_module+0x0/0x1ec) from [<c002b140>] > (ret_fast_syscall+0x0/0x30) > r7:00000080 r6:00024c72 r5:00000002 r4:0008f6b0 > Code: e89da800 bf002428 bf00205c bf001f47 (e1a0c00d) > ---[ end trace 11affbcdd9483af2 ]--- > Segmentation fault > ****************************************************************************** > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel at lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 5:59 ` [PATCH 0/4] Patches for -next Shilimkar, Santosh 2010-05-21 6:06 ` Shilimkar, Santosh @ 2010-05-21 9:52 ` Catalin Marinas 2010-05-21 10:57 ` Shilimkar, Santosh 2010-05-21 19:04 ` Russell King - ARM Linux 1 sibling, 2 replies; 23+ messages in thread From: Catalin Marinas @ 2010-05-21 9:52 UTC (permalink / raw) To: linux-arm-kernel Santosh, On Fri, 2010-05-21 at 06:59 +0100, Shilimkar, Santosh wrote: > > Catalin Marinas (4): > > ARM: Remove the domain switching on ARMv6k/v7 CPUs > > ARM: Use lazy cache flushing on ARMv7 SMP systems > > ARM: Assume new page cache pages have dirty D-cache > > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > > > > I just gave quick try with these patches on OMAP4 yesterday. > "ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP" > patch is creating regression when loading the module. Thanks for testing and reporting this. I missed the fact the modules allocation uses set_pte_at(). Below is an updated patch which makes sure that L_PTE_EXEC is set for addresses greater than TASK_SIZE. We shouldn't have a race for modules as initialisation only happens on a single CPU. Could you please try the updated patch below? ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP From: Catalin Marinas <catalin.marinas@arm.com> On SMP systems, there is a small chance of a PTE becoming visible to a different CPU before the cache maintenance operations in update_mmu_cache(). This patch clears the L_PTE_EXEC bit in set_pte_at() but sets it later in update_mmu_cache() if vm_flags & VM_EXEC. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/pgtable.h | 15 +++++++++++++++ arch/arm/mm/fault-armv.c | 17 ++++++++++++----- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index 1139768..ee8cc13 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -278,9 +278,24 @@ extern struct page *empty_zero_page; #define set_pte_ext(ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext) +#ifndef CONFIG_SMP #define set_pte_at(mm,addr,ptep,pteval) do { \ set_pte_ext(ptep, pteval, (addr) >= TASK_SIZE ? 0 : PTE_EXT_NG); \ } while (0) +#else +/* + * The L_PTE_EXEC attribute is later set in update_mmu_cache() to avoid a + * race with SMP systems executing from the new mapping before the cache + * flushing took place. + */ +#define set_pte_at(mm,addr,ptep,pteval) do { \ + if ((addr) >= TASK_SIZE) \ + set_pte_ext(ptep, pteval, 0); \ + else \ + set_pte_ext(ptep, __pte(pte_val(pteval) & ~L_PTE_EXEC), \ + PTE_EXT_NG); \ + } while (0) +#endif /* * The following only work if pte_present() is true. diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index f9e9cbb..c01356d 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -172,11 +172,18 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, mapping = page_mapping(page); if (!test_and_set_bit(PG_dcache_clean, &page->flags)) __flush_dcache_page(mapping, page); - if (mapping) { - if (cache_is_vivt()) - make_coherent(mapping, vma, addr, ptep, pfn); - else if (vma->vm_flags & VM_EXEC) - __flush_icache_all(); + if (!mapping) + return; + + if (cache_is_vivt()) + make_coherent(mapping, vma, addr, ptep, pfn); + else if (vma->vm_flags & VM_EXEC) { + __flush_icache_all(); +#ifdef CONFIG_SMP + set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), + addr >= TASK_SIZE ? 0 : PTE_EXT_NG); + flush_tlb_page(vma, addr); +#endif } } -- Catalin ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 9:52 ` Catalin Marinas @ 2010-05-21 10:57 ` Shilimkar, Santosh 2010-05-21 14:49 ` Catalin Marinas 2010-05-21 19:04 ` Russell King - ARM Linux 1 sibling, 1 reply; 23+ messages in thread From: Shilimkar, Santosh @ 2010-05-21 10:57 UTC (permalink / raw) To: linux-arm-kernel Catalin, > -----Original Message----- > From: Catalin Marinas [mailto:catalin.marinas at arm.com] > Sent: Friday, May 21, 2010 3:22 PM > To: Shilimkar, Santosh > Cc: Russell King - ARM Linux; linux-arm-kernel at lists.infradead.org > Subject: RE: [PATCH 0/4] Patches for -next > > Santosh, > > On Fri, 2010-05-21 at 06:59 +0100, Shilimkar, Santosh wrote: > > > Catalin Marinas (4): > > > ARM: Remove the domain switching on ARMv6k/v7 CPUs > > > ARM: Use lazy cache flushing on ARMv7 SMP systems > > > ARM: Assume new page cache pages have dirty D-cache > > > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > > > > > > > I just gave quick try with these patches on OMAP4 yesterday. > > "ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP" > > patch is creating regression when loading the module. > > Thanks for testing and reporting this. I missed the fact the modules > allocation uses set_pte_at(). Below is an updated patch which makes sure > that L_PTE_EXEC is set for addresses greater than TASK_SIZE. We > shouldn't have a race for modules as initialisation only happens on a > single CPU. > > Could you please try the updated patch below? > This one works fine!! You can add, Tested-by: Santosh Shilimkar <Santosh.shilimkar@ti.com> > > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > From: Catalin Marinas <catalin.marinas@arm.com> > > On SMP systems, there is a small chance of a PTE becoming visible to a > different CPU before the cache maintenance operations in > update_mmu_cache(). This patch clears the L_PTE_EXEC bit in set_pte_at() > but sets it later in update_mmu_cache() if vm_flags & VM_EXEC. > > Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> > --- > arch/arm/include/asm/pgtable.h | 15 +++++++++++++++ > arch/arm/mm/fault-armv.c | 17 ++++++++++++----- > 2 files changed, 27 insertions(+), 5 deletions(-) > > diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h > index 1139768..ee8cc13 100644 > --- a/arch/arm/include/asm/pgtable.h > +++ b/arch/arm/include/asm/pgtable.h > @@ -278,9 +278,24 @@ extern struct page *empty_zero_page; > > #define set_pte_ext(ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext) > > +#ifndef CONFIG_SMP > #define set_pte_at(mm,addr,ptep,pteval) do { \ > set_pte_ext(ptep, pteval, (addr) >= TASK_SIZE ? 0 : PTE_EXT_NG); \ > } while (0) > +#else > +/* > + * The L_PTE_EXEC attribute is later set in update_mmu_cache() to avoid a > + * race with SMP systems executing from the new mapping before the cache > + * flushing took place. > + */ > +#define set_pte_at(mm,addr,ptep,pteval) do { \ > + if ((addr) >= TASK_SIZE) \ > + set_pte_ext(ptep, pteval, 0); \ > + else \ > + set_pte_ext(ptep, __pte(pte_val(pteval) & ~L_PTE_EXEC), \ > + PTE_EXT_NG); \ > + } while (0) > +#endif > > /* > * The following only work if pte_present() is true. > diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c > index f9e9cbb..c01356d 100644 > --- a/arch/arm/mm/fault-armv.c > +++ b/arch/arm/mm/fault-armv.c > @@ -172,11 +172,18 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, > mapping = page_mapping(page); > if (!test_and_set_bit(PG_dcache_clean, &page->flags)) > __flush_dcache_page(mapping, page); > - if (mapping) { > - if (cache_is_vivt()) > - make_coherent(mapping, vma, addr, ptep, pfn); > - else if (vma->vm_flags & VM_EXEC) > - __flush_icache_all(); > + if (!mapping) > + return; > + > + if (cache_is_vivt()) > + make_coherent(mapping, vma, addr, ptep, pfn); > + else if (vma->vm_flags & VM_EXEC) { > + __flush_icache_all(); > +#ifdef CONFIG_SMP > + set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), > + addr >= TASK_SIZE ? 0 : PTE_EXT_NG); > + flush_tlb_page(vma, addr); > +#endif > } > } > > > > -- Regards, Santosh ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 10:57 ` Shilimkar, Santosh @ 2010-05-21 14:49 ` Catalin Marinas 2010-05-21 14:52 ` Shilimkar, Santosh 0 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-21 14:49 UTC (permalink / raw) To: linux-arm-kernel On Fri, 2010-05-21 at 11:57 +0100, Shilimkar, Santosh wrote: > Catalin, > > > -----Original Message----- > > From: Catalin Marinas [mailto:catalin.marinas at arm.com] > > Sent: Friday, May 21, 2010 3:22 PM > > To: Shilimkar, Santosh > > Cc: Russell King - ARM Linux; linux-arm-kernel at lists.infradead.org > > Subject: RE: [PATCH 0/4] Patches for -next > > > > Santosh, > > > > On Fri, 2010-05-21 at 06:59 +0100, Shilimkar, Santosh wrote: > > > > Catalin Marinas (4): > > > > ARM: Remove the domain switching on ARMv6k/v7 CPUs > > > > ARM: Use lazy cache flushing on ARMv7 SMP systems > > > > ARM: Assume new page cache pages have dirty D-cache > > > > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > > > > > > > > > > I just gave quick try with these patches on OMAP4 yesterday. > > > "ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP" > > > patch is creating regression when loading the module. > > > > Thanks for testing and reporting this. I missed the fact the modules > > allocation uses set_pte_at(). Below is an updated patch which makes sure > > that L_PTE_EXEC is set for addresses greater than TASK_SIZE. We > > shouldn't have a race for modules as initialisation only happens on a > > single CPU. > > > > Could you please try the updated patch below? > > > This one works fine!! > You can add, Tested-by: Santosh Shilimkar <Santosh.shilimkar@ti.com> Thanks for testing. Have you tried the whole series or just this one? -- Catalin ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 14:49 ` Catalin Marinas @ 2010-05-21 14:52 ` Shilimkar, Santosh 0 siblings, 0 replies; 23+ messages in thread From: Shilimkar, Santosh @ 2010-05-21 14:52 UTC (permalink / raw) To: linux-arm-kernel > -----Original Message----- > From: Catalin Marinas [mailto:catalin.marinas at arm.com] > Sent: Friday, May 21, 2010 8:20 PM > To: Shilimkar, Santosh > Cc: Russell King - ARM Linux; linux-arm-kernel at lists.infradead.org > Subject: RE: [PATCH 0/4] Patches for -next > > On Fri, 2010-05-21 at 11:57 +0100, Shilimkar, Santosh wrote: > > Catalin, > > > > > -----Original Message----- > > > From: Catalin Marinas [mailto:catalin.marinas at arm.com] > > > Sent: Friday, May 21, 2010 3:22 PM > > > To: Shilimkar, Santosh > > > Cc: Russell King - ARM Linux; linux-arm-kernel at lists.infradead.org > > > Subject: RE: [PATCH 0/4] Patches for -next > > > > > > Santosh, > > > > > > On Fri, 2010-05-21 at 06:59 +0100, Shilimkar, Santosh wrote: > > > > > Catalin Marinas (4): > > > > > ARM: Remove the domain switching on ARMv6k/v7 CPUs > > > > > ARM: Use lazy cache flushing on ARMv7 SMP systems > > > > > ARM: Assume new page cache pages have dirty D-cache > > > > > ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP > > > > > > > > > > > > > > I just gave quick try with these patches on OMAP4 yesterday. > > > > "ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP" > > > > patch is creating regression when loading the module. > > > > > > Thanks for testing and reporting this. I missed the fact the modules > > > allocation uses set_pte_at(). Below is an updated patch which makes sure > > > that L_PTE_EXEC is set for addresses greater than TASK_SIZE. We > > > shouldn't have a race for modules as initialisation only happens on a > > > single CPU. > > > > > > Could you please try the updated patch below? > > > > > This one works fine!! > > You can add, Tested-by: Santosh Shilimkar <Santosh.shilimkar@ti.com> > > Thanks for testing. > > Have you tried the whole series or just this one? Whole series. Regards Santosh ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 9:52 ` Catalin Marinas 2010-05-21 10:57 ` Shilimkar, Santosh @ 2010-05-21 19:04 ` Russell King - ARM Linux 2010-05-24 11:31 ` Catalin Marinas 1 sibling, 1 reply; 23+ messages in thread From: Russell King - ARM Linux @ 2010-05-21 19:04 UTC (permalink / raw) To: linux-arm-kernel On Fri, May 21, 2010 at 10:52:21AM +0100, Catalin Marinas wrote: > + if (cache_is_vivt()) > + make_coherent(mapping, vma, addr, ptep, pfn); > + else if (vma->vm_flags & VM_EXEC) { > + __flush_icache_all(); > +#ifdef CONFIG_SMP > + set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), > + addr >= TASK_SIZE ? 0 : PTE_EXT_NG); You don't need this test - pages which have VMAs are userspace pages, and therefore the address is always < TASK_SIZE. Therefore, they're always non-global: set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), PTE_EXT_NG); ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-21 19:04 ` Russell King - ARM Linux @ 2010-05-24 11:31 ` Catalin Marinas 2010-05-24 12:08 ` Russell King - ARM Linux 0 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-24 11:31 UTC (permalink / raw) To: linux-arm-kernel On Fri, 2010-05-21 at 20:04 +0100, Russell King - ARM Linux wrote: > On Fri, May 21, 2010 at 10:52:21AM +0100, Catalin Marinas wrote: > > + if (cache_is_vivt()) > > + make_coherent(mapping, vma, addr, ptep, pfn); > > + else if (vma->vm_flags & VM_EXEC) { > > + __flush_icache_all(); > > +#ifdef CONFIG_SMP > > + set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), > > + addr >= TASK_SIZE ? 0 : PTE_EXT_NG); > > You don't need this test - pages which have VMAs are userspace pages, > and therefore the address is always < TASK_SIZE. Therefore, they're > always non-global: > > set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), > PTE_EXT_NG); Good point. Thanks. So, are you ok for me to push these patches to -next? -- Catalin ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-24 11:31 ` Catalin Marinas @ 2010-05-24 12:08 ` Russell King - ARM Linux 2010-05-24 13:06 ` Catalin Marinas 0 siblings, 1 reply; 23+ messages in thread From: Russell King - ARM Linux @ 2010-05-24 12:08 UTC (permalink / raw) To: linux-arm-kernel On Mon, May 24, 2010 at 12:31:04PM +0100, Catalin Marinas wrote: > On Fri, 2010-05-21 at 20:04 +0100, Russell King - ARM Linux wrote: > > On Fri, May 21, 2010 at 10:52:21AM +0100, Catalin Marinas wrote: > > > + if (cache_is_vivt()) > > > + make_coherent(mapping, vma, addr, ptep, pfn); > > > + else if (vma->vm_flags & VM_EXEC) { > > > + __flush_icache_all(); > > > +#ifdef CONFIG_SMP > > > + set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), > > > + addr >= TASK_SIZE ? 0 : PTE_EXT_NG); > > > > You don't need this test - pages which have VMAs are userspace pages, > > and therefore the address is always < TASK_SIZE. Therefore, they're > > always non-global: > > > > set_pte_ext(ptep, __pte(pte_val(*ptep) | L_PTE_EXEC), > > PTE_EXT_NG); > > Good point. Thanks. > > So, are you ok for me to push these patches to -next? I don't think these patches are quite ready for -next just yet. Apart from my comment on patch 1, I think patch 4 is buggy in three other ways: 1. VIVT cached CPUs end up with L_PTE_EXEC clear on their mappings with this patch. Luckily for us, nothing tests for this (there used to be a pte_exec() helper for testing this, but I removed it as there were no users in the kernel) - but even so, I think it should be documented that this is the case. 2. L_PTE_EXEC is also forced-clear on non-page cache pages by this patch, even if executable permission was requested. I'm guessing this will be bad for tasks which generate code on the fly. 3. Kernel mappings are also created with set_pte_at(), and so module mappings will have L_PTE_EXEC cleared, resulting in kernel modules being non-executable - which is rather bad. Not sure what the right fix for this is at the moment - and if we're going to make set_pte_at() more complicated, it should be moved out of line. ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-24 12:08 ` Russell King - ARM Linux @ 2010-05-24 13:06 ` Catalin Marinas 2010-05-26 13:06 ` Catalin Marinas 0 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-24 13:06 UTC (permalink / raw) To: linux-arm-kernel On Mon, 2010-05-24 at 13:08 +0100, Russell King - ARM Linux wrote: > On Mon, May 24, 2010 at 12:31:04PM +0100, Catalin Marinas wrote: > > So, are you ok for me to push these patches to -next? > > I don't think these patches are quite ready for -next just yet. Apart > from my comment on patch 1, I think patch 4 is buggy in three other > ways: > > 1. VIVT cached CPUs end up with L_PTE_EXEC clear on their mappings > with this patch. Luckily for us, nothing tests for this (there > used to be a pte_exec() helper for testing this, but I removed it > as there were no users in the kernel) - but even so, I think it > should be documented that this is the case. The L_PTE_EXEC is only cleared in set_pte_at() if SMP. We won't have any VIVT SMP system. I could add a comment for this. > 2. L_PTE_EXEC is also forced-clear on non-page cache pages by this > patch, even if executable permission was requested. I'm guessing > this will be bad for tasks which generate code on the fly. Attributes changing via sys_mprotect() won't be handled, so this would be a problem. > 3. Kernel mappings are also created with set_pte_at(), and so module > mappings will have L_PTE_EXEC cleared, resulting in kernel modules > being non-executable - which is rather bad. Yes, this was already pointed at by Santosh and fixed in the latest version of the patch (it checks for addr < TASK_SIZE). > Not sure what the right fix for this is at the moment - and if we're > going to make set_pte_at() more complicated, it should be moved out > of line. The only way I can think of is to do the cache flushing in set_pte_at(). It would have a similar approach with clearing the exec bit, flushing the cache and then setting it (since we may not always have a kernel mapping). The exec/no-exec would only be done for SMP. If we go this route, for UP and all the other cache types, should we move the lazy cache flushing out of update_mmu_cache() as well? A side-effect of such change is that mprotect(exec) may trigger a cache flush, which I think it's fine. Thanks. -- Catalin ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-24 13:06 ` Catalin Marinas @ 2010-05-26 13:06 ` Catalin Marinas 2010-05-26 18:28 ` Russell King - ARM Linux 0 siblings, 1 reply; 23+ messages in thread From: Catalin Marinas @ 2010-05-26 13:06 UTC (permalink / raw) To: linux-arm-kernel On Mon, 2010-05-24 at 14:06 +0100, Catalin Marinas wrote: > On Mon, 2010-05-24 at 13:08 +0100, Russell King - ARM Linux wrote: > > Not sure what the right fix for this is at the moment - and if we're > > going to make set_pte_at() more complicated, it should be moved out > > of line. > > The only way I can think of is to do the cache flushing in set_pte_at(). Here's a new variant dealing with cache flushing via set_pte_at(). It could be optimised to avoid whole I-cache flushing on non-aliasing VIPT (well, all SMP systems). We could even make this method the default for UP v6+ systems. If you are OK with this approach, I'll repost the patches I have for -next. Thanks. ARM: Synchronise the I and D caches via set_pte_at() on SMP systems From: Catalin Marinas <catalin.marinas@arm.com> On SMP systems, there is a small chance of a PTE becoming visible to a different CPU before the cache maintenance operations in update_mmu_cache(). This patch follows the IA-64 and PowerPC approach of synchronising the I and D caches via the set_pte_at() function. In this case, there is no need for update_mmu_cache() to be implemented since lazy cache flushing is already handled by the time this function is called. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/pgtable.h | 21 ++++++++++++++++++--- arch/arm/include/asm/tlbflush.h | 10 +++++++++- arch/arm/mm/fault-armv.c | 2 ++ arch/arm/mm/flush.c | 13 +++++++++++++ 4 files changed, 42 insertions(+), 4 deletions(-) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index ab68cf1..9043f60 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -278,9 +278,24 @@ extern struct page *empty_zero_page; #define set_pte_ext(ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext) -#define set_pte_at(mm,addr,ptep,pteval) do { \ - set_pte_ext(ptep, pteval, (addr) >= TASK_SIZE ? 0 : PTE_EXT_NG); \ - } while (0) +#ifndef CONFIG_SMP +static inline void __sync_icache_dcache(pte_t pteval) +{ +} +#else +extern void __sync_icache_dcache(pte_t pteval); +#endif + +static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval) +{ + if (addr >= TASK_SIZE) + set_pte_ext(ptep, pteval, 0); + else { + __sync_icache_dcache(pteval); + set_pte_ext(ptep, pteval, PTE_EXT_NG); + } +} /* * The following only work if pte_present() is true. diff --git a/arch/arm/include/asm/tlbflush.h b/arch/arm/include/asm/tlbflush.h index 40a7092..6aabedd 100644 --- a/arch/arm/include/asm/tlbflush.h +++ b/arch/arm/include/asm/tlbflush.h @@ -554,10 +554,18 @@ extern void flush_tlb_kernel_range(unsigned long start, unsigned long end); /* * If PG_dcache_clean is not set for the page, we need to ensure that any * cache entries for the kernels virtual memory range are written - * back to the page. + * back to the page. On SMP systems, the cache coherency is handled in the + * set_pte_at() function. */ +#ifndef CONFIG_SMP extern void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); +#else +static inline void update_mmu_cache(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ +} +#endif #endif diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 91a691f..030f1da 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -28,6 +28,7 @@ static unsigned long shared_pte_mask = L_PTE_MT_BUFFERABLE; +#ifndef CONFIG_SMP /* * We take the easy way out of this problem - we make the * PTE uncacheable. However, we leave the write buffer on. @@ -177,6 +178,7 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, __flush_icache_all(); } } +#endif /* !CONFIG_SMP */ /* * Check whether the write buffer has physical address aliasing diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c index 6f55e36..90a6c60 100644 --- a/arch/arm/mm/flush.c +++ b/arch/arm/mm/flush.c @@ -212,6 +212,19 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p flush_dcache_mmap_unlock(mapping); } +#ifdef CONFIG_SMP +void __sync_icache_dcache(pte_t pteval) +{ + struct page *page = pte_page(pteval); + struct address_space *mapping = page_mapping(page); + + if (!test_and_set_bit(PG_dcache_clean, &page->flags)) + __flush_dcache_page(mapping, page); + if (pte_val(pteval) & L_PTE_EXEC) + __flush_icache_all(); +} +#endif + /* * Ensure cache coherency between kernel mapping and userspace mapping * of this page. -- Catalin ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-26 13:06 ` Catalin Marinas @ 2010-05-26 18:28 ` Russell King - ARM Linux 2010-05-27 11:04 ` Catalin Marinas 0 siblings, 1 reply; 23+ messages in thread From: Russell King - ARM Linux @ 2010-05-26 18:28 UTC (permalink / raw) To: linux-arm-kernel On Wed, May 26, 2010 at 02:06:35PM +0100, Catalin Marinas wrote: > +#ifdef CONFIG_SMP > +void __sync_icache_dcache(pte_t pteval) > +{ > + struct page *page = pte_page(pteval); > + struct address_space *mapping = page_mapping(page); Err. no. You're assuming that the physical page associated with the PTE always has a struct page. What if it's a device mapping? What if it's a mapping for some memory which isn't being handled by the kernel? You want this to be: unsigned long pfn = pte_pfn(pteval); if (pfn_valid(pfn)) { struct page *page = pfn_to_page(pfn); struct address_space *mapping = page_mapping(page); ^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 0/4] Patches for -next 2010-05-26 18:28 ` Russell King - ARM Linux @ 2010-05-27 11:04 ` Catalin Marinas 0 siblings, 0 replies; 23+ messages in thread From: Catalin Marinas @ 2010-05-27 11:04 UTC (permalink / raw) To: linux-arm-kernel On Wed, 2010-05-26 at 19:28 +0100, Russell King - ARM Linux wrote: > On Wed, May 26, 2010 at 02:06:35PM +0100, Catalin Marinas wrote: > > +#ifdef CONFIG_SMP > > +void __sync_icache_dcache(pte_t pteval) > > +{ > > + struct page *page = pte_page(pteval); > > + struct address_space *mapping = page_mapping(page); > > Err. no. You're assuming that the physical page associated with the > PTE always has a struct page. What if it's a device mapping? What > if it's a mapping for some memory which isn't being handled by the > kernel? > > You want this to be: > > unsigned long pfn = pte_pfn(pteval); > > if (pfn_valid(pfn)) { > struct page *page = pfn_to_page(pfn); > struct address_space *mapping = page_mapping(page); BTW, if we only target non-aliasing VIPT hardware, is there a need for "mapping" above? See below for an updated patch. I got some ideas from IA-64 to do additional checks on pteval before deciding to flush the cache. With this change, the D-cache won't be flushed at all for a page cache page which is not mapped as executable (update_mmu_cache is a no-op). It shouldn't matter since we don't have aliases. ARM: Synchronise the I and D caches via set_pte_at() on SMP systems From: Catalin Marinas <catalin.marinas@arm.com> On SMP systems, there is a small chance of a PTE becoming visible to a different CPU before the cache maintenance operations in update_mmu_cache(). This patch follows the IA-64 and PowerPC approach of synchronising the I and D caches via the set_pte_at() function. In this case, there is no need for update_mmu_cache() to be implemented since lazy cache flushing is already handled by the time this function is called. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> --- arch/arm/include/asm/pgtable.h | 25 ++++++++++++++++++++++--- arch/arm/include/asm/tlbflush.h | 10 +++++++++- arch/arm/mm/fault-armv.c | 2 ++ arch/arm/mm/flush.c | 15 +++++++++++++++ 4 files changed, 48 insertions(+), 4 deletions(-) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index ab68cf1..a3752f5 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -278,9 +278,24 @@ extern struct page *empty_zero_page; #define set_pte_ext(ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext) -#define set_pte_at(mm,addr,ptep,pteval) do { \ - set_pte_ext(ptep, pteval, (addr) >= TASK_SIZE ? 0 : PTE_EXT_NG); \ - } while (0) +#ifndef CONFIG_SMP +static inline void __sync_icache_dcache(pte_t pteval) +{ +} +#else +extern void __sync_icache_dcache(pte_t pteval); +#endif + +static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pteval) +{ + if (addr >= TASK_SIZE) + set_pte_ext(ptep, pteval, 0); + else { + __sync_icache_dcache(pteval); + set_pte_ext(ptep, pteval, PTE_EXT_NG); + } +} /* * The following only work if pte_present() is true. @@ -292,6 +307,10 @@ extern struct page *empty_zero_page; #define pte_young(pte) (pte_val(pte) & L_PTE_YOUNG) #define pte_special(pte) (0) +#define pte_present_exec_user(pte) \ + ((pte_val(pte) & (L_PTE_PRESENT | L_PTE_EXEC | L_PTE_USER)) == \ + (L_PTE_PRESENT | L_PTE_EXEC | L_PTE_USER)) + #define PTE_BIT_FUNC(fn,op) \ static inline pte_t pte_##fn(pte_t pte) { pte_val(pte) op; return pte; } diff --git a/arch/arm/include/asm/tlbflush.h b/arch/arm/include/asm/tlbflush.h index 40a7092..6aabedd 100644 --- a/arch/arm/include/asm/tlbflush.h +++ b/arch/arm/include/asm/tlbflush.h @@ -554,10 +554,18 @@ extern void flush_tlb_kernel_range(unsigned long start, unsigned long end); /* * If PG_dcache_clean is not set for the page, we need to ensure that any * cache entries for the kernels virtual memory range are written - * back to the page. + * back to the page. On SMP systems, the cache coherency is handled in the + * set_pte_at() function. */ +#ifndef CONFIG_SMP extern void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); +#else +static inline void update_mmu_cache(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ +} +#endif #endif diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 91a691f..030f1da 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -28,6 +28,7 @@ static unsigned long shared_pte_mask = L_PTE_MT_BUFFERABLE; +#ifndef CONFIG_SMP /* * We take the easy way out of this problem - we make the * PTE uncacheable. However, we leave the write buffer on. @@ -177,6 +178,7 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, __flush_icache_all(); } } +#endif /* !CONFIG_SMP */ /* * Check whether the write buffer has physical address aliasing diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c index 6f55e36..2d08a5e 100644 --- a/arch/arm/mm/flush.c +++ b/arch/arm/mm/flush.c @@ -212,6 +212,21 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p flush_dcache_mmap_unlock(mapping); } +#ifdef CONFIG_SMP +void __sync_icache_dcache(pte_t pteval) +{ + unsigned long pfn = pte_pfn(pteval); + + if (pfn_valid(pfn) && pte_present_exec_user(pteval)) { + struct page *page = pfn_to_page(pfn); + + if (!test_and_set_bit(PG_dcache_clean, &page->flags)) + __flush_dcache_page(NULL, page); + __flush_icache_all(); + } +} +#endif + /* * Ensure cache coherency between kernel mapping and userspace mapping * of this page. -- Catalin ^ permalink raw reply related [flat|nested] 23+ messages in thread
end of thread, other threads:[~2010-05-27 11:04 UTC | newest] Thread overview: 23+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-05-20 14:29 [PATCH 0/4] Patches for -next Catalin Marinas 2010-05-20 14:30 ` [PATCH 1/4] ARM: Remove the domain switching on ARMv6k/v7 CPUs Catalin Marinas 2010-05-24 11:45 ` Russell King - ARM Linux 2010-05-24 13:25 ` Catalin Marinas 2010-05-24 13:52 ` Catalin Marinas 2010-05-24 13:59 ` Russell King - ARM Linux 2010-05-24 14:02 ` Catalin Marinas 2010-05-20 14:30 ` [PATCH 2/4] ARM: Use lazy cache flushing on ARMv7 SMP systems Catalin Marinas 2010-05-20 14:30 ` [PATCH 3/4] ARM: Assume new page cache pages have dirty D-cache Catalin Marinas 2010-05-20 14:30 ` [PATCH 4/4] ARM: Defer the L_PTE_EXEC flag setting to update_mmu_cache() on SMP Catalin Marinas 2010-05-21 5:59 ` [PATCH 0/4] Patches for -next Shilimkar, Santosh 2010-05-21 6:06 ` Shilimkar, Santosh 2010-05-21 9:52 ` Catalin Marinas 2010-05-21 10:57 ` Shilimkar, Santosh 2010-05-21 14:49 ` Catalin Marinas 2010-05-21 14:52 ` Shilimkar, Santosh 2010-05-21 19:04 ` Russell King - ARM Linux 2010-05-24 11:31 ` Catalin Marinas 2010-05-24 12:08 ` Russell King - ARM Linux 2010-05-24 13:06 ` Catalin Marinas 2010-05-26 13:06 ` Catalin Marinas 2010-05-26 18:28 ` Russell King - ARM Linux 2010-05-27 11:04 ` Catalin Marinas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).