* [PATCH V2 0/2] arm64: errata: Workaround Cortex-A715 errata #2645198 @ 2022-11-13 1:26 Anshuman Khandual 2022-11-13 1:26 ` [PATCH V2 1/2] arm64: Add Cortex-715 CPU part definition Anshuman Khandual 2022-11-13 1:26 ` [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Anshuman Khandual 0 siblings, 2 replies; 11+ messages in thread From: Anshuman Khandual @ 2022-11-13 1:26 UTC (permalink / raw) To: linux-arm-kernel, linux-kernel, catalin.marinas, will Cc: Anshuman Khandual, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc This series adds Cortex-A715 partnumber and workarounds the errata #2645198 which gets triggered when an userspace page mapping permission changes from executable to non-executable, corrupting both ESR_EL1/FAR_EL1 registers when an instruction abort is taken. This series applies on v6.1-rc4. The errata description can be found here. https://developer.arm.com/documentation/SDEN2148827/1000/?lang=en Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Suzuki K Poulose <suzuki.poulose@arm.com> Cc: James Morse <james.morse@arm.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Mark Rutland <mark.rutland@arm.com> Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org Changes in V2: - Added ptep_modify_prot_start/commit declaration into [asm/pgtable.h] - Moved ptep_modify_prot_start/commit definitions into [arch/arm64/mm/mmu.c] - Above code movements solved ptep_clear_flush() related build dependency - Replaced __flush_tlb_range() with ptep_clear_flush()/huge_ptep_clear_flush() Changes in V1: https://lore.kernel.org/all/20221027023915.1318100-1-anshuman.khandual@arm.com/ Anshuman Khandual (2): arm64: Add Cortex-715 CPU part definition arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Documentation/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 16 ++++++++++++++++ arch/arm64/include/asm/cputype.h | 2 ++ arch/arm64/include/asm/hugetlb.h | 9 +++++++++ arch/arm64/include/asm/pgtable.h | 9 +++++++++ arch/arm64/kernel/cpu_errata.c | 7 +++++++ arch/arm64/mm/hugetlbpage.c | 21 +++++++++++++++++++++ arch/arm64/mm/mmu.c | 21 +++++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + 9 files changed, 88 insertions(+) -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH V2 1/2] arm64: Add Cortex-715 CPU part definition 2022-11-13 1:26 [PATCH V2 0/2] arm64: errata: Workaround Cortex-A715 errata #2645198 Anshuman Khandual @ 2022-11-13 1:26 ` Anshuman Khandual 2022-11-15 13:51 ` Catalin Marinas 2022-11-13 1:26 ` [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Anshuman Khandual 1 sibling, 1 reply; 11+ messages in thread From: Anshuman Khandual @ 2022-11-13 1:26 UTC (permalink / raw) To: linux-arm-kernel, linux-kernel, catalin.marinas, will Cc: Anshuman Khandual, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc Add the CPU Partnumbers for the new Arm designs. Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Suzuki K Poulose <suzuki.poulose@arm.com> Cc: James Morse <james.morse@arm.com> Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> --- arch/arm64/include/asm/cputype.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index abc418650fec..4b1ad810436f 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -80,6 +80,7 @@ #define ARM_CPU_PART_CORTEX_X1 0xD44 #define ARM_CPU_PART_CORTEX_A510 0xD46 #define ARM_CPU_PART_CORTEX_A710 0xD47 +#define ARM_CPU_PART_CORTEX_A715 0xD4D #define ARM_CPU_PART_CORTEX_X2 0xD48 #define ARM_CPU_PART_NEOVERSE_N2 0xD49 #define ARM_CPU_PART_CORTEX_A78C 0xD4B @@ -142,6 +143,7 @@ #define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1) #define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510) #define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710) +#define MIDR_CORTEX_A715 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A715) #define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2) #define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2) #define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C) -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH V2 1/2] arm64: Add Cortex-715 CPU part definition 2022-11-13 1:26 ` [PATCH V2 1/2] arm64: Add Cortex-715 CPU part definition Anshuman Khandual @ 2022-11-15 13:51 ` Catalin Marinas 0 siblings, 0 replies; 11+ messages in thread From: Catalin Marinas @ 2022-11-15 13:51 UTC (permalink / raw) To: Anshuman Khandual Cc: linux-arm-kernel, linux-kernel, will, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On Sun, Nov 13, 2022 at 06:56:44AM +0530, Anshuman Khandual wrote: > Add the CPU Partnumbers for the new Arm designs. > > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will@kernel.org> > Cc: Suzuki K Poulose <suzuki.poulose@arm.com> > Cc: James Morse <james.morse@arm.com> > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-kernel@vger.kernel.org > Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-13 1:26 [PATCH V2 0/2] arm64: errata: Workaround Cortex-A715 errata #2645198 Anshuman Khandual 2022-11-13 1:26 ` [PATCH V2 1/2] arm64: Add Cortex-715 CPU part definition Anshuman Khandual @ 2022-11-13 1:26 ` Anshuman Khandual 2022-11-15 13:38 ` Will Deacon ` (2 more replies) 1 sibling, 3 replies; 11+ messages in thread From: Anshuman Khandual @ 2022-11-13 1:26 UTC (permalink / raw) To: linux-arm-kernel, linux-kernel, catalin.marinas, will Cc: Anshuman Khandual, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc If a Cortex-A715 cpu sees a page mapping permissions change from executable to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers, on the next instruction abort caused by permission fault. Only user-space does executable to non-executable permission transition via mprotect() system call which calls ptep_modify_prot_start() and ptep_modify _prot_commit() helpers, while changing the page mapping. The platform code can override these helpers via __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION. Work around the problem via doing a break-before-make TLB invalidation, for all executable user space mappings, that go through mprotect() system call. This overrides ptep_modify_prot_start() and ptep_modify_prot_commit(), via defining HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION on the platform thus giving an opportunity to intercept user space exec mappings, and do the necessary TLB invalidation. Similar interceptions are also implemented for HugeTLB. Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Mark Rutland <mark.rutland@arm.com> Cc: linux-arm-kernel@lists.infradead.org Cc: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> --- Documentation/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 16 ++++++++++++++++ arch/arm64/include/asm/hugetlb.h | 9 +++++++++ arch/arm64/include/asm/pgtable.h | 9 +++++++++ arch/arm64/kernel/cpu_errata.c | 7 +++++++ arch/arm64/mm/hugetlbpage.c | 21 +++++++++++++++++++++ arch/arm64/mm/mmu.c | 21 +++++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + 8 files changed, 86 insertions(+) diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst index 808ade4cc008..ec5f889d7681 100644 --- a/Documentation/arm64/silicon-errata.rst +++ b/Documentation/arm64/silicon-errata.rst @@ -120,6 +120,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A710 | #2224489 | ARM64_ERRATUM_2224489 | +----------------+-----------------+-----------------+-----------------------------+ +| ARM | Cortex-A715 | #2645198 | ARM64_ERRATUM_2645198 | ++----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-X2 | #2119858 | ARM64_ERRATUM_2119858 | +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-X2 | #2224489 | ARM64_ERRATUM_2224489 | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 505c8a1ccbe0..56c3381e9d94 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -964,6 +964,22 @@ config ARM64_ERRATUM_2457168 If unsure, say Y. +config ARM64_ERRATUM_2645198 + bool "Cortex-A715: 2645198: Workaround possible [ESR|FAR]_ELx corruption" + default y + help + This option adds the workaround for ARM Cortex-A715 erratum 2645198. + + If a Cortex-A715 cpu sees a page mapping permissions change from executable + to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers on the + next instruction abort caused by permission fault. + + Only user-space does executable to non-executable permission transition via + mprotect() system call. Workaround the problem by doing a break-before-make + TLB invalidation, for all changes to executable user space mappings. + + If unsure, say Y. + config CAVIUM_ERRATUM_22375 bool "Cavium erratum 22375, 24313" default y diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index d20f5da2d76f..6a4a1ab8eb23 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -49,6 +49,15 @@ extern pte_t huge_ptep_get(pte_t *ptep); void __init arm64_hugetlb_cma_reserve(void); +#define huge_ptep_modify_prot_start huge_ptep_modify_prot_start +extern pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + +#define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit +extern void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t new_pte); + #include <asm-generic/hugetlb.h> #endif /* __ASM_HUGETLB_H */ diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 71a1af42f0e8..fe76e5823d91 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1096,6 +1096,15 @@ static inline bool pud_sect_supported(void) } +#define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION +#define ptep_modify_prot_start ptep_modify_prot_start +extern pte_t ptep_modify_prot_start(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + +#define ptep_modify_prot_commit ptep_modify_prot_commit +extern void ptep_modify_prot_commit(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t new_pte); #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 89ac00084f38..307faa2b4395 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -661,6 +661,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = { CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus), }, #endif +#ifdef CONFIG_ARM64_ERRATUM_2645198 + { + .desc = "ARM erratum 2645198", + .capability = ARM64_WORKAROUND_2645198, + ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A715) + }, +#endif #ifdef CONFIG_ARM64_ERRATUM_2077057 { .desc = "ARM erratum 2077057", diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 35e9a468d13e..6552947ca7fa 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -559,3 +559,24 @@ bool __init arch_hugetlb_valid_size(unsigned long size) { return __hugetlb_valid_size(size); } + +pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) +{ + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { + pte_t pte = READ_ONCE(*ptep); + /* + * Break-before-make (BBM) is required for all user space mappings + * when the permission changes from executable to non-executable + * in cases where cpu is affected with errata #2645198. + */ + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) + return huge_ptep_clear_flush(vma, addr, ptep); + } + return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); +} + +void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t pte) +{ + set_huge_pte_at(vma->vm_mm, addr, ptep, pte); +} diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 9a7c38965154..c1fb0ce1473c 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) } early_initcall(prevent_bootmem_remove_init); #endif + +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) +{ + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { + pte_t pte = READ_ONCE(*ptep); + /* + * Break-before-make (BBM) is required for all user space mappings + * when the permission changes from executable to non-executable + * in cases where cpu is affected with errata #2645198. + */ + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) + return ptep_clear_flush(vma, addr, ptep); + } + return ptep_get_and_clear(vma->vm_mm, addr, ptep); +} + +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, + pte_t old_pte, pte_t pte) +{ + __set_pte_at(vma->vm_mm, addr, ptep, pte); +} diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index f1c0347ec31a..2274d836fcfe 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -70,6 +70,7 @@ WORKAROUND_2038923 WORKAROUND_2064142 WORKAROUND_2077057 WORKAROUND_2457168 +WORKAROUND_2645198 WORKAROUND_2658417 WORKAROUND_TRBE_OVERWRITE_FILL_MODE WORKAROUND_TSB_FLUSH_FAILURE -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-13 1:26 ` [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Anshuman Khandual @ 2022-11-15 13:38 ` Will Deacon 2022-11-15 13:42 ` Will Deacon 2022-11-15 14:02 ` Catalin Marinas 2022-11-16 5:04 ` Anshuman Khandual 2 siblings, 1 reply; 11+ messages in thread From: Will Deacon @ 2022-11-15 13:38 UTC (permalink / raw) To: Anshuman Khandual Cc: linux-arm-kernel, linux-kernel, catalin.marinas, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On Sun, Nov 13, 2022 at 06:56:45AM +0530, Anshuman Khandual wrote: > If a Cortex-A715 cpu sees a page mapping permissions change from executable > to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers, on the > next instruction abort caused by permission fault. > > Only user-space does executable to non-executable permission transition via > mprotect() system call which calls ptep_modify_prot_start() and ptep_modify > _prot_commit() helpers, while changing the page mapping. The platform code > can override these helpers via __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION. > > Work around the problem via doing a break-before-make TLB invalidation, for > all executable user space mappings, that go through mprotect() system call. > This overrides ptep_modify_prot_start() and ptep_modify_prot_commit(), via > defining HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION on the platform thus giving > an opportunity to intercept user space exec mappings, and do the necessary > TLB invalidation. Similar interceptions are also implemented for HugeTLB. > > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will@kernel.org> > Cc: Jonathan Corbet <corbet@lwn.net> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-doc@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> > --- > Documentation/arm64/silicon-errata.rst | 2 ++ > arch/arm64/Kconfig | 16 ++++++++++++++++ > arch/arm64/include/asm/hugetlb.h | 9 +++++++++ > arch/arm64/include/asm/pgtable.h | 9 +++++++++ > arch/arm64/kernel/cpu_errata.c | 7 +++++++ > arch/arm64/mm/hugetlbpage.c | 21 +++++++++++++++++++++ > arch/arm64/mm/mmu.c | 21 +++++++++++++++++++++ > arch/arm64/tools/cpucaps | 1 + > 8 files changed, 86 insertions(+) [...] > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 9a7c38965154..c1fb0ce1473c 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) > } > early_initcall(prevent_bootmem_remove_init); > #endif > + > +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > +{ > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > + pte_t pte = READ_ONCE(*ptep); > + /* > + * Break-before-make (BBM) is required for all user space mappings > + * when the permission changes from executable to non-executable > + * in cases where cpu is affected with errata #2645198. > + */ > + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + return ptep_clear_flush(vma, addr, ptep); > + } > + return ptep_get_and_clear(vma->vm_mm, addr, ptep); > +} > + > +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, > + pte_t old_pte, pte_t pte) > +{ > + __set_pte_at(vma->vm_mm, addr, ptep, pte); > +} So these are really similar to the generic copies and, in looking at change_pte_range(), it appears that we already invalidate the TLB, it just happens _after_ writing the new version. So with your change, I think we end up invalidating twice. Can we instead change the generic code to invalidate the TLB before writing the new entry? Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-15 13:38 ` Will Deacon @ 2022-11-15 13:42 ` Will Deacon 2022-11-16 5:49 ` Anshuman Khandual 0 siblings, 1 reply; 11+ messages in thread From: Will Deacon @ 2022-11-15 13:42 UTC (permalink / raw) To: Anshuman Khandual Cc: linux-arm-kernel, linux-kernel, catalin.marinas, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On Tue, Nov 15, 2022 at 01:38:54PM +0000, Will Deacon wrote: > On Sun, Nov 13, 2022 at 06:56:45AM +0530, Anshuman Khandual wrote: > > If a Cortex-A715 cpu sees a page mapping permissions change from executable > > to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers, on the > > next instruction abort caused by permission fault. > > > > Only user-space does executable to non-executable permission transition via > > mprotect() system call which calls ptep_modify_prot_start() and ptep_modify > > _prot_commit() helpers, while changing the page mapping. The platform code > > can override these helpers via __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION. > > > > Work around the problem via doing a break-before-make TLB invalidation, for > > all executable user space mappings, that go through mprotect() system call. > > This overrides ptep_modify_prot_start() and ptep_modify_prot_commit(), via > > defining HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION on the platform thus giving > > an opportunity to intercept user space exec mappings, and do the necessary > > TLB invalidation. Similar interceptions are also implemented for HugeTLB. > > > > Cc: Catalin Marinas <catalin.marinas@arm.com> > > Cc: Will Deacon <will@kernel.org> > > Cc: Jonathan Corbet <corbet@lwn.net> > > Cc: Mark Rutland <mark.rutland@arm.com> > > Cc: linux-arm-kernel@lists.infradead.org > > Cc: linux-doc@vger.kernel.org > > Cc: linux-kernel@vger.kernel.org > > Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> > > --- > > Documentation/arm64/silicon-errata.rst | 2 ++ > > arch/arm64/Kconfig | 16 ++++++++++++++++ > > arch/arm64/include/asm/hugetlb.h | 9 +++++++++ > > arch/arm64/include/asm/pgtable.h | 9 +++++++++ > > arch/arm64/kernel/cpu_errata.c | 7 +++++++ > > arch/arm64/mm/hugetlbpage.c | 21 +++++++++++++++++++++ > > arch/arm64/mm/mmu.c | 21 +++++++++++++++++++++ > > arch/arm64/tools/cpucaps | 1 + > > 8 files changed, 86 insertions(+) > > [...] > > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > > index 9a7c38965154..c1fb0ce1473c 100644 > > --- a/arch/arm64/mm/mmu.c > > +++ b/arch/arm64/mm/mmu.c > > @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) > > } > > early_initcall(prevent_bootmem_remove_init); > > #endif > > + > > +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > > +{ > > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > > + pte_t pte = READ_ONCE(*ptep); > > + /* > > + * Break-before-make (BBM) is required for all user space mappings > > + * when the permission changes from executable to non-executable > > + * in cases where cpu is affected with errata #2645198. > > + */ > > + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > > + return ptep_clear_flush(vma, addr, ptep); > > + } > > + return ptep_get_and_clear(vma->vm_mm, addr, ptep); > > +} > > + > > +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, > > + pte_t old_pte, pte_t pte) > > +{ > > + __set_pte_at(vma->vm_mm, addr, ptep, pte); > > +} > > So these are really similar to the generic copies and, in looking at > change_pte_range(), it appears that we already invalidate the TLB, it just > happens _after_ writing the new version. > > So with your change, I think we end up invalidating twice. Can we instead > change the generic code to invalidate the TLB before writing the new entry? Bah, scratch that, the invalidations are all batched, aren't they? It just seems silly that we have to add all this code just to do a TLB invalidation. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-15 13:42 ` Will Deacon @ 2022-11-16 5:49 ` Anshuman Khandual 0 siblings, 0 replies; 11+ messages in thread From: Anshuman Khandual @ 2022-11-16 5:49 UTC (permalink / raw) To: Will Deacon Cc: linux-arm-kernel, linux-kernel, catalin.marinas, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On 11/15/22 19:12, Will Deacon wrote: > On Tue, Nov 15, 2022 at 01:38:54PM +0000, Will Deacon wrote: >> On Sun, Nov 13, 2022 at 06:56:45AM +0530, Anshuman Khandual wrote: >>> If a Cortex-A715 cpu sees a page mapping permissions change from executable >>> to non-executable, it may corrupt the ESR_ELx and FAR_ELx registers, on the >>> next instruction abort caused by permission fault. >>> >>> Only user-space does executable to non-executable permission transition via >>> mprotect() system call which calls ptep_modify_prot_start() and ptep_modify >>> _prot_commit() helpers, while changing the page mapping. The platform code >>> can override these helpers via __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION. >>> >>> Work around the problem via doing a break-before-make TLB invalidation, for >>> all executable user space mappings, that go through mprotect() system call. >>> This overrides ptep_modify_prot_start() and ptep_modify_prot_commit(), via >>> defining HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION on the platform thus giving >>> an opportunity to intercept user space exec mappings, and do the necessary >>> TLB invalidation. Similar interceptions are also implemented for HugeTLB. >>> >>> Cc: Catalin Marinas <catalin.marinas@arm.com> >>> Cc: Will Deacon <will@kernel.org> >>> Cc: Jonathan Corbet <corbet@lwn.net> >>> Cc: Mark Rutland <mark.rutland@arm.com> >>> Cc: linux-arm-kernel@lists.infradead.org >>> Cc: linux-doc@vger.kernel.org >>> Cc: linux-kernel@vger.kernel.org >>> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> >>> --- >>> Documentation/arm64/silicon-errata.rst | 2 ++ >>> arch/arm64/Kconfig | 16 ++++++++++++++++ >>> arch/arm64/include/asm/hugetlb.h | 9 +++++++++ >>> arch/arm64/include/asm/pgtable.h | 9 +++++++++ >>> arch/arm64/kernel/cpu_errata.c | 7 +++++++ >>> arch/arm64/mm/hugetlbpage.c | 21 +++++++++++++++++++++ >>> arch/arm64/mm/mmu.c | 21 +++++++++++++++++++++ >>> arch/arm64/tools/cpucaps | 1 + >>> 8 files changed, 86 insertions(+) >> >> [...] >> >>> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c >>> index 9a7c38965154..c1fb0ce1473c 100644 >>> --- a/arch/arm64/mm/mmu.c >>> +++ b/arch/arm64/mm/mmu.c >>> @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) >>> } >>> early_initcall(prevent_bootmem_remove_init); >>> #endif >>> + >>> +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) >>> +{ >>> + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { >>> + pte_t pte = READ_ONCE(*ptep); >>> + /* >>> + * Break-before-make (BBM) is required for all user space mappings >>> + * when the permission changes from executable to non-executable >>> + * in cases where cpu is affected with errata #2645198. >>> + */ >>> + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) >>> + return ptep_clear_flush(vma, addr, ptep); >>> + } >>> + return ptep_get_and_clear(vma->vm_mm, addr, ptep); >>> +} >>> + >>> +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, >>> + pte_t old_pte, pte_t pte) >>> +{ >>> + __set_pte_at(vma->vm_mm, addr, ptep, pte); >>> +} >> >> So these are really similar to the generic copies and, in looking at >> change_pte_range(), it appears that we already invalidate the TLB, it just >> happens _after_ writing the new version. >> >> So with your change, I think we end up invalidating twice. Can we instead >> change the generic code to invalidate the TLB before writing the new entry? > > Bah, scratch that, the invalidations are all batched, aren't they? Right. > > It just seems silly that we have to add all this code just to do a TLB > invalidation. Right, but only when affected by this errata. Otherwise it is just same as the existing generic definitions. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-13 1:26 ` [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Anshuman Khandual 2022-11-15 13:38 ` Will Deacon @ 2022-11-15 14:02 ` Catalin Marinas 2022-11-16 4:42 ` Anshuman Khandual 2022-11-16 5:04 ` Anshuman Khandual 2 siblings, 1 reply; 11+ messages in thread From: Catalin Marinas @ 2022-11-15 14:02 UTC (permalink / raw) To: Anshuman Khandual Cc: linux-arm-kernel, linux-kernel, will, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On Sun, Nov 13, 2022 at 06:56:45AM +0530, Anshuman Khandual wrote: > diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c > index 35e9a468d13e..6552947ca7fa 100644 > --- a/arch/arm64/mm/hugetlbpage.c > +++ b/arch/arm64/mm/hugetlbpage.c > @@ -559,3 +559,24 @@ bool __init arch_hugetlb_valid_size(unsigned long size) > { > return __hugetlb_valid_size(size); > } > + > +pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > +{ > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > + pte_t pte = READ_ONCE(*ptep); Not sure whether the generated code would be any different but we should probably add the check for the CPU capability in the 'if' condition above, before the READ_ONCE (which expands to some asm volatile): if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { pte_t pte = ... ... } > + /* > + * Break-before-make (BBM) is required for all user space mappings > + * when the permission changes from executable to non-executable > + * in cases where cpu is affected with errata #2645198. > + */ > + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + return huge_ptep_clear_flush(vma, addr, ptep); > + } > + return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); > +} > + > +void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, > + pte_t old_pte, pte_t pte) > +{ > + set_huge_pte_at(vma->vm_mm, addr, ptep, pte); > +} > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 9a7c38965154..c1fb0ce1473c 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) > } > early_initcall(prevent_bootmem_remove_init); > #endif > + > +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > +{ > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > + pte_t pte = READ_ONCE(*ptep); > + /* > + * Break-before-make (BBM) is required for all user space mappings > + * when the permission changes from executable to non-executable > + * in cases where cpu is affected with errata #2645198. > + */ > + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + return ptep_clear_flush(vma, addr, ptep); > + } Same here. > + return ptep_get_and_clear(vma->vm_mm, addr, ptep); > +} > + > +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, > + pte_t old_pte, pte_t pte) > +{ > + __set_pte_at(vma->vm_mm, addr, ptep, pte); > +} -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-15 14:02 ` Catalin Marinas @ 2022-11-16 4:42 ` Anshuman Khandual 2022-11-16 9:00 ` Catalin Marinas 0 siblings, 1 reply; 11+ messages in thread From: Anshuman Khandual @ 2022-11-16 4:42 UTC (permalink / raw) To: Catalin Marinas Cc: linux-arm-kernel, linux-kernel, will, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On 11/15/22 19:32, Catalin Marinas wrote: > On Sun, Nov 13, 2022 at 06:56:45AM +0530, Anshuman Khandual wrote: >> diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c >> index 35e9a468d13e..6552947ca7fa 100644 >> --- a/arch/arm64/mm/hugetlbpage.c >> +++ b/arch/arm64/mm/hugetlbpage.c >> @@ -559,3 +559,24 @@ bool __init arch_hugetlb_valid_size(unsigned long size) >> { >> return __hugetlb_valid_size(size); >> } >> + >> +pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) >> +{ >> + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { >> + pte_t pte = READ_ONCE(*ptep); > > Not sure whether the generated code would be any different but we should > probably add the check for the CPU capability in the 'if' condition > above, before the READ_ONCE (which expands to some asm volatile): Sure, will do. > > if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) && > cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { > pte_t pte = ... > ... > } The local variable 'pte_t pte' can be dropped as well. > >> + /* >> + * Break-before-make (BBM) is required for all user space mappings >> + * when the permission changes from executable to non-executable >> + * in cases where cpu is affected with errata #2645198. >> + */ >> + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) >> + return huge_ptep_clear_flush(vma, addr, ptep); >> + } >> + return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); >> +} >> + >> +void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, >> + pte_t old_pte, pte_t pte) >> +{ >> + set_huge_pte_at(vma->vm_mm, addr, ptep, pte); >> +} >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c >> index 9a7c38965154..c1fb0ce1473c 100644 >> --- a/arch/arm64/mm/mmu.c >> +++ b/arch/arm64/mm/mmu.c >> @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) >> } >> early_initcall(prevent_bootmem_remove_init); >> #endif >> + >> +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) >> +{ >> + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { >> + pte_t pte = READ_ONCE(*ptep); >> + /* >> + * Break-before-make (BBM) is required for all user space mappings >> + * when the permission changes from executable to non-executable >> + * in cases where cpu is affected with errata #2645198. >> + */ >> + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) >> + return ptep_clear_flush(vma, addr, ptep); >> + } > > Same here. Sure, will do. > >> + return ptep_get_and_clear(vma->vm_mm, addr, ptep); >> +} >> + >> +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, >> + pte_t old_pte, pte_t pte) >> +{ >> + __set_pte_at(vma->vm_mm, addr, ptep, pte); >> +} > Planning to apply the following change after this patch. diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 6552947ca7fa..cd8d96e1fa1a 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -562,14 +562,14 @@ bool __init arch_hugetlb_valid_size(unsigned long size) pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { - if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { - pte_t pte = READ_ONCE(*ptep); + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) && + cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { /* * Break-before-make (BBM) is required for all user space mappings * when the permission changes from executable to non-executable * in cases where cpu is affected with errata #2645198. */ - if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) + if (pte_user_exec(READ_ONCE(*ptep))) return huge_ptep_clear_flush(vma, addr, ptep); } return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index c1fb0ce1473c..ec305ea3942c 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1705,14 +1705,14 @@ early_initcall(prevent_bootmem_remove_init); pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { - if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { - pte_t pte = READ_ONCE(*ptep); + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) && + cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { /* * Break-before-make (BBM) is required for all user space mappings * when the permission changes from executable to non-executable * in cases where cpu is affected with errata #2645198. */ - if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) + if (pte_user_exec(READ_ONCE(*ptep))) return ptep_clear_flush(vma, addr, ptep); } return ptep_get_and_clear(vma->vm_mm, addr, ptep); _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-16 4:42 ` Anshuman Khandual @ 2022-11-16 9:00 ` Catalin Marinas 0 siblings, 0 replies; 11+ messages in thread From: Catalin Marinas @ 2022-11-16 9:00 UTC (permalink / raw) To: Anshuman Khandual Cc: linux-arm-kernel, linux-kernel, will, Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On Wed, Nov 16, 2022 at 10:12:34AM +0530, Anshuman Khandual wrote: > Planning to apply the following change after this patch. > > diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c > index 6552947ca7fa..cd8d96e1fa1a 100644 > --- a/arch/arm64/mm/hugetlbpage.c > +++ b/arch/arm64/mm/hugetlbpage.c > @@ -562,14 +562,14 @@ bool __init arch_hugetlb_valid_size(unsigned long size) > > pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > { > - if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > - pte_t pte = READ_ONCE(*ptep); > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) && > + cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { > /* > * Break-before-make (BBM) is required for all user space mappings > * when the permission changes from executable to non-executable > * in cases where cpu is affected with errata #2645198. > */ > - if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + if (pte_user_exec(READ_ONCE(*ptep))) > return huge_ptep_clear_flush(vma, addr, ptep); > } > return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index c1fb0ce1473c..ec305ea3942c 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -1705,14 +1705,14 @@ early_initcall(prevent_bootmem_remove_init); > > pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > { > - if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > - pte_t pte = READ_ONCE(*ptep); > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198) && > + cpus_have_const_cap(ARM64_WORKAROUND_2645198)) { > /* > * Break-before-make (BBM) is required for all user space mappings > * when the permission changes from executable to non-executable > * in cases where cpu is affected with errata #2645198. > */ > - if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + if (pte_user_exec(READ_ONCE(*ptep))) > return ptep_clear_flush(vma, addr, ptep); > } > return ptep_get_and_clear(vma->vm_mm, addr, ptep); It looks fine to me. Thanks. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption 2022-11-13 1:26 ` [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Anshuman Khandual 2022-11-15 13:38 ` Will Deacon 2022-11-15 14:02 ` Catalin Marinas @ 2022-11-16 5:04 ` Anshuman Khandual 2 siblings, 0 replies; 11+ messages in thread From: Anshuman Khandual @ 2022-11-16 5:04 UTC (permalink / raw) To: linux-arm-kernel, linux-kernel, catalin.marinas, will Cc: Suzuki K Poulose, James Morse, Jonathan Corbet, Mark Rutland, linux-doc On 11/13/22 06:56, Anshuman Khandual wrote: > +pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > +{ > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > + pte_t pte = READ_ONCE(*ptep); > + /* > + * Break-before-make (BBM) is required for all user space mappings > + * when the permission changes from executable to non-executable > + * in cases where cpu is affected with errata #2645198. > + */ > + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + return huge_ptep_clear_flush(vma, addr, ptep); > + } > + return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep); > +} > + > +void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, > + pte_t old_pte, pte_t pte) > +{ > + set_huge_pte_at(vma->vm_mm, addr, ptep, pte); > +} > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > index 9a7c38965154..c1fb0ce1473c 100644 > --- a/arch/arm64/mm/mmu.c > +++ b/arch/arm64/mm/mmu.c > @@ -1702,3 +1702,24 @@ static int __init prevent_bootmem_remove_init(void) > } > early_initcall(prevent_bootmem_remove_init); > #endif > + > +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) > +{ > + if (IS_ENABLED(CONFIG_ARM64_WORKAROUND_2645198)) { > + pte_t pte = READ_ONCE(*ptep); > + /* > + * Break-before-make (BBM) is required for all user space mappings > + * when the permission changes from executable to non-executable > + * in cases where cpu is affected with errata #2645198. > + */ > + if (pte_user_exec(pte) && cpus_have_const_cap(ARM64_WORKAROUND_2645198)) > + return ptep_clear_flush(vma, addr, ptep); > + } > + return ptep_get_and_clear(vma->vm_mm, addr, ptep); > +} > + > +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, > + pte_t old_pte, pte_t pte) > +{ > + __set_pte_at(vma->vm_mm, addr, ptep, pte); > +} Will change this __set_pte_at() to set_pte_at() instead like the generic version, which also makes sure that page_table_check_pte_set() gets called on the way, also making it similar to the HugeTLB implementation huge_ptep_modify_prot_commit(). _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-11-16 9:05 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-11-13 1:26 [PATCH V2 0/2] arm64: errata: Workaround Cortex-A715 errata #2645198 Anshuman Khandual 2022-11-13 1:26 ` [PATCH V2 1/2] arm64: Add Cortex-715 CPU part definition Anshuman Khandual 2022-11-15 13:51 ` Catalin Marinas 2022-11-13 1:26 ` [PATCH V2 2/2] arm64: errata: Workaround possible Cortex-A715 [ESR|FAR]_ELx corruption Anshuman Khandual 2022-11-15 13:38 ` Will Deacon 2022-11-15 13:42 ` Will Deacon 2022-11-16 5:49 ` Anshuman Khandual 2022-11-15 14:02 ` Catalin Marinas 2022-11-16 4:42 ` Anshuman Khandual 2022-11-16 9:00 ` Catalin Marinas 2022-11-16 5:04 ` Anshuman Khandual
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).