[PATCH 0/3] powerpc/64s: Restrict NMMU workarounds

[PATCH 0/3] powerpc/64s: Restrict NMMU workarounds

[Nicholas Piggin]

POWER10 doesn't require the two Nest MMU workarounds according to the
workbook. Also remove the last vestige of the spurious fault flushing
for NMMU which shouldn't have been required anyway.

Thanks,
Nick

Nicholas Piggin (3):
  powerpc/64s: POWER10 nest MMU does not require flush escalation
    workaround
  powerpc/64s: POWER10 nest MMU can upgrade PTE access authority without
    TLB flush
  powerpc/64s: Remove spurious fault flushing for NMMU

 arch/powerpc/include/asm/book3s/64/tlbflush.h | 28 +++++++++++++--
 arch/powerpc/mm/book3s64/radix_hugetlbpage.c  | 10 +++---
 arch/powerpc/mm/book3s64/radix_pgtable.c      | 35 ++++++++++++-------
 arch/powerpc/mm/book3s64/radix_tlb.c          | 14 ++++++--
 4 files changed, 64 insertions(+), 23 deletions(-)

-- 
2.35.1

[PATCH 1/3] powerpc/64s: POWER10 nest MMU does not require flush escalation workaround

[Nicholas Piggin]

Per (non-public) Nest MMU Workbook, POWER10 and POWER9P NMMU does not
cache PTEs in PWC, so does not require PWC flush to invalidate these
translations.

Skip the workaround on POWER10 and later.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 arch/powerpc/mm/book3s64/radix_tlb.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c
index 7724af19ed7e..7e233829b453 100644
--- a/arch/powerpc/mm/book3s64/radix_tlb.c
+++ b/arch/powerpc/mm/book3s64/radix_tlb.c
@@ -755,10 +755,18 @@ EXPORT_SYMBOL(radix__local_flush_tlb_page);
 static bool mm_needs_flush_escalation(struct mm_struct *mm)
 {
 	/*
-	 * P9 nest MMU has issues with the page walk cache
-	 * caching PTEs and not flushing them properly when
-	 * RIC = 0 for a PID/LPID invalidate
+	 * The P9 nest MMU has issues with the page walk cache caching PTEs
+	 * and not flushing them when RIC = 0 for a PID/LPID invalidate.
+	 *
+	 * This may have been fixed in shipping firmware (by disabling PWC
+	 * or preventing it from caching PTEs), but until that is confirmed,
+	 * this workaround is required - escalate all RIC=0 IS=1/2/3 flushes
+	 * to RIC=2.
+	 *
+	 * POWER10 (and P9P) does not have this problem.
 	 */
+	if (cpu_has_feature(CPU_FTR_ARCH_31))
+		return false;
 	if (atomic_read(&mm->context.copros) > 0)
 		return true;
 	return false;
-- 
2.35.1

[PATCH 2/3] powerpc/64s: POWER10 nest MMU can upgrade PTE access authority without TLB flush

[Nicholas Piggin]

The nest MMU in POWER9 does not re-fetch the PTE in response to
permission mismatch, contrary to the architecture[*] and unlike the core
MMU. This requires a TLB flush before upgrading permissions of valid
PTEs, for any address space with a coprocessor attached.

Per (non-public) Nest MMU Workbook, POWER10 nest MMU conforms to the
architecture in this regard, so skip the workaround.

[*] See: Power ISA Version 3.1B, 6.10.1.2 Modifying a Translation Table
    Entry, Setting a Reference or Change Bit or Upgrading Access
    Authority (PTE Subject to Atomic Hardware Updates):

      "If the only change being made to a valid PTE that is subject to
       atomic hardware updates is to set the Reference or Change bit to
       1 or to upgrade access authority, a simpler sequence suffices
       because the translation hardware will refetch the PTE if an
       access is attempted for which the only problems were reference
       and/or change bits needing to be set or insufficient access
       authority."

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 arch/powerpc/mm/book3s64/radix_hugetlbpage.c | 10 +++---
 arch/powerpc/mm/book3s64/radix_pgtable.c     | 35 ++++++++++++--------
 2 files changed, 28 insertions(+), 17 deletions(-)

diff --git a/arch/powerpc/mm/book3s64/radix_hugetlbpage.c b/arch/powerpc/mm/book3s64/radix_hugetlbpage.c
index 23d3e08911d3..78618c9b618b 100644
--- a/arch/powerpc/mm/book3s64/radix_hugetlbpage.c
+++ b/arch/powerpc/mm/book3s64/radix_hugetlbpage.c
@@ -103,11 +103,13 @@ void radix__huge_ptep_modify_prot_commit(struct vm_area_struct *vma,
 	struct mm_struct *mm = vma->vm_mm;
 
 	/*
-	 * To avoid NMMU hang while relaxing access we need to flush the tlb before
-	 * we set the new value.
+	 * POWER9 NMMU must flush the TLB after clearing the PTE before
+	 * installing a PTE with more relaxed access permissions, see
+	 * radix__ptep_set_access_flags.
 	 */
-	if (is_pte_rw_upgrade(pte_val(old_pte), pte_val(pte)) &&
-	    (atomic_read(&mm->context.copros) > 0))
+	if (!cpu_has_feature(CPU_FTR_ARCH_31) &&
+	    is_pte_rw_upgrade(pte_val(old_pte), pte_val(pte)) &&
+	    atomic_read(&mm->context.copros) > 0)
 		radix__flush_hugetlb_page(vma, addr);
 
 	set_huge_pte_at(vma->vm_mm, addr, ptep, pte);
diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index def04631a74d..195719a6c41c 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -1018,16 +1018,21 @@ void radix__ptep_set_access_flags(struct vm_area_struct *vma, pte_t *ptep,
 
 	unsigned long change = pte_val(entry) ^ pte_val(*ptep);
 	/*
-	 * To avoid NMMU hang while relaxing access, we need mark
-	 * the pte invalid in between.
+	 * On POWER9, the NMMU is not able to relax PTE access permissions
+	 * for a translation with a TLB. The PTE must be invalidated, TLB
+	 * flushed before the new PTE is installed.
+	 *
+	 * This only needs to be done for radix, because hash translation does
+	 * flush when updating the linux pte (and we don't support NMMU
+	 * accelerators on HPT on POWER9 anyway XXX: do we?).
+	 *
+	 * POWER10 (and P9P) NMMU does behave as per ISA.
 	 */
-	if ((change & _PAGE_RW) && atomic_read(&mm->context.copros) > 0) {
+	if (!cpu_has_feature(CPU_FTR_ARCH_31) && (change & _PAGE_RW) &&
+	    atomic_read(&mm->context.copros) > 0) {
 		unsigned long old_pte, new_pte;
 
 		old_pte = __radix_pte_update(ptep, _PAGE_PRESENT, _PAGE_INVALID);
-		/*
-		 * new value of pte
-		 */
 		new_pte = old_pte | set;
 		radix__flush_tlb_page_psize(mm, address, psize);
 		__radix_pte_update(ptep, _PAGE_INVALID, new_pte);
@@ -1035,9 +1040,12 @@ void radix__ptep_set_access_flags(struct vm_area_struct *vma, pte_t *ptep,
 		__radix_pte_update(ptep, 0, set);
 		/*
 		 * Book3S does not require a TLB flush when relaxing access
-		 * restrictions when the address space is not attached to a
-		 * NMMU, because the core MMU will reload the pte after taking
-		 * an access fault, which is defined by the architecture.
+		 * restrictions when the address space (modulo the POWER9 nest
+		 * MMU issue above) because the MMU will reload the PTE after
+		 * taking an access fault, as defined by the architecture. See
+		 * "Setting a Reference or Change Bit or Upgrading Access
+		 *  Authority (PTE Subject to Atomic Hardware Updates)" in
+		 *  Power ISA Version 3.1B.
 		 */
 	}
 	/* See ptesync comment in radix__set_pte_at */
@@ -1050,11 +1058,12 @@ void radix__ptep_modify_prot_commit(struct vm_area_struct *vma,
 	struct mm_struct *mm = vma->vm_mm;
 
 	/*
-	 * To avoid NMMU hang while relaxing access we need to flush the tlb before
-	 * we set the new value. We need to do this only for radix, because hash
-	 * translation does flush when updating the linux pte.
+	 * POWER9 NMMU must flush the TLB after clearing the PTE before
+	 * installing a PTE with more relaxed access permissions, see
+	 * radix__ptep_set_access_flags.
 	 */
-	if (is_pte_rw_upgrade(pte_val(old_pte), pte_val(pte)) &&
+	if (!cpu_has_feature(CPU_FTR_ARCH_31) &&
+	    is_pte_rw_upgrade(pte_val(old_pte), pte_val(pte)) &&
 	    (atomic_read(&mm->context.copros) > 0))
 		radix__flush_tlb_page(vma, addr);
 
-- 
2.35.1

[PATCH 3/3] powerpc/64s: Remove spurious fault flushing for NMMU

[Nicholas Piggin]

Commit 6d8278c414cb2 ("powerpc/64s/radix: do not flush TLB on spurious
fault") removed the TLB flush for spurious faults, except when a
coprocessor (nest MMU) maps the address space. This is not needed
because the NMMU workaround in the PTE permission upgrade paths
prevents PTEs existing with less restrictive access permissions than
their corresponding TLB entries have.

Remove it and replace with a comment.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 arch/powerpc/include/asm/book3s/64/tlbflush.h | 28 +++++++++++++++++--
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush.h b/arch/powerpc/include/asm/book3s/64/tlbflush.h
index d2e80f178b6d..ab01938f6c82 100644
--- a/arch/powerpc/include/asm/book3s/64/tlbflush.h
+++ b/arch/powerpc/include/asm/book3s/64/tlbflush.h
@@ -138,9 +138,31 @@ static inline void flush_all_mm(struct mm_struct *mm)
 static inline void flush_tlb_fix_spurious_fault(struct vm_area_struct *vma,
 						unsigned long address)
 {
-	/* See ptep_set_access_flags comment */
-	if (atomic_read(&vma->vm_mm->context.copros) > 0)
-		flush_tlb_page(vma, address);
+	/*
+	 * Book3S 64 does not require spurious fault flushes because the PTE
+	 * must be re-fetched in case of an access permission problem. So the
+	 * only reason for a spurious fault should be concurrent modification
+	 * to the PTE, in which case the PTE will eventually be re-fetched by
+	 * the MMU when it attempts the access again.
+	 *
+	 * See: Power ISA Version 3.1B, 6.10.1.2 Modifying a Translation Table
+	 * Entry, Setting a Reference or Change Bit or Upgrading Access
+	 * Authority (PTE Subject to Atomic Hardware Updates):
+	 *
+         * "If the only change being made to a valid PTE that is subject to
+         *  atomic hardware updates is to set the Reference or Change bit to
+         *  1 or to upgrade access authority, a simpler sequence suffices
+         *  because the translation hardware will refetch the PTE if an
+         *  access is attempted for which the only problems were reference
+         *  and/or change bits needing to be set or insufficient access
+         *  authority."
+	 */
+
+	/*
+	 * The nest MMU in POWER9 does not perform this PTE re-fetch, but
+	 * it avoids the spurious fault problem by flushing the TLB before
+	 * upgrading PTE permissions, see radix__ptep_set_access_flags.
+	 */
 }
 
 extern bool tlbie_capable;
-- 
2.35.1

Re: [PATCH 0/3] powerpc/64s: Restrict NMMU workarounds

[Michael Ellerman]

On Wed, 25 May 2022 12:23:55 +1000, Nicholas Piggin wrote:
> POWER10 doesn't require the two Nest MMU workarounds according to the
> workbook. Also remove the last vestige of the spurious fault flushing
> for NMMU which shouldn't have been required anyway.
> 
> Thanks,
> Nick
> 
> [...]

Applied to powerpc/next.

[1/3] powerpc/64s: POWER10 nest MMU does not require flush escalation workaround
      https://git.kernel.org/powerpc/c/abf0878ce95f8a9b47d8ecf2de1d4617bec21711
[2/3] powerpc/64s: POWER10 nest MMU can upgrade PTE access authority without TLB flush
      https://git.kernel.org/powerpc/c/2a8a0f420f74425bf5f80760fd14d7a2c3abb87d
[3/3] powerpc/64s: Remove spurious fault flushing for NMMU
      https://git.kernel.org/powerpc/c/fd193f85d3206cc7e7aeea2b6033d105cca38d01

cheers