[PATCH] arm64: hugetlb: fix BBM for mprotect() on contiguous PTEs

[PATCH] arm64: hugetlb: fix BBM for mprotect() on contiguous PTEs

[Karl Mehltretter]

huge_ptep_modify_prot_start() clears a hugetlb entry before changing its
permissions. For contiguous PTE mappings, break-before-make (BBM)
requires a TLB invalidation after clearing the set and before making any
entry valid again.

Commit fb396bb459c1 ("arm64/hugetlb: Drop TLB flush from
get_clear_flush()") removed this invalidation, relying on the deferred
flush from the core code. Commit 410982303772 ("arm64: hugetlb: Restore
TLB invalidation for BBM on contiguous ptes") restored it for
huge_ptep_set_{access_flags,wrprotect}(), since a deferred flush is too
late for the break step. The modify-prot path has the same problem.

Use huge_ptep_clear_flush() for contiguous entries so that the TLB is
invalidated during the break step. Leave huge_ptep_get_and_clear()
unchanged because it is also used by teardown paths, where the deferred
flush is sufficient.

Fixes: fb396bb459c1 ("arm64/hugetlb: Drop TLB flush from get_clear_flush()")
Assisted-by: LLM
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
---
An instrumented QEMU detected the missing break-step TLBI on an unpatched
kernel and none with this change. A fork() control exercising
huge_ptep_set_wrprotect() remained clean. No user-visible failure was
reproduced.

The QEMU checker was exercised with 4K and 64K base-page kernels. The
patched kernel passed the LTP hugetlb tests with both -cpu max and -cpu
cortex-a72 (16 TPASS and no failures).

Testing on Neoverse N1 hardware would be welcome, as it can use the
contiguous hint and can be configured to report TLB conflicts.

 arch/arm64/mm/hugetlbpage.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 8e799c1fe0aa..bb53a04b73b2 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -517,6 +517,11 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
 {
 	unsigned long psize = huge_page_size(hstate_vma(vma));
+	pte_t pte = __ptep_get(ptep);
+
+	/* The break step for contiguous PTEs must include the TLB flush. */
+	if (pte_cont(pte))
+		return huge_ptep_clear_flush(vma, addr, ptep);
 
 	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_2645198)) {
 		/*
@@ -524,7 +529,7 @@ pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr
 		 * when the permission changes from executable to non-executable
 		 * in cases where cpu is affected with errata #2645198.
 		 */
-		if (pte_user_exec(__ptep_get(ptep)))
+		if (pte_user_exec(pte))
 			return huge_ptep_clear_flush(vma, addr, ptep);
 	}
 	return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep, psize);

base-commit: 786262be6048deab760f68c8acc2c85607165894
-- 
2.53.0

Re: [PATCH] arm64: hugetlb: fix BBM for mprotect() on contiguous PTEs

[Anshuman Khandual]

On Tue, Sep 01, 2026 at 03:18:23PM +0200, Karl Mehltretter wrote:
> huge_ptep_modify_prot_start() clears a hugetlb entry before changing its
> permissions. For contiguous PTE mappings, break-before-make (BBM)
> requires a TLB invalidation after clearing the set and before making any
> entry valid again.

Agreed.

> 
> Commit fb396bb459c1 ("arm64/hugetlb: Drop TLB flush from
> get_clear_flush()") removed this invalidation, relying on the deferred
> flush from the core code. Commit 410982303772 ("arm64: hugetlb: Restore
> TLB invalidation for BBM on contiguous ptes") restored it for
> huge_ptep_set_{access_flags,wrprotect}(), since a deferred flush is too
> late for the break step. The modify-prot path has the same problem.

The commit 410982303772 ("arm64: hugetlb: Restore TLB invalidation for BBM
on contiguous ptes") should also have fixed huge_ptep_modify_prot_start()
while at it ?

> 
> Use huge_ptep_clear_flush() for contiguous entries so that the TLB is
> invalidated during the break step. Leave huge_ptep_get_and_clear()
> unchanged because it is also used by teardown paths, where the deferred
> flush is sufficient.

Makes sense.

> 
> Fixes: fb396bb459c1 ("arm64/hugetlb: Drop TLB flush from get_clear_flush()")

Probably OK as the earlier commit 410982303772 also blames the same.

> Assisted-by: LLM

So you did not actually see this problem on a system ?

> Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
> ---
> An instrumented QEMU detected the missing break-step TLBI on an unpatched
> kernel and none with this change. A fork() control exercising
> huge_ptep_set_wrprotect() remained clean. No user-visible failure was
> reproduced.

No user-visiable failure was reported even without this change ?
Missing TLBI as seen in QEMU is the only clue here ?

> 
> The QEMU checker was exercised with 4K and 64K base-page kernels. The
> patched kernel passed the LTP hugetlb tests with both -cpu max and -cpu
> cortex-a72 (16 TPASS and no failures).
> 
> Testing on Neoverse N1 hardware would be welcome, as it can use the
> contiguous hint and can be configured to report TLB conflicts.

But you have not seen any real TLB conflicts even on custom QEMU ?

> 
>  arch/arm64/mm/hugetlbpage.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
> index 8e799c1fe0aa..bb53a04b73b2 100644
> --- a/arch/arm64/mm/hugetlbpage.c
> +++ b/arch/arm64/mm/hugetlbpage.c
> @@ -517,6 +517,11 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
>  pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
>  {
>  	unsigned long psize = huge_page_size(hstate_vma(vma));
> +	pte_t pte = __ptep_get(ptep);
> +
> +	/* The break step for contiguous PTEs must include the TLB flush. */

Probably helpful to mention here that subsequent huge_ptep_get_and_clearI()
in the function depends on deferred TLB flush mechanism which would not be
accurate for contig HugeTLB pages.

> +	if (pte_cont(pte))
> +		return huge_ptep_clear_flush(vma, addr, ptep);
>  
>  	if (alternative_has_cap_unlikely(ARM64_WORKAROUND_2645198)) {
>  		/*
> @@ -524,7 +529,7 @@ pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long addr
>  		 * when the permission changes from executable to non-executable
>  		 * in cases where cpu is affected with errata #2645198.
>  		 */
> -		if (pte_user_exec(__ptep_get(ptep)))
> +		if (pte_user_exec(pte))
>  			return huge_ptep_clear_flush(vma, addr, ptep);
>  	}
>  	return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep, psize);
> 
> base-commit: 786262be6048deab760f68c8acc2c85607165894
> -- 
> 2.53.0

Overall LGTM but will need some more testing.