[PATCH 0/3] mm: fix PTE table overruns with folded PMDs

[PATCH 0/3] mm: fix PTE table overruns with folded PMDs

[David Hildenbrand (Arm)]

Fix 3 pmd_addr_end() abusers that can result in PTE table overruns with
folded PMDs. Let's factor out a helper from a place where we
recently stumbled over something related, and use it to fix these 3
cases.

I'd call these more urgent because they can corrupt memory, although
folded PMDs are uncommon.

I'm planning on picking all 3 up.

Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
David Hildenbrand (Arm) (3):
      mm/userfaultfd: fix PTE table overruns in UFFDIO_MOVE
      mm/rmap: fix PTE table overruns during batched unmap
      mm/rmap: fix PTE table overruns during batched reference checks

 include/linux/pgtable.h | 24 ++++++++++++++++++++++++
 mm/memory.c             |  7 +------
 mm/rmap.c               |  4 ++--
 mm/userfaultfd.c        |  4 ++--
 4 files changed, 29 insertions(+), 10 deletions(-)

---

base-commit: d369ba5a46c65c54b94c94713ca1fb4ad6a87ebf

change-id: 20261111-pte_table_clamped_end-69bbcaf44fe7

--

Cheers,

David

[PATCH 1/3] mm/userfaultfd: fix PTE table overruns in UFFDIO_MOVE

[David Hildenbrand (Arm)]

With a folded PMD, pmd_addr_end() simply returns the supplied end.
move_present_ptes() can therefore walk past the source or destination
PTE table during UFFDIO_MOVE.

pmd_addr_end() is only intended for page table traversal, in patterns
like "next = pmd_addr_end(addr, end);". With folded PMDs, a higher level
has already clamped the range to the PTE table.

Folded PMDs are uncommon, but can occur even on arm64, for example
with 64K pages and a 42-bit virtual address space.

Extract the calculation from folio_pte_range() into
pte_table_clamped_end() and use it to clamp both ranges. This preserves
the wraparound handling from commit f681d4580d5c ("mm: fix address
wraparound when scanning large folio PTEs").

Fixes: 50944692052b ("userfaultfd: opportunistic TLB-flush batching for present pages in MOVE")
Cc: stable@vger.kernel.org
Reported-by: Chris Mason <mason@kernel.org>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
 include/linux/pgtable.h | 24 ++++++++++++++++++++++++
 mm/memory.c             |  7 +------
 mm/userfaultfd.c        |  4 ++--
 3 files changed, 27 insertions(+), 8 deletions(-)

diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 780fe849ff8b7..66d369870a740 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -12,6 +12,7 @@
 #ifdef CONFIG_MMU
 
 #include <linux/mm_types.h>
+#include <linux/minmax.h>
 #include <linux/bug.h>
 #include <linux/errno.h>
 #include <asm-generic/pgtable_uffd.h>
@@ -1518,6 +1519,29 @@ static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio)
 })
 #endif
 
+/**
+ * pte_table_clamped_end() - Limit a range to a single PTE table.
+ * @addr: Start address.
+ * @end: Exclusive end address.
+ *
+ * Only use pmd_addr_end() when traversing page tables, in a pattern like::
+ *
+ *     next = pmd_addr_end(addr, end);
+ *
+ * Higher levels already bound the range when PMDs are folded. Use this
+ * helper elsewhere to clamp the range regardless of PMD folding.
+ *
+ * Return: The exclusive end of the PTE table containing @addr, capped at @end.
+ */
+static inline unsigned long pte_table_clamped_end(unsigned long addr,
+		unsigned long end)
+{
+	const unsigned long pmd_end = (addr & PMD_MASK) + PMD_SIZE;
+
+	/* Compare inclusive ends to avoid wraparounds. */
+	return min(pmd_end - 1, end - 1) + 1;
+}
+
 /*
  * When walking page tables, we usually want to skip any p?d_none entries;
  * and any p?d_bad entries - reporting the error before resetting to none.
diff --git a/mm/memory.c b/mm/memory.c
index ec3d9d99ccee2..ee792a2c0ccea 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4959,13 +4959,8 @@ static void folio_pte_range(const struct folio *folio,
 {
 	/* Stay within the page table and VMA boundaries. */
 	const unsigned long pmd_start = addr & PMD_MASK;
-	const unsigned long pmd_end = pmd_start + PMD_SIZE;
 	const unsigned long lower = max(pmd_start, vma->vm_start);
-	/*
-	 * We cannot use pmd_addr_end(), as it does not clamp reliably. To
-	 * avoid wraparounds, compute using the inclusive end.
-	 */
-	const unsigned long upper = min(pmd_end - 1, vma->vm_end - 1) + 1;
+	const unsigned long upper = pte_table_clamped_end(addr, vma->vm_end);
 	const unsigned long offset = idx << PAGE_SHIFT;
 
 	/*
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 169f681f9849b..57650e6f040d1 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -1266,8 +1266,8 @@ static long move_present_ptes(struct mm_struct *mm,
 	unsigned long src_start = src_addr;
 	unsigned long src_end;
 
-	len = pmd_addr_end(dst_addr, dst_addr + len) - dst_addr;
-	src_end = pmd_addr_end(src_addr, src_addr + len);
+	len = pte_table_clamped_end(dst_addr, dst_addr + len) - dst_addr;
+	src_end = pte_table_clamped_end(src_addr, src_addr + len);
 	flush_cache_range(src_vma, src_addr, src_end);
 	double_pt_lock(dst_ptl, src_ptl);
 

-- 
2.43.0

Re: [PATCH 1/3] mm/userfaultfd: fix PTE table overruns in UFFDIO_MOVE

[Baolin Wang]

On 11/12/26 5:59 AM, David Hildenbrand (Arm) wrote:
> With a folded PMD, pmd_addr_end() simply returns the supplied end.
> move_present_ptes() can therefore walk past the source or destination
> PTE table during UFFDIO_MOVE.
> 
> pmd_addr_end() is only intended for page table traversal, in patterns
> like "next = pmd_addr_end(addr, end);". With folded PMDs, a higher level
> has already clamped the range to the PTE table.
> 
> Folded PMDs are uncommon, but can occur even on arm64, for example
> with 64K pages and a 42-bit virtual address space.
> 
> Extract the calculation from folio_pte_range() into
> pte_table_clamped_end() and use it to clamp both ranges. This preserves
> the wraparound handling from commit f681d4580d5c ("mm: fix address
> wraparound when scanning large folio PTEs").
> 
> Fixes: 50944692052b ("userfaultfd: opportunistic TLB-flush batching for present pages in MOVE")
> Cc: stable@vger.kernel.org
> Reported-by: Chris Mason <mason@kernel.org>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> ---
>   include/linux/pgtable.h | 24 ++++++++++++++++++++++++
>   mm/memory.c             |  7 +------
>   mm/userfaultfd.c        |  4 ++--
>   3 files changed, 27 insertions(+), 8 deletions(-)
> 
> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> index 780fe849ff8b7..66d369870a740 100644
> --- a/include/linux/pgtable.h
> +++ b/include/linux/pgtable.h
> @@ -12,6 +12,7 @@
>   #ifdef CONFIG_MMU
>   
>   #include <linux/mm_types.h>
> +#include <linux/minmax.h>
>   #include <linux/bug.h>
>   #include <linux/errno.h>
>   #include <asm-generic/pgtable_uffd.h>
> @@ -1518,6 +1519,29 @@ static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio)
>   })
>   #endif
>   
> +/**
> + * pte_table_clamped_end() - Limit a range to a single PTE table.
> + * @addr: Start address.
> + * @end: Exclusive end address.
> + *
> + * Only use pmd_addr_end() when traversing page tables, in a pattern like::
> + *
> + *     next = pmd_addr_end(addr, end);
> + *
> + * Higher levels already bound the range when PMDs are folded. Use this
> + * helper elsewhere to clamp the range regardless of PMD folding.
> + *
> + * Return: The exclusive end of the PTE table containing @addr, capped at @end.
> + */
> +static inline unsigned long pte_table_clamped_end(unsigned long addr,
> +		unsigned long end)
> +{
> +	const unsigned long pmd_end = (addr & PMD_MASK) + PMD_SIZE;
> +
> +	/* Compare inclusive ends to avoid wraparounds. */
> +	return min(pmd_end - 1, end - 1) + 1;
> +}

Thanks. The helper makes the code more readable.
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>

[PATCH 2/3] mm/rmap: fix PTE table overruns during batched unmap

[David Hildenbrand (Arm)]

With a folded PMD, pmd_addr_end() simply returns the supplied end.
folio_unmap_pte_batch() can therefore walk past the PTE table when
unmapping a large folio.

Folded PMDs are uncommon, but can occur even on arm64, for example
with 64K pages and a 42-bit virtual address space.

Use pte_table_clamped_end() to clamp the range to the PTE table and
VMA boundaries.

Fixes: 354dffd29575 ("mm: support batched unmap for lazyfree large folios during reclamation")
Cc: stable@vger.kernel.org
Reported-by: Chris Mason <mason@kernel.org>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
 mm/rmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/rmap.c b/mm/rmap.c
index 0faef46806e5c..491a954b7fa44 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1978,7 +1978,7 @@ static inline unsigned int folio_unmap_pte_batch(struct folio *folio,
 		return 1;
 
 	/* We may only batch within a single VMA and a single page table. */
-	end_addr = pmd_addr_end(addr, vma->vm_end);
+	end_addr = pte_table_clamped_end(addr, vma->vm_end);
 	max_nr = (end_addr - addr) >> PAGE_SHIFT;
 
 	/* We only support lazyfree or file folios batching for now ... */

-- 
2.43.0

Re: [PATCH 2/3] mm/rmap: fix PTE table overruns during batched unmap

[Baolin Wang]

On 11/12/26 6:00 AM, David Hildenbrand (Arm) wrote:
> With a folded PMD, pmd_addr_end() simply returns the supplied end.
> folio_unmap_pte_batch() can therefore walk past the PTE table when
> unmapping a large folio.
> 
> Folded PMDs are uncommon, but can occur even on arm64, for example
> with 64K pages and a 42-bit virtual address space.
> 
> Use pte_table_clamped_end() to clamp the range to the PTE table and
> VMA boundaries.
> 
> Fixes: 354dffd29575 ("mm: support batched unmap for lazyfree large folios during reclamation")
> Cc: stable@vger.kernel.org
> Reported-by: Chris Mason <mason@kernel.org>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> ---

LGTM.
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>

[PATCH 3/3] mm/rmap: fix PTE table overruns during batched reference checks

[David Hildenbrand (Arm)]

With a folded PMD, pmd_addr_end() simply returns the supplied end.
folio_referenced_one() can therefore walk past the PTE table when
checking references to a large folio.

Folded PMDs are uncommon, but can occur even on arm64, for example
with 64K pages and a 42-bit virtual address space.

Use pte_table_clamped_end() to clamp the range to the PTE table and
VMA boundaries.

Fixes: 52e054f71840 ("mm: rmap: support batched checks of the references for large folios")
Cc: stable@vger.kernel.org
Reported-by: Chris Mason <mason@kernel.org>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
---
 mm/rmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/rmap.c b/mm/rmap.c
index 491a954b7fa44..65646ed6fc453 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -973,7 +973,7 @@ static bool folio_referenced_one(struct folio *folio,
 			pteval = ptep_get(pvmw.pte);
 
 		if (pvmw.pte && pte_present(pteval) && folio_test_large(folio)) {
-			const unsigned long end_addr = pmd_addr_end(address, vma->vm_end);
+			const unsigned long end_addr = pte_table_clamped_end(address, vma->vm_end);
 			const unsigned int max_nr = (end_addr - address) >> PAGE_SHIFT;
 
 			nr = folio_pte_batch(folio, pvmw.pte, pteval, max_nr);

-- 
2.43.0

Re: [PATCH 3/3] mm/rmap: fix PTE table overruns during batched reference checks

[Baolin Wang]

On 11/12/26 6:00 AM, David Hildenbrand (Arm) wrote:
> With a folded PMD, pmd_addr_end() simply returns the supplied end.
> folio_referenced_one() can therefore walk past the PTE table when
> checking references to a large folio.
> 
> Folded PMDs are uncommon, but can occur even on arm64, for example
> with 64K pages and a 42-bit virtual address space.
> 
> Use pte_table_clamped_end() to clamp the range to the PTE table and
> VMA boundaries.
> 
> Fixes: 52e054f71840 ("mm: rmap: support batched checks of the references for large folios")
> Cc: stable@vger.kernel.org
> Reported-by: Chris Mason <mason@kernel.org>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> ---

LGTM.
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>

BTW, the 'end' calculation for the batch operation in 
lru_gen_look_around() could also use the pte_table_clamped_end() helper. 
But this can be a follow-up.

end = min(addr | ~PMD_MASK, vma->vm_end - 1) + 1;

Re: [PATCH 3/3] mm/rmap: fix PTE table overruns during batched reference checks

[David Hildenbrand (Arm)]

On 11/12/26 08:25, Baolin Wang wrote:
> 
> 
> On 11/12/26 6:00 AM, David Hildenbrand (Arm) wrote:
>> With a folded PMD, pmd_addr_end() simply returns the supplied end.
>> folio_referenced_one() can therefore walk past the PTE table when
>> checking references to a large folio.
>>
>> Folded PMDs are uncommon, but can occur even on arm64, for example
>> with 64K pages and a 42-bit virtual address space.
>>
>> Use pte_table_clamped_end() to clamp the range to the PTE table and
>> VMA boundaries.
>>
>> Fixes: 52e054f71840 ("mm: rmap: support batched checks of the references for
>> large folios")
>> Cc: stable@vger.kernel.org
>> Reported-by: Chris Mason <mason@kernel.org>
>> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
>> ---
> 
> LGTM.
> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> 
> BTW, the 'end' calculation for the batch operation in lru_gen_look_around()
> could also use the pte_table_clamped_end() helper. But this can be a follow-up.
> 
> end = min(addr | ~PMD_MASK, vma->vm_end - 1) + 1;

Cool, can you send a cleanup patch? I'd pick it up once these patches are upstream.

I'll move these into core.git today, so you can send against that for-next later
today.

-- 
Cheers,

David

Re: [PATCH 3/3] mm/rmap: fix PTE table overruns during batched reference checks

[Baolin Wang]

On 11/12/26 4:29 PM, David Hildenbrand (Arm) wrote:
> On 11/12/26 08:25, Baolin Wang wrote:
>>
>>
>> On 11/12/26 6:00 AM, David Hildenbrand (Arm) wrote:
>>> With a folded PMD, pmd_addr_end() simply returns the supplied end.
>>> folio_referenced_one() can therefore walk past the PTE table when
>>> checking references to a large folio.
>>>
>>> Folded PMDs are uncommon, but can occur even on arm64, for example
>>> with 64K pages and a 42-bit virtual address space.
>>>
>>> Use pte_table_clamped_end() to clamp the range to the PTE table and
>>> VMA boundaries.
>>>
>>> Fixes: 52e054f71840 ("mm: rmap: support batched checks of the references for
>>> large folios")
>>> Cc: stable@vger.kernel.org
>>> Reported-by: Chris Mason <mason@kernel.org>
>>> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
>>> ---
>>
>> LGTM.
>> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
>>
>> BTW, the 'end' calculation for the batch operation in lru_gen_look_around()
>> could also use the pte_table_clamped_end() helper. But this can be a follow-up.
>>
>> end = min(addr | ~PMD_MASK, vma->vm_end - 1) + 1;
> 
> Cool, can you send a cleanup patch? I'd pick it up once these patches are upstream.
> 
> I'll move these into core.git today, so you can send against that for-next later
> today.

Sure.

Re: [PATCH 0/3] mm: fix PTE table overruns with folded PMDs

[David Hildenbrand]

On Wed, 11 Nov 2026 22:59:58 +0100, David Hildenbrand (Arm) wrote:
> Fix 3 pmd_addr_end() abusers that can result in PTE table overruns with
> folded PMDs. Let's factor out a helper from a place where we
> recently stumbled over something related, and use it to fix these 3
> cases.
> 
> I'd call these more urgent because they can corrupt memory, although
> folded PMDs are uncommon.
> 
> [...]

Applied, thanks!

tree: mm/core.git
branch: for-next-fixes

Applied patches will appear soon in mm-next, followed by linux-next.

Please report any outstanding bugs that were missed during review, so we
can either drop the patch series or decide how to fix it up.

It is encouraged to provide Acked-bys, Reviewed-bys, and Tested-bys even
though the patches have already been applied. If possible, patch trailers
will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. Further, this is not a guarantee that the
patches will go upstream. If in doubt, please check the listed branch.

[1/3] mm/userfaultfd: fix PTE table overruns in UFFDIO_MOVE
      https://git.kernel.org/mm/core/c/654b0d18
[2/3] mm/rmap: fix PTE table overruns during batched unmap
      https://git.kernel.org/mm/core/c/bd94d906
[3/3] mm/rmap: fix PTE table overruns during batched reference checks
      https://git.kernel.org/mm/core/c/dbedd8e6

-- 
Cheers,

David