[PATCH v2] mm/hugetlb: fix swap entry corruption when clearing uffd-wp at fork()

[PATCH v2] mm/hugetlb: fix swap entry corruption when clearing uffd-wp at fork()

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

copy_hugetlb_page_range() clears the uffd-wp bit of migration and
hwpoison entries with huge_pte_clear_uffd_wp(), which operates on the
present-PTE bit position. Swap entries keep the uffd-wp state elsewhere
-- the migration branch reads and sets it with pte_swp_uffd_wp() and
pte_swp_mkuffd_wp() -- and the present-PTE position falls into the swap
payload. On x86-64 it lands in the inverted swap offset, where a
naturally-aligned hugetlb PFN always has the affected bit set, so the
clear advances the encoded PFN by two pages.

No userfaultfd needs to be involved: the clear is guarded only by the
child VMA not being uffd-wp registered, so a plain fork() with an
in-flight hugetlb migration entry (or a poisoned hugetlb page) corrupts
the entry copied into the child. Instrumenting the clear and forking
after MADV_HWPOISON on a 2MB anon hugetlb page shows:

  offset before=120e00
  offset after =120e02

The fallout is mostly latent: rmap walks match migration entries by
folio range and remove_migration_pte() rebuilds the PTE from the folio,
so a within-folio PFN skew heals once migration completes. But any path
that re-encodes the corrupted offset -- e.g. hugetlb_change_protection()
rewriting a writable migration entry via
make_readable_migration_entry(swp_offset(entry)) -- propagates it.

Migration entries legitimately carry uffd-wp, so clear it with
pte_swp_clear_uffd_wp(), matching copy_nonpresent_pte() and
move_huge_pte().

A hwpoison entry, on the other hand, never carries the uffd-wp bit: it
is installed fresh by make_hwpoison_entry() (try_to_unmap_one() does not
preserve uffd-wp on the hwpoison path) and hugetlb_change_protection()
leaves hwpoison entries untouched. There was nothing to clear there,
only the corruption, so drop the clear entirely.

Reported-by: Sashiko AI review <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260703140011.99E601F000E9@smtp.kernel.org/
Suggested-by: David Hildenbrand <david@kernel.org>
Fixes: bc70fbf269fd ("mm/hugetlb: handle uffd-wp during fork()")
Cc: stable@vger.kernel.org
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Assisted-by: Claude:claude-fable-5
---

Changes since v1 [1]:
  - Drop the clear on the hwpoison branch entirely rather than
    switching it to pte_swp_clear_uffd_wp(): a hugetlb hwpoison entry
    never carries the uffd-wp bit, so there is nothing to clear.
    (David Hildenbrand)

[1] https://lore.kernel.org/all/20260703161833.57416-1-kirill@shutemov.name/
 mm/hugetlb.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 571212b80835..bca2707d02e3 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -4917,8 +4917,12 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
 
 		softleaf = softleaf_from_pte(entry);
 		if (unlikely(softleaf_is_hwpoison(softleaf))) {
-			if (!userfaultfd_wp(dst_vma))
-				entry = huge_pte_clear_uffd_wp(entry);
+			/*
+			 * A hwpoison entry never carries the uffd-wp bit: it is
+			 * installed fresh by make_hwpoison_entry() and
+			 * hugetlb_change_protection() leaves it untouched, so
+			 * there is nothing to clear for the child.
+			 */
 			set_huge_pte_at(dst, addr, dst_pte, entry, sz);
 		} else if (unlikely(softleaf_is_migration(softleaf))) {
 			bool uffd_wp = pte_swp_uffd_wp(entry);
@@ -4936,7 +4940,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
 				set_huge_pte_at(src, addr, src_pte, entry, sz);
 			}
 			if (!userfaultfd_wp(dst_vma))
-				entry = huge_pte_clear_uffd_wp(entry);
+				entry = pte_swp_clear_uffd_wp(entry);
 			set_huge_pte_at(dst, addr, dst_pte, entry, sz);
 		} else if (unlikely(pte_is_marker(entry))) {
 			const pte_marker marker = copy_pte_marker(softleaf, dst_vma);

base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
-- 
2.54.0

Re: [PATCH v2] mm/hugetlb: fix swap entry corruption when clearing uffd-wp at fork()

[David Hildenbrand (Arm)]

On 7/8/26 11:01, Kiryl Shutsemau wrote:
> From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
> 
> copy_hugetlb_page_range() clears the uffd-wp bit of migration and
> hwpoison entries with huge_pte_clear_uffd_wp(), which operates on the
> present-PTE bit position. Swap entries keep the uffd-wp state elsewhere
> -- the migration branch reads and sets it with pte_swp_uffd_wp() and
> pte_swp_mkuffd_wp() -- and the present-PTE position falls into the swap
> payload. On x86-64 it lands in the inverted swap offset, where a
> naturally-aligned hugetlb PFN always has the affected bit set, so the
> clear advances the encoded PFN by two pages.
> 
> No userfaultfd needs to be involved: the clear is guarded only by the
> child VMA not being uffd-wp registered, so a plain fork() with an
> in-flight hugetlb migration entry (or a poisoned hugetlb page) corrupts
> the entry copied into the child. Instrumenting the clear and forking
> after MADV_HWPOISON on a 2MB anon hugetlb page shows:
> 
>   offset before=120e00
>   offset after =120e02
> 
> The fallout is mostly latent: rmap walks match migration entries by
> folio range and remove_migration_pte() rebuilds the PTE from the folio,
> so a within-folio PFN skew heals once migration completes. But any path
> that re-encodes the corrupted offset -- e.g. hugetlb_change_protection()
> rewriting a writable migration entry via
> make_readable_migration_entry(swp_offset(entry)) -- propagates it.
> 
> Migration entries legitimately carry uffd-wp, so clear it with
> pte_swp_clear_uffd_wp(), matching copy_nonpresent_pte() and
> move_huge_pte().
> 
> A hwpoison entry, on the other hand, never carries the uffd-wp bit: it
> is installed fresh by make_hwpoison_entry() (try_to_unmap_one() does not
> preserve uffd-wp on the hwpoison path) and hugetlb_change_protection()
> leaves hwpoison entries untouched. There was nothing to clear there,
> only the corruption, so drop the clear entirely.
> 
> Reported-by: Sashiko AI review <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/all/20260703140011.99E601F000E9@smtp.kernel.org/
> Suggested-by: David Hildenbrand <david@kernel.org>
> Fixes: bc70fbf269fd ("mm/hugetlb: handle uffd-wp during fork()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
> Assisted-by: Claude:claude-fable-5
> ---

Thanks!

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David