[PATCH v7] mm/userfaultfd: detect VMA type change after copy retry in mfill_copy_folio_retry()

[PATCH v7] mm/userfaultfd: detect VMA type change after copy retry in mfill_copy_folio_retry()

[David Carlier]

mfill_copy_folio_retry() drops mmap_lock for the copy_from_user() call.
During this window, the VMA can be replaced with a different type (e.g.
hugetlb), making the caller's ops pointer stale.  Subsequent use of the
stale ops would dispatch into the wrong per-vma handlers.

Capture the VMA's ops via vma_uffd_ops() before dropping the lock and
compare against the current vma_uffd_ops() after re-acquiring it.
Return -EAGAIN if they differ so the operation can be retried.  This
avoids comparing against the caller's ops which may have been
overridden to anon_uffd_ops for MAP_PRIVATE file-backed mappings.

Fixes: 6ab703034f14 ("userfaultfd: mfill_atomic(): remove retry logic")
Reported-by: Usama Arif <usama.arif@linux.dev>
Closes: https://lore.kernel.org/all/20260410114809.3592720-1-usama.arif@linux.dev/
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: David Carlier <devnexen@gmail.com>
---
v7: (akpm review)
 - update Fixes: to the current mm-unstable hash
 - add Closes: link to Usama's report
 - drop "kernel crash" wording; no observed reproducer
 - align Reported-by address to usama.arif@linux.dev
 - carry Mike's Ack from v5

v6: capture ops via vma_uffd_ops() before dropping the lock so
    MAP_PRIVATE shmem (which overrides to anon_uffd_ops) no longer
    triggers spurious -EAGAIN (Usama).
    Drop unused ops parameter from mfill_copy_folio_retry().

v5: initial ops-compare approach.

Tested under virtme-ng (DEBUG_VM, LOCKDEP, PROVE_LOCKING):
  uffd-unit-tests: 67 pass, 0 skip, 0 fail
  uffd-stress {anon,shmem,shmem-private}: 4 bounces each, clean

 mm/userfaultfd.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 885da1e56466..180bad42fc79 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -443,8 +443,10 @@ static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr)
 	return ret;
 }
 
-static int mfill_copy_folio_retry(struct mfill_state *state, struct folio *folio)
+static int mfill_copy_folio_retry(struct mfill_state *state,
+				  struct folio *folio)
 {
+	const struct vm_uffd_ops *orig_ops = vma_uffd_ops(state->vma);
 	unsigned long src_addr = state->src_addr;
 	void *kaddr;
 	int err;
@@ -465,6 +467,14 @@ static int mfill_copy_folio_retry(struct mfill_state *state, struct folio *folio
 	if (err)
 		return err;
 
+	/*
+	 * The VMA type may have changed while the lock was dropped
+	 * (e.g. replaced with a hugetlb mapping), making the caller's
+	 * ops pointer stale.
+	 */
+	if (vma_uffd_ops(state->vma) != orig_ops)
+		return -EAGAIN;
+
 	err = mfill_establish_pmd(state);
 	if (err)
 		return err;
-- 
2.53.0