Linux kernel -stable discussions
 help / color / mirror / Atom feed
* [PATCH v2 0/9] userfaultfd: assorted fixes (part 1)
@ 2026-11-16 20:02 Tal Zussman
  2026-11-16 20:02 ` [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings Tal Zussman
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Tal Zussman @ 2026-11-16 20:02 UTC (permalink / raw)
  To: Andrew Morton, Mike Rapoport, Peter Xu, James Houghton,
	Andrea Arcangeli, Suren Baghdasaryan, Balbir Singh,
	Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka, Jann Horn,
	Pedro Falcato, David Hildenbrand, Michal Hocko,
	Michał Mirosław, Andrei Vagin, Muhammad Usama Anjum,
	Stephen Rothwell, Arnd Bergmann, Zi Yan, Matthew Brost,
	Joshua Hahn, Byungchul Park, Gregory Price, Ying Huang,
	Alistair Popple, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
	Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau, Muchun Song,
	Oscar Salvador, Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, stable, Tal Zussman,
	Breno Leitao, Thomas Hellström, Lyude Paul, Danilo Krummrich,
	Felix Kuehling, Jason Gunthorpe, Leon Romanovsky, dri-devel,
	intel-xe, nouveau, amd-gfx

This is a first batch of userfaultfd fixes, with more to come.

These were found by a mix of regular old LLM review and by having an LLM
write a TLA+ model of uffd interactions with PTE bits and other state
and seeing what issues fell out.

Most of these fix places where mm code loses uffd-wp or RWP protection
while moving or migrating an entry. Userspace then either misses writes
and accesses, or gets pages reported that nobody touched.
There is also a hang in UFFDIO_MOVE, and UFFDIO_ZEROPAGE modifying a
file through a private mapping.

Patches 1-2:   UFFDIO_ZEROPAGE and UFFDIO_MOVE fixes, cc stable
Patches 3-5:   PAGEMAP_SCAN write-protect vs. migration and
               device-private entries, patch 3 cc stable
Patch 6:       UFFDIO_MOVE of the huge zero page from an RWP source
Patch 7:       mremap() losing uffd state when a move fails halfway
Patch 8:       migrate_vma losing uffd state of a zero page
Patch 9:       UFFDIO_UNREGISTER with a misaligned hugetlb end

A few of the fixes change what userspace sees:

 - UFFDIO_ZEROPAGE on a private file mapping no longer fails with
   -EEXIST where the file already has a page, on an mm that cannot use
   the zero page (patch 1)
 - UFFDIO_MOVE returns -EEXIST for any huge destination PMD, not only
   for a present THP (patch 2)
 - UFFDIO_UNREGISTER with a misaligned hugetlb end fails before
   unregistering anything (patch 9)

Patch 6 has little user impact on its own. It is included because,
together with patch 7, it allows removing clear_uffd_wp_pmd().

Patch 7 adds spurious write faults on shmem and hugetlb after mremap():
when dropping uffd protection there, change_protection() makes writable
pages that were never protected read-only. UFFDIO_UNREGISTER and
UFFDIO_WRITEPROTECT already behave this way and lead to the same
spurious faults. This will be fixed in change_protection() in a separate
series.

Based on mm-new.

---
Changes in v2:
- Collect tags (Thanks Kiryl, Lorenzo, Gregory, Alistair!)
- Patch 2: say "softleaf entry" in the commit message, per Lorenzo
- Patch 3: add softleaf_is_writable() and softleaf_mkreadable() to
  leafops.h and rename the helpers to say softleaf, per Lorenzo
- Patch 3: add Reported-by and Closes for Breno's report, and cc stable
- Patch 4: explain in the commit message how PAGEMAP_SCAN set the uffd
  bit on writable migration entries, per Lorenzo
- Patch 5: use pmd_is_valid_softleaf() rather than open-coding it, per
  Lorenzo
- Patch 7: say "write faults" rather than "minor faults" in the commit
  message, per Kiryl
- Patch 7: trim the commit message, per Lorenzo and Gregory
- Patch 8: trim the commit message, per Gregory
- Patch 8: Cc the GPU driver maintainers and lists, per Gregory and
  Alistair
- Patch 9: use IS_ALIGNED() and a local variable for the VMA's ctx, and
  shorten the comment, per Lorenzo
- Drop the three guard region patches, per Lorenzo
- Link to v1: https://patch.msgid.link/20261111-uffd-state-fixes-v1-0-00713d368976@columbia.edu

---
Tal Zussman (9):
      userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings
      userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD
      fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect
      mm/migrate: keep the uffd bit when restoring a writable migration entry
      fs/proc/task_mmu: write-protect device-private THPs in PAGEMAP_SCAN
      mm/huge_memory: build the moved huge zero PMD from the destination VMA
      mm/mremap: resolve uffd state when the context is dropped, not during the move
      mm/migrate_device: do not migrate a uffd-protected zero page
      mm/userfaultfd: check hugetlb end alignment in UFFDIO_UNREGISTER pre-scan

 fs/proc/task_mmu.c      | 42 +++++++++++++++++++++++++++---
 include/linux/leafops.h | 51 ++++++++++++++++++++++++++++++++++++
 mm/huge_memory.c        | 31 +++-------------------
 mm/hugetlb.c            | 27 +------------------
 mm/migrate.c            |  2 +-
 mm/migrate_device.c     | 23 +++++++++++++++--
 mm/mremap.c             | 46 +--------------------------------
 mm/userfaultfd.c        | 69 ++++++++++++++++++++++++-------------------------
 8 files changed, 150 insertions(+), 141 deletions(-)
---
base-commit: 67f3b04b277373438f17300e056306adbd9edf7b
change-id: 20261106-uffd-state-fixes-d2111e2abd48

Best regards,
--  
Tal Zussman <tz2294@columbia.edu>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings
  2026-11-16 20:02 [PATCH v2 0/9] userfaultfd: assorted fixes (part 1) Tal Zussman
@ 2026-11-16 20:02 ` Tal Zussman
  2026-11-16 20:15   ` David Hildenbrand (Arm)
  2026-11-16 20:16   ` David Hildenbrand (Arm)
  2026-11-16 20:02 ` [PATCH v2 2/9] userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD Tal Zussman
  2026-11-16 20:02 ` [PATCH v2 3/9] fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect Tal Zussman
  2 siblings, 2 replies; 8+ messages in thread
From: Tal Zussman @ 2026-11-16 20:02 UTC (permalink / raw)
  To: Andrew Morton, Mike Rapoport, Peter Xu, James Houghton,
	Andrea Arcangeli, Suren Baghdasaryan, Balbir Singh,
	Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka, Jann Horn,
	Pedro Falcato, David Hildenbrand, Michal Hocko,
	Michał Mirosław, Andrei Vagin, Muhammad Usama Anjum,
	Stephen Rothwell, Arnd Bergmann, Zi Yan, Matthew Brost,
	Joshua Hahn, Byungchul Park, Gregory Price, Ying Huang,
	Alistair Popple, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
	Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau, Muchun Song,
	Oscar Salvador, Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, stable, Tal Zussman

If the mm cannot map the shared zero page, UFFDIO_ZEROPAGE falls back to
mfill_atomic_pte_zeroed_folio(), which allocates a zeroed folio and
installs it through vma_uffd_ops(), no matter what kind of mapping it
is. An s390 KVM guest with storage keys or protected virtualization is
such an mm.

For a MAP_PRIVATE shmem mapping, that adds the zeroed folio to the page
cache of the file. That is wrong: a private mapping must never modify
the file it maps, and the file might even be mapped from a read-only fd.

The ioctl can also fail where it should succeed. A missing fault is only
delivered for a hole in the file, but UFFDIO_ZEROPAGE can be called on
an address that did not fault, or can race with a write to the file. If
the file has a page at that offset by then, inserting the zeroed folio
fails and the ioctl returns -EEXIST although nothing is mapped at the
destination address.

What we want is what UFFDIO_ZEROPAGE does on a private mapping when the
shared zero page can be used, and what UFFDIO_COPY does: only check that
nothing is mapped at the destination address, map zeroed memory there,
and leave the page cache alone. The zeroed folio path did that before it
was moved to vm_uffd_ops.

To fix it, pick the ops in the common folio path, as
mfill_atomic_pte_copy() did for UFFDIO_COPY: anon_uffd_ops for a private
mapping, the ops of the VMA otherwise. That makes
mfill_atomic_pte_copy() and mfill_atomic_pte_zeroed_folio() redundant,
so remove them and rename __mfill_atomic_pte() to
mfill_atomic_pte_folio().

UFFDIO_ZEROPAGE no longer modifies the file, and only fails with -EEXIST
if something is already mapped at the destination address.

Fixes: f74991b4e383 ("shmem, userfaultfd: implement shmem uffd operations using vm_uffd_ops")
Cc: stable@vger.kernel.org
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
 mm/userfaultfd.c | 46 ++++++++++++++++------------------------------
 1 file changed, 16 insertions(+), 30 deletions(-)

diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index d65184d9a480..61a712e1f9f5 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -531,15 +531,27 @@ static int mfill_copy_folio_retry(struct mfill_state *mfill_state,
 	return 0;
 }
 
-static int __mfill_atomic_pte(struct mfill_state *state,
-			      const struct vm_uffd_ops *ops)
+static int mfill_atomic_pte_folio(struct mfill_state *state)
 {
+	const struct vm_uffd_ops *ops = vma_uffd_ops(state->vma);
 	unsigned long dst_addr = state->dst_addr;
 	unsigned long src_addr = state->src_addr;
 	uffd_flags_t flags = state->flags;
 	struct folio *folio;
 	int ret;
 
+	/*
+	 * The normal page fault path for a MAP_PRIVATE mapping in a
+	 * file-backed VMA will invoke the fault, fill the hole in the file and
+	 * COW it right away. The result generates plain anonymous memory.
+	 * So when we are asked to fill a hole in a MAP_PRIVATE mapping, we'll
+	 * generate anonymous memory directly without actually filling the
+	 * hole. For the MAP_PRIVATE case the robustness check only happens in
+	 * the pagetable (to verify it's still none) and not in the page cache.
+	 */
+	if (!(state->vma->vm_flags & VM_SHARED))
+		ops = &anon_uffd_ops;
+
 	if (!ops) {
 		VM_WARN_ONCE(1, "UFFDIO_COPY for unsupported VMA");
 		return -EOPNOTSUPP;
@@ -597,32 +609,6 @@ static int __mfill_atomic_pte(struct mfill_state *state,
 	return ret;
 }
 
-static int mfill_atomic_pte_copy(struct mfill_state *state)
-{
-	const struct vm_uffd_ops *ops = vma_uffd_ops(state->vma);
-
-	/*
-	 * The normal page fault path for a MAP_PRIVATE mapping in a
-	 * file-backed VMA will invoke the fault, fill the hole in the file and
-	 * COW it right away. The result generates plain anonymous memory.
-	 * So when we are asked to fill a hole in a MAP_PRIVATE mapping, we'll
-	 * generate anonymous memory directly without actually filling the
-	 * hole. For the MAP_PRIVATE case the robustness check only happens in
-	 * the pagetable (to verify it's still none) and not in the page cache.
-	 */
-	if (!(state->vma->vm_flags & VM_SHARED))
-		ops = &anon_uffd_ops;
-
-	return __mfill_atomic_pte(state, ops);
-}
-
-static int mfill_atomic_pte_zeroed_folio(struct mfill_state *state)
-{
-	const struct vm_uffd_ops *ops = vma_uffd_ops(state->vma);
-
-	return __mfill_atomic_pte(state, ops);
-}
-
 static int mfill_atomic_pte_zeropage(struct mfill_state *state)
 {
 	struct vm_area_struct *dst_vma = state->vma;
@@ -634,7 +620,7 @@ static int mfill_atomic_pte_zeropage(struct mfill_state *state)
 
 	if (mm_forbids_zeropage(dst_vma->vm_mm) ||
 	    (dst_vma->vm_flags & VM_SHARED))
-		return mfill_atomic_pte_zeroed_folio(state);
+		return mfill_atomic_pte_folio(state);
 
 	_dst_pte = pte_mkspecial(pfn_pte(zero_pfn(dst_addr),
 					 dst_vma->vm_page_prot));
@@ -927,7 +913,7 @@ static __always_inline ssize_t mfill_atomic_pte(struct mfill_state *state)
 	if (uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON))
 		return mfill_atomic_pte_poison(state);
 	if (uffd_flags_mode_is(flags, MFILL_ATOMIC_COPY))
-		return mfill_atomic_pte_copy(state);
+		return mfill_atomic_pte_folio(state);
 	if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE))
 		return mfill_atomic_pte_zeropage(state);
 

-- 
2.39.5


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 2/9] userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD
  2026-11-16 20:02 [PATCH v2 0/9] userfaultfd: assorted fixes (part 1) Tal Zussman
  2026-11-16 20:02 ` [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings Tal Zussman
@ 2026-11-16 20:02 ` Tal Zussman
  2026-11-16 20:18   ` David Hildenbrand (Arm)
  2026-11-16 20:02 ` [PATCH v2 3/9] fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect Tal Zussman
  2 siblings, 1 reply; 8+ messages in thread
From: Tal Zussman @ 2026-11-16 20:02 UTC (permalink / raw)
  To: Andrew Morton, Mike Rapoport, Peter Xu, James Houghton,
	Andrea Arcangeli, Suren Baghdasaryan, Balbir Singh,
	Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka, Jann Horn,
	Pedro Falcato, David Hildenbrand, Michal Hocko,
	Michał Mirosław, Andrei Vagin, Muhammad Usama Anjum,
	Stephen Rothwell, Arnd Bergmann, Zi Yan, Matthew Brost,
	Joshua Hahn, Byungchul Park, Gregory Price, Ying Huang,
	Alistair Popple, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
	Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau, Muchun Song,
	Oscar Salvador, Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, stable, Tal Zussman

move_pages() refuses to move into a destination PMD that already maps a
THP, but checks for that with pmd_trans_huge(), which is only true for a
present THP. A device-private THP or a THP under migration is a PMD
softleaf entry and is not refused.

The destination PMD is then treated as if it pointed to a PTE table. It
is not empty, so pte_alloc() leaves it alone, and move_pages_ptes()
fails to map a PTE table from it and returns -EAGAIN. move_pages()
retries on -EAGAIN, with both VMA read locks held. If the source is a
THP, it is split first for nothing.

For a THP under migration, the loop ends once the migration finishes.
A device-private THP can remain in device memory indefinitely, so the
caller of UFFDIO_MOVE hangs until it receives a fatal signal, and
anything in the mm that needs to write-lock one of the VMAs blocks along
with it. This requires a driver that migrates THPs to device memory.

To fix it, check the destination with pmd_is_huge(), which is true for
a present THP and for a PMD softleaf entry. UFFDIO_MOVE now fails with
-EEXIST in these cases, as it already does for a present THP.

Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI")
Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
Cc: stable@vger.kernel.org
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Tested-by: Gregory Price (Meta) <gourry@gourry.net>
Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
 mm/userfaultfd.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 61a712e1f9f5..ad9d243dc64b 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -2021,12 +2021,13 @@ static ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,
 
 		dst_pmdval = pmdp_get_lockless(dst_pmd);
 		/*
-		 * If the dst_pmd is mapped as THP don't override it and just
-		 * be strict. If dst_pmd changes into TPH after this check, the
-		 * move_pages_huge_pmd() will detect the change and retry
-		 * while move_pages_pte() will detect the change and fail.
+		 * If the dst_pmd is a huge leaf, present or not, don't
+		 * override it and just be strict. If dst_pmd changes into a
+		 * leaf after this check, move_pages_huge_pmd() will detect the
+		 * change and retry while move_pages_ptes() will fail to map
+		 * the PTE table and the next iteration will see the leaf.
 		 */
-		if (unlikely(pmd_trans_huge(dst_pmdval))) {
+		if (unlikely(pmd_is_huge(dst_pmdval))) {
 			err = -EEXIST;
 			break;
 		}

-- 
2.39.5


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH v2 3/9] fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect
  2026-11-16 20:02 [PATCH v2 0/9] userfaultfd: assorted fixes (part 1) Tal Zussman
  2026-11-16 20:02 ` [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings Tal Zussman
  2026-11-16 20:02 ` [PATCH v2 2/9] userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD Tal Zussman
@ 2026-11-16 20:02 ` Tal Zussman
  2026-11-16 20:21   ` David Hildenbrand (Arm)
  2 siblings, 1 reply; 8+ messages in thread
From: Tal Zussman @ 2026-11-16 20:02 UTC (permalink / raw)
  To: Andrew Morton, Mike Rapoport, Peter Xu, James Houghton,
	Andrea Arcangeli, Suren Baghdasaryan, Balbir Singh,
	Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka, Jann Horn,
	Pedro Falcato, David Hildenbrand, Michal Hocko,
	Michał Mirosław, Andrei Vagin, Muhammad Usama Anjum,
	Stephen Rothwell, Arnd Bergmann, Zi Yan, Matthew Brost,
	Joshua Hahn, Byungchul Park, Gregory Price, Ying Huang,
	Alistair Popple, Baolin Wang, Nico Pache, Ryan Roberts, Dev Jain,
	Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau, Muchun Song,
	Oscar Salvador, Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, Breno Leitao, stable,
	Tal Zussman

PAGEMAP_SCAN with PM_SCAN_WP_MATCHING write-protects pages in a VMA
registered for async uffd-wp, and sets the uffd bit on non-present
entries as they are. An entry with the uffd bit set must not be
writable, so that the page is write-protected once it is mapped again.
change_protection() therefore downgrades writable migration and
device-private entries to readable ones first, but PAGEMAP_SCAN does
not.

For a writable migration entry at the PTE level,
remove_migration_pte() restores a writable PTE without the uffd bit. So
if a page is under migration while PAGEMAP_SCAN write-protects it, the
write protection is lost. A user tracking writes with async uffd-wp
sees the page reported as written on the next scan although nothing
wrote to it.

With CONFIG_PAGE_TABLE_CHECK, setting the uffd bit on a writable
non-present PTE or PMD also triggers a warning.

To fix it, downgrade writable migration and device-private entries
before setting the uffd bit, as change_protection() does, at the PTE,
PMD, and hugetlb levels.

Add softleaf_is_writable() and softleaf_mkreadable() to leafops.h for
the downgrade. The open-coded copies in change_protection() can be
converted to them separately.

Fixes: 52526ca7fdb9 ("fs/proc/task_mmu: implement IOCTL to get and optionally clear info about PTEs")
Reported-by: Breno Leitao <leitao@debian.org>
Closes: https://lore.kernel.org/all/avNEgmvYdoukt6MS@gmail.com/
Cc: stable@vger.kernel.org
Tested-by: Gregory Price (Meta) <gourry@gourry.net>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
 fs/proc/task_mmu.c      | 40 +++++++++++++++++++++++++++++++++++---
 include/linux/leafops.h | 51 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 88 insertions(+), 3 deletions(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 164b4efc5e09..ee7f509d376e 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -2405,6 +2405,25 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p,
 	return categories;
 }
 
+/*
+ * An armed entry must not come back writable, so downgrade a writable
+ * migration or device-private entry before giving it the bit, as
+ * change_softleaf_pte() does.
+ */
+static pte_t set_uffd_wp_softleaf_pte(pte_t pte)
+{
+	softleaf_t entry = softleaf_from_pte(pte);
+	pte_t newpte;
+
+	if (!softleaf_is_writable(entry))
+		return pte_swp_mkuffd(pte);
+
+	newpte = softleaf_to_pte(softleaf_mkreadable(entry));
+	if (pte_swp_soft_dirty(pte))
+		newpte = pte_swp_mksoft_dirty(newpte);
+	return pte_swp_mkuffd(newpte);
+}
+
 static void make_uffd_wp_pte(struct vm_area_struct *vma,
 			     unsigned long addr, pte_t *pte, pte_t ptent)
 {
@@ -2418,7 +2437,7 @@ static void make_uffd_wp_pte(struct vm_area_struct *vma,
 		set_pte_at(vma->vm_mm, addr, pte,
 			   make_pte_marker(PTE_MARKER_UFFD_WP));
 	} else {
-		ptent = pte_swp_mkuffd(ptent);
+		ptent = set_uffd_wp_softleaf_pte(ptent);
 		set_pte_at(vma->vm_mm, addr, pte, ptent);
 	}
 }
@@ -2477,6 +2496,21 @@ static unsigned long pagemap_thp_category(struct pagemap_scan_private *p,
 	return categories;
 }
 
+/* Same as set_uffd_wp_softleaf_pte(), for a PMD. */
+static pmd_t set_uffd_wp_softleaf_pmd(pmd_t pmd)
+{
+	softleaf_t entry = softleaf_from_pmd(pmd);
+	pmd_t newpmd;
+
+	if (!softleaf_is_writable(entry))
+		return pmd_swp_mkuffd(pmd);
+
+	newpmd = softleaf_to_pmd(softleaf_mkreadable(entry));
+	if (pmd_swp_soft_dirty(pmd))
+		newpmd = pmd_swp_mksoft_dirty(newpmd);
+	return pmd_swp_mkuffd(newpmd);
+}
+
 static void make_uffd_wp_pmd(struct vm_area_struct *vma,
 			     unsigned long addr, pmd_t *pmdp)
 {
@@ -2487,7 +2521,7 @@ static void make_uffd_wp_pmd(struct vm_area_struct *vma,
 		pmd = pmd_mkuffd(old);
 		set_pmd_at(vma->vm_mm, addr, pmdp, pmd);
 	} else if (pmd_is_migration_entry(pmd)) {
-		pmd = pmd_swp_mkuffd(pmd);
+		pmd = set_uffd_wp_softleaf_pmd(pmd);
 		set_pmd_at(vma->vm_mm, addr, pmdp, pmd);
 	}
 }
@@ -2557,7 +2591,7 @@ static void make_uffd_wp_huge_pte(struct vm_area_struct *vma,
 
 	if (softleaf_is_migration(entry)) {
 		set_huge_pte_at(vma->vm_mm, addr, ptep,
-				pte_swp_mkuffd(ptent), psize);
+				set_uffd_wp_softleaf_pte(ptent), psize);
 	} else {
 		pte_t old_pte, new_pte;
 
diff --git a/include/linux/leafops.h b/include/linux/leafops.h
index 4c1476ae3234..4c016fc28690 100644
--- a/include/linux/leafops.h
+++ b/include/linux/leafops.h
@@ -434,6 +434,57 @@ static inline struct folio *softleaf_to_folio(softleaf_t entry)
 	return folio;
 }
 
+/**
+ * softleaf_is_writable() - Is this leaf entry writable?
+ * @entry: Leaf entry.
+ *
+ * Returns: true for a writable migration or device private entry.
+ */
+static inline bool softleaf_is_writable(softleaf_t entry)
+{
+	return softleaf_is_migration_write(entry) ||
+	       softleaf_is_device_private_write(entry);
+}
+
+/**
+ * softleaf_migration_mkreadable() - Downgrade a writable migration entry to a
+ * readable one.
+ * @entry: Leaf entry, softleaf_is_migration_write(@entry) must return true.
+ *
+ * Returns: A readable migration entry for the same PFN, which is exclusive if
+ * the folio is anonymous.
+ */
+static inline softleaf_t softleaf_migration_mkreadable(softleaf_t entry)
+{
+	const pgoff_t offset = swp_offset(entry);
+
+	VM_WARN_ON_ONCE(!softleaf_is_migration_write(entry));
+
+	/* A writable anon migration entry implies AnonExclusive. */
+	if (folio_test_anon(softleaf_to_folio(entry)))
+		return make_readable_exclusive_migration_entry(offset);
+	return make_readable_migration_entry(offset);
+}
+
+/**
+ * softleaf_mkreadable() - Downgrade a writable leaf entry to a readable one.
+ * @entry: Leaf entry.
+ *
+ * Returns: A readable entry if @entry is a writable migration or device
+ * private entry, otherwise @entry unchanged.
+ */
+static inline softleaf_t softleaf_mkreadable(softleaf_t entry)
+{
+	switch (softleaf_type(entry)) {
+	case SOFTLEAF_DEVICE_PRIVATE_WRITE:
+		return make_readable_device_private_entry(swp_offset(entry));
+	case SOFTLEAF_MIGRATION_WRITE:
+		return softleaf_migration_mkreadable(entry);
+	default:
+		return entry;
+	}
+}
+
 /**
  * softleaf_is_poison_marker() - Is this leaf entry a poison marker?
  * @entry: Leaf entry.

-- 
2.39.5


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings
  2026-11-16 20:02 ` [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings Tal Zussman
@ 2026-11-16 20:15   ` David Hildenbrand (Arm)
  2026-11-16 20:16   ` David Hildenbrand (Arm)
  1 sibling, 0 replies; 8+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 20:15 UTC (permalink / raw)
  To: Tal Zussman, Andrew Morton, Mike Rapoport, Peter Xu,
	James Houghton, Andrea Arcangeli, Suren Baghdasaryan,
	Balbir Singh, Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka,
	Jann Horn, Pedro Falcato, Michal Hocko, Michał Mirosław,
	Andrei Vagin, Muhammad Usama Anjum, Stephen Rothwell,
	Arnd Bergmann, Zi Yan, Matthew Brost, Joshua Hahn, Byungchul Park,
	Gregory Price, Ying Huang, Alistair Popple, Baolin Wang,
	Nico Pache, Ryan Roberts, Dev Jain, Barry Song, Lance Yang,
	Usama Arif, Kiryl Shutsemau, Muchun Song, Oscar Salvador,
	Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, stable

On 11/16/26 21:02, Tal Zussman wrote:
> If the mm cannot map the shared zero page, UFFDIO_ZEROPAGE falls back to
> mfill_atomic_pte_zeroed_folio(), which allocates a zeroed folio and
> installs it through vma_uffd_ops(), no matter what kind of mapping it
> is. An s390 KVM guest with storage keys or protected virtualization is
> such an mm.
> 
> For a MAP_PRIVATE shmem mapping, that adds the zeroed folio to the page
> cache of the file. That is wrong: a private mapping must never modify
> the file it maps, and the file might even be mapped from a read-only fd.
> 
> The ioctl can also fail where it should succeed. A missing fault is only
> delivered for a hole in the file, but UFFDIO_ZEROPAGE can be called on
> an address that did not fault, or can race with a write to the file. If
> the file has a page at that offset by then, inserting the zeroed folio
> fails and the ioctl returns -EEXIST although nothing is mapped at the
> destination address.
> 
> What we want is what UFFDIO_ZEROPAGE does on a private mapping when the
> shared zero page can be used, and what UFFDIO_COPY does: only check that
> nothing is mapped at the destination address, map zeroed memory there,
> and leave the page cache alone. The zeroed folio path did that before it
> was moved to vm_uffd_ops.
> 
> To fix it, pick the ops in the common folio path, as
> mfill_atomic_pte_copy() did for UFFDIO_COPY: anon_uffd_ops for a private
> mapping, the ops of the VMA otherwise. That makes
> mfill_atomic_pte_copy() and mfill_atomic_pte_zeroed_folio() redundant,
> so remove them and rename __mfill_atomic_pte() to
> mfill_atomic_pte_folio().
> 
> UFFDIO_ZEROPAGE no longer modifies the file, and only fails with -EEXIST
> if something is already mapped at the destination address.
> 
> Fixes: f74991b4e383 ("shmem, userfaultfd: implement shmem uffd operations using vm_uffd_ops")
> Cc: stable@vger.kernel.org
> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings
  2026-11-16 20:02 ` [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings Tal Zussman
  2026-11-16 20:15   ` David Hildenbrand (Arm)
@ 2026-11-16 20:16   ` David Hildenbrand (Arm)
  1 sibling, 0 replies; 8+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 20:16 UTC (permalink / raw)
  To: Tal Zussman, Andrew Morton, Mike Rapoport, Peter Xu,
	James Houghton, Andrea Arcangeli, Suren Baghdasaryan,
	Balbir Singh, Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka,
	Jann Horn, Pedro Falcato, Michal Hocko, Michał Mirosław,
	Andrei Vagin, Muhammad Usama Anjum, Stephen Rothwell,
	Arnd Bergmann, Zi Yan, Matthew Brost, Joshua Hahn, Byungchul Park,
	Gregory Price, Ying Huang, Alistair Popple, Baolin Wang,
	Nico Pache, Ryan Roberts, Dev Jain, Barry Song, Lance Yang,
	Usama Arif, Kiryl Shutsemau, Muchun Song, Oscar Salvador,
	Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, stable

On 11/16/26 21:02, Tal Zussman wrote:
> If the mm cannot map the shared zero page, UFFDIO_ZEROPAGE falls back to
> mfill_atomic_pte_zeroed_folio(), which allocates a zeroed folio and
> installs it through vma_uffd_ops(), no matter what kind of mapping it
> is. An s390 KVM guest with storage keys or protected virtualization is
> such an mm.
> 
> For a MAP_PRIVATE shmem mapping, that adds the zeroed folio to the page
> cache of the file. That is wrong: a private mapping must never modify
> the file it maps, and the file might even be mapped from a read-only fd.
> 
> The ioctl can also fail where it should succeed. A missing fault is only
> delivered for a hole in the file, but UFFDIO_ZEROPAGE can be called on
> an address that did not fault, or can race with a write to the file. If
> the file has a page at that offset by then, inserting the zeroed folio
> fails and the ioctl returns -EEXIST although nothing is mapped at the
> destination address.
> 
> What we want is what UFFDIO_ZEROPAGE does on a private mapping when the
> shared zero page can be used, and what UFFDIO_COPY does: only check that
> nothing is mapped at the destination address, map zeroed memory there,
> and leave the page cache alone. The zeroed folio path did that before it
> was moved to vm_uffd_ops.
> 
> To fix it, pick the ops in the common folio path, as
> mfill_atomic_pte_copy() did for UFFDIO_COPY: anon_uffd_ops for a private
> mapping, the ops of the VMA otherwise. That makes
> mfill_atomic_pte_copy() and mfill_atomic_pte_zeroed_folio() redundant,
> so remove them and rename __mfill_atomic_pte() to
> mfill_atomic_pte_folio().
> 
> UFFDIO_ZEROPAGE no longer modifies the file, and only fails with -EEXIST
> if something is already mapped at the destination address.
> 
> Fixes: f74991b4e383 ("shmem, userfaultfd: implement shmem uffd operations using vm_uffd_ops")
> Cc: stable@vger.kernel.org
> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---
>  mm/userfaultfd.c | 46 ++++++++++++++++------------------------------
>  1 file changed, 16 insertions(+), 30 deletions(-)
> 
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index d65184d9a480..61a712e1f9f5 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -531,15 +531,27 @@ static int mfill_copy_folio_retry(struct mfill_state *mfill_state,
>  	return 0;
>  }
>  
> -static int __mfill_atomic_pte(struct mfill_state *state,
> -			      const struct vm_uffd_ops *ops)
> +static int mfill_atomic_pte_folio(struct mfill_state *state)
>  {
> +	const struct vm_uffd_ops *ops = vma_uffd_ops(state->vma);
>  	unsigned long dst_addr = state->dst_addr;
>  	unsigned long src_addr = state->src_addr;
>  	uffd_flags_t flags = state->flags;
>  	struct folio *folio;
>  	int ret;
>  
> +	/*
> +	 * The normal page fault path for a MAP_PRIVATE mapping in a
> +	 * file-backed VMA will invoke the fault, fill the hole in the file and
> +	 * COW it right away. The result generates plain anonymous memory.
> +	 * So when we are asked to fill a hole in a MAP_PRIVATE mapping, we'll
> +	 * generate anonymous memory directly without actually filling the
> +	 * hole. For the MAP_PRIVATE case the robustness check only happens in
> +	 * the pagetable (to verify it's still none) and not in the page cache.
> +	 */
> +	if (!(state->vma->vm_flags & VM_SHARED))
> +		ops = &anon_uffd_ops;

Oh, but one comment: why not check for actual cow mappings?

	vma_is_cow_mapping()

is your friend :)

-- 
Cheers,

David

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v2 2/9] userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD
  2026-11-16 20:02 ` [PATCH v2 2/9] userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD Tal Zussman
@ 2026-11-16 20:18   ` David Hildenbrand (Arm)
  0 siblings, 0 replies; 8+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 20:18 UTC (permalink / raw)
  To: Tal Zussman, Andrew Morton, Mike Rapoport, Peter Xu,
	James Houghton, Andrea Arcangeli, Suren Baghdasaryan,
	Balbir Singh, Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka,
	Jann Horn, Pedro Falcato, Michal Hocko, Michał Mirosław,
	Andrei Vagin, Muhammad Usama Anjum, Stephen Rothwell,
	Arnd Bergmann, Zi Yan, Matthew Brost, Joshua Hahn, Byungchul Park,
	Gregory Price, Ying Huang, Alistair Popple, Baolin Wang,
	Nico Pache, Ryan Roberts, Dev Jain, Barry Song, Lance Yang,
	Usama Arif, Kiryl Shutsemau, Muchun Song, Oscar Salvador,
	Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, stable

On 11/16/26 21:02, Tal Zussman wrote:
> move_pages() refuses to move into a destination PMD that already maps a
> THP, but checks for that with pmd_trans_huge(), which is only true for a
> present THP. A device-private THP or a THP under migration is a PMD
> softleaf entry and is not refused.
> 
> The destination PMD is then treated as if it pointed to a PTE table. It
> is not empty, so pte_alloc() leaves it alone, and move_pages_ptes()
> fails to map a PTE table from it and returns -EAGAIN. move_pages()
> retries on -EAGAIN, with both VMA read locks held. If the source is a
> THP, it is split first for nothing.
> 
> For a THP under migration, the loop ends once the migration finishes.
> A device-private THP can remain in device memory indefinitely, so the
> caller of UFFDIO_MOVE hangs until it receives a fatal signal, and
> anything in the mm that needs to write-lock one of the VMAs blocks along
> with it. This requires a driver that migrates THPs to device memory.
> 
> To fix it, check the destination with pmd_is_huge(), which is true for
> a present THP and for a PMD softleaf entry. UFFDIO_MOVE now fails with
> -EEXIST in these cases, as it already does for a present THP.
> 
> Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI")
> Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
> Cc: stable@vger.kernel.org
> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Tested-by: Gregory Price (Meta) <gourry@gourry.net>
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
> Signed-off-by: Tal Zussman <tz2294@columbia.edu>
> ---
>  mm/userfaultfd.c | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
> index 61a712e1f9f5..ad9d243dc64b 100644
> --- a/mm/userfaultfd.c
> +++ b/mm/userfaultfd.c
> @@ -2021,12 +2021,13 @@ static ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,
>  
>  		dst_pmdval = pmdp_get_lockless(dst_pmd);
>  		/*
> -		 * If the dst_pmd is mapped as THP don't override it and just
> -		 * be strict. If dst_pmd changes into TPH after this check, the
> -		 * move_pages_huge_pmd() will detect the change and retry
> -		 * while move_pages_pte() will detect the change and fail.
> +		 * If the dst_pmd is a huge leaf, present or not, don't
> +		 * override it and just be strict. If dst_pmd changes into a
> +		 * leaf after this check, move_pages_huge_pmd() will detect the
> +		 * change and retry while move_pages_ptes() will fail to map
> +		 * the PTE table and the next iteration will see the leaf.
>  		 */
> -		if (unlikely(pmd_trans_huge(dst_pmdval))) {
> +		if (unlikely(pmd_is_huge(dst_pmdval))) {

Insert the usual ranting about stupid thp helpers

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v2 3/9] fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect
  2026-11-16 20:02 ` [PATCH v2 3/9] fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect Tal Zussman
@ 2026-11-16 20:21   ` David Hildenbrand (Arm)
  0 siblings, 0 replies; 8+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-16 20:21 UTC (permalink / raw)
  To: Tal Zussman, Andrew Morton, Mike Rapoport, Peter Xu,
	James Houghton, Andrea Arcangeli, Suren Baghdasaryan,
	Balbir Singh, Liam R. Howlett, Lorenzo Stoakes, Vlastimil Babka,
	Jann Horn, Pedro Falcato, Michal Hocko, Michał Mirosław,
	Andrei Vagin, Muhammad Usama Anjum, Stephen Rothwell,
	Arnd Bergmann, Zi Yan, Matthew Brost, Joshua Hahn, Byungchul Park,
	Gregory Price, Ying Huang, Alistair Popple, Baolin Wang,
	Nico Pache, Ryan Roberts, Dev Jain, Barry Song, Lance Yang,
	Usama Arif, Kiryl Shutsemau, Muchun Song, Oscar Salvador,
	Mikołaj Lenczewski
  Cc: linux-mm, linux-kernel, linux-fsdevel, Breno Leitao, stable

On 11/16/26 21:02, Tal Zussman wrote:
> PAGEMAP_SCAN with PM_SCAN_WP_MATCHING write-protects pages in a VMA

Today on "things that should have never been merged": PAGEMAP_SCAN and
PM_SCAN_WP_MATCHING

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-16 20:21 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 20:02 [PATCH v2 0/9] userfaultfd: assorted fixes (part 1) Tal Zussman
2026-11-16 20:02 ` [PATCH v2 1/9] userfaultfd: use anonymous memory for UFFDIO_ZEROPAGE on private file mappings Tal Zussman
2026-11-16 20:15   ` David Hildenbrand (Arm)
2026-11-16 20:16   ` David Hildenbrand (Arm)
2026-11-16 20:02 ` [PATCH v2 2/9] userfaultfd: refuse UFFDIO_MOVE into a non-present huge destination PMD Tal Zussman
2026-11-16 20:18   ` David Hildenbrand (Arm)
2026-11-16 20:02 ` [PATCH v2 3/9] fs/proc/task_mmu: downgrade writable migration entries in PAGEMAP_SCAN write-protect Tal Zussman
2026-11-16 20:21   ` David Hildenbrand (Arm)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox