[PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[James Houghton]

Userfaultfd minor faults provides userspace with the ability to manually
install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can
map holes in the VMA when a naturally-aligned THP is present without
explicit action from userspace.

This is a problem, as it bypasses userfaultfd minor faults that
userspace is expecting to handle.

If userspace implements post-copy live migration using userfaultfd minor
faults, this situation is currently possible:
1. The VMA for guest memory is userfaultfd-minor-registered and nothing
   is mapped in the page tables.
2. A stale copy of a page is present in a naturally-aligned THP (from
   pre-copy live migration).
3. khugepaged collapses the mapping of the THP, installs a PMD.
4. The VM now has access to the stale contents => VM is broken.
5. After installing the correct contents, userspace attempts to map the
   page with UFFDIO_CONTINUE; it gets EEXIST, indicating that something
   unexpectedly mapped the page.

The naturally-aligned THP case is the only case where this is a problem.
khugepaged otherwise requires all PTEs to be present for
userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
correct. This check is essentially bypassed for naturally-aligned THPs.

No changes are needed for file_backed_vma_is_retractable(), as zapping
PTEs is safe. Userspace must already handle cases where PTEs are zapped
without explicit action (e.g. due to reclaim).

Fixes: 58ac9a8993a1 ("mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by pmds")
Cc: <stable@vger.kernel.org> # 6.1
Suggested-by: Lance Yang <lance.yang@linux.dev>
Tested-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: James Houghton <jthoughton@google.com>
---
v1->v2:
 - Applied Lance's change to move the userfaultfd_minor() check to the
   right place.
 - Suggested-by: and Tested-by: Lance. Thank you!
 - Adjusted the commit description.

v1: https://lore.kernel.org/linux-mm/20260828005004.2870750-1-jthoughton@google.com/
---
 mm/khugepaged.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index b237f6e7662a..22356229c1e4 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1899,6 +1899,13 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
 	if (userfaultfd_protected(vma))
 		return SCAN_PTE_UFFD;
 
+	/*
+	 * Userfaultfd-minor-registered VMAs should not be collapsed, as
+	 * userspace is expecting to explicitly install PTEs.
+	 */
+	if (userfaultfd_minor(vma))
+		return SCAN_PTE_UFFD;
+
 	folio = filemap_lock_folio(vma->vm_file->f_mapping,
 			       linear_page_index(vma, haddr));
 	if (IS_ERR(folio))

base-commit: 26260251022fbc2f248a3d747a9b2b961b18d2d8
-- 
2.55.0.897.gb25b4bd76c-goog

Re: [PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[Andrew Morton]

On Fri, 28 Aug 2026 22:26:40 +0000 James Houghton <jthoughton@google.com> wrote:

> Userfaultfd minor faults provides userspace with the ability to manually
> install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can
> map holes in the VMA when a naturally-aligned THP is present without
> explicit action from userspace.
> 
> This is a problem, as it bypasses userfaultfd minor faults that
> userspace is expecting to handle.
> 
> If userspace implements post-copy live migration using userfaultfd minor
> faults, this situation is currently possible:
> 1. The VMA for guest memory is userfaultfd-minor-registered and nothing
>    is mapped in the page tables.
> 2. A stale copy of a page is present in a naturally-aligned THP (from
>    pre-copy live migration).
> 3. khugepaged collapses the mapping of the THP, installs a PMD.
> 4. The VM now has access to the stale contents => VM is broken.
> 5. After installing the correct contents, userspace attempts to map the
>    page with UFFDIO_CONTINUE; it gets EEXIST, indicating that something
>    unexpectedly mapped the page.

Did I read somewhere that this it biting you in a real-life scenario? 
That would use useful info to spell out.

Or is this a theoretical thing?

> The naturally-aligned THP case is the only case where this is a problem.
> khugepaged otherwise requires all PTEs to be present for
> userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
> correct. This check is essentially bypassed for naturally-aligned THPs.
> 
> No changes are needed for file_backed_vma_is_retractable(), as zapping
> PTEs is safe. Userspace must already handle cases where PTEs are zapped
> without explicit action (e.g. due to reclaim).
> 
> Fixes: 58ac9a8993a1 ("mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by pmds")
> Cc: <stable@vger.kernel.org> # 6.1

From the changelog it isn't clear why a -stable backport is proposed? 
I assume the userspace-visible runtime effect is a rare and bogus
-EEXIST from UFFDIO_CONTINUE?

> Suggested-by: Lance Yang <lance.yang@linux.dev>
> Tested-by: Lance Yang <lance.yang@linux.dev>

Wondering what Lance tested.  Was he able to reproduce the bug?

Sashiko might have found another race in there:

	https://sashiko.dev/#/patchset/20260828222640.1638457-1-jthoughton@google.com

Re: [PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[James Houghton]

On Fri, Aug 28, 2026 at 5:33 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Fri, 28 Aug 2026 22:26:40 +0000 James Houghton <jthoughton@google.com> wrote:
>
> > Userfaultfd minor faults provides userspace with the ability to manually
> > install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can
> > map holes in the VMA when a naturally-aligned THP is present without
> > explicit action from userspace.
> >
> > This is a problem, as it bypasses userfaultfd minor faults that
> > userspace is expecting to handle.
> >
> > If userspace implements post-copy live migration using userfaultfd minor
> > faults, this situation is currently possible:
> > 1. The VMA for guest memory is userfaultfd-minor-registered and nothing
> >    is mapped in the page tables.
> > 2. A stale copy of a page is present in a naturally-aligned THP (from
> >    pre-copy live migration).
> > 3. khugepaged collapses the mapping of the THP, installs a PMD.
> > 4. The VM now has access to the stale contents => VM is broken.
> > 5. After installing the correct contents, userspace attempts to map the
> >    page with UFFDIO_CONTINUE; it gets EEXIST, indicating that something
> >    unexpectedly mapped the page.
>
> Did I read somewhere that this it biting you in a real-life scenario?
> That would use useful info to spell out.
>
> Or is this a theoretical thing?

So far it is only theoretical (but please do keep reading). I caught
this while debugging a related, real problem.

The real problem was that userspace was doing MADV_COLLAPSE (*without*
userfaultfd registered) and assuming that page tables wouldn't be
created. This is of course incorrect, but for some reason we only
started hitting this after moving from a 6.6-based kernel to a
6.18-based kernel. I don't know what change caused this (most likely a
downstream change), but the real fix was for userspace to instead do
something like:
1. MADV_COLLAPSE (let's assume this must be done)
2. UFFD_REGISTER_MODE_MINOR
3. MADV_DONTNEED
and we're good to go, right? Wrong, in fact!

I noticed that there was indeed a circumstance where khugepaged could
create PMDs even when userfaultfd *was* registered, making the
userspace fix technically incomplete. That's what this patch fixes.

I haven't yet seen this in our production environment (as far as I am
aware, hence "theoretical"); we don't actually have that many VMs
running a userfaultfd-based live migration stack anymore though (maybe
tens of thousands).

I hope that extra context is helpful.

> > The naturally-aligned THP case is the only case where this is a problem.
> > khugepaged otherwise requires all PTEs to be present for
> > userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
> > correct. This check is essentially bypassed for naturally-aligned THPs.
> >
> > No changes are needed for file_backed_vma_is_retractable(), as zapping
> > PTEs is safe. Userspace must already handle cases where PTEs are zapped
> > without explicit action (e.g. due to reclaim).
> >
> > Fixes: 58ac9a8993a1 ("mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by pmds")
> > Cc: <stable@vger.kernel.org> # 6.1
>
> From the changelog it isn't clear why a -stable backport is proposed?
> I assume the userspace-visible runtime effect is a rare and bogus
> -EEXIST from UFFDIO_CONTINUE?

Although this bug is likely pretty inconsequential (I'm not sure how
many users of userfaultfd minor faults there are), but rarely getting
EEXIST back when UFFDIO_CONTINUE is a userspace-visible bug that
doesn't really have a great workaround.

Because this bug essentially breaks userfaultfd minor faults,
*technically* live migration implemented with it can sometimes corrupt
guest memory, and the hypervisor (like QEMU, but QEMU doesn't use
minor faults) only learns about it after the fact, after we get EEXIST
back. IMO this is enough to backport the patch to stable. But if
others disagree, I don't really mind.

>
> > Suggested-by: Lance Yang <lance.yang@linux.dev>
> > Tested-by: Lance Yang <lance.yang@linux.dev>
>
> Wondering what Lance tested.  Was he able to reproduce the bug?

I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance,
what did you test?

> Sashiko might have found another race in there:
>
>         https://sashiko.dev/#/patchset/20260828222640.1638457-1-jthoughton@google.com

Let me get back to you on this. Best case, I'll send another
(separate) patch. :)

Re: [PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[James Houghton]

On Fri, Aug 28, 2026 at 6:09 PM James Houghton <jthoughton@google.com> wrote:
>
> On Fri, Aug 28, 2026 at 5:33 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > Wondering what Lance tested.  Was he able to reproduce the bug?
>
> I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance,
> what did you test?

Ah, just to be clear, I linked a repro on v1. Here[1] is that repro.
Sorry, slipped my mind to also include it in the notes for this v2.

[1] https://gist.github.com/48ca/d399bf534158e80241fb4937ef1ff664

Re: [PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[Andrew Morton]

On Fri, 28 Aug 2026 18:09:39 -0700 James Houghton <jthoughton@google.com> wrote:

> [lots of good stuff]
>

OK, thanks.

> >
> > > Suggested-by: Lance Yang <lance.yang@linux.dev>
> > > Tested-by: Lance Yang <lance.yang@linux.dev>
> >
> > Wondering what Lance tested.  Was he able to reproduce the bug?
> 
> I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance,
> what did you test?

Wait, there was a reproducer?

Oh yeah, found it.  You changed the patch Subject: between versions,
dangnabbit.

I asked Gemini to write me a reproducer and it basically said "sorry
andrew, I can't do that".  ChatGPT was much more accommodating but
after a few iterations it was still pretty mind-bending.  I came away
thinking "nobody has ever hit this".

But that's not the point.  Now that it's out there and known about,
Black Hats might be working away to find and utilize a reproducer, so
yes, we backport it.

> > Sashiko might have found another race in there:
> >
> >         https://sashiko.dev/#/patchset/20260828222640.1638457-1-jthoughton@google.com
> 
> Let me get back to you on this. Best case, I'll send another
> (separate) patch. :)

Great, another patch ;)

Re: [PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[Andrew Morton]

On Fri, 28 Aug 2026 18:18:11 -0700 James Houghton <jthoughton@google.com> wrote:

> On Fri, Aug 28, 2026 at 6:09 PM James Houghton <jthoughton@google.com> wrote:
> >
> > On Fri, Aug 28, 2026 at 5:33 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > Wondering what Lance tested.  Was he able to reproduce the bug?
> >
> > I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance,
> > what did you test?
> 
> Ah, just to be clear, I linked a repro on v1. Here[1] is that repro.
> Sorry, slipped my mind to also include it in the notes for this v2.
> 
> [1] https://gist.github.com/48ca/d399bf534158e80241fb4937ef1ff664

Cool thanks, I'll slap this into the changelog.

# ./a.out
TAP version 13
1..1
madvise(MADV_COLLAPSE) succeeded
ioctl(UFFDIO_CONTINUE) failed: File exists (errno=17)
not ok 1 - memfd userfaultfd MINOR continue test


We do have a ton of uffd selftesting code in there.  Adding this case
to uffd-unit-tests.c would presumably be pretty simple.  We don't
really have a policy on adding a test-case after fixing something -
it's common practice but why?  We already fixed it!

Re: [PATCH v2] mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

[James Houghton]

On Fri, Aug 28, 2026 at 6:38 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Fri, 28 Aug 2026 18:18:11 -0700 James Houghton <jthoughton@google.com> wrote:
> > Ah, just to be clear, I linked a repro on v1. Here[1] is that repro.
> > Sorry, slipped my mind to also include it in the notes for this v2.
> >
> > [1] https://gist.github.com/48ca/d399bf534158e80241fb4937ef1ff664
>
> Cool thanks, I'll slap this into the changelog.
>
> # ./a.out
> TAP version 13
> 1..1
> madvise(MADV_COLLAPSE) succeeded
> ioctl(UFFDIO_CONTINUE) failed: File exists (errno=17)
> not ok 1 - memfd userfaultfd MINOR continue test
>
>
> We do have a ton of uffd selftesting code in there.  Adding this case
> to uffd-unit-tests.c would presumably be pretty simple.  We don't
> really have a policy on adding a test-case after fixing something -
> it's common practice but why?  We already fixed it!

Good idea. The below patch should work; I'll send it properly on Monday.

diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index ef9b3956bdcf..227bcd6aef9d 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -518,19 +518,34 @@ static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uff
 	uffd_wp_fork_pin_test_common(gopts, args, true);
 }
 
-static void check_memory_contents(uffd_global_test_opts_t *gopts, char *p)
+static int __check_memory_contents(unsigned long offset,
+				   unsigned long nr_pages,
+				   uffd_global_test_opts_t *gopts,
+				   char *p)
 {
 	unsigned long i, j;
 	uint8_t expected_byte;
 
-	for (i = 0; i < gopts->nr_pages; ++i) {
+	if (nr_pages + offset < nr_pages)
+		err("overflow in memory check");
+	if (nr_pages + offset > gopts->nr_pages)
+		err("out of bounds memory check");
+
+	for (i = offset; i < nr_pages; ++i) {
 		expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
 		for (j = 0; j < gopts->page_size; j++) {
 			uint8_t v = *(uint8_t *)(p + (i * gopts->page_size) + j);
 			if (v != expected_byte)
-				err("unexpected page contents");
+				return 1;
 		}
 	}
+
+	return 0;
+}
+
+static int check_memory_contents(uffd_global_test_opts_t *gopts, char *p)
+{
+	return __check_memory_contents(0, gopts->nr_pages, gopts, p);
 }
 
 static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_collapse, bool test_wp)
@@ -539,6 +554,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 	pthread_t uffd_mon;
 	char c = '\0';
 	struct uffd_args args = { 0 };
+	unsigned long checked = 0;
+	bool bad_contents;
 	args.gopts = gopts;
 
 	/*
@@ -564,19 +581,51 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 	if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args))
 		err("uffd_poll_thread create");
 
+	if (test_collapse) {
+		/*
+		 * Read just a single page and try collapsing. The collapse
+		 * should either be rejected or be a no-op.
+		 */
+		if (__check_memory_contents(0, 1, gopts, gopts->area_dst_alias))
+			err("unexpected memory contents before collapse");
+
+		madvise(gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,
+			MADV_COLLAPSE);
+		/*
+		 * If the above collapse mapped pages that were not explicitly
+		 * CONTINUE'd, the below __check_memory_contents() will not
+		 * fault on some pages, resulting in incorrect contents. The
+		 * PTE for the first page may get retracted, so avoid checking
+		 * that page, as we might take a second fault, flipping the
+		 * contents a second time.
+		 */
+		checked = 1;
+	}
+
 	/*
 	 * Read each of the pages back using the UFFD-registered mapping. We
 	 * expect that the first time we touch a page, it will result in a minor
 	 * fault. uffd_poll_thread will resolve the fault by bit-flipping the
 	 * page's contents, and then issuing a CONTINUE ioctl.
 	 */
-	check_memory_contents(gopts, gopts->area_dst_alias);
+	bad_contents = !!__check_memory_contents(checked, gopts->nr_pages - checked,
+						    gopts, gopts->area_dst_alias);
 
 	if (write(gopts->pipefd[1], &c, sizeof(c)) != sizeof(c))
 		err("pipe write");
 	if (pthread_join(uffd_mon, NULL))
 		err("join() failed");
 
+	if (bad_contents && test_collapse) {
+		uffd_test_fail("unexpected memory contents after collapse");
+		return;
+	}
+
+	if (bad_contents) {
+		uffd_test_fail("unexpected memory contents");
+		return;
+	}
+
 	if (test_collapse) {
 		if (madvise(gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,
 			    MADV_COLLAPSE)) {
@@ -593,7 +642,10 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 		 * This won't cause uffd-fault - it purely just makes sure there
 		 * was no corruption.
 		 */
-		check_memory_contents(gopts, gopts->area_dst_alias);
+		if (check_memory_contents(gopts, gopts->area_dst_alias)) {
+			uffd_test_fail("unexpected memory contents");
+			return;
+		}
 	}
 
 	if (args.missing_faults != 0 || args.minor_faults != gopts->nr_pages)