[PATCH v2] fs/proc/task_mmu: fix PAGEMAP_SCAN written state for PMD holes

[PATCH v2] fs/proc/task_mmu: fix PAGEMAP_SCAN written state for PMD holes

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

PAGEMAP_SCAN reports an unpopulated PTE in a uffd-wp VMA as written
(pagemap_page_category() and the PAGE_IS_WRITTEN fast path), but a range
with no page table at all -- a PMD hole -- is skipped.
pagemap_scan_pte_hole() evaluates the hole against p->cur_vma_category,
which pagemap_scan_test_walk() builds from only PAGE_IS_WPALLOWED and
PAGE_IS_SOFT_DIRTY, so PAGE_IS_WRITTEN is never set: the hole is neither
reported nor, under PM_SCAN_WP_MATCHING, armed.

This is reachable. An anonymous THP is write-protected in place as a huge
PMD (change_huge_pmd(), anon is not split), and a full-PMD MADV_DONTNEED
clears it to pmd_none. A WP-async consumer such as CRIU then misses the
2MB drop -- the range is not reported written and the next incremental
dump keeps stale data. (A file/shmem THP is split on write-protect, so a
later DONTNEED leaves a populated page table of pte_none entries, which
are already reported; only anon THP reaches the hole path.)

Add PAGE_IS_WRITTEN to the categories evaluated for a hole in a
non-hugetlb uffd-wp VMA, matching the pte_none handling in
pagemap_page_category(). The existing PM_SCAN_WP_MATCHING path then also
arms the range: uffd_wp_range() allocates the page table and installs
markers under WP_UNPOPULATED, so the next scan sees it clean until
re-written.

hugetlb is excluded on purpose: an allocated-but-empty huge entry reads
as not-written via pagemap_hugetlb_category(), so reporting an
unallocated hugetlb hole (which also reaches this path) as written would
be inconsistent within the same VMA. hugetlb hole handling is left as-is.

Add a pagemap_ioctl selftest that forms an anon THP, drops it with
MADV_DONTNEED and checks the resulting PMD hole is reported written.

Fixes: 2bad466cc9d9 ("mm/uffd: UFFD_FEATURE_WP_UNPOPULATED")
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Assisted-by: Claude:claude-fable-5
---

Changes since v1 [1], addressing Sashiko AI review:
  - Exclude hugetlb from the hole PAGE_IS_WRITTEN report. An unallocated
    hugetlb slot also reaches pagemap_scan_pte_hole(), and reporting it
    written while an allocated-but-empty huge entry reads not-written
    (pagemap_hugetlb_category()) would be inconsistent within the VMA.
    This also keeps the pre-existing pagemap_scan_hugetlb_hole_wp() range
    concern out of scope here.

Based on the pte_none fix [2] (same selftest file).

[1] https://lore.kernel.org/all/20260708093444.145566-1-kirill@shutemov.name/
[2] https://lore.kernel.org/all/20260707151349.92143-1-kirill@shutemov.name/
 fs/proc/task_mmu.c                         | 25 ++++++++-
 tools/testing/selftests/mm/pagemap_ioctl.c | 60 +++++++++++++++++++++-
 2 files changed, 82 insertions(+), 3 deletions(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index d45c729ab6bb..fad648f9a40c 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -3049,12 +3049,33 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
 {
 	struct pagemap_scan_private *p = walk->private;
 	struct vm_area_struct *vma = walk->vma;
+	unsigned long categories;
 	int ret, err;
 
-	if (!vma || !pagemap_scan_is_interesting_page(p->cur_vma_category, p))
+	if (!vma)
 		return 0;
 
-	ret = pagemap_scan_output(p->cur_vma_category, p, addr, &end);
+	/*
+	 * An unpopulated range with no page table -- e.g. a 2MB anon THP
+	 * dropped via MADV_DONTNEED, which pagemap_page_category() never sees
+	 * -- reads as written on a uffd-wp VMA, matching the pte_none case
+	 * there. Reporting it also lets the PM_SCAN_WP_MATCHING arming below
+	 * install markers (uffd_wp_range() allocates the page table under
+	 * WP_UNPOPULATED), so the next scan sees it clean until re-written.
+	 *
+	 * hugetlb is excluded: an allocated-but-empty huge entry reads as
+	 * not-written via pagemap_hugetlb_category(), so reporting an
+	 * unallocated hugetlb hole as written here would be inconsistent
+	 * within the same VMA.
+	 */
+	categories = p->cur_vma_category;
+	if (userfaultfd_wp(vma) && !is_vm_hugetlb_page(vma))
+		categories |= PAGE_IS_WRITTEN;
+
+	if (!pagemap_scan_is_interesting_page(categories, p))
+		return 0;
+
+	ret = pagemap_scan_output(categories, p, addr, &end);
 	if (addr == end)
 		return ret;
 
diff --git a/tools/testing/selftests/mm/pagemap_ioctl.c b/tools/testing/selftests/mm/pagemap_ioctl.c
index 550d1f2c059b..c0e45d0f7478 100644
--- a/tools/testing/selftests/mm/pagemap_ioctl.c
+++ b/tools/testing/selftests/mm/pagemap_ioctl.c
@@ -25,6 +25,10 @@
 #include "kselftest.h"
 #include "hugepage_settings.h"
 
+#ifndef MADV_COLLAPSE
+#define MADV_COLLAPSE 25
+#endif
+
 #define PAGEMAP_BITS_ALL		(PAGE_IS_WPALLOWED | PAGE_IS_WRITTEN |	\
 					 PAGE_IS_FILE | PAGE_IS_PRESENT |	\
 					 PAGE_IS_SWAPPED | PAGE_IS_PFNZERO |	\
@@ -1102,6 +1106,59 @@ static void unpopulated_scan_test(void)
 	munmap(mem, mem_size);
 }
 
+/*
+ * A 2MB anon THP dropped with MADV_DONTNEED leaves a pmd_none hole with no
+ * page table, which pagemap_page_category() never sees. PAGEMAP_SCAN must
+ * still report it as written on a uffd-wp VMA, via pagemap_scan_pte_hole().
+ */
+static void unpopulated_thp_hole_test(void)
+{
+	long npages, written = 0, ret, i;
+	struct page_region regions[16];
+	char *area, *mem;
+
+	if (!hpage_size) {
+		ksft_test_result_skip("%s THP not supported\n", __func__);
+		return;
+	}
+	npages = hpage_size / page_size;
+
+	/* Get a PMD-aligned range so the range can be a single THP. */
+	area = mmap(NULL, 2 * hpage_size, PROT_READ | PROT_WRITE,
+		    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (area == MAP_FAILED)
+		ksft_exit_fail_msg("%s mmap failed\n", __func__);
+	mem = (char *)(((unsigned long)area + hpage_size - 1) & ~(hpage_size - 1));
+
+	memset(mem, 1, hpage_size);
+	if (madvise(mem, hpage_size, MADV_COLLAPSE) ||
+	    !check_huge_anon(mem, 1, hpage_size)) {
+		ksft_test_result_skip("%s could not form a THP\n", __func__);
+		munmap(area, 2 * hpage_size);
+		return;
+	}
+
+	wp_init(mem, hpage_size);
+
+	/* Drop the whole PMD: it is cleared to a pmd_none hole. */
+	if (madvise(mem, hpage_size, MADV_DONTNEED))
+		ksft_exit_fail_msg("%s MADV_DONTNEED failed\n", __func__);
+
+	ret = pagemap_ioctl(mem, hpage_size, regions, 16, 0, 0,
+			    PAGE_IS_WRITTEN, 0, 0, PAGE_IS_WRITTEN);
+	if (ret < 0)
+		ksft_exit_fail_msg("%s scan failed\n", __func__);
+	for (i = 0; i < ret; i++)
+		written += LEN(regions[i]);
+
+	ksft_test_result(written == npages,
+			 "%s pmd-hole reported written (%ld of %ld)\n",
+			 __func__, written, npages);
+
+	wp_free(mem, hpage_size);
+	munmap(area, 2 * hpage_size);
+}
+
 int sanity_tests(void)
 {
 	unsigned long long mem_size, vec_size;
@@ -1610,7 +1667,7 @@ int main(int __attribute__((unused)) argc, char *argv[])
 	if (!hugetlb_setup_default(4))
 		ksft_print_msg("HugeTLB test will be skipped\n");
 
-	ksft_set_plan(118);
+	ksft_set_plan(119);
 
 	page_size = getpagesize();
 	hpage_size = read_pmd_pagesize();
@@ -1790,6 +1847,7 @@ int main(int __attribute__((unused)) argc, char *argv[])
 
 	/* 18. Unpopulated pte scan-path consistency */
 	unpopulated_scan_test();
+	unpopulated_thp_hole_test();
 
 	close(pagemap_fd);
 	ksft_finished();

base-commit: 9795ad96d277c4af049fe30de1cebd4e39d7bcbe
-- 
2.54.0

Re: [PATCH v2] fs/proc/task_mmu: fix PAGEMAP_SCAN written state for PMD holes

[Andrew Morton]

On Wed,  8 Jul 2026 11:34:29 +0100 Kiryl Shutsemau <kirill@shutemov.name> wrote:

> From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
> 
> PAGEMAP_SCAN reports an unpopulated PTE in a uffd-wp VMA as written
> (pagemap_page_category() and the PAGE_IS_WRITTEN fast path), but a range
> with no page table at all -- a PMD hole -- is skipped.
> pagemap_scan_pte_hole() evaluates the hole against p->cur_vma_category,
> which pagemap_scan_test_walk() builds from only PAGE_IS_WPALLOWED and
> PAGE_IS_SOFT_DIRTY, so PAGE_IS_WRITTEN is never set: the hole is neither
> reported nor, under PM_SCAN_WP_MATCHING, armed.
> 
> This is reachable. An anonymous THP is write-protected in place as a huge
> PMD (change_huge_pmd(), anon is not split), and a full-PMD MADV_DONTNEED
> clears it to pmd_none. A WP-async consumer such as CRIU then misses the
> 2MB drop -- the range is not reported written and the next incremental
> dump keeps stale data. (A file/shmem THP is split on write-protect, so a
> later DONTNEED leaves a populated page table of pte_none entries, which
> are already reported; only anon THP reaches the hole path.)
> 
> Add PAGE_IS_WRITTEN to the categories evaluated for a hole in a
> non-hugetlb uffd-wp VMA, matching the pte_none handling in
> pagemap_page_category(). The existing PM_SCAN_WP_MATCHING path then also
> arms the range: uffd_wp_range() allocates the page table and installs
> markers under WP_UNPOPULATED, so the next scan sees it clean until
> re-written.
> 
> hugetlb is excluded on purpose: an allocated-but-empty huge entry reads
> as not-written via pagemap_hugetlb_category(), so reporting an
> unallocated hugetlb hole (which also reaches this path) as written would
> be inconsistent within the same VMA. hugetlb hole handling is left as-is.
> 
> Add a pagemap_ioctl selftest that forms an anon THP, drops it with
> MADV_DONTNEED and checks the resulting PMD hole is reported written.

hoo boy, that was heavy going.

> Assisted-by: Claude:claude-fable-5

OK ;)

But what do our users see?  afaict the result of the bug is "the next
incremental CRIU dump keeps stale data".  Why is this a problem?  How
would operators look at a user bug report and figure out that this
patch will address it?  Is there some Reported-by/Closes?

In other words, (please train Claude to) always describe the userspace
visible effects of a bug when fixing it.

>
> --- a/tools/testing/selftests/mm/pagemap_ioctl.c
> +++ b/tools/testing/selftests/mm/pagemap_ioctl.c
> @@ -25,6 +25,10 @@
>  #include "kselftest.h"
>  #include "hugepage_settings.h"
>  
> +#ifndef MADV_COLLAPSE
> +#define MADV_COLLAPSE 25
> +#endif

Why would this be undefined?  It's right there in mman-common.h?

> +/*
> + * A 2MB anon THP dropped with MADV_DONTNEED leaves a pmd_none hole with no
> + * page table, which pagemap_page_category() never sees. PAGEMAP_SCAN must
> + * still report it as written on a uffd-wp VMA, via pagemap_scan_pte_hole().
> + */
> +static void unpopulated_thp_hole_test(void)
> +{
> +	long npages, written = 0, ret, i;
> +	struct page_region regions[16];
> +	char *area, *mem;
> +
> +	if (!hpage_size) {
> +		ksft_test_result_skip("%s THP not supported\n", __func__);
> +		return;
> +	}
> +	npages = hpage_size / page_size;
> +
> +	/* Get a PMD-aligned range so the range can be a single THP. */
> +	area = mmap(NULL, 2 * hpage_size, PROT_READ | PROT_WRITE,
> +		    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> +	if (area == MAP_FAILED)
> +		ksft_exit_fail_msg("%s mmap failed\n", __func__);
> +	mem = (char *)(((unsigned long)area + hpage_size - 1) & ~(hpage_size - 1));

Do selftests not have ALIGN and friends?  Seems not, given how many of
them have own implementations.

> +
> +	memset(mem, 1, hpage_size);
> +	if (madvise(mem, hpage_size, MADV_COLLAPSE) ||
> +	    !check_huge_anon(mem, 1, hpage_size)) {
> +		ksft_test_result_skip("%s could not form a THP\n", __func__);
> +		munmap(area, 2 * hpage_size);
> +		return;
> +	}
>
> ...
>

Re: [PATCH v2] fs/proc/task_mmu: fix PAGEMAP_SCAN written state for PMD holes

[Kiryl Shutsemau]

On Wed, Jul 08, 2026 at 08:08:44PM -0700, Andrew Morton wrote:
> On Wed,  8 Jul 2026 11:34:29 +0100 Kiryl Shutsemau <kirill@shutemov.name> wrote:
> 
> > From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
> > 
> > PAGEMAP_SCAN reports an unpopulated PTE in a uffd-wp VMA as written
> > (pagemap_page_category() and the PAGE_IS_WRITTEN fast path), but a range
> > with no page table at all -- a PMD hole -- is skipped.
> > pagemap_scan_pte_hole() evaluates the hole against p->cur_vma_category,
> > which pagemap_scan_test_walk() builds from only PAGE_IS_WPALLOWED and
> > PAGE_IS_SOFT_DIRTY, so PAGE_IS_WRITTEN is never set: the hole is neither
> > reported nor, under PM_SCAN_WP_MATCHING, armed.
> > 
> > This is reachable. An anonymous THP is write-protected in place as a huge
> > PMD (change_huge_pmd(), anon is not split), and a full-PMD MADV_DONTNEED
> > clears it to pmd_none. A WP-async consumer such as CRIU then misses the
> > 2MB drop -- the range is not reported written and the next incremental
> > dump keeps stale data. (A file/shmem THP is split on write-protect, so a
> > later DONTNEED leaves a populated page table of pte_none entries, which
> > are already reported; only anon THP reaches the hole path.)
> > 
> > Add PAGE_IS_WRITTEN to the categories evaluated for a hole in a
> > non-hugetlb uffd-wp VMA, matching the pte_none handling in
> > pagemap_page_category(). The existing PM_SCAN_WP_MATCHING path then also
> > arms the range: uffd_wp_range() allocates the page table and installs
> > markers under WP_UNPOPULATED, so the next scan sees it clean until
> > re-written.
> > 
> > hugetlb is excluded on purpose: an allocated-but-empty huge entry reads
> > as not-written via pagemap_hugetlb_category(), so reporting an
> > unallocated hugetlb hole (which also reaches this path) as written would
> > be inconsistent within the same VMA. hugetlb hole handling is left as-is.
> > 
> > Add a pagemap_ioctl selftest that forms an anon THP, drops it with
> > MADV_DONTNEED and checks the resulting PMD hole is reported written.
> 
> hoo boy, that was heavy going.

Will make it brief in v3.

> > Assisted-by: Claude:claude-fable-5
> 
> OK ;)
> 
> But what do our users see?  afaict the result of the bug is "the next
> incremental CRIU dump keeps stale data".  Why is this a problem?  How
> would operators look at a user bug report and figure out that this
> patch will address it?

The core point is that MADV_DONTNEED has fill-with-zeros semantics and
should be treated as write for write-tracking purposes.

Ideally, we want to have PMD marker here, but we don't have enough infra
to handle non-present PMD entries. Usama works on this.

> Is there some Reported-by/Closes?

Closes: https://sashiko.dev/#/patchset/20260707151349.92143-1-kirill@shutemov.name

-- 
  Kiryl Shutsemau / Kirill A. Shutemov