Re: [PATCH v4 1/2] mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs

[Kiryl Shutsemau <kas@kernel.org>]

On Sat, Oct 03, 2026 at 12:18:58AM +0000, James Houghton wrote:
> Userfaultfd minor faults provides userspace with the ability to manually
> install PTEs with UFFDIO_CONTINUE. Right now, MADV_COLLAPSE can map
> holes in the VMA when a naturally-aligned THP is present. This is not
> true for khugepaged collapse: the PTEs will be retracted, but a PMD will
> not be installed.
> 
> When MADV_COLLAPSE installs a PMD that mapped holes in the VMA,
> userspace is likely to expect UFFDIO_CONTINUE to succeed on the
> should-be holes. UFFDIO_CONTINUE will fail and return EEXIST.
> 
> This is not inherently a problem, as MADV_COLLAPSE is an explicit
> userspace action. But, especially because MADV_COLLAPSE can be invoked
> by an external process via process_madvise(), a rogue caller could break
> a userfaultfd-minor resolver thread.
> 
> Userspace cannot generally use MADV_COLLAPSE to resolve userfault minor
> faults, as MADV_COLLAPSE will only resolve such faults if a
> naturally-aligned THP is present, so this is not a functional
> regression for userspace.
> 
> The naturally-aligned THP case is the only case where this quirk exists.
> Collapsing otherwise requires all PTEs to be present for
> userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
> correct. This check is essentially bypassed for naturally-aligned THPs.
> 
> Suggested-by: Lance Yang <lance.yang@linux.dev>
> Tested-by: Lance Yang <lance.yang@linux.dev>
> Signed-off-by: James Houghton <jthoughton@google.com>
> ---
> Changes since v3:
>   - Prevent page table retraction in patch 1. Please see the comment
>     next to it. So this patch now has two hunks instead of one.
>   - Reworded the comment and combined the userfaultfd checks in patch 1.
>     (Thanks David)
>   - Simplified the selftest a bit given the behavior change in patch 1.
>   - Undid a change in v3's selftest that incorrectly handled the case
>     where MADV_COLLAPSE was not supported entirely.
>   - Rebased on top of mm-unstable, which includes Kiryl's changes.
> v3: https://lore.kernel.org/linux-mm/20260910023411.514987-1-jthoughton@google.com/
> v2: https://lore.kernel.org/linux-mm/20260828222640.1638457-1-jthoughton@google.com/
> v1: https://lore.kernel.org/linux-mm/20260828005004.2870750-1-jthoughton@google.com/
> ---
>  mm/khugepaged.c | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index 913086eaf17b..438c4af1c058 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -1868,10 +1868,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
>  		return SCAN_VMA_CHECK;
>  
>  	/*
> -	 * Keep pmd pgtable while the uffd bit is in use; see comment in
> -	 * retract_page_tables().
> +	 * Don't collapse if there might be PTE markers for userfaultfd-based
> +	 * access protection or if collapsing might bypass userfaultfd minor
> +	 * faults.
>  	 */
> -	if (userfaultfd_protected(vma))
> +	if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
>  		return SCAN_PTE_UFFD;
>  
>  	folio = filemap_lock_folio(vma->vm_file->f_mapping,
> @@ -2093,8 +2094,13 @@ static bool file_backed_vma_is_retractable(struct vm_area_struct *vma)
>  	 * and cannot be recycled to a shared PMD. Other vmas can still
>  	 * have the same file mapped hugely, but skip this one: it will
>  	 * always be mapped in small page size for these registrations.
> +	 *
> +	 * Userfaultfd-minor-registered VMAs should also not be retracted.
> +	 * PMDs will not be installed, as doing so can suppress minor faults.
> +	 * If retraction were allowed, khugepaged might continually cause
> +	 * unnecessary userfaultfd minor faults for already-CONTINUE'd pages.
>  	 */
> -	if (userfaultfd_protected(vma))
> +	if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
>  		return false;
>  
>  	/*


Did you run 2/2 on current mm-unstable?

Here, on e8d0f6a1b2a4 plus this series, minor-collapse fails every run:

  # failed reason: [unexpected memory contents after collapse]
  not ok 1 minor-collapse on shmem

This patch refuses the collapse too late.

collapse_scan_file() has no VMA, so the uffd check in
collapse_max_ptes_none() never applies, and collapse_file() goes ahead:
it try_to_unmap()s the PTEs userspace already CONTINUE'd and replaces
the page cache.

Only then does try_collapse_pte_mapped_thp() return SCAN_PTE_UFFD.

MADV_COLLAPSE fails with -EINVAL, but the page the test read beforehand
is already unmapped, takes a second minor fault, and gets flipped twice.

Also, your 2/2 missing Signed-off-by.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov