Re: [PATCH] mm/khugepaged: Don't collapse uffd-minor-registered VMAs

[Lance Yang <lance.yang@linux.dev>]

On Sat, Aug 29, 2026 at 12:15:23PM +0800, Lance Yang wrote:
>
>On Fri, Aug 28, 2026 at 02:07:07PM +0100, Kiryl Shutsemau wrote:
>>On Fri, Aug 28, 2026 at 05:47:03PM +0800, Lance Yang wrote:
>>> 
>>> On Fri, Aug 28, 2026 at 12:50:04AM +0000, James Houghton wrote:
>>> >Userfaultfd minor faults provides userspace with the ability to manually
>>> >install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can
>>> >map holes in the VMA when a naturally-aligned THP is present without
>>> >explicit action from userspace.
>>> >
>>> >This is a problem, as it bypasses userfaultfd minor faults that
>>> >userspace is expecting to handle.
>>> 
>>> One basic question first. Should MADV_COLLAPSE refuse to collapse a
>>> UFFD-minor-registered VMA, regardless of whether all PTEs are present?
>>
>>I think the line is not between MADV_COLLAPSE vs khugepaged, it is if
>>vma->vm_mm is current->mm or not.
>>
>>In other words, we might want to allow collapse if workload itself asks
>>for it, but not for khugepaged or *external* process_madvise(MADV_COLLAPSE).
>>is_khugepaged is not the right proxy.
>
>Right, IIUC, we have two options so far:
>
>A) Conservatively reject collapse for any UFFD-minor-registered VMA, as
>   this patch does.
>
>B) Allow collapse when the workload itself asks for it, but reject
>   khugepaged and external process_madvise(MADV_COLLAPSE).

One more correction to option B above. Background khugepaged already
passes install_pmd=false and cannot trigger the problem, so the
distinction is only between the workload's own madvise(MADV_COLLAPSE)
and external process_madvise(MADV_COLLAPSE) :)

Actually:

A) Reject MADV_COLLAPSE for any UFFD-minor-registered VMA.

B) Allow MADV_COLLAPSE when the workload itself asks for it, but reject
   external process_madvise(MADV_COLLAPSE).

>
>Maybe there is a third option as well. Would be good to hear more thoughts
>on the expected semantics, especially from the maintainers, before
>deciding what the check should look like.
>
>@James, thanks for spinning v2 so quickly :) Please don't send another
>revision until we settle the expected semantics here. Hope you don't mind
>giving us a little time to sort this out :D
>
>Cheers, Lance
>