Re: [PATCH] mm/khugepaged: Don't collapse uffd-minor-registered VMAs

[Lance Yang <lance.yang@linux.dev>]

On Fri, Aug 28, 2026 at 02:07:07PM +0100, Kiryl Shutsemau wrote:
>On Fri, Aug 28, 2026 at 05:47:03PM +0800, Lance Yang wrote:
>> 
>> On Fri, Aug 28, 2026 at 12:50:04AM +0000, James Houghton wrote:
>> >Userfaultfd minor faults provides userspace with the ability to manually
>> >install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can
>> >map holes in the VMA when a naturally-aligned THP is present without
>> >explicit action from userspace.
>> >
>> >This is a problem, as it bypasses userfaultfd minor faults that
>> >userspace is expecting to handle.
>> 
>> One basic question first. Should MADV_COLLAPSE refuse to collapse a
>> UFFD-minor-registered VMA, regardless of whether all PTEs are present?
>
>I think the line is not between MADV_COLLAPSE vs khugepaged, it is if
>vma->vm_mm is current->mm or not.
>
>In other words, we might want to allow collapse if workload itself asks
>for it, but not for khugepaged or *external* process_madvise(MADV_COLLAPSE).
>is_khugepaged is not the right proxy.

Right, IIUC, we have two options so far:

A) Conservatively reject collapse for any UFFD-minor-registered VMA, as
   this patch does.

B) Allow collapse when the workload itself asks for it, but reject
   khugepaged and external process_madvise(MADV_COLLAPSE).

Maybe there is a third option as well. Would be good to hear more thoughts
on the expected semantics, especially from the maintainers, before
deciding what the check should look like.

@James, thanks for spinning v2 so quickly :) Please don't send another
revision until we settle the expected semantics here. Hope you don't mind
giving us a little time to sort this out :D

Cheers, Lance