mm/khugepaged: collapse file/shmem compound pages

["Zach O'Keefe" <zokeefe@google.com>]

Hey Matthew,

I'm leading an attempt to add a new madvise mode, MADV_COLLAPSE, to
allow userspace-directed collapse of memory into THPs[1]. The initial
proposal only supports anonymous memory, but I'm
working on adding support for file-backed and shmem memory.

The intended behavior of MADV_COLLAPSE is that it should return
"success" if all hugepage-aligned / sized regions requested are backed
by pmd-mapped THPs on return (races aside). IOW: we were able to
successfully collapse the memory, or it was already backed by
pmd-mapped THPs.

Currently there is a nice "XXX: khugepaged should compact smaller
compound pages into a PMD sized page" in khugepaged_scan_file() when
we encounter a compound page during scanning. Do you know what kind of
gotchas or technical difficulties would be involved in doing this? I
presume this work would also benefit those relying on khugepaged to
collapse read-only file and shmem memory, and I'd be happy to help
move it forward.

Thanks for your time,
Zach


[1] https://lore.kernel.org/linux-mm/20220504214437.2850685-1-zokeefe@google.com/