[PATCH v2 0/3] ksm: perform a range-walk to jump over holes in break_ksm

Pedro Demarchi Gomes
2025-10-31T17:48:32Z

When unmerging an address range, unmerge_ksm_pages function walks every
page address in the specified range to locate ksm pages. This becomes
highly inefficient when scanning large virtual memory areas that contain
mostly unmapped regions, causing the process to get blocked for several
minutes.

This patch makes break_ksm, function called by unmerge_ksm_pages for
every page in an address range, perform a range walk, allowing it to skip
over entire unmapped holes in a VMA, avoiding unnecessary lookups.

As pointed by David Hildenbrand in [1], unmerge_ksm_pages() is called
from:

* ksm_madvise() through madvise(MADV_UNMERGEABLE). There are not a lot
of users of that function.

* __ksm_del_vma() through ksm_del_vmas(). Effectively called when
disabling KSM for a process either through the sysctl or from s390x gmap
code when enabling storage keys for a VM.

Consider the following test program which creates a 32 TiB mapping in
the virtual address space but only populates a single page:

#include <unistd.h>
#include <stdio.h>
#include <sys/mman.h>

/* 32 TiB */
const size_t size = 32ul * 1024 * 1024 * 1024 * 1024;

int main() {
char *area = mmap(NULL, size, PROT_READ | PROT_WRITE,
MAP_NORESERVE | MAP_PRIVATE | MAP_ANON, -1, 0);

if (area == MAP_FAILED) {
perror("mmap() failed\n");
return -1;
}

/* Populate a single page such that we get an anon_vma. */
*area = 0;

/* Enable KSM. */
madvise(area, size, MADV_MERGEABLE);
madvise(area, size, MADV_UNMERGEABLE);
return 0;
}

Without this patch, this program takes 9 minutes to finish, while with
this patch it finishes in less then 5 seconds.

[1] https://lore.kernel.org/linux-mm/e0886fdf-d198-4130-bd9a-be276c59da37@redhat.com/

Changelog:

v2:
- Use folios in break_ksm_pmd_entry
- Pass address pointer to walk_page_range_vma private parameter
instead of struct break_ksm_arg
- Do not unnecessarily initialize start_ptep
- Order local variable declarations from longest to shortest line
- Improve patch 2 description (do not reference the cover letter)
- Quite a few styling fixes and improvements as suggested by reviewer

v1: https://lore.kernel.org/all/20251028131945.26445-1-pedrodemargomes@gmail.com/

Suggested-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Pedro Demarchi Gomes <pedrodemargomes@gmail.com>

Pedro Demarchi Gomes (3):
Revert "mm/ksm: convert break_ksm() from walk_page_range_vma() to
folio_walk"
ksm: perform a range-walk in break_ksm
ksm: replace function unmerge_ksm_pages with break_ksm

mm/ksm.c | 141 +++++++++++++++++++++++++++++++++----------------------
1 file changed, 86 insertions(+), 55 deletions(-)

--
2.43.0