[PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[xu.xin]

From: "Xu Xin (vivo)" <xu.xin@linux.dev>

Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
which, when enabled, makes ksmd merge only empty (zeroed) anonymous
pages with the kernel zero page and skip the stable and unstable trees
entirely for pages that carry data.

The attribute requires use_zero_pages to be 1, and the two are kept
consistent by symmetric constraints: it cannot be enabled while
use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
off while it is active (-EBUSY). The new attribute can be changed at
any time and is only effective for pages merged after the change.

Benefits in three aspects
=========================
- Memory saving:
  On memory-constrained systems such as Android and embedded devices,
  zero-filled anonymous pages are the largest source of duplicate
  anonymous memory. They are produced by userspace allocators: the
  zeroing of freshly allocated Java objects and buffers in the ART heap
  (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
  preallocated media buffers. Merging them with the kernel zero page
  frees a significant amount of physical memory, which matters as DRAM
  costs keep rising. With this enabled, we measured 100MB–300MB saving
  tested with several popular applications on aarch64 (Android).

- CPU overhead reduction:
  Merging only zero pages also reduces the overhead of ksmd. A page that
  is not merged with the zero page carries data and can never be merged
  with the zero page, so there is no point in searching the stable tree
  or inserting it into the unstable tree for it. Skipping those tree
  operations for data-bearing pages reduces both the CPU time and the
  rhashtable memory footprint of ksmd, which makes zero page merging
  practical on low-resource devices where full KSM scanning would be too
  expensive.

- Security:
  Finally, this shrinks the KSM side-channel attack surface: only zero
  pages are ever merged, so the sharing status of data-bearing pages is
  no longer observable through KSM timing side channels.

Note
=====
- Suitable for Android, embedded and low-memory devices, where zero-page
  deduplication is wanted but full KSM scanning is too costly, and reduced
  KSM side-channel exposure is desired. Many handset vendors may prefer
  this knob, which can be used as a default fallback policy.

- Not suitable for server and virtualization workloads containing many
  non-zero duplicate pages, where vanilla KSM provides better memory
  savings.

Signed-off-by: Xu Xin (vivo) <xu.xin@linux.dev>
Reviewed-by: Huan Yang <link@vivo.com>
Cc: Wang Xiao <xiao.wang@vivo.com>
---
 Documentation/admin-guide/mm/ksm.rst | 40 ++++++++++++++++++++-
 mm/ksm.c                             | 53 ++++++++++++++++++++++++++++
 2 files changed, 92 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst
index c329ca747b8c..dd8ef2e4fdc2 100644
--- a/Documentation/admin-guide/mm/ksm.rst
+++ b/Documentation/admin-guide/mm/ksm.rst
@@ -128,10 +128,48 @@ use_zero_pages
         KSM for some workloads, for example if the checksums of pages
         candidate for merging match the checksum of an empty
         page. This setting can be changed at any time, it is only
-        effective for pages merged after the change.
+        effective for pages merged after the change. Turning this off
+        while ``merge_only_zero_pages`` is enabled fails with EBUSY;
+        clear ``merge_only_zero_pages`` first.
 
         Default: 0 (normal KSM behaviour as in earlier releases)
 
+merge_only_zero_pages
+        specifies whether ksmd should only merge empty (zeroed)
+        anonymous pages, with the kernel zero page, and skip the stable
+        and unstable trees entirely for pages that carry data. This
+        requires ``use_zero_pages`` to be 1; it cannot be enabled while
+        ``use_zero_pages`` is 0, and ``use_zero_pages`` cannot be turned
+        off while it is enabled (the write fails with EBUSY).
+
+        This is useful on memory-constrained systems (e.g. Android and
+        embedded devices) where the main source of duplicate anonymous
+        memory is zero-filled pages produced by userspace allocators
+        (for example the zeroing of freshly allocated Java objects and
+        buffers in the ART heap, ``calloc``/``memset`` in native code,
+        and pre-allocated media buffers). Merging only those pages
+        avoids the per-page cost of searching and inserting into the
+        stable and unstable trees for data-bearing pages, which reduces
+        the CPU and memory overhead of ksmd. It also shrinks the KSM
+        side-channel attack surface, since only zero pages are ever
+        merged and the sharing state of data-bearing pages is no longer
+        observable through KSM.
+
+        - Suitable for: Android, embedded and low-memory devices, where
+          zero-page deduplication is wanted but full KSM scanning is
+          too costly, and reduced KSM side-channel exposure is desired.
+          Many handset vendors may prefer this knob, which can be used
+          as a default fallback policy.
+
+        - Not suitable for: Server and virtualization workloads containing
+          many non-zero duplicate pages, where vanilla KSM provides better
+          memory savings.
+
+        This setting can be changed at any time, it is only effective
+        for pages merged after the change.
+
+        Default: 0 (normal KSM behaviour)
+
 max_page_sharing
         Maximum sharing allowed for each KSM page. This enforces a
         deduplication limit to avoid high latency for virtual memory
diff --git a/mm/ksm.c b/mm/ksm.c
index 64f9e803b99e..a89b0f1c1065 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -297,6 +297,13 @@ static unsigned int zero_checksum __read_mostly;
 /* Whether to merge empty (zeroed) pages with actual zero pages */
 static bool ksm_use_zero_pages __read_mostly;
 
+/*
+ * When set, ksmd only merges empty (zeroed) anonymous pages with the kernel
+ * zero page and skips the stable/unstable tree entirely for pages that carry
+ * data. This requires ksm_use_zero_pages to be enabled.
+ */
+static bool ksm_merge_only_zero_pages __read_mostly;
+
 /* Skip pages that couldn't be de-duplicated previously */
 /* Default to true at least temporarily, for testing */
 static bool ksm_smart_scan = true;
@@ -2369,6 +2376,15 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite
 
 		if (!try_to_merge_with_zero_page(rmap_item, page))
 			return;
+
+		/*
+		 * merge_only_zero_pages: only empty (zeroed) pages are merged,
+		 * with the kernel zero page. A page that reached here was not
+		 * merged with the zero page, so it carries data; skip the
+		 * stable and unstable trees for it.
+		 */
+		if (ksm_merge_only_zero_pages)
+			return;
 	}
 
 	/* Start by searching for the folio in the stable tree */
@@ -3730,12 +3746,48 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
 	if (err)
 		return -EINVAL;
 
+	/*
+	 * merge_only_zero_pages depends on use_zero_pages: refuse to turn
+	 * the latter off while the former is active, so the two cannot
+	 * silently diverge.
+	 */
+	if (!value && ksm_merge_only_zero_pages)
+		return -EBUSY;
+
 	ksm_use_zero_pages = value;
 
 	return count;
 }
 KSM_ATTR(use_zero_pages);
 
+static ssize_t merge_only_zero_pages_show(struct kobject *kobj,
+					  struct kobj_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%u\n", ksm_merge_only_zero_pages);
+}
+static ssize_t merge_only_zero_pages_store(struct kobject *kobj,
+					   struct kobj_attribute *attr,
+					   const char *buf, size_t count)
+{
+	int err;
+	bool value;
+
+	err = kstrtobool(buf, &value);
+	if (err)
+		return -EINVAL;
+
+	if (value && !ksm_use_zero_pages) {
+		pr_warn_once("ksm: merge_only_zero_pages can only be enabled "
+			     "when use_zero_pages is 1\n");
+		return -EINVAL;
+	}
+
+	ksm_merge_only_zero_pages = value;
+
+	return count;
+}
+KSM_ATTR(merge_only_zero_pages);
+
 static ssize_t max_page_sharing_show(struct kobject *kobj,
 				     struct kobj_attribute *attr, char *buf)
 {
@@ -4061,6 +4113,7 @@ static struct attribute *ksm_attrs[] = {
 	&stable_node_dups_attr.attr,
 	&stable_node_chains_prune_millisecs_attr.attr,
 	&use_zero_pages_attr.attr,
+	&merge_only_zero_pages_attr.attr,
 	&general_profit_attr.attr,
 	&smart_scan_attr.attr,
 	&advisor_mode_attr.attr,
-- 
2.34.1

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] mm: ksm: unsynchronized state checks in zero-page sysfs store
  handlers
--

commit 9b8576e88012c85cc90e3b5aa732dfb204eefebc
Author: Xu Xin (vivo) <xu.xin@linux.dev>

ksm: add merge_only_zero_pages to merge only empty anonymous pages

This commit adds the /sys/kernel/mm/ksm/merge_only_zero_pages sysfs attribute
so ksmd can merge only zero-filled anonymous pages with the kernel zero page
while skipping the stable and unstable trees for pages carrying data.

> diff --git a/mm/ksm.c b/mm/ksm.c
> index 340c65aa14e58..35ed40747e30e 100644
> --- a/mm/ksm.c
> +++ b/mm/ksm.c

[ ... ]

> @@ -3679,12 +3695,48 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
>  	if (err)
>  		return -EINVAL;
>  
> +	/*
> +	 * merge_only_zero_pages depends on use_zero_pages: refuse to turn
> +	 * the latter off while the former is active, so the two cannot
> +	 * silently diverge.
> +	 */
> +	if (!value && ksm_merge_only_zero_pages)
> +		return -EBUSY;
> +
>  	ksm_use_zero_pages = value;
>  
>  	return count;
>  }
>  KSM_ATTR(use_zero_pages);
>  
> +static ssize_t merge_only_zero_pages_show(struct kobject *kobj,
> +					  struct kobj_attribute *attr, char *buf)
> +{
> +	return sysfs_emit(buf, "%u\n", ksm_merge_only_zero_pages);
> +}
> +static ssize_t merge_only_zero_pages_store(struct kobject *kobj,
> +					   struct kobj_attribute *attr,
> +					   const char *buf, size_t count)
> +{
> +	int err;
> +	bool value;
> +
> +	err = kstrtobool(buf, &value);
> +	if (err)
> +		return -EINVAL;
> +
> +	if (value && !ksm_use_zero_pages) {
> +		pr_warn_once("ksm: merge_only_zero_pages can only be enabled "
> +			     "when use_zero_pages is 1\n");
> +		return -EINVAL;
> +	}
> +
> +	ksm_merge_only_zero_pages = value;

[Severity: Medium]
Can concurrent writes to use_zero_pages and merge_only_zero_pages cause
ksm_use_zero_pages and ksm_merge_only_zero_pages to diverge?

Since these are separate sysfs attributes, kernfs does not serialize
use_zero_pages_store() against merge_only_zero_pages_store(), and neither
handler takes ksm_thread_mutex or another shared lock.

Starting from ksm_use_zero_pages == true and ksm_merge_only_zero_pages ==
false, if one task writes 0 to use_zero_pages while another concurrently
writes 1 to merge_only_zero_pages:

CPU 0                                   CPU 1
use_zero_pages_store()                  merge_only_zero_pages_store()
  if (!value &&                           if (value &&
      ksm_merge_only_zero_pages)              !ksm_use_zero_pages)
    /* sees false */                        /* sees true */
  ksm_use_zero_pages = false;             ksm_merge_only_zero_pages = true;

With ksm_use_zero_pages set to false and ksm_merge_only_zero_pages set to
true, try_to_merge_with_zero_page() always returns -EFAULT and the
subsequent ksm_merge_only_zero_pages check in cmp_and_merge_page() returns
early:

mm/ksm.c:cmp_and_merge_page() {
    ...
		if (!try_to_merge_with_zero_page(rmap_item, page))
			return;

		if (ksm_merge_only_zero_pages)
			return;
    ...
}

Wouldn't this leave ksmd scanning and checksumming pages without ever
merging either zero pages or non-zero pages?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106034841.233085-1-xu.xin@linux.dev?part=1

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Jinjiang Tu]

在 2026/11/6 11:48, xu.xin@linux.dev 写道:
> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
>
> Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
> which, when enabled, makes ksmd merge only empty (zeroed) anonymous
> pages with the kernel zero page and skip the stable and unstable trees
> entirely for pages that carry data.
>
> The attribute requires use_zero_pages to be 1, and the two are kept
> consistent by symmetric constraints: it cannot be enabled while
> use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
> off while it is active (-EBUSY). The new attribute can be changed at
> any time and is only effective for pages merged after the change.
>
> Benefits in three aspects
> =========================
> - Memory saving:
>    On memory-constrained systems such as Android and embedded devices,
>    zero-filled anonymous pages are the largest source of duplicate
>    anonymous memory. They are produced by userspace allocators: the
>    zeroing of freshly allocated Java objects and buffers in the ART heap
>    (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
>    preallocated media buffers. Merging them with the kernel zero page
>    frees a significant amount of physical memory, which matters as DRAM
>    costs keep rising. With this enabled, we measured 100MB–300MB saving
>    tested with several popular applications on aarch64 (Android).
>
> - CPU overhead reduction:
>    Merging only zero pages also reduces the overhead of ksmd. A page that
>    is not merged with the zero page carries data and can never be merged
>    with the zero page, so there is no point in searching the stable tree
>    or inserting it into the unstable tree for it. Skipping those tree
>    operations for data-bearing pages reduces both the CPU time and the
>    rhashtable memory footprint of ksmd, which makes zero page merging
>    practical on low-resource devices where full KSM scanning would be too
>    expensive.
>
> - Security:
>    Finally, this shrinks the KSM side-channel attack surface: only zero
>    pages are ever merged, so the sharing status of data-bearing pages is
>    no longer observable through KSM timing side channels.
>
> Note
> =====
> - Suitable for Android, embedded and low-memory devices, where zero-page
>    deduplication is wanted but full KSM scanning is too costly, and reduced
>    KSM side-channel exposure is desired. Many handset vendors may prefer
>    this knob, which can be used as a default fallback policy.
>
> - Not suitable for server and virtualization workloads containing many
>    non-zero duplicate pages, where vanilla KSM provides better memory
>    savings.

It’s a coincidence that I’m also working on similar optimizations.
In our scenario, we have a MySQL instance with over 700 GB of anonymous
memory. Most of this memory belongs to the buffer pool, which caches
data from storage; these pages cannot be merged. Of the remaining memory,
around 15 GB is consumed by hashtables, which contain many zero-filled
pages that are eligible for merging to reduce memory footprint.

When we enabled KSM for the entire MySQL instance, ksmd created rmap_item
metadata for each candidate page.The massive metadata overhead wiped out
any gains from page merging, and even resulted in a negative benefit.

For this reason, I’m trying to make ksmd merge only zero pages. The goal
is to have ksmd skip scanning non-zero pages entirely, thereby avoiding
the rmap_item metadata cost introduced by scanning non-mergeable pages.

1) Before get_next_rmap_item(), check whether the page is all zeros.
If not, skip it directly to avoid allocating an rmap_item.

2) When walking page tables to find candidate pages in ksm_next_page_pmd_entry(),
check whether the first and last contents are zero as a quick filter.
This is intended to speed up page-table traversal.

Would you mind to collaborate? For example, by including my optimization
as the second patch in your patch series?

Here is the diff. Basic functionality has been tested.

diff --git a/mm/ksm.c b/mm/ksm.c
index dbfb8f98b189..8a02976b0aba 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -296,6 +296,9 @@ static unsigned int zero_checksum __read_mostly;
  /* Whether to merge empty (zeroed) pages with actual zero pages */
  static bool ksm_use_zero_pages __read_mostly;
  
+/* Whether to merge empty (zeroed) pages only */
+static bool ksm_only_merge_zero_pages __read_mostly;
+
  /* Skip pages that couldn't be de-duplicated previously */
  /* Default to true at least temporarily, for testing */
  static bool ksm_smart_scan = true;
@@ -2524,8 +2527,42 @@ struct ksm_next_page_arg {
  	struct folio *folio;
  	struct page *page;
  	unsigned long addr;
+	unsigned int nr_to_skip;
  };
  
+static inline bool page_maybe_zero(struct page *page)
+{
+	unsigned long *p;
+	bool ret = true;
+	unsigned int last_pos = PAGE_SIZE / sizeof(*p) - 1;
+
+	p = kmap_local_page(page);
+	if (p[0] || p[last_pos])
+		ret = false;
+	kunmap_local(p);
+	return ret;
+}
+
+static struct page *ksm_skip_nonzero_pages(struct page *page,
+					   unsigned long *addr,
+					   unsigned long nr,
+					   unsigned int *nr_to_skip)
+{
+	unsigned long i;
+
+	for (i = 0; i < nr; i++) {
+		if (!*nr_to_skip)
+			break;
+		(*nr_to_skip)--;
+		if (page_maybe_zero(page + i))
+			break;
+	}
+	if (i == nr)
+		return NULL;
+	*addr += i << PAGE_SHIFT;
+	return page + i;
+}
+
  static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long end,
  		struct mm_walk *walk)
  {
@@ -2563,6 +2600,13 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
  				goto not_found_unlock;
  
  			page += ((addr & (PMD_SIZE - 1)) >> PAGE_SHIFT);
+			if (ksm_only_merge_zero_pages) {
+				page = ksm_skip_nonzero_pages(page, &addr,
+					(end - addr) >> PAGE_SHIFT,
+					&private->nr_to_skip);
+				if (!page)
+					goto not_found_unlock;
+			}
  			goto found_unlock;
  		}
  		spin_unlock(ptl);
@@ -2585,6 +2629,14 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
  
  		if (folio_is_zone_device(folio) || !folio_test_anon(folio))
  			continue;
+
+		if (ksm_only_merge_zero_pages && !folio_test_ksm(folio)) {
+			page = ksm_skip_nonzero_pages(page, &addr, 1,
+						      &private->nr_to_skip);
+			if (!page)
+				continue;
+		}
+
  		goto found_unlock;
  	}
  
@@ -2609,6 +2661,33 @@ static struct mm_walk_ops ksm_next_page_ops = {
  	.walk_lock = PGWALK_RDLOCK,
  };
  
+static bool should_skip_page(struct ksm_rmap_item **rmap_list,
+				  unsigned long addr, struct folio *folio,
+				  struct page *page)
+{
+	struct ksm_rmap_item *rmap_item;
+
+	if (!ksm_only_merge_zero_pages)
+		return false;
+
+	if (folio_test_ksm(folio))
+		return false;
+
+	if (pages_identical(page, ZERO_PAGE(0)))
+		return false;
+
+	while (*rmap_list) {
+		rmap_item = *rmap_list;
+		if ((rmap_item->address & PAGE_MASK) > addr)
+			break;
+		*rmap_list = rmap_item->rmap_list;
+		remove_rmap_item_from_tree(rmap_item);
+		free_rmap_item(rmap_item);
+	}
+	return true;
+}
+
+
  static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
  {
  	struct mm_struct *mm;
@@ -2705,6 +2784,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
  
  			int found;
  
+			ksm_next_page_arg.nr_to_skip = 1024;
  			found = walk_page_range_vma(vma, ksm_scan.address,
  						    vma->vm_end,
  						    &ksm_next_page_ops,
@@ -2722,6 +2802,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
  			if (tmp_page) {
  				flush_anon_page(vma, tmp_page, ksm_scan.address);
  				flush_dcache_page(tmp_page);
+
+				if (should_skip_page(ksm_scan.rmap_list, ksm_scan.address, folio, tmp_page)) {
+					folio_put(folio);
+					ksm_scan.address += PAGE_SIZE;
+					mmap_read_unlock(mm);
+					return ERR_PTR(-EAGAIN);
+				}
+
  				rmap_item = get_next_rmap_item(mm_slot,
  					ksm_scan.rmap_list, ksm_scan.address);
  				if (rmap_item) {
@@ -2824,6 +2912,10 @@ static void ksm_do_scan(unsigned int scan_npages)
  		rmap_item = scan_get_next_rmap_item(&page);
  		if (!rmap_item)
  			return;
+		if (rmap_item == ERR_PTR(-EAGAIN)) {
+			ksm_pages_scanned++;
+			continue;
+		}
  		cmp_and_merge_page(page, rmap_item);
  		put_page(page);
  		ksm_pages_scanned++;
@@ -3674,6 +3766,28 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
  }
  KSM_ATTR(use_zero_pages);

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[David Hildenbrand (Arm)]

On 11/6/26 04:48, xu.xin@linux.dev wrote:
> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
> 
> Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
> which, when enabled, makes ksmd merge only empty (zeroed) anonymous
> pages with the kernel zero page and skip the stable and unstable trees
> entirely for pages that carry data.
> 
> The attribute requires use_zero_pages to be 1, and the two are kept
> consistent by symmetric constraints: it cannot be enabled while
> use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
> off while it is active (-EBUSY). The new attribute can be changed at
> any time and is only effective for pages merged after the change.
> 
> Benefits in three aspects
> =========================
> - Memory saving:
>   On memory-constrained systems such as Android and embedded devices,
>   zero-filled anonymous pages are the largest source of duplicate
>   anonymous memory. They are produced by userspace allocators: the
>   zeroing of freshly allocated Java objects and buffers in the ART heap
>   (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
>   preallocated media buffers. Merging them with the kernel zero page
>   frees a significant amount of physical memory, which matters as DRAM
>   costs keep rising. With this enabled, we measured 100MB–300MB saving
>   tested with several popular applications on aarch64 (Android).
> 
> - CPU overhead reduction:
>   Merging only zero pages also reduces the overhead of ksmd. A page that
>   is not merged with the zero page carries data and can never be merged
>   with the zero page, so there is no point in searching the stable tree
>   or inserting it into the unstable tree for it. Skipping those tree
>   operations for data-bearing pages reduces both the CPU time and the
>   rhashtable memory footprint of ksmd, which makes zero page merging
>   practical on low-resource devices where full KSM scanning would be too
>   expensive.
> 
> - Security:
>   Finally, this shrinks the KSM side-channel attack surface: only zero
>   pages are ever merged, so the sharing status of data-bearing pages is
>   no longer observable through KSM timing side channels.

Lol. Did you come up with that or an LLM :)

> 
> Note
> =====
> - Suitable for Android, embedded and low-memory devices, where zero-page
>   deduplication is wanted but full KSM scanning is too costly, and reduced
>   KSM side-channel exposure is desired. Many handset vendors may prefer
>   this knob, which can be used as a default fallback policy.
> 
> - Not suitable for server and virtualization workloads containing many
>   non-zero duplicate pages, where vanilla KSM provides better memory
>   savings.

I proposed something similar when I was still at RH. My concern back then was
that we don't want/need all of the stable/unstable trees.

IIUC that's all still in place in your implementation, right?

-- 
Cheers,

David

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Xu Xin (vivo)]

On Sat, Nov 07, 2026 at 07:28:50PM +0100, David Hildenbrand (Arm) wrote:
> On 11/6/26 04:48, xu.xin@linux.dev wrote:
> > From: "Xu Xin (vivo)" <xu.xin@linux.dev>
> > 
> > Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
> > which, when enabled, makes ksmd merge only empty (zeroed) anonymous
> > pages with the kernel zero page and skip the stable and unstable trees
> > entirely for pages that carry data.
> > 
> > The attribute requires use_zero_pages to be 1, and the two are kept
> > consistent by symmetric constraints: it cannot be enabled while
> > use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
> > off while it is active (-EBUSY). The new attribute can be changed at
> > any time and is only effective for pages merged after the change.
> > 
> > Benefits in three aspects
> > =========================
> > - Memory saving:
> >   On memory-constrained systems such as Android and embedded devices,
> >   zero-filled anonymous pages are the largest source of duplicate
> >   anonymous memory. They are produced by userspace allocators: the
> >   zeroing of freshly allocated Java objects and buffers in the ART heap
> >   (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
> >   preallocated media buffers. Merging them with the kernel zero page
> >   frees a significant amount of physical memory, which matters as DRAM
> >   costs keep rising. With this enabled, we measured 100MB–300MB saving
> >   tested with several popular applications on aarch64 (Android).
> > 
> > - CPU overhead reduction:
> >   Merging only zero pages also reduces the overhead of ksmd. A page that
> >   is not merged with the zero page carries data and can never be merged
> >   with the zero page, so there is no point in searching the stable tree
> >   or inserting it into the unstable tree for it. Skipping those tree
> >   operations for data-bearing pages reduces both the CPU time and the
> >   rhashtable memory footprint of ksmd, which makes zero page merging
> >   practical on low-resource devices where full KSM scanning would be too
> >   expensive.
> > 
> > - Security:
> >   Finally, this shrinks the KSM side-channel attack surface: only zero
> >   pages are ever merged, so the sharing status of data-bearing pages is
> >   no longer observable through KSM timing side channels.
> 
> Lol. Did you come up with that or an LLM :)

Ha, fair question :) This came from our own investigation. We were looking
into whether KSM can actually provide benefits on Android phones, and this
observation came from real-world application workloads. To be fully honest:
I did use an LLM to help polish the English wording of the commit description,
but I carefully rechecked and verified every sentence myself..

> 
> > 
> > Note
> > =====
> > - Suitable for Android, embedded and low-memory devices, where zero-page
> >   deduplication is wanted but full KSM scanning is too costly, and reduced
> >   KSM side-channel exposure is desired. Many handset vendors may prefer
> >   this knob, which can be used as a default fallback policy.
> > 
> > - Not suitable for server and virtualization workloads containing many
> >   non-zero duplicate pages, where vanilla KSM provides better memory
> >   savings.
> 
> I proposed something similar when I was still at RH. My concern back then was
> that we don't want/need all of the stable/unstable trees.
> 
> IIUC that's all still in place in your implementation, right?

We still keep the stable/unstable tree data structures, but zero-page merging
does not participate in the stable/unstable tree search path. One more reason
for keeping them is that merge_only_zero_page is designed as a dynamic knob
that can be toggled at runtime, so when it is disabled KSM can fall back to
the traditional full KSM behavior.

> 
> -- 
> Cheers,
> 
> David

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Xu Xin (vivo)]

On Sat, Nov 07, 2026 at 04:13:40PM +0800, Jinjiang Tu wrote:
> 
> 在 2026/11/6 11:48, xu.xin@linux.dev 写道:
> > From: "Xu Xin (vivo)" <xu.xin@linux.dev>
> > 
> >    KSM side-channel exposure is desired. Many handset vendors may prefer
> >    this knob, which can be used as a default fallback policy.
> > 
> > - Not suitable for server and virtualization workloads containing many
> >    non-zero duplicate pages, where vanilla KSM provides better memory
> >    savings.
> 
> It’s a coincidence that I’m also working on similar optimizations.
> In our scenario, we have a MySQL instance with over 700 GB of anonymous
> memory. Most of this memory belongs to the buffer pool, which caches
> data from storage; these pages cannot be merged. Of the remaining memory,
> around 15 GB is consumed by hashtables, which contain many zero-filled
> pages that are eligible for merging to reduce memory footprint.
> 
> When we enabled KSM for the entire MySQL instance, ksmd created rmap_item
> metadata for each candidate page.The massive metadata overhead wiped out
> any gains from page merging, and even resulted in a negative benefit.
> 
> For this reason, I’m trying to make ksmd merge only zero pages. The goal
> is to have ksmd skip scanning non-zero pages entirely, thereby avoiding
> the rmap_item metadata cost introduced by scanning non-mergeable pages.

Skip generating ksm_rmap_item for non-zero pages ? That dose make sense to me.

> 
> 1) Before get_next_rmap_item(), check whether the page is all zeros.
> If not, skip it directly to avoid allocating an rmap_item.
> 
> 2) When walking page tables to find candidate pages in ksm_next_page_pmd_entry(),
> check whether the first and last contents are zero as a quick filter.
> This is intended to speed up page-table traversal.
> 
> Would you mind to collaborate? For example, by including my optimization
> as the second patch in your patch series?

Happy to do it.

> 
> Here is the diff. Basic functionality has been tested.
> 
> diff --git a/mm/ksm.c b/mm/ksm.c
> index dbfb8f98b189..8a02976b0aba 100644
> --- a/mm/ksm.c
> +++ b/mm/ksm.c
> @@ -296,6 +296,9 @@ static unsigned int zero_checksum __read_mostly;
>  /* Whether to merge empty (zeroed) pages with actual zero pages */
>  static bool ksm_use_zero_pages __read_mostly;
> +/* Whether to merge empty (zeroed) pages only */
> +static bool ksm_only_merge_zero_pages __read_mostly;
> +
>  /* Skip pages that couldn't be de-duplicated previously */
>  /* Default to true at least temporarily, for testing */
>  static bool ksm_smart_scan = true;
> @@ -2524,8 +2527,42 @@ struct ksm_next_page_arg {
>  	struct folio *folio;
>  	struct page *page;
>  	unsigned long addr;
> +	unsigned int nr_to_skip;
>  };
> +static inline bool page_maybe_zero(struct page *page)
> +{
> +	unsigned long *p;
> +	bool ret = true;
> +	unsigned int last_pos = PAGE_SIZE / sizeof(*p) - 1;
> +
> +	p = kmap_local_page(page);
> +	if (p[0] || p[last_pos])
> +		ret = false;
> +	kunmap_local(p);
> +	return ret;
> +}
> +
> +static struct page *ksm_skip_nonzero_pages(struct page *page,
> +					   unsigned long *addr,
> +					   unsigned long nr,
> +					   unsigned int *nr_to_skip)
> +{
> +	unsigned long i;
> +
> +	for (i = 0; i < nr; i++) {
> +		if (!*nr_to_skip)
> +			break;
> +		(*nr_to_skip)--;
> +		if (page_maybe_zero(page + i))
> +			break;
> +	}
> +	if (i == nr)
> +		return NULL;
> +	*addr += i << PAGE_SHIFT;
> +	return page + i;
> +}
> +
>  static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long end,
>  		struct mm_walk *walk)
>  {
> @@ -2563,6 +2600,13 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>  				goto not_found_unlock;
>  			page += ((addr & (PMD_SIZE - 1)) >> PAGE_SHIFT);
> +			if (ksm_only_merge_zero_pages) {
> +				page = ksm_skip_nonzero_pages(page, &addr,
> +					(end - addr) >> PAGE_SHIFT,
> +					&private->nr_to_skip);
> +				if (!page)
> +					goto not_found_unlock;
> +			}
>  			goto found_unlock;
>  		}
>  		spin_unlock(ptl);
> @@ -2585,6 +2629,14 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>  		if (folio_is_zone_device(folio) || !folio_test_anon(folio))
>  			continue;
> +
> +		if (ksm_only_merge_zero_pages && !folio_test_ksm(folio)) {
> +			page = ksm_skip_nonzero_pages(page, &addr, 1,
> +						      &private->nr_to_skip);
> +			if (!page)
> +				continue;
> +		}
> +
>  		goto found_unlock;
>  	}
> @@ -2609,6 +2661,33 @@ static struct mm_walk_ops ksm_next_page_ops = {
>  	.walk_lock = PGWALK_RDLOCK,
>  };
> +static bool should_skip_page(struct ksm_rmap_item **rmap_list,
> +				  unsigned long addr, struct folio *folio,
> +				  struct page *page)
> +{
> +	struct ksm_rmap_item *rmap_item;
> +
> +	if (!ksm_only_merge_zero_pages)
> +		return false;
> +
> +	if (folio_test_ksm(folio))
> +		return false;
> +
> +	if (pages_identical(page, ZERO_PAGE(0)))
> +		return false;
> +
> +	while (*rmap_list) {
> +		rmap_item = *rmap_list;
> +		if ((rmap_item->address & PAGE_MASK) > addr)
> +			break;
> +		*rmap_list = rmap_item->rmap_list;
> +		remove_rmap_item_from_tree(rmap_item);
> +		free_rmap_item(rmap_item);
> +	}
> +	return true;
> +}
> +
> +
>  static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>  {
>  	struct mm_struct *mm;
> @@ -2705,6 +2784,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>  			int found;
> +			ksm_next_page_arg.nr_to_skip = 1024;
>  			found = walk_page_range_vma(vma, ksm_scan.address,
>  						    vma->vm_end,
>  						    &ksm_next_page_ops,
> @@ -2722,6 +2802,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>  			if (tmp_page) {
>  				flush_anon_page(vma, tmp_page, ksm_scan.address);
>  				flush_dcache_page(tmp_page);
> +
> +				if (should_skip_page(ksm_scan.rmap_list, ksm_scan.address, folio, tmp_page)) {
> +					folio_put(folio);
> +					ksm_scan.address += PAGE_SIZE;
> +					mmap_read_unlock(mm);
> +					return ERR_PTR(-EAGAIN);
> +				}
> +
>  				rmap_item = get_next_rmap_item(mm_slot,
>  					ksm_scan.rmap_list, ksm_scan.address);
>  				if (rmap_item) {
> @@ -2824,6 +2912,10 @@ static void ksm_do_scan(unsigned int scan_npages)
>  		rmap_item = scan_get_next_rmap_item(&page);
>  		if (!rmap_item)
>  			return;
> +		if (rmap_item == ERR_PTR(-EAGAIN)) {
> +			ksm_pages_scanned++;
> +			continue;
> +		}
>  		cmp_and_merge_page(page, rmap_item);
>  		put_page(page);
>  		ksm_pages_scanned++;
> @@ -3674,6 +3766,28 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
>  }
>  KSM_ATTR(use_zero_pages);
> 

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Jinjiang Tu]

在 2026/11/9 10:02, Xu Xin (vivo) 写道:
> On Sat, Nov 07, 2026 at 04:13:40PM +0800, Jinjiang Tu wrote:
>> 在 2026/11/6 11:48, xu.xin@linux.dev 写道:
>>> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
>>>
>>>     KSM side-channel exposure is desired. Many handset vendors may prefer
>>>     this knob, which can be used as a default fallback policy.
>>>
>>> - Not suitable for server and virtualization workloads containing many
>>>     non-zero duplicate pages, where vanilla KSM provides better memory
>>>     savings.
>> It’s a coincidence that I’m also working on similar optimizations.
>> In our scenario, we have a MySQL instance with over 700 GB of anonymous
>> memory. Most of this memory belongs to the buffer pool, which caches
>> data from storage; these pages cannot be merged. Of the remaining memory,
>> around 15 GB is consumed by hashtables, which contain many zero-filled
>> pages that are eligible for merging to reduce memory footprint.
>>
>> When we enabled KSM for the entire MySQL instance, ksmd created rmap_item
>> metadata for each candidate page.The massive metadata overhead wiped out
>> any gains from page merging, and even resulted in a negative benefit.
>>
>> For this reason, I’m trying to make ksmd merge only zero pages. The goal
>> is to have ksmd skip scanning non-zero pages entirely, thereby avoiding
>> the rmap_item metadata cost introduced by scanning non-mergeable pages.
> Skip generating ksm_rmap_item for non-zero pages ? That dose make sense to me.

Yes.

>
>> 1) Before get_next_rmap_item(), check whether the page is all zeros.
>> If not, skip it directly to avoid allocating an rmap_item.
>>
>> 2) When walking page tables to find candidate pages in ksm_next_page_pmd_entry(),
>> check whether the first and last contents are zero as a quick filter.
>> This is intended to speed up page-table traversal.
>>
>> Would you mind to collaborate? For example, by including my optimization
>> as the second patch in your patch series?
> Happy to do it.

Thanks.

>
>> Here is the diff. Basic functionality has been tested.
>>
>> diff --git a/mm/ksm.c b/mm/ksm.c
>> index dbfb8f98b189..8a02976b0aba 100644
>> --- a/mm/ksm.c
>> +++ b/mm/ksm.c
>> @@ -296,6 +296,9 @@ static unsigned int zero_checksum __read_mostly;
>>   /* Whether to merge empty (zeroed) pages with actual zero pages */
>>   static bool ksm_use_zero_pages __read_mostly;
>> +/* Whether to merge empty (zeroed) pages only */
>> +static bool ksm_only_merge_zero_pages __read_mostly;
>> +
>>   /* Skip pages that couldn't be de-duplicated previously */
>>   /* Default to true at least temporarily, for testing */
>>   static bool ksm_smart_scan = true;
>> @@ -2524,8 +2527,42 @@ struct ksm_next_page_arg {
>>   	struct folio *folio;
>>   	struct page *page;
>>   	unsigned long addr;
>> +	unsigned int nr_to_skip;
>>   };
>> +static inline bool page_maybe_zero(struct page *page)
>> +{
>> +	unsigned long *p;
>> +	bool ret = true;
>> +	unsigned int last_pos = PAGE_SIZE / sizeof(*p) - 1;
>> +
>> +	p = kmap_local_page(page);
>> +	if (p[0] || p[last_pos])
>> +		ret = false;
>> +	kunmap_local(p);
>> +	return ret;
>> +}
>> +
>> +static struct page *ksm_skip_nonzero_pages(struct page *page,
>> +					   unsigned long *addr,
>> +					   unsigned long nr,
>> +					   unsigned int *nr_to_skip)
>> +{
>> +	unsigned long i;
>> +
>> +	for (i = 0; i < nr; i++) {
>> +		if (!*nr_to_skip)
>> +			break;
>> +		(*nr_to_skip)--;
>> +		if (page_maybe_zero(page + i))
>> +			break;
>> +	}
>> +	if (i == nr)
>> +		return NULL;
>> +	*addr += i << PAGE_SHIFT;
>> +	return page + i;
>> +}
>> +
>>   static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long end,
>>   		struct mm_walk *walk)
>>   {
>> @@ -2563,6 +2600,13 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>>   				goto not_found_unlock;
>>   			page += ((addr & (PMD_SIZE - 1)) >> PAGE_SHIFT);
>> +			if (ksm_only_merge_zero_pages) {
>> +				page = ksm_skip_nonzero_pages(page, &addr,
>> +					(end - addr) >> PAGE_SHIFT,
>> +					&private->nr_to_skip);
>> +				if (!page)
>> +					goto not_found_unlock;
>> +			}
>>   			goto found_unlock;
>>   		}
>>   		spin_unlock(ptl);
>> @@ -2585,6 +2629,14 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>>   		if (folio_is_zone_device(folio) || !folio_test_anon(folio))
>>   			continue;
>> +
>> +		if (ksm_only_merge_zero_pages && !folio_test_ksm(folio)) {
>> +			page = ksm_skip_nonzero_pages(page, &addr, 1,
>> +						      &private->nr_to_skip);
>> +			if (!page)
>> +				continue;
>> +		}
>> +
>>   		goto found_unlock;
>>   	}
>> @@ -2609,6 +2661,33 @@ static struct mm_walk_ops ksm_next_page_ops = {
>>   	.walk_lock = PGWALK_RDLOCK,
>>   };
>> +static bool should_skip_page(struct ksm_rmap_item **rmap_list,
>> +				  unsigned long addr, struct folio *folio,
>> +				  struct page *page)
>> +{
>> +	struct ksm_rmap_item *rmap_item;
>> +
>> +	if (!ksm_only_merge_zero_pages)
>> +		return false;
>> +
>> +	if (folio_test_ksm(folio))
>> +		return false;
>> +
>> +	if (pages_identical(page, ZERO_PAGE(0)))
>> +		return false;
>> +
>> +	while (*rmap_list) {
>> +		rmap_item = *rmap_list;
>> +		if ((rmap_item->address & PAGE_MASK) > addr)
>> +			break;
>> +		*rmap_list = rmap_item->rmap_list;
>> +		remove_rmap_item_from_tree(rmap_item);
>> +		free_rmap_item(rmap_item);
>> +	}
>> +	return true;
>> +}
>> +
>> +
>>   static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>>   {
>>   	struct mm_struct *mm;
>> @@ -2705,6 +2784,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>>   			int found;
>> +			ksm_next_page_arg.nr_to_skip = 1024;
>>   			found = walk_page_range_vma(vma, ksm_scan.address,
>>   						    vma->vm_end,
>>   						    &ksm_next_page_ops,
>> @@ -2722,6 +2802,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>>   			if (tmp_page) {
>>   				flush_anon_page(vma, tmp_page, ksm_scan.address);
>>   				flush_dcache_page(tmp_page);
>> +
>> +				if (should_skip_page(ksm_scan.rmap_list, ksm_scan.address, folio, tmp_page)) {
>> +					folio_put(folio);
>> +					ksm_scan.address += PAGE_SIZE;
>> +					mmap_read_unlock(mm);
>> +					return ERR_PTR(-EAGAIN);
>> +				}
>> +
>>   				rmap_item = get_next_rmap_item(mm_slot,
>>   					ksm_scan.rmap_list, ksm_scan.address);
>>   				if (rmap_item) {
>> @@ -2824,6 +2912,10 @@ static void ksm_do_scan(unsigned int scan_npages)
>>   		rmap_item = scan_get_next_rmap_item(&page);
>>   		if (!rmap_item)
>>   			return;
>> +		if (rmap_item == ERR_PTR(-EAGAIN)) {
>> +			ksm_pages_scanned++;
>> +			continue;
>> +		}
>>   		cmp_and_merge_page(page, rmap_item);
>>   		put_page(page);
>>   		ksm_pages_scanned++;
>> @@ -3674,6 +3766,28 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
>>   }
>>   KSM_ATTR(use_zero_pages);
>>

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Xu Xin (vivo)]

On Sat, Nov 07, 2026 at 04:13:40PM +0800, Jinjiang Tu wrote:
> 
> 在 2026/11/6 11:48, xu.xin@linux.dev 写道:
> > From: "Xu Xin (vivo)" <xu.xin@linux.dev>
> > 
> > Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
> > which, when enabled, makes ksmd merge only empty (zeroed) anonymous
> > pages with the kernel zero page and skip the stable and unstable trees
> > entirely for pages that carry data.
> > 
> > The attribute requires use_zero_pages to be 1, and the two are kept
> > consistent by symmetric constraints: it cannot be enabled while
> > use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
> > off while it is active (-EBUSY). The new attribute can be changed at
> > any time and is only effective for pages merged after the change.
> > 
> > Benefits in three aspects
> > =========================
> > - Memory saving:
> >    On memory-constrained systems such as Android and embedded devices,
> >    zero-filled anonymous pages are the largest source of duplicate
> >    anonymous memory. They are produced by userspace allocators: the
> >    zeroing of freshly allocated Java objects and buffers in the ART heap
> >    (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
> >    preallocated media buffers. Merging them with the kernel zero page
> >    frees a significant amount of physical memory, which matters as DRAM
> >    costs keep rising. With this enabled, we measured 100MB–300MB saving
> >    tested with several popular applications on aarch64 (Android).
> > 
> > - CPU overhead reduction:
> >    Merging only zero pages also reduces the overhead of ksmd. A page that
> >    is not merged with the zero page carries data and can never be merged
> >    with the zero page, so there is no point in searching the stable tree
> >    or inserting it into the unstable tree for it. Skipping those tree
> >    operations for data-bearing pages reduces both the CPU time and the
> >    rhashtable memory footprint of ksmd, which makes zero page merging
> >    practical on low-resource devices where full KSM scanning would be too
> >    expensive.
> > 
> > - Security:
> >    Finally, this shrinks the KSM side-channel attack surface: only zero
> >    pages are ever merged, so the sharing status of data-bearing pages is
> >    no longer observable through KSM timing side channels.
> > 
> > Note
> > =====
> > - Suitable for Android, embedded and low-memory devices, where zero-page
> >    deduplication is wanted but full KSM scanning is too costly, and reduced
> >    KSM side-channel exposure is desired. Many handset vendors may prefer
> >    this knob, which can be used as a default fallback policy.
> > 
> > - Not suitable for server and virtualization workloads containing many
> >    non-zero duplicate pages, where vanilla KSM provides better memory
> >    savings.
> 
> It’s a coincidence that I’m also working on similar optimizations.
> In our scenario, we have a MySQL instance with over 700 GB of anonymous
> memory. Most of this memory belongs to the buffer pool, which caches
> data from storage; these pages cannot be merged. Of the remaining memory,
> around 15 GB is consumed by hashtables, which contain many zero-filled
> pages that are eligible for merging to reduce memory footprint.
> 
> When we enabled KSM for the entire MySQL instance, ksmd created rmap_item
> metadata for each candidate page.The massive metadata overhead wiped out
> any gains from page merging, and even resulted in a negative benefit.
> 
> For this reason, I’m trying to make ksmd merge only zero pages. The goal
> is to have ksmd skip scanning non-zero pages entirely, thereby avoiding
> the rmap_item metadata cost introduced by scanning non-mergeable pages.
> 
> 1) Before get_next_rmap_item(), check whether the page is all zeros.
> If not, skip it directly to avoid allocating an rmap_item.
> 
> 2) When walking page tables to find candidate pages in ksm_next_page_pmd_entry(),
> check whether the first and last contents are zero as a quick filter.
> This is intended to speed up page-table traversal.
> 
> Would you mind to collaborate? For example, by including my optimization
> as the second patch in your patch series?
>
 
Thanks for sharing your work — the motivation is clear and our goals are compatible

> Here is the diff. Basic functionality has been tested.
> 
> diff --git a/mm/ksm.c b/mm/ksm.c
> index dbfb8f98b189..8a02976b0aba 100644
> --- a/mm/ksm.c
> +++ b/mm/ksm.c
> @@ -296,6 +296,9 @@ static unsigned int zero_checksum __read_mostly;
>  /* Whether to merge empty (zeroed) pages with actual zero pages */
>  static bool ksm_use_zero_pages __read_mostly;
> +/* Whether to merge empty (zeroed) pages only */
> +static bool ksm_only_merge_zero_pages __read_mostly;
> +
>  /* Skip pages that couldn't be de-duplicated previously */
>  /* Default to true at least temporarily, for testing */
>  static bool ksm_smart_scan = true;
> @@ -2524,8 +2527,42 @@ struct ksm_next_page_arg {
>  	struct folio *folio;
>  	struct page *page;
>  	unsigned long addr;
> +	unsigned int nr_to_skip;
>  };
> +static inline bool page_maybe_zero(struct page *page)
> +{
> +	unsigned long *p;
> +	bool ret = true;
> +	unsigned int last_pos = PAGE_SIZE / sizeof(*p) - 1;
> +
> +	p = kmap_local_page(page);
> +	if (p[0] || p[last_pos])
> +		ret = false;
> +	kunmap_local(p);
> +	return ret;
> +}
> +
> +static struct page *ksm_skip_nonzero_pages(struct page *page,
> +					   unsigned long *addr,
> +					   unsigned long nr,
> +					   unsigned int *nr_to_skip)
> +{
> +	unsigned long i;
> +
> +	for (i = 0; i < nr; i++) {
> +		if (!*nr_to_skip)
> +			break;
> +		(*nr_to_skip)--;
> +		if (page_maybe_zero(page + i))
> +			break;
> +	}
> +	if (i == nr)
> +		return NULL;
> +	*addr += i << PAGE_SHIFT;
> +	return page + i;
> +}
> +
>  static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long end,
>  		struct mm_walk *walk)
>  {
> @@ -2563,6 +2600,13 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>  				goto not_found_unlock;
>  			page += ((addr & (PMD_SIZE - 1)) >> PAGE_SHIFT);
> +			if (ksm_only_merge_zero_pages) {
> +				page = ksm_skip_nonzero_pages(page, &addr,
> +					(end - addr) >> PAGE_SHIFT,
> +					&private->nr_to_skip);
> +				if (!page)
> +					goto not_found_unlock;
> +			}
>  			goto found_unlock;
>  		}
>  		spin_unlock(ptl);
> @@ -2585,6 +2629,14 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>  		if (folio_is_zone_device(folio) || !folio_test_anon(folio))
>  			continue;
> +
> +		if (ksm_only_merge_zero_pages && !folio_test_ksm(folio)) {
> +			page = ksm_skip_nonzero_pages(page, &addr, 1,
> +						      &private->nr_to_skip);
> +			if (!page)
> +				continue;
> +		}
> +
>  		goto found_unlock;
>  	}

I'd like to take you up on the collaboration, with a few adjustments to your diff:

I'd keep only the precise check in should_skip_page() before get_next_rmap_item(),
and remove page_maybe_zero(), ksm_skip_nonzero_pages(), and nr_to_skip.

Two concerns drive this:

(1) 'nr_to_skip' conflicts semantically with 'pages_to_scan'. nr_to_skip (initialized to 1024,
reset per walk) is an independent cap on how much work one walk may do, decoupled from the
user-facing pages_to_scan knob. pages_to_scan controls how many pages one scan cycle processes,
but the pages skipped inside a walk are not accounted for in that budget, so the actual amount
of work ksmd does is no longer fully described by pages_to_scan. A user tuning pages_to_scan to
bound ksmd's CPU/power on a battery-powered device would no longer get a bound they can reason
about.

(2) The same non-zero page can go through page_maybe_zero() more than once. It's only a heuristic
(first and last word), so a page that is zero at both ends but carries data in the middle passes
the fast filter, is returned as a candidate, and is then rejected by the exact pages_identical()
check in should_skip_page(). Because KSM re-walks the same address range on every subsequent scan
cycle, that same page is fed through page_maybe_zero() again on each cycle and rejected again
— the cost is paid repeatedly while the page is never actually filtered out.

Given the above, I'd like to drop ksm _skip_nonzero_pages() and the coarse filter in the page
walking path from this round of submission, to keep the code simpler and clearer overall.
The precise check in should_skip_page() is kept, so non-zero pages still never get an rmap_item
— the metadata saving you're after is preserved. Also, I'll keep the attribute name consistent
with yours.

On the tags: I'd like to credit you as Co-developed-by and add your Signed-off-by, per
submitting-patches.rst, with my own Signed-off-by above them.

If you have concerns, let me know before I post.

> @@ -2609,6 +2661,33 @@ static struct mm_walk_ops ksm_next_page_ops = {
>  	.walk_lock = PGWALK_RDLOCK,
>  };
> +static bool should_skip_page(struct ksm_rmap_item **rmap_list,
> +				  unsigned long addr, struct folio *folio,
> +				  struct page *page)
> +{
> +	struct ksm_rmap_item *rmap_item;
> +
> +	if (!ksm_only_merge_zero_pages)
> +		return false;
> +
> +	if (folio_test_ksm(folio))
> +		return false;
> +
> +	if (pages_identical(page, ZERO_PAGE(0)))
> +		return false;
> +
> +	while (*rmap_list) {
> +		rmap_item = *rmap_list;
> +		if ((rmap_item->address & PAGE_MASK) > addr)
> +			break;
> +		*rmap_list = rmap_item->rmap_list;
> +		remove_rmap_item_from_tree(rmap_item);
> +		free_rmap_item(rmap_item);
> +	}
> +	return true;
> +}
> +
> +
>  static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>  {
>  	struct mm_struct *mm;
> @@ -2705,6 +2784,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>  			int found;
> +			ksm_next_page_arg.nr_to_skip = 1024;
>  			found = walk_page_range_vma(vma, ksm_scan.address,
>  						    vma->vm_end,
>  						    &ksm_next_page_ops,
> @@ -2722,6 +2802,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>  			if (tmp_page) {
>  				flush_anon_page(vma, tmp_page, ksm_scan.address);
>  				flush_dcache_page(tmp_page);
> +
> +				if (should_skip_page(ksm_scan.rmap_list, ksm_scan.address, folio, tmp_page)) {
> +					folio_put(folio);
> +					ksm_scan.address += PAGE_SIZE;
> +					mmap_read_unlock(mm);
> +					return ERR_PTR(-EAGAIN);
> +				}
> +
>  				rmap_item = get_next_rmap_item(mm_slot,
>  					ksm_scan.rmap_list, ksm_scan.address);
>  				if (rmap_item) {
> @@ -2824,6 +2912,10 @@ static void ksm_do_scan(unsigned int scan_npages)
>  		rmap_item = scan_get_next_rmap_item(&page);
>  		if (!rmap_item)
>  			return;
> +		if (rmap_item == ERR_PTR(-EAGAIN)) {
> +			ksm_pages_scanned++;
> +			continue;
> +		}
>  		cmp_and_merge_page(page, rmap_item);
>  		put_page(page);
>  		ksm_pages_scanned++;
> @@ -3674,6 +3766,28 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
>  }
>  KSM_ATTR(use_zero_pages);
> 

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[David Hildenbrand (Arm)]

On 11/9/26 02:39, Xu Xin (vivo) wrote:
> On Sat, Nov 07, 2026 at 07:28:50PM +0100, David Hildenbrand (Arm) wrote:
>> On 11/6/26 04:48, xu.xin@linux.dev wrote:
>>> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
>>>
>>> Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
>>> which, when enabled, makes ksmd merge only empty (zeroed) anonymous
>>> pages with the kernel zero page and skip the stable and unstable trees
>>> entirely for pages that carry data.
>>>
>>> The attribute requires use_zero_pages to be 1, and the two are kept
>>> consistent by symmetric constraints: it cannot be enabled while
>>> use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
>>> off while it is active (-EBUSY). The new attribute can be changed at
>>> any time and is only effective for pages merged after the change.
>>>
>>> Benefits in three aspects
>>> =========================
>>> - Memory saving:
>>>   On memory-constrained systems such as Android and embedded devices,
>>>   zero-filled anonymous pages are the largest source of duplicate
>>>   anonymous memory. They are produced by userspace allocators: the
>>>   zeroing of freshly allocated Java objects and buffers in the ART heap
>>>   (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
>>>   preallocated media buffers. Merging them with the kernel zero page
>>>   frees a significant amount of physical memory, which matters as DRAM
>>>   costs keep rising. With this enabled, we measured 100MB–300MB saving
>>>   tested with several popular applications on aarch64 (Android).
>>>
>>> - CPU overhead reduction:
>>>   Merging only zero pages also reduces the overhead of ksmd. A page that
>>>   is not merged with the zero page carries data and can never be merged
>>>   with the zero page, so there is no point in searching the stable tree
>>>   or inserting it into the unstable tree for it. Skipping those tree
>>>   operations for data-bearing pages reduces both the CPU time and the
>>>   rhashtable memory footprint of ksmd, which makes zero page merging
>>>   practical on low-resource devices where full KSM scanning would be too
>>>   expensive.
>>>
>>> - Security:
>>>   Finally, this shrinks the KSM side-channel attack surface: only zero
>>>   pages are ever merged, so the sharing status of data-bearing pages is
>>>   no longer observable through KSM timing side channels.
>>
>> Lol. Did you come up with that or an LLM :)
> 
> Ha, fair question :) This came from our own investigation. We were looking
> into whether KSM can actually provide benefits on Android phones, and this
> observation came from real-world application workloads. To be fully honest:
> I did use an LLM to help polish the English wording of the commit description,
> but I carefully rechecked and verified every sentence myself..

The reason why I ask is that this effectively disables most deduplication where
KSM was originally invented for: VMs running the same kernels, libaries,
executables etc.

So yeah, disabling most deduplication shrinks side-channels ... by disabling the
core functionality of KSM, to dedupicate pages with the same content :)

> 
>>
>>>
>>> Note
>>> =====
>>> - Suitable for Android, embedded and low-memory devices, where zero-page
>>>   deduplication is wanted but full KSM scanning is too costly, and reduced
>>>   KSM side-channel exposure is desired. Many handset vendors may prefer
>>>   this knob, which can be used as a default fallback policy.
>>>
>>> - Not suitable for server and virtualization workloads containing many
>>>   non-zero duplicate pages, where vanilla KSM provides better memory
>>>   savings.
>>
>> I proposed something similar when I was still at RH. My concern back then was
>> that we don't want/need all of the stable/unstable trees.
>>
>> IIUC that's all still in place in your implementation, right?
> 
> We still keep the stable/unstable tree data structures, but zero-page merging
> does not participate in the stable/unstable tree search path. One more reason
> for keeping them is that merge_only_zero_page is designed as a dynamic knob
> that can be toggled at runtime, so when it is disabled KSM can fall back to
> the traditional full KSM behavior.


What's the memory overhead from keeping these datastructures in the
merge_only_zero_page-only path?


-- 
Cheers,

David

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[David Hildenbrand (Arm)]

On 11/9/26 03:02, Xu Xin (vivo) wrote:
> On Sat, Nov 07, 2026 at 04:13:40PM +0800, Jinjiang Tu wrote:
>>
>> 在 2026/11/6 11:48, xu.xin@linux.dev 写道:
>>> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
>>>
>>>    KSM side-channel exposure is desired. Many handset vendors may prefer
>>>    this knob, which can be used as a default fallback policy.
>>>
>>> - Not suitable for server and virtualization workloads containing many
>>>    non-zero duplicate pages, where vanilla KSM provides better memory
>>>    savings.
>>
>> It’s a coincidence that I’m also working on similar optimizations.
>> In our scenario, we have a MySQL instance with over 700 GB of anonymous
>> memory. Most of this memory belongs to the buffer pool, which caches
>> data from storage; these pages cannot be merged. Of the remaining memory,
>> around 15 GB is consumed by hashtables, which contain many zero-filled
>> pages that are eligible for merging to reduce memory footprint.
>>
>> When we enabled KSM for the entire MySQL instance, ksmd created rmap_item
>> metadata for each candidate page.The massive metadata overhead wiped out
>> any gains from page merging, and even resulted in a negative benefit.
>>
>> For this reason, I’m trying to make ksmd merge only zero pages. The goal
>> is to have ksmd skip scanning non-zero pages entirely, thereby avoiding
>> the rmap_item metadata cost introduced by scanning non-mergeable pages.
> 
> Skip generating ksm_rmap_item for non-zero pages ? That dose make sense to me.

Yes, we should avoid any such memory overhead in that case.

-- 
Cheers,

David

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Xu Xin (vivo)]

On Mon, Nov 09, 2026 at 06:41:39PM +0100, David Hildenbrand (Arm) wrote:
> On 11/9/26 02:39, Xu Xin (vivo) wrote:
> > On Sat, Nov 07, 2026 at 07:28:50PM +0100, David Hildenbrand (Arm) wrote:
> >> On 11/6/26 04:48, xu.xin@linux.dev wrote:
> >>> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
> >>>
> >>> Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
> >>> which, when enabled, makes ksmd merge only empty (zeroed) anonymous
> >>> pages with the kernel zero page and skip the stable and unstable trees
> >>> entirely for pages that carry data.
> >>>
> >>> The attribute requires use_zero_pages to be 1, and the two are kept
> >>> consistent by symmetric constraints: it cannot be enabled while
> >>> use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
> >>> off while it is active (-EBUSY). The new attribute can be changed at
> >>> any time and is only effective for pages merged after the change.
> >>>
> >>> Benefits in three aspects
> >>> =========================
> >>> - Memory saving:
> >>>   On memory-constrained systems such as Android and embedded devices,
> >>>   zero-filled anonymous pages are the largest source of duplicate
> >>>   anonymous memory. They are produced by userspace allocators: the
> >>>   zeroing of freshly allocated Java objects and buffers in the ART heap
> >>>   (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
> >>>   preallocated media buffers. Merging them with the kernel zero page
> >>>   frees a significant amount of physical memory, which matters as DRAM
> >>>   costs keep rising. With this enabled, we measured 100MB–300MB saving
> >>>   tested with several popular applications on aarch64 (Android).
> >>>
> >>> - CPU overhead reduction:
> >>>   Merging only zero pages also reduces the overhead of ksmd. A page that
> >>>   is not merged with the zero page carries data and can never be merged
> >>>   with the zero page, so there is no point in searching the stable tree
> >>>   or inserting it into the unstable tree for it. Skipping those tree
> >>>   operations for data-bearing pages reduces both the CPU time and the
> >>>   rhashtable memory footprint of ksmd, which makes zero page merging
> >>>   practical on low-resource devices where full KSM scanning would be too
> >>>   expensive.
> >>>
> >>> - Security:
> >>>   Finally, this shrinks the KSM side-channel attack surface: only zero
> >>>   pages are ever merged, so the sharing status of data-bearing pages is
> >>>   no longer observable through KSM timing side channels.
> >>
> >> Lol. Did you come up with that or an LLM :)
> > 
> > Ha, fair question :) This came from our own investigation. We were looking
> > into whether KSM can actually provide benefits on Android phones, and this
> > observation came from real-world application workloads. To be fully honest:
> > I did use an LLM to help polish the English wording of the commit description,
> > but I carefully rechecked and verified every sentence myself..
> 
> The reason why I ask is that this effectively disables most deduplication where
> KSM was originally invented for: VMs running the same kernels, libaries,
> executables etc.

Yes, it closes the original scenarios like KVM, so I wrote a Note to say that in
commit description and Documentations. But it opens up some new application scenarios
 — scenarios with high security and low cpu-consuming requirements, where merging
zero pages alone can already bring benefits.

> 
> So yeah, disabling most deduplication shrinks side-channels ... by disabling the
> core functionality of KSM, to dedupicate pages with the same content :)
> 
> > 
> >>
> >>>
> >>> Note
> >>> =====
> >>> - Suitable for Android, embedded and low-memory devices, where zero-page
> >>>   deduplication is wanted but full KSM scanning is too costly, and reduced
> >>>   KSM side-channel exposure is desired. Many handset vendors may prefer
> >>>   this knob, which can be used as a default fallback policy.
> >>>
> >>> - Not suitable for server and virtualization workloads containing many
> >>>   non-zero duplicate pages, where vanilla KSM provides better memory
> >>>   savings.
> >>
> >> I proposed something similar when I was still at RH. My concern back then was
> >> that we don't want/need all of the stable/unstable trees.
> >>
> >> IIUC that's all still in place in your implementation, right?
> > 
> > We still keep the stable/unstable tree data structures, but zero-page merging
> > does not participate in the stable/unstable tree search path. One more reason
> > for keeping them is that merge_only_zero_page is designed as a dynamic knob
> > that can be toggled at runtime, so when it is disabled KSM can fall back to
> > the traditional full KSM behavior.
> 
> 
> What's the memory overhead from keeping these datastructures in the
> merge_only_zero_page-only path?

In the merge_only_zero_page-only path of current implementation, the main memory
overhead comes from the ksm_rmap_items for non-zero pages, RB trees of stable and
unstable, and mm_slots.

If the allocation of rmap_item can be avoided (following Tu Jinjiang’s idea), then
only the stable_tree and unstable_tree remain, which are basically empty, and these
overheads become so low that they are negligible. The mm_slots are essential so can't
be avoid, but won't take up much memory.

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[Jinjiang Tu]

在 2026/11/9 15:58, Xu Xin (vivo) 写道:
> On Sat, Nov 07, 2026 at 04:13:40PM +0800, Jinjiang Tu wrote:
>> 在 2026/11/6 11:48, xu.xin@linux.dev 写道:
>>> From: "Xu Xin (vivo)" <xu.xin@linux.dev>
>>>
>>> Add a new sysfs attribute, /sys/kernel/mm/ksm/merge_only_zero_pages,
>>> which, when enabled, makes ksmd merge only empty (zeroed) anonymous
>>> pages with the kernel zero page and skip the stable and unstable trees
>>> entirely for pages that carry data.
>>>
>>> The attribute requires use_zero_pages to be 1, and the two are kept
>>> consistent by symmetric constraints: it cannot be enabled while
>>> use_zero_pages is 0 (-EINVAL), and use_zero_pages cannot be turned
>>> off while it is active (-EBUSY). The new attribute can be changed at
>>> any time and is only effective for pages merged after the change.
>>>
>>> Benefits in three aspects
>>> =========================
>>> - Memory saving:
>>>     On memory-constrained systems such as Android and embedded devices,
>>>     zero-filled anonymous pages are the largest source of duplicate
>>>     anonymous memory. They are produced by userspace allocators: the
>>>     zeroing of freshly allocated Java objects and buffers in the ART heap
>>>     (new byte[]/int[], Bitmap buffers), calloc/memset in native code, and
>>>     preallocated media buffers. Merging them with the kernel zero page
>>>     frees a significant amount of physical memory, which matters as DRAM
>>>     costs keep rising. With this enabled, we measured 100MB–300MB saving
>>>     tested with several popular applications on aarch64 (Android).
>>>
>>> - CPU overhead reduction:
>>>     Merging only zero pages also reduces the overhead of ksmd. A page that
>>>     is not merged with the zero page carries data and can never be merged
>>>     with the zero page, so there is no point in searching the stable tree
>>>     or inserting it into the unstable tree for it. Skipping those tree
>>>     operations for data-bearing pages reduces both the CPU time and the
>>>     rhashtable memory footprint of ksmd, which makes zero page merging
>>>     practical on low-resource devices where full KSM scanning would be too
>>>     expensive.
>>>
>>> - Security:
>>>     Finally, this shrinks the KSM side-channel attack surface: only zero
>>>     pages are ever merged, so the sharing status of data-bearing pages is
>>>     no longer observable through KSM timing side channels.
>>>
>>> Note
>>> =====
>>> - Suitable for Android, embedded and low-memory devices, where zero-page
>>>     deduplication is wanted but full KSM scanning is too costly, and reduced
>>>     KSM side-channel exposure is desired. Many handset vendors may prefer
>>>     this knob, which can be used as a default fallback policy.
>>>
>>> - Not suitable for server and virtualization workloads containing many
>>>     non-zero duplicate pages, where vanilla KSM provides better memory
>>>     savings.
>> It’s a coincidence that I’m also working on similar optimizations.
>> In our scenario, we have a MySQL instance with over 700 GB of anonymous
>> memory. Most of this memory belongs to the buffer pool, which caches
>> data from storage; these pages cannot be merged. Of the remaining memory,
>> around 15 GB is consumed by hashtables, which contain many zero-filled
>> pages that are eligible for merging to reduce memory footprint.
>>
>> When we enabled KSM for the entire MySQL instance, ksmd created rmap_item
>> metadata for each candidate page.The massive metadata overhead wiped out
>> any gains from page merging, and even resulted in a negative benefit.
>>
>> For this reason, I’m trying to make ksmd merge only zero pages. The goal
>> is to have ksmd skip scanning non-zero pages entirely, thereby avoiding
>> the rmap_item metadata cost introduced by scanning non-mergeable pages.
>>
>> 1) Before get_next_rmap_item(), check whether the page is all zeros.
>> If not, skip it directly to avoid allocating an rmap_item.
>>
>> 2) When walking page tables to find candidate pages in ksm_next_page_pmd_entry(),
>> check whether the first and last contents are zero as a quick filter.
>> This is intended to speed up page-table traversal.
>>
>> Would you mind to collaborate? For example, by including my optimization
>> as the second patch in your patch series?
>>
>   
> Thanks for sharing your work — the motivation is clear and our goals are compatible
>
>> Here is the diff. Basic functionality has been tested.
>>
>> diff --git a/mm/ksm.c b/mm/ksm.c
>> index dbfb8f98b189..8a02976b0aba 100644
>> --- a/mm/ksm.c
>> +++ b/mm/ksm.c
>> @@ -296,6 +296,9 @@ static unsigned int zero_checksum __read_mostly;
>>   /* Whether to merge empty (zeroed) pages with actual zero pages */
>>   static bool ksm_use_zero_pages __read_mostly;
>> +/* Whether to merge empty (zeroed) pages only */
>> +static bool ksm_only_merge_zero_pages __read_mostly;
>> +
>>   /* Skip pages that couldn't be de-duplicated previously */
>>   /* Default to true at least temporarily, for testing */
>>   static bool ksm_smart_scan = true;
>> @@ -2524,8 +2527,42 @@ struct ksm_next_page_arg {
>>   	struct folio *folio;
>>   	struct page *page;
>>   	unsigned long addr;
>> +	unsigned int nr_to_skip;
>>   };
>> +static inline bool page_maybe_zero(struct page *page)
>> +{
>> +	unsigned long *p;
>> +	bool ret = true;
>> +	unsigned int last_pos = PAGE_SIZE / sizeof(*p) - 1;
>> +
>> +	p = kmap_local_page(page);
>> +	if (p[0] || p[last_pos])
>> +		ret = false;
>> +	kunmap_local(p);
>> +	return ret;
>> +}
>> +
>> +static struct page *ksm_skip_nonzero_pages(struct page *page,
>> +					   unsigned long *addr,
>> +					   unsigned long nr,
>> +					   unsigned int *nr_to_skip)
>> +{
>> +	unsigned long i;
>> +
>> +	for (i = 0; i < nr; i++) {
>> +		if (!*nr_to_skip)
>> +			break;
>> +		(*nr_to_skip)--;
>> +		if (page_maybe_zero(page + i))
>> +			break;
>> +	}
>> +	if (i == nr)
>> +		return NULL;
>> +	*addr += i << PAGE_SHIFT;
>> +	return page + i;
>> +}
>> +
>>   static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long end,
>>   		struct mm_walk *walk)
>>   {
>> @@ -2563,6 +2600,13 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>>   				goto not_found_unlock;
>>   			page += ((addr & (PMD_SIZE - 1)) >> PAGE_SHIFT);
>> +			if (ksm_only_merge_zero_pages) {
>> +				page = ksm_skip_nonzero_pages(page, &addr,
>> +					(end - addr) >> PAGE_SHIFT,
>> +					&private->nr_to_skip);
>> +				if (!page)
>> +					goto not_found_unlock;
>> +			}
>>   			goto found_unlock;
>>   		}
>>   		spin_unlock(ptl);
>> @@ -2585,6 +2629,14 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
>>   		if (folio_is_zone_device(folio) || !folio_test_anon(folio))
>>   			continue;
>> +
>> +		if (ksm_only_merge_zero_pages && !folio_test_ksm(folio)) {
>> +			page = ksm_skip_nonzero_pages(page, &addr, 1,
>> +						      &private->nr_to_skip);
>> +			if (!page)
>> +				continue;
>> +		}
>> +
>>   		goto found_unlock;
>>   	}
> I'd like to take you up on the collaboration, with a few adjustments to your diff:
>
> I'd keep only the precise check in should_skip_page() before get_next_rmap_item(),
> and remove page_maybe_zero(), ksm_skip_nonzero_pages(), and nr_to_skip.
>
> Two concerns drive this:
>
> (1) 'nr_to_skip' conflicts semantically with 'pages_to_scan'. nr_to_skip (initialized to 1024,
> reset per walk) is an independent cap on how much work one walk may do, decoupled from the
> user-facing pages_to_scan knob. pages_to_scan controls how many pages one scan cycle processes,
> but the pages skipped inside a walk are not accounted for in that budget, so the actual amount
> of work ksmd does is no longer fully described by pages_to_scan. A user tuning pages_to_scan to
> bound ksmd's CPU/power on a battery-powered device would no longer get a bound they can reason
> about.
>
> (2) The same non-zero page can go through page_maybe_zero() more than once. It's only a heuristic
> (first and last word), so a page that is zero at both ends but carries data in the middle passes
> the fast filter, is returned as a candidate, and is then rejected by the exact pages_identical()
> check in should_skip_page(). Because KSM re-walks the same address range on every subsequent scan
> cycle, that same page is fed through page_maybe_zero() again on each cycle and rejected again
> — the cost is paid repeatedly while the page is never actually filtered out.
>
> Given the above, I'd like to drop ksm _skip_nonzero_pages() and the coarse filter in the page
> walking path from this round of submission, to keep the code simpler and clearer overall.
> The precise check in should_skip_page() is kept, so non-zero pages still never get an rmap_item
> — the metadata saving you're after is preserved. Also, I'll keep the attribute name consistent
> with yours.
>
> On the tags: I'd like to credit you as Co-developed-by and add your Signed-off-by, per
> submitting-patches.rst, with my own Signed-off-by above them.
>
> If you have concerns, let me know before I post.

I have no problem with that. Thanks.

>
>> @@ -2609,6 +2661,33 @@ static struct mm_walk_ops ksm_next_page_ops = {
>>   	.walk_lock = PGWALK_RDLOCK,
>>   };
>> +static bool should_skip_page(struct ksm_rmap_item **rmap_list,
>> +				  unsigned long addr, struct folio *folio,
>> +				  struct page *page)
>> +{
>> +	struct ksm_rmap_item *rmap_item;
>> +
>> +	if (!ksm_only_merge_zero_pages)
>> +		return false;
>> +
>> +	if (folio_test_ksm(folio))
>> +		return false;
>> +
>> +	if (pages_identical(page, ZERO_PAGE(0)))
>> +		return false;
>> +
>> +	while (*rmap_list) {
>> +		rmap_item = *rmap_list;
>> +		if ((rmap_item->address & PAGE_MASK) > addr)
>> +			break;
>> +		*rmap_list = rmap_item->rmap_list;
>> +		remove_rmap_item_from_tree(rmap_item);
>> +		free_rmap_item(rmap_item);
>> +	}
>> +	return true;
>> +}
>> +
>> +
>>   static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>>   {
>>   	struct mm_struct *mm;
>> @@ -2705,6 +2784,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>>   			int found;
>> +			ksm_next_page_arg.nr_to_skip = 1024;
>>   			found = walk_page_range_vma(vma, ksm_scan.address,
>>   						    vma->vm_end,
>>   						    &ksm_next_page_ops,
>> @@ -2722,6 +2802,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
>>   			if (tmp_page) {
>>   				flush_anon_page(vma, tmp_page, ksm_scan.address);
>>   				flush_dcache_page(tmp_page);
>> +
>> +				if (should_skip_page(ksm_scan.rmap_list, ksm_scan.address, folio, tmp_page)) {
>> +					folio_put(folio);
>> +					ksm_scan.address += PAGE_SIZE;
>> +					mmap_read_unlock(mm);
>> +					return ERR_PTR(-EAGAIN);
>> +				}
>> +
>>   				rmap_item = get_next_rmap_item(mm_slot,
>>   					ksm_scan.rmap_list, ksm_scan.address);
>>   				if (rmap_item) {
>> @@ -2824,6 +2912,10 @@ static void ksm_do_scan(unsigned int scan_npages)
>>   		rmap_item = scan_get_next_rmap_item(&page);
>>   		if (!rmap_item)
>>   			return;
>> +		if (rmap_item == ERR_PTR(-EAGAIN)) {
>> +			ksm_pages_scanned++;
>> +			continue;
>> +		}
>>   		cmp_and_merge_page(page, rmap_item);
>>   		put_page(page);
>>   		ksm_pages_scanned++;
>> @@ -3674,6 +3766,28 @@ static ssize_t use_zero_pages_store(struct kobject *kobj,
>>   }
>>   KSM_ATTR(use_zero_pages);
>>

Re: [PATCH resend] ksm: add merge_only_zero_pages to merge only empty anonymous pages

[David Hildenbrand (Arm)]

On 11/10/26 02:34, Xu Xin (vivo) wrote:
> On Mon, Nov 09, 2026 at 06:41:39PM +0100, David Hildenbrand (Arm) wrote:
>> On 11/9/26 02:39, Xu Xin (vivo) wrote:
>>>
>>> Ha, fair question :) This came from our own investigation. We were looking
>>> into whether KSM can actually provide benefits on Android phones, and this
>>> observation came from real-world application workloads. To be fully honest:
>>> I did use an LLM to help polish the English wording of the commit description,
>>> but I carefully rechecked and verified every sentence myself..
>>
>> The reason why I ask is that this effectively disables most deduplication where
>> KSM was originally invented for: VMs running the same kernels, libaries,
>> executables etc.
> 
> Yes, it closes the original scenarios like KVM, so I wrote a Note to say that in
> commit description and Documentations. But it opens up some new application scenarios
>  — scenarios with high security and low cpu-consuming requirements, where merging
> zero pages alone can already bring benefits.
> 
>>
>> So yeah, disabling most deduplication shrinks side-channels ... by disabling the
>> core functionality of KSM, to dedupicate pages with the same content :)
>>
>>>
>>>
>>> We still keep the stable/unstable tree data structures, but zero-page merging
>>> does not participate in the stable/unstable tree search path. One more reason
>>> for keeping them is that merge_only_zero_page is designed as a dynamic knob
>>> that can be toggled at runtime, so when it is disabled KSM can fall back to
>>> the traditional full KSM behavior.
>>
>>
>> What's the memory overhead from keeping these datastructures in the
>> merge_only_zero_page-only path?
> 
> In the merge_only_zero_page-only path of current implementation, the main memory
> overhead comes from the ksm_rmap_items for non-zero pages, RB trees of stable and
> unstable, and mm_slots.
> 
> If the allocation of rmap_item can be avoided (following Tu Jinjiang’s idea), then
> only the stable_tree and unstable_tree remain, which are basically empty, and these
> overheads become so low that they are negligible. The mm_slots are essential so can't
> be avoid, but won't take up much memory.

Okay, so that's the right approach then.

-- 
Cheers,

David