Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH v4 0/4] mTHP-friendly compression in zsmalloc and zram based on multi-pages
@ 2026-11-15  2:44 Nanzhe Zhao
  2026-11-15  2:44 ` [RFC PATCH v4 1/4] mm: zsmalloc: support objects compressed based on multiple pages Nanzhe Zhao
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: Nanzhe Zhao @ 2026-11-15  2:44 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Nanzhe Zhao, linux-mm, linux-block, linux-kernel, Jens Axboe,
	Sergey Senozhatsky, Minchan Kim, Kairui Song, Johannes Weiner,
	Nhat Pham, Usama Arif, Yosry Ahmed, Chengming Zhou, Chris Li,
	David Hildenbrand, Ying Huang, Suren Baghdasaryan, Ryan Roberts,
	Kanchana P Sridhar, Wajdi K Feghali, Bala Seshasayee,
	Nick Terrell, Barry Song, Tangquan Zheng, Bo Zhang

When large folios are compressed at a larger granularity, we observe
a notable reduction in CPU usage and a significant improvement in
compression ratios.

mTHP's ability to be swapped out without splitting and swapped back in
as a whole allows compression and decompression at larger granularities.

This patchset enhances zsmalloc and zram by adding support for dividing
large folios into multi-page blocks, typically configured with a
2-order granularity. Without this patchset, a large folio is always
divided into nr_pages 4KiB blocks.

The granularity can be set using the ZSMALLOC_MULTI_PAGES_ORDER
setting, where the default of 2 allows all anonymous THP to benefit.

Examples include:
* A 16KiB large folio will be compressed and stored as a single 16KiB
  block.
* A 64KiB large folio will be compressed and stored as four 16KiB
  blocks.

We captured anonymous pages from four Android applications on a Pixel 6.
To evaluate throughput, we allocate a buffer via mmap(MAP_PRIVATE |
MAP_ANONYMOUS), populate it with the captured pages, and invoke
madvise(MADV_PAGEOUT) to swap the buffer out to zram, measuring the
pageout duration. We then sequentially touch every 4 KiB page to fault
them back in from zram, measuring the swapin duration.

For example, testing the 143 MiB WeChat corpus (times in seconds;
w/o: without the patch, w/: with the patch):

Algorithm  pageout w/o (s)  pageout w/            swapin w/o (s)  swapin w/
lz4        0.247            0.210 (-14.8%)        0.137           0.113 (-17.6%)
lzo-rle    0.240            0.207 (-13.7%)        0.149           0.123 (-17.5%)
zstd       0.567            0.452 (-20.4%)        0.258           0.192 (-25.6%)

v3 left two issues to discuss:

The first issue is internal fragmentation. A compressed 16 KiB block
is rounded up to the next size-class boundary. If it lands in a class
with a large gap, the gap becomes per-object padding.

On all four applications, unused pages in partially filled zspages
increase by ~1.8 to 2.4 MiB, and the per-object class gap is wider.
However, because the number of allocated objects drops by ~74% (from
four 4 KiB objects to one 16 KiB object), total rounding waste
actually falls by 13% to 31%. Combined with 5% to 15% better
compression ratios from multi-page compression, total zsmalloc memory
occupancy (mem_used_total) is reduced across all applications.

For example, below is the memory breakdown under lz4 for our four apps
(lzo-rle and zstd show the same trend, saving 3.8% to 9.3% total memory):

App          objects (w/o -> w/)        rounding waste (MiB)    unused pages (MiB)      mem_used_total (MiB)
WPS          27,530 ->  7,285 (-73.5%)  0.80 -> 0.56 (-29.8%)   1.07 -> 3.35 (+214.0%)   36.4 ->  34.9 (-4.0%)
WeChat       32,460 ->  8,450 (-74.0%)  0.89 -> 0.62 (-30.6%)   1.35 -> 3.32 (+145.8%)   42.8 ->  40.0 (-6.7%)
Douyin lite  62,668 -> 16,218 (-74.1%)  2.07 -> 1.61 (-22.3%)   1.37 -> 3.78 (+176.0%)  100.8 ->  93.6 (-7.1%)
JK chess     71,950 -> 18,584 (-74.2%)  1.75 -> 1.52 (-13.0%)   1.43 -> 3.28 (+129.7%)  117.0 -> 112.7 (-3.6%)

Here, objects is obj_allocated, rounding waste is total class padding
across all objects, unused pages is memory left in partially filled
zspages, and mem_used_total is the total memory allocated by zsmalloc
(including fragmentation and metadata).

We also built a 256 MiB lz4 synthetic extreme case: 16 KiB blocks with
the first K bytes random and the rest zero, with K tuned so the object
lands one byte above the previous class. Below is the code to generate
this pattern:

#define BLOCK 16384

static void make_block(unsigned char *buf, size_t k)
{
	size_t i;

	for (i = 0; i < k; i++)
		buf[i] = rand();
	memset(buf + k, 0, BLOCK - k);
}

static void write_corpus(int fd, size_t k, size_t nblocks)
{
	unsigned char *buf = malloc(BLOCK);
	size_t i;

	make_block(buf, k);
	for (i = 0; i < nblocks; i++)
		write(fd, buf, BLOCK);
}

With the default CONFIG_ZSMALLOC_CHAIN_SIZE=8, the largest non-huge
class gap is 576 bytes. Setting K to 10,781 bytes lands the object in
class 11,456 with a 575 B/object padding, increasing total memory by
5.0% over the baseline without the patch (179.3 MiB vs 170.7 MiB).

With CONFIG_ZSMALLOC_CHAIN_SIZE=10, the classes are denser and the
largest gap drops to 448 bytes. At K=10,781 bytes, the memory overhead
falls to 3.9% (177.3 MiB vs 170.7 MiB).

A potential future optimization is during size-class merging: we can
avoid merging classes if the resulting gap exceeds a threshold (e.g.
ZS_MAX_MERGE_GAP = 256 bytes), for example:

--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ can_merge() @@
+#define ZS_MAX_MERGE_GAP	256
+
+	if (prev->size - size > ZS_MAX_MERGE_GAP)
+		return false;
+
 	if (prev->zpdescs_per_zspage == zpdescs_per_zspage &&
 		prev->objs_per_zspage == objs_per_zspage)
 		return true;

The second issue is partial read. If the order-2 allocation for the
swapped-in folio fails and falls back to order-0, we must decompress
the entire 16 KiB block just for a 4 KiB swapin.

v3's 4/4 ("mm: fall back to four small folios if mTHP allocation
fails") handled this by building four small folios. Usama asked whether
defrag=always could replace this patch. But we only need direct reclaim
for the do_swap_page order-2 allocation; otherwise all anonymous mTHP
allocations would enter direct reclaim. So the new 4/4 adds
__GFP_RECLAIM to the order-2 swapin allocation in do_swap_page().

We measured the cost of entering direct reclaim in the swapin path on
an i7-14700HX with no_turbo=1: a 1 GiB memcg, zram (zstd), 16 KiB mTHP,
and an arm64 defconfig vmlinux build with -j20 in the background. The
bench was pinned to one CPU and the build to the rest. We compared
forcing direct reclaim with the existing behavior, 10 totals per case,
each total 30 rounds of 100 MiB anonymous pageout/swapin.

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <time.h>

static double now(void)
{
	struct timespec ts;

	clock_gettime(CLOCK_MONOTONIC, &ts);
	return ts.tv_sec + ts.tv_nsec / 1e9;
}

int main(void)
{
	const size_t size = 100UL << 20;
	unsigned long *p, sum = 0;
	size_t i, off;
	double t0, t1;

	p = mmap(NULL, size, PROT_READ | PROT_WRITE,
		 MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
	if (p == MAP_FAILED) {
		perror("mmap");
		return 1;
	}

	for (i = 0; i < size / sizeof(*p); i++)
		p[i] = i & 0xfff;

	t0 = now();
	if (madvise(p, size, MADV_PAGEOUT)) {
		perror("madvise(MADV_PAGEOUT)");
		return 1;
	}
	t1 = now();
	printf("pageout %.6f\n", t1 - t0);

	t0 = now();
	for (off = 0; off < size; off += 4096)
		sum += *(volatile unsigned long *)((char *)p + off);
	t1 = now();
	printf("swapin %.6f sum %lu\n", t1 - t0, sum);

	return 0;
}

                         w/o DR          DR
partial reads            235,832         218
swapin total (s)         24.85 +/- 0.56    22.80 +/- 0.73
pswpout (pages)          11.4M           11.3M

Here, partial reads is the zram_bio_read_multi_pages_partial counter.
The direct-reclaim case did not show a measurable cost. Almost all
partial decompressions were eliminated, making swapin 8.3% faster.

-v4:
 * Rebased onto latest akpm mm-new.
 * Patch 1: Fixed multi-page page accounting (NR_ZSPAGES, zs_compact,
   and debugfs) to use physical pages via class_pages_per_zspage().
 * Patch 2: Adapted to upstream's new zcomp read/write stream separation;
   added a preallocated decomp_buf in zcomp_dstrm for partial reads to
   avoid memory allocation in the swapin path.
 * Patch 3: Updated zstd backend to pass ZCOMP_MULTI_PAGES_SIZE to
   zstd_setup_params().
 * Patch 4: Dropped v3's 4/4 fallback and added a new patch to allow
   direct reclaim for synchronous multi-page swapin.

[1] v3: https://lore.kernel.org/all/20241121222521.83458-1-21cnbao@gmail.com/

Barry Song (1):
  zram: backend_zstd: use multi-page size for zstd parameter setup

Nanzhe Zhao (1):
  mm: swap: allow direct reclaim for synchronous multi-page swapin

Tangquan Zheng (2):
  mm: zsmalloc: support objects compressed based on multiple pages
  zram: support compression at the granularity of multi-pages

 drivers/block/zram/Kconfig        |   9 +
 drivers/block/zram/backend_zstd.c |   4 +-
 drivers/block/zram/zcomp.c        |  31 +-
 drivers/block/zram/zcomp.h        |  19 +-
 drivers/block/zram/zram_drv.c     | 499 +++++++++++++++++++++++++++---
 drivers/block/zram/zram_drv.h     |  43 +++
 include/linux/zsmalloc.h          |  10 +-
 mm/Kconfig                        |  29 +-
 mm/swap_state.c                   |   7 +
 mm/zsmalloc.c                     | 242 ++++++++++-----
 10 files changed, 757 insertions(+), 136 deletions(-)

-- 
2.43.0



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-16  2:47 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-15  2:44 [RFC PATCH v4 0/4] mTHP-friendly compression in zsmalloc and zram based on multi-pages Nanzhe Zhao
2026-11-15  2:44 ` [RFC PATCH v4 1/4] mm: zsmalloc: support objects compressed based on multiple pages Nanzhe Zhao
2026-11-16  2:46   ` sashiko-bot
2026-11-15  2:44 ` [RFC PATCH v4 2/4] zram: support compression at the granularity of multi-pages Nanzhe Zhao
2026-11-16  2:46   ` sashiko-bot
2026-11-15  2:44 ` [RFC PATCH v4 3/4] zram: backend_zstd: use multi-page size for zstd parameter setup Nanzhe Zhao
2026-11-15  2:44 ` [RFC PATCH v4 4/4] mm: swap: allow direct reclaim for synchronous multi-page swapin Nanzhe Zhao
2026-11-16  2:46   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox