[PATCH] erofs: cap Zstandard stream pool size

[PATCH] erofs: cap Zstandard stream pool size

[Zhan Xusheng]

From: Zhan Xusheng <zhanxusheng@xiaomi.com>

fs/erofs/decompressor_zstd.c sizes the module-global Zstandard stream
pool from num_possible_cpus() when the zstd_streams module parameter is
unset, and z_erofs_load_zstd_config() then preallocates one workspace per
stream, grown to the largest dictionary of any mounted image (up to
Z_EROFS_ZSTD_MAX_DICT_SIZE, i.e. Z_EROFS_PCLUSTER_MAX_SIZE).  On high-CPU
systems this can pin a large amount of vmalloc-backed decoder state until
the erofs module is unloaded, mirroring the LZMA case fixed in commit
c9b47e6b2311 ("erofs: cap LZMA stream pool size").

Bound the default stream count by a new
CONFIG_EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS option, default 16, while
preserving the per-image workspace sizing.  An explicit zstd_streams
module parameter is still honoured as-is.

Fixes: 7c35de4df105 ("erofs: Zstandard compression support")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
---
 fs/erofs/Kconfig             | 14 ++++++++++++++
 fs/erofs/decompressor_zstd.c |  3 ++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig
index 8ca1767dafb6..a27114d3c82f 100644
--- a/fs/erofs/Kconfig
+++ b/fs/erofs/Kconfig
@@ -167,6 +167,20 @@ config EROFS_FS_ZIP_ZSTD
 
 	  If unsure, say N.
 
+config EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS
+	int "EROFS Zstandard default maximum decompression streams"
+	depends on EROFS_FS_ZIP_ZSTD
+	range 1 NR_CPUS
+	default 16
+	help
+	  By default EROFS allocates one Zstandard decompression stream per
+	  CPU.  Each stream preallocates a workspace sized for the largest
+	  dictionary of any mounted image (up to 1 MiB), so on systems with
+	  many CPUs this can reserve a lot of memory.  This caps the default;
+	  the zstd_streams module parameter still overrides it.
+
+	  If unsure, keep the default of 16.
+
 config EROFS_FS_ZIP_ACCEL
 	bool "EROFS hardware decompression support"
 	depends on EROFS_FS_ZIP
diff --git a/fs/erofs/decompressor_zstd.c b/fs/erofs/decompressor_zstd.c
index ab318a2400f9..44f9d89674e3 100644
--- a/fs/erofs/decompressor_zstd.c
+++ b/fs/erofs/decompressor_zstd.c
@@ -53,7 +53,8 @@ static int __init z_erofs_zstd_init(void)
 {
 	/* by default, use # of possible CPUs instead */
 	if (!z_erofs_zstd_nstrms)
-		z_erofs_zstd_nstrms = num_possible_cpus();
+		z_erofs_zstd_nstrms = min_t(unsigned int, num_possible_cpus(),
+				CONFIG_EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS);
 
 	for (; z_erofs_zstd_avail_strms < z_erofs_zstd_nstrms;
 	     ++z_erofs_zstd_avail_strms) {
-- 
2.43.0

Re: [PATCH] erofs: cap Zstandard stream pool size

[Gao Xiang]

Hi Xusheng,

On Tue, Jul 28, 2026 at 10:18:31AM +0800, Zhan Xusheng wrote:
> From: Zhan Xusheng <zhanxusheng@xiaomi.com>
> 
> fs/erofs/decompressor_zstd.c sizes the module-global Zstandard stream
> pool from num_possible_cpus() when the zstd_streams module parameter is
> unset, and z_erofs_load_zstd_config() then preallocates one workspace per
> stream, grown to the largest dictionary of any mounted image (up to
> Z_EROFS_ZSTD_MAX_DICT_SIZE, i.e. Z_EROFS_PCLUSTER_MAX_SIZE).  On high-CPU
> systems this can pin a large amount of vmalloc-backed decoder state until
> the erofs module is unloaded, mirroring the LZMA case fixed in commit
> c9b47e6b2311 ("erofs: cap LZMA stream pool size").
> 
> Bound the default stream count by a new
> CONFIG_EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS option, default 16, while
> preserving the per-image workspace sizing.  An explicit zstd_streams
> module parameter is still honoured as-is.
> 
> Fixes: 7c35de4df105 ("erofs: Zstandard compression support")
> Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>

Thanks for the patch.

Unlike LZMA, each Zstandard only takes 1MiB at most (although erofs-utils
only generates 4MiB LZ77 dictionary at most for LZMA, but on-disk format
allows 8MiB so just in case.)

So I think for Zstandard, 1MiB each stream should fulfill to most
platforms (and for embedded systems for example, the vendors should
control the dictionary size when generating the image; but for servers,
I think 1MiB at most for each server CPU is OK).

Unless there is a particular need, I don't expect every decompressor need
a strict customized Kconfig for this (of course, DEFLATE and LZ4 takes
32k/64k so it doesn't matter.)

Thanks,
Gao Xiang