From: Gao Xiang <xiang@kernel.org>
To: Zhan Xusheng <zhanxusheng1024@gmail.com>
Cc: Gao Xiang <xiang@kernel.org>, Chao Yu <chao@kernel.org>,
Yue Hu <zbestahu@gmail.com>,
Jeffle Xu <jefflexu@linux.alibaba.com>,
Sandeep Dhavale <dhavale@google.com>,
Hongbo Li <hongbohbli@tencent.com>,
Chunhai Guo <guochunhai@vivo.com>, Nick Terrell <terrelln@fb.com>,
David Sterba <dsterba@suse.com>,
linux-erofs@lists.ozlabs.org, linux-kernel@vger.kernel.org,
Zhan Xusheng <zhanxusheng@xiaomi.com>
Subject: Re: [PATCH] erofs: cap Zstandard stream pool size
Date: Tue, 28 Jul 2026 14:40:26 +0800 [thread overview]
Message-ID: <amhO2uzAdAZYb0VW@XiangdeMacBook-Pro.local> (raw)
In-Reply-To: <20260728021831.1532541-1-zhanxusheng1024@gmail.com>
Hi Xusheng,
On Tue, Jul 28, 2026 at 10:18:31AM +0800, Zhan Xusheng wrote:
> From: Zhan Xusheng <zhanxusheng@xiaomi.com>
>
> fs/erofs/decompressor_zstd.c sizes the module-global Zstandard stream
> pool from num_possible_cpus() when the zstd_streams module parameter is
> unset, and z_erofs_load_zstd_config() then preallocates one workspace per
> stream, grown to the largest dictionary of any mounted image (up to
> Z_EROFS_ZSTD_MAX_DICT_SIZE, i.e. Z_EROFS_PCLUSTER_MAX_SIZE). On high-CPU
> systems this can pin a large amount of vmalloc-backed decoder state until
> the erofs module is unloaded, mirroring the LZMA case fixed in commit
> c9b47e6b2311 ("erofs: cap LZMA stream pool size").
>
> Bound the default stream count by a new
> CONFIG_EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS option, default 16, while
> preserving the per-image workspace sizing. An explicit zstd_streams
> module parameter is still honoured as-is.
>
> Fixes: 7c35de4df105 ("erofs: Zstandard compression support")
> Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Thanks for the patch.
Unlike LZMA, each Zstandard only takes 1MiB at most (although erofs-utils
only generates 4MiB LZ77 dictionary at most for LZMA, but on-disk format
allows 8MiB so just in case.)
So I think for Zstandard, 1MiB each stream should fulfill to most
platforms (and for embedded systems for example, the vendors should
control the dictionary size when generating the image; but for servers,
I think 1MiB at most for each server CPU is OK).
Unless there is a particular need, I don't expect every decompressor need
a strict customized Kconfig for this (of course, DEFLATE and LZ4 takes
32k/64k so it doesn't matter.)
Thanks,
Gao Xiang
prev parent reply other threads:[~2026-07-28 6:40 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-28 2:18 [PATCH] erofs: cap Zstandard stream pool size Zhan Xusheng
2026-07-28 6:40 ` Gao Xiang [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=amhO2uzAdAZYb0VW@XiangdeMacBook-Pro.local \
--to=xiang@kernel.org \
--cc=chao@kernel.org \
--cc=dhavale@google.com \
--cc=dsterba@suse.com \
--cc=guochunhai@vivo.com \
--cc=hongbohbli@tencent.com \
--cc=jefflexu@linux.alibaba.com \
--cc=linux-erofs@lists.ozlabs.org \
--cc=linux-kernel@vger.kernel.org \
--cc=terrelln@fb.com \
--cc=zbestahu@gmail.com \
--cc=zhanxusheng1024@gmail.com \
--cc=zhanxusheng@xiaomi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox