The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Gao Xiang <xiang@kernel.org>
To: Zhan Xusheng <zhanxusheng1024@gmail.com>
Cc: Gao Xiang <xiang@kernel.org>, Chao Yu <chao@kernel.org>,
	Yue Hu <zbestahu@gmail.com>,
	Jeffle Xu <jefflexu@linux.alibaba.com>,
	Sandeep Dhavale <dhavale@google.com>,
	Hongbo Li <hongbohbli@tencent.com>,
	Chunhai Guo <guochunhai@vivo.com>, Nick Terrell <terrelln@fb.com>,
	David Sterba <dsterba@suse.com>,
	linux-erofs@lists.ozlabs.org, linux-kernel@vger.kernel.org,
	Zhan Xusheng <zhanxusheng@xiaomi.com>
Subject: Re: [PATCH] erofs: cap Zstandard stream pool size
Date: Tue, 28 Jul 2026 14:40:26 +0800	[thread overview]
Message-ID: <amhO2uzAdAZYb0VW@XiangdeMacBook-Pro.local> (raw)
In-Reply-To: <20260728021831.1532541-1-zhanxusheng1024@gmail.com>

Hi Xusheng,

On Tue, Jul 28, 2026 at 10:18:31AM +0800, Zhan Xusheng wrote:
> From: Zhan Xusheng <zhanxusheng@xiaomi.com>
> 
> fs/erofs/decompressor_zstd.c sizes the module-global Zstandard stream
> pool from num_possible_cpus() when the zstd_streams module parameter is
> unset, and z_erofs_load_zstd_config() then preallocates one workspace per
> stream, grown to the largest dictionary of any mounted image (up to
> Z_EROFS_ZSTD_MAX_DICT_SIZE, i.e. Z_EROFS_PCLUSTER_MAX_SIZE).  On high-CPU
> systems this can pin a large amount of vmalloc-backed decoder state until
> the erofs module is unloaded, mirroring the LZMA case fixed in commit
> c9b47e6b2311 ("erofs: cap LZMA stream pool size").
> 
> Bound the default stream count by a new
> CONFIG_EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS option, default 16, while
> preserving the per-image workspace sizing.  An explicit zstd_streams
> module parameter is still honoured as-is.
> 
> Fixes: 7c35de4df105 ("erofs: Zstandard compression support")
> Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>

Thanks for the patch.

Unlike LZMA, each Zstandard only takes 1MiB at most (although erofs-utils
only generates 4MiB LZ77 dictionary at most for LZMA, but on-disk format
allows 8MiB so just in case.)

So I think for Zstandard, 1MiB each stream should fulfill to most
platforms (and for embedded systems for example, the vendors should
control the dictionary size when generating the image; but for servers,
I think 1MiB at most for each server CPU is OK).

Unless there is a particular need, I don't expect every decompressor need
a strict customized Kconfig for this (of course, DEFLATE and LZ4 takes
32k/64k so it doesn't matter.)

Thanks,
Gao Xiang


      reply	other threads:[~2026-07-28  6:40 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-28  2:18 [PATCH] erofs: cap Zstandard stream pool size Zhan Xusheng
2026-07-28  6:40 ` Gao Xiang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=amhO2uzAdAZYb0VW@XiangdeMacBook-Pro.local \
    --to=xiang@kernel.org \
    --cc=chao@kernel.org \
    --cc=dhavale@google.com \
    --cc=dsterba@suse.com \
    --cc=guochunhai@vivo.com \
    --cc=hongbohbli@tencent.com \
    --cc=jefflexu@linux.alibaba.com \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=terrelln@fb.com \
    --cc=zbestahu@gmail.com \
    --cc=zhanxusheng1024@gmail.com \
    --cc=zhanxusheng@xiaomi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox