All of lore.kernel.org
 help / color / mirror / Atom feed
From: Gao Xiang <xiang@kernel.org>
To: Zhan Xusheng <zhanxusheng1024@gmail.com>
Cc: Gao Xiang <xiang@kernel.org>, Chao Yu <chao@kernel.org>,
	Yue Hu <zbestahu@gmail.com>,
	Jeffle Xu <jefflexu@linux.alibaba.com>,
	Sandeep Dhavale <dhavale@google.com>,
	Hongbo Li <hongbohbli@tencent.com>,
	Chunhai Guo <guochunhai@vivo.com>, Nick Terrell <terrelln@fb.com>,
	David Sterba <dsterba@suse.com>,
	linux-erofs@lists.ozlabs.org, linux-kernel@vger.kernel.org,
	Zhan Xusheng <zhanxusheng@xiaomi.com>
Subject: Re: [PATCH] erofs: cap Zstandard stream pool size
Date: Tue, 28 Jul 2026 14:40:26 +0800	[thread overview]
Message-ID: <amhO2uzAdAZYb0VW@XiangdeMacBook-Pro.local> (raw)
In-Reply-To: <20260728021831.1532541-1-zhanxusheng1024@gmail.com>

Hi Xusheng,

On Tue, Jul 28, 2026 at 10:18:31AM +0800, Zhan Xusheng wrote:
> From: Zhan Xusheng <zhanxusheng@xiaomi.com>
> 
> fs/erofs/decompressor_zstd.c sizes the module-global Zstandard stream
> pool from num_possible_cpus() when the zstd_streams module parameter is
> unset, and z_erofs_load_zstd_config() then preallocates one workspace per
> stream, grown to the largest dictionary of any mounted image (up to
> Z_EROFS_ZSTD_MAX_DICT_SIZE, i.e. Z_EROFS_PCLUSTER_MAX_SIZE).  On high-CPU
> systems this can pin a large amount of vmalloc-backed decoder state until
> the erofs module is unloaded, mirroring the LZMA case fixed in commit
> c9b47e6b2311 ("erofs: cap LZMA stream pool size").
> 
> Bound the default stream count by a new
> CONFIG_EROFS_FS_ZIP_ZSTD_DEFAULT_MAX_STREAMS option, default 16, while
> preserving the per-image workspace sizing.  An explicit zstd_streams
> module parameter is still honoured as-is.
> 
> Fixes: 7c35de4df105 ("erofs: Zstandard compression support")
> Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>

Thanks for the patch.

Unlike LZMA, each Zstandard only takes 1MiB at most (although erofs-utils
only generates 4MiB LZ77 dictionary at most for LZMA, but on-disk format
allows 8MiB so just in case.)

So I think for Zstandard, 1MiB each stream should fulfill to most
platforms (and for embedded systems for example, the vendors should
control the dictionary size when generating the image; but for servers,
I think 1MiB at most for each server CPU is OK).

Unless there is a particular need, I don't expect every decompressor need
a strict customized Kconfig for this (of course, DEFLATE and LZ4 takes
32k/64k so it doesn't matter.)

Thanks,
Gao Xiang



      reply	other threads:[~2026-07-28  6:40 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-28  2:18 [PATCH] erofs: cap Zstandard stream pool size Zhan Xusheng
2026-07-28  6:40 ` Gao Xiang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=amhO2uzAdAZYb0VW@XiangdeMacBook-Pro.local \
    --to=xiang@kernel.org \
    --cc=chao@kernel.org \
    --cc=dhavale@google.com \
    --cc=dsterba@suse.com \
    --cc=guochunhai@vivo.com \
    --cc=hongbohbli@tencent.com \
    --cc=jefflexu@linux.alibaba.com \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=terrelln@fb.com \
    --cc=zbestahu@gmail.com \
    --cc=zhanxusheng1024@gmail.com \
    --cc=zhanxusheng@xiaomi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.