Maybe we can set default zstd compression level to 1 when SSD detected?

Maybe we can set default zstd compression level to 1 when SSD detected?

[Integral]

Hi,


When SSD is detected, maybe we can set default zstd compression level to 1.


Current default compression level for zstd is 3, which is not optimal 
for SSDs.

This GitHub Gist [1] can serve as a reference.


An example is Fedora Workstation [2], which uses `zstd:1` as default

compression option.


[1] Link: 
https://gist.github.com/braindevices/fde49c6a8f6b9aaf563fb977562aafec

[2] Link: https://fedoraproject.org/wiki/Changes/BtrfsTransparentCompression


Sincerely,

Integral

Re: Maybe we can set default zstd compression level to 1 when SSD detected?

[David Sterba]

On Sun, Apr 13, 2025 at 12:07:26PM +0800, Integral wrote:
> Hi,
> 
> When SSD is detected, maybe we can set default zstd compression level to 1.
> 
> Current default compression level for zstd is 3, which is not optimal 
> for SSDs.
> 
> This GitHub Gist [1] can serve as a reference.

Well, while the linked gist is thorough I don't see that zstd:1 clearly
wins against zstd:3. The compression brings overhead (more extents, CPU
cost) so the preferred criteria should be space savings. The runtimes of
read and write seem to be roughly the same.

I haven't found any description or classification of the input data
(other than known to be incompressible). This is an important factor.

> An example is Fedora Workstation [2], which uses `zstd:1` as default
> compression option.
> 
> [1] Link: 
> https://gist.github.com/braindevices/fde49c6a8f6b9aaf563fb977562aafec
> 
> [2] Link: https://fedoraproject.org/wiki/Changes/BtrfsTransparentCompression

Unfortunately the Fedora evaluation disqualifies itself because it uses
/dev/urandom (practically incompressible) and /dev/zero (trivially
compressible). I would not select the default based on that benchmark
for the wole distro, it's IMHO flawed or incomplete at best.

For the evaluation I recommend some commonly found data types based on
their compressibility, like we did for the recent fast zstd levels
https://lore.kernel.org/linux-btrfs/20250128132235.1356769-1-neelx@suse.com/ .
Binaries, documentation, enwik9 (commonly used for compression
benchmarks), linux sources.

The classes are not exact but should represent files that are common
and have a chance of being considered for compression.  Already
compressed files or other hard to compress files like media are
detected and not considered by the heuristic.

Changing defaults is possible but it affects everybody and from past
experience breaks somebody's use case or negatively affects performance.

The evaluation from the gist could be enhanced with more input data
types and CPU strengths.

Re: Maybe we can set default zstd compression level to 1 when SSD detected?

[Neal Gompa]

On Wed, Apr 16, 2025 at 6:06 AM David Sterba <dsterba@suse.cz> wrote:
>
> On Sun, Apr 13, 2025 at 12:07:26PM +0800, Integral wrote:
> > Hi,
> >
> > When SSD is detected, maybe we can set default zstd compression level to 1.
> >
> > Current default compression level for zstd is 3, which is not optimal
> > for SSDs.
> >
> > This GitHub Gist [1] can serve as a reference.
>
> Well, while the linked gist is thorough I don't see that zstd:1 clearly
> wins against zstd:3. The compression brings overhead (more extents, CPU
> cost) so the preferred criteria should be space savings. The runtimes of
> read and write seem to be roughly the same.
>
> I haven't found any description or classification of the input data
> (other than known to be incompressible). This is an important factor.
>
> > An example is Fedora Workstation [2], which uses `zstd:1` as default
> > compression option.
> >
> > [1] Link:
> > https://gist.github.com/braindevices/fde49c6a8f6b9aaf563fb977562aafec
> >
> > [2] Link: https://fedoraproject.org/wiki/Changes/BtrfsTransparentCompression
>
> Unfortunately the Fedora evaluation disqualifies itself because it uses
> /dev/urandom (practically incompressible) and /dev/zero (trivially
> compressible). I would not select the default based on that benchmark
> for the wole distro, it's IMHO flawed or incomplete at best.
>

You didn't read it properly. There were two factors being tested:
compression ratio and CPU load.

The compression ratio was tested with a typical Fedora install and
produced different rates of compression based on the different
compression levels selected. The /dev/urandom and /dev/zero tests were
for testing the CPU load at different extremes across those levels.

There was not enough gain to warrant zstd:2 or zstd:3 for the
increased CPU load on average or at the extremes.


-- 
真実はいつも一つ！/ Always, there's only one truth!