* Re: apply different compress/compress-force settings on different subvolumes ?
2024-02-21 13:15 ` Roland
@ 2024-02-21 17:45 ` Austin S. Hemmelgarn
2024-02-22 3:25 ` Qu Wenruo
2024-02-22 8:33 ` Roland
2 siblings, 0 replies; 7+ messages in thread
From: Austin S. Hemmelgarn @ 2024-02-21 17:45 UTC (permalink / raw)
To: Roland, Qu Wenruo, linux-btrfs
On 2/21/24 08:15, Roland wrote:
> Am 21.02.24 um 10:33 schrieb Qu Wenruo:
>
>> 在 2024/2/21 19:19, Roland 写道:
>>> and why is compress-force silently ignored? (see below)
>> Compress-force has no coresponding per-inode prop option though.
>>
>> But I don't think compress-force would cause much difference against
>> regular compress.
>
> apparently, force-compress can make a big difference (and compress only
> can lead to no compression at all):
>
> https://bugzilla.proxmox.com/show_bug.cgi?id=5250#c6
I can attest that this is indeed the case, though it’s very dependent on
the exact usage. For the case of large files with highly varied data
(think filesystem images, VM disk images, large databases, etc) that
don’t include inherent compression, it’s very normal for compression to
actually be a huge help, but the early parts of the file to have very
high entropy and thus not compress well, resulting in regular `compress`
not actually helping at all (because BTRFS quickly decides it’s not
worth it for that file and chooses to not compress it at all).
As an example, the VM disk images I’m storing on my home server take up
roughly 40% less space with `compress-force=zstd:1` compared to just
`compress=zstd:1`.
In comparison, I see no effective difference when doing it on the root
filesystem of said home server, so like I said above, it’s rather
dependent on exact usage.
There is of course some extra overhead to attempting compression n every
write, but at least for me, on the lower compression levels when dealing
with ‘slow’ storage, it’s not enough to cause problems.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: apply different compress/compress-force settings on different subvolumes ?
2024-02-21 13:15 ` Roland
2024-02-21 17:45 ` Austin S. Hemmelgarn
@ 2024-02-22 3:25 ` Qu Wenruo
2024-02-23 1:06 ` Chris Murphy
2024-02-22 8:33 ` Roland
2 siblings, 1 reply; 7+ messages in thread
From: Qu Wenruo @ 2024-02-22 3:25 UTC (permalink / raw)
To: Roland, linux-btrfs
在 2024/2/21 23:45, Roland 写道:
> Am 21.02.24 um 10:33 schrieb Qu Wenruo:
>
>> 在 2024/2/21 19:19, Roland 写道:
>>> hello,
>>>
>>> what can be the reason , that multiple compress mount option do not work
>>> on subvolume level , i.e. it's always the first that wins ?
>> In that case, you may want to use "btrfs prop" subcommand to setup
>> compression for each subvolume.
>>
>> The mount option one is really affecting the whole filesystem.
>>>
>>> and why is compress-force silently ignored? (see below)
>> Compress-force has no coresponding per-inode prop option though.
>>
>> But I don't think compress-force would cause much difference against
>> regular compress.
>
> apparently, force-compress can make a big difference (and compress only
> can lead to no compression at all):
>
> https://bugzilla.proxmox.com/show_bug.cgi?id=5250#c6
OK, didn't expect the compression ratio detection to cause so much
difference.
Which also means, we can further improve the compression detection.
In that case, I would purpose to change "compress-force" to
"skip_compress_heuristic" or something similar, and get rid of the
compression string/level for the new option.
The "compress-force=%s" is really confusing against the existing
"comrepss=%s" option.
To me, this sounds like a change which would improve the user interface.
And with the new option named determined, I'm pretty happy to add a new
prop to skip the heuristic.
Thanks,
Qu
>
>
>>> regarding compress mount option, it seems that this can be overriden via
>>> subvolume property, which can work around the problem and have multiple
>>> compression settings.
>>>
>>> but how to fix compress-force in the same way, i.e. how can we have
>>> differenty compress-force settings with one btrfs fs ?
>>>
>>> wouldn't it make sense to introduce compress-force property for this ?
>>>
>>> what about replacing compress with compress-force (i.e. make it the
>>> default), as there is not much overhead with modern/fast cpu ?
>>>
>>> i think compress-force / compress is VERY confusing from and end user
>>> perspective - and even worse, i have set "compress" on a subvolume used
>>> for virtual machines and they do not get compressed at all (not a single
>>> bit) . so i think, compress option is pretty short-sighted and not, what
>>> an average user would expect (
>>> https://marc.info/?l=linux-btrfs&m=154523409314147&w=2 )
>>
>> IIRC compress prop needs to be set to all inodes.
>> If you set it on a directory, only new inodes would inherit the prop,
>> the existing ones won't get the new prop.
>
>
> that's clear, and no different from zfs.
>
> the big advantage of btrfs is, that we have btrfs filesystem defragment
> to recompress everything
> in place, whereas in zfs we need to move it forth and back to a
> different dataset.
>
> regards
> Roland
>
>>
>> Thanks,
>> Qu
>>>
>>> i'd be happy on some feedback. not subscribed to this list, so please
>>> CC.
>>>
>>> roland
>>>
>>>
>>> zstd compression applied to all subvolumes, though specified otherwise:
>>>
>>> cat /etc/fstab|grep btrfs
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> mount|grep btrfs
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>> different order in fstab has different result - first wins:
>>>
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>> compress-force silently ignored:
>>>
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>>
>>> this is the write speed i get with compress-force=zstd on i7-7700
>>>
>>> # dd if=/dev/zero of=test.dat bs=1024k count=10240 ;time sync
>>> 10240+0 records in
>>> 10240+0 records out
>>> 10737418240 bytes (11 GB, 10 GiB) copied, 3.76111 s, 2.9 GB/s
>>>
>>> real 0m0.224s
>>> user 0m0.000s
>>> sys 0m0.081s
>>>
>>>
>>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: apply different compress/compress-force settings on different subvolumes ?
2024-02-22 3:25 ` Qu Wenruo
@ 2024-02-23 1:06 ` Chris Murphy
0 siblings, 0 replies; 7+ messages in thread
From: Chris Murphy @ 2024-02-23 1:06 UTC (permalink / raw)
To: Qu Wenruo, Roland, Btrfs BTRFS
On Wed, Feb 21, 2024, at 8:25 PM, Qu Wenruo wrote:
> OK, didn't expect the compression ratio detection to cause so much
> difference.
>
> Which also means, we can further improve the compression detection.
>
> In that case, I would purpose to change "compress-force" to
> "skip_compress_heuristic" or something similar, and get rid of the
> compression string/level for the new option.
But which heuristic is skipped? I assume it can only be the btrfs built-in one? I'm not sure about other compression algorithms but zstandard has its own bail-out heuristic.
Another potentially large difference between compress and compress-force is extent size is limited to 512 KIB even when not compressed using compress-force. Whereas the upper limit on extent size for compress option, is (I think) bg size. An upper limit of 512 KiB on extent size can significantly increase the metadata requirement, leading to tall trees. That then makes me wonder if such setups should have a larger node/leaf size than 16KiB default.
I'm also not sure which heuristic is cheaper, or more accurate, the built-in Btrfs one or zstandard.
--
Chris Murphy
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: apply different compress/compress-force settings on different subvolumes ?
2024-02-21 13:15 ` Roland
2024-02-21 17:45 ` Austin S. Hemmelgarn
2024-02-22 3:25 ` Qu Wenruo
@ 2024-02-22 8:33 ` Roland
2 siblings, 0 replies; 7+ messages in thread
From: Roland @ 2024-02-22 8:33 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
re-send becaus my answer accidentally contained html and that was
rejected by mailinglist
>The "compress-force=%s" is really confusing against the existing
>"comrepss=%s" option.
yes, it is
>To me, this sounds like a change which would improve the user interface.
>And with the new option named determined, I'm pretty happy to add a new
>prop to skip the heuristic.
fantastic. thank you for positive response
btw,
https://btrfs.readthedocs.io/en/latest/Compression.html
"unless the filesystem is mounted with /compress-force/. In that case
compression will always be attempted on the file only to be later
discarded. This is not optimal and subject to optimizations and further
development."
afaik, zfs is exactly doing that, i.e. it's always using
"compress-force", afaik there is no "more intelligent" heuristics to
determine if data is compressible.
if you choose to have compression, you are tradingcpu cycles for
increased diskspace but also decreased i/o latency and bandwidth. that's
the deal. users excpect that, but they don't expect that they only have
"compression lite" when they use "compress=zstd".
i read some days ago, that zfs plans more cpu friendly check if data is
compressible at all, by first using faster compression algorithm and
then later on recompress with better one (
https://www.truenas.com/community/threads/game-changer-for-zfs-2-2-intelligent-compression-that-saves-cpu-and-space.113247/
) . that sounds interesting.
regards
roland
Am 21.02.24 um 14:15 schrieb Roland:
> Am 21.02.24 um 10:33 schrieb Qu Wenruo:
>
>> 在 2024/2/21 19:19, Roland 写道:
>>> hello,
>>>
>>> what can be the reason , that multiple compress mount option do not
>>> work
>>> on subvolume level , i.e. it's always the first that wins ?
>> In that case, you may want to use "btrfs prop" subcommand to setup
>> compression for each subvolume.
>>
>> The mount option one is really affecting the whole filesystem.
>>>
>>> and why is compress-force silently ignored? (see below)
>> Compress-force has no coresponding per-inode prop option though.
>>
>> But I don't think compress-force would cause much difference against
>> regular compress.
>
> apparently, force-compress can make a big difference (and compress
> only can lead to no compression at all):
>
> https://bugzilla.proxmox.com/show_bug.cgi?id=5250#c6
>
>
>>> regarding compress mount option, it seems that this can be overriden
>>> via
>>> subvolume property, which can work around the problem and have multiple
>>> compression settings.
>>>
>>> but how to fix compress-force in the same way, i.e. how can we have
>>> differenty compress-force settings with one btrfs fs ?
>>>
>>> wouldn't it make sense to introduce compress-force property for this ?
>>>
>>> what about replacing compress with compress-force (i.e. make it the
>>> default), as there is not much overhead with modern/fast cpu ?
>>>
>>> i think compress-force / compress is VERY confusing from and end user
>>> perspective - and even worse, i have set "compress" on a subvolume used
>>> for virtual machines and they do not get compressed at all (not a
>>> single
>>> bit) . so i think, compress option is pretty short-sighted and not,
>>> what
>>> an average user would expect (
>>> https://marc.info/?l=linux-btrfs&m=154523409314147&w=2 )
>>
>> IIRC compress prop needs to be set to all inodes.
>> If you set it on a directory, only new inodes would inherit the prop,
>> the existing ones won't get the new prop.
>
>
> that's clear, and no different from zfs.
>
> the big advantage of btrfs is, that we have btrfs filesystem
> defragment to recompress everything
> in place, whereas in zfs we need to move it forth and back to a
> different dataset.
>
> regards
> Roland
>
>>
>> Thanks,
>> Qu
>>>
>>> i'd be happy on some feedback. not subscribed to this list, so
>>> please CC.
>>>
>>> roland
>>>
>>>
>>> zstd compression applied to all subvolumes, though specified otherwise:
>>>
>>> cat /etc/fstab|grep btrfs
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> mount|grep btrfs
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>> different order in fstab has different result - first wins:
>>>
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>> compress-force silently ignored:
>>>
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>>
>>> this is the write speed i get with compress-force=zstd on i7-7700
>>>
>>> # dd if=/dev/zero of=test.dat bs=1024k count=10240 ;time sync
>>> 10240+0 records in
>>> 10240+0 records out
>>> 10737418240 bytes (11 GB, 10 GiB) copied, 3.76111 s, 2.9 GB/s
>>>
>>> real 0m0.224s
>>> user 0m0.000s
>>> sys 0m0.081s
>>>
>>>
>>>
^ permalink raw reply [flat|nested] 7+ messages in thread