Re: apply different compress/compress-force settings on different subvolumes ?

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

在 2024/2/21 23:45, Roland 写道:
> Am 21.02.24 um 10:33 schrieb Qu Wenruo:
>
>> 在 2024/2/21 19:19, Roland 写道:
>>> hello,
>>>
>>> what can be the reason , that multiple compress mount option do not work
>>> on subvolume level , i.e. it's always the first that wins ?
>> In that case, you may want to use "btrfs prop" subcommand to setup
>> compression for each subvolume.
>>
>> The mount option one is really affecting the whole filesystem.
>>>
>>> and why is compress-force silently ignored? (see below)
>> Compress-force has no coresponding per-inode prop option though.
>>
>> But I don't think compress-force would cause much difference against
>> regular compress.
>
> apparently, force-compress can make a big difference (and compress only
> can lead to no compression at all):
>
> https://bugzilla.proxmox.com/show_bug.cgi?id=5250#c6

OK, didn't expect the compression ratio detection to cause so much
difference.

Which also means, we can further improve the compression detection.

In that case, I would purpose to change "compress-force" to
"skip_compress_heuristic" or something similar, and get rid of the
compression string/level for the new option.

The "compress-force=%s" is really confusing against the existing
"comrepss=%s" option.

To me, this sounds like a change which would improve the user interface.
And with the new option named determined, I'm pretty happy to add a new
prop to skip the heuristic.

Thanks,
Qu
>
>
>>> regarding compress mount option, it seems that this can be overriden via
>>> subvolume property, which can work around the problem and have multiple
>>> compression settings.
>>>
>>> but how to fix compress-force in the same way, i.e. how can we have
>>> differenty compress-force settings with one btrfs fs ?
>>>
>>> wouldn't it make sense to introduce compress-force property for this ?
>>>
>>> what about replacing compress with compress-force (i.e. make it the
>>> default), as there is not much overhead with modern/fast cpu ?
>>>
>>> i think compress-force / compress is VERY confusing from and end user
>>> perspective - and even worse, i have set "compress" on a subvolume used
>>> for virtual machines and they do not get compressed at all (not a single
>>> bit) . so i think, compress option is pretty short-sighted and not, what
>>> an average user would expect (
>>> https://marc.info/?l=linux-btrfs&m=154523409314147&w=2 )
>>
>> IIRC compress prop needs to be set to all inodes.
>> If you set it on a directory, only new inodes would inherit the prop,
>> the existing ones won't get the new prop.
>
>
> that's clear, and no different from zfs.
>
> the big advantage of btrfs is, that we have btrfs filesystem defragment
> to recompress everything
> in place, whereas in zfs we need to move it forth and back to a
> different dataset.
>
> regards
> Roland
>
>>
>> Thanks,
>> Qu
>>>
>>> i'd be happy on some feedback. not subscribed to this list, so please
>>> CC.
>>>
>>> roland
>>>
>>>
>>> zstd compression applied to all subvolumes, though specified otherwise:
>>>
>>> cat /etc/fstab|grep btrfs
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo  btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> mount|grep btrfs
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>> different order in fstab has different result - first wins:
>>>
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo  btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=lzo,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>> compress-force silently ignored:
>>>
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd btrfs
>>> subvol=zstd,compress=zstd,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/lzo  btrfs
>>> subvol=lzo,compress=lzo,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/none btrfs
>>> subvol=none,compress=none,defaults 0 1
>>> UUID=da0fadf1-5db4-4f5b-a77e-2f0730f4e872 /btrfs/zstd-force btrfs
>>> subvol=zstd-force,compress-force=zstd,defaults 0 1
>>>
>>> /dev/sdb1 on /btrfs/zstd type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=259,subvol=/zstd)
>>>
>>> /dev/sdb1 on /btrfs/lzo type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/lzo)
>>>
>>> /dev/sdb1 on /btrfs/none type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=258,subvol=/none)
>>>
>>> /dev/sdb1 on /btrfs/zstd-force type btrfs
>>> (rw,relatime,compress=zstd:3,ssd,discard=async,space_cache=v2,subvolid=260,subvol=/zstd-force)
>>>
>>>
>>>
>>>
>>> this is the write speed i get with compress-force=zstd on i7-7700
>>>
>>> # dd if=/dev/zero of=test.dat bs=1024k count=10240 ;time sync
>>> 10240+0 records in
>>> 10240+0 records out
>>> 10737418240 bytes (11 GB, 10 GiB) copied, 3.76111 s, 2.9 GB/s
>>>
>>> real    0m0.224s
>>> user    0m0.000s
>>> sys    0m0.081s
>>>
>>>
>>>
>