Re: BTRFS doesn't compress on the fly

[Gerhard Wiesinger <lists@wiesinger.com>]

Hello Qu,

Thank you for the answers, see inline.

Any further ideas?

Ciao,
Gerhard.

On 30.11.2023 21:53, Qu Wenruo wrote:
>
>
> On 2023/11/30 21:51, Gerhard Wiesinger wrote:
>> Dear All,
>>
>> I created a new BTRFS volume with migrating an existing PostgreSQL
>> database on it. Versions are recent.
>
> Does the data base directory has something like NODATACOW or NODATASUM 
> set?
> The other possibility is preallocation, for the first write on
> preallocated range, no matter if the compression is enabled, the write
> would be treated as NOCOW.
>
I don't think so. How to find out (googled already a lot)?

At least it is not mounted with these options (see also original post).

# Mounted via force
findmnt -vno OPTIONS /var/lib/pgsql
rw,relatime,compress-force=zstd:3,space_cache=v2,subvolid=5,subvol=/'

According to the following link it should compress anyway with the -o 
compress-force option:

https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Compression.html#What.27s_the_precedence_of_all_the_options_affecting_compression.3F
Compression to newly written data happens:
always -- if the filesystem is mounted with -o compress-force
never -- if the NOCOMPRESS flag is set per-file/-directory
if possible -- if the COMPRESS per-file flag (aka chattr +c) is set, but 
it may get converted to NOCOMPRESS eventually
if possible -- if the -o compress mount option is specified
Note, that mounting with -o compress will not set the +c file attribute.

>>
>> Compression is not done on the fly although everything is IMHO
>> configured correctly to do so.
>>
>> I need to run the following command that everything gets compressed:
>> btrfs filesystem defragment -r -v -czstd /var/lib/pgsql
>>
>> Had also a problem that
>> chattr -R +c /var/lib/pgsql
>> didn't work for some files.
>>
>> Find further details below.
>>
>> Looks like a bug to me.
>>
>> Any ideas?
>>
>> Thanx.
>>
>> Ciao,
>> Gerhard
>>
>> uname -a
>> Linux myhostname 6.5.12-300.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Nov
>> 20 22:44:24 UTC 2023 x86_64 GNU/Linux
>>
>> btrfs --version
>> btrfs-progs v6.5.1
>>
>> btrfs filesystem show
>> Label: 'database'  uuid: 6ad6ef90-30fa-4979-9509-99803f7545aa
>>          Total devices 1 FS bytes used 15.76GiB
>>          devid    1 size 129.98GiB used 21.06GiB path /dev/mapper/datab
>>
>> btrfs filesystem df /var/lib/pgsql
>> Data, single: total=19.00GiB, used=15.61GiB
>> System, DUP: total=32.00MiB, used=16.00KiB
>> Metadata, DUP: total=1.00GiB, used=151.92MiB
>> GlobalReserve, single: total=85.38MiB, used=0.00B
>>
>> # Mounted via force
>> findmnt -vno OPTIONS /var/lib/pgsql
>> rw,relatime,compress-force=zstd:3,space_cache=v2,subvolid=5,subvol=/'
>>
>> # all files even have "c" attribute, set after creation of the 
>> filesystem
>> lsattr /var/lib/pgsql
>> --------c------------- /var/lib/pgsql/16
>>
>> # Should be empty and is empty, so everything has the comressed
>> attribute (after creation and also all new files)
>> lsattr -R /var/lib/pgsql | grep -v "^/" | grep -v "^$" | grep -v
>> "^........c"
>>
>> # Stays here at this compression level
>> compsize -x /var/lib/pgsql
>> Processed 5332 files, 575858 regular extents (591204 refs), 40 inline.
>> Type       Perc     Disk Usage   Uncompressed Referenced
>> TOTAL       63%       51G          80G          80G
>> none       100%       40G          40G          40G
>> zstd        27%       10G          40G          40G
>> prealloc   100%      5.0M         5.0M         5.5M
>
> Not sure if the preallocation is the cause, but maybe you can try
> disabling preallocation of postgresql?
>
> As preallocation doesn't make that much sense on btrfs, there are too
> many cases that can break the preallocation.


I googled a lot and didn't find anything useful with preallocation and 
postgresql (looks like it doesn'use fallocate).

How can I find something about preallocation out?