Re: BTRFS doesn't compress on the fly

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2023/12/3 20:15, Gerhard Wiesinger wrote:
> On 03.12.2023 10:11, Qu Wenruo wrote:
>>
>>>
>>> Thank you for reproducting it. Think we nailed it down.
>>>
>>> Is there a way to get the output of the file of the chunks/items?
>>
>> You can always dump the full subvolume (`btrfs ins dump-tree -t
>> <subvolid> <device>`), then try to grep the inode which has PREALLOC
>> alloc (`| grep -C 5 "flags.*PREALLOC"), which would include the inode
>> number, then you can ping down the inodes which has PREALLOC flags and
>> not undergoing compression.
>>
>> I won't be surprised most (if not all) files of postgresql would have
>> that flag.
>
> Looks like only a small part has PREALLOC:
>
> find /var/lib/pgsql -type f | wc -l
> 5569
>
> btrfs inspect-internal dump-tree /dev/mapper/datab | grep -i PREALLOC |
> wc -l
> 95
>
> For reference:
>
> How to find the file at a certain btrfs inode
> https://serverfault.com/questions/746938/how-to-find-the-file-at-a-certain-btrfs-inode
>
> btrfs inspect-internal inode-resolve 13269 /var/lib/pgsql
> /var/lib/pgsql/16/data/base/16400/16419
>
> find /var/lib/pgsql -xdev -inum 13269
> /var/lib/pgsql/16/data/base/16400/16419
>
> # Get  files from inodes
>
> btrfs inspect-internal dump-tree /dev/mapper/datab | grep -C 5
> "flags.*PREALLOC" | grep -i INODE | perl -pe 's/.*?\((.*?) .*/$1/' |
> sort | uniq | while read INODE; do echo -n "$INODE: ";btrfs
> inspect-internal inode-resolve ${INODE} /var/lib/pgsql; done
>
> # Number of inodes, count is consistent
>
> btrfs inspect-internal dump-tree /dev/mapper/datab | grep -C 5
> "flags.*PREALLOC" | grep -i INODE | perl -pe 's/.*?\((.*?) .*/$1/' |
> sort | uniq | while read INODE; do echo -n "$INODE: ";btrfs
> inspect-internal inode-resolve ${INODE} /var/lib/pgsql; done | wc -l
>
> 95
>
> All files are in subdirectories: /var/lib/pgsql/16/data/base/
>
> Already an idea for the fix?

We can copy the files (without using reflink) to a temporary location
(better out of btrfs), then copy the temporary copy back to overwrite
all the existing files.

The problem is still inside pgsql, as long as they do preallocation, the
same problem would still happen.

>
> BTW:
>
> if compression is forced, should be then just any "block" be compressed?

There is a long existing problem with compression with preallocation.

One easy example is, if we go compression for the preallocated range,
what we do with the gap (compressed size is always smaller than the real
size).

If we leave the gap, then the read performance can be even worse, as now
we have to read several small extents with gaps between them, vs a large
contig read.

IIRC years ago when I was a btrfs newbie, that's the direction I tried
to go, but never reached upstream.

Thus you can see some of the reason why we do not go compression for
preallocated range.

But I still don't believe we should go as the current behavior.
We should still try to go compression as long as we know the write still
needs COW, thus we should fix it.

Thanks,
Qu

>
> Or, what's the problem of the logic?
>
> Thnx.
>
> Ciao,
>
> Gerhard
>