Re: BTRFS doesn't compress on the fly

[Gerhard Wiesinger <lists@wiesinger.com>]

On 02.12.2023 22:56, Qu Wenruo wrote:
>>
>>>
>>> How can I find something about preallocation out?
>>
>> Above compsize is already showing there is some preallocated space.
>>
>> Thus I'm wondering if the preallocation is the cause.
>>
>> As should_nocow() would also check the PREALLOC inode flag, and tries
>> NOCOW path first (then falls to COW if needed)
>
> Yep, I just reproduced it, for any INODE with PREALLOC flag (aka, the
> file has some preallocated range), even we're writing into the range
> that needs COW anyway (e.g. new writes which would enlarge the file),
> the compression would not work anyway.
>
>  # mkfs.btrfs test.img
>  # mount test.img -o compress-force=zstd /mnt/btrfs
>  # fallocate -l 128k /mnt/btrfs/file1
>  # xfs_io -f -c "pwrite 128k 128k" /mnt/btrfs/file1
>  # xfs_io -f -c "pwrite 128k 128k" /mnt/btrfs/file2
>  # sync
>
> Since file1 has 128K preallocated range, thus the inode has PREALLOC
> flag, and would lead to no compression:
>
>     item 6 key (257 INODE_ITEM 0) itemoff 15811 itemsize 160
>         generation 8 transid 8 size 262144 nbytes 262144
>         block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0
>         sequence 33 flags 0x10(PREALLOC) <<<<
>     item 7 key (257 INODE_REF 256) itemoff 15796 itemsize 15
>         index 2 namelen 5 name: file1
>     item 8 key (257 EXTENT_DATA 0) itemoff 15743 itemsize 53
>         generation 8 type 2 (prealloc)
>         prealloc data disk byte 13631488 nr 131072
>         prealloc data offset 0 nr 131072
>     item 9 key (257 EXTENT_DATA 131072) itemoff 15690 itemsize 53
>         generation 8 type 1 (regular)
>         extent data disk byte 13762560 nr 131072
>         extent data offset 0 nr 131072 ram 131072
>         extent compression 0 (none) <<<
>
> Meanwhile for the other file, which has no prealloc, would go regular
> compression path.
>
>     item 10 key (258 INODE_ITEM 0) itemoff 15530 itemsize 160
>         generation 8 transid 8 size 262144 nbytes 131072
>         block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
>         sequence 32 flags 0x0(none)
>     item 11 key (258 INODE_REF 256) itemoff 15515 itemsize 15
>         index 3 namelen 5 name: file2
>     item 12 key (258 EXTENT_DATA 131072) itemoff 15462 itemsize 53
>         generation 8 type 1 (regular)
>         extent data disk byte 13893632 nr 4096
>         extent data offset 0 nr 131072 ram 131072
>         extent compression 3 (zstd)
>
> To me, this looks a bug, and the reason is exactly what I explained 
> before.
>
> The worst thing is, as long as the inode has PREALLOC flag, even if all
> preallocated extents are used, it would prevent compression from
> happening, forever for that inode.
>
> Let me try to fix the fallback to COW path to include compression.


Thank you for reproducting it. Think we nailed it down.

Is there a way to get the output of the file of the chunks/items?

Thnx.

Ciao,

Gerhard