Re: 'btrfs filesystem defragment' makes files explode in size, especially fallocated ones

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

在 2024/8/6 03:46, Hanabishi 写道:
> On 8/4/24 22:19, Qu Wenruo wrote:
>> Mind to dump the filemap output (xfs_io -c "fiemap -v") before and
>> after the defrag?
>>
>> Thanks,
>> Qu
>
> Sure.
>
> # compsize mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst
> Processed 1 file, 1 regular extents (1 refs), 0 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL      100%      224M         224M         224M
> none       100%      224M         224M         224M
>
> # xfs_io -c "fiemap -v" mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst
> mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst:
>   EXT: FILE-OFFSET      BLOCK-RANGE           TOTAL FLAGS
>     0: [0..460303]:     545974648..546434951 460304   0x1

Weird, there is no fallocated space involved at all.

>
> # btrfs filesystem defragment -t 1G

Oh you're using non-default threshold.
Unfortunately 1G makes no sense, as btrfs's largest extent size is only
128M.

(Although the above output shows an extent with 224M size, it's because
btrfs merges the fiemap result internally when possible).

It's recommended to go the default values anyway.

> mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst
> mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst
>
> # compsize mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst
> Processed 1 file, 8 regular extents (8 refs), 0 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL      100%      420M         420M         224M
> none       100%      420M         420M         224M
>
> # xfs_io -c "fiemap -v" mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst
> mingw-w64-gcc-13.1.0-1-x86_64.pkg.tar.zst:
>   EXT: FILE-OFFSET      BLOCK-RANGE           TOTAL FLAGS
>     0: [0..511]:        15754800..15755311      512   0x0
>     1: [512..2559]:     22070192..22072239     2048   0x0
>     2: [2560..6655]:    22632216..22636311     4096   0x0
>     3: [6656..14335]:   22072240..22079919     7680   0x0
>     4: [14336..385023]: 546434952..546805639 370688   0x0
>     5: [385024..400383]: 44592672..44608031    15360   0x0

All the above extents are new extents.

>     6: [400384..460303]: 546375032..546434951  59920   0x1
>

While this one is the old one.

This looks like a recent bug fix e42b9d8b9ea2 ("btrfs: defrag: avoid
unnecessary defrag caused by incorrect extent size"), which is merged in
v6.8 kernel.

Mind to provide the kernel version?


Furthermore, there is another problem, according to your fiemap result,
the fs seems to cause fragmented new extents by somehow.

Is there any memory pressure or the fs itself is fragmented?
Btrfs defrag is only re-dirty the data, then write them back.
This expects them to be written in a continuous extent, but both memory
pressure and fragmented fs can all break such assumption.

Thanks,
Qu