'btrfs filesystem defragment' makes files explode in size, especially fallocated ones

[i.r.e.c.c.a.k.u.n+kernel.org@gmail.com]

(Originally reported on Kernel.org Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=219033)

There is a very weird quirk I found with 'btrfs filesystem defragment' command. And no, it's not about reflinks removal, I'm aware of that.

It is kinda hard to replicate, but I found a somewhat reliable way. It reaches extremes with fallocated files specifically.

1. Create a file on a Btrfs filesystem using 'fallocate' and fill it. The easy way to do that is just to copy some files with 'rsync --preallocate'.

2. Check compsize info:

# compsize foo
Processed 71 files, 71 regular extents (71 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL      100%      630M         630M         630M
none       100%      630M         630M         630M

All is fine here for now. 1 extent per 1 file, "Disk Usage" = "Referenced".

3. Run defragment:

# btrfs filesystem defragment -r foo

4. Check compsize again:

# compsize foo
Processed 71 files, 76 regular extents (76 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL      100%      638M         638M         630M
none       100%      638M         638M         630M

Oops, besides the fact that the amount of extents is actually increased, which means 'btrfs filesystem defragment' actually made fragmentation worse, physical disk usage increased for no reason. And I didn't find any way to shrink it back.

---

The end result seems to be random though. But I managed to achieve some truly horrifying results.

# compsize foo
Processed 45 files, 45 regular extents (45 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL      100%      360M         360M         360M
none       100%      360M         360M         360M

# btrfs filesystem defragment -r -t 1G foo

# compsize foo
Processed 45 files, 144 regular extents (144 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL      100%      716M         716M         360M
none       100%      716M         716M         360M

Yikes! Triple the extents! Double increase in size!