Re: 'btrfs filesystem defragment' makes files explode in size, especially fallocated ones

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

在 2024/8/6 19:53, Hanabishi 写道:
> On 8/6/24 09:55, Qu Wenruo wrote:
>
>> So either there is something like cgroup involved (which can limits the
>> dirty page cache and trigger write backs), or some other weird
>> behavior/bugs.
>
> Yes, this line reveals something. I do have modified dirty page cache
> values. I tend to keep it on low values.
>
> Now playing around with it - yes, it is seems to be the cause. When I
> tune 'vm.dirty_ratio' and 'vm.dirty_background_ratio' up to higher
> values, the problem becames less prevalent.
>
> Which means lowering them cranks up the problem to extremes. E.g. try
>
> # sysctl -w vm.dirty_bytes=8192
> # sysctl -w vm.dirty_background_ratio=0
>
> With that setup defrag completely obliterates files even with default
> threshold value.

At least I no longer need to live under the fear of new defrag bugs.

This also explains why defrag (even with default values) would trigger
rewrites of extents, because although fiemap is only showing a single
extent, it will be a lot of small extents on the larger pre-allocated range.

Thus btrfs believe it can merge all of them into a larger extent, but VM
settings forces btrfs to write them early, causing extra data COW, and
cause worse fragmentation.

Too low values means kernel will trigger dirty writeback aggressively, I
believe for all extent based file systems (ext4/xfs/btrfs etc), it would
cause a huge waste of metadata, due to the huge amount of small extents.

So yes, that setting is the cause, although it will reduce the memory
used by page cache (it still counts as memory pressure), but the cost is
more fragmented extents and overall worse fs performance and possibly
more wear on NAND based storage.

Thanks,
Qu