Re: Btrfs autodefrag wrote 5TB in one day to a 0.5TB SSD without a measurable benefit

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2022/3/12 07:28, Jan Ziak wrote:
> On Sat, Mar 12, 2022 at 12:04 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>> As stated before, autodefrag is not really that useful for database.
>
> Do you realize that you are claiming that btrfs autodefrag should not
> - by design - be effective in the case of high-fragmentation files?

Unfortunately, that's exactly what I mean.

We all know random writes would cause fragments, but autodefrag is not
like regular defrag ioctl, as it only scan newer extents.

For example:

Our autodefrag is required to defrag writes newer than gen 100, and our
inode has the following layout:

|---Ext A---|--- Ext B---|---Ext C---|---Ext D---|---Ext E---|
     Gen 50       Gen 101     Gen 49      Gen 30      Gen 30

Then autodefrag will only try to defrag extent B and extent C.

Extent B meets the generation requirement, and is mergable with the next
extent C.

But all the remaining extents A, D, E will not be defragged as their
generations don't meet the requirement.


While for regular defrag ioctl, we don't have such generation
requirement, and is able to defrag all extents from A to E.
(But cause way more IO).



Furthermore, autodefrag works by marking the target range dirty, and
wait for writeback (and hopefully get more writes near it, so it can get
even larger)

But if the application, like the database, is calling fsync()
frequently, such re-dirtied range is going to writeback almost
immediately, without any further chance to get merged larger.

Thus the autodefrag effectiveness is almost zero for random writes +
frequently fsync(), which is exactly the database workload.

> If
> it isn't supposed to be useful for high-fragmentation files then where
> is it supposed to be useful? Low-fragmentation files?

Frequently append writes, or less frequently fsync() calls.

Thanks,
Qu

>
> -Jan