Re: Kernel 5.4 - BTRFS FS shows full with about 600 GB Free ?

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2019/12/22 下午3:55, Swâmi Petaramesh wrote:
> Hi again,
>
> Le 21/12/2019 à 13:00, Qu Wenruo a écrit :
>> A known regression introduced in v5.4.
>>
>> The new metadata over-commit behavior conflicts with an existing check
>> in btrfs_statfs().
>> It is completely a runtime false behavior, and had*no*  other bad effect.
>
> I'm painfully unable to understand how the same said « metadata
> over-commit » would allow, on my system, “df” to work properly on some
> filesystems and fail miserably on others.

Here comes the full explanation if you like tech details:

For v5.4, btrfs allows reserved metadata space to go beyond current
metadata free space, as long as the fs can allocate enough metadata.

E.g you have a 1T fs, with 2G metadata (SINGLE) chunks allocated.
And among all these 2G metadata chunks, you have 1G unused space.

In v5.4, btrfs is allowed to reserve extra metadata space, near (1T -
1G). Since btrfs knows we can allocate new metadata chunks for (1T -
2G). And we also have 1G free metadata space.

However, in btrfs_statfs(), there is a check to ensure we have at least
4M free space in allocated metadata chunks.
If that condition can't be met, then btrfs_statfs() return f_bavail = 0,
indicating no available space.

That check is from 2015, which is kinda OK at that time, but definitely
not works with the new over-commit behavior.

So if you have enough uncommitted metadata, that check will be triggered
and suddenly returns 0 available space, even 1 sec early you still get
tons of available space.


>
> AND show “100% full” filesystems with still more than 600 GB free...
> That would make quite much of an overcommit isn't it ?
>
> So I wonder if there isn't something else or if the “overcommit”
> calculation is not really completely broke in the first place...

It's the old check from 2015 completely broken with 2019 new behavior.

If you really want to dig into the code, please check btrfs_statfs() of
fs/btrfs/super.c.

        if (!mixed && total_free_meta - thresh < block_rsv->size)
                buf->f_bavail = 0;

Which is explained in my patch mentioned in previous reply.

Thanks,
Qu

>
> ॐ
>