Re: btrfs blocks root from writing even when there is plenty of statfs->f_bfree

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2023/8/25 04:33, Jeff Layton wrote:
> I've been doing some testing with btrfs exported via nfsd and hit this
> bug. root is unable to write to the filesystem, even though statfs
> reports that there should be 400k+ blocks available.
>
> The reproducer is pretty simple. First I fill up the filesystem as an
> unprivileged user (let this run until I hit ENOSPC):
>
> [vagrant@kdevops-nfs-default btrfs]$ dd if=/dev/urandom of=/media/btrfs/bigfile bs=1M
>
> ...and then try to write to a file on the fs as root:
>
> [root@kdevops-nfs-default btrfs]# strace -f -e statfs df .
> statfs(".", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096, f_blocks=26214400, f_bfree=452204, f_bavail=0, f_files=0, f_ffree=0, f_fsid={val=[0xa9d43863, 0xb08b34cc]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_RELATIME}) = 0

Note that, unlike XFS/EXT4, btrfs goes strict limits between data and
metadata.

Metadata space can only be used to store tree blocks, never data.
And both metadata and data space are allocated dynamically, thus we have
have problems when the workload is unbalanced.
(This is a little like the inodes limits on XFS/EXT4, but more dynamic)

In your particular case, the data space is all used up, but metadata
space still has some free space.

And f_btree returned by btrfs is including the metadata space, as
although that's inaccurate, it's the best what we can do to report.


> Filesystem     1K-blocks      Used Available Use% Mounted on
> /dev/nvme2n1   104857600 103048784         0 100% /media/btrfs
> +++ exited with 0 +++
> [root@kdevops-nfs-default btrfs]# xfs_io -f -c "pwrite -b 4096 4096 4096" ./file1
> pwrite: No space left on device
>
> This works on ext4 and xfs. The kernel is Linus' master branch as of a
> couple of days ago:
>
>      6.5.0-rc7-g081b0d4bef5d
>
> Let me know if you need more info.

That's why for btrfs we recommend to go "btrfs fi usage" to get a more
comprehensive understand on the space usage.

In your case, Data usage should be 100%, without any spare ones.

Thanks,
Qu