Re: [bug] Balance fails due to ENOSPC

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2021/1/12 下午6:35, Anand Jain wrote:
>
>
> Balance fails due to ENOSPC on 5.11.0-rc2+.
>
>
> [ 2460.219094] BTRFS info (device sdb6): balance: start -f
> -dconvert=single -mconvert=single -sconvert=single
> [ 2460.219224] BTRFS info (device sdb6): relocating block group
> 194572713984 flags data
> [ 2460.241299] ------------[ cut here ]------------
> [ 2460.241302] BTRFS: Transaction aborted (error -28)
> [ 2460.241328] WARNING: CPU: 3 PID: 4135 at fs/btrfs/volumes.c:3003
> btrfs_remove_chunk+0x60e/0x6c0 [btrfs]
> ::
> [ 2460.241447] RIP: 0010:btrfs_remove_chunk+0x60e/0x6c0 [btrfs]
> ::
> [ 2460.241497] Call Trace:
> [ 2460.241500]  btrfs_relocate_chunk+0x9d/0xd0 [btrfs]
> [ 2460.241533]  __btrfs_balance+0x413/0xa30 [btrfs]
> [ 2460.241566]  btrfs_balance+0x436/0x4d0 [btrfs]
> [ 2460.241597]  btrfs_ioctl_balance+0x2d5/0x380 [btrfs]
> [ 2460.241647]  btrfs_ioctl+0x565/0x22b0 [btrfs]
> [ 2460.241693]  ? selinux_file_ioctl+0x174/0x220
> [ 2460.241698]  ? handle_mm_fault+0xd7/0x2b0
> [ 2460.241702]  __x64_sys_ioctl+0x91/0xc0
> [ 2460.241705]  ? __x64_sys_ioctl+0x91/0xc0
> [ 2460.241707]  do_syscall_64+0x38/0x50
> [ 2460.241711]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> ::
> [ 2460.241735] ---[ end trace 95b8b7b84503a477 ]---
> [ 2460.241738] BTRFS: error (device sdb6) in btrfs_remove_chunk:3003:
> errno=-28 No space left
> [ 2460.241743] BTRFS info (device sdb6): forced readonly
> [ 2460.242646] BTRFS info (device sdb6): 1 enospc errors during balance
> [ 2460.242649] BTRFS info (device sdb6): balance: ended with status: -30
>
>
> btrfs fi us /Volumes/
> Overall:
>      Device size:         154.97GiB
>      Device allocated:         154.97GiB
>      Device unallocated:           2.00MiB
>      Device missing:             0.00B
>      Used:             147.91GiB
>      Free (estimated):           2.75GiB    (min: 2.75GiB)
>      Data ratio:                  1.99
>      Metadata ratio:              1.52
>      Global reserve:         183.72MiB    (used: 0.00B)
>
> Data,single: Size:894.00MiB, Used:0.00B (0.00%)
>     /dev/sdb6     894.00MiB
>
> Data,RAID1: Size:75.53GiB, Used:73.23GiB (96.96%)
>     /dev/sdb6      75.53GiB
>     /dev/sdc2      75.53GiB
>
> Metadata,single: Size:964.00MiB, Used:0.00B (0.00%)
>     /dev/sdb6     964.00MiB
>
> Metadata,RAID1: Size:1.00GiB, Used:737.25MiB (72.00%)
>     /dev/sdb6       1.00GiB
>     /dev/sdc2       1.00GiB

Metadata is only 1G, with 737M used, and 183 reserved as GlobalRSV.

Although the GlobalRSV is a little smaller than older kernel, but it
should be more or less enough.

In theory, we should be able to modify the fs stealing from GlobalRSV,
and since btrfs_remove_chunk:3003 is updating device time, either it
means we didn't really reserve enough metadata when we start the trans,
or we didn't do the steal when we really need.

Would you like to provide the dmesg if you can reproduce with "-o
enospc_debug" mount option?

Thanks,
Qu
>
> System,single: Size:32.00MiB, Used:0.00B (0.00%)
>     /dev/sdb6      32.00MiB
>
> System,RAID1: Size:32.00MiB, Used:16.00KiB (0.05%)
>     /dev/sdb6      32.00MiB
>     /dev/sdc2      32.00MiB
>
> Unallocated:
>     /dev/sdb6       1.00MiB
>     /dev/sdc2       1.00MiB