Re: unable to remove device due to enospc

[Ross Vandegrift <ross@kallisti.us>]

On Mon, Jan 10, 2022 at 11:22:10PM -0800, Ross Vandegrift wrote:
> Hoping someone can help with an enospc issue.  I'm converting a single 8TB fs
> -> 2*10TB raid1.  So far I:
>   1. added the two new devices
>   2. rebalanced single -> raid1
>   3. tried to remove the 8TB device.
> 
> The balance in #2 failed with enospc and left 35G of data in the single
> profile.  The remove in #3 sucessfully moved everything but 35G off.  There is
> 2.5TB unallocated on the new devices:
> 
> Overall:
>     Device size:	          25.34TiB
>     Device allocated:	          13.00TiB
>     Device unallocated:		  12.34TiB
>     Device missing:	             0.00B
>     Used:		          12.94TiB
>     Free (estimated):	           6.21TiB	(min: 6.20TiB)
>     Free (statfs, df):	           5.15TiB
>     Data ratio:	             	      1.99
>     Metadata ratio:	              2.00
>     Global reserve:	         512.00MiB	(used: 0.00B)
>     Multiple profiles:	               yes	(data)
> 
> Data,single: Size:35.00GiB, Used:34.98GiB (99.94%)
>    /dev/mapper/backup	  35.00GiB
> 
> Data,RAID1: Size:6.46TiB, Used:6.44TiB (99.60%)
>    /dev/mapper/backup_a    6.46TiB
>    /dev/mapper/backup_b    6.46TiB
> 
> Metadata,RAID1: Size:19.00GiB, Used:17.71GiB (93.22%)
>    /dev/mapper/backup_a   19.00GiB
>    /dev/mapper/backup_b   19.00GiB
> 
> System,RAID1: Size:32.00MiB, Used:936.00KiB (2.86%)
>    /dev/mapper/backup	  32.00MiB
>    /dev/mapper/backup_a   32.00MiB
> 
> Unallocated:
>    /dev/mapper/backup	   7.24TiB
>    /dev/mapper/backup_a    2.55TiB
>    /dev/mapper/backup_b    2.55TiB

Still having this issue.  Did some more reading, found similar cases.

There's the "when you're not in the 'usual' case" problem from the wiki.
That's me.  It refers to [1].  But I'm not out of space for file writes, only
on balance/dev remove.

There's a reddit thread at [2] where someone has a similar issue at [2].  It
mentions a bug mention in 5.4, but I was on 5.10, and now 5.14.  It also
mentions issues related to balances on both data & metadata.  `btrfs bal start
-dprofiles=single` fails with enospc too.

I tried shrinking /dev/mapper/backup.  Shrinking to 750G worked, but even with
just has 35G of data, trying to shrink to 700G fails with enospc.  

btrfs check passes without errors.

Thanks,
Ross

[1] - https://bugzilla.kernel.org/show_bug.cgi?id=74101
[2] - https://www.reddit.com/r/btrfs/comments/k17v4z/no_space_left_during_rebalance/