Re: Btrfs ENOSPC / Stuck in RO with "exclusive operation balance paused in progress"

[Alban Browaeys <alban.browaeys@gmail.com>]

Hi,

I had space after the sdb3 btrfs /var partition, a 9GB swap partition. I 
deleted this swap partition and expanded the sdb3 btrfs size with parted
But I don't seem to be able to expand the btrfs filesystem inside sdb3 
if I cannot mount it read write, is it so? Or is it only due to my 
pending operation?
Can I get this btrfs partition back?

I have an image of this partiton but it seems I will have the same issue 
with this image and will be unable to restore it.
Is there a way to copy the content of the partition mounted in ro in its 
current state, create a new partition and copy the content back to it?
There are docker subvolumes on it wich seems they can be recreated by 
docker later on (I hope so) so rsync might be an option but if the 
docker subvolume need to be backed up rsync is not able to backup them 
correctly.
And btrfs restore gives me thousands of

Also should I expect this behavior to happen everytime I do a btrfs 
filessytem conversion without checking I have enough metadata space 
available beforehand or will this be prevented by some new code one day ?


# parted /dev/sdb
GNU Parted 3.7
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) select /dev/sdb
Using /dev/sdb
(parted) p
Model: ATA SanDisk SD9SB8W1 (scsi)
Disk /dev/sdb: 128GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name                  Flags
  1      1049kB  903MB   902MB   fat32           EFI system partition 
boot, hidden, esp
  2      903MB   39.8GB  38.9GB  btrfs           root
  3      43.2GB  117GB   73.5GB  btrfs           ssdhome
  4      119GB   128GB   9197MB  linux-swap(v1)  SSDSWAP               swap

(parted) rm 4
(parted) resizepart 3 100%
(parted) p
Model: ATA SanDisk SD9SB8W1 (scsi)
Disk /dev/sdb: 128GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name                  Flags
  1      1049kB  903MB   902MB   fat32        EFI system partition 
boot, hidden, esp
  2      903MB   39.8GB  38.9GB  btrfs        root
  3      43.2GB  128GB   84.8GB  btrfs        ssdhome



# mount /dev/sdb3 /mnt/3 
  

[441290.450174] BTRFS: device label SSDHOME devid 1 transid 44655875 
/dev/sdb3 (8:19) scanned by mount (7976) 
                           [441290.461454] BTRFS info (device sdb3): 
first mount of filesystem 13af326c-631f-482b-9c34-b59b4f100608
[441290.471065] BTRFS info (device sdb3): using crc32c checksum 
algorithm 

[441290.518919] BTRFS warning (device sdb3): Skipping commit of aborted 
transaction. 

[441290.527296] ------------[ cut here ]------------ 
  

[441290.532875] BTRFS: Transaction aborted (error -28) 

[441290.538625] WARNING: fs/btrfs/transaction.c:2060 at 
btrfs_commit_transaction+0xaaa/0xe20 [btrfs], CPU#2: mount/7976
[441290.550177] Modules linked in: nls_ascii nls_cp437 vfat fat exfat 
ntfs3 btrfs libblake2b raid10 raid456 async_raid6_recov async_memcpy 
async_pq async_xor async_tx x
or raid6_pq raid1 raid0 md_mod autofs4 hid_logitech_hidpp 
hid_logitech_dj hid_generic usbhid hid sr_mod cdrom uas usb_storage i915 
drm_buddy ttm i2c_algo_bit drm_displa
y_helper cec sd_mod rc_core drm_client_lib drm_kms_helper iTCO_wdt 
xhci_pci ahci intel_pmc_bxt drm libahci watchdog xhci_hcd firewire_ohci 
libata firewire_core r8169 vi
deo realtek crc_itu_t usbcore scsi_mod fan i2c_i801 wmi button 
usb_common i2c_smbus scsi_common efivarfs 

[441290.605209] CPU: 2 UID: 0 PID: 7976 Comm: mount Tainted: G        W 
          7.0.10+deb14-amd64 #1 PREEMPT(lazy)  Debian 7.0.10-1 

[441290.617556] Tainted: [W]=WARN 
  

[441290.621060] Hardware name: LENOVO 10M8S14C42/3102, BIOS M16KT71A 
09/14/2023
[441290.628556] RIP: 0010:btrfs_commit_transaction+0xaac/0xe20 [btrfs] 

[441290.635306] Code: ff 48 8b 7c 24 08 31 c9 48 8d b7 48 01 00 00 e8 9a 
7c 0b 00 48 c7 43 18 00 00 00 00 e9 73 fe ff ff 48 8d 3d 66 fd 34 00 89 
ee <67> 48 0f b9 3a e9
f8 fe ff ff 49 8b bc 24 48 03 00 00 be 02 00 00
[441290.654597] RSP: 0018:ffffce9fcc3fb6b8 EFLAGS: 00010292
[441290.660386] RAX: 0000000000000002 RBX: ffff8ac4c06a1c78 RCX: 
0000000000000003
[441290.668074] RDX: 0000000002040001 RSI: 00000000ffffffe4 RDI: 
ffffffffc1028080
[441290.675771] RBP: 00000000ffffffe4 R08: 0000000000000000 R09: 
ffffce9fcc3fb4a0
[441290.683464] R10: ffff8acbe23fff90 R11: 00000000fff7ffff R12: 
ffff8ac4c584f000
[441290.691152] R13: ffff8ac4c7211600 R14: ffff8ac4c584f4c0 R15: 
ffff8ac545b71b80
[441290.698842] FS:  00007f16a636e940(0000) GS:ffff8acc3a3fc000(0000) 
knlGS:0000000000000000 

[441290.707489] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
  
                        [441290.713789] CR2: 00007f71f5dcb19c CR3: 
000000034a2fa002 CR4: 00000000003706f0
[441290.721478] Call Trace: 
  

[441290.724485]  <TASK> 

[441290.727145]  ? start_transaction+0x228/0x880 [btrfs] 

[441290.732712]  flush_space+0x3c3/0x5a0 [btrfs] 

[441290.737591]  ? __schedule+0x489/0x1010 

[441290.741896]  ? btrfs_reduce_alloc_profile+0x5a/0x1d0 [btrfs] 

[441290.748152]  ? calc_available_free_space.isra.0+0x70/0xc0 [btrfs] 

[441290.754844]  priority_reclaim_metadata_space+0x96/0x130 [btrfs]
[441290.761360]  handle_reserve_ticket+0x5b/0x290 [btrfs]
[441290.767011]  ? btrfs_reduce_alloc_profile+0x5a/0x1d0 [btrfs] 
  

[441290.773268]  reserve_bytes+0x268/0x4c0 [btrfs] 
  

[441290.778314]  btrfs_reserve_metadata_bytes+0x1a/0xe0 [btrfs] 
  

[441290.784484]  btrfs_block_rsv_refill+0x60/0x90 [btrfs] 
  
                        [441290.790127]  evict_refill_and_join+0x47/0xe0 
[btrfs] 

[441290.795671]  btrfs_evict_inode+0x332/0x400 [btrfs] 
  

[441290.801048]  evict+0xfd/0x270 
  

[441290.804556]  btrfs_orphan_cleanup+0x21f/0x390 [btrfs] 
  

[441290.810191]  btrfs_start_pre_rw_mount+0x2a0/0x6c0 [btrfs] 

[441290.816159]  open_ctree+0x12fb/0x1730 [btrfs] 
  

[441290.821099]  btrfs_get_tree.cold+0xb/0xb6 [btrfs] 
  

[441290.826385]  ? ksys_read+0x73/0xf0 
  

[441290.830311]  vfs_get_tree+0x29/0xd0 
  

[441290.834324]  vfs_cmd_create+0x57/0xd0 
  

[441290.838474]  __do_sys_fsconfig+0x2fc/0x5e0 
  

[441290.843059]  do_syscall_64+0xcd/0x15f0 
  

[441290.847326]  ? from_kuid_munged+0x13/0x30 

[441290.851847]  ? __do_sys_geteuid+0x27/0x30 

[441290.856373]  ? do_syscall_64+0x10a/0x15f0 
  

[441290.860889]  ? __x64_sys_mkdir+0x39/0x50
[441290.865317]  ? do_syscall_64+0x10a/0x15f0
[441290.869841]  ? __x64_sys_statx+0x73/0xd0
[441290.874261]  ? do_syscall_64+0x10a/0x15f0 

[441290.878766]  ? irqentry_exit+0x78/0x680 

[441290.883089]  ? exc_page_fault+0x82/0x1d0 

[441290.887504]  entry_SYSCALL_64_after_hwframe+0x76/0x7e 

[441290.893048] RIP: 0033:0x7f16a67adb4a 
  

[441290.897115] Code: 73 01 c3 48 8b 0d c6 32 0d 00 f7 d8 64 89 01 48 83 
c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 af 01 00 00 0f 
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 96 32 0d 00 f7 d8 64 89 01 48 

[441290.916373] RSP: 002b:00007ffded6881b8 EFLAGS: 00000246 ORIG_RAX: 
00000000000001af 

[441290.924455] RAX: ffffffffffffffda RBX: 00005603bb8eba50 RCX: 
00007f16a67adb4a
[441290.932099] RDX: 0000000000000000 RSI: 0000000000000006 RDI: 
0000000000000003
[441290.939743] RBP: 0000000000000003 R08: 0000000000000000 R09: 
0000000000000000
[441290.947387] R10: 0000000000000000 R11: 0000000000000246 R12: 
00005603bb8ecb80
[441290.955041] R13: 00007f16a694027c R14: 00007f16a693e460 R15: 
00007f16a636e7f0
[441290.962695]  </TASK> 

[441290.965396] ---[ end trace 0000000000000000 ]---
[441290.970507] BTRFS info (device sdb3 state A): dumping space info:
[441290.977124] BTRFS info (device sdb3 state A): space_info DATA 
(sub-group id 0) has 17422925824 free, is not full 

[441290.987784] BTRFS info (device sdb3 state A): space_info 
total=70262980608, used=52839989248, pinned=0, reserved=0, may_use=0, 
readonly=65536 zone_unusable=0
[441291.002378] BTRFS info (device sdb3 state A): space_info METADATA 
(sub-group id 0) has -190382080 free, is full 

[441291.012959] BTRFS info (device sdb3 state A): space_info 
total=3221225472, used=2142470144, pinned=0, reserved=0, 
may_use=190382080, readonly=1078755328 zone_unusable=0 
  

[441291.028472] BTRFS info (device sdb3 state A): space_info SYSTEM 
(sub-group id 0) has 0 free, is not full 

[441291.038428] BTRFS info (device sdb3 state A): space_info 
total=33554432, used=16384, pinned=0, reserved=0, may_use=0, 
readonly=33538048 zone_unusable=0
[441291.052461] BTRFS info (device sdb3 state A): global_block_rsv: size 
190382080 reserved 190382080 

[441291.061805] BTRFS info (device sdb3 state A): trans_block_rsv: size 
0 reserved 0
[441291.069675] BTRFS info (device sdb3 state A): chunk_block_rsv: size 
0 reserved 0 

[441291.077552] BTRFS info (device sdb3 state A): remap_block_rsv: size 
0 reserved 0 

[441291.085436] BTRFS info (device sdb3 state A): delayed_block_rsv: 
size 0 reserved 0 

[441291.093455] BTRFS info (device sdb3 state A): delayed_refs_rsv: size 
0 reserved 0 

[441291.101367] BTRFS: error (device sdb3 state A) in 
cleanup_transaction:2060: errno=-28 No space left 

[441291.110841] BTRFS warning (device sdb3 state EA): could not allocate 
space for delete; will truncate on mount 

[441291.121191] BTRFS error (device sdb3 state EA): Error removing 
orphan entry, stopping orphan cleanup 

[441291.130780] BTRFS error (device sdb3 state EA): could not do orphan 
cleanup -28
[441291.138588] BTRFS error (device sdb3 state EA): commit super block 
returned -30
[441291.147315] BTRFS error (device sdb3 state EA): open_ctree failed: 
-28 

mount: /mnt/3: fsconfig() failed: No space left on device.
        dmesg(1) may have more information after failed mount system call.


# mount -o ro,rescue=all /dev/sdb3 /var
[442011.834443] BTRFS: device label SSDHOME devid 1 transid 44655875 
/dev/sdb3 (8:19) scanned by mount (8020)
[442011.845803] BTRFS info (device sdb3 state S): first mount of 
filesystem 13af326c-631f-482b-9c34-b59b4f100608
[442011.856089] BTRFS info (device sdb3 state S): using crc32c checksum 
algorithm
[442011.897204] BTRFS info (device sdb3 state ECS): enabling ssd 
optimizations
[442011.904566] BTRFS info (device sdb3 state ECS): disabling log replay 
at mount time
[442011.912626] BTRFS info (device sdb3 state ECS): turning on async discard
[442011.919811] BTRFS info (device sdb3 state ECS): enabling free space tree
[442011.927008] BTRFS info (device sdb3 state ECS): ignoring bad roots
[442011.933661] BTRFS info (device sdb3 state ECS): ignoring data csums
[442011.940357] BTRFS info (device sdb3 state ECS): ignoring meta csums
[442011.947051] BTRFS info (device sdb3 state ECS): ignoring unknown 
super block flags
# ls /var
@var
# btrfs fi resize max /var
ERROR: unable to start resize, another exclusive operation 'balance 
paused' in progress




Le 11/06/2026 à 23:19, Alban Browaeys a écrit :
> Here is the debug dump of this partition sdb3_var_metadata-only- 
> debug.img at https://drive.google.com/file/ 
> d/1OzEZLaTUNpWsL1ohGKOMW4GiR9oQakdP/view?usp=drive_link done with 
> "btrfs-image -c 9 -t 4 /dev/sdb3 /mnt/4/hermes-var-20260610-enospc- 
> vanilla/sdb3_var_metadata-only-debug.img".
> 
> I tried to btrfs-resotre this partition but I get a lot of errors:
> 
> btrfs restore -v -m -x -i /dev/sdb3 /mnt/4/hermes-var-20260610-enospc- 
> vanilla/restored_var/
> 
> Skipping existing file /mnt/4/hermes-var-20260610-enospc-vanilla/ 
> restored_var/@var/backups/group.bak
> If you wish to overwrite use -o
> ERROR: zstd frame incomplete
> ERROR: copying data for /mnt/4/hermes-var-20260610-enospc-vanilla/ 
> restored_var/@var/backups/dpkg.arch.5.gz failed
> ERROR: zstd frame incomplete
> (...)
> 
> Alban
> 
> Le 10/06/2026 à 23:05, Alban Browaeys a écrit :
>> Le 09/06/2026 à 17:25, Chris Murphy a écrit :
>>> Try:
>>>
>>> mount -o skip_balance,nospace_cache,enospc_debug /dev/sdb3 /var
>>> btrfs balance start -dlimit=1 /var
>>
>>
>> Thanks for the suggestions. Unfortunately, the bypass did not work.
>>
>> # mount -o skip_balance,nospace_cache,enospc_debug /dev/sdb3 /var
>> [117052.507168] BTRFS: device label SSDHOME devid 1 transid 44655875 / 
>> dev/sdb3 (8:19) scanned by mount (2004)
>> [117052.526475] BTRFS info (device sdb3): first mount of filesystem 
>> 13af326c-631f-482b-9c34-b59b4f100608
>> [117052.536146] BTRFS info (device sdb3): using crc32c checksum algorithm
>> [117052.543125] BTRFS error (device sdb3): cannot disable free-space-tree
>> [117052.550097] BTRFS error (device sdb3): open_ctree failed: -22
>> mount: /var: wrong fs type, bad option, bad superblock on /dev/sdb3, 
>> missing codepage or helper program, or other error.
>>        dmesg(1) may have more information after failed mount system call.
>>
>>
>> I believe this issue is blocking me:
>>
>> [117052.543125] BTRFS error (device sdb3): cannot disable free-space-tree
>>
>> So I tried with enospc_debug without nospace_cache, to get debug infos:
>>
>> # mount -o skip_balance,enospc_debug /dev/sdb3 /var
>> [117261.590047] BTRFS: device label SSDHOME devid 1 transid 44655875 / 
>> dev/sdb3 (8:19) scanned by mount (2008)
>> [117261.612863] BTRFS info (device sdb3): first mount of filesystem 
>> 13af326c-631f-482b-9c34-b59b4f100608
>> [117261.622548] BTRFS info (device sdb3): using crc32c checksum algorithm
>> [117261.662415] BTRFS info (device sdb3): left=0, need=393216, flags=36
>> [117261.669328] BTRFS info (device sdb3): space_info SYSTEM (sub-group 
>> id 0) has 0 free, is not full
>> [117261.678792] BTRFS info (device sdb3): space_info total=33554432, 
>> used=16384, pinned=0, reserved=0, may_use=0, readonly=33538048 
>> zone_unusable=0
>> [117261.692344] BTRFS info (device sdb3): global_block_rsv: size 
>> 190382080 reserved 190382080
>> [117261.701131] BTRFS info (device sdb3): trans_block_rsv: size 0 
>> reserved 0
>> [117261.708349] BTRFS info (device sdb3): chunk_block_rsv: size 0 
>> reserved 0
>> [117261.715571] BTRFS info (device sdb3): remap_block_rsv: size 0 
>> reserved 0
>> [117261.722782] BTRFS info (device sdb3): delayed_block_rsv: size 0 
>> reserved 0
>> [117261.730154] BTRFS info (device sdb3): delayed_refs_rsv: size 0 
>> reserved 0
>> [117261.737462] BTRFS error (device sdb3): allocation failed flags 36, 
>> wanted 16384 tree-log 0, relocation: 0
>> [117261.747497] BTRFS info (device sdb3): space_info METADATA (sub- 
>> group id 0) has -190382080 free, is full
>> [117261.757371] BTRFS info (device sdb3): space_info total=3221225472, 
>> used=2142470144, pinned=0, reserved=0, may_use=190382080, 
>> readonly=1078755328 zone_unusable=0
>> [117261.772172] BTRFS info (device sdb3): global_block_rsv: size 
>> 190382080 reserved 190365696
>> [117261.780844] BTRFS info (device sdb3): trans_block_rsv: size 0 
>> reserved 0
>> [117261.787995] BTRFS info (device sdb3): chunk_block_rsv: size 0 
>> reserved 0
>> [117261.795127] BTRFS info (device sdb3): remap_block_rsv: size 0 
>> reserved 0
>> [117261.802251] BTRFS info (device sdb3): delayed_block_rsv: size 0 
>> reserved 0
>> [117261.809551] BTRFS info (device sdb3): delayed_refs_rsv: size 0 
>> reserved 0
>> [117261.816785] BTRFS info (device sdb3): block group 55856594944 has 
>> 1073741824 bytes, 1007976448 used 0 pinned 0 reserved 0 delalloc 0 
>> super 0 zone_unusable (65765376
>>  bytes available) [readonly]
>> [117261.834327] BTRFS info (device sdb3): block group has cluster?: no
>> [117261.840947] BTRFS info (device sdb3): 0 free space entries at or 
>> bigger than 16384 bytes
>> [117261.849500] BTRFS info (device sdb3): block group 101368987648 has 
>> 1073741824 bytes, 500957184 used 0 pinned 0 reserved 0 delalloc 0 
>> super 0 zone_unusable (57278464
>> 0 bytes available) [readonly]
>> [117261.867163] BTRFS info (device sdb3): block group has cluster?: no
>> [117261.873784] BTRFS info (device sdb3): 0 free space entries at or 
>> bigger than 16384 bytes [117261.882313] BTRFS info (device sdb3): 
>> block group 102442729472 has 1073741824 bytes, 633536512 used 0 pinned 
>> 0 reserved 0 delalloc 0 super 0 zone_unusable (44020531
>> 2 bytes available) [readonly]
>> [117261.899994] BTRFS info (device sdb3): block group has cluster?: no
>> [117261.906630] BTRFS info (device sdb3): 0 free space entries at or 
>> bigger than 16384 bytes
>> [117261.915176] BTRFS info (device sdb3): 1078755328 bytes available 
>> across all block groups
>> [117261.923750] BTRFS warning (device sdb3): Skipping commit of 
>> aborted transaction.
>> [117261.931608] ------------[ cut here ]------------
>> [117261.936687] BTRFS: Transaction aborted (error -28)
>> [117261.941950] WARNING: fs/btrfs/transaction.c:2060 at 
>> btrfs_commit_transaction+0xaaa/0xe20 [btrfs], CPU#6: mount/2008
>> [117261.952905] Modules linked in: ntfs3 btrfs libblake2b raid10 
>> raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor 
>> raid6_pq raid1 raid0 md_mod auto
>> fs4 hid_logitech_hidpp hid_logitech_dj hid_generic usbhid hid sr_mod 
>> cdrom uas usb_storage i915 drm_buddy ttm i2c_algo_bit 
>> drm_display_helper cec sd_mod rc_core drm_cli
>> ent_lib drm_kms_helper iTCO_wdt xhci_pci ahci intel_pmc_bxt drm 
>> libahci watchdog xhci_hcd firewire_ohci libata firewire_core r8169 
>> video realtek crc_itu_t usbcore scsi_
>> mod fan i2c_i801 wmi button usb_common i2c_smbus scsi_common efivarfs
>> [117262.004021] CPU: 6 UID: 0 PID: 2008 Comm: mount Tainted: G 
>> W           7.0.10+deb14-amd64 #1 PREEMPT(lazy)  Debian 7.0.10-1
>> [117262.016343] Tainted: [W]=WARN
>> [117262.019853] Hardware name: LENOVO 10M8S14C42/3102, BIOS M16KT71A 
>> 09/14/2023
>> [117262.027348] RIP: 0010:btrfs_commit_transaction+0xaac/0xe20 [btrfs]
>> [117262.034117] Code: ff 48 8b 7c 24 08 31 c9 48 8d b7 48 01 00 00 e8 
>> 9a 7c 0b 00 48 c7 43 18 00 00 00 00 e9 73 fe ff ff 48 8d 3d 66 fd 34 
>> 00 89 ee <67> 48 0f b9 3a e9
>> f8 fe ff ff 49 8b bc 24 48 03 00 00 be 02 00 00
>> [117262.053426] RSP: 0018:ffffce9fc8b57418 EFLAGS: 00010292
>> [117262.059233] RAX: 0000000000000002 RBX: ffff8ac4d181a738 RCX: 
>> 0000000000000003
>> [117262.066946] RDX: 0000000002040001 RSI: 00000000ffffffe4 RDI: 
>> ffffffffc1028080
>> [117262.074658] RBP: 00000000ffffffe4 R08: 0000000000000000 R09: 
>> ffffce9fc8b57200
>> [117262.082364] R10: ffff8acbe23fff90 R11: 00000000fff7ffff R12: 
>> ffff8ac545885000
>> [117262.090075] R13: ffff8ac4c242ea00 R14: ffff8ac5458854c0 R15: 
>> ffff8ac4cff01b80
>> [117262.097779] FS:  00007f22c28cf940(0000) GS:ffff8acc3a4fc000(0000) 
>> knlGS:0000000000000000
>> [117262.106429] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [117262.112740] CR2: 00007f22c2cdfdd0 CR3: 00000001867ac001 CR4: 
>> 00000000003706f0
>> [117262.120434] Call Trace:
>> [117262.123451]  <TASK>
>> [117262.126121]  ? start_transaction+0x228/0x880 [btrfs]
>> [117262.131676]  flush_space+0x3c3/0x5a0 [btrfs]
>> [117262.136574]  ? btrfs_reduce_alloc_profile+0x5a/0x1d0 [btrfs]
>> [117262.142855]  ? calc_available_free_space.isra.0+0x70/0xc0 [btrfs]
>> [117262.149597]  priority_reclaim_metadata_space+0x96/0x130 [btrfs]
>> [117262.156132]  handle_reserve_ticket+0x5b/0x290 [btrfs]
>> [117262.161800]  ? btrfs_reduce_alloc_profile+0x5a/0x1d0 [btrfs]
>> [117262.168076]  reserve_bytes+0x268/0x4c0 [btrfs] [117262.173127] 
>> btrfs_reserve_metadata_bytes+0x1a/0xe0 [btrfs]
>> [117262.179306]  btrfs_block_rsv_refill+0x60/0x90 [btrfs]
>> [117262.184965]  evict_refill_and_join+0x47/0xe0 [btrfs]
>> [117262.190531]  btrfs_evict_inode+0x332/0x400 [btrfs]
>> [117262.195922]  evict+0xfd/0x270
>> [117262.199441]  btrfs_orphan_cleanup+0x21f/0x390 [btrfs]
>> [117262.205085]  btrfs_start_pre_rw_mount+0x2a0/0x6c0 [btrfs]
>> [117262.211071]  open_ctree+0x12fb/0x1730 [btrfs]
>> [117262.216028]  btrfs_get_tree.cold+0xb/0xb6 [btrfs]
>> [117262.221324]  ? security_inode_permission+0x4a/0x110
>> [117262.226747]  vfs_get_tree+0x29/0xd0
>> [117262.230776]  vfs_cmd_create+0x57/0xd0
>> [117262.234947]  __do_sys_fsconfig+0x2fc/0x5e0
>> [117262.239581]  do_syscall_64+0xcd/0x15f0
>> [117262.243873]  ? __fs_parse+0x68/0x1c0
>> [117262.247989]  ? aa_file_perm+0x18a/0x550
>> [117262.252368]  ? btrfs_parse_param+0x4f/0x960 [btrfs]
>> [117262.257818]  ? apparmor_file_permission+0x44/0xb0
>> [117262.263052]  ? fscontext_read+0x171/0x190
>> [117262.267594]  ? rw_verify_area+0x56/0x180
>> [117262.272039]  ? vfs_read+0xbf/0x390
>> [117262.275964]  ? __fs_parse+0x68/0x1c0
>> [117262.280063]  ? aa_file_perm+0x18a/0x550
>> [117262.284422]  ? btrfs_parse_param+0x4f/0x960 [btrfs]
>> [117262.289858]  ? apparmor_file_permission+0x44/0xb0
>> [117262.295082]  ? fscontext_read+0x171/0x190
>> [117262.299605]  ? rw_verify_area+0x56/0x180
>> [117262.304036]  ? vfs_read+0xbf/0x390
>> [117262.307943]  ? do_syscall_64+0x10a/0x15f0
>> [117262.312451]  ? from_kgid_munged+0x17/0x30
>> [117262.316955]  ? __do_sys_getgid+0x27/0x30
>> [117262.321369]  ? do_syscall_64+0x10a/0x15f0
>> [117262.325856]  ? from_kgid_munged+0x17/0x30
>> [117262.330340]  ? ksys_read+0x73/0xf0
>> [117262.334213]  ? do_syscall_64+0x10a/0x15f0
>> [117262.338694]  ? irqentry_exit+0x78/0x680
>> [117262.343000]  ? exc_page_fault+0x82/0x1d0
>> [117262.347384]  ? __irq_exit_rcu+0x3d/0xe0 [117262.351683] 
>> entry_SYSCALL_64_after_hwframe+0x76/0x7e
>> [117262.357197] RIP: 0033:0x7f22c2d0eb4a
>> [117262.361218] Code: 73 01 c3 48 8b 0d c6 32 0d 00 f7 d8 64 89 01 48 
>> 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 49 89 ca b8 af 01 00 
>> 00 0f 05 <48> 3d 01 f0 ff ff
>> 73 01 c3 48 8b 0d 96 32 0d 00 f7 d8 64 89 01 48
>> [117262.380431] RSP: 002b:00007ffd5d547f68 EFLAGS: 00000246 ORIG_RAX: 
>> 00000000000001af
>> [117262.388464] RAX: ffffffffffffffda RBX: 0000557471e4da50 RCX: 
>> 00007f22c2d0eb4a
>> [117262.396057] RDX: 0000000000000000 RSI: 0000000000000006 RDI: 
>> 0000000000000003
>> [117262.403650] RBP: 0000000000000003 R08: 0000000000000000 R09: 
>> 0000000000000000
>> [117262.411240] R10: 0000000000000000 R11: 0000000000000246 R12: 
>> 0000557471e4ec40
>> [117262.418831] R13: 00007f22c2ea127c R14: 00007f22c2e9f460 R15: 
>> 00007f22c28cf7f0
>> [117262.426410]  </TASK>
>> [117262.429024] ---[ end trace 0000000000000000 ]---
>> [117262.434074] BTRFS info (device sdb3 state A): dumping space info:
>> [117262.440595] BTRFS info (device sdb3 state A): space_info DATA 
>> (sub-group id 0) has 17422925824 free, is not full
>> [117262.451184] BTRFS info (device sdb3 state A): space_info 
>> total=70262980608, used=52839989248, pinned=0, reserved=0, may_use=0, 
>> readonly=65536 zone_unusable=0
>> [117262.465659] BTRFS info (device sdb3 state A): space_info METADATA 
>> (sub-group id 0) has -190382080 free, is full
>> [117262.476171] BTRFS info (device sdb3 state A): space_info 
>> total=3221225472, used=2142470144, pinned=0, reserved=0, 
>> may_use=190382080, readonly=1078755328 zone_unusable=0
>> [117262.491598] BTRFS info (device sdb3 state A): space_info SYSTEM 
>> (sub-group id 0) has 0 free, is not full
>> [117262.501477] BTRFS info (device sdb3 state A): space_info 
>> total=33554432, used=16384, pinned=0, reserved=0, may_use=0, 
>> readonly=33538048 zone_unusable=0
>> [117262.515439] BTRFS info (device sdb3 state A): global_block_rsv: 
>> size 190382080 reserved 190382080
>> [117262.524731] BTRFS info (device sdb3 state A): trans_block_rsv: 
>> size 0 reserved 0
>> [117262.532538] BTRFS info (device sdb3 state A): chunk_block_rsv: 
>> size 0 reserved 0
>> [117262.540337] BTRFS info (device sdb3 state A): remap_block_rsv: 
>> size 0 reserved 0
>> [117262.548136] BTRFS info (device sdb3 state A): delayed_block_rsv: 
>> size 0 reserved 0
>> [117262.556110] BTRFS info (device sdb3 state A): delayed_refs_rsv: 
>> size 0 reserved 0
>> [117262.563998] BTRFS: error (device sdb3 state A) in 
>> cleanup_transaction:2060: errno=-28 No space left
>> [117262.573453] BTRFS info (device sdb3 state EA): space_info METADATA 
>> (sub-group id 0) has -190382080 free, is full
>> [117262.584033] BTRFS info (device sdb3 state EA): space_info 
>> total=3221225472, used=2142470144, pinned=0, reserved=0, 
>> may_use=190382080, readonly=1078755328 zone_unusable=0
>> [117262.599563] BTRFS info (device sdb3 state EA): global_block_rsv: 
>> size 190382080 reserved 190382080
>> [117262.608943] BTRFS info (device sdb3 state EA): trans_block_rsv: 
>> size 0 reserved 0
>> [117262.616846] BTRFS info (device sdb3 state EA): chunk_block_rsv: 
>> size 0 reserved 0
>> [117262.624749] BTRFS info (device sdb3 state EA): remap_block_rsv: 
>> size 0 reserved 0
>> [117262.632643] BTRFS info (device sdb3 state EA): delayed_block_rsv: 
>> size 0 reserved 0
>> [117262.640715] BTRFS info (device sdb3 state EA): delayed_refs_rsv: 
>> size 0 reserved 0
>> [117262.648696] BTRFS info (device sdb3 state EA): space_info METADATA 
>> (sub-group id 0) has -190382080 free, is full
>> [117262.659312] BTRFS info (device sdb3 state EA): space_info 
>> total=3221225472, used=2142470144, pinned=0, reserved=0, 
>> may_use=190382080, readonly=1078755328 zone_unusable=0
>> [117262.674875] BTRFS info (device sdb3 state EA): global_block_rsv: 
>> size 190382080 reserved 190382080
>> [117262.684287] BTRFS info (device sdb3 state EA): trans_block_rsv: 
>> size 0 reserved 0
>> [117262.692225] BTRFS info (device sdb3 state EA): chunk_block_rsv: 
>> size 0 reserved 0
>> [117262.700155] BTRFS info (device sdb3 state EA): remap_block_rsv: 
>> size 0 reserved 0
>> [117262.708086] BTRFS info (device sdb3 state EA): delayed_block_rsv: 
>> size 0 reserved 0
>> [117262.716191] BTRFS info (device sdb3 state EA): delayed_refs_rsv: 
>> size 0 reserved 0
>> [117262.724260] BTRFS warning (device sdb3 state EA): could not 
>> allocate space for delete; will truncate on mount
>> [117262.734622] BTRFS error (device sdb3 state EA): Error removing 
>> orphan entry, stopping orphan cleanup
>> [117262.744220] BTRFS error (device sdb3 state EA): could not do 
>> orphan cleanup -28
>> [117262.752021] BTRFS error (device sdb3 state EA): commit super block 
>> returned -30
>> [117262.760739] BTRFS error (device sdb3 state EA): open_ctree failed: 
>> -28
>> mount: /var: fsconfig() failed: No space left on device.
>>        dmesg(1) may have more information after failed mount system call.
>>
>>
>>
>>
>>> If the first command works but the second command does not, you could 
>>> insert a `btrfs device add` command to add a partition. I'm not such 
>>> a big fan of adding USB sticks or zram devices because if anything 
>>> goes wrong, the file system ends up broken.
>>>
>>> This is a bit of a heavy hammer to fix this problem but it might 
>>> actually be faster than troubleshooting Btrfs enospc:
>>
>> I am all for troubleshooting this enospc issue because it is the first 
>> time I cannot even add a device to fix the enospc... in fact an hour 
>> before this issue I had an enospc issue this time with the root 
>> partition that had very few space left, and adding a device worked 
>> well and it was pretty easy to fix. But here I have gigabytes of 
>> unused data and metadata, I am puzzled.
>>
>> I was switching my SSD btrfs partitions (I plan to merge them all but 
>> for historical reason they are split as separate btrfs partitions 
>> instead of subvolumes of a single btrfs partition) from single to DUP, 
>> the new default.
>>
>>
>>
>>  I believe the main issue here is that the conversion attempted to use 
>> until the last megabyte of metadata before errorring out, preventing 
>> any simple rescue operation afterwards, and even preventing from 
>> balancing data to free chunks for metadata.
>>
>> It would have been fine for the conversion to error out and let me 
>> balance data then resume the conversion. The main issue is that the 
>> conversion used so much metadata space that I cannot even mount the 
>> btrfs partition to balance its data space.
>>
>> Maybe the metadata conversion should guard/ straight error out when 
>> there is not enough space in metadata (here 3GB to duplicate 1,59GB 
>> does not fit) and there is no way to get a new free chunk for metadata 
>> and tell to free chunks first ?
>>
>>
>>> It should still be possible to make the troubled Btrfs a read-only 
>>> seed device using btrfs-tune; then mount it ro; use `btrfs dev add` 
>>> using a device that's equal to or bigger; remount rw; then `btrfs dev 
>>> rem` the 1st device, i.e. the ro seed. This will replicate/clone the 
>>> file system from device 1 to device 2. And it'll take a while. But in 
>>> the end you'll have a balanced file system on device 2.
>>>
>>> It isn't strictly necessary for device 2 to be equal to or bigger. It 
>>> is possible to add a smaller device, but before you remove the 1st 
>>> device, you wlil need to delete files. The bigger files you choose to 
>>> delete, the fewer you will need to delete to get to the 
>>> underprovisioned 2nd device size. Then remove the 1st device. Now 
>>> only the files that have not been deleted will have extents 
>>> replicated to the 2nd device.
>>>
>>> At the very least the seed-device is not being modified and shouldn't 
>>> get any worse than it is.
>>> Chris Murphy


So with I tried with what I understood:
with a copy of the image I did of the partition: 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
# btrfstune -S 1 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
ERROR: please finish/cancel the running replace/balance before running 
this command

I tried:
# btrfs check --clear-space-cache v2 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
Opening filesystem to check...
Checking filesystem on 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
UUID: 13af326c-631f-482b-9c34-b59b4f100608
WARNING: --clear-space-cache option is deprecated, please use "btrfs 
rescue clear-space-cache" instead
ERROR: please finish/cancel the running replace/balance before running 
this command

# btrfs balance cancel 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
ERROR: not a directory: 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img

# btrfs rescue clear-uuid-tree 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
ERROR: please finish/cancel the running replace/balance before running 
this command



and without the seed flag:
# dd if=/dev/zero 
of=/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var_target.img 
bs=1M count=90000
90000+0 records in
90000+0 records out
94371840000 bytes (94 GB, 88 GiB) copied, 1340.52 s, 70.4 MB/s


# losetup -f --show 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var.img
[475088.702700] loop0: detected capacity change from 0 to 166119424
/dev/loop0

# mkdir -p /mnt/seed_workspace

# mount -o ro,rescue=all /dev/loop0 /mnt/seed_workspace
[475287.536430] BTRFS info: device /dev/loop0 (7:0) using temp-fsid 
7d3aa685-b9cd-4cbc-87ed-c2d0288fdeba
[475287.546680] BTRFS: device label SSDHOME devid 1 transid 44655875 
/dev/loop0 (7:0) scanned by mount (8576)
[475287.556933] BTRFS info (device loop0 state S): first mount of 
filesystem 13af326c-631f-482b-9c34-b59b4f100608
[475287.567325] BTRFS info (device loop0 state S): using crc32c checksum 
algorithm
[475288.554301] BTRFS info (device loop0 state ECS): enabling ssd 
optimizations
[475288.562226] BTRFS info (device loop0 state ECS): disabling log 
replay at mount time
[475288.570394] BTRFS info (device loop0 state ECS): turning on async 
discard
[475288.577648] BTRFS info (device loop0 state ECS): enabling free space 
tree
[475288.584915] BTRFS info (device loop0 state ECS): ignoring bad roots
[475288.591652] BTRFS info (device loop0 state ECS): ignoring data csums
[475288.598482] BTRFS info (device loop0 state ECS): ignoring meta csums
[475288.605308] BTRFS info (device loop0 state ECS): ignoring unknown 
super block flags


# losetup -f --show 
/mnt/4/hermes-var-20260610-enospc-vanilla/tests/sdb3_var_target.img
[475310.285452] loop1: detected capacity change from 0 to 184320000
/dev/loop1


# btrfs device add /dev/loop1 /mnt/seed_workspace
Performing full device TRIM /dev/loop1 (87.89GiB) ...
ERROR: error adding device '/dev/loop1': Read-only file system



>>
>> I don't know yet how to proceed with such a read-only seed option, I 
>> will look into it. But is there other flags or tune I could do that 
>> could let me cancel the pause operation, the balance the data space, 
>> then retry the conversion ?
>>
>>
>> Alban
>>

Alban