* kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly @ 2025-12-11 1:28 Chris Murphy 2025-12-11 1:55 ` Qu Wenruo 0 siblings, 1 reply; 12+ messages in thread From: Chris Murphy @ 2025-12-11 1:28 UTC (permalink / raw) To: Btrfs BTRFS User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. Downstream bug is here, which has the complete dmesg attached: https://bugzilla.redhat.com/show_bug.cgi?id=2421293 $ sudo btrfs check -p --readonly /dev/sda3 Opening filesystem to check... Checking filesystem on /dev/sda3 UUID: afdbb979-0b91-499b-976c-0244ba2ed38f [1/8] checking log skipped (none written) [1/7] checking root items (0:00:00 elapsed, 659535 items checked) [2/7] checking extents (0:00:03 elapsed, 59194 items checked) We have a space info key for a block group that doesn't existed) [3/7] checking free space tree (0:00:00 elapsed, 172 items checked) [4/7] chunresolved ref dir 1924 index 134016 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 4, no inode ref [4/7] checking fs roots (0:00:02 elapsed, 45654 items checked) ERROR: errors found in fs roots found 140957597696 bytes used, error(s) found total csum bytes: 136358540 total tree bytes: 969637888 total fs tree bytes: 750862336 total extent tree bytes: 62783488 btree space waste bytes: 178909301 file data blocks allocated: 488208621568 referenced 162958045184 dmesg excerpt [ 126.803576] BTRFS critical (device sda3): failed to delete reference to AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E, root 256 inode 730455 parent 1924 [ 126.803583] ------------[ cut here ]------------ [ 126.803584] BTRFS: Transaction aborted (error -2) [ 126.803590] WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode+0x416/0x440 [ 126.803596] Modules linked in: vfat fat exfat uinput rfcomm snd_seq_dummy snd_hrtimer nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables qrtr bnep iwlmvm mac80211 libarc4 snd_hda_codec_alc882 snd_hda_codec_realtek_lib snd_usb_audio snd_hda_codec_generic snd_hda_codec_atihdmi btusb iwlwifi snd_hda_codec_hdmi btmtk snd_hda_intel btrtl amd_atl snd_hda_codec btbcm intel_rapl_msr intel_rapl_common btintel snd_usbmidi_lib snd_hda_core mc cfg80211 snd_intel_dspcfg snd_ump snd_intel_sdw_acpi bluetooth snd_rawmidi snd_hwdep edac_mce_amd snd_seq snd_seq_device kvm_amd eeepc_wmi snd_pcm asus_wmi snd_timer snd ee1004 sparse_keymap kvm platform_profile wmi_bmof joydev soundcore rfkill i2c_piix4 gpio_amdpt irqbypass k10temp i2c_smbus pcspkr gpio_generic rapl tun zram lz4hc_compress lz4_compress overlay erofs netfs isofs amdgpu hid_logitech_hidpp amdxcp [ 126.803693] drm_panel_backlight_quirks gpu_sched drm_buddy drm_ttm_helper ttm video drm_exec i2c_algo_bit drm_suballoc_helper drm_display_helper r8169 uas polyval_clmulni cec ghash_clmulni_intel usb_storage realtek sp5100_tco nvme nvme_tcp nvme_fabrics wmi hid_logitech_dj nvme_core nvme_keyring nvme_auth hkdf sunrpc be2iscsi bnx2i cnic uio cxgb4i cxgb4 tls cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi loop fuse i2c_dev nfnetlink [ 126.803747] CPU: 5 UID: 1000 PID: 7181 Comm: rm Not tainted 6.18.0-65.fc44.x86_64 #1 PREEMPT(lazy) [ 126.803750] Hardware name: MM-Vision Computer/TUF B450M-PLUS GAMING, BIOS 4604 03/22/2024 [ 126.803752] RIP: 0010:__btrfs_unlink_inode+0x416/0x440 [ 126.803755] Code: bc 89 04 24 e8 5b 61 9e ff 0f 0b 8b 04 24 41 b8 01 00 00 00 e9 b7 d8 86 ff 89 c6 48 c7 c7 28 49 a4 bc 89 04 24 e8 3a 61 9e ff <0f> 0b 8b 04 24 41 b8 01 00 00 00 e9 6e d8 86 ff b8 f4 ff ff ff e9 [ 126.803757] RSP: 0018:ffffd1e8926c7b48 EFLAGS: 00010246 [ 126.803760] RAX: 0000000000000000 RBX: ffff8f351fbdfc00 RCX: 0000000000000027 [ 126.803762] RDX: ffff8f36dec9cfc8 RSI: 0000000000000001 RDI: ffff8f36dec9cfc0 [ 126.803764] RBP: ffff8f34c79b4400 R08: 0000000000000000 R09: ffffd1e8926c79f0 [ 126.803765] R10: ffffffffbd53c268 R11: 00000000ffffdfff R12: ffff8f34fbb818c0 [ 126.803767] R13: ffff8f34c7aaa1f8 R14: ffff8f33c564f800 R15: ffffd1e8926c7bc0 [ 126.803769] FS: 00007ff6c7bc7740(0000) GS:ffff8f37208aa000(0000) knlGS:0000000000000000 [ 126.803771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 126.803773] CR2: 000055cce3131618 CR3: 00000001ef110000 CR4: 0000000000f50ef0 [ 126.803775] PKRU: 55555554 [ 126.803777] Call Trace: [ 126.803779] <TASK> [ 126.803785] btrfs_unlink+0xd9/0x150 [ 126.803790] vfs_unlink+0x117/0x2a0 [ 126.803795] do_unlinkat+0x268/0x2f0 [ 126.803801] __x64_sys_unlinkat+0x61/0xd0 [ 126.803803] do_syscall_64+0x7e/0x7f0 [ 126.803807] ? charge_memcg+0x48/0x80 [ 126.803811] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803814] ? blk_cgroup_congested+0x65/0x70 [ 126.803818] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803821] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803823] ? __lruvec_stat_mod_folio+0x85/0xd0 [ 126.803827] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803829] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803832] ? set_ptes.isra.0+0x36/0x80 [ 126.803835] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803838] ? do_anonymous_page+0x100/0x490 [ 126.803842] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803844] ? __handle_mm_fault+0x551/0x6a0 [ 126.803849] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803852] ? count_memcg_events+0xd6/0x220 [ 126.803856] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803858] ? handle_mm_fault+0x248/0x360 [ 126.803861] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803864] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803866] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803869] ? do_syscall_64+0xb6/0x7f0 [ 126.803871] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803874] ? do_user_addr_fault+0x21a/0x690 [ 126.803878] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803881] ? srso_alias_return_thunk+0x5/0xfbef5 [ 126.803883] ? irqentry_exit_to_user_mode+0x2c/0x1c0 [ 126.803886] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ 126.803889] RIP: 0033:0x7ff6c7cb314b [ 126.803908] Code: 77 05 c3 0f 1f 40 00 48 8b 15 a9 fc 0f 00 f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 07 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 7d fc 0f 00 f7 d8 64 89 01 48 [ 126.803910] RSP: 002b:00007ffdb498d188 EFLAGS: 00000206 ORIG_RAX: 0000000000000107 [ 126.803913] RAX: ffffffffffffffda RBX: 000055cce3131730 RCX: 00007ff6c7cb314b [ 126.803914] RDX: 0000000000000000 RSI: 000055cce3130510 RDI: 00000000ffffff9c [ 126.803916] RBP: 00007ffdb498d260 R08: 0000000000000002 R09: 00007ffdb498d2bc [ 126.803918] R10: 0000000000000000 R11: 0000000000000206 R12: 000055cce3130480 [ 126.803919] R13: 0000000000000000 R14: 00007ffdb498d2c0 R15: 0000000000000000 [ 126.803925] </TASK> [ 126.803926] ---[ end trace 0000000000000000 ]--- [ 126.803956] BTRFS: error (device sda3 state A) in __btrfs_unlink_inode:4297: errno=-2 No such entry [ 126.803963] BTRFS info (device sda3 state EA): forced readonly -- Chris Murphy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-11 1:28 kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly Chris Murphy @ 2025-12-11 1:55 ` Qu Wenruo 2025-12-11 18:01 ` Chris Murphy 0 siblings, 1 reply; 12+ messages in thread From: Qu Wenruo @ 2025-12-11 1:55 UTC (permalink / raw) To: Chris Murphy, Btrfs BTRFS 在 2025/12/11 11:58, Chris Murphy 写道: > User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. > > Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. > > User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. This looks like a previous memory corruption caused on-disk metadata corruption. Tree-checker is not a memtest tool, it can only detect very obvious problems, it can not do cross-reference, and unfortunately this is exact cross-reference case. For this particular one, I'd recommend to do a "btrfs check --repair" then "btrfs check" to verify. At least it should not make the situation worse, and I believe the chance of repair is pretty high. > > Downstream bug is here, which has the complete dmesg attached: > https://bugzilla.redhat.com/show_bug.cgi?id=2421293 > > $ sudo btrfs check -p --readonly /dev/sda3 > Opening filesystem to check... > Checking filesystem on /dev/sda3 > UUID: afdbb979-0b91-499b-976c-0244ba2ed38f > [1/8] checking log skipped (none written) > [1/7] checking root items (0:00:00 elapsed, 659535 items checked) > [2/7] checking extents (0:00:03 elapsed, 59194 items checked) > We have a space info key for a block group that doesn't existed) This one is safe to ignore. Thanks, Qu > [3/7] checking free space tree (0:00:00 elapsed, 172 items checked) > [4/7] chunresolved ref dir 1924 index 134016 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 4, no inode ref > [4/7] checking fs roots (0:00:02 elapsed, 45654 items checked) > ERROR: errors found in fs roots > found 140957597696 bytes used, error(s) found > total csum bytes: 136358540 > total tree bytes: 969637888 > total fs tree bytes: 750862336 > total extent tree bytes: 62783488 > btree space waste bytes: 178909301 > file data blocks allocated: 488208621568 > referenced 162958045184 > > > dmesg excerpt > > [ 126.803576] BTRFS critical (device sda3): failed to delete reference to AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E, root 256 inode 730455 parent 1924 > [ 126.803583] ------------[ cut here ]------------ > [ 126.803584] BTRFS: Transaction aborted (error -2) > [ 126.803590] WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode+0x416/0x440 > [ 126.803596] Modules linked in: vfat fat exfat uinput rfcomm snd_seq_dummy snd_hrtimer nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables qrtr bnep iwlmvm mac80211 libarc4 snd_hda_codec_alc882 snd_hda_codec_realtek_lib snd_usb_audio snd_hda_codec_generic snd_hda_codec_atihdmi btusb iwlwifi snd_hda_codec_hdmi btmtk snd_hda_intel btrtl amd_atl snd_hda_codec btbcm intel_rapl_msr intel_rapl_common btintel snd_usbmidi_lib snd_hda_core mc cfg80211 snd_intel_dspcfg snd_ump snd_intel_sdw_acpi bluetooth snd_rawmidi snd_hwdep edac_mce_amd snd_seq snd_seq_device kvm_amd eeepc_wmi snd_pcm asus_wmi snd_timer snd ee1004 sparse_keymap kvm platform_profile wmi_bmof joydev soundcore rfkill i2c_piix4 gpio_amdpt irqbypass k10temp i2c_smbus pcspkr gpio_generic rapl tun zram lz4hc_compress lz4_compress overlay erofs netfs isofs amdgpu hid_logitech_hidpp amdxcp > [ 126.803693] drm_panel_backlight_quirks gpu_sched drm_buddy drm_ttm_helper ttm video drm_exec i2c_algo_bit drm_suballoc_helper drm_display_helper r8169 uas polyval_clmulni cec ghash_clmulni_intel usb_storage realtek sp5100_tco nvme nvme_tcp nvme_fabrics wmi hid_logitech_dj nvme_core nvme_keyring nvme_auth hkdf sunrpc be2iscsi bnx2i cnic uio cxgb4i cxgb4 tls cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi loop fuse i2c_dev nfnetlink > [ 126.803747] CPU: 5 UID: 1000 PID: 7181 Comm: rm Not tainted 6.18.0-65.fc44.x86_64 #1 PREEMPT(lazy) > [ 126.803750] Hardware name: MM-Vision Computer/TUF B450M-PLUS GAMING, BIOS 4604 03/22/2024 > [ 126.803752] RIP: 0010:__btrfs_unlink_inode+0x416/0x440 > [ 126.803755] Code: bc 89 04 24 e8 5b 61 9e ff 0f 0b 8b 04 24 41 b8 01 00 00 00 e9 b7 d8 86 ff 89 c6 48 c7 c7 28 49 a4 bc 89 04 24 e8 3a 61 9e ff <0f> 0b 8b 04 24 41 b8 01 00 00 00 e9 6e d8 86 ff b8 f4 ff ff ff e9 > [ 126.803757] RSP: 0018:ffffd1e8926c7b48 EFLAGS: 00010246 > [ 126.803760] RAX: 0000000000000000 RBX: ffff8f351fbdfc00 RCX: 0000000000000027 > [ 126.803762] RDX: ffff8f36dec9cfc8 RSI: 0000000000000001 RDI: ffff8f36dec9cfc0 > [ 126.803764] RBP: ffff8f34c79b4400 R08: 0000000000000000 R09: ffffd1e8926c79f0 > [ 126.803765] R10: ffffffffbd53c268 R11: 00000000ffffdfff R12: ffff8f34fbb818c0 > [ 126.803767] R13: ffff8f34c7aaa1f8 R14: ffff8f33c564f800 R15: ffffd1e8926c7bc0 > [ 126.803769] FS: 00007ff6c7bc7740(0000) GS:ffff8f37208aa000(0000) knlGS:0000000000000000 > [ 126.803771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 126.803773] CR2: 000055cce3131618 CR3: 00000001ef110000 CR4: 0000000000f50ef0 > [ 126.803775] PKRU: 55555554 > [ 126.803777] Call Trace: > [ 126.803779] <TASK> > [ 126.803785] btrfs_unlink+0xd9/0x150 > [ 126.803790] vfs_unlink+0x117/0x2a0 > [ 126.803795] do_unlinkat+0x268/0x2f0 > [ 126.803801] __x64_sys_unlinkat+0x61/0xd0 > [ 126.803803] do_syscall_64+0x7e/0x7f0 > [ 126.803807] ? charge_memcg+0x48/0x80 > [ 126.803811] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803814] ? blk_cgroup_congested+0x65/0x70 > [ 126.803818] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803821] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803823] ? __lruvec_stat_mod_folio+0x85/0xd0 > [ 126.803827] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803829] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803832] ? set_ptes.isra.0+0x36/0x80 > [ 126.803835] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803838] ? do_anonymous_page+0x100/0x490 > [ 126.803842] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803844] ? __handle_mm_fault+0x551/0x6a0 > [ 126.803849] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803852] ? count_memcg_events+0xd6/0x220 > [ 126.803856] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803858] ? handle_mm_fault+0x248/0x360 > [ 126.803861] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803864] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803866] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803869] ? do_syscall_64+0xb6/0x7f0 > [ 126.803871] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803874] ? do_user_addr_fault+0x21a/0x690 > [ 126.803878] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803881] ? srso_alias_return_thunk+0x5/0xfbef5 > [ 126.803883] ? irqentry_exit_to_user_mode+0x2c/0x1c0 > [ 126.803886] entry_SYSCALL_64_after_hwframe+0x76/0x7e > [ 126.803889] RIP: 0033:0x7ff6c7cb314b > [ 126.803908] Code: 77 05 c3 0f 1f 40 00 48 8b 15 a9 fc 0f 00 f7 d8 64 89 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 07 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 7d fc 0f 00 f7 d8 64 89 01 48 > [ 126.803910] RSP: 002b:00007ffdb498d188 EFLAGS: 00000206 ORIG_RAX: 0000000000000107 > [ 126.803913] RAX: ffffffffffffffda RBX: 000055cce3131730 RCX: 00007ff6c7cb314b > [ 126.803914] RDX: 0000000000000000 RSI: 000055cce3130510 RDI: 00000000ffffff9c > [ 126.803916] RBP: 00007ffdb498d260 R08: 0000000000000002 R09: 00007ffdb498d2bc > [ 126.803918] R10: 0000000000000000 R11: 0000000000000206 R12: 000055cce3130480 > [ 126.803919] R13: 0000000000000000 R14: 00007ffdb498d2c0 R15: 0000000000000000 > [ 126.803925] </TASK> > [ 126.803926] ---[ end trace 0000000000000000 ]--- > [ 126.803956] BTRFS: error (device sda3 state A) in __btrfs_unlink_inode:4297: errno=-2 No such entry > [ 126.803963] BTRFS info (device sda3 state EA): forced readonly > > > > -- > Chris Murphy > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-11 1:55 ` Qu Wenruo @ 2025-12-11 18:01 ` Chris Murphy 2025-12-11 18:52 ` Chris Murphy 2025-12-15 9:02 ` Qu Wenruo 0 siblings, 2 replies; 12+ messages in thread From: Chris Murphy @ 2025-12-11 18:01 UTC (permalink / raw) To: Qu WenRuo, Btrfs BTRFS On Wed, Dec 10, 2025, at 6:55 PM, Qu Wenruo wrote: > 在 2025/12/11 11:58, Chris Murphy 写道: >> User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. >> >> Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. >> >> User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. > > This looks like a previous memory corruption caused on-disk metadata > corruption. > > Tree-checker is not a memtest tool, it can only detect very obvious > problems, it can not do cross-reference, and unfortunately this is exact > cross-reference case. > > For this particular one, I'd recommend to do a "btrfs check --repair" > then "btrfs check" to verify. Looks like --repair changed from "errors 4" to "errors 6" [1/8] checking log [2/8] checking root items [3/8] checking extents [4/8] checking free space tree [5/8] checking fs roots unresolved ref dir 1924 index 0 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir index, no inode ref ERROR: errors found in fs roots Opening filesystem to check... Checking filesystem on /dev/sda3 UUID: afdbb979-0b91-499b-976c-0244ba2ed38f found 140964491264 bytes used, error(s) found total csum bytes: 136339904 total tree bytes: 969474048 total fs tree bytes: 751108096 total extent tree bytes: 62439424 btree space waste bytes: 178561905 file data blocks allocated: 490371264512 referenced 162984435712 End user has btrfs-image before and after repair if that's useful. But additional --repair attempts do not appear to fix the "errors 6" message - and the kernel still produces a warning and goes read-only if the file is accessed. -- Chris Murphy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-11 18:01 ` Chris Murphy @ 2025-12-11 18:52 ` Chris Murphy 2025-12-11 21:07 ` Chris Murphy 2025-12-15 9:02 ` Qu Wenruo 1 sibling, 1 reply; 12+ messages in thread From: Chris Murphy @ 2025-12-11 18:52 UTC (permalink / raw) To: Qu WenRuo, Btrfs BTRFS On Thu, Dec 11, 2025, at 11:01 AM, Chris Murphy wrote: > On Wed, Dec 10, 2025, at 6:55 PM, Qu Wenruo wrote: >> 在 2025/12/11 11:58, Chris Murphy 写道: >>> User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. >>> >>> Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. >>> >>> User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. >> >> This looks like a previous memory corruption caused on-disk metadata >> corruption. >> >> Tree-checker is not a memtest tool, it can only detect very obvious >> problems, it can not do cross-reference, and unfortunately this is exact >> cross-reference case. >> >> For this particular one, I'd recommend to do a "btrfs check --repair" >> then "btrfs check" to verify. > > Looks like --repair changed from "errors 4" to "errors 6" > > > [1/8] checking log > [2/8] checking root items > [3/8] checking extents > [4/8] checking free space tree > [5/8] checking fs roots > unresolved ref dir 1924 index 0 namelen 40 name > AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir > index, no inode ref > ERROR: errors found in fs roots > Opening filesystem to check... > Checking filesystem on /dev/sda3 > UUID: afdbb979-0b91-499b-976c-0244ba2ed38f > found 140964491264 bytes used, error(s) found > total csum bytes: 136339904 > total tree bytes: 969474048 > total fs tree bytes: 751108096 > total extent tree bytes: 62439424 > btree space waste bytes: 178561905 > file data blocks allocated: 490371264512 > referenced 162984435712 > > End user has btrfs-image before and after repair if that's useful. But > additional --repair attempts do not appear to fix the "errors 6" > message - and the kernel still produces a warning and goes read-only if > the file is accessed. Looks like inode 730455 is found in more than one leaf. I'm kinda wondering if there is a way to compel btrfs to delete the affected leafs, thus fixing the file system. Backups are done so none of the files in any of the leafs need saving. So if I were to have the user delete all the files in all the suspect leaves - is it possible Btrfs would just drop the leaf without hitting the splat? -- Chris Murphy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-11 18:52 ` Chris Murphy @ 2025-12-11 21:07 ` Chris Murphy 0 siblings, 0 replies; 12+ messages in thread From: Chris Murphy @ 2025-12-11 21:07 UTC (permalink / raw) To: Qu WenRuo, Btrfs BTRFS On Thu, Dec 11, 2025, at 11:52 AM, Chris Murphy wrote: > On Thu, Dec 11, 2025, at 11:01 AM, Chris Murphy wrote: >> On Wed, Dec 10, 2025, at 6:55 PM, Qu Wenruo wrote: >>> 在 2025/12/11 11:58, Chris Murphy 写道: >>>> User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. >>>> >>>> Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. >>>> >>>> User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. >>> >>> This looks like a previous memory corruption caused on-disk metadata >>> corruption. >>> >>> Tree-checker is not a memtest tool, it can only detect very obvious >>> problems, it can not do cross-reference, and unfortunately this is exact >>> cross-reference case. >>> >>> For this particular one, I'd recommend to do a "btrfs check --repair" >>> then "btrfs check" to verify. >> >> Looks like --repair changed from "errors 4" to "errors 6" >> >> >> [1/8] checking log >> [2/8] checking root items >> [3/8] checking extents >> [4/8] checking free space tree >> [5/8] checking fs roots >> unresolved ref dir 1924 index 0 namelen 40 name >> AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir >> index, no inode ref >> ERROR: errors found in fs roots >> Opening filesystem to check... >> Checking filesystem on /dev/sda3 >> UUID: afdbb979-0b91-499b-976c-0244ba2ed38f >> found 140964491264 bytes used, error(s) found >> total csum bytes: 136339904 >> total tree bytes: 969474048 >> total fs tree bytes: 751108096 >> total extent tree bytes: 62439424 >> btree space waste bytes: 178561905 >> file data blocks allocated: 490371264512 >> referenced 162984435712 >> >> End user has btrfs-image before and after repair if that's useful. But >> additional --repair attempts do not appear to fix the "errors 6" >> message - and the kernel still produces a warning and goes read-only if >> the file is accessed. > > > Looks like inode 730455 is found in more than one leaf. I'm kinda > wondering if there is a way to compel btrfs to delete the affected > leafs, thus fixing the file system. Backups are done so none of the > files in any of the leafs need saving. So if I were to have the user > delete all the files in all the suspect leaves - is it possible Btrfs > would just drop the leaf without hitting the splat? search for inode 730455 https://paste.mikkel.cc/3gCsnhb2/+inline The subvolid the file appears in is 256. >[ 46.847056] BTRFS info (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 33, gen 0 The corruption counter has increasing values but we're not seeing any csum/checksum errors. -- Chris Murphy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-11 18:01 ` Chris Murphy 2025-12-11 18:52 ` Chris Murphy @ 2025-12-15 9:02 ` Qu Wenruo 2025-12-15 14:47 ` Chris Murphy 1 sibling, 1 reply; 12+ messages in thread From: Qu Wenruo @ 2025-12-15 9:02 UTC (permalink / raw) To: Chris Murphy, Qu WenRuo, Btrfs BTRFS 在 2025/12/12 04:31, Chris Murphy 写道: > > > On Wed, Dec 10, 2025, at 6:55 PM, Qu Wenruo wrote: >> 在 2025/12/11 11:58, Chris Murphy 写道: >>> User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. >>> >>> Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. >>> >>> User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. >> >> This looks like a previous memory corruption caused on-disk metadata >> corruption. >> >> Tree-checker is not a memtest tool, it can only detect very obvious >> problems, it can not do cross-reference, and unfortunately this is exact >> cross-reference case. >> >> For this particular one, I'd recommend to do a "btrfs check --repair" >> then "btrfs check" to verify. > > Looks like --repair changed from "errors 4" to "errors 6" > > > [1/8] checking log > [2/8] checking root items > [3/8] checking extents > [4/8] checking free space tree > [5/8] checking fs roots > unresolved ref dir 1924 index 0 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir index, no inode ref And repair again? If after repair, readonly check still shows error, I can craft a quick fix branch for the reporter. Thanks, Qu > ERROR: errors found in fs roots > Opening filesystem to check... > Checking filesystem on /dev/sda3 > UUID: afdbb979-0b91-499b-976c-0244ba2ed38f > found 140964491264 bytes used, error(s) found > total csum bytes: 136339904 > total tree bytes: 969474048 > total fs tree bytes: 751108096 > total extent tree bytes: 62439424 > btree space waste bytes: 178561905 > file data blocks allocated: 490371264512 > referenced 162984435712 > > End user has btrfs-image before and after repair if that's useful. But additional --repair attempts do not appear to fix the "errors 6" message - and the kernel still produces a warning and goes read-only if the file is accessed. > > > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-15 9:02 ` Qu Wenruo @ 2025-12-15 14:47 ` Chris Murphy 2025-12-15 22:03 ` Mikkel Jeppesen [not found] ` <acefecca-9fbb-4268-a26a-b889756019e7@mikkel.cc> 0 siblings, 2 replies; 12+ messages in thread From: Chris Murphy @ 2025-12-15 14:47 UTC (permalink / raw) To: Qu Wenruo, Qu WenRuo, Btrfs BTRFS On Mon, Dec 15, 2025, at 2:02 AM, Qu Wenruo wrote: > 在 2025/12/12 04:31, Chris Murphy 写道: >> >> >> On Wed, Dec 10, 2025, at 6:55 PM, Qu Wenruo wrote: >>> 在 2025/12/11 11:58, Chris Murphy 写道: >>>> User reports root file system going read-only at some point after startup. Seems to be when a Firefox cache file is accessed. >>>> >>>> Initial report is kernel 6.17.8-300.fc43.x86_64, but the problem also happens with 6.18.0-65.fc44.x86_64. >>>> >>>> User previously discovered bad RAM and has replaced it, so I guess it's possible we have a bad write that made it to disk despite write time tree checker (?) and now can't handle the issue when reading the file. But I haven't seen this kind of error or call trace before, so I'm not sure what to recommend next. If --repair can fix it. >>> >>> This looks like a previous memory corruption caused on-disk metadata >>> corruption. >>> >>> Tree-checker is not a memtest tool, it can only detect very obvious >>> problems, it can not do cross-reference, and unfortunately this is exact >>> cross-reference case. >>> >>> For this particular one, I'd recommend to do a "btrfs check --repair" >>> then "btrfs check" to verify. >> >> Looks like --repair changed from "errors 4" to "errors 6" >> >> >> [1/8] checking log >> [2/8] checking root items >> [3/8] checking extents >> [4/8] checking free space tree >> [5/8] checking fs roots >> unresolved ref dir 1924 index 0 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir index, no inode ref > > And repair again? No change. sudo btrfs-image -t 16 -c 9 -ss /dev/sda3 fs_post_repair.img https://paste.mikkel.cc/gNwpLmyw > If after repair, readonly check still shows error, I can craft a quick > fix branch for the reporter. I'll check to see if they still have the file system. I asked them to make a btrfs-image before starting over. Thanks Qu! -- Chris Murphy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-15 14:47 ` Chris Murphy @ 2025-12-15 22:03 ` Mikkel Jeppesen [not found] ` <acefecca-9fbb-4268-a26a-b889756019e7@mikkel.cc> 1 sibling, 0 replies; 12+ messages in thread From: Mikkel Jeppesen @ 2025-12-15 22:03 UTC (permalink / raw) To: lists; +Cc: linux-btrfs, quwenruo.btrfs, wqu Hi there. Affected user here :) On Monday, December 15, 2025 3:47:13 PM Central European Standard Time Chris Murphy wrote: > ... > >> Looks like --repair changed from "errors 4" to "errors 6" > >> > >> > >> [1/8] checking log > >> [2/8] checking root items > >> [3/8] checking extents > >> [4/8] checking free space tree > >> [5/8] checking fs roots > >> > >> unresolved ref dir 1924 index 0 namelen 40 name > >> AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir > >> index, no inode ref> > > And repair again? > > No change. > > sudo btrfs-image -t 16 -c 9 -ss /dev/sda3 fs_post_repair.img > > https://paste.mikkel.cc/gNwpLmyw > > > If after repair, readonly check still shows error, I can craft a quick > > fix branch for the reporter. > > I'll check to see if they still have the file system. I asked them to make a > btrfs-image before starting over. Thanks Qu! I installed a new F43 install on a different SSD, so I still have the affected file system/install available for any testing. Thanks for the quick and good responses to this! :) ^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <acefecca-9fbb-4268-a26a-b889756019e7@mikkel.cc>]
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly [not found] ` <acefecca-9fbb-4268-a26a-b889756019e7@mikkel.cc> @ 2025-12-15 23:05 ` Qu Wenruo 2025-12-16 1:14 ` mikkel 0 siblings, 1 reply; 12+ messages in thread From: Qu Wenruo @ 2025-12-15 23:05 UTC (permalink / raw) To: mikkel+btrfs, lists; +Cc: linux-btrfs, quwenruo.btrfs 在 2025/12/16 08:31, Mikkel Jeppesen 写道: > Hi there. Affected user here :) > > On Monday, December 15, 2025 3:47:13 PM Central European Standard Time > Chris > > Murphy wrote: > > > ... > > > >> Looks like --repair changed from "errors 4" to "errors 6" > > > >> > > > >> > > > >> [1/8] checking log > > > >> [2/8] checking root items > > > >> [3/8] checking extents > > > >> [4/8] checking free space tree > > > >> [5/8] checking fs roots > > > >> > > > >> unresolved ref dir 1924 index 0 namelen 40 name > > > >> AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir > > > >> index, no inode ref> > > > > And repair again? > > > > > > No change. > > > > > > sudo btrfs-image -t 16 -c 9 -ss /dev/sda3 fs_post_repair.img > > > > > > https://paste.mikkel.cc/gNwpLmyw > > > > > > > If after repair, readonly check still shows error, I can craft a quick > > > > fix branch for the reporter. > > > > > > I'll check to see if they still have the file system. I asked them to > make a > > > btrfs-image before starting over. Thanks Qu! > > I installed a new F43 install on a different SSD, so I still have the > affected > > file system/install available for any testing. Thanks a lot. If your fs doesn't contain confidential info in the file names (btrfs-image dump doesn't contain regular data, but may still contain inlined data), a regular btrfs-image dump would be the best way for us to test. Unfortunately this corruption involves filename, thus the "-s"/"-ss" option is screwing up the result. If there is confidential filenames involved, please provide the following dump from the original fs: # btrfs ins dump-tree <device> | grep -C 7 "AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E" And unfortunately we will need to communicate for each new dump, so this will take some time. And before posting the content of above dump, make sure no confidential filename is involved. Thanks, Qu > > Thanks for the quick and good responses to this! :) > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-15 23:05 ` Qu Wenruo @ 2025-12-16 1:14 ` mikkel 2025-12-16 5:42 ` Qu Wenruo 0 siblings, 1 reply; 12+ messages in thread From: mikkel @ 2025-12-16 1:14 UTC (permalink / raw) To: Qu Wenruo, mikkel+btrfs, lists; +Cc: linux-btrfs, quwenruo.btrfs On Tuesday, December 16, 2025 12:05:54 AM Central European Standard Time Qu Wenruo wrote: > > 在 2025/12/16 08:31, Mikkel Jeppesen 写道: > Hi there. Affected user here :) > > On Monday, December 15, 2025 3:47:13 PM Central European Standard Time > Chris > > Murphy wrote: > > ... > > >> Looks like --repair changed from "errors 4" to "errors 6" > > >> > > >> > > >> [1/8] checking log > > >> [2/8] checking root items > > >> [3/8] checking extents > > >> [4/8] checking free space tree > > >> [5/8] checking fs roots > > >> > > >> unresolved ref dir 1924 index 0 namelen 40 name > > >> AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 6, no dir > > >> index, no inode ref > > > > > > And repair again? > > > > No change. > > > > sudo btrfs-image -t 16 -c 9 -ss /dev/sda3 fs_post_repair.img > > > > https://paste.mikkel.cc/gNwpLmyw > > > > > If after repair, readonly check still shows error, I can craft a > > > quick > > > > > > fix branch for the reporter. > > > > I'll check to see if they still have the file system. I asked them to > > make a > > > btrfs-image before starting over. Thanks Qu! > > I installed a new F43 install on a different SSD, so I still have the > affected > > file system/install available for any testing. > > Thanks a lot. If your fs doesn't contain confidential info in the file > names (btrfs-image dump doesn't contain regular data, but may still > contain inlined data), a regular btrfs-image dump would be the best way > for us to test. > > Unfortunately this corruption involves filename, thus the "-s"/"-ss" > option is screwing up the result. There shouldn't be any issues there. I hadn't used the system for much other than one steam game before this happened. Here's the image: https://paste.mikkel.cc/FH3jEN9z > > If there is confidential filenames involved, please provide the > following dump from the original fs: > > # btrfs ins dump-tree <device> | grep -C 7 > "AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E" > > And unfortunately we will need to communicate for each new dump, so this > will take some time. > That's all good. I'm heading off for tonight, but I'll try and keep an eye on my email when I'm home :) > > And before posting the content of above dump, make sure no confidential > filename is involved. > > Thanks, > Qu > > Thanks for the quick and good responses to this! :) > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-16 1:14 ` mikkel @ 2025-12-16 5:42 ` Qu Wenruo 2025-12-17 18:42 ` Mikkel Jeppesen 0 siblings, 1 reply; 12+ messages in thread From: Qu Wenruo @ 2025-12-16 5:42 UTC (permalink / raw) To: mikkel+btrfs, Qu Wenruo, lists; +Cc: linux-btrfs 在 2025/12/16 11:44, mikkel@mikkel.cc 写道: > On Tuesday, December 16, 2025 12:05:54 AM Central European Standard Time Qu > Wenruo wrote: > [...] >> Unfortunately this corruption involves filename, thus the "-s"/"-ss" >> option is screwing up the result. > > There shouldn't be any issues there. I hadn't used the system for much other > than one steam game before this happened. > Here's the image: > https://paste.mikkel.cc/FH3jEN9z It's indeed a bitflip, and in a pretty bad location, making the initial repair doing a bad job. Firstly there is a DIR_ITEM that is the offending one in subvolume 256: item 148 key (1924 DIR_ITEM 2435677723) itemoff 5853 itemsize 70 location key (730455 INODE_ITEM 0) type FILE transid 7280 data_len 0 name_len 40 name: AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E However there is no DIR_INDEX one for it. I believe this is caused by the initial repair. Then for the offending inode, it exists! item 10 key (730455 INODE_ITEM 0) itemoff 15456 itemsize 160 generation 7280 transid 9794 size 13829 nbytes 16384 block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 sequence 11 flags 0x0(none) atime 1765397443.29231914 (2025-12-11 06:40:43) ctime 1764798591.882909548 (2025-12-04 08:19:51) mtime 1764798591.882909548 (2025-12-04 08:19:51) otime 1764712848.413821734 (2025-12-03 08:30:48) item 11 key (730455 UNKNOWN.8 1924) itemoff 15406 itemsize 50 Note the item 11, which has an unknown type 8. But normally this should be INODE_REF, and the size matches the namelen + 10. bin(8) = 0b1000 bin(12) = 0b1100 Diff is exact one bit, so it's again a bitflip. This means there are several problems in the original mode at least:(lowmem mode should be able to do better, but the metadata is too large thus lowmem mode is too slow to be practical) - Unknown key type detection Every time we hit an unknown key type, we should give some warning to help debugging at least. - Inode nlink checks is not properly checking INODE_REF Since this has a back INODE_REF item, I have to manually fix it, you can fetch the following branch: https://github.com/adam900710/btrfs-progs/tree/dirty_fix Then compile it, you will get "btrfs-corrupt-block" at the project top directory. Then: ./btrfs-corrupt-block -X <device> Which will fix the inode ref. Now btrfs check should report a different error: ... [5/8] checking fs roots unresolved ref dir 1924 index 134016 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 2, no dir index ERROR: errors found in fs roots ... This is the indicator that the repair is done correctly (errors 2). Then "btrfs check --repair" should be able to repair it: enabling repair mode Opening filesystem to check... Checking filesystem on /home/adam/test.img UUID: afdbb979-0b91-499b-976c-0244ba2ed38f [1/8] checking log skipped (none written) [2/8] checking root items Fixed 0 roots. [3/8] checking extents super bytes used 166880690176 mismatches actual used 166880575488 No device size related problem found [4/8] checking free space tree [5/8] checking fs roots repairing missing dir index item for inode 730455 reset isize for dir 1924 root 256 [6/8] checking only csums items (without verifying data) [7/8] checking root refs [8/8] checking quota groups skipped (not enabled on this FS) found 333761265664 bytes used, no error found total csum bytes: 322896656 total tree bytes: 2385526784 total fs tree bytes: 1857060864 total extent tree bytes: 164102144 btree space waste bytes: 411870645 file data blocks allocated: 1029749325824 referenced 404780113920 Thanks, Qu ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly 2025-12-16 5:42 ` Qu Wenruo @ 2025-12-17 18:42 ` Mikkel Jeppesen 0 siblings, 0 replies; 12+ messages in thread From: Mikkel Jeppesen @ 2025-12-17 18:42 UTC (permalink / raw) To: mikkel+btrfs, Qu Wenruo, lists, Qu Wenruo; +Cc: linux-btrfs On Tuesday, December 16, 2025 6:42:28 AM Central European Standard Time Qu Wenruo wrote: > 在 2025/12/16 11:44, mikkel@mikkel.cc 写道: > > On Tuesday, December 16, 2025 12:05:54 AM Central European Standard Time > > Qu > > > Wenruo wrote: > [...] > It's indeed a bitflip, and in a pretty bad location, making the initial > repair doing a bad job. > > > Firstly there is a DIR_ITEM that is the offending one in subvolume 256: > > item 148 key (1924 DIR_ITEM 2435677723) itemoff 5853 itemsize 70 > location key (730455 INODE_ITEM 0) type FILE > transid 7280 data_len 0 name_len 40 > name: AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E > > However there is no DIR_INDEX one for it. > I believe this is caused by the initial repair. > > Then for the offending inode, it exists! > > item 10 key (730455 INODE_ITEM 0) itemoff 15456 itemsize 160 > generation 7280 transid 9794 size 13829 nbytes 16384 > block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0 > sequence 11 flags 0x0(none) > atime 1765397443.29231914 (2025-12-11 06:40:43) > ctime 1764798591.882909548 (2025-12-04 08:19:51) > mtime 1764798591.882909548 (2025-12-04 08:19:51) > otime 1764712848.413821734 (2025-12-03 08:30:48) > item 11 key (730455 UNKNOWN.8 1924) itemoff 15406 itemsize 50 > > Note the item 11, which has an unknown type 8. But normally this should > be INODE_REF, and the size matches the namelen + 10. > > bin(8) = 0b1000 > bin(12) = 0b1100 > > Diff is exact one bit, so it's again a bitflip. > > This means there are several problems in the original mode at > least:(lowmem mode should be able to do better, but the metadata is too > large thus lowmem mode is too slow to be practical) > > - Unknown key type detection > Every time we hit an unknown key type, we should give some warning to > help debugging at least. > > - Inode nlink checks is not properly checking INODE_REF > > Since this has a back INODE_REF item, I have to manually fix it, you can > fetch the following branch: > > https://github.com/adam900710/btrfs-progs/tree/dirty_fix > > Then compile it, you will get "btrfs-corrupt-block" at the project top > directory. > > Then: > ./btrfs-corrupt-block -X <device> > > Which will fix the inode ref. > > Now btrfs check should report a different error: > > ... > [5/8] checking fs roots > unresolved ref dir 1924 index 134016 namelen 40 name > AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 2, no dir index > ERROR: errors found in fs roots > ... > > This is the indicator that the repair is done correctly (errors 2). > > Then "btrfs check --repair" should be able to repair it: > > enabling repair mode > Opening filesystem to check... > Checking filesystem on /home/adam/test.img > UUID: afdbb979-0b91-499b-976c-0244ba2ed38f > [1/8] checking log skipped (none written) > [2/8] checking root items > Fixed 0 roots. > [3/8] checking extents > super bytes used 166880690176 mismatches actual used 166880575488 > No device size related problem found > [4/8] checking free space tree > [5/8] checking fs roots > repairing missing dir index item for inode 730455 > reset isize for dir 1924 root 256 > [6/8] checking only csums items (without verifying data) > [7/8] checking root refs > [8/8] checking quota groups skipped (not enabled on this FS) > found 333761265664 bytes used, no error found > total csum bytes: 322896656 > total tree bytes: 2385526784 > total fs tree bytes: 1857060864 > total extent tree bytes: 164102144 > btree space waste bytes: 411870645 > file data blocks allocated: 1029749325824 > referenced 404780113920 > > Thanks, > Qu This did indeed do the trick! :) sudo ./btrfs-corrupt-block -X /dev/sda3 Repaired the bad inode ref key sudo btrfs check /dev/sda3 Opening filesystem to check... Checking filesystem on /dev/sda3 UUID: afdbb979-0b91-499b-976c-0244ba2ed38f [1/8] checking log skipped (none written) [2/8] checking root items [3/8] checking extents [4/8] checking free space tree [5/8] checking fs roots unresolved ref dir 1924 index 134016 namelen 40 name AC1E6A9C763DC6BC77494D6E8DE724C240D36C9E filetype 1 errors 2, no dir index ERROR: errors found in fs roots found 167377907712 bytes used, error(s) found total csum bytes: 161874964 total tree bytes: 1243725824 total fs tree bytes: 976257024 total extent tree bytes: 84213760 btree space waste bytes: 214400581 file data blocks allocated: 511281180672 referenced 203289292800 sudo btrfs check --repair /dev/sda3 enabling repair mode WARNING: Do not use --repair unless you are advised to do so by a developer or an experienced user, and then only after having accepted that no fsck can successfully repair all types of filesystem corruption. E.g. some software or hardware bugs can fatally damage a volume. The operation will start in 10 seconds. Use Ctrl-C to stop it. 10 9 8 7 6 5 4 3 2 1 Starting repair. Opening filesystem to check... Checking filesystem on /dev/sda3 UUID: afdbb979-0b91-499b-976c-0244ba2ed38f [1/8] checking log skipped (none written) [2/8] checking root items Fixed 0 roots. [3/8] checking extents No device size related problem found [4/8] checking free space tree [5/8] checking fs roots repairing missing dir index item for inode 730455 reset isize for dir 1924 root 256 [6/8] checking only csums items (without verifying data) [7/8] checking root refs [8/8] checking quota groups skipped (not enabled on this FS) found 167377907712 bytes used, no error found total csum bytes: 161874964 total tree bytes: 1243725824 total fs tree bytes: 976257024 total extent tree bytes: 84213760 btree space waste bytes: 214400581 file data blocks allocated: 511281180672 referenced 203289292800 sudo btrfs check /dev/sda3 Opening filesystem to check... Checking filesystem on /dev/sda3 UUID: afdbb979-0b91-499b-976c-0244ba2ed38f [1/8] checking log skipped (none written) [2/8] checking root items [3/8] checking extents [4/8] checking free space tree [5/8] checking fs roots [6/8] checking only csums items (without verifying data) [7/8] checking root refs [8/8] checking quota groups skipped (not enabled on this FS) found 167377907712 bytes used, no error found total csum bytes: 161874964 total tree bytes: 1243725824 total fs tree bytes: 976257024 total extent tree bytes: 84213760 btree space waste bytes: 214400236 file data blocks allocated: 511281180672 referenced 203289292800 ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2025-12-17 18:43 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-11 1:28 kernel 6.17 and 6.18, WARNING: CPU: 5 PID: 7181 at fs/btrfs/inode.c:4297 __btrfs_unlink_inode, forced readonly Chris Murphy
2025-12-11 1:55 ` Qu Wenruo
2025-12-11 18:01 ` Chris Murphy
2025-12-11 18:52 ` Chris Murphy
2025-12-11 21:07 ` Chris Murphy
2025-12-15 9:02 ` Qu Wenruo
2025-12-15 14:47 ` Chris Murphy
2025-12-15 22:03 ` Mikkel Jeppesen
[not found] ` <acefecca-9fbb-4268-a26a-b889756019e7@mikkel.cc>
2025-12-15 23:05 ` Qu Wenruo
2025-12-16 1:14 ` mikkel
2025-12-16 5:42 ` Qu Wenruo
2025-12-17 18:42 ` Mikkel Jeppesen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox