Re: read time block corruption on root partition, Not booting

Qu Wenruo
2019-12-21T14:19:26Z

[-- Attachment #1.1: Type: text/plain, Size: 12959 bytes --]

On 2019/12/21 下午10:06, jollemeyer@gmx.de wrote:
> Dear all,
>
> on my BTRFS root partition (/dev/sda), a "read time block corruption" was detected. The system refuses to boot.
> Current kernel is 5.4.2-1-Manjaro.
>
> Systemd's journalctl generated the following output:
>
> Dez 21 10:46:41 Phoenix kernel: BTRFS critical (device sda): corrupt leaf: block=2089035464704 slot=85 extent bytenr=1937096425472 len=4096 invalid extent refs, have 1 expect >= inline 1048577
> Dez 21 10:46:41 Phoenix kernel: BTRFS error (device sda): block=2089035464704 read time tree block corruption detected
> Dez 21 10:46:41 Phoenix kernel: BTRFS critical (device sda): corrupt leaf: block=2089035464704 slot=85 extent bytenr=1937096425472 len=4096 invalid extent refs, have 1 expect >= inline 1048577
> Dez 21 10:46:41 Phoenix kernel: BTRFS error (device sda): block=2089035464704 read time tree block corruption detected
> Dez 21 10:46:41 Phoenix kernel: BTRFS critical (device sda): corrupt leaf: block=2089035464704 slot=85 extent bytenr=1937096425472 len=4096 invalid extent refs, have 1 expect >= inline 1048577
> Dez 21 10:46:41 Phoenix kernel: BTRFS error (device sda): block=2089035464704 read time tree block corruption detected
> Dez 21 10:46:41 Phoenix kernel: BTRFS critical (device sda): corrupt leaf: block=2089035464704 slot=85 extent bytenr=1937096425472 len=4096 invalid extent refs, have 1 expect >= inline 1048577
> Dez 21 10:46:41 Phoenix kernel: BTRFS error (device sda): block=2089035464704 read time tree block corruption detected
> Dez 21 10:46:41 Phoenix kernel: BTRFS: error (device sda) in btrfs_replay_log:2293: errno=-5 IO failure (Failed to recover log tree)

"btrfs ins dump-tree -b 2089035464704 /dev/sda" output please.

Thanks,
Qu

> Dez 21 10:46:41 Phoenix kernel: ------------[ cut here ]------------
> Dez 21 10:46:41 Phoenix kernel: WARNING: CPU: 7 PID: 621 at fs/btrfs/block-group.c:132 btrfs_put_block_group+0x42/0x50 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: Modules linked in: usblp snd_hda_codec_hdmi intel_rapl_msr snd_hda_codec_realtek intel_rapl_common snd_hda_codec_generic ext4 ledtrig_audio x86_pkg_temp_thermal intel_powerclamp coretemp crc16 kvm_intel mbcache i915 jbd2 kvm snd_hda_intel snd_intel_nhlt i2c_algo_bit irqbypass drm_kms_helper snd_hda_codec crct10dif_pclmul crc32_pclmul snd_hda_core mousedev ghash_clmulni_intel drm snd_hwdep snd_pcm aesni_intel crypto_simd cryptd glue_helper mei_hdcp intel_cstate iTCO_wdt intel_gtt snd_timer iTCO_vendor_support intel_uncore mei_me intel_rapl_perf input_leds agpgart i2c_i801 snd syscopyarea mei sysfillrect lpc_ich r8168(OE) sysimgblt soundcore fb_sys_fops evdev ie31200_edac mac_hid vboxpci(OE) vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) sg crypto_user ip_tables x_tables hid_generic usbhid hid btrfs libcrc32c crc32c_generic xor uas usb_storage raid6_pq sr_mod cdrom sd_mod ahci serio_raw atkbd libahci libps2 libata xhci_pci scsi_mod xhci_hcd ehci_pci crc32c_intel ehci_hcd i8042 serio
> Dez 21 10:46:41 Phoenix kernel: CPU: 7 PID: 621 Comm: mount Tainted: G OE 5.4.2-1-MANJARO #1
> Dez 21 10:46:41 Phoenix kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./H77 Pro4/MVP, BIOS P1.40 09/04/2012
> Dez 21 10:46:41 Phoenix kernel: RIP: 0010:btrfs_put_block_group+0x42/0x50 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: Code: 2d 48 83 7d 50 00 75 22 48 8b 85 e8 01 00 00 48 85 c0 75 1e 48 8b bd d8 00 00 00 e8 a8 08 66 d9 48 89 ef 5d e9 9f 08 66 d9 c3 <0f> 0b eb da 0f 0b eb cf 0f 0b eb de 66 90 0f 1f 44 00 00 31 d2 e9
> Dez 21 10:46:41 Phoenix kernel: RSP: 0018:ffffa95900f47ae8 EFLAGS: 00010206
> Dez 21 10:46:41 Phoenix kernel: RAX: 0000000000000001 RBX: ffff8a1eb7bfe0e0 RCX: 0000000000000000
> Dez 21 10:46:41 Phoenix kernel: RDX: 0000000000000001 RSI: ffff8a1ecf5f0250 RDI: ffff8a1eb7bfe000
> Dez 21 10:46:41 Phoenix kernel: RBP: ffff8a1eb7bfe000 R08: 0000000000000000 R09: 0000000000000001
> Dez 21 10:46:41 Phoenix kernel: R10: 00000000003f6c00 R11: 0000000000000000 R12: ffff8a1ec81b0080
> Dez 21 10:46:41 Phoenix kernel: R13: ffff8a1ec81b0090 R14: ffff8a1eb7bfe000 R15: dead000000000100
> Dez 21 10:46:41 Phoenix kernel: FS: 00007f77003d7500(0000) GS:ffff8a1ecf5c0000(0000) knlGS:0000000000000000
> Dez 21 10:46:41 Phoenix kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> Dez 21 10:46:41 Phoenix kernel: CR2: 0000559652364460 CR3: 00000003fbaec005 CR4: 00000000001606e0
> Dez 21 10:46:41 Phoenix kernel: Call Trace:
> Dez 21 10:46:41 Phoenix kernel: btrfs_free_block_groups+0x11c/0x260 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: open_ctree+0x187a/0x1bc0 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: btrfs_mount_root+0x57b/0x670 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: legacy_get_tree+0x27/0x40
> Dez 21 10:46:41 Phoenix kernel: vfs_get_tree+0x25/0xb0
> Dez 21 10:46:41 Phoenix kernel: fc_mount+0xe/0x30
> Dez 21 10:46:41 Phoenix kernel: vfs_kern_mount.part.0+0x71/0x90
> Dez 21 10:46:41 Phoenix kernel: btrfs_mount+0x18e/0x930 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: ? filename_lookup+0x105/0x190
> Dez 21 10:46:41 Phoenix kernel: ? legacy_get_tree+0x27/0x40
> Dez 21 10:46:41 Phoenix kernel: ? btrfs_remount+0x4d0/0x4d0 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: legacy_get_tree+0x27/0x40
> Dez 21 10:46:41 Phoenix kernel: vfs_get_tree+0x25/0xb0
> Dez 21 10:46:41 Phoenix kernel: do_mount+0x77a/0xa30
> Dez 21 10:46:41 Phoenix kernel: ksys_mount+0x7e/0xc0
> Dez 21 10:46:41 Phoenix kernel: __x64_sys_mount+0x21/0x30
> Dez 21 10:46:41 Phoenix kernel: do_syscall_64+0x4e/0x140
> Dez 21 10:46:41 Phoenix kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
> Dez 21 10:46:41 Phoenix kernel: RIP: 0033:0x7f770055ae4e
> Dez 21 10:46:41 Phoenix kernel: Code: 48 8b 0d 35 00 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 02 00 0c 00 f7 d8 64 89 01 48
> Dez 21 10:46:41 Phoenix kernel: RSP: 002b:00007ffedb3135c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
> Dez 21 10:46:41 Phoenix kernel: RAX: ffffffffffffffda RBX: 00007f7700681204 RCX: 00007f770055ae4e
> Dez 21 10:46:41 Phoenix kernel: RDX: 000055d9705f5650 RSI: 000055d9705f56f0 RDI: 000055d9705f73d0
> Dez 21 10:46:41 Phoenix kernel: RBP: 000055d9705f5440 R08: 000055d9705f5670 R09: 0000000000000000
> Dez 21 10:46:41 Phoenix kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> Dez 21 10:46:41 Phoenix kernel: R13: 000055d9705f73d0 R14: 000055d9705f5650 R15: 000055d9705f5440
> Dez 21 10:46:41 Phoenix kernel: ---[ end trace 71465d442bb4c509 ]---
> Dez 21 10:46:41 Phoenix kernel: ------------[ cut here ]------------
> Dez 21 10:46:41 Phoenix kernel: WARNING: CPU: 0 PID: 621 at fs/btrfs/block-group.c:3166 btrfs_free_block_groups+0x1ea/0x260 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: Modules linked in: usblp snd_hda_codec_hdmi intel_rapl_msr snd_hda_codec_realtek intel_rapl_common snd_hda_codec_generic ext4 ledtrig_audio x86_pkg_temp_thermal intel_powerclamp coretemp crc16 kvm_intel mbcache i915 jbd2 kvm snd_hda_intel snd_intel_nhlt i2c_algo_bit irqbypass drm_kms_helper snd_hda_codec crct10dif_pclmul crc32_pclmul snd_hda_core mousedev ghash_clmulni_intel drm snd_hwdep snd_pcm aesni_intel crypto_simd cryptd glue_helper mei_hdcp intel_cstate iTCO_wdt intel_gtt snd_timer iTCO_vendor_support intel_uncore mei_me intel_rapl_perf input_leds agpgart i2c_i801 snd syscopyarea mei sysfillrect lpc_ich r8168(OE) sysimgblt soundcore fb_sys_fops evdev ie31200_edac mac_hid vboxpci(OE) vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) sg crypto_user ip_tables x_tables hid_generic usbhid hid btrfs libcrc32c crc32c_generic xor uas usb_storage raid6_pq sr_mod cdrom sd_mod ahci serio_raw atkbd libahci libps2 libata xhci_pci scsi_mod xhci_hcd ehci_pci crc32c_intel ehci_hcd i8042 serio
> Dez 21 10:46:41 Phoenix kernel: CPU: 0 PID: 621 Comm: mount Tainted: G W OE 5.4.2-1-MANJARO #1
> Dez 21 10:46:41 Phoenix kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./H77 Pro4/MVP, BIOS P1.40 09/04/2012
> Dez 21 10:46:41 Phoenix kernel: RIP: 0010:btrfs_free_block_groups+0x1ea/0x260 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: Code: 49 bd 22 01 00 00 00 00 ad de e8 51 13 d1 d9 e8 0c d3 4e d9 48 89 ef e8 64 9b ff ff 48 8b 85 00 10 00 00 49 39 c4 75 3c eb 5f <0f> 0b 31 c9 31 d2 4c 89 fe 48 89 ef e8 55 85 ff ff 48 8b 43 08 48
> Dez 21 10:46:41 Phoenix kernel: RSP: 0018:ffffa95900f47af8 EFLAGS: 00010206
> Dez 21 10:46:41 Phoenix kernel: RAX: ffff8a1eca367488 RBX: ffff8a1eca367488 RCX: 0000000000000000
> Dez 21 10:46:41 Phoenix kernel: RDX: 0000000000000001 RSI: ffff8a1eca364200 RDI: 00000000ffffffff
> Dez 21 10:46:41 Phoenix kernel: RBP: ffff8a1ec81b0000 R08: 0000000000000000 R09: 0000000020000000
> Dez 21 10:46:41 Phoenix kernel: R10: 0000000000000005 R11: ffffffffffd5ce37 R12: ffff8a1ec81b1000
> Dez 21 10:46:41 Phoenix kernel: R13: dead000000000122 R14: dead000000000100 R15: ffff8a1eca367400
> Dez 21 10:46:41 Phoenix kernel: FS: 00007f77003d7500(0000) GS:ffff8a1ecf400000(0000) knlGS:0000000000000000
> Dez 21 10:46:41 Phoenix kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> Dez 21 10:46:41 Phoenix kernel: CR2: 0000559006e55d68 CR3: 00000003fbaec006 CR4: 00000000001606f0
> Dez 21 10:46:41 Phoenix kernel: Call Trace:
> Dez 21 10:46:41 Phoenix kernel: open_ctree+0x187a/0x1bc0 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: btrfs_mount_root+0x57b/0x670 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: legacy_get_tree+0x27/0x40
> Dez 21 10:46:41 Phoenix kernel: vfs_get_tree+0x25/0xb0
> Dez 21 10:46:41 Phoenix kernel: fc_mount+0xe/0x30
> Dez 21 10:46:41 Phoenix kernel: vfs_kern_mount.part.0+0x71/0x90
> Dez 21 10:46:41 Phoenix kernel: btrfs_mount+0x18e/0x930 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: ? filename_lookup+0x105/0x190
> Dez 21 10:46:41 Phoenix kernel: ? legacy_get_tree+0x27/0x40
> Dez 21 10:46:41 Phoenix kernel: ? btrfs_remount+0x4d0/0x4d0 [btrfs]
> Dez 21 10:46:41 Phoenix kernel: legacy_get_tree+0x27/0x40
> Dez 21 10:46:41 Phoenix kernel: vfs_get_tree+0x25/0xb0
> Dez 21 10:46:41 Phoenix kernel: do_mount+0x77a/0xa30
> Dez 21 10:46:41 Phoenix kernel: ksys_mount+0x7e/0xc0
> Dez 21 10:46:41 Phoenix kernel: __x64_sys_mount+0x21/0x30
> Dez 21 10:46:41 Phoenix kernel: do_syscall_64+0x4e/0x140
> Dez 21 10:46:41 Phoenix kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
> Dez 21 10:46:41 Phoenix kernel: RIP: 0033:0x7f770055ae4e
> Dez 21 10:46:41 Phoenix kernel: Code: 48 8b 0d 35 00 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 02 00 0c 00 f7 d8 64 89 01 48
> Dez 21 10:46:41 Phoenix kernel: RSP: 002b:00007ffedb3135c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
> Dez 21 10:46:41 Phoenix kernel: RAX: ffffffffffffffda RBX: 00007f7700681204 RCX: 00007f770055ae4e
> Dez 21 10:46:41 Phoenix kernel: RDX: 000055d9705f5650 RSI: 000055d9705f56f0 RDI: 000055d9705f73d0
> Dez 21 10:46:41 Phoenix kernel: RBP: 000055d9705f5440 R08: 000055d9705f5670 R09: 0000000000000000
> Dez 21 10:46:41 Phoenix kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> Dez 21 10:46:41 Phoenix kernel: R13: 000055d9705f73d0 R14: 000055d9705f5650 R15: 000055d9705f5440
> Dez 21 10:46:41 Phoenix kernel: ---[ end trace 71465d442bb4c50a ]---
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): space_info 1 has 733175808 free, is not full
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): space_info total=1386200694784, used=1385467318272, pinned=0, reserved=4096, may_use=0, readonly=196608
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): global_block_rsv: size 0 reserved 0
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): trans_block_rsv: size 0 reserved 0
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): chunk_block_rsv: size 0 reserved 0
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): delayed_block_rsv: size 0 reserved 0
> Dez 21 10:46:41 Phoenix kernel: BTRFS info (device sda): delayed_refs_rsv: size 0 reserved 0
> Dez 21 10:46:41 Phoenix kernel: BTRFS error (device sda): open_ctree failed
> -- Subject: Unit process exited
> -- Defined-By: systemd
>
> Btrfs check --readonly /dev/sda also found errors:
>
> Opening filesystem to check...
> Checking filesystem on /dev/sda
> UUID: 014bdf0a-bcb3-4a7c-b58d-c1fc17281f7e
> found 1386975199232 bytes used, error(s) found
> total csum bytes: 1352697008
> total tree bytes: 1508327424
> total fs tree bytes: 61161472
> total extent tree bytes: 37421056
> btree space waste bytes: 55303865
> file data blocks allocated: 1388605296640
> referenced 1385670742016
>
> Btrfs scrub does not recognize any errors.
>
> Kindly help me to recover this error.
>
> many thanks,
>
> Jörg Meyer
>
>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]