Re: kernel BUG at fs/btrfs/ctree.c:5213

[Liu Bo <bo.li.liu@oracle.com>]

On Fri, Sep 02, 2016 at 04:58:43PM +0200, stek14@srv1.eu wrote:
> Hi,
> 
> since a few weeks ago, my BTRFS (the root file system) sporadically crashes
> leaving a "kernel BUG" message in dmesg. I also noticed some files behaving
> like this:
> 
> $ ls -lai
> ls: cannot access 'nlpsolver.jar': Stale file handle
> ls: cannot access 'EvolutionarySolver.jar': Stale file handle
> ls: cannot access 'description-en-US.txt': Stale file handle
> ls: cannot access 'components.rdb': Stale file handle
> ls: cannot access 'description.xml': Stale file handle
> total 0
> 1546821 drwxr-xr-x 1 root root 230 Apr 17 11:44 .
> 1546727 drwxr-xr-x 1 root root  68 Apr 17 11:44 ..
>       ? -????????? ? ?    ?      ?            ? components.rdb
>       ? -????????? ? ?    ?      ?            ? description-en-US.txt
>       ? -????????? ? ?    ?      ?            ? description.xml
>       ? -????????? ? ?    ?      ?            ? EvolutionarySolver.jar
> 1546828 drwxr-xr-x 1 root root  58 Apr 17 11:44 help
> 1546829 drwxr-xr-x 1 root root 268 Apr 17 11:44 locale
> 1546825 drwxr-xr-x 1 root root  24 Apr 17 11:44 META-INF
>       ? -????????? ? ?    ?      ?            ? nlpsolver.jar
> 1546826 drwxr-xr-x 1 root root  14 Apr 17 11:44 registration
> 
> 
> Every time I try to access them, I get the following in dmesg, always with
> the same numbers:
> 
> [  274.448070] BTRFS error (device sda8): parent transid verify failed on
> 31042928640 wanted 18412968017025564884 found 135380
> 

We have a commit[1] in 4.8 which has cleaned up this BUG_ON(), but
apparently some metadata blocks have invalid content, I'd recommand trying
btrfsck --repair, it's offline.

[1]:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=fb770ae414d018255afa7a70b14ba1f8620762dd

Thanks,

-liubo

> 
> Then suddenly, usually while doing something unrelated like loading a
> website in my browser, a "kernel BUG" appears in dmesg, most programs freeze
> and I can only recover by rebooting. It was still possible to save the
> output of dmesg, which I copied below. I guess that even if my file system
> is currupted, this should not happen?
> 
> In addition to reporting this bug, I would like to ask what I can do to
> repair the file system? btrfs scrub does not return any errors, and the
> usual articles describing the "parent transid verify failed" error assume
> the "wanted" and "found" IDs differ only by a small number, so I guess they
> don't apply to my situation? Can someone give me some advice? If I can't
> repair it, it would at least be good to have a list which files are currupt,
> so I can set up a new file system and restore them from a backup.
> 
> Thank you!
> 
> Florian
> 
> 
> 
> Here is some more information:
> 
> dmesg output:
> [18101.021285] BTRFS error (device sda8): parent transid verify failed on
> 31042928640 wanted 18412968017025564884 found 135380
> [18101.021479] BTRFS error (device sda8): parent transid verify failed on
> 31042928640 wanted 18412968017025564884 found 135380
> [27168.533582] verify_parent_transid: 12 callbacks suppressed
> [27168.533587] BTRFS error (device sda8): parent transid verify failed on
> 31042928640 wanted 18412968017025564884 found 135380
> [27168.533627] ------------[ cut here ]------------
> [27168.534696] kernel BUG at fs/btrfs/ctree.c:5213!
> [27168.535684] invalid opcode: 0000 [#1] PREEMPT SMP
> [27168.536613] Modules linked in: sha256_ssse3 sha256_generic hmac drbg
> ansi_cprng ctr ccm joydev snd_hda_codec_hdmi mousedev snd_hda_codec_realtek
> snd_hda_codec_generic mei_wdt dell_wmi sparse_keymap iTCO_wdt
> iTCO_vendor_support ppdev intel_rapl x86_pkg_temp_thermal intel_powerclamp
> coretemp kvm_intel kvm arc4 irqbypass crct10dif_pclmul crc32_pclmul
> dell_laptop ghash_clmulni_intel dell_smbios nls_iso8859_1 dcdbas
> dell_smm_hwmon nls_cp437 vfat i915 fat aesni_intel aes_x86_64 lrw gf128mul
> glue_helper ablk_helper cryptd intel_cstate iwlmvm intel_rapl_perf uvcvideo
> mac80211 evdev input_leds mac_hid videobuf2_vmalloc drm_kms_helper pcspkr
> videobuf2_memops videobuf2_v4l2 videobuf2_core psmouse btusb serio_raw btrtl
> drm btbcm btintel iwlwifi videodev snd_hda_intel i2c_i801 snd_hda_codec
> bluetooth intel_gtt
> [27168.539869]  syscopyarea media sysfillrect sysimgblt cfg80211
> snd_soc_rt5640 fb_sys_fops i2c_algo_bit mei_me crc16 e1000e lpc_ich mei
> snd_hda_core snd_soc_rl6231 parport_pc snd_soc_ssm4567 ptp snd_soc_core
> parport pps_core snd_hwdep shpchp wmi snd_compress thermal battery
> int3403_thermal dell_smo8800 fan snd_pcm_dmaengine ac97_bus snd_pcm
> snd_soc_sst_acpi snd_soc_sst_match snd_timer elan_i2c fjes i2c_hid 8250_dw
> snd hid soundcore spi_pxa2xx_platform processor_thermal_device
> i2c_designware_platform dell_rbtn int3406_thermal i2c_designware_core
> int3400_thermal video tpm_tis acpi_als kfifo_buf industrialio tpm
> int3402_thermal acpi_pad int340x_thermal_zone intel_soc_dts_iosf ac button
> acpi_thermal_rel rfkill sch_fq_codel ip_tables x_tables btrfs xor raid6_pq
> sd_mod atkbd libps2 crc32c_intel ahci libahci
> [27168.543479]  libata scsi_mod ehci_pci ehci_hcd xhci_pci xhci_hcd
> sdhci_pci usbcore usb_common i8042 serio sdhci_acpi sdhci led_class mmc_core
> [27168.546045] CPU: 2 PID: 1012 Comm: mozStorage #1 Tainted: G U
> 4.7.2-1-ARCH #1
> [27168.547318] Hardware name: Dell Inc. Latitude E7450/0D8H72, BIOS A06
> 08/31/2015
> [27168.548587] task: ffff8800c2afaac0 ti: ffff8800c1344000 task.ti:
> ffff8800c1344000
> [27168.549845] RIP: 0010:[<ffffffffa01f5661>] [<ffffffffa01f5661>]
> btrfs_search_forward+0x251/0x330 [btrfs]
> [27168.551109] RSP: 0018:ffff8800c1347a90  EFLAGS: 00010246
> [27168.552368] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [27168.553644] RDX: 0000000000000001 RSI: 0000000000000000 RDI:
> ffff8800da2251a0
> [27168.554910] RBP: ffff8800c1347af8 R08: 0000000701400000 R09:
> 0000000741400000
> [27168.556176] R10: 0000000000000000 R11: ffffffff81a7ba4d R12:
> ffff8800b310fc40
> [27168.557447] R13: 0000000000000001 R14: 0000000000000000 R15:
> 000000000003a9f1
> [27168.558718] FS:  00007f616b2ff700(0000) GS:ffff88021e500000(0000)
> knlGS:0000000000000000
> [27168.560040] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [27168.561308] CR2: 00007f615d790000 CR3: 00000001f7432000 CR4:
> 00000000003406e0
> [27168.562574] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [27168.563841] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
> 0000000000000400
> [27168.565087] Stack:
> [27168.566321]  01b16b0056bdd1d0 00ff480000000000 ffff8800c1347bc3
> ffff880214045000
> [27168.567561]  46000000000000b1 010000000000431b 0000000000000000
> 00000000f358f9a8
> [27168.568797]  ffff8801f2660a50 ffff8800da101688 ffff8801f2660a50
> ffff8800b310fc40
> [27168.570033] Call Trace:
> [27168.571270]  [<ffffffffa029b2cd>] btrfs_log_inode+0x394/0xb57 [btrfs]
> [27168.572498]  [<ffffffffa021af50>] ? do_async_commit+0x40/0x40 [btrfs]
> [27168.573723]  [<ffffffffa029bc81>] log_new_dir_dentries+0x1f1/0x528
> [btrfs]
> [27168.574946]  [<ffffffffa025e385>] btrfs_log_inode_parent+0x8b5/0x960
> [btrfs]
> [27168.576163]  [<ffffffffa0217a98>] ? wait_current_trans.isra.8+0x58/0x110
> [btrfs]
> [27168.577384]  [<ffffffffa02173c2>] ? join_transaction.isra.2+0x112/0x410
> [btrfs]
> [27168.578606]  [<ffffffffa025f422>] btrfs_log_dentry_safe+0x62/0x80 [btrfs]
> [27168.579811]  [<ffffffffa023230f>] btrfs_sync_file+0x2af/0x3f0 [btrfs]
> [27168.580986]  [<ffffffff8122db6b>] vfs_fsync_range+0x4b/0xb0
> [27168.582158]  [<ffffffff8122dc2d>] do_fsync+0x3d/0x70
> [27168.583363]  [<ffffffff8122dee3>] SyS_fdatasync+0x13/0x20
> [27168.584593]  [<ffffffff815de7b2>] entry_SYSCALL_64_fastpath+0x1a/0xa4
> [27168.585759] Code: 48 8b 55 a8 48 8b 7d b0 4d 89 f8 44 89 e9 4c 89 e6 e8
> 44 fb ff ff 85 c0 0f 85 c1 00 00 00 4c 89 e7 e8 d4 7d ff ff e9 f8 fd ff ff
> <0f> 0b 48 8d 04 92 43 89 54 ac 40 48 8d 75 bf b9 11 00 00 00 48
> [27168.587061] RIP  [<ffffffffa01f5661>] btrfs_search_forward+0x251/0x330
> [btrfs]
> [27168.588311]  RSP <ffff8800c1347a90>
> [27168.595772] ---[ end trace a1ed8484eb98289b ]---
> [27393.213272] systemd[1]: systemd-journald.service: State 'stop-sigabrt'
> timed out. Terminating.
> 
> $ uname -a
> Linux dell 4.7.2-1-ARCH #1 SMP PREEMPT Sat Aug 20 23:02:56 CEST 2016 x86_64
> GNU/Linux
> $ btrfs --version
> btrfs-progs v4.7
> $ btrfs fi show
> $ btrfs fi df /
> Data, single: total=107.01GiB, used=78.94GiB
> System, single: total=32.00MiB, used=16.00KiB
> Metadata, single: total=2.01GiB, used=639.95MiB
> GlobalReserve, single: total=224.00MiB, used=0.00B
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html