* [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists @ 2016-08-28 13:29 Kai Krakow 2017-02-02 12:01 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Kai Krakow @ 2016-08-28 13:29 UTC (permalink / raw) To: linux-btrfs Hello list! It happened again. While using VirtualBox the following crash happened, btrfs check found a lot of errors which it couldn't repair. Earlier that day my system crashed which may already introduced errors into my filesystem. Apparently, I couldn't create an image (not enough space available), I only can give this trace from dmesg: [44819.903435] ------------[ cut here ]------------ [44819.903443] WARNING: CPU: 3 PID: 2787 at fs/btrfs/extent-tree.c:2963 btrfs_run_delayed_refs+0x26c/0x290 [44819.903444] BTRFS: Transaction aborted (error -17) [44819.903445] Modules linked in: nls_iso8859_15 nls_cp437 vfat fat fuse rfcomm veth af_packet ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack bridge stp llc w83627ehf bnep hwmon_vid cachefiles btusb btintel bluetooth snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec rfkill snd_hwdep snd_hda_core snd_pcm snd_timer coretemp hwmon snd r8169 mii kvm_intel kvm iTCO_wdt iTCO_vendor_support rtc_cmos irqbypass soundcore ip_tables uas usb_storage nvidia_drm(PO) vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) nvidia_modeset(PO) nvidia(PO) efivarfs unix ipv6 [44819.903484] CPU: 3 PID: 2787 Comm: BrowserBlocking Tainted: P O 4.7.2-gentoo #2 [44819.903485] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z68 Pro3, BIOS L2.16A 02/22/2013 [44819.903487] 0000000000000000 ffffffff8130af2d ffff8800b7d03d20 0000000000000000 [44819.903489] ffffffff810865fa ffff880409374428 ffff8800b7d03d70 ffff8803bf299760 [44819.903491] 0000000000000000 00000000ffffffef ffff8803f677f000 ffffffff8108666a [44819.903493] Call Trace: [44819.903496] [<ffffffff8130af2d>] ? dump_stack+0x46/0x59 [44819.903500] [<ffffffff810865fa>] ? __warn+0xba/0xe0 [44819.903502] [<ffffffff8108666a>] ? warn_slowpath_fmt+0x4a/0x50 [44819.903504] [<ffffffff8121351c>] ? btrfs_run_delayed_refs+0x26c/0x290 [44819.903507] [<ffffffff811feb1e>] ? btrfs_release_path+0xe/0x80 [44819.903509] [<ffffffff81216afa>] ? btrfs_start_dirty_block_groups+0x2da/0x420 [44819.903511] [<ffffffff812279f3>] ? btrfs_commit_transaction+0x143/0x990 [44819.903514] [<ffffffff8116a2c5>] ? kmem_cache_free+0x165/0x180 [44819.903516] [<ffffffff8124396c>] ? btrfs_wait_ordered_range+0x7c/0x110 [44819.903518] [<ffffffff8123ecf6>] ? btrfs_sync_file+0x286/0x360 [44819.903522] [<ffffffff811ae343>] ? do_fsync+0x33/0x60 [44819.903524] [<ffffffff811ae57a>] ? SyS_fdatasync+0xa/0x10 [44819.903528] [<ffffffff8162299b>] ? entry_SYSCALL_64_fastpath+0x13/0x8f [44819.903529] ---[ end trace 6944811e170a0e57 ]--- [44819.903531] BTRFS: error (device bcache2) in btrfs_run_delayed_refs:2963: errno=-17 Object already exists [44819.903533] BTRFS info (device bcache2): forced readonly Since I had to get back up and running fast, I restored from backup. I now bought some extra 3TB backup space and created a rescue system including all tools on a USB stick, so next time it happens I may be able to create an image of the broken filesystem. Btrfs --repair refused to repair the filesystem telling me something about compressed extents and an unsupported case, wanting me to take an image and send it to the devs. *sigh* System is kernel 4.7.2, Gentoo Linux, latest VirtualBox stable. VirtualBox was using VDI image format without nocow. I now reverted back to using nocow on VDI files and hope it doesn't strike again too soon. I didn't try again yet, first I need to refresh my backup which takes a while. The filesystem runs on 3x SATA 1TB mraid1 draid0 through bcache in writeback mode, backed by a 500GB 850 Evo - if that matters. The problem occurred during high IO on 4.7.2. I previously ran 4.6.6 which didn't show this problem. Part of the culprit may be that I was using bfq patches - I removed them for now and went back to deadline io scheduler. The bfq patches froze my system a few times when I booted 4.7.2 which may already have broken my btrfs (although it shouldn't, right? btrfs is transactional). Last time this happened (on an earlier kernel), bfq may have been part of the problem, too. So I think bfq does something to btrfs which may break the fs, or at least interferes badly with the transaction as otherwise it shouldn't break. You may want to run your test suites with bfq also (or different io schedulers in general). My home partition is mounted as a subvolume: /dev/bcache0 on /home type btrfs (rw,noatime,compress=lzo,nossd,space_cache,autodefrag,subvolid=261,subvol=/home) -- Regards, Kai Replies to list-only preferred. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2016-08-28 13:29 [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists Kai Krakow @ 2017-02-02 12:01 ` Marc Joliet 2017-02-03 22:44 ` Kai Krakow 2017-03-02 0:43 ` Qu Wenruo 0 siblings, 2 replies; 23+ messages in thread From: Marc Joliet @ 2017-02-02 12:01 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 21084 bytes --] On Sunday 28 August 2016 15:29:08 Kai Krakow wrote: > Hello list! Hi list > It happened again. While using VirtualBox the following crash happened, > btrfs check found a lot of errors which it couldn't repair. Earlier > that day my system crashed which may already introduced errors into my > filesystem. Apparently, I couldn't create an image (not enough space > available), I only can give this trace from dmesg: > > [44819.903435] ------------[ cut here ]------------ > [44819.903443] WARNING: CPU: 3 PID: 2787 at fs/btrfs/extent-tree.c:2963 > btrfs_run_delayed_refs+0x26c/0x290 [44819.903444] BTRFS: Transaction > aborted (error -17) > [44819.903445] Modules linked in: nls_iso8859_15 nls_cp437 vfat fat fuse > rfcomm veth af_packet ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat > nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack bridge stp > llc w83627ehf bnep hwmon_vid cachefiles btusb btintel bluetooth > snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic > snd_hda_intel snd_hda_codec rfkill snd_hwdep snd_hda_core snd_pcm snd_timer > coretemp hwmon snd r8169 mii kvm_intel kvm iTCO_wdt iTCO_vendor_support > rtc_cmos irqbypass soundcore ip_tables uas usb_storage nvidia_drm(PO) > vboxpci(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O) nvidia_modeset(PO) > nvidia(PO) efivarfs unix ipv6 [44819.903484] CPU: 3 PID: 2787 Comm: > BrowserBlocking Tainted: P O 4.7.2-gentoo #2 [44819.903485] > Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z68 Pro3, BIOS > L2.16A 02/22/2013 [44819.903487] 0000000000000000 ffffffff8130af2d > ffff8800b7d03d20 0000000000000000 [44819.903489] ffffffff810865fa > ffff880409374428 ffff8800b7d03d70 ffff8803bf299760 [44819.903491] > 0000000000000000 00000000ffffffef ffff8803f677f000 ffffffff8108666a > [44819.903493] Call Trace: > [44819.903496] [<ffffffff8130af2d>] ? dump_stack+0x46/0x59 > [44819.903500] [<ffffffff810865fa>] ? __warn+0xba/0xe0 > [44819.903502] [<ffffffff8108666a>] ? warn_slowpath_fmt+0x4a/0x50 > [44819.903504] [<ffffffff8121351c>] ? btrfs_run_delayed_refs+0x26c/0x290 > [44819.903507] [<ffffffff811feb1e>] ? btrfs_release_path+0xe/0x80 > [44819.903509] [<ffffffff81216afa>] ? > btrfs_start_dirty_block_groups+0x2da/0x420 [44819.903511] > [<ffffffff812279f3>] ? btrfs_commit_transaction+0x143/0x990 [44819.903514] > [<ffffffff8116a2c5>] ? kmem_cache_free+0x165/0x180 [44819.903516] > [<ffffffff8124396c>] ? btrfs_wait_ordered_range+0x7c/0x110 [44819.903518] > [<ffffffff8123ecf6>] ? btrfs_sync_file+0x286/0x360 [44819.903522] > [<ffffffff811ae343>] ? do_fsync+0x33/0x60 > [44819.903524] [<ffffffff811ae57a>] ? SyS_fdatasync+0xa/0x10 > [44819.903528] [<ffffffff8162299b>] ? entry_SYSCALL_64_fastpath+0x13/0x8f > [44819.903529] ---[ end trace 6944811e170a0e57 ]--- > [44819.903531] BTRFS: error (device bcache2) in btrfs_run_delayed_refs:2963: > errno=-17 Object already exists [44819.903533] BTRFS info (device bcache2): > forced readonly I got the same error myself, with this stack trace: -- Logs begin at Fr 2016-04-01 17:07:28 CEST, end at Mi 2017-02-01 22:03:57 CET. -- Feb 01 01:46:26 diefledermaus kernel: ------------[ cut here ]------------ Feb 01 01:46:26 diefledermaus kernel: WARNING: CPU: 1 PID: 16727 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 01 01:46:26 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 01 01:46:26 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 01 01:46:27 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 01 01:46:27 diefledermaus kernel: Modules linked in: msr ctr ccm tun arc4 snd_hda_codec_idt applesmc snd_hda_codec_generic input_polldev hwmon snd_hda_intel ath5k snd_hda_codec mac80211 snd_hda_core ath snd_pcm cfg80211 snd_timer video acpi_cpufreq snd backlight sky2 rfkill processor button soundcore sg usb_storage sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 01 01:46:27 diefledermaus kernel: CPU: 1 PID: 16727 Comm: kworker/u4:0 Not tainted 4.9.6-gentoo #1 Feb 01 01:46:27 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 01 01:46:27 diefledermaus kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper Feb 01 01:46:27 diefledermaus kernel: 0000000000000000 ffffffff812cf739 ffffc9000285fd60 0000000000000000 Feb 01 01:46:27 diefledermaus kernel: ffffffff8104908a ffff8800428df1e0 ffffc9000285fdb0 0000000000000020 Feb 01 01:46:27 diefledermaus kernel: ffff880003c1b1b8 ffff8800bb73e900 0000000000000000 ffffffff810490fa Feb 01 01:46:27 diefledermaus kernel: Call Trace: Feb 01 01:46:27 diefledermaus kernel: [<ffffffff812cf739>] ? dump_stack+0x46/0x5d Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81201fa8>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81202064>] ? delayed_ref_async_start+0x84/0xa0 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105ce76>] ? process_one_work+0x126/0x310 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105a893>] ? pwq_activate_delayed_work+0x33/0x60 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105d0a5>] ? worker_thread+0x45/0x450 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8104c7e7>] ? do_group_exit+0x37/0xa0 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81061c99>] ? kthread+0xb9/0xd0 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81061be0>] ? kthread_park+0x50/0x50 Feb 01 01:46:27 diefledermaus kernel: [<ffffffff814f6252>] ? ret_from_fork+0x22/0x30 Feb 01 01:46:27 diefledermaus kernel: ---[ end trace 5d090b8f212318e0 ]--- Feb 01 01:46:27 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 01 07:33:52 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:52 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:33 diefledermaus kernel: verify_parent_transid: 14 callbacks suppressed Feb 01 09:03:34 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:34 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:34 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:34 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:36 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:36 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: verify_parent_transid: 10 callbacks suppressed Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:10 diefledermaus kernel: verify_parent_transid: 14 callbacks suppressed Feb 01 09:04:10 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:10 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:10 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:10 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:19 diefledermaus kernel: verify_parent_transid: 14 callbacks suppressed Feb 01 09:04:19 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:04:19 diefledermaus kernel: BTRFS error (device sdb2): parent transid verify failed on 4722283290624 wanted 283904 found 194340 Feb 01 09:10:20 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 01 09:11:41 diefledermaus kernel: usb 1-6: USB disconnect, device number 2 Feb 01 09:11:41 diefledermaus kernel: sd 6:0:0:0: [sdb] Synchronizing SCSI cache Feb 01 09:11:41 diefledermaus kernel: sd 6:0:0:0: [sdb] Synchronize Cache(10) failed: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK (This is the entire dmesg output for that day up to that time. The non-dmesg log before that is just getmail running (on a different FS) with the previous entries a few minutes beforehand, and afterwards it's just a bunch of SSH related messages from btrbk trying to continue the backups, followed by unrelated log entries.) > Since I had to get back up and running fast, I restored from backup. I > now bought some extra 3TB backup space and created a rescue system > including all tools on a USB stick, so next time it happens I may be > able to create an image of the broken filesystem. I wanted to create an image, but btrfs-image segfaults after a few seconds. > Btrfs --repair refused to repair the filesystem telling me something > about compressed extents and an unsupported case, wanting me to take an > image and send it to the devs. *sigh* I haven't tried a repair yet; it's a big file system, and btrfs-check is still running: # btrfs check -p /dev/sdd2 Checking filesystem on /dev/sdd2 UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 parent transid verify failed on 3829276291072 wanted 224274 found 283858 parent transid verify failed on 3829276291072 wanted 224274 found 283858 parent transid verify failed on 3829276291072 wanted 224274 found 283858 parent transid verify failed on 3829276291072 wanted 224274 found 283858 Ignoring transid failure leaf parent key incorrect 3829276291072 bad block 3829276291072 ERROR: errors found in extent allocation tree or chunk allocation block group 4722282987520 has wrong amount of free space failed to load free space cache for block group 4722282987520 checking free space cache [O] root 32018 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32089 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32091 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32092 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32107 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32189 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32190 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32191 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32265 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32266 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32409 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32410 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32411 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32412 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32413 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32631 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32632 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32633 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32634 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32635 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32636 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32718 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32732 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 checking fs roots [o] I know that the "file extend discount" errors are fixable from my previous email to this ML, but what about the rest? From looking through the ML archives it seems that --repair won't be able to fix the transid failures. It seems that one person had success with the "usebackuproot" mount option, which I haven't tried yet. > System is kernel 4.7.2, Gentoo Linux, latest VirtualBox stable. > VirtualBox was using VDI image format without nocow. I now reverted > back to using nocow on VDI files and hope it doesn't strike again too > soon. I didn't try again yet, first I need to refresh my backup which > takes a while. > > The filesystem runs on 3x SATA 1TB mraid1 draid0 through bcache in > writeback mode, backed by a 500GB 850 Evo - if that matters. > > The problem occurred during high IO on 4.7.2. I previously ran 4.6.6 > which didn't show this problem. Part of the culprit may be that I was > using bfq patches - I removed them for now and went back to deadline io > scheduler. The bfq patches froze my system a few times when I booted > 4.7.2 which may already have broken my btrfs (although it shouldn't, > right? btrfs is transactional). Last time this happened (on an earlier > kernel), bfq may have been part of the problem, too. So I think bfq > does something to btrfs which may break the fs, or at least interferes > badly with the transaction as otherwise it shouldn't break. You may > want to run your test suites with bfq also (or different io schedulers > in general). > > My home partition is mounted as a subvolume: > /dev/bcache0 on /home type btrfs > (rw,noatime,compress=lzo,nossd,space_cache,autodefrag,subvolid=261,subvol=/ > home) The system the drive runs on is: % uname -a Linux diefledermaus 4.9.7-gentoo #1 SMP Wed Feb 1 23:52:56 CET 2017 x86_64 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux However during the crash it was running 4.9.6-gentoo. The system uses the standard CFQ scheduler, so perhaps BFQ is not at fault in Kai's case. The system I am running btrfs-check on is: % uname -a Linux thetick 4.9.6-gentoo #1 SMP PREEMPT Fri Jan 27 00:50:02 CET 2017 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux Both have btrfs-progs 4.9: % /sbin/btrfs --version btrfs-progs v4.9 And the file system in question: % sudo /sbin/btrfs fi show /dev/sdd2 Label: 'MARCEC_BACKUP' uuid: f97b3cda-15e8-418b-bb9b-235391ef2a38 Total devices 1 FS bytes used 842.50GiB devid 1 size 976.56GiB used 877.31GiB path /dev/sdd2 The file system is mounted with "noatime,compress,comment=systemd.automount". In my case the crash also happened during high I/O load (three btrfs- send/receive backups running at the same time). If "usebackuproot" (now called "recovery"?) fails, then I'll just wipe the FS and start the backups from scratch. Since I would like to have that done by Saturday: is there any information I can provide that might help fix whatever bug(s) caused this? Should I file a bug if one doesn't exist yet (I haven't checked yet, sorry)? Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-02 12:01 ` Marc Joliet @ 2017-02-03 22:44 ` Kai Krakow 2017-02-10 22:15 ` Marc Joliet 2017-03-02 0:43 ` Qu Wenruo 1 sibling, 1 reply; 23+ messages in thread From: Kai Krakow @ 2017-02-03 22:44 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 21975 bytes --] Am Thu, 02 Feb 2017 13:01:03 +0100 schrieb Marc Joliet <marcec@gmx.de>: > On Sunday 28 August 2016 15:29:08 Kai Krakow wrote: > > Hello list! > > Hi list > > > It happened again. While using VirtualBox the following crash > > happened, btrfs check found a lot of errors which it couldn't > > repair. Earlier that day my system crashed which may already > > introduced errors into my filesystem. Apparently, I couldn't create > > an image (not enough space available), I only can give this trace > > from dmesg: > > > > [44819.903435] ------------[ cut here ]------------ > > [44819.903443] WARNING: CPU: 3 PID: 2787 at > > fs/btrfs/extent-tree.c:2963 btrfs_run_delayed_refs+0x26c/0x290 > > [44819.903444] BTRFS: Transaction aborted (error -17) > > [44819.903445] Modules linked in: nls_iso8859_15 nls_cp437 vfat fat > > fuse rfcomm veth af_packet ipt_MASQUERADE nf_nat_masquerade_ipv4 > > iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat > > nf_conntrack bridge stp llc w83627ehf bnep hwmon_vid cachefiles > > btusb btintel bluetooth snd_hda_codec_hdmi snd_hda_codec_realtek > > snd_hda_codec_generic snd_hda_intel snd_hda_codec rfkill snd_hwdep > > snd_hda_core snd_pcm snd_timer coretemp hwmon snd r8169 mii > > kvm_intel kvm iTCO_wdt iTCO_vendor_support rtc_cmos irqbypass > > soundcore ip_tables uas usb_storage nvidia_drm(PO) vboxpci(O) > > vboxnetadp(O) vboxnetflt(O) vboxdrv(O) nvidia_modeset(PO) > > nvidia(PO) efivarfs unix ipv6 [44819.903484] CPU: 3 PID: 2787 Comm: > > BrowserBlocking Tainted: P O 4.7.2-gentoo #2 > > [44819.903485] Hardware name: To Be Filled By O.E.M. To Be Filled > > By O.E.M./Z68 Pro3, BIOS L2.16A 02/22/2013 [44819.903487] > > 0000000000000000 ffffffff8130af2d ffff8800b7d03d20 0000000000000000 > > [44819.903489] ffffffff810865fa ffff880409374428 ffff8800b7d03d70 > > ffff8803bf299760 [44819.903491] 0000000000000000 00000000ffffffef > > ffff8803f677f000 ffffffff8108666a [44819.903493] Call Trace: > > [44819.903496] [<ffffffff8130af2d>] ? dump_stack+0x46/0x59 > > [44819.903500] [<ffffffff810865fa>] ? __warn+0xba/0xe0 > > [44819.903502] [<ffffffff8108666a>] ? warn_slowpath_fmt+0x4a/0x50 > > [44819.903504] [<ffffffff8121351c>] ? > > btrfs_run_delayed_refs+0x26c/0x290 [44819.903507] > > [<ffffffff811feb1e>] ? btrfs_release_path+0xe/0x80 [44819.903509] > > [<ffffffff81216afa>] ? btrfs_start_dirty_block_groups+0x2da/0x420 > > [44819.903511] [<ffffffff812279f3>] ? > > btrfs_commit_transaction+0x143/0x990 [44819.903514] > > [<ffffffff8116a2c5>] ? kmem_cache_free+0x165/0x180 [44819.903516] > > [<ffffffff8124396c>] ? btrfs_wait_ordered_range+0x7c/0x110 > > [44819.903518] [<ffffffff8123ecf6>] ? btrfs_sync_file+0x286/0x360 > > [44819.903522] [<ffffffff811ae343>] ? do_fsync+0x33/0x60 > > [44819.903524] [<ffffffff811ae57a>] ? SyS_fdatasync+0xa/0x10 > > [44819.903528] [<ffffffff8162299b>] ? > > entry_SYSCALL_64_fastpath+0x13/0x8f [44819.903529] ---[ end trace > > 6944811e170a0e57 ]--- [44819.903531] BTRFS: error (device bcache2) > > in btrfs_run_delayed_refs:2963: errno=-17 Object already exists > > [44819.903533] BTRFS info (device bcache2): forced readonly > > I got the same error myself, with this stack trace: > > -- Logs begin at Fr 2016-04-01 17:07:28 CEST, end at Mi 2017-02-01 > 22:03:57 CET. -- > Feb 01 01:46:26 diefledermaus kernel: ------------[ cut here > ]------------ Feb 01 01:46:26 diefledermaus kernel: WARNING: CPU: 1 > PID: 16727 at fs/btrfs/extent-tree.c:2967 > btrfs_run_delayed_refs+0x278/0x2b0 Feb 01 01:46:26 diefledermaus > kernel: BTRFS: Transaction aborted (error -17) Feb 01 01:46:26 > diefledermaus kernel: BTRFS: error (device sdb2) in > btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 01 > 01:46:27 diefledermaus kernel: BTRFS info (device sdb2): forced > readonly Feb 01 01:46:27 diefledermaus kernel: Modules linked in: msr > ctr ccm tun arc4 snd_hda_codec_idt applesmc snd_hda_codec_generic > input_polldev hwmon snd_hda_intel ath5k snd_hda_codec mac80211 > snd_hda_core ath snd_pcm cfg80211 snd_timer video acpi_cpufreq snd > backlight sky2 rfkill processor button soundcore sg usb_storage > sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix > libata ehci_pci ehci_hcd Feb 01 01:46:27 diefledermaus kernel: CPU: 1 > PID: 16727 Comm: kworker/u4:0 Not tainted 4.9.6-gentoo #1 > Feb 01 01:46:27 diefledermaus kernel: Hardware name: Apple Inc. > Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 > 06/28/07 Feb 01 01:46:27 diefledermaus kernel: Workqueue: > btrfs-extent-refs btrfs_extent_refs_helper > Feb 01 01:46:27 diefledermaus kernel: 0000000000000000 > ffffffff812cf739 ffffc9000285fd60 0000000000000000 > Feb 01 01:46:27 diefledermaus kernel: ffffffff8104908a > ffff8800428df1e0 ffffc9000285fdb0 0000000000000020 > Feb 01 01:46:27 diefledermaus kernel: ffff880003c1b1b8 > ffff8800bb73e900 0000000000000000 ffffffff810490fa > Feb 01 01:46:27 diefledermaus kernel: Call Trace: > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff812cf739>] ? > dump_stack+0x46/0x5d > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8104908a>] ? > __warn+0xba/0xe0 Feb 01 01:46:27 diefledermaus kernel: > [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81201fa8>] ? > btrfs_run_delayed_refs+0x278/0x2b0 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81202064>] ? > delayed_ref_async_start+0x84/0xa0 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105ce76>] ? > process_one_work+0x126/0x310 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105a893>] ? > pwq_activate_delayed_work+0x33/0x60 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105d0a5>] ? > worker_thread+0x45/0x450 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105d060>] ? > process_one_work+0x310/0x310 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8105d060>] ? > process_one_work+0x310/0x310 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff8104c7e7>] ? > do_group_exit+0x37/0xa0 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81061c99>] ? > kthread+0xb9/0xd0 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff81061be0>] ? > kthread_park+0x50/0x50 > Feb 01 01:46:27 diefledermaus kernel: [<ffffffff814f6252>] ? > ret_from_fork+0x22/0x30 > Feb 01 01:46:27 diefledermaus kernel: ---[ end trace 5d090b8f212318e0 > ]--- Feb 01 01:46:27 diefledermaus kernel: BTRFS: error (device sdb2) > in btrfs_run_delayed_refs:2967: errno=-17 Object already exists > Feb 01 07:33:52 diefledermaus kernel: BTRFS error (device sdb2): > parent transid verify failed on 4722283290624 wanted 283904 found > 194340 Feb 01 07:33:52 diefledermaus kernel: BTRFS error (device > sdb2): parent transid verify failed on 4722283290624 wanted 283904 > found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:53 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 07:33:54 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:33 diefledermaus kernel: > verify_parent_transid: 14 callbacks suppressed > Feb 01 09:03:34 diefledermaus kernel: BTRFS error (device sdb2): > parent transid verify failed on 4722283290624 wanted 283904 found > 194340 Feb 01 09:03:34 diefledermaus kernel: BTRFS error (device > sdb2): parent transid verify failed on 4722283290624 wanted 283904 > found 194340 Feb 01 09:03:34 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:34 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:35 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:36 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:36 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: > verify_parent_transid: 10 callbacks suppressed > Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device sdb2): > parent transid verify failed on 4722283290624 wanted 283904 found > 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error (device > sdb2): parent transid verify failed on 4722283290624 wanted 283904 > found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:03:39 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:10 diefledermaus kernel: > verify_parent_transid: 14 callbacks suppressed > Feb 01 09:04:10 diefledermaus kernel: BTRFS error (device sdb2): > parent transid verify failed on 4722283290624 wanted 283904 found > 194340 Feb 01 09:04:10 diefledermaus kernel: BTRFS error (device > sdb2): parent transid verify failed on 4722283290624 wanted 283904 > found 194340 Feb 01 09:04:10 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:10 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:11 diefledermaus kernel: BTRFS error > (device sdb2): parent transid verify failed on 4722283290624 wanted > 283904 found 194340 Feb 01 09:04:19 diefledermaus kernel: > verify_parent_transid: 14 callbacks suppressed > Feb 01 09:04:19 diefledermaus kernel: BTRFS error (device sdb2): > parent transid verify failed on 4722283290624 wanted 283904 found > 194340 Feb 01 09:04:19 diefledermaus kernel: BTRFS error (device > sdb2): parent transid verify failed on 4722283290624 wanted 283904 > found 194340 Feb 01 09:10:20 diefledermaus kernel: BTRFS error > (device sdb2): cleaner transaction attach returned -30 > Feb 01 09:11:41 diefledermaus kernel: usb 1-6: USB disconnect, device > number 2 Feb 01 09:11:41 diefledermaus kernel: sd 6:0:0:0: [sdb] > Synchronizing SCSI cache > Feb 01 09:11:41 diefledermaus kernel: sd 6:0:0:0: [sdb] Synchronize > Cache(10) failed: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK > > (This is the entire dmesg output for that day up to that time. The > non-dmesg log before that is just getmail running (on a different FS) > with the previous entries a few minutes beforehand, and afterwards > it's just a bunch of SSH related messages from btrbk trying to > continue the backups, followed by unrelated log entries.) > > > Since I had to get back up and running fast, I restored from > > backup. I now bought some extra 3TB backup space and created a > > rescue system including all tools on a USB stick, so next time it > > happens I may be able to create an image of the broken filesystem. > > I wanted to create an image, but btrfs-image segfaults after a few > seconds. > > > Btrfs --repair refused to repair the filesystem telling me something > > about compressed extents and an unsupported case, wanting me to > > take an image and send it to the devs. *sigh* > > I haven't tried a repair yet; it's a big file system, and btrfs-check > is still running: > > # btrfs check -p /dev/sdd2 > Checking filesystem on /dev/sdd2 > UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 > parent transid verify failed on 3829276291072 wanted 224274 found > 283858 parent transid verify failed on 3829276291072 wanted 224274 > found 283858 parent transid verify failed on 3829276291072 wanted > 224274 found 283858 parent transid verify failed on 3829276291072 > wanted 224274 found 283858 Ignoring transid failure > leaf parent key incorrect 3829276291072 > bad block 3829276291072 > > ERROR: errors found in extent allocation tree or chunk allocation > block group 4722282987520 has wrong amount of free space > failed to load free space cache for block group 4722282987520 > checking free space cache [O] > root 32018 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32089 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32091 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32092 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32107 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32189 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32190 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32191 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32265 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32266 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32409 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32410 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32411 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32412 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32413 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32631 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32632 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32633 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32634 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32635 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32636 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32718 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32732 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > checking fs roots [o] > > I know that the "file extend discount" errors are fixable from my > previous email to this ML, but what about the rest? From looking > through the ML archives it seems that --repair won't be able to fix > the transid failures. It seems that one person had success with the > "usebackuproot" mount option, which I haven't tried yet. > > > System is kernel 4.7.2, Gentoo Linux, latest VirtualBox stable. > > VirtualBox was using VDI image format without nocow. I now reverted > > back to using nocow on VDI files and hope it doesn't strike again > > too soon. I didn't try again yet, first I need to refresh my backup > > which takes a while. > > > > The filesystem runs on 3x SATA 1TB mraid1 draid0 through bcache in > > writeback mode, backed by a 500GB 850 Evo - if that matters. > > > > The problem occurred during high IO on 4.7.2. I previously ran 4.6.6 > > which didn't show this problem. Part of the culprit may be that I > > was using bfq patches - I removed them for now and went back to > > deadline io scheduler. The bfq patches froze my system a few times > > when I booted 4.7.2 which may already have broken my btrfs > > (although it shouldn't, right? btrfs is transactional). Last time > > this happened (on an earlier kernel), bfq may have been part of the > > problem, too. So I think bfq does something to btrfs which may > > break the fs, or at least interferes badly with the transaction as > > otherwise it shouldn't break. You may want to run your test suites > > with bfq also (or different io schedulers in general). > > > > My home partition is mounted as a subvolume: > > /dev/bcache0 on /home type btrfs > > (rw,noatime,compress=lzo,nossd,space_cache,autodefrag,subvolid=261,subvol=/ > > home) > > The system the drive runs on is: > > % uname -a > Linux diefledermaus 4.9.7-gentoo #1 SMP Wed Feb 1 23:52:56 CET 2017 > x86_64 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux > > However during the crash it was running 4.9.6-gentoo. The system > uses the standard CFQ scheduler, so perhaps BFQ is not at fault in > Kai's case. > > The system I am running btrfs-check on is: > > % uname -a > Linux thetick 4.9.6-gentoo #1 SMP PREEMPT Fri Jan 27 00:50:02 CET > 2017 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ > AuthenticAMD GNU/Linux > > Both have btrfs-progs 4.9: > > % /sbin/btrfs --version > btrfs-progs v4.9 > > And the file system in question: > > % sudo /sbin/btrfs fi show /dev/sdd2 > Label: 'MARCEC_BACKUP' uuid: > f97b3cda-15e8-418b-bb9b-235391ef2a38 Total devices 1 FS bytes used > 842.50GiB devid 1 size 976.56GiB used 877.31GiB path /dev/sdd2 > > The file system is mounted with > "noatime,compress,comment=systemd.automount". So our common denominator is only "compress". Well, and systemd and Gentoo - but that probably doesn't count. It looks like you're running without bcache and on a single disk. Do you use files that should maybe better be marked nocow? (chattr +C)... in terms like VirtualBox images or database files? I'm running 4.9.6-gentoo also and this problem didn't happen again yet. And I switched from deadline back to cfq (but I will probably revert this). I'm backing up with borgbackup to an internal drive running XFS, this runs flawless so far and without major performance impacts. > In my case the crash also happened during high I/O load (three btrfs- > send/receive backups running at the same time). If > "usebackuproot" (now called "recovery"?) fails, then I'll just wipe > the FS and start the backups from scratch. > > Since I would like to have that done by Saturday: is there any > information I can provide that might help fix whatever bug(s) caused > this? Should I file a bug if one doesn't exist yet (I haven't > checked yet, sorry)? > > Greetings -- Regards, Kai Replies to list-only preferred. [-- Attachment #2: Digitale Signatur von OpenPGP --] [-- Type: application/pgp-signature, Size: 163 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-03 22:44 ` Kai Krakow @ 2017-02-10 22:15 ` Marc Joliet 2017-02-11 2:01 ` Kai Krakow 0 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-02-10 22:15 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 20371 bytes --] Sorry for the late reply, see below for why :) . On Friday 03 February 2017 23:44:10 Kai Krakow wrote: > Am Thu, 02 Feb 2017 13:01:03 +0100 > > schrieb Marc Joliet <marcec@gmx.de>: [...] > > > Btrfs --repair refused to repair the filesystem telling me something > > > about compressed extents and an unsupported case, wanting me to > > > take an image and send it to the devs. *sigh* > > > > I haven't tried a repair yet; it's a big file system, and btrfs-check > > is still running: > > > > # btrfs check -p /dev/sdd2 > > Checking filesystem on /dev/sdd2 > > UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 > > parent transid verify failed on 3829276291072 wanted 224274 found > > 283858 parent transid verify failed on 3829276291072 wanted 224274 > > found 283858 parent transid verify failed on 3829276291072 wanted > > 224274 found 283858 parent transid verify failed on 3829276291072 > > wanted 224274 found 283858 Ignoring transid failure > > leaf parent key incorrect 3829276291072 > > bad block 3829276291072 > > > > ERROR: errors found in extent allocation tree or chunk allocation > > block group 4722282987520 has wrong amount of free space > > failed to load free space cache for block group 4722282987520 > > checking free space cache [O] > > root 32018 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32089 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32091 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32092 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32107 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32189 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32190 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32191 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32265 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32266 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32409 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32410 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32411 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32412 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32413 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32631 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32632 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32633 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32634 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32635 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32636 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32718 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32732 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > checking fs roots [o] > > > > I know that the "file extend discount" errors are fixable from my > > previous email to this ML, but what about the rest? From looking > > through the ML archives it seems that --repair won't be able to fix > > the transid failures. It seems that one person had success with the > > "usebackuproot" mount option, which I haven't tried yet. Well, it turns out that the problem fixed itself. After check finished, I tried mounting the FS normally, which succeeded. Shortly after, btrfs-cleaner started removing deleted snapshots. About 40 minutes later I got the error again: Feb 02 22:49:14 thetick kernel: BTRFS: device label MARCEC_BACKUP devid 1 transid 283903 /dev/sdd2 Feb 02 22:49:19 thetick kernel: EXT4-fs (sdd1): mounted filesystem with ordered data mode. Opts: (null) Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): use zlib compression Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): disk space caching is enabled Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): has skinny extents Feb 03 00:20:09 thetick kernel: BTRFS info (device sdd2): The free space cache file (3967375376384) is invalid. skip it Feb 03 01:05:58 thetick kernel: ------------[ cut here ]------------ Feb 03 01:05:58 thetick kernel: WARNING: CPU: 1 PID: 26544 at fs/btrfs/extent- tree.c:2967 btrfs_run_delayed_refs+0x26c/0x290 Feb 03 01:05:58 thetick kernel: BTRFS: Transaction aborted (error -17) Feb 03 01:05:58 thetick kernel: Modules linked in: dummy iptable_filter ip_tables x_tables joydev hid_logitech_hidpp hid_logitech_dj snd_hda_codec_hdmi snd_hda_codec_analog snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core sn Feb 03 01:05:58 thetick kernel: ehci_pci ohci_hcd ehci_hcd Feb 03 01:05:58 thetick kernel: CPU: 1 PID: 26544 Comm: btrfs-cleaner Not tainted 4.9.7-gentoo #1 Feb 03 01:05:58 thetick kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 1701 10/30/2008 Feb 03 01:05:58 thetick kernel: 0000000000000000 ffffffff8137af83 ffffc90000a4fd30 0000000000000000 Feb 03 01:05:58 thetick kernel: ffffffff810537ad ffff880138b2ec80 ffffc90000a4fd80 ffff880138b4e850 Feb 03 01:05:58 thetick kernel: 00000000ffffffef 00000000000000f2 ffff880138b2ec80 ffffffff8105381a Feb 03 01:05:58 thetick kernel: Call Trace: Feb 03 01:05:58 thetick kernel: [<ffffffff8137af83>] ? dump_stack+0x46/0x63 Feb 03 01:05:58 thetick kernel: [<ffffffff810537ad>] ? __warn+0xed/0x110 Feb 03 01:05:58 thetick kernel: [<ffffffff8105381a>] ? warn_slowpath_fmt+0x4a/0x50 Feb 03 01:05:58 thetick kernel: [<ffffffff81286f7a>] ? __btrfs_run_delayed_refs+0x11ba/0x1390 Feb 03 01:05:58 thetick kernel: [<ffffffff8128a2ec>] ? btrfs_run_delayed_refs+0x26c/0x290 Feb 03 01:05:58 thetick kernel: [<ffffffff81287fbe>] ? walk_up_tree+0xde/0x1d0 Feb 03 01:05:58 thetick kernel: [<ffffffff8129f6d5>] ? btrfs_should_end_transaction+0x45/0x80 Feb 03 01:05:58 thetick kernel: [<ffffffff81288851>] ? btrfs_drop_snapshot+0x401/0x7a0 Feb 03 01:05:58 thetick kernel: [<ffffffff812a09f6>] ? btrfs_clean_one_deleted_snapshot+0xb6/0xf0 Feb 03 01:05:58 thetick kernel: [<ffffffff8129757f>] ? cleaner_kthread+0x13f/0x1b0 Feb 03 01:05:58 thetick kernel: [<ffffffff81297440>] ? btrfs_destroy_pinned_extent+0xc0/0xc0 Feb 03 01:05:58 thetick kernel: [<ffffffff81070ee9>] ? kthread+0xb9/0xd0 Feb 03 01:05:58 thetick kernel: [<ffffffff81070e30>] ? kthread_park+0x70/0x70 Feb 03 01:05:58 thetick kernel: [<ffffffff8160dfd2>] ? ret_from_fork+0x22/0x30 Feb 03 01:05:58 thetick kernel: ---[ end trace 9b83fb0ab757cd99 ]--- Feb 03 01:05:58 thetick kernel: BTRFS: error (device sdd2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 03 01:05:58 thetick kernel: BTRFS info (device sdd2): forced readonly Feb 03 01:26:39 thetick kernel: BTRFS error (device sdd2): cleaner transaction attach returned -30 Feb 03 01:26:59 thetick kernel: BTRFS info (device sdd2): use zlib compression Feb 03 01:26:59 thetick kernel: BTRFS info (device sdd2): disk space caching is enabled Feb 03 01:26:59 thetick kernel: BTRFS info (device sdd2): has skinny extents Feb 03 01:27:44 thetick kernel: BTRFS info (device sdd2): The free space cache file (4690842484736) is invalid. skip it I then mounted it again, after which btrfs-cleaner resumed its task: Feb 03 09:54:05 thetick kernel: BTRFS info (device sdd2): use zlib compression Feb 03 09:54:05 thetick kernel: BTRFS info (device sdd2): disk space caching is enabled Feb 03 09:54:05 thetick kernel: BTRFS info (device sdd2): has skinny extents This time it finished successfully, after that there are no more messages pertaining to that file system in dmesg. I then ran btrfs-check again, which didn't find the transid errors any more: # btrfs check -p /dev/sdd2 Checking filesystem on /dev/sdd2 UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 checking extents [o] block group 4722282987520 has wrong amount of free space failed to load free space cache for block group 4722282987520 checking free space cache [o] root 32018 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32089 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32091 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32092 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32107 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32189 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32190 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32191 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32265 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32266 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32409 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32410 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32411 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32412 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32413 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32631 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32632 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32633 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32634 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32635 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32636 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32718 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32732 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32830 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32832 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32833 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32834 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32903 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 32961 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 root 33002 inode 95066 errors 100, file extent discount Found file extent holes: start: 413696, len: 4096 checking fs roots [O] found 901576467411 bytes used err is 1 total csum bytes: 871524676 total tree bytes: 11151159296 total fs tree bytes: 8709533696 total extent tree bytes: 1307844608 btree space waste bytes: 2428036988 file data blocks allocated: 17604037283840 referenced 1003776770048 So, since the error was now limited to something btrfs-check could repair, I ran check with --repair: # btrfs check -p --repair /dev/sdd2 enabling repair mode Checking filesystem on /dev/sdd2 UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 checking extents [O]o] Fixed 0 roots. cache and super generation don't match, space cache will be invalidated Fixed discount file extents for inode: 95066 in root: 32018 Fixed discount file extents for inode: 95066 in root: 32089 Fixed discount file extents for inode: 95066 in root: 32091 Fixed discount file extents for inode: 95066 in root: 32092 Fixed discount file extents for inode: 95066 in root: 32107 Fixed discount file extents for inode: 95066 in root: 32189 Fixed discount file extents for inode: 95066 in root: 32190 Fixed discount file extents for inode: 95066 in root: 32191 Fixed discount file extents for inode: 95066 in root: 32265 Fixed discount file extents for inode: 95066 in root: 32266 Fixed discount file extents for inode: 95066 in root: 32409 Fixed discount file extents for inode: 95066 in root: 32410 Fixed discount file extents for inode: 95066 in root: 32411 Fixed discount file extents for inode: 95066 in root: 32412 Fixed discount file extents for inode: 95066 in root: 32413 Fixed discount file extents for inode: 95066 in root: 32631 Fixed discount file extents for inode: 95066 in root: 32632 Fixed discount file extents for inode: 95066 in root: 32633 Fixed discount file extents for inode: 95066 in root: 32634 Fixed discount file extents for inode: 95066 in root: 32635 Fixed discount file extents for inode: 95066 in root: 32636 Fixed discount file extents for inode: 95066 in root: 32718 Fixed discount file extents for inode: 95066 in root: 32732 Fixed discount file extents for inode: 95066 in root: 32830ts [O] Fixed discount file extents for inode: 95066 in root: 32832 Fixed discount file extents for inode: 95066 in root: 32833 btrfs unable to find ref byte nr 4607605415936 parent 0 root 32834 owner 0 offset 1 Fixed discount file extents for inode: 95066 in root: 32834 Fixed discount file extents for inode: 95066 in root: 32903 Fixed discount file extents for inode: 95066 in root: 32961 Fixed discount file extents for inode: 95066 in root: 33002 warning line 4112 [.] checking csums checking root refs found 901576467411 bytes used err is 0 total csum bytes: 871524676 total tree bytes: 11151159296 total fs tree bytes: 8709533696 total extent tree bytes: 1307844608 btree space waste bytes: 2428036988 file data blocks allocated: 17604037283840 referenced 1003776770048 This, BTW, *just* completed a few hours ago, i.e., it took almost a full week (it started on Friday, February 3rd, at about 20:53:35, and finished today at 20:42:48). Since it required between about 75% and 85% of the measly 4GB of RAM my desktop has, I decided not to use it for the duration of the --repair run, which is why I haven't replied until now (since that's the machine I use for private email). Running check without --repair, on the other hand, took only 9-10 hours. Right now I have a scrub running, the status of which is, at the moment of writing: # btrfs scrub status /media/MARCEC_BACKUP scrub status for f97b3cda-15e8-418b-bb9b-235391ef2a38 scrub started at Fri Feb 10 21:00:36 2017, running for 02:14:05 total bytes scrubbed: 631.51GiB with 0 errors (which is about 2x slower than expected so far) Otherwise, since I have the FS mounted now, I can now show this: # btrfs filesystem show /media/MARCEC_BACKUP/ Label: 'MARCEC_BACKUP' uuid: f97b3cda-15e8-418b-bb9b-235391ef2a38 Total devices 1 FS bytes used 841.75GiB devid 1 size 976.56GiB used 878.25GiB path /dev/sdd2 # btrfs filesystem usage /media/MARCEC_BACKUP/ Overall: Device size: 976.56GiB Device allocated: 878.25GiB Device unallocated: 98.31GiB Device missing: 0.00B Used: 852.13GiB Free (estimated): 117.95GiB (min: 68.80GiB) Data ratio: 1.00 Metadata ratio: 1.92 Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:851.00GiB, Used:831.36GiB /dev/sdd2 851.00GiB Metadata,single: Size:1.12GiB, Used:0.00B /dev/sdd2 1.12GiB Metadata,DUP: Size:13.00GiB, Used:10.38GiB /dev/sdd2 26.00GiB System,DUP: Size:64.00MiB, Used:120.00KiB /dev/sdd2 128.00MiB Unallocated: /dev/sdd2 98.31GiB # btrfs filesystem df /media/MARCEC_BACKUP/ Data, single: total=851.00GiB, used=831.36GiB System, DUP: total=64.00MiB, used=120.00KiB Metadata, DUP: total=13.00GiB, used=10.38GiB Metadata, single: total=1.12GiB, used=0.00B GlobalReserve, single: total=512.00MiB, used=0.00B Hmm, I take it that the single metadata is a leftover from running --repair? [...] > > The system the drive runs on is: > > > > % uname -a > > Linux diefledermaus 4.9.7-gentoo #1 SMP Wed Feb 1 23:52:56 CET 2017 > > x86_64 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux > > > > However during the crash it was running 4.9.6-gentoo. The system > > uses the standard CFQ scheduler, so perhaps BFQ is not at fault in > > Kai's case. > > > > The system I am running btrfs-check on is: > > > > % uname -a > > Linux thetick 4.9.6-gentoo #1 SMP PREEMPT Fri Jan 27 00:50:02 CET > > 2017 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ > > AuthenticAMD GNU/Linux > > > > Both have btrfs-progs 4.9: > > > > % /sbin/btrfs --version > > btrfs-progs v4.9 > > > > And the file system in question: > > > > % sudo /sbin/btrfs fi show /dev/sdd2 > > Label: 'MARCEC_BACKUP' uuid: > > f97b3cda-15e8-418b-bb9b-235391ef2a38 Total devices 1 FS bytes used > > 842.50GiB devid 1 size 976.56GiB used 877.31GiB path /dev/sdd2 > > > > The file system is mounted with > > "noatime,compress,comment=systemd.automount". > > So our common denominator is only "compress". Well, and systemd and > Gentoo - but that probably doesn't count. Yeah, probably not :) . > It looks like you're running without bcache and on a single disk. Exactly. > Do you use files that should maybe better be marked nocow? (chattr > +C)... in terms like VirtualBox images or database files? Nope. It's used exclusively as a target for btrfs-receive and rsync (OK, my music backups are also used by MPD on the home server it's normally attached to, as an alternative to leaving my desktop running just to hear music, but that's it). [...] Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-10 22:15 ` Marc Joliet @ 2017-02-11 2:01 ` Kai Krakow 2017-02-11 10:18 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Kai Krakow @ 2017-02-11 2:01 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2429 bytes --] Am Fri, 10 Feb 2017 23:15:03 +0100 schrieb Marc Joliet <marcec@gmx.de>: > # btrfs filesystem df /media/MARCEC_BACKUP/ > Data, single: total=851.00GiB, used=831.36GiB > System, DUP: total=64.00MiB, used=120.00KiB > Metadata, DUP: total=13.00GiB, used=10.38GiB > Metadata, single: total=1.12GiB, used=0.00B > GlobalReserve, single: total=512.00MiB, used=0.00B > > Hmm, I take it that the single metadata is a leftover from running > --repair? It's more probably a remnant of an incomplete balance operation or an older mkfs version. I'd simply rebalance metadata to fix this. I don't think that btrfs-repair would migrate missing metadata duplicates back to single profile, it would more likely trigger recreating the missing duplicates. But I'm not sure. If it is a result of the repair operation, that could be an interesting clue. Could it explain "error -17" from your logs? But that would mean the duplicates were already missing before the repair operation and triggered that problem. So the question is, why are those duplicates missing in the first place as a result of normal operation? From your logs: ---8<--- snip Feb 02 22:49:14 thetick kernel: BTRFS: device label MARCEC_BACKUP devid 1 transid 283903 /dev/sdd2 Feb 02 22:49:19 thetick kernel: EXT4-fs (sdd1): mounted filesystem with ordered data mode. Opts: (null) Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): use zlib compression Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): disk space caching is enabled Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): has skinny extents Feb 03 00:20:09 thetick kernel: BTRFS info (device sdd2): The free space cache file (3967375376384) is invalid. skip it Feb 03 01:05:58 thetick kernel: ------------[ cut here ]------------ Feb 03 01:05:58 thetick kernel: WARNING: CPU: 1 PID: 26544 at fs/btrfs/extent- tree.c:2967 btrfs_run_delayed_refs+0x26c/0x290 Feb 03 01:05:58 thetick kernel: BTRFS: Transaction aborted (error -17) --->8--- snap "error -17" being "object already exists". My only theory would be this has a direct connection to you finding the single metadata profile. Like in "the kernel thinks the objects already exists when it really didn't, and as a result the object is there only once now" aka "single metadata". But I'm no dev and no expert on the internals. -- Regards, Kai Replies to list-only preferred. [-- Attachment #2: Digitale Signatur von OpenPGP --] [-- Type: application/pgp-signature, Size: 163 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-11 2:01 ` Kai Krakow @ 2017-02-11 10:18 ` Marc Joliet 2017-02-14 12:52 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-02-11 10:18 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 55579 bytes --] On Saturday 11 February 2017 03:01:39 Kai Krakow wrote: > Am Fri, 10 Feb 2017 23:15:03 +0100 > > schrieb Marc Joliet <marcec@gmx.de>: > > # btrfs filesystem df /media/MARCEC_BACKUP/ > > Data, single: total=851.00GiB, used=831.36GiB > > System, DUP: total=64.00MiB, used=120.00KiB > > Metadata, DUP: total=13.00GiB, used=10.38GiB > > Metadata, single: total=1.12GiB, used=0.00B > > GlobalReserve, single: total=512.00MiB, used=0.00B > > > > Hmm, I take it that the single metadata is a leftover from running > > --repair? > > It's more probably a remnant of an incomplete balance operation or an > older mkfs version. I'd simply rebalance metadata to fix this. > > I don't think that btrfs-repair would migrate missing metadata > duplicates back to single profile, it would more likely trigger > recreating the missing duplicates. But I'm not sure. I'm fairly certain it's from the repair operation, the device didn't have any single metadata before (and it still didn't really, notice that used=0.00B). So my best guess is that it was allocated during the --repair. > If it is a result of the repair operation, that could be an > interesting clue. Could it explain "error -17" from your logs? But that > would mean the duplicates were already missing before the repair > operation and triggered that problem. So the question is, why are those > duplicates missing in the first place as a result of normal operation? > From your logs: I certainly didn't create them myself, the device has always had metadata=dup (you can check previous threads of mine, which also contain "btrfs fi df" output). So I don't think it has anything to do with the failures. > ---8<--- snip > Feb 02 22:49:14 thetick kernel: BTRFS: device label MARCEC_BACKUP devid > 1 transid 283903 /dev/sdd2 > Feb 02 22:49:19 thetick kernel: EXT4-fs (sdd1): mounted filesystem with > ordered data mode. Opts: (null) > Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): use zlib > compression Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): > disk space caching is enabled > Feb 03 00:18:52 thetick kernel: BTRFS info (device sdd2): has skinny > extents Feb 03 00:20:09 thetick kernel: BTRFS info (device sdd2): The > free space cache file (3967375376384) is invalid. skip it > Feb 03 01:05:58 thetick kernel: ------------[ cut here ]------------ > Feb 03 01:05:58 thetick kernel: WARNING: CPU: 1 PID: 26544 at > fs/btrfs/extent- tree.c:2967 btrfs_run_delayed_refs+0x26c/0x290 > Feb 03 01:05:58 thetick kernel: BTRFS: Transaction aborted (error -17) > --->8--- snap > > "error -17" being "object already exists". My only theory would be this > has a direct connection to you finding the single metadata profile. > Like in "the kernel thinks the objects already exists when it really > didn't, and as a result the object is there only once now" aka "single > metadata". > > But I'm no dev and no expert on the internals. Again, I don't think it has anything to do with the single metadata. A "btrfs balance start -mprofile=single" got rid of the (empty) single metadata blocks. What I *can* say is that the error seems to be transient. For example, I ran an rsync last night that failed with the same error as before. After unmounting and mounting the FS again, I could run it to completion (and verified some files). Anyway, here's the whole log since this morning. I can't correlate individual stack traces to specific actions any more, but the rough order of actions on my part (not counting unmount/mount cycles, those are easy enough to find): - backups from the home server run, then - btrfs-cleaner drops old snapshots, after which - I run rsync (which fails), then - I run rsync again (which succeeds), then - I run "balance start -mprofile=single", which succeeds (I might have run this before rsync, I'm not sure), after which - I start backups on my laptop (which fails after I finally went to bed, at about 3:58). After that I just try "btrbk -r run" a few times, which managed to transfer the first two snapshots before failing. Feb 11 00:00:06 diefledermaus kernel: usb 1-1: new high-speed USB device number 4 using ehci-pci Feb 11 00:00:06 diefledermaus kernel: usb 1-1: New USB device found, idVendor=0480, idProduct=d010 Feb 11 00:00:06 diefledermaus kernel: usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 Feb 11 00:00:06 diefledermaus kernel: usb 1-1: Product: External USB 3.0 Feb 11 00:00:06 diefledermaus kernel: usb 1-1: Manufacturer: Toshiba Feb 11 00:00:06 diefledermaus kernel: usb 1-1: SerialNumber: 20130421020612 Feb 11 00:00:07 diefledermaus kernel: usb-storage 1-1:1.0: USB Mass Storage device detected Feb 11 00:00:07 diefledermaus kernel: usb-storage 1-1:1.0: Quirks match for vid 0480 pid d010: 20000000 Feb 11 00:00:07 diefledermaus kernel: scsi host6: usb-storage 1-1:1.0 Feb 11 00:00:07 diefledermaus kernel: usbcore: registered new interface driver usb-storage Feb 11 00:00:16 diefledermaus kernel: scsi 6:0:0:0: Direct-Access Toshiba External USB 3.0 0201 PQ: 0 ANSI: 6 Feb 11 00:00:16 diefledermaus kernel: sd 6:0:0:0: Attached scsi generic sg2 type 0 Feb 11 00:00:16 diefledermaus kernel: sd 6:0:0:0: [sdb] Very big device. Trying to use READ CAPACITY(16). Feb 11 00:00:16 diefledermaus kernel: sd 6:0:0:0: [sdb] 732566645 4096-byte logical blocks: (3.00 TB/2.73 TiB) Feb 11 00:00:16 diefledermaus kernel: sd 6:0:0:0: [sdb] Assuming Write Enabled Feb 11 00:00:16 diefledermaus kernel: sd 6:0:0:0: [sdb] Assuming drive cache: write back Feb 11 00:00:16 diefledermaus kernel: sdb: sdb1 sdb2 Feb 11 00:00:16 diefledermaus kernel: sd 6:0:0:0: [sdb] Attached SCSI disk Feb 11 00:00:16 diefledermaus kernel: BTRFS: device label MARCEC_BACKUP devid 1 transid 284246 /dev/sdb2 Feb 11 00:00:16 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 00:00:16 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 00:00:16 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 00:00:17 diefledermaus kernel: EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null) Feb 11 00:01:21 diefledermaus kernel: BTRFS warning (device sdb2): block group 3831010164736 has wrong amount of free space Feb 11 00:01:21 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3831010164736, rebuilding it now Feb 11 00:01:21 diefledermaus kernel: BTRFS warning (device sdb2): block group 3831547035648 has wrong amount of free space Feb 11 00:01:21 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3831547035648, rebuilding it now Feb 11 00:01:21 diefledermaus kernel: BTRFS warning (device sdb2): block group 3833694519296 has wrong amount of free space Feb 11 00:01:21 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3833694519296, rebuilding it now Feb 11 00:01:22 diefledermaus kernel: BTRFS warning (device sdb2): block group 4649738305536 has wrong amount of free space Feb 11 00:01:22 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 4649738305536, rebuilding it now Feb 11 00:01:27 diefledermaus kernel: BTRFS warning (device sdb2): block group 4607325503488 has wrong amount of free space Feb 11 00:01:27 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 4607325503488, rebuilding it now Feb 11 00:01:37 diefledermaus kernel: BTRFS warning (device sdb2): block group 3832083906560 has wrong amount of free space Feb 11 00:01:37 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3832083906560, rebuilding it now Feb 11 00:01:37 diefledermaus kernel: BTRFS warning (device sdb2): block group 3832620777472 has wrong amount of free space Feb 11 00:01:37 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3832620777472, rebuilding it now Feb 11 00:01:38 diefledermaus kernel: BTRFS warning (device sdb2): block group 3833157648384 has wrong amount of free space Feb 11 00:01:38 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3833157648384, rebuilding it now Feb 11 00:03:06 diefledermaus kernel: BTRFS warning (device sdb2): block group 3829936422912 has wrong amount of free space Feb 11 00:03:06 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3829936422912, rebuilding it now Feb 11 00:06:26 diefledermaus kernel: BTRFS warning (device sdb2): block group 4735906086912 has wrong amount of free space Feb 11 00:06:26 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 4735906086912, rebuilding it now Feb 11 00:06:53 diefledermaus kernel: BTRFS warning (device sdb2): block group 4736442957824 has wrong amount of free space Feb 11 00:06:53 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 4736442957824, rebuilding it now Feb 11 00:06:55 diefledermaus kernel: BTRFS warning (device sdb2): block group 3830473293824 has wrong amount of free space Feb 11 00:06:55 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 3830473293824, rebuilding it now Feb 11 00:07:11 diefledermaus kernel: BTRFS warning (device sdb2): block group 4745066446848 has wrong amount of free space Feb 11 00:07:11 diefledermaus kernel: BTRFS warning (device sdb2): failed to load free space cache for block group 4745066446848, rebuilding it now Feb 11 00:47:39 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 00:47:39 diefledermaus kernel: WARNING: CPU: 1 PID: 8963 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 00:47:39 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 00:47:39 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 11 00:47:39 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 11 00:47:39 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 00:47:39 diefledermaus kernel: CPU: 1 PID: 8963 Comm: kworker/u4:4 Not tainted 4.9.7-gentoo #1 Feb 11 00:47:39 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 00:47:39 diefledermaus kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper Feb 11 00:47:39 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 ffffc900051a7d60 0000000000000000 Feb 11 00:47:39 diefledermaus kernel: ffffffff8104908a ffff88007edc63c0 ffffc900051a7db0 0000000000001732 Feb 11 00:47:39 diefledermaus kernel: ffff88008f47a1f8 ffff8800beac3700 0000000000000000 ffffffff810490fa Feb 11 00:47:39 diefledermaus kernel: Call Trace: Feb 11 00:47:39 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 00:47:39 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff81202188>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff81202244>] ? delayed_ref_async_start+0x84/0xa0 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff8105ce76>] ? process_one_work+0x126/0x310 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff8105d0a5>] ? worker_thread+0x45/0x450 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff81061c99>] ? kthread+0xb9/0xd0 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff81061be0>] ? kthread_park+0x50/0x50 Feb 11 00:47:39 diefledermaus kernel: [<ffffffff814f6412>] ? ret_from_fork+0x22/0x30 Feb 11 00:47:39 diefledermaus kernel: ---[ end trace ba59f5981cc5a675 ]--- Feb 11 00:47:39 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 01:07:43 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 11 01:07:43 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 01:07:43 diefledermaus kernel: WARNING: CPU: 0 PID: 9450 at fs/btrfs/extent-tree.c:134 btrfs_put_block_group+0x55/0x60 Feb 11 01:07:43 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 01:07:43 diefledermaus kernel: CPU: 0 PID: 9450 Comm: umount Tainted: G W 4.9.7-gentoo #1 Feb 11 01:07:43 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 01:07:43 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 0000000000000000 0000000000000000 Feb 11 01:07:43 diefledermaus kernel: ffffffff8104908a ffff88004203b200 ffff88004203b200 ffff8800174160a0 Feb 11 01:07:43 diefledermaus kernel: ffff880017416000 ffff880017416090 ffff8800371193f8 ffffffff811f9e05 Feb 11 01:07:43 diefledermaus kernel: Call Trace: Feb 11 01:07:43 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811f9e05>] ? btrfs_put_block_group+0x55/0x60 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff81201295>] ? btrfs_free_block_groups+0xf5/0x420 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8121231b>] ? close_ctree+0xfb/0x320 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8111105f>] ? generic_shutdown_super+0x5f/0xe0 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811112e9>] ? kill_anon_super+0x9/0x20 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811e70ed>] ? btrfs_kill_super+0xd/0x90 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811114bf>] ? deactivate_locked_super+0x2f/0x60 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8112b006>] ? cleanup_mnt+0x36/0x80 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff810605cd>] ? task_work_run+0x6d/0x90 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff81000fe6>] ? exit_to_usermode_loop+0x66/0x70 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff810012ee>] ? syscall_return_slowpath+0x4e/0x50 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff814f625f>] ? entry_SYSCALL_64_fastpath+0x92/0x94 Feb 11 01:07:43 diefledermaus kernel: ---[ end trace ba59f5981cc5a676 ]--- Feb 11 01:07:43 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 01:07:43 diefledermaus kernel: WARNING: CPU: 0 PID: 9450 at fs/btrfs/extent-tree.c:10045 btrfs_free_block_groups+0x28d/0x420 Feb 11 01:07:43 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 01:07:43 diefledermaus kernel: CPU: 0 PID: 9450 Comm: umount Tainted: G W 4.9.7-gentoo #1 Feb 11 01:07:43 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 01:07:43 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 0000000000000000 0000000000000000 Feb 11 01:07:43 diefledermaus kernel: ffffffff8104908a ffff8800bbcc6800 0000000000000000 ffff8800bbcc6888 Feb 11 01:07:43 diefledermaus kernel: ffff880017416000 ffff8800bb6569f8 ffff880017416a70 ffffffff8120142d Feb 11 01:07:43 diefledermaus kernel: Call Trace: Feb 11 01:07:43 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8120142d>] ? btrfs_free_block_groups+0x28d/0x420 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8121231b>] ? close_ctree+0xfb/0x320 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8111105f>] ? generic_shutdown_super+0x5f/0xe0 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811112e9>] ? kill_anon_super+0x9/0x20 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811e70ed>] ? btrfs_kill_super+0xd/0x90 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff811114bf>] ? deactivate_locked_super+0x2f/0x60 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff8112b006>] ? cleanup_mnt+0x36/0x80 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff810605cd>] ? task_work_run+0x6d/0x90 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff81000fe6>] ? exit_to_usermode_loop+0x66/0x70 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff810012ee>] ? syscall_return_slowpath+0x4e/0x50 Feb 11 01:07:43 diefledermaus kernel: [<ffffffff814f625f>] ? entry_SYSCALL_64_fastpath+0x92/0x94 Feb 11 01:07:43 diefledermaus kernel: ---[ end trace ba59f5981cc5a677 ]--- Feb 11 01:07:43 diefledermaus kernel: BTRFS info (device sdb2): space_info 4 has 2744418304 free, is not full Feb 11 01:07:43 diefledermaus kernel: BTRFS info (device sdb2): space_info total=15166603264, used=11214192640, pinned=0, reserved=32768, may_use=0, readonly=1207959552 Feb 11 01:07:57 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 01:07:57 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 01:07:57 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 02:02:58 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757540306944 flags 4 Feb 11 02:03:06 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757531918336 flags 4 Feb 11 02:03:07 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757523529728 flags 4 Feb 11 02:03:08 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757515141120 flags 4 Feb 11 02:03:08 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757506752512 flags 4 Feb 11 02:03:09 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757498363904 flags 4 Feb 11 02:03:10 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757489975296 flags 4 Feb 11 02:03:11 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757481586688 flags 4 Feb 11 02:03:12 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757473198080 flags 4 Feb 11 02:03:13 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757464809472 flags 4 Feb 11 02:03:14 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757456420864 flags 4 Feb 11 02:03:15 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757448032256 flags 4 Feb 11 02:03:16 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757439643648 flags 4 Feb 11 02:03:18 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757431255040 flags 4 Feb 11 02:03:19 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757422866432 flags 4 Feb 11 02:03:23 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757414477824 flags 4 Feb 11 02:03:25 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757406089216 flags 4 Feb 11 02:03:28 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757397700608 flags 4 Feb 11 02:03:30 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757389312000 flags 4 Feb 11 02:03:33 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757380923392 flags 4 Feb 11 02:03:35 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757372534784 flags 4 Feb 11 02:03:38 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757364146176 flags 4 Feb 11 02:03:40 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757355757568 flags 4 Feb 11 02:03:43 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757347368960 flags 4 Feb 11 02:03:45 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757338980352 flags 4 Feb 11 02:03:47 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757330591744 flags 4 Feb 11 02:03:49 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757322203136 flags 4 Feb 11 02:03:51 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757313814528 flags 4 Feb 11 02:03:53 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757305425920 flags 4 Feb 11 02:03:55 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757297037312 flags 4 Feb 11 02:03:57 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757288648704 flags 4 Feb 11 02:03:59 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757280260096 flags 4 Feb 11 02:04:01 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757271871488 flags 4 Feb 11 02:04:03 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757263482880 flags 4 Feb 11 02:04:06 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757255094272 flags 4 Feb 11 02:04:07 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757246705664 flags 4 Feb 11 02:04:09 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757238317056 flags 4 Feb 11 02:04:11 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757229928448 flags 4 Feb 11 02:04:12 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757221539840 flags 4 Feb 11 02:04:13 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757213151232 flags 4 Feb 11 02:04:15 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757204762624 flags 4 Feb 11 02:04:16 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757196374016 flags 4 Feb 11 02:04:17 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757187985408 flags 4 Feb 11 02:04:19 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757179596800 flags 4 Feb 11 02:04:20 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757171208192 flags 4 Feb 11 02:04:21 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757162819584 flags 4 Feb 11 02:04:22 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757154430976 flags 4 Feb 11 02:04:24 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757146042368 flags 4 Feb 11 02:04:25 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757137653760 flags 4 Feb 11 02:04:26 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757129265152 flags 4 Feb 11 02:04:27 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757120876544 flags 4 Feb 11 02:04:28 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757112487936 flags 4 Feb 11 02:04:29 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757104099328 flags 4 Feb 11 02:04:30 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757095710720 flags 4 Feb 11 02:04:31 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757087322112 flags 4 Feb 11 02:04:37 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757078933504 flags 4 Feb 11 02:04:38 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757070544896 flags 4 Feb 11 02:04:39 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757062156288 flags 4 Feb 11 02:04:40 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757053767680 flags 4 Feb 11 02:04:41 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757045379072 flags 4 Feb 11 02:04:42 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757036990464 flags 4 Feb 11 02:04:43 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757028601856 flags 4 Feb 11 02:04:44 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757020213248 flags 4 Feb 11 02:04:45 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757011824640 flags 4 Feb 11 02:04:47 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4757003436032 flags 4 Feb 11 02:04:47 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756995047424 flags 4 Feb 11 02:04:48 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756986658816 flags 4 Feb 11 02:04:49 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756978270208 flags 4 Feb 11 02:04:50 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756969881600 flags 4 Feb 11 02:04:50 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756961492992 flags 4 Feb 11 02:04:51 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756953104384 flags 4 Feb 11 02:04:52 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756944715776 flags 4 Feb 11 02:04:53 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756936327168 flags 4 Feb 11 02:04:53 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756927938560 flags 4 Feb 11 02:04:54 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756919549952 flags 4 Feb 11 02:04:55 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756911161344 flags 4 Feb 11 02:04:57 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756902772736 flags 4 Feb 11 02:04:58 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756894384128 flags 4 Feb 11 02:04:58 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756885995520 flags 4 Feb 11 02:05:00 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756877606912 flags 4 Feb 11 02:05:01 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756869218304 flags 4 Feb 11 02:05:03 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756860829696 flags 4 Feb 11 02:05:04 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756852441088 flags 4 Feb 11 02:05:04 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756844052480 flags 4 Feb 11 02:05:05 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756835663872 flags 4 Feb 11 02:05:06 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756827275264 flags 4 Feb 11 02:05:06 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756818886656 flags 4 Feb 11 02:05:07 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756810498048 flags 4 Feb 11 02:05:08 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756802109440 flags 4 Feb 11 02:05:09 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756793720832 flags 4 Feb 11 02:05:09 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756785332224 flags 4 Feb 11 02:05:10 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756776943616 flags 4 Feb 11 02:05:11 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756768555008 flags 4 Feb 11 02:05:13 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756760166400 flags 4 Feb 11 02:05:13 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756751777792 flags 4 Feb 11 02:05:14 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756743389184 flags 4 Feb 11 02:05:15 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756735000576 flags 4 Feb 11 02:05:15 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756726611968 flags 4 Feb 11 02:05:16 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756718223360 flags 4 Feb 11 02:05:17 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756709834752 flags 4 Feb 11 02:05:20 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756701446144 flags 4 Feb 11 02:05:20 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756693057536 flags 4 Feb 11 02:05:22 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756684668928 flags 4 Feb 11 02:05:23 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756676280320 flags 4 Feb 11 02:05:24 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756667891712 flags 4 Feb 11 02:05:26 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756659503104 flags 4 Feb 11 02:05:27 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756651114496 flags 4 Feb 11 02:05:45 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756642725888 flags 4 Feb 11 02:05:46 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756634337280 flags 4 Feb 11 02:05:48 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756625948672 flags 4 Feb 11 02:05:49 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756617560064 flags 4 Feb 11 02:05:51 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756609171456 flags 4 Feb 11 02:05:52 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756600782848 flags 4 Feb 11 02:05:54 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756592394240 flags 4 Feb 11 02:05:56 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756584005632 flags 4 Feb 11 02:05:58 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756575617024 flags 4 Feb 11 02:05:59 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756567228416 flags 4 Feb 11 02:06:01 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756558839808 flags 4 Feb 11 02:06:02 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756550451200 flags 4 Feb 11 02:06:04 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4756542062592 flags 4 Feb 11 02:06:05 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751164964864 flags 4 Feb 11 02:06:07 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751156576256 flags 4 Feb 11 02:06:09 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751148187648 flags 4 Feb 11 02:06:10 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751139799040 flags 4 Feb 11 02:06:12 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751131410432 flags 4 Feb 11 02:06:13 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751123021824 flags 4 Feb 11 02:06:15 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751114633216 flags 4 Feb 11 02:06:17 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751106244608 flags 4 Feb 11 02:06:18 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751097856000 flags 4 Feb 11 02:06:20 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751089467392 flags 4 Feb 11 02:06:21 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751081078784 flags 4 Feb 11 02:06:22 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751072690176 flags 4 Feb 11 02:06:24 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751064301568 flags 4 Feb 11 02:06:25 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751055912960 flags 4 Feb 11 02:06:26 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751047524352 flags 4 Feb 11 02:06:27 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751039135744 flags 4 Feb 11 02:06:28 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751030747136 flags 4 Feb 11 02:06:29 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751022358528 flags 4 Feb 11 02:06:30 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751013969920 flags 4 Feb 11 02:06:32 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4751005581312 flags 4 Feb 11 02:06:33 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4750997192704 flags 4 Feb 11 02:06:34 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4750988804096 flags 4 Feb 11 02:06:35 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4750980415488 flags 4 Feb 11 02:06:36 diefledermaus kernel: BTRFS info (device sdb2): relocating block group 4750972026880 flags 4 Feb 11 02:27:34 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 02:27:35 diefledermaus kernel: WARNING: CPU: 0 PID: 10268 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 02:27:35 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 02:27:35 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 11 02:27:35 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 11 02:27:35 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 02:27:35 diefledermaus kernel: CPU: 0 PID: 10268 Comm: kworker/u4:4 Tainted: G W 4.9.7-gentoo #1 Feb 11 02:27:35 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 02:27:35 diefledermaus kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper Feb 11 02:27:35 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 ffffc90005fe3d60 0000000000000000 Feb 11 02:27:35 diefledermaus kernel: ffffffff8104908a ffff880052a738c0 ffffc90005fe3db0 00000000000001c2 Feb 11 02:27:35 diefledermaus kernel: ffff880019a0ea50 ffff8800bc583c00 0000000000000000 ffffffff810490fa Feb 11 02:27:35 diefledermaus kernel: Call Trace: Feb 11 02:27:35 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 02:27:35 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff81202188>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff81202244>] ? delayed_ref_async_start+0x84/0xa0 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff8105ce76>] ? process_one_work+0x126/0x310 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff8105d0a5>] ? worker_thread+0x45/0x450 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff81061c99>] ? kthread+0xb9/0xd0 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff81061be0>] ? kthread_park+0x50/0x50 Feb 11 02:27:35 diefledermaus kernel: [<ffffffff814f6412>] ? ret_from_fork+0x22/0x30 Feb 11 02:27:35 diefledermaus kernel: ---[ end trace ba59f5981cc5a678 ]--- Feb 11 02:27:35 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 02:29:09 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 11 02:29:11 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 02:29:11 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 02:29:11 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 02:54:20 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 02:54:20 diefledermaus kernel: WARNING: CPU: 0 PID: 10382 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 02:54:20 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 11 02:54:20 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 02:54:20 diefledermaus kernel: CPU: 0 PID: 10382 Comm: btrfs-cleaner Tainted: G W 4.9.7-gentoo #1 Feb 11 02:54:20 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 02:54:20 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 ffffc9000601bd28 0000000000000000 Feb 11 02:54:20 diefledermaus kernel: ffffffff8104908a ffff8800037f3780 ffffc9000601bd78 00000000000003a0 Feb 11 02:54:20 diefledermaus kernel: ffff88005b83d3b0 ffff8800225f19df ffff8800037f3780 ffffffff810490fa Feb 11 02:54:20 diefledermaus kernel: Call Trace: Feb 11 02:54:20 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 02:54:20 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff81202188>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff81215dda>] ? btrfs_should_end_transaction+0x3a/0x60 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff81200787>] ? btrfs_drop_snapshot+0x417/0x7b0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff81217012>] ? btrfs_clean_one_deleted_snapshot+0xb2/0xe0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff8120e04f>] ? cleaner_kthread+0x13f/0x1b0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff8120df10>] ? btrfs_destroy_pinned_extent+0xc0/0xc0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff81061c99>] ? kthread+0xb9/0xd0 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff8104b678>] ? do_exit+0x698/0xa70 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff81061be0>] ? kthread_park+0x50/0x50 Feb 11 02:54:20 diefledermaus kernel: [<ffffffff814f6412>] ? ret_from_fork+0x22/0x30 Feb 11 02:54:20 diefledermaus kernel: ---[ end trace ba59f5981cc5a679 ]--- Feb 11 02:54:20 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 02:54:20 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 11 02:55:18 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 11 02:55:21 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 02:55:21 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 02:55:21 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 03:04:21 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 03:04:21 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 03:04:21 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 03:30:30 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 03:30:30 diefledermaus kernel: WARNING: CPU: 0 PID: 10620 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 03:30:30 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 11 03:30:30 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 03:30:30 diefledermaus kernel: CPU: 0 PID: 10620 Comm: kworker/u4:4 Tainted: G W 4.9.7-gentoo #1 Feb 11 03:30:30 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 03:30:30 diefledermaus kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper Feb 11 03:30:30 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 ffffc9000645bd60 0000000000000000 Feb 11 03:30:31 diefledermaus kernel: ffffffff8104908a ffff8800b7bd5a00 ffffc9000645bdb0 0000000000000020 Feb 11 03:30:31 diefledermaus kernel: ffff880003394898 ffff8800bc583500 0000000000000000 ffffffff810490fa Feb 11 03:30:31 diefledermaus kernel: Call Trace: Feb 11 03:30:31 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 03:30:31 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff81202188>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff81202244>] ? delayed_ref_async_start+0x84/0xa0 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff8105ce76>] ? process_one_work+0x126/0x310 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff8105a893>] ? pwq_activate_delayed_work+0x33/0x60 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff8105d0a5>] ? worker_thread+0x45/0x450 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff81061c99>] ? kthread+0xb9/0xd0 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff81061be0>] ? kthread_park+0x50/0x50 Feb 11 03:30:31 diefledermaus kernel: [<ffffffff814f6412>] ? ret_from_fork+0x22/0x30 Feb 11 03:30:31 diefledermaus kernel: ---[ end trace ba59f5981cc5a67a ]--- Feb 11 03:30:31 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 03:30:31 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 11 03:31:39 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 11 03:31:41 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 03:31:41 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 03:31:41 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 03:58:38 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 03:58:38 diefledermaus kernel: WARNING: CPU: 0 PID: 11174 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 03:58:38 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 11 03:58:38 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 03:58:38 diefledermaus kernel: CPU: 0 PID: 11174 Comm: btrfs Tainted: G W 4.9.7-gentoo #1 Feb 11 03:58:38 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 03:58:38 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 ffffc90006bfbb88 0000000000000000 Feb 11 03:58:38 diefledermaus kernel: ffffffff8104908a ffff880047adb1e0 ffffc90006bfbbd8 0000000000000000 Feb 11 03:58:38 diefledermaus kernel: ffff88000a6f8898 0000000000000a34 ffff880023d59240 ffffffff810490fa Feb 11 03:58:38 diefledermaus kernel: Call Trace: Feb 11 03:58:38 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff81202188>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff812170eb>] ? btrfs_commit_transaction.part.25+0x5b/0x970 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8124c04d>] ? btrfs_mksubvol+0x45d/0x470 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff810796b0>] ? wake_up_bit+0x20/0x20 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8124c1f5>] ? btrfs_ioctl_snap_create_transid+0x195/0x1a0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8110bfcd>] ? __check_object_size+0x11d/0x1d2 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8124c38a>] ? btrfs_ioctl_snap_create_v2+0xfa/0x150 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8124f386>] ? btrfs_ioctl+0x656/0x22b0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8111d7ad>] ? do_filp_open+0x8d/0xd0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff810ecf92>] ? handle_mm_fault+0xd42/0xdb0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8110bfcd>] ? __check_object_size+0x11d/0x1d2 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8111f756>] ? do_vfs_ioctl+0x86/0x5c0 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff8111fcc6>] ? SyS_ioctl+0x36/0x70 Feb 11 03:58:38 diefledermaus kernel: [<ffffffff814f61e0>] ? entry_SYSCALL_64_fastpath+0x13/0x94 Feb 11 03:58:38 diefledermaus kernel: ---[ end trace ba59f5981cc5a67b ]--- Feb 11 03:58:38 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 03:58:39 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 11 09:47:57 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 11 09:48:10 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 09:48:10 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 09:48:10 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 10:06:46 diefledermaus kernel: ------------[ cut here ]------------ Feb 11 10:06:46 diefledermaus kernel: WARNING: CPU: 1 PID: 12677 at fs/btrfs/extent-tree.c:2967 btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 10:06:46 diefledermaus kernel: BTRFS: Transaction aborted (error -17) Feb 11 10:06:46 diefledermaus kernel: Modules linked in: usb_storage msr ctr ccm tun arc4 snd_hda_codec_idt snd_hda_codec_generic ath5k applesmc snd_hda_intel input_polldev snd_hda_codec mac80211 hwmon ath snd_hda_core snd_pcm cfg80211 sky2 rfkill snd_timer acpi_cpufreq video snd backlight soundcore processor button sg sr_mod cdrom ata_generic pata_acpi uhci_hcd ahci libahci ata_piix libata ehci_pci ehci_hcd Feb 11 10:06:46 diefledermaus kernel: CPU: 1 PID: 12677 Comm: kworker/u4:6 Tainted: G W 4.9.7-gentoo #1 Feb 11 10:06:46 diefledermaus kernel: Hardware name: Apple Inc. Macmini2,1/Mac-F4208EAA, BIOS MM21.88Z.009A.B00.0706281359 06/28/07 Feb 11 10:06:46 diefledermaus kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper Feb 11 10:06:46 diefledermaus kernel: 0000000000000000 ffffffff812cf8f9 ffffc90001c73d60 0000000000000000 Feb 11 10:06:46 diefledermaus kernel: ffffffff8104908a ffff8800258696e0 ffffc90001c73db0 0000000000000020 Feb 11 10:06:46 diefledermaus kernel: ffff8800214131f8 ffff880087de7f00 0000000000000000 ffffffff810490fa Feb 11 10:06:46 diefledermaus kernel: Call Trace: Feb 11 10:06:46 diefledermaus kernel: [<ffffffff812cf8f9>] ? dump_stack+0x46/0x5d Feb 11 10:06:46 diefledermaus kernel: [<ffffffff8104908a>] ? __warn+0xba/0xe0 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff810490fa>] ? warn_slowpath_fmt+0x4a/0x50 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff81202188>] ? btrfs_run_delayed_refs+0x278/0x2b0 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff81202244>] ? delayed_ref_async_start+0x84/0xa0 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff8105ce76>] ? process_one_work+0x126/0x310 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff8105d0a5>] ? worker_thread+0x45/0x450 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff8105d060>] ? process_one_work+0x310/0x310 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff81061c99>] ? kthread+0xb9/0xd0 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff81061be0>] ? kthread_park+0x50/0x50 Feb 11 10:06:46 diefledermaus kernel: [<ffffffff814f6412>] ? ret_from_fork+0x22/0x30 Feb 11 10:06:46 diefledermaus kernel: ---[ end trace ba59f5981cc5a67c ]--- Feb 11 10:06:46 diefledermaus kernel: BTRFS: error (device sdb2) in btrfs_run_delayed_refs:2967: errno=-17 Object already exists Feb 11 10:06:46 diefledermaus kernel: BTRFS info (device sdb2): forced readonly Feb 11 10:18:06 diefledermaus kernel: BTRFS error (device sdb2): cleaner transaction attach returned -30 Feb 11 10:18:19 diefledermaus kernel: BTRFS info (device sdb2): use zlib compression Feb 11 10:18:19 diefledermaus kernel: BTRFS info (device sdb2): disk space caching is enabled Feb 11 10:18:19 diefledermaus kernel: BTRFS info (device sdb2): has skinny extents Feb 11 10:24:09 diefledermaus kernel: BTRFS info (device sdb2): The free space cache file (3833157648384) is invalid. skip it Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-11 10:18 ` Marc Joliet @ 2017-02-14 12:52 ` Marc Joliet 2017-02-17 8:19 ` Kai Krakow 0 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-02-14 12:52 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 778 bytes --] Hi again, so, it seems that I've solved the problem: After having to umount/mount the FS several times to get btrfs-cleaner to finish, I thought of the "failed to load free space [...], rebuilding it now" type errors and decided to try the clear_cache mount option. Since then, my home server has been running its backups regularly again. Furthermore, I was able back up my desktop again via send/recv (the rsync based backup is still running, but I expect it to succeed). The kernel log has also stayed clean. Kai, I'd be curious whether clear_cache will help in your case, too, if you haven't tried it already. Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-14 12:52 ` Marc Joliet @ 2017-02-17 8:19 ` Kai Krakow 2017-02-28 22:14 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Kai Krakow @ 2017-02-17 8:19 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1109 bytes --] Am Tue, 14 Feb 2017 13:52:48 +0100 schrieb Marc Joliet <marcec@gmx.de>: > Hi again, > > so, it seems that I've solved the problem: After having to > umount/mount the FS several times to get btrfs-cleaner to finish, I > thought of the "failed to load free space [...], rebuilding it now" > type errors and decided to try the clear_cache mount option. Since > then, my home server has been running its backups regularly again. > Furthermore, I was able back up my desktop again via send/recv (the > rsync based backup is still running, but I expect it to succeed). > The kernel log has also stayed clean. > > Kai, I'd be curious whether clear_cache will help in your case, too, > if you haven't tried it already. The FS in question has been rebuilt from scratch. AFAIR it wasn't even mountable back then, or at it froze the system short after mounting. I needed the system back in usable state, so I had to recreate and restore the system. Next time, if it happens again (finger crossed it does not), I'll give it a try. -- Regards, Kai Replies to list-only preferred. [-- Attachment #2: Digitale Signatur von OpenPGP --] [-- Type: application/pgp-signature, Size: 163 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-17 8:19 ` Kai Krakow @ 2017-02-28 22:14 ` Marc Joliet 2017-03-01 8:23 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-02-28 22:14 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 32389 bytes --] =2D-nextPart2283738.6rFPRxsZn9 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="utf-8" (Resend with the log compressed this time.) On Friday 17 February 2017 09:19:15 Kai Krakow wrote: > Am Tue, 14 Feb 2017 13:52:48 +0100 > > schrieb Marc Joliet <marcec@gmx.de>: > > Hi again, > > > > so, it seems that I've solved the problem: After having to > > umount/mount the FS several times to get btrfs-cleaner to finish, I > > thought of the "failed to load free space [...], rebuilding it now" > > type errors and decided to try the clear_cache mount option. Since > > then, my home server has been running its backups regularly again. > > Furthermore, I was able back up my desktop again via send/recv (the > > rsync based backup is still running, but I expect it to succeed). > > The kernel log has also stayed clean. *sigh* It was working fine... until the 26th (so almost 2 weeks without incident). Now the error keeps occurring, and the clear_cache mount option stopped helping. Just out of curiosity, I used "clear_cache,space_cache=v2", which also didn't help. I've attached the full log, in case it's in any way useful. This is all with: % uname -a Linux diefledermaus 4.9.12-gentoo #1 SMP Fri Feb 24 20:09:28 CET 2017 x86_64 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux % /sbin/btrfs --version btrfs-progs v4.9 > > Kai, I'd be curious whether clear_cache will help in your case, too, > > if you haven't tried it already. > > The FS in question has been rebuilt from scratch. AFAIR it wasn't even > mountable back then, or at it froze the system short after mounting. > > I needed the system back in usable state, so I had to recreate and > restore the system. I think I'm at that point now myself, unless anybody has any other ideas. What I *am* going to do before giving up and reformatting is create an image. I found out that using the -w option to btrfs-image keeps it from segfaulting, so I guess the underlying problem has to do with the extent tree? However, the image is... pretty damn big (8.6 GB by now, *after* compressing with "xz =2D9"), and it's not even finished yet. So, under the assumption that the image could be useful to somebody, I would welcome any suggestions for where to upload such an image once it's done. > Next time, if it happens again (finger crossed it does not), I'll give > it a try. Yeah, I hope it doesn't come back, either, for either of us. Greetings =2D- Marc Joliet =2D- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup =2D-nextPart2283738.6rFPRxsZn9 Content-Disposition: attachment; filename="dmesg_since_20170226.log.zip" Content-Transfer-Encoding: base64 Content-Type: application/zip; name="dmesg_since_20170226.log.zip" UEsDBBQAAAAIAMW5XErwNMkOxlQAAAC9BAAYAAAAZG1lc2dfc2luY2VfMjAxNzAyMjYubG9n7Z1Z kxtHkqDf91ekzT6MZCuScR+w1axRR8/SpkX1iOyemW1rg+FIsGCsAmoAlNjsX78RmahiHpGRcSUS BaIkUazL3dMzItw9ji/+kM8zxDIAJhhOCM+W63x1my/z3d3sYZ99zHeb/HaSvXn7h18n2WG2/5jN D7vVfoIoxTKb324XH/NlttrusrvtLs8ON7NNBhHI9vliu1nuX/6PPzjJLz/ebg9Kx3pzUCLJS/kS ohcf8s1hu83+J3QU9E/54mabgexfslf3u+3i1f7z/lX5vVc3D5sPU/0I08P6Lt8+HKbKyP0/KUH7 2fw23yvj1/vsLt/vZx9yV8MLb2RfPn7Sf4CsdE/xPyXi7+D44eoN0PjIVupDCABmhCBKKl+AKyJX tPKF+WIBKRLOqh5/D3OqPkgpaCGVUsg4XiJRfKH4KUhWKGcg46vaR+6rqmUzgYAt0bwledVyhKOq H2e3t9n73WyRT1yN++v/bj7nv/wt+z/ZdLpf3OTLh9v8f4G/QwFegb8rJ4UJxYgsC6EVkWipJLq/ rapAhFHOaSGwaIXT/WG2O0y3O/Vb+XKa//2g+o5SQWiwCsAlk6UjPs0+5tOH++l8rWUi7QgUaDZG bF4x+9Ns/cXq3WzzQftlPlcKIAzTgJCY8YqGw+5hs5gdtFzMteGzQLnqDS6qDs8Ps8Nhp/2BFkou DvMIRIKTvJC72R7Wq8/Txc3RDYjkWm4eJhcscsQKuctt1QmUKaEyVOZKyELm73XPQiT1K2OBYpcA gkdT1aBdMxcEm0tWjBx7suoMu8/Td//17sfXf/zjlJHparY/3M8ON9p2rDWQqgY8oWDIYGiR7xcM LYIGCYYWfeHB0OaNxMHQpipxMHRRlSgYWlR1BEObccHB0F2oYzB0FRgRDF1V+ARDd7NDg6GzBs9g 6C7XLxi6yvUNhs5yPYKhh0yfYOgu1i8Yune60GBIJxQPGQwt8v2CoUXQIMHQoi88GNq8kTgY2lQl DoYuqhIFQ4uqjmBoMy44GLoLdQyGrgIjgqGrCp9g6G52aDB01uAZDN3l+gVDV7m+wdBZrkcw9JDp EwzdxfoFQ/dOFxoM+YSyIYOhRb5fMLQIGiQYWvSFB0ObNxIHQ5uqxMHQRVWiYGhR1REMbcYFB0N3 oY7B0FVgRDB0VeETDN3NDg2Gzho8g6G7XL9g6CrXNxg6y/UIhh4yfYKhu1i/YOje6UKDoZxQOWQw tMj3C4YWQYMEQ4u+8GBo80biYGhTlTgYuqhKFAwtqjqCoc244GDoLtQxGLoKjAiGrip8gqG72aHB 0FmDZzB0l+sXDF3l+gZDZ7kewdBDpk8wdBfrFwzdO11gMCRoAtCAwdAm3ysY2gQNEQxt+oKDodUb aYOhVVXaYOikKk0wtKkyB0OrcaHB0EOoWzB0FhgeDJ1VeARDD7MDg6G7Br9g6CHXKxg6y/UMhu5y 3YOhj0yPYOgh1isYenS60GBIJoAMGQwt8v2CoUXQIMHQoi88GNq8kTgY2lQlDoYuqhIFQ4uqjmCo foMmD4buQh2DoavAiGDoqsInGLqbHRoMnTV4BkN3uX7B0FWubzB0lusRDD1k+gRDd7F+wdC904UE Q4gnSNWeXaPLi8rHX7PFwyG7UW03+1v16w1RXcb+x+vf3r55+69qxPrTnycqbvzpzU+TjAqWzQ7Z av+qaGavyp784rDL85eLCZMUqWGpbIEr9bVjT3+53u9mLzlVj8OkHlWWX3pPjxW/bPVotM9u1xsd zdebSXa332WLg/pvcZcdHjbZbLcg2X6znN4sZ9PFdpkvpuvlofEVFazz3Xrx9FUdyW/Vo9zQj/Wf zGb390rh3ULpuleB+H57e7vMf8/uZgsBEISVn1aOVQKym093203x5Xtl0WL14cvP6Wi+y35fL/Nt Nlvcr6eL+wfll//O5rPFx9v1h5vCzEynAyq2q0xlt/q4VvFh//EzyuYPh4MWvH3YLAtl+w/Zw36u xtLtTqUB2X43vdsus8Vyt71ThsyenvFef6LVZbObxbr41v16/Xflw3nxhQf1x/RmsSy+cJhluf78 Xn0jP37D8d00G8aP27u7Y9rxYnGbz5Q1TnlTj5r/O9stP83U829md/kke63fT/Zms3iZ/TJb3K03 a/QdfKX++uIPBAHx8+vX32U/vPn1XZHz/PILgi+F+H8vAZCvX/4AwEvAAUMCYiozwF4h8QpwRzvM qc9x/F2spMqqWj/RlST0qarEOSIBmz0Jekxsyi9gPl+pdAVW8zAEEIGSeakqsrOlGpJNNj/9hFws lNtm1eeGq5mEzu2lnfr0GVeLcYWPy4H44e5e5xSLjzqJ0CGDuhpRTyIK5x6zKdXGNjq8z5S43Nl9 tVBReOMoLuUo2NCSc1bXogLSdJnfzj7rNCVf7XVkWmkVKnsN0oFUJrwClXzCoEEWidY8TD6kOZpV 85Wb7cOtygz1oKkSrf1scVhv9evA+nWEPgTgy2qypUZK1Ww2s3ulrUg/oU65eOgj6JGkIr0Y8qbb Ta4dlasBr6pqjoJbFQI5kKtC0XFUnX483Ozy2bJID1b6NQc+AliuWPUtL/P9Ybf9rMLFZlPN0xf6 VS+CdAAGF7zstV+snut8bBkmj8wZFY/pWP73IudnUo3jf5/xUAvnC1C1cHo/2318zO5okFCV3TFZ to5dflDDwPZuutoWUpFuCthVapHIqW6RHfTAmc2XsxyqCpBCuVIFtShSO0dRP7z/7Q/vVFqz2mbf qJRmrcTtl3P07SRTzWqVYYGBkIJKzCXKDtvD7Da7P+z2GVGFthrGsv29tgAKgV09cvxYH/I7lSh8 zD9n3ygtEEiJJNdaIFNB89viB7aqpsaSw+KT/fofeYaaWW+fnsePst1mepzKIMhUwpEhChCj2ep2 9mGvPhEhTwCNT8BZhpEEuOspnJtP40Pn1OUsVpEtqocJsRlZbGbwPG3GnTZjld6cp82k22Yo+Xna TLttRoCcp83MZjM7T5u5xWZ6pu1ZdNuMz7UPym6b2bmOz9AcFgVS4RhxRCEUmMHkDXt/owrqZQLz 6zERA5UdiHZUx+KLydg3e3j8qEX1IqhDwhgnj0E92XvMdtvtIXOeFqj5ox5v1R9AgLY/AP1a/IEb /iC6Pzb9ITgaxB9ICUZ4IH+owC5cp7DqPiENn+iO3vYJll9LG6nnIRgxwkjbH4AN5Q89d3lO/qjn OCreUtYeQzjHX0v74A1/qDfWHkM4AV+LP+q5GSZIkHbM5YAPM6YSQlSGElpIP36YEpDsXn1NadMT EZRKKSBlYYW6bHiISdIeUfRrHaTFqCCnLBjaQxgygjkFvpMk5bRAPe3ElELc7lOMwGHakIQMyYH6 lK+80h+NPJap0qzdpxis9Kk0s1M0RZMpHwEZH6GY6lEZRtdjjDw9hbuNVtXQmRpNOo3GkJMzNZra jKZnajSzGc3O1GhuMVp0jicjGy1sRuMzNVrajD7TjoiBzegzHaexOT4ejZZnanR3RFRFxJkGF9wd ETE+146ILRGRoHP1tCUiqhLzTI22RESKzzS44GZE5AS16zHKK7OiJLQsqGbX6FjBC0SGrccIx4xi KhFgJOWUfKXeU46jTAeIoPoGN8I7hxS155QIrczDQpSk4HuciVWF6lCzbLq7pmvgTzKpTC8TpbcT iwFkYt+ZqX6ZSKa3E5H0dkLvHuYi03elwEVmen8C4dvmy9GlkYdzyWF7RUN998vowkI7V3VwKSdP EOKUwejx/dw97SCTs9QykUxvJ5IkaAMZaRROQo187Tk6TIdZBzi/OUvSqMkEl6CdVWE40DrA+fmj Xu4RAKBhhyESw8z6I8GpHGqdSEU6GDTPT3DDJwQZ9mcgOsw8//mtnRHS8IcaRNp9Rm+/GSJS6a3G MD5S2SsRQgECKgRDHBoN7PJVjYM514eIQ9+oVT6TykVAlTpikEpKVQqcCMG5qgkHqtQQEhDobVtB yzikPitBICKGHUSQDLM7BAnJERxsFMNh1SthDZ8wathFBOEwO0TOdGSvz6moypIZdhEBMcwukbPx SXmmId/ttrvmoYaHjWYnZIdttlpvltr6bP75kGebXVbfbXnsuuAxPxWYZNn2kz6+CDLlxn1+cJ5A G/0Arug5esaB9Dp6Vjh4oo/tPR7Kymbz7U6f5/ymdPsL9K2jrOth3ud9mPd9eZB3kv3rYxf+j+P/ L/9or/6kZI8QuprjIY/21lWJkY72auuWM9fWc+lHe0tvHHEju810f7v9pAkN09VdcZJ0FnxSUB8a PmIr0o7c10PDYQ9xPTTcI/l6aPh5HxqW3oeGJ6YMW+VUphGrSEKLX9hsv3+BsrfbbP+wuCkhNw2l XTzx7pPKyh2LXCfzs+V2c+slr/MhzIOVEsi4/Tm4Bv6of1WC6F2dHIuOYtxaL7PfVbK3+pytZmsl JVOZot7iyxllDKnMR8WcgqCCJKKqOFnpLFJ/8rSeeLXkasnVkqsll2sJ6rKklD8ttU6PWicZyxaq JtGl+V6N2/f3O1WT50tHmefzdKe3pOvCq6/ZkoDZx6slV0uulihLunLyjnGbuozb9rrhlE/XdXH8 1ZKrJVVLxNWSqyX9luAujn4KSxgG5HlZAieAT+A55IHnZsk5xPerJVdLnqMl55A9npslXRGwI0sn 9izdLvN8nu5qyTlY0nWR3tWSqyUVSxC4WlK3RATNGz/u5qpsmchmh8NscaNXnR/U7y+zF7ihqPPG ke6l2od9nv3jdj3PFtu7IkAoRdFCi/Xf4hF2682HbLtSEvYfVTRa3OTRwgtRJeVaC9QK1vssL/aR LqOl38xUrPy43mw+HzfU7isiIeq+Msp9T2mvqPqeUlhu90OS0aS3uvSacd0IOtpG0N5302oZ5U7Q j5+2O/Ujrx7IhHlvBO1VeqKNoP39Qz3kfz/kD/nj1tdjVyi2vpc9oPxKsUVkepPf3uc7R9np7o/p V9W/yfR4f8xsuVyyvLoVFCxncyQWXqr07/HlihD7JlMoFitIu+6P6W+ZzU2m/cZ5bzL1E9m7ydRP XOj9Mb5aQraCejqGLVYAl9sPb/LFx6lKAPK7+8NU39X3cK/3IAJ9sRkKewT/naa+8hGSpGwqX6RO Z/vPm0V5l6FSIIjePhjmHrrIaXn93TFKFFtAP5V7/SDSLRKHuQbQJRDl1YvliD192kBZ3LZIaLBY ArosVrZGWEwWfMEfN2d+2G0f7h+3aBZXIga6uHcLqbc8lw2ffkJdN3y6pIiWDZ+zyobPXlG9t8Qg RjGiknMGarfEsNotMRgwV49Uj7U9nfFHTFU7nKiO2D6fa7wlxlmP8VQbOB7QpfoMfeOArt8TQOMT ON4S4/0QXYAmP5uRxebeW2JGshl32uxwS8xINpNum/tviRnJZtptc/8tMSPZzGw2912mMZLN3GJz 7y0xI9ksum3uvyVmJJtlt839t8SMZDM0h0XPW2K8je++JcbT/GZMJNhAn5OwYrJ72dH4qEZ1coRr Sy5aV78leJNPfKhHWGESB1dJGKpkBlhfdz+AfCywhiWp94FBaApll48Yh4QiymRQKgibWYkgsE19 ENxwt1BcLli2Gk4RwNGtpu8NEMwlQ08sEk8PNXIgypiBiFW7KURluAlcJI9XQUCOWldBJOxYWIY2 TBvaMLQz2dCGrvOc7jLDBxULeC3Yzh7kkB6rBOGQDdFNNFxVtTTBsUQpI1kFaYQo4EzDoIYYyFXv FFJwLr9c1uHZzRtlAwPcAHozXoCTYiDEAPLwmtjNQypbBIQDJrxDXemhRpHCiABlglEtXhmrYL14 6INUPcQeoUn6BrMaDCfhSECR6+qFj8x0CW0LwptS5iOEN6XMAZ79CcLr2W4bhSoTQhqu4UEG2FeC nq13F6jKYph26yuv9EejCOb6Epg2KFwaQF8p/MGZJGDgkQ4xDIGqSBAJm11slNxClX4GlDozYL9S eEgIStlgIx2Svslp6ZNGSS8oMiDzKB4mPg48+geOLKgxYyAkNiDziDTAYZP4BGDRAl2mzGSJr7Gl VxoTERKTIzSvmifo6yefZUvxze5Ln9TLbAEgM1y/iVmlzPau500+ESfJnUT6qpSyIfKx9BUkEekr XTJAlU8GeHb/vvD40TulhVSdi0hY9P5y3d2xr1FuuNpVNYdhKjk1sH1Z3Uzsc98+UfqDNPwhheFq V1UcfiX5L6rXsQJivfeyhetmA9UDglOMh8vuYNDsB6rXSAIpOe34hPBANUGaOc7e+S0OBQQgcFTh DQ+pP9ujip7N/opaTb1O0vflGi64hXygObPzG2llwx+6SGr7Aw9TD5xoMQVzShChMGjsxfWKSRAV 6tu9CIjKOENDk4yqh/BjHgw5HKwXETREvRHaQCwyvVc9yndXr+sEJdJwARGgw4yAg/Z2KsM20uFG WafkGNYGQTW1ujbnc2nOjTJBTxmS5jQFkqwSvIJXC9v3JQ5ckp/R9DtulB96xG8N+apKGGiS7BlM N+NGQaJK3/YChWowz3TCLKzVNAoSrq/CafuEGe5U+lpSqUZBIjBtL1KoOuVZ79yJ81CjPFHRsj09 jzioTAQ93cca56KkY89pF5ifbhBNuc2Gpx9Xnu52TCpzCDvTT7yC4H0VDvdaplxASv+OkKShe+uS L3ThRrkvqWjfDqUnsweK2wxIOljchmE+IY0CX0rJWhWRypeGmVyFRMAES6L2HcOIqxbDVbEXFpVI vYyWgID2UiBiYKCpVko5bd3Tnjofljol5pSF5X4ENTwkYHsBB1E+0FSrRAgONdUa1mLqlaqECLUX cBDFcJBK9QSZXrG5FTKOhf/WRLcWqTNJBqh8uk/T8w2QxhtguL1kpKo/OcgbUNWPysuGG+lF+qV2 KIIWFEi9EpYq9W8vQ6kvDjM2FjfHxp9hsUcPCiBW+QaQztEj9PbF6ikmj9sXe+0Zn5Tjef2im4fd rl/slXWl7lypOzWlV+pO/WpHnOfzuaoJhqPu1FXNxTjUnerVjv0t89KpOz5XO/pJDr3a0VfLlefT K//K87nyfC6G5zP35vkkvsDxSakd6u18gaObvEEucFR6SfdlB2/e/uHXSXaY7T8eM5MnaOoEQ0rL Sko9jnqq7E7nlYebmarWEMj2+WK7We5fOioqP8IyPIvYf8oXN1tVR/1L9kp381f7z/tX5fde3Txs Pkz1kxWZ9Val58rk/T8pQXtdue3Vo6z32Z0aGVSC7PoYDSdlP2nbQZYVzio+UAb+bsqrbL7pzG30 odcFqdyQLZZkyWnlJ+aLBdTHgVxVffk9soSUgC8ZG4EC63u/q70dqT5pzRZdVLVsnq3EiuWqEKsI RbXnVtZxTOtLLRZVHSmczbj6qKaf85hg7FX812WkjmuiGCmJ8xPXhWJEyqylIhLpJMX9bdXTILZa sWMatz5Mi655DD8FHljlLNsPmkOsEy7WCkSuegCXTIKjno/5VImfF8ENFQlKkEykctBVmV7p2ZLq aKyjJtGp1WIRKJqqWqOaWuX7fPd7RTzPdWK4CpOOABa8lnuqzFA1t+1iulrf5lM1Enwo2grQoR8H agFoDssMvXyT+5l6gH1+KDJPNNMZOg5rhso9y9Wieqe4zgany/Xu8LnahHSjgatFO+FyVgQ5zKuX ly9Ugb8+VC8uf6kSkMNLRAt36WeSPFAVQqR85RXp1fu/58Urz0OlA1nN1QueuOoF9SvYSXH9Nwlr tK7Jnoc892TPfQTzSfbYBHclN2mTDIuimCTDInaQJMOiLzzJsPkmcZJhU5U4yXBRlSjJsKjqSDJs xgUnGe5CHZMMV4GxSYarHp8kw1VmQJLhLjokyXCWHpVkuD+Dd5Lh4Z64JMNZUXyS4a4qJMnwkB6c ZDj3Mcckw0Oee5LhPoL5JBligruudUqbZFgUxSQZFrGDJBkWfeFJhs03iZMMm6rESYaLqkRJhkVV R5JhMy44yXAX6phkuAqMTTJc9fgkGa4yA5IMd9EhSYaz9Kgkw/0ZvJMMD/fEJRnOiuKTDHdVIUmG h/TgJMO5jzkmGR7y3JMM9xHMI8lAtPv+O8/9VzZRxv1XRfBNflOZzYzrnqlx90zZ3k2rZZR7ppqZ W1AqatN7ym1TNjsSXyhmVVUZgXouFMvlQgXH5oViKz9VblubNOkPyVom27pQzNqAjNmk1biwrU3O It22NjmLi7pQzENL8AYkdx35Imekmt2oQFXs0ipSm/XmQ5FeKg2B8lcqNeM1+Wpcv9GbwYpNYIti 54pzIEyxu8nHNyvOa4mxSmT2eeEfHcN5sxTxMJ3my5lbUomWs1ZS6a4oOqn0UBWQVPpID00q3UcJ t6TSR55zUuksNGVSadkLtWjuhbKJcrjbjBBVpEKJWfVuM717tnK3Gawd9LR65PhRHCGpnD8kBHFM mek+C1mhetRva3BR9PhhvK2BEm48QuIj2JW1mUgmc06GnGWSAZ69zrBMI7POsEwiE0np3JedZYJg f9qPI2Gk+qJQvZENIV/fOcC47oSUkJDeDBu9WXIDckrQLnZMaGeuQPdMJ6zTNL06dM/ZJajuEvX2 DMgpgbpgMXEugUQgHD++9eF0GNHeqd8i4+wg3HAQl8BwkQeU1whwjQDXCHDuEYDUezPVK1vtyztE 14ngyx/uaMNBDJhuN+mkMkaGSImGC5G+8gp3sLo7GICGy00Y6kLSXmLGwBsu0bCw9s0dYpgk6jl0 IdFwkMCm617oMCnV+XUhWXeHxhoabnpBXei9y28vsDHJwBk13fsiutB7kQ1Gs4fMtPSUHtKMc0Ek 8pVfeqhRuAnA2mgslSJ23cFwaV0KNqo2QXgbi6WJu19vn2qUbUII0w05ogscFushRgAfqMU06NLu LmnkvrKgEbcvMqkMMzS0ljHSpQU3XUOWqjyU6WWi5GVXg1/r/u4aabnkwHjpylczANbzcgEgNF26 0slKvDh/8IY/qMqyDBeudNHHQ/1xJIoRFX/YgJNUCPqOeI8fPZMJkFIMCMAibDpZNLwuseGaG9h5 YV9kkAEMcjNfLGEY5lQKzJl0X6Kveaie/AuoHNSOObDz8r7LT1QQaHhIlfyGS2/wUOURE9J050OK fus7CVz6o57qC9X1seFaEDlMMfQsWgxqeIhyw7VAgFf6FPId3oweAo9NRmUfwS4qHwEbH0HlhxoY jrseI106FmQ0sRjN4JkaTTuN1hfJnKnRrNtoFU3P1GjebTQC5EyNFjaj2ZkaLS1G0zNt0xh0G43P tSPiZjCuGM3OdZzG5vgoUJHOIgqhwAwmb9ummB9kfjM2Sr37ohXeyTAJ4fmVtrgRdjGWyHDHH/h6 E0LciPFY3wfevhKLVRa1r8V/dPGPG0kKQbC9mQxJPNBKuV6KNl9NkbL41zcmASa830DpoUZGRBhq 7y1T+f5Ai+fnN5I1ki0KcHsrGeKyeoHUdSvZdSvZdSvZWW4lw40yhOqTb+3uzL6WNQjSqHCoIIZ7 Jzkaasn1/BM10iinGKKGWygZq3jIe9rd5CFykhXYYFO/2lVd0qhUGWPtrXSI4We+G1Vf3AdoWI9p FMMccMOtpAwMk2Q/Cw81ymNORHt7HaJ8mCT7WXioUR5zIQ33tlL8zDesxnioUcqqzMhw8yQFX8uO VdIoXAWHhlsnCf9apuBIo3CVEBnumCR4oA2q5+ePRuYvKTbcKEkueTdW+C1yX6bkfW+R67VniBsz tFIYpDTuhotevWbn3+eb5XrzIVvsH+722Vq1GaovjvkSFtgEgAkMkVy+rQKJsF5mv+e79epztpqt lZRsuynfLddHRfV1op9mmt/z1FY1j6j4BMKrJVdLrpZcLblcSyCYUDqBXRcSlPKnpdbpUeskY9li dnurg/FeRYT7e80KzZeOMs/n6a6WXC25WnK15NlaQrDfuK0PnvSP3J1Sz+f5rpZcLblacrXkmVrC 1b/+lhSkx3xXpUpms8NhtrjRBMQH9fvL7MUX7KFWJHrubTWyCh/2efaP2/U8W2zvigChFEULLe5l LR5hp+d8tislYf+xpNRHCy9ElfBELVArWO+zvJjcW0ZLv5mpWPlxvdl8Ps5w7p9EwglQ/3ZN9Png yntEGXHlVOPf0uLKe8y44spHxJX3vJtWyyhx5R/La71fPZAJCGCV9yg9Gau8r3+oh/zvh/whf+Sz H7tCsRZR9oDyK8XE9vQmv73Pd46yU3LQ+1Q5ctAJyedsKesc9JyJ6gU+Lqr07yFJ5tjKQadzlucc dHPQ+1pmm4PeZ1wAB91HpAMH3UdcOAfdT0sYB93LMWyxAuX9OiowLz7qS2ryu/vDVN/gU1wiA4G+ YweFPUIIqtxPPlLNuWwqX6ROZ/vPm0XJFlcKBKldaeTlHrrIaXl10jFKTLebfPqp5E5DpFskDnMN oEsgaHlbUjFiT5/Q24TWwee+YgnosljZGmGxAybcU54bJtxHqDsmvD+Zs2DClzVMeI8oB0w4hkJS ojelVjDhhNYw4RhiV49UdwQct76qmoVBBFVlIg2H+GWVmvjlBI6jHtOGgOMJHMH0fp3GhgBPsd2b G2GwqX2nS/QWGilUDhPiclh3uSoVDcfk9FExwx6MCJc/7cEgsgV4SuRyX3nlodiGOwQ2nJITZipj vDsoEKKFkRjTHbjuDoyIidtvZjBeYOsgdXcQlcmYKN/GDW0X6A7acAfhJqa3GbiYwB0qQODBhmtK g4ZT1nCJrlHaLjETFhO4BEgphnNJWFDndZeoMdWAQuZmyOKFthLRcAmTJhyymap4geOIrLuDQWBA ITNqPL9xge6AjTRYFQkG1C9DxtMal+iPRo6qj1O3RxAqB8pRBaCitU841QiiEkwa5JNGoiooM4BZ qfkOlgQ+kUwM1Uawer1h7aSRreoDKiZE9kDZKsUC4PYJlYTFnn4wBFVckPQLbMHLQ40EVqrc3YDI lgMlsM/BQ/WclgEGDSxKQp9hThvWp+oJLSsO7RoQ2UMltGcXi+rZLEOIGqiBWA6UzT6HHiQaHmLM MKOk19EG8RATgsjBPQSLCTckwqbc6vku08cEDQRx9LXkuwg0/EGEYZYJyYHy3WfQYhBseEifHWx7 iA2UAT8HD9XzYUYKlFnLQ2igfPg5eAg3PMShYbZOQwmfPERdt1rYPPR0bwE1gbZGnIlxkOk94dUv E8ugeg/VM3dGITJMLKrs7GuJGY08nVJkmFWE8KtZt2rk6aq0M0wpAj7Q+Hd+/mjk6ay4+KnlD3Qd 7c5ytGvUEIxTA69GDpURnqzKkgwCFFRloUYNodEabXaCHCojPLv+jhs1BKfcQKeRX3H+hxs1BJfC QKcR8mtZy8eNikFgaaDT6Dv9htlsNOSqQhX2O/ZKxSOI+8nPAhioNwI+3/kgJlWeSRDBHIa1xEZu Lxlqz8ojbr5NOd5DHGBVXQzpoRowPshD9WyfA4Db95Qhbr5OOcFq15ltDCx9who+IaR9VxniEA40 fp2ZT5zbYcxmScwbPte0Wjt59OrzWJ/XawGuoWntTLeD7pliSyaj8ZWcPbNDKvxBzDgJqwWwbHiI 8fYaXRfdM95DhOiBd+D4ITQfTtVK7seVqh4i9eqAq6bYvg2ti+75HOpJrJ6HSAqA5DJsZytseEjf Me1K90yQgwhChm9DiCKVXFEelIMQ1PCQbo6udM/n0Iai5yQeKcOPHlIhpb2u2cX7/Do8RBoe4shw xUwHAfQ5eChkHApjYNaLJS8GZo894x+NFz1HNTmQXkc1j4DN91U4w3y70zSIb0q/v0DfOsq6HrO/ HrOvKb0esy+O2etPFlJ9i83gfM7EgMfs66rmYpxj9tq65cy1ZV76MfvSG+URaSVsur/dflLd/ma6 utPnusks+JSwPsAvpMMBft+ocD3A7yv/eoD/eoB/hAP8ufcB/uQE96NSO12yk62lM/fZcru59ZI3 BBFe6YVkgkSa1N8mqp76gzIpY5Cx9FQsmxnXdH3cdN32bloto0zXy+z1iaQXlrHb9J4yY7fZkRhe ZVXVn1WX8CoK5isIZi14Vc68VLll1YwQglSAtMGrrA3ImFVbjQvLqp1FumXVzuKi4FUeWoJzX3cd +SJnpJKc7lXCUBQIxTzaevNBZ0w6eQyUv1LpKa/JV+P6ja5DivpjwXVeioNkBybWPr5Z8artu/w2 n+3zwj86l9K2i0DTaX4sy45u0Xn6dLneHT6Xnp9+2G0f7vUjoOWslQq7K4Ic5lVFCzWWr49863LO 7aVKOA8vES3qHK1K8kBVCJG8UFWRPv2SJsN5rltqWJdT0oGsvuwC7fpwX32SokHpF67+DBol3JJ7 H3nOyb2zUK/kvieptCT3q2ZybxPlQudSuZO+71nU6FyoRudCtZ1cVo9UVy+eFlEJEUQSjKjhGL4k lQUw6pqS2BYvjtduc1TZMFdbvPAR3L39QQXs1DKRBDhQZsh2KucXCY0vUv2nLGaw62U696Je94TY jDptxlDy87QZd9uMADlPm4nNZnaeNlOLzRSdp82s22YMztRm3rCZsfYOcixh1/p+aAB4XL0miIjo ANC/k1cAod6Fr6cLB4m6gwjg7S3lWODKRiwe+hxVB7HH7YGQmuhfaeIjHSDm1o8cpZFZh8ekkYmk b8LULxOSgXIDwRgAqh0D3/dVNGHZaMJEGLhCXKDESd7TuTmV1gzWhAmV6Zube2HqLhM7T6FVXx1s JOgUAwPdh5OukxaXPz7DRuZLOWyfI8CsukMt5QBNGSJDljBUDCCT+bbFfpkEhQ4ZPfuoIaAIE5VM hfbJvqFVo685R95ll6N8Drhq2SreBA3dsFEjMYjaJ0Iwldf0Y8j0Y4BU4XJTmrLdNupkRrGBuUY7 zzJHxi2JKDCdxEzhE195pT8aNbgaE9qnFTBFXXvxL84fjfpeQG5grpFOcs01Jx0xJ23McwgqDDQ4 0smtOeO2HOaPxhyKJMDAOqvxA69t+VzacmN6RwpooLBhAq/v7vzeXX1egwKETHy4KkP6+u7O5N0h 0Hh3DBvIdaiTdR0bQyDH5JzyIVSfxaAQEAMLDoFrPnSGbRk13l1xkVaLW1e9IOT67s7l3eHGuxPU wNiDnTeZxM6vUoTMJIJ0M2iqNULM1JBXYz+7e6heuSqDmYm6V13RvM5AXWeLki+AScgExTxocwyi jTbMeJs9iMFgq9zPoJfXa3qKgehjD15j2LnEsPr8AyUYGqh33ZzES5uPQaLhD45MdLrr3OI5tuVG Ta+Cafu+H019ub67s3t3uFHTU0pMzMPOe1TOeBwKqulxo6an6t+ytoBVf3TemhI9x0GR6d6hsTLn 0ieNWplhZiIcAnHt3+fXvxu1MuPcRErkXfdGRbZnASEHg2fRgkCVMPCwOgM3amWusvF29KKSX2vl 51UrD1PXFrwrVdwSAuQwu4+wvsFZEIHDqkLcqJtVrDJQKCm5ZtJnOFo3KnoBgImPWWUbklAnm7i2 g767c/Jzo/qWCJsIiVWOddI+MuiYf7nzjuW7a8wUSEZM7EY01C61c1uFxPXqmwHC2ivqCIuBZpK4 INRMrU+a4WHGimNOIR4ioOEhwdvr1gjTLmLsV+ChetXLVIhprxchJCpVAgsNxFUP0ZPkwNfdxSON 26FE2epRTF+ibK89QzC5tFIZpDSeoWXVa3b+fb5ZrjcfssX+4W6frfdZAe+tehHjicpFvQUXjAr1 Zg5Vwu3hMFOm7/LDg/r9ZfYCNxSpwcemyERZeNjn2T9u1/Nssb273+X7vVIULbRApBWPsNPO2a6U hP3HbKGsz6OFF6JK7IMWqBUov+dFL1hGS7+Z7bP9x/Vm8/k4vO6rIpmS2iHSE7RmE2VkLDMCUHrQ ms2MK2htXNCa7d20WkaLi0zDKGs2paekrFn7RyQX2SY7McHNqsqR4Ib5PF9IfQS6QnATTNUhwkuV /j0MVjC3EtwKLjJcWAlu1pZpJLhZjQsjuDmLdCO4OYuLIrh5aAkmuLn7WkOVjvRiHasfe818rQFr CDeRUO6Cw/BqHvKDuMXuLcabW+wh2odb7CXWj1t8FqLdqGkePX3OqCjbxVa15qIdMyl0Q+ChFjpz 2JyFenHYenLObg7bEjQ5bDZR/Rw2jgiG5fRuhcOmbxKrctjcPVKdNTlOK2GBAVcFq+CwvdCCVWnV Ma3kosc0aVJOK+mXjLhxWslHcN9SFNeHGvR8WYiDYN1BUo3dBk4R6tpLEe0gwM237CR0EAEEI4lZ bS3Q2UGo4SCKDQdqhey6Ui/eQcw8MZnOQYhLQCkR9cuSnR2EGw6SxHBqVWWVgzgIYWX3sC2IMMQp oYgJhIK6GKk6iALAmeEYq/rmMA5CAkAzSCVhFwNcswgYC3MQrTsISmk4s8Zl1+pIZBdDasQz7yJP OUgjNf5wREmQg1jdQfpqx/YhNs66Fkcu30G84SABDCfFOO46hxDZxYig2ExzStnFJJRE6OPuIQ4S dQepF2o4jqUS2mfuICGIcJ8mqTpINhzEkOE0FmNde/Auf5CGoO4hArDhrI+mrwwyCBEGVdRsrz/6 iHU5DeHuD9jwByHtkz2YyoEy52cwKEPU8JAwQbApGyp1ppIN7qHiTgpJUVhtAXHdQ/oAnYFehIfJ nZ9DcQEbyTNlJkysSqqfeeAKj+ywkT0zYKKQEjZM9nwiD3EGOAQwcBxqpM9M7y9qewgPlD4r19Pz ilyNbJlJE/uUgK83W4aNdFlVDgb2KWZfb7oMG/ky5yZ+JsbD5MsniVzKQ4xjQtWrD5o3bOTLAppI jaia/dT264V66LhfD6uYy6I9ZLmuQ7ou7rvL5NJ3fHd5k4QjwJDeglxn0SZsKQghNYzyOkY8YQ7F oNRnPj12ANRaYqNSESqfNzCQYNfZ39CWiFO2xOsbtr3hRqUlJDUsc0Ex2DLXCUZjRAAnQJXZQXXE I4/o0UMSmzixsJP3fvnVOqpXWrp8bp+3wXoS5uvIklG9rtKXRBiWbYAYaF3rGWTJiDU8JEycSECH qc2fRZ+qV1raZMOpFlmdI7xG3ucVeUXjDTMDX1I5aKBR8zn0gXqlCFUPaK/NITnQHOdzqKUxaHiI GFiJSA40x/kc2hCGDQ8JAyuxTp7yXuQyegiUjQhIxMIbUQ3Q0niE8iJJgLsew7dBPX5YL7FzNxpb jK7efnlWRpNOo/XFpWdqNO02unZn51kZzbqNrl3aeVZGc5vR7EyNFhaj6bm26WbkrxiNz7UjkmYw rhjNznWcJub4KFD9/uPUxptifpD5jdhIkAEji0TnxSfPJUUML7VJIxATZsLICvj1poikEfUpkO3N JojzgZbBhy+1Y0rh0kONFINS0N5sgjgZZhn8WXiokc9QaQL+8k6c9VfgoUbyxLAJAcz4MAvjz2Ic amRqjJtAu6plffGQ8J2fNnmIP+LBVPscaBq9SICSy6Q8vUwSvJJuk5l8ER2rRCe5TBzcnCwyA/tC owDg0AQupnSg0UKFNyGH35rGNb1R0KANoLRRbXBK2xuNVIE30LZzCTEBA40WrvLCAET1SscXQGSz Z3zciug5/s+B9Dr+f6Qbva8Cf+bb3UG16m9Kv79A3zrKuqJbruiWmtIruqVAt+hPFlJ9i+VoPldV w3DolrqquRgH3aKtW85cW+alo1tKb5TwDSVsur/dflLd/ma6utOkCDILRjpoKIyQDlAY36hwhcK4 y79CYa5QGFOnv0JhvkBhoDcUZgjKp1ZKrJVGJ1ZSFxiz5XZz6yVvIGooBRNA01QoNlHmCgVznh4I aTPjWlWMW1XY3k2rZZRVRZlkP0FkwwoLm95TFhY2OxJzG62q+pP/R27jakUBbHEbV36q3JJ/KJYr PFtZuY3WBmRM/q3GhSX/ziLdkn9ncVHcRg8twSm6u458kTNSyaT3KmEo6phigm+9+aBzMJ3pBspf qVya1+Srcf1Gl0tFmbTgOnfCQbIDqwAf36x41fZdrqqYfV74R+dS2nYRaDrNj9Xj0S26qJgu17vD 59Lz0w+77cO9fgS0nLXydndFkMO8qmihxvL1YVphgb9UCefhJdLNFQKtSvJAVQiRvFBVkT79knjD ea5baliXU9KBrL7sgmr+cF99kqJB6Reu/gwaJdzKBY9Rx7Fc8LHQuVxwFupVLvSkqZZyATXLBZso J4YkgVDoIzoVhiQRNYakENjVI9VFmi9rNBBIiSTnBnaS5JUV3drmYxc9xjWa4+ZjChAzs4GcnwAa n6C997j+FM7Np3dFKMRmZLG5uvX4nGzGnTbXdx6fk82k2+baxuNzspl221zbd3xONjObzew8beYW m+mZtmfRbTM+1z4ou21m5zo+Q3NY7N5znMb4nj3H7ubXYyJWZa4B+Cdx196U4Kj+eNyZcdMVdXHv sX3c2d0f9Xir/gAGnJ0EXTtRLs4fuOEPAg3wOsG7kApx/kBKsPGS8RT+UIFduM7C1X1CGj7RHd1A Mu6CKFxcG6nnIXovrAG9JkDX4d9of1BhQiaM5496jqPiLTWA1jjvOsdxce2DN/yh3pgBS0y6Tm1c nD/quRkmSBigYRx0ndGIHFMJISpDCS2kHz/69oNSKqWAlIUV6rLhISYN0DD9WgdpMSrIQfOO2ZQe wpARzCnwnSQppwXqaSem1HSjByNdZzQi25CEDMmB+pSvvNIfjTyWYdP9FAxW+lSa2SmaosmUj4CM j1BM9agMo+sxRp6ewt1Gq2roTI0mnUZjyMmZGk1tRtMzNZrZjGZnajS3GC06x5ORjRY2o/GZGi1t Rp9pR8TAZvSZjtPYHB+PRsszNbo7IupDY2dqdHdExPhcOyK2RESCztXTloioSswzNdoSESk+0+CC mxGRmy43o7wyK0pCy4Jqdo2OFbxAZNh6jHDMKNaoT0ZSTslXz1MDSZkOEEH1DW6Edw6pAUtKaGUe FqIkBd/jTKwqVIeaZdPdNV0Df5JJZXqZKL2d2H23lLtM7Dsz1S8TyfR2IpLeTujdw1xk+q4UuMhM 708gfNt8Obo08nAuOTRc9FCF/LLQzmWA9CNOO64ouiRPO8jkLLVMJNPbiSQJ2kBGGoWTUCOf4bKM TlDypc1ZkkZNJrjpwk8MB1oHOD9/1Ms9AoDpdkYkhpn1R4JTOdQ6kYp0MGiev0I8K31CTBcyIjrM PP/5rZ0R0vCHMF2/WLviIWGk0luNYXykslcihAIEVAiGODQa9KLKMeeU1hl8yeQzqVwEVKkjBqmk VKXAiRCcq5pwoEoNaYKg3rYVtIxD6rMSBCLThZjdV2zEjmKSIxP5KlFdEVa9EtbwCTNdgdl9qcZF juz1ORVVWZouvey+WONCfBLMParstvTlHvXaM8SRbK2UBSmNP0Jt1Wt2/n2+Wa43H7LF/uFun61V I+GIy1ZICpBcnFFSr+ZQBTEdDjNl+y4/PKjfX2YvWsdruFWR6UzMwz7P/nG7nmeL7d39Lt/vlaJo ocv1/uPxHM1CWaw9pHyTFy21eQjTX/rNbJ/tP643m8/HLrv3Eml091a9yemi6Kqr2TqFlVfH9jgW TSibQPuI5+3YUKFujg2VbnFsIRJ1iXzYzzP4Ak6yP7/7Qdu42G42+eLwXXaUv3m4m6tBAjmK2y8z NgHFP9lflW1/y9593ixudtvN+h/6kd/9+O5N8fx5sMDD7MP/BNkfXr/5488/Zb/l+4fbg3r+7f6g g9L3P735afr21+mPv759+/OP77Plbv17viu/89ubv/z82/TXf4vU/ONPP0yU3tnyG6gyBKTZWhlY ZmKR4aXec1ocXs8AzCqn3+2a5rcfpw/3y9khVxHlvx/yvXqgN69+LZt78Sb0a/4u26sXo9o/FCop plRUrrC2y7d0nrkW/kr98Up/Qf/MfpJ9UvH6u2ynnuY7lZk87G/0p4vtbvdwf9B/1YmL67MF60aD 6i5HCp3T6HOqx+SpPNU8yV7QBK09z37UzbxoJKW2yZCtNXyARF1j7tPQsMk/ZTfrDzcv9ve5clox UtRGB6p+VvduzfJ4cb9Y+yp4qxRUpK40feS7bL38i0p+trvvgarc9Kd/2m2XD4vD90sAm03AV8X+ sFMGq3f9y2r3vWrmj6JVq3uX79az27fFk32PffUcBU2yn9UwvNvMbgut+KW3wb/MNg8rlZY97PLd JHu/3d+s5zNfIdVnmWQI6OuuEASqxoHNEd0i68Uj/UXLhC9BGSx+me332bvjd45uXeYHNUb5tLC2 7H9/WO8+7rO72UFloqttgbLJdBPI7tVf9LvXT9Igmdj17Bf7ddHpmFFnXQqGNilP/f2n9U496ovX C03RKcqw4zvKWq8+Uw6H2Z/+faLKoNdv372ZZMxVZWWAeV3k5qr/FXY8sYU+oOzw+T5vDopOIssx 6y/57nM2X384vsSX2fvdZ92b1eCoM6Hffn79U/bj6z+9/vHN+//6BrJvXwZr4hhRxhihGQGSvSiq yNvth/VC+aoYglWH/EY5DGTvf3iFXnKcvV//8G2wutd7VTTpJ/mPnards59NuVaIuGKYLrMYHbC0 bF13OwrWw7IWP4fFAB1uzmN7KHIqnV46iopJfAOEeiS+AdL7El+MOkT+/J/vyYvVPvtGvwkl6E4F Hc0gXqkIuf+811NDn9aHm2y7U7+pvr7U1Kq77VL1j1/vD7qhbh5ub780TjwBYEK66nAvSJpdlBmS JlVnSQxJs5txhaSNCUmzv5tWy2ihl0UIIc2u9HSEtJ7+EYVetstOSl/rURVBX4MriCEkXqoKsLL6 1R76GiZ5TpYW+lpPyzTQ13qMC6GveYh0oa95iIugr3lpCaSv+TiGcSnmJSD5Jl98nKqMIL+7P0wX D7ud9vm8ThjzeY0hcDQv+QGIZB/XeCKSvUS7I5I9xfpwjM9ENFnwBX9klBXTM4+kMszD354LSc1P niP3zEOoB/esN/O0cM9wnXtmF9XPPWNCYCiBvi/RjXvW45HqQm0U98xNj3Gdtod75vEEsdwzv4ew HDnwsDmWezaGzbjTZjfu2Rg2x3LPxrA5lns2hs2x3LMxbI7lno1hcyz3bAybY7lnY9icjHvmZ7yV e+ZjfhT3LCKq2/bVxr7H5r5aH39Ecc8u0B+44Q8v7lmsPzq4Z2n8Ueee+fgkint2gW0kinuWwB8m 7tmY/ojinl1g+4jinl2gP6K4Z9Fjahf3LD4BsXPPfDwUxT2LbjFdEKu0Hmpzz3ymBaK4Z9FtyHxm Lk2f8pVX+iOKexbhjx7umc8jxHLPRpmewt1GO3HPRjE6lns2itGx3LNRjI7lno1idCz3bBSjY7ln oxgdyz0bZXY7lns2itGx3LNRjI7lno1idHdEdOOejWJ0LPdsFKNjuWejGB3LPRvF6CjuWXh23cM9 S1ePmblnSes9A/fM5w3Ecc8iChwr9yzVTCxx3yLlLrPCPUsmE6W3E7vv4XKXiX1npvplVrlnyWSS 9HZC7x7mItN3pcBFZnp/VmlcPqNLFPcsfHDp4Z5dlqcdZFa4Z4lk1rhnyWSSoA1kcdyzy5uzjOOe XaA/orhnCdaaTSSYZJEOBs3zx3HPLm/tLI57FhupOrlnKSsRE/cspXwT9yyhfCP3LKV8E/csaaVm 4J75tNAo7ln8KGbkniWrK8KqVxLFPbvQkT2Ke3YxPjk998xuz/gHhUXPwTUOpNfBtSNU7X0VMzbf 7vRZ629Kv79Arseor4eOr4eOa0qvh46LQ8f6k4VU34KAzOdMDHjouK5qLsY5dKytW85cW+alHzou vVEeGlXCpvvb7SfV7W+mqzt9VpLMgo8h6uPMQjocZ/aNCtfjzD7yr8eZr8eZuxv29Tjz43Fm4n2c OT1iuFQqrPWG6Qy1csciXxY8wu3m1kveIMhipZdOsP05koCF7Yp8CyKLKGNBxAFFloIIAkBoVmkK Rb1Zdl49riOhg1JjXdZiw7WCGbmCsbybVrMoK5iHAg0WWLtY1J20drHYkRpqZFPVqi8ec/zZSrI5 qiT9xRcWzS8gIXxU1eqL+hdk4wszXntuAAl3bjfm+sJmXGB94SrSsb5wtrDwRiXBTTES1jVAhNkx O7/d7vOSV6ukrnRajsOEFgsBq0LocSiZ7m8eDsvtJ1UiPagCvCgqgr2iPlB+TLzUIDedbbZf5Or8 K9DqnENcdXYh/FGu9rEMNZfMV8cqQofl3zVwWr9DlRY8ikfaHSxMvqpuADu+QpUBqOT2blPktqxV WTm3Y5XZzUpfHGb7j0UWrtMZzc8K94SezshLS3X6PT1spw979QsqXCp/bO+1dG0zD5SumkVeSN9/ 3i/U2DAt056nslz37byVSrvKV6k0WZbvsUjYpu/+692Pr//4xykj09VsfzhqkDqtlsQ9y7Kk1bSV VvdmhqYMt+D/TIvvwALciQnCesWIYlYQgr7TZNDNViVjD7e3KdUVMKLvJaQSQM6p4Og7DTtdfi8E ZExwfXoNfJfdrzcqOf1eA8hz1SB+L/9+N/us28f3UBDCOCGAYw4kJZyi4ifLfP17qKrC+vIUnRDg bbWVwRoqtKgtim6psdfZdpUVVNY6iD9UuBvgNVS6BfCqRLIJIYkKB4uoeuEAHjNEAQdArlrMuNYO I9cOlnfTahll7VAuBjxWxGElhEXrSUsIix2pSwibqlYJ0VqiKD4zclEFyXkjmexTVfwew3OjzV9+ guLVfEHsXFRb8zGXEDbjAksIV5GOJYSruDguqruW8IUEdx3qjdNmMXR4qoh00aKnthEOEh+6mOAs H9IczSry9zfbh9vlNNdD9JcZO523z1q1gPtDAL7kFSVqXFatcjO7V9qKJSmoJ7x56CPogaoivRhf i2l65aj8oOuZL6rmKLjRIpADufpS0uS76Zcpdoh1rQQDHwEsV6z6lpf5/rDbfp6WKeixWygtC/2q F0E6XBcGPOS5Lwy4CvVbGLCne5YKhrUqGIsoB84pUVWEZHrHlDPn1OaR6sasWM6pgx7jvqx+zqnr EyTgnHo8hP2IoavNCTinJ7cZd9rszDk9uc0JOKcntzkB5/TkNifgnJ7c5gSc05PbnIBzenKbE3BO T25zSs6ph/F9nFNn82M5p6FRveccTdR7NJyjcfZHLOf00vyBG/7w5ZxG+aObc5rAHy3OqbNPYjmn l9ZGYjmnsf7o4JyO5o9YzumltY9Yzuml+SOWcxo3plo4p5EJSC/n1NlDsZzTuBZjgVYm9JCRc+o8 LRDLOY1rQ51n5BP0KV95pT9iOaeh/ujnnDo/QgLO6emnp3C30a6c09MbnYBzenqjE3BOT290As7p 6Y1OwDk9vdEJOKenNzoB5/T0s9sJOKenNzoB5/T0RifgnJ7e6O6I6Mw5Pb3RCTinpzc6Aef09EYn 4Jye3uhYzmlgdt3POU1Uj3VyTtPVe2bOqfMbiOachhY4fZzTJDOxxH07j7vMOuc0jUz3PXrOMhuc 0zQyse/MVL/MBuc0jUyS3k7o3cNcZPquFLjITO/PBn3TeXSJ5ZwGDi79nNML8rSDzDrnNIXMJuc0 jUwStIEsmnN6YXOW0ZzTS/NHLOc0dq25g/yWJtLBoHn+aM7pha2dkVjOaVSksnFOk1UiHZzTZPI7 OKep5HdxTpPJ7+CcpqvUzJxT5xYayzmNHMW6OKdp6oqw6jWac3qJI3ss5/QyfDIK59Riz/inc4M4 p30e9uGcWmRdT/peT/rWtT6jk776k5IQSharOR7ypG9dlRjppG8bRmprPpd+0tcfRuoqOQ5G6q7l eoa4T/71DLHHY1zPEF/GGWLufYZ4ELioUkqtSb4nXLRP3kBwUUgn0E4DSgQXhWyCcIcilVhm8AWc ZH9+94P6kf1iq3rO4vBddlS6ebibK2XUUdx+mbEJKP7J/qoM/lv27vNmcbPbbtb/0Fikdz++e9OC LoUKzFUqoSR9A1Udu5qtb3W2+Vu+f7g9TLKb7f6ga7jvf3rz0/Ttr9Mff3379ucf32fL3fr3fFd+ 57c3f/n5t+mv/9YwpYuzVboKT7JN/im7UUn8i/19rpxdeK7mLaZ+Vj+tzqtfqAS7oUD2KXirFFSk rnQl8F22Xv5F9czt7ntABNCf/mmniqTF4fslgM3X7atCDZvK4L2qu1a77+F32aNo9F32TtUWs9u3 xZN9j331HAVNsp9VH99tZreFVvzS2+BfZpuHlWruD7t8p2rM7f5GFTG+QqrPMslUkoABQRAgNSgj d1kvHisxLRO+BGXn+WW232fvjt85unWpwujiUGOAecv+94f17uNe1aEH1cPVKKbLykw3gexe/UW/ e/0krbrCpme/2K+LHsKMOutScNfYWEh56pw/rXfqUV+8XuiKtii6ju8oa736TDkcZn/6d11Evn77 7s0kY64qK6PB62LMU/2vsOOpzv+AssPn+/o+JleR5QDzl3z3OZuvPxxf4ktVq3zWvfmwLdB0v/38 +qfsx9d/ev3jm/f/9Q1k375saOqibbc1qfqMMsZU9CFAshfFnNPt9sN6oXxVZMGqQ36jHAay9z+8 Qi85zt6vf/g2WN3r/f7hTj/Jf+zWStPPLT5doLhiTC1H9Un2qZCt5zkcBeu4psXPYRHhws15bA9F jNFIvqooPkH2C5ACSIRhQh1JhGHCXUmEYdKtJEIoJhimmeu0iTIjzBEg6UmENjOu85Pjzk/a3k2r ZSSbn7RpPeX8pM2OxCRCq6r++clOEqEqLZd85qdK/x5CMzazzk/S5YrmcGElEVqbj3F+0mpc2Pyk s0i3+UlncVEkQg8twbOI7r4Om+Zzlx8xzefxEAHTfB6PEDfN5/EYntN8HpKDp/ncu5jbNJ+PPOdp PmehXtN8PfmYZZpPNKf5bKKcUIGSCagR566oQKtHqnsbIlGBLnqMWxt6UYHOTxCPCvR5COspHWeb 41GBp7cZd9rsigo8vc3xqMDT2xyPCjy9zfGowNPbHI8KPL3N8ajA09scjwo8vc0JUYE+xvegAt3N j0QFBkd1+1b0uPfY3oru7o9IVODF+QM3/OGJCozzRycqMIU/mqhAd59EogIvro1EogKj/WFGBY7n j0hU4MW1j0hU4MX5IxIVGDmmdqMCYxOQPlSgu4ciUYGRLaab+5bSQyZUoPu0QCQqMLINdR0zTdGn fOWV/ohEBQb7oxcV6P4I8ajAEaancLfRjqjAEYyORwWOYHQ8KnAEo+NRgSMYHY8KHMHoeFTgCEbH owJHmN2ORwWOYHQ8KnAEo+NRgSMY3R0RXVGBIxgdjwocweh4VOAIRsejAkcwOhIVGJpd96ICU9Vj XajAhPWeERXo/gZiUYHBBU4PKjDNTCxx387jLrOGCkwkE6W3s44KTCQT+85M9cusowITySTp7YTe PcxFpu9KgYvM9P6sA+zcR5dIVGDo4NKLCrwkTzvIrKECk8hsoAITySRBG8hiUYGXNmcZiwq8OH9E ogKj15rN8KREkQ4GzfPHogIvbe0sFhUYF6ksqMB0lYgZFZhOvhkVmEx+ByownXwzKjBhpWZEBbq3 0EhUYOwo1oEKTFRXhFWvsajAixzZI1GBF+KTMVCBNnvGPz4bggrs9bAHKtAm63oU93oUt671GR3F 1Z+U/D62xD2oQKuq/qO4dVVipKO4LVSgtflc+lFcb1Sgs+QoVKCHlushX8eHuB7y7ZF8PeR7ykO+ 0vuQ7xAsP60UWbNwP5Zfr7yhWH5yQrq4cUlZfghMgL1sCQAZhQl1BBmFCS+QRVps5ZC4LkcSuGII qVYykhIJ7VR6k8iFat4Hu61UToC91ZkEq8qzkAtVaZqtlJIHlSnrojuZZGSXDP39q5rU4qMW/ec/ v/mp7gaiOz3kHSKf0H92viVzFBfAtwwXmJxvWZrSS0ns5VvyLr6lo4IYvmWYCn++paOefr6loyA7 39JRiBPfsldWBN8yQHYQ39Kux5VvWUhBXUPzIHzLHpUhfEt3kbF8S19NkXxLX3U9fMtgcX18S7vg Lr6ltzndfEui809sRzz3hkDuKC4sBAYKHCIEqlS9l4bdGwKFJQTKbij6k4LIEBigIigEuuhxCoEu gnpDoIsQ1xBol1UJGdA/BPrKDg2BFj3dIRA2Q6Cc0K6SeagQaFMZGAIdRSYIgV6a4kOgl7r+EBgm ziEEWgRbQqCfOT0hkHaN64EzI+FCbXV7qFTLHINd5M//+Z68WO2zb/QbUILuVLDRK60rFU/3n/d6 4f/T+nCTbXfqN9XXl3rF7267VP3i1/uDbqCbh9vbaqNEoHu6y2ux2i7KfK+d3v2VmPVsN+O6wDzm ArP93bRaRqIFZrvW0y0w2+1IynruUdW/wNzJeoaSEj4nXqr0783gIs+tC8xyjvL5nFlYzz3Nx7DA 3GNcyAKzh0iXBWYPcRGsZy8tgcvAPr4OWQb2kR+8DOz1EN7LwF6PELMM7PUYXsvAXpIDl4F9upjL MrCfPMdlYA+hHsvAvfmYZRl4Vl8GtotyYD1jQhCmAAhH1nOPR6qbU6NYz256jHtTe1jPHk8Qy3r2 ewjLMWsPm2NZz2PYjDttdmM9j2FzLOt5DJtjWc9j2BzLeh7D5ljW8xg2x7Kex7A5lvU8hs3JWM9+ xltZzz7mR7GeI6K67Sxh7HtsniX08UcU6/kC/YEb/vBiPcf6o4P1nMYfddazj0+iWM8X2EaiWM8J /GFiPY/pjyjW8wW2jyjW8wX6I4r1HD2mdrGe4xMQO+vZx0NRrOfoFtMF7k3roTbr2WdaIIr1HN2G zJyQNH3KV17pjyjWc4Q/eljPPo8Qy3oeZXoKdxvtxHoexehY1vMoRseynkcxOpb1PIrRsaznUYyO ZT2PYnQs63mU2e1Y1vMoRseynkcxOpb1PIrR3RHRjfU8itGxrOdRjI5lPY9idCzreRSjo1jP4dl1 D+s5XT1mZj0nrfcMrGefNxDHeo4ocKys51QzscR9O4+7zArrOZlMlN7OKus5mUzsOzPVL7PKek4m k6S3E3r3MBeZvisFLjLT+7NKIPYZXaJYz+GDSw/r+bI87SCTs9Qya6znZDJJ0AayONbz5c1ZxrGe L9AfUaznBGvNJvplskgHg+b541jPl7d2Fsd6jo1UnaznlJWIifWcUr6J9ZxQvpH1nFK+ifWctFIz sJ59WmgU6zl+FDOynpPVFWHVaxzr+UJH9ijW88X45PSsZ7s94x+f9Wc9O3jYmfVsl3U9ins9ilvX +oyO4upPSgDzEs2trOceVf1HceuqxEhHcRus557mc+lHcT1Zzx6SI1jPXlquh3wdH+J6yLdH8vWQ 7ykP+c69D/mmZz2XSruIjf6sZwd5g7CelV48oQHVhC/rWSkiE5CcaNQvVAcnzS9qyP3nXb7Y/p7v Pv9ztt5ny1zJXszUaPNdofGf1R86i3241zWQ+pHN/qBeWfTTHGpgrlJDWWapGqYAGWU6q07gNTuy KUyqFdmEaDdp2rcKtIgy3/jD1S+mhyhZzLhWbiNXbpZ302oZ6So3i9aTVm4WO1JDlGyq+iu3TogS mC0wpDMvVfr3yGq2ZNbKbTabSzRb2SFKtuZjrtxsxgVWbq4iHSs3V3FxECV3LeH1lbOvA+srZ/kx 9ZX7Q4TUV+6PEFlfuT+Gb33lLjm8vnLuYo71lYc89/rKVahffWXPxyz11aJVX1lEOUCUkGrAhAjJ 3CFKNo9UV31iIUoOeoyLPv0QJdcnSABR8ngI+/5lV5sTQJRObjPutNkZonRymxNAlE5ucwKI0slt TgBROrnNCSBKJ7c5AUTp5DYngCid3OaUECUP4/sgSs7mx0KUQqN6zya9qPdo2KTn7I9YiNKl+QM3 /OELUYryRzdEKYE/WhAlZ5/EQpQurY3EQpRi/dEBURrNH7EQpUtrH7EQpUvzRyxEKW5MtUCUIhOQ XoiSs4diIUpxLcZCxEnoISNEyXlaIBaiFNeGOg/gJOhTvvJKf8RClEL90Q9Rcn6EBBCl009P4W6j XSFKpzc6AUTp9EYngCid3ugEEKXTG50AonR6oxNAlE5vdAKI0ulntxNAlE5vdAKI0umNTgBROr3R 3RHRGaJ0eqMTQJROb3QCiNLpjU4AUTq90bEQpcDsuh+ilKge64Qopav3zBAl5zcQDVEKLXD6IEpJ ZmKJ+3Yed5l1iFIamSi9nQ2IUhqZ2Hdmql9mA6KURiZJbyf07mEuMn1XClxkpvdnA+3jPLrEQpQC B5d+iNIFedpBZh2ilEJmE6KURiYJ2kAWDVG6sDnLaIjSpfkjFqIUu9bcgZVIE+lg0Dx/NETpwtbO oiFKUZHKBlFKVol0QJSSye+AKKWS3wVRSia/A6KUrlIzQ5ScW2gsRClyFOuCKKWpK8Kq12iI0iWO 7LEQpcvwySgQJYs94x+fDYIo9XnYB6JkkXU9ins9ilvX+oyO4upPSrJRDnEfRMmmqv8obl2VGOko bhuiZGs+l34U1x+i5Co5DqLkruV6yNfxIa6HfHskXw/5nvKQ79L7kO8gEKXeLNwTohT8EJEQJTYB xL+aCIAo8QnoojWVWdX6TmV+f+WciL9Nsn3+YTV7uC3wQqrAnS1UcF7fVwI2IXpXdLYvv8ZVC+SC 0aUKB0eLyZNzjpJJ8Xv/S+ULAPytapmYqFrZ91X24Z0ChfbQj4Kk2ulHcqKK5zTlm0WUmYGrqokB 6EcWM64l18gll+XdtFpGupLLovWkJZfFjtT0I5uq/pKr+KyTfjTzU6V/j+dysbSWXJRBIRdzO/3I 1nzMJZfNuMCSy1WkY8nlKi6OfuSuJbwwcteh3jitpMzFc+hRf1pMgCoFK6rrIhwkPrTucpYfU3e5 P0RI3eX+CJF1l/tj+NZd7pLD6y7nHuxYd3nIc6+7XIX61V32dM9Sd+WtussiygWuBCWlTDDiDley eaS6GhQLV3LQY1wM6ocruT5BAriSx0PY9zW72pwArnRym3Gnzc5wpZPbnACudHKbE8CVTm5zArjS yW1OAFc6uc0J4EontzkBXOnkNqeEK3kY3wdXcjY/Fq4UGtV7Nu9FvUfD5j1nf8TClS7NH7jhD1+4 UpQ/uuFKCfzRgis5+yQWrnRpbSQWrhTrjw640mj+iIUrXVr7iIUrPR9/ON1NGQ1XclDU7Q8bXMlD cBBcydlDsXClKA/ZSDkJPWSEK7l6KBquFNeGOg/meIh1Opjj7I9YuFKoP/rhSs6PkACuFOX+IKNx t9GucKXTG50ArnR6oxPAlU5vdAK40umNTgBXOr3RCeBKpzc6AVzp5EangCud3ugEcKXTG50ArnR6 o7sjojNc6fRGJ4Arnd7oBHCl0xudAK50eqNj4UqB2XU/XClRPdYJV0pX75nhSs5vIBquFFrg9MGV ElR8LbhSGpl1uFIamSi9nQ24UhqZ2Hdmql9mA66URiZJbyf07mEuMpsrBSlkpvdnA/njPLrEwpUC B5d+uNIFedpBZh2ulEJmE66URibxjSzlFptYuNKFzVlGw5UuzR+xcKU4f3TiJtJEOhg0zx8NV4pb CbGtncX6JGjtLBquFBWpbHClZJVIB1wpmfwOuFIq+V1wpWTyO+BK6So1M1zJuYXGwpUiR7EuuFKa uiKseo2GK13iyB4LV7oMn4wCV7LYE3A6t0tU4OncILhSn4d94EoWWdeTvuOf9O16N8Oe9O3SeuqT vp1jU/9JX/1JSTxakV64kk1V/0nfuqomXKl90teqKvikbxuuZGs+nSd9O40LP+nrJNL9pK+zOD+4 kqvkOLiSu5aoM8SOOkLPELs1kvAzxG7yI88QOz5E4Blix0eIP0Ps+BgBZ4gdJUedIXbrzO5niF3l eZ0hdhLqfYbYkpRazhCvTGeIe7LBAdhNvUm+J7sp+CHi2E0YTmAACdaX3SQmAEwA9naYlZAULtRG SAqVaiEkKZFwQrpEetVgdlHGGkxCLlITkuxmXOumMesm+7tptYxq3fTUnUMqJ7ve01VOdjuSMpJ6 VPVXTp2MJDFfzTD1U1VUTjM0y62VE6eSUbK0MJJ6GpChcuoxLqRy8hDpUjl5iItgJHlpCaxvfHwd UoD4yIcc53k1e1djyfpQLT5eqrzy8BJpd2GQK1WSB6kCXDIJjuXsx3z6cD+dr3VWjQr7A81HiJTm Vyyu1gVzbTEMa0ZKOpCgWdoou+ulGSnKArII84pLXeAnz7Eu8BDqURf0JkrddUEO6nWBXZQDWwij AhCKkSNbqMcj1cWQKLaQmx7jWkgPW8jjCWLZQn4PYdnW62FzLFtoDJtxp81ubKExbI5lC41hcyxb aAybY9lCY9gcyxYaw+ZYttAYNseyhcawORlbyM94K1vIx/wotlBEVLftXYt9j829az7+iGILXaA/ cMMfXmyhWH90sIXS+KPOFvLxSRRb6ALbSBRbKIE/TGyhMf0RxRa6wPYRxRa6QH9EsYWix9QutlB8 AmJnC/l4KIotFN1iukAxaT3UZgv5TAtEsYWi25D5XEqaPuUrr/RHFFsowh89bCGfR4hlC40yPYW7 jXZiC41idCxbaBSjY9lCoxgdyxYaxehYttAoRseyhUYxOpYtNMrsdixbaBSjY9lCoxgdyxYaxeju iOjGFhrF6Fi20ChGx7KFRjE6li00itFRbKHw7LqHLZSuHjOzhZLWewa2kM8biGMLRRQ4VrZQqplY 4r5VyF1mhS2UTKb7nh1nmVW2UDKZ2Hdmql9mlS2UTCZJbyf07mEuMn1XClxkpvdnlXjjM7pEsYXC B5cettBledpBZoUtlEhmjS2UTCYJ2kAWxxa6vDnLOLbQBfojii2UYK3ZRFtIFulg0Dw/wQ2feLGF Lm/tLI4tFBupOtlCKSsRE1sopXwTWyihfCNbKKV8E1soaaVmYAv5tNAotlD8KGZkCyWrK8Kq1zi2 0IWO7FFsoYvxyenZQnZ7xj/X6s8WcvCwM1vILut6RvZ6Rrap9xmdkdWfFMgfDGfzuUqkhzsjW1e1 FOOckW3QhXoa0KWfkfWkC3lIjqALeWm5nr71ed3X07cmr1xP3z6dvoXep2/TU3lKpcSaHvtQeRzk DULlKfVS63PoUVdTbBqP8u7j+v5ef70cC1TK/pShVocFH1Wdj2joURMoKHB4PownEPiXMZ7UIQQm UE6wHW/kSx2KEGqhDgVL7aYOKZEIToC9GQU8fahQ+9OHSTU8/f8HUEsBAhQDFAAAAAgAxblcSvA0 yQ7GVAAAAL0EABgAAAAAAAAAAAAAAKSBAAAAAGRtZXNnX3NpbmNlXzIwMTcwMjI2LmxvZ1BLBQYA AAAAAQABAEYAAAD8VAAAAAA= =2D-nextPart2283738.6rFPRxsZn9-- This is a multi-part message in MIME format. [-- Attachment #1.2: Type: text/plain, Size: 2475 bytes --] (Resend with the log compressed this time.) On Friday 17 February 2017 09:19:15 Kai Krakow wrote: > Am Tue, 14 Feb 2017 13:52:48 +0100 > > schrieb Marc Joliet <marcec@gmx.de>: > > Hi again, > > > > so, it seems that I've solved the problem: After having to > > umount/mount the FS several times to get btrfs-cleaner to finish, I > > thought of the "failed to load free space [...], rebuilding it now" > > type errors and decided to try the clear_cache mount option. Since > > then, my home server has been running its backups regularly again. > > Furthermore, I was able back up my desktop again via send/recv (the > > rsync based backup is still running, but I expect it to succeed). > > The kernel log has also stayed clean. *sigh* It was working fine... until the 26th (so almost 2 weeks without incident). Now the error keeps occurring, and the clear_cache mount option stopped helping. Just out of curiosity, I used "clear_cache,space_cache=v2", which also didn't help. I've attached the full log, in case it's in any way useful. This is all with: % uname -a Linux diefledermaus 4.9.12-gentoo #1 SMP Fri Feb 24 20:09:28 CET 2017 x86_64 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux % /sbin/btrfs --version btrfs-progs v4.9 > > Kai, I'd be curious whether clear_cache will help in your case, too, > > if you haven't tried it already. > > The FS in question has been rebuilt from scratch. AFAIR it wasn't even > mountable back then, or at it froze the system short after mounting. > > I needed the system back in usable state, so I had to recreate and > restore the system. I think I'm at that point now myself, unless anybody has any other ideas. What I *am* going to do before giving up and reformatting is create an image. I found out that using the -w option to btrfs-image keeps it from segfaulting, so I guess the underlying problem has to do with the extent tree? However, the image is... pretty damn big (8.6 GB by now, *after* compressing with "xz -9"), and it's not even finished yet. So, under the assumption that the image could be useful to somebody, I would welcome any suggestions for where to upload such an image once it's done. > Next time, if it happens again (finger crossed it does not), I'll give > it a try. Yeah, I hope it doesn't come back, either, for either of us. Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #1.3: dmesg_since_20170226.log.zip --] [-- Type: application/zip, Size: 21848 bytes --] [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-28 22:14 ` Marc Joliet @ 2017-03-01 8:23 ` Marc Joliet 2017-03-01 9:32 ` Qu Wenruo 0 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-03-01 8:23 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 497 bytes --] On Tuesday 28 February 2017 23:14:54 Marc Joliet wrote: > I think I'm at that point now myself, unless anybody has any other ideas. For example, could the --init-extent-tree option to btrfs-check help, given that I needed to pass -w to btrfs-image? Also, the image is complete, so I only need to find somewhere where I can upload a 9.4 GB file. Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-01 8:23 ` Marc Joliet @ 2017-03-01 9:32 ` Qu Wenruo 2017-03-01 18:14 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Qu Wenruo @ 2017-03-01 9:32 UTC (permalink / raw) To: Marc Joliet, linux-btrfs At 03/01/2017 04:23 PM, Marc Joliet wrote: > On Tuesday 28 February 2017 23:14:54 Marc Joliet wrote: >> I think I'm at that point now myself, unless anybody has any other ideas. > > For example, could the --init-extent-tree option to btrfs-check help, given > that I needed to pass -w to btrfs-image? --init-extent-tree should be avoided normally, unless you're 100% sure about only extent tree is corrupted, and you have enough faith that --init-extent-tree can finish. Or it will mostly make things worse. Before trying any RW btrfs check operations, did you run btrfs check on the image? And btrfs check --mode=lowmem is also recommended as in some rare case, low mem mode can detect bug which original mode doesn't. (And it also helps us to enhance lowmem mode) > > Also, the image is complete, so I only need to find somewhere where I can > upload a 9.4 GB file. Is it a compressed dump? Dumped with btrfs-image -c9? If so, such large one won't help much unless there is some developer really interested in inspecting the image. Thanks, Qu > > Greetings > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-01 9:32 ` Qu Wenruo @ 2017-03-01 18:14 ` Marc Joliet 2017-03-01 18:27 ` Marc Joliet ` (3 more replies) 0 siblings, 4 replies; 23+ messages in thread From: Marc Joliet @ 2017-03-01 18:14 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2503 bytes --] On Wednesday 01 March 2017 17:32:35 Qu Wenruo wrote: > At 03/01/2017 04:23 PM, Marc Joliet wrote: > > On Tuesday 28 February 2017 23:14:54 Marc Joliet wrote: > >> I think I'm at that point now myself, unless anybody has any other ideas. > > > > For example, could the --init-extent-tree option to btrfs-check help, > > given > > that I needed to pass -w to btrfs-image? (First off, thanks for taking the time to respond!) > --init-extent-tree should be avoided normally, unless you're 100% sure > about only extent tree is corrupted, and you have enough faith that > --init-extent-tree can finish. > > Or it will mostly make things worse. OK, that's why I asked :) . > Before trying any RW btrfs check operations, did you run btrfs check on > the image? If by image you mean the device in question, not since the last time (see my previous messages). Or can I really run btrfs check on the image? In any case, I started btrfs-check on the device itself. > And btrfs check --mode=lowmem is also recommended as in some rare case, > low mem mode can detect bug which original mode doesn't. I did see differences in output the last time around (again, see my previous messages in this thread), so I'll run with lowmem. It won't be done until tomorrow, though. > (And it also helps us to enhance lowmem mode) OK > > Also, the image is complete, so I only need to find somewhere where I can > > upload a 9.4 GB file. > > Is it a compressed dump? Dumped with btrfs-image -c9? It was created with: btrfs-image -s -w /dev/sdb2 - | xz -9 --stdout > ./btrfs_backup_drive_2.img.xz (Mainly because I felt more comfortable using a separate compression utility, not for any rational reason. Although if you really meant "image" above, I have the feeling I'll regret this decision.) > If so, such large one won't help much unless there is some developer > really interested in inspecting the image. When I started without compression, it was something like 46 GB, after only 1-2 hours, so I expect the uncompressed size to be... very large :-/ . > Thanks, > Qu > > > Greetings > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-01 18:14 ` Marc Joliet @ 2017-03-01 18:27 ` Marc Joliet 2017-03-01 18:43 ` Marc Joliet ` (2 subsequent siblings) 3 siblings, 0 replies; 23+ messages in thread From: Marc Joliet @ 2017-03-01 18:27 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 698 bytes --] On Wednesday 01 March 2017 19:14:07 Marc Joliet wrote: > > And btrfs check --mode=lowmem is also recommended as in some rare case, > > low mem mode can detect bug which original mode doesn't. > > I did see differences in output the last time around (again, see my > previous messages in this thread), so I'll run with lowmem. OK, just to prevent confusion: it turns out I never posted the output of btrfs-check with --mode=lowmem from the first time I was seeing these errors, but I remember the output being (slightly) different from the original mode. -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-01 18:14 ` Marc Joliet 2017-03-01 18:27 ` Marc Joliet @ 2017-03-01 18:43 ` Marc Joliet 2017-03-02 9:44 ` Marc Joliet 2017-03-05 23:53 ` Marc Joliet 3 siblings, 0 replies; 23+ messages in thread From: Marc Joliet @ 2017-03-01 18:43 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1038 bytes --] On Wednesday 01 March 2017 19:14:07 Marc Joliet wrote: > > > Also, the image is complete, so I only need to find somewhere where I > > > can > > > upload a 9.4 GB file. > > > > > > > > Is it a compressed dump? Dumped with btrfs-image -c9? > > It was created with: > > btrfs-image -s -w /dev/sdb2 - | xz -9 --stdout > > ./btrfs_backup_drive_2.img.xz > > (Mainly because I felt more comfortable using a separate compression > utility, not for any rational reason. Although if you really meant > "image" above, I have the feeling I'll regret this decision.) Ah, never mind, it's not as big as I was worried it would be: % xz -l btrfs_backup_drive.img.xz Str. Blöcke Kompr. Unkompr. Verh. Check Dateiname 1 1 9.589,6 MiB 81,4 GiB 0,115 CRC64 btrfs_backup_drive.img.xz So 81.4 GB uncompressed. I have the space to uncompress it if need be. -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-01 18:14 ` Marc Joliet 2017-03-01 18:27 ` Marc Joliet 2017-03-01 18:43 ` Marc Joliet @ 2017-03-02 9:44 ` Marc Joliet 2017-03-03 1:09 ` Qu Wenruo 2017-03-05 23:53 ` Marc Joliet 3 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-03-02 9:44 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 5809 bytes --] On Wednesday 01 March 2017 19:14:07 Marc Joliet wrote: > In any > case, I started btrfs-check on the device itself. OK, it's still running, but the output so far is: # btrfs check --mode=lowmem --progress /dev/sdb2 Checking filesystem on /dev/sdb2 UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 ERROR: shared extent[3826242740224 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3826442825728 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3826744471552 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827106349056 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827141001216 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827150958592 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827251724288 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827433795584 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827536166912 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827536183296 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3827621646336 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3828179406848 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3828267970560 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3828284530688 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3828714246144 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3828794187776 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3829161340928 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3829373693952 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3830252130304 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3830421159936 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3830439141376 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3830441398272 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3830785138688 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831099297792 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831128768512 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831371513856 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831535570944 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831591952384 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831799398400 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831829250048 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3831829512192 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832011440128 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832011767808 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832023920640 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832024678400 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832027316224 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832028762112 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832030236672 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832030330880 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832161079296 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832164904960 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832164945920 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3832613765120 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3833727565824 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3833914073088 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3833929310208 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: shared extent[3833930141696 4096] lost its parent (parent: 3827251183616, level: 0) ERROR: extent[3837768077312, 24576] referencer count mismatch (root: 33174, owner: 1277577, offset: 4767744) wanted: 1, have: 0 [snip many more referencer count mismatches] ERROR: extent[3878247383040, 8192] referencer count mismatch (root: 33495, owner: 2688918, offset: 3874816) wanted: 2, have: 3 ERROR: block group[3879328546816 1073741824] used 1072840704 but extent items used 1129164800 ERROR: extent[3879516581888, 40960] referencer count mismatch (root: 33530, owner: 1277577, offset: 23461888) wanted: 1, have: 0 [snip many more referencer count mismatches] ERROR: extent[3951758860288, 16384] referencer count mismatch (root: 33189, owner: 1277577, offset: 519880704) wanted: 1, have: 0 This is 56 lines of output, the unabridged output contains enough "referencer count mismatches" to fill 1329 lines of output. Could that be because there are deleted snapshots that btrfs-cleaner hasn't been able to clean up? Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-02 9:44 ` Marc Joliet @ 2017-03-03 1:09 ` Qu Wenruo 2017-03-03 11:26 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Qu Wenruo @ 2017-03-03 1:09 UTC (permalink / raw) To: Marc Joliet, linux-btrfs At 03/02/2017 05:44 PM, Marc Joliet wrote: > On Wednesday 01 March 2017 19:14:07 Marc Joliet wrote: >> In any >> case, I started btrfs-check on the device itself. > > OK, it's still running, but the output so far is: > > # btrfs check --mode=lowmem --progress /dev/sdb2 > Checking filesystem on /dev/sdb2 > UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 > ERROR: shared extent[3826242740224 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3826442825728 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3826744471552 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827106349056 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827141001216 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827150958592 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827251724288 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827433795584 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827536166912 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827536183296 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3827621646336 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3828179406848 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3828267970560 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3828284530688 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3828714246144 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3828794187776 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3829161340928 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3829373693952 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3830252130304 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3830421159936 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3830439141376 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3830441398272 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3830785138688 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831099297792 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831128768512 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831371513856 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831535570944 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831591952384 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831799398400 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831829250048 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3831829512192 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832011440128 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832011767808 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832023920640 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832024678400 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832027316224 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832028762112 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832030236672 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832030330880 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832161079296 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832164904960 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832164945920 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3832613765120 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3833727565824 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3833914073088 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3833929310208 4096] lost its parent (parent: > 3827251183616, level: 0) > ERROR: shared extent[3833930141696 4096] lost its parent (parent: > 3827251183616, level: 0) The "shared extent lost its parent" is all about the same extent, 3827251183616. It would be nice if you could paste the output of btrfs-debug-tree -b 3827251183616 to check what tree it belongs to. > ERROR: extent[3837768077312, 24576] referencer count mismatch (root: 33174, > owner: 1277577, offset: 4767744) wanted: 1, have: 0 > [snip many more referencer count mismatches] > ERROR: extent[3878247383040, 8192] referencer count mismatch (root: 33495, > owner: 2688918, offset: 3874816) wanted: 2, have: 3 That's a little dangerous now. Some *DATA* extent even lost its referencer now. For example, data extent at 3837768077312, length 24K exists on disk, but no one is referring to it. At least the good news is, these just mean there is some extra space taken on disk, but not vice verse. > ERROR: block group[3879328546816 1073741824] used 1072840704 but extent items > used 1129164800 > ERROR: extent[3879516581888, 40960] referencer count mismatch (root: 33530, > owner: 1277577, offset: 23461888) wanted: 1, have: 0 > [snip many more referencer count mismatches] > ERROR: extent[3951758860288, 16384] referencer count mismatch (root: 33189, > owner: 1277577, offset: 519880704) wanted: 1, have: 0 > > This is 56 lines of output, the unabridged output contains enough "referencer > count mismatches" to fill 1329 lines of output. Could that be because there > are deleted snapshots that btrfs-cleaner hasn't been able to clean up? Ah, this makes sense. But normally cleaner should cleanup subvolume tree block and modify extent tree. How did you stop btrfs-cleaner? Thanks, Qu > > Greetings > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-03 1:09 ` Qu Wenruo @ 2017-03-03 11:26 ` Marc Joliet 0 siblings, 0 replies; 23+ messages in thread From: Marc Joliet @ 2017-03-03 11:26 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 14339 bytes --] On Friday 03 March 2017 09:09:57 Qu Wenruo wrote: > At 03/02/2017 05:44 PM, Marc Joliet wrote: > > On Wednesday 01 March 2017 19:14:07 Marc Joliet wrote: > >> In any > >> case, I started btrfs-check on the device itself. > > > > OK, it's still running, but the output so far is: > > > > # btrfs check --mode=lowmem --progress /dev/sdb2 > > Checking filesystem on /dev/sdb2 > > UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 > > ERROR: shared extent[3826242740224 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3826442825728 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3826744471552 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827106349056 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827141001216 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827150958592 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827251724288 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827433795584 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827536166912 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827536183296 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3827621646336 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3828179406848 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3828267970560 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3828284530688 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3828714246144 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3828794187776 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3829161340928 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3829373693952 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3830252130304 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3830421159936 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3830439141376 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3830441398272 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3830785138688 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831099297792 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831128768512 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831371513856 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831535570944 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831591952384 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831799398400 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831829250048 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3831829512192 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832011440128 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832011767808 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832023920640 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832024678400 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832027316224 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832028762112 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832030236672 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832030330880 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832161079296 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832164904960 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832164945920 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3832613765120 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3833727565824 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3833914073088 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3833929310208 4096] lost its parent (parent: > > 3827251183616, level: 0) > > ERROR: shared extent[3833930141696 4096] lost its parent (parent: > > 3827251183616, level: 0) > > The "shared extent lost its parent" is all about the same extent, > 3827251183616. > > It would be nice if you could paste the output of btrfs-debug-tree -b > 3827251183616 to check what tree it belongs to. # btrfs-debug-tree -b 3827251183616 /dev/sdb2 btrfs-progs v4.9 leaf 3827251183616 items 33 free space 1712 generation 291223 owner 2 fs uuid f97b3cda-15e8-418b-bb9b-235391ef2a38 chunk uuid 1cee580c-3442-4717-9300-8514dd8ff297 item 0 key (3988449177600 EXTENT_ITEM 8192) itemoff 3942 itemsize 53 extent refs 1 gen 271366 flags DATA extent data backref root 32126 objectid 1921863 offset 302157824 count 1 item 1 key (3988449185792 EXTENT_ITEM 4096) itemoff 3889 itemsize 53 extent refs 1 gen 271366 flags DATA extent data backref root 32126 objectid 1921863 offset 302534656 count 1 item 2 key (3988449189888 EXTENT_ITEM 4096) itemoff 3852 itemsize 37 extent refs 1 gen 244831 flags DATA shared data backref parent 4691081252864 count 1 item 3 key (3988449193984 EXTENT_ITEM 8192) itemoff 3815 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644000256 count 1 item 4 key (3988449202176 EXTENT_ITEM 4096) itemoff 3778 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644000256 count 1 item 5 key (3988449206272 EXTENT_ITEM 4096) itemoff 3741 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644000256 count 1 item 6 key (3988449210368 EXTENT_ITEM 4096) itemoff 3688 itemsize 53 extent refs 1 gen 288553 flags DATA extent data backref root 33295 objectid 1921863 offset 233136128 count 1 item 7 key (3988449214464 EXTENT_ITEM 8192) itemoff 3635 itemsize 53 extent refs 1 gen 283033 flags DATA extent data backref root 32994 objectid 1921863 offset 694497280 count 1 item 8 key (3988449222656 EXTENT_ITEM 4096) itemoff 3598 itemsize 37 extent refs 1 gen 279261 flags DATA shared data backref parent 3829567111168 count 1 item 9 key (3988449226752 EXTENT_ITEM 12288) itemoff 3545 itemsize 53 extent refs 1 gen 283033 flags DATA extent data backref root 32994 objectid 1921863 offset 694648832 count 1 item 10 key (3988449239040 EXTENT_ITEM 4096) itemoff 3508 itemsize 37 extent refs 1 gen 244831 flags DATA shared data backref parent 4691081252864 count 1 item 11 key (3988449243136 EXTENT_ITEM 16384) itemoff 3426 itemsize 82 extent refs 3 gen 287487 flags DATA extent data backref root 33204 objectid 214635 offset 389120 count 1 extent data backref root 33205 objectid 214635 offset 389120 count 2 item 12 key (3988449259520 EXTENT_ITEM 4096) itemoff 3389 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644000256 count 1 item 13 key (3988449263616 EXTENT_ITEM 4096) itemoff 3352 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644016640 count 1 item 14 key (3988449267712 EXTENT_ITEM 4096) itemoff 3315 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644016640 count 1 item 15 key (3988449271808 EXTENT_ITEM 4096) itemoff 3278 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644045312 count 1 item 16 key (3988449275904 EXTENT_ITEM 8192) itemoff 3225 itemsize 53 extent refs 1 gen 291191 flags DATA extent data backref root 33530 objectid 1277577 offset 392007680 count 1 item 17 key (3988449284096 EXTENT_ITEM 8192) itemoff 3188 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644016640 count 1 item 18 key (3988449292288 EXTENT_ITEM 8192) itemoff 3135 itemsize 53 extent refs 1 gen 287570 flags DATA extent data backref root 33214 objectid 211018 offset 131072 count 1 item 19 key (3988449300480 EXTENT_ITEM 8192) itemoff 3098 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644016640 count 1 item 20 key (3988449308672 EXTENT_ITEM 4096) itemoff 3061 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644045312 count 1 item 21 key (3988449312768 EXTENT_ITEM 4096) itemoff 3024 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644045312 count 1 item 22 key (3988449316864 EXTENT_ITEM 8192) itemoff 2987 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644053504 count 1 item 23 key (3988449325056 EXTENT_ITEM 8192) itemoff 2934 itemsize 53 extent refs 1 gen 287571 flags DATA extent data backref root 33214 objectid 212344 offset 131072 count 1 item 24 key (3988449333248 EXTENT_ITEM 4096) itemoff 2897 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644053504 count 1 item 25 key (3988449337344 EXTENT_ITEM 4096) itemoff 2860 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644053504 count 1 item 26 key (3988449341440 EXTENT_ITEM 20480) itemoff 2807 itemsize 53 extent refs 1 gen 291191 flags DATA extent data backref root 33530 objectid 1277577 offset 391614464 count 1 item 27 key (3988449361920 EXTENT_ITEM 4096) itemoff 2754 itemsize 53 extent refs 1 gen 291191 flags DATA extent data backref root 33530 objectid 1277577 offset 392032256 count 1 item 28 key (3988449366016 EXTENT_ITEM 8192) itemoff 2701 itemsize 53 extent refs 1 gen 288553 flags DATA extent data backref root 33295 objectid 1921863 offset 233062400 count 1 item 29 key (3988449374208 EXTENT_ITEM 4096) itemoff 2664 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644053504 count 1 item 30 key (3988449378304 EXTENT_ITEM 4096) itemoff 2627 itemsize 37 extent refs 1 gen 154664 flags DATA shared data backref parent 4607644053504 count 1 item 31 key (3988449382400 EXTENT_ITEM 8192) itemoff 2590 itemsize 37 extent refs 1 gen 154663 flags DATA shared data backref parent 4607644053504 count 1 item 32 key (3988449390592 EXTENT_ITEM 8192) itemoff 2537 itemsize 53 extent refs 1 gen 271366 flags DATA extent data backref root 32126 objectid 1921863 offset 302276608 count 1 > > ERROR: extent[3837768077312, 24576] referencer count mismatch (root: > > 33174, > > owner: 1277577, offset: 4767744) wanted: 1, have: 0 > > [snip many more referencer count mismatches] > > ERROR: extent[3878247383040, 8192] referencer count mismatch (root: 33495, > > owner: 2688918, offset: 3874816) wanted: 2, have: 3 > > That's a little dangerous now. > > Some *DATA* extent even lost its referencer now. > > For example, data extent at 3837768077312, length 24K exists on disk, > but no one is referring to it. > > At least the good news is, these just mean there is some extra space > taken on disk, but not vice verse. > > > ERROR: block group[3879328546816 1073741824] used 1072840704 but extent > > items used 1129164800 > > ERROR: extent[3879516581888, 40960] referencer count mismatch (root: > > 33530, > > owner: 1277577, offset: 23461888) wanted: 1, have: 0 > > [snip many more referencer count mismatches] > > ERROR: extent[3951758860288, 16384] referencer count mismatch (root: > > 33189, > > owner: 1277577, offset: 519880704) wanted: 1, have: 0 > > > > This is 56 lines of output, the unabridged output contains enough > > "referencer count mismatches" to fill 1329 lines of output. Could that > > be because there are deleted snapshots that btrfs-cleaner hasn't been > > able to clean up? > Ah, this makes sense. > > But normally cleaner should cleanup subvolume tree block and modify > extent tree. > How did you stop btrfs-cleaner? See the log I posted on 28th February. The file system gets forced RO due to "Transaction aborted (error -2)" errors. Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-01 18:14 ` Marc Joliet ` (2 preceding siblings ...) 2017-03-02 9:44 ` Marc Joliet @ 2017-03-05 23:53 ` Marc Joliet 2017-03-06 11:18 ` Marc Joliet 3 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-03-05 23:53 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2015 bytes --] On Mittwoch, 1. März 2017 19:14:07 CET you wrote: > In any > case, I started btrfs-check on the device itself. *Sigh*, I had to restart it, because I forgot to redirect to a file and quite frankly wasn't expecting this flood of output, but here's a summary of the output after about 2 days: % wc -l btrfs_check_output_20170303.log 1360252 btrfs_check_output_20170303.log % grep -v "backref lost" btrfs_check_output_20170303.log | grep -v "check \ (leaf\|node\) failed" | grep -v "lost its parent" | grep -v "referencer count" checking extents ERROR: block group[3879328546816 1073741824] used 1072840704 but extent items used 1129164800 ERROR: block group[4163870130176 1073741824] used 1072259072 but extent items used 0 ERROR: block group[4223999672320 1073741824] used 1073664000 but extent items used 1074188288 ERROR: block group[4278760505344 1073741824] used 1073377280 but extent items used 1073901568 ERROR: block group[4406535782400 1073741824] used 1073627136 but extent items used 0 ERROR: extent [3830028140544 4096] referencer bytenr mismatch, wanted: 3830028140544, have: 3826183847936 % tail btrfs_check_output_20170303.log ERROR: data extent[3924538445824 4096] backref lost ERROR: data extent[3933464903680 4096] backref lost ERROR: data extent[3924538531840 4096] backref lost ERROR: data extent[3839131480064 4096] backref lost ERROR: data extent[3834701750272 4096] backref lost ERROR: data extent[3873087918080 4096] backref lost ERROR: data extent[3873072283648 4096] backref lost ERROR: data extent[3873088090112 8192] backref lost ERROR: data extent[3873072287744 4096] backref lost ERROR: data extent[3856294449152 4096] backref lost That's right, slowly approaching 1.5 million lines of btrfs-check output! That's *way* more than I ran it the last time this error happened a few weeks ago. Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-05 23:53 ` Marc Joliet @ 2017-03-06 11:18 ` Marc Joliet 0 siblings, 0 replies; 23+ messages in thread From: Marc Joliet @ 2017-03-06 11:18 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2981 bytes --] On Montag, 6. März 2017 00:53:40 CET Marc Joliet wrote: > On Mittwoch, 1. März 2017 19:14:07 CET you wrote: > > In any > > > > case, I started btrfs-check on the device itself. > > *Sigh*, I had to restart it, because I forgot to redirect to a file and > quite frankly wasn't expecting this flood of output, but here's a summary > of the output after about 2 days: > [snip old output] OK, it finished last night. Here's the summary again: % wc -l btrfs_check_output_20170303.log 3028222 btrfs_check_output_20170303.log % grep -v "backref lost" btrfs_check_output_20170303.log | grep -v "check \ (leaf\|node\) failed" | grep -v "lost its parent" | grep -v "referencer count" checking extents ERROR: block group[3879328546816 1073741824] used 1072840704 but extent items used 1129164800 ERROR: block group[4163870130176 1073741824] used 1072259072 but extent items used 0 ERROR: block group[4223999672320 1073741824] used 1073664000 but extent items used 1074188288 ERROR: block group[4278760505344 1073741824] used 1073377280 but extent items used 1073901568 ERROR: block group[4406535782400 1073741824] used 1073627136 but extent items used 0 ERROR: extent [3830028140544 4096] referencer bytenr mismatch, wanted: 3830028140544, have: 3826183847936 ERROR: errors found in extent allocation tree or chunk allocation checking free space cache checking fs roots Checking filesystem on /dev/sdb2 UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 found 892572778496 bytes used err is -5 total csum bytes: 860790216 total tree bytes: 36906336256 total fs tree bytes: 34551476224 total extent tree bytes: 1230610432 btree space waste bytes: 7446885892 file data blocks allocated: 16359581663232 referenced 2358137831424 > That's right, slowly approaching 1.5 million lines of btrfs-check output! > That's *way* more than I ran it the last time this error happened a few > weeks ago. As can be seen above, that ballooned to over 3 million lines. Since the output is 4.2 MB, even after XZ compression, I put it up on my Dropbox, just in case it's interesting to anybody: https://www.dropbox.com/s/h6ftqpygfr4vsks/btrfs_check_output_20170303.log.xz? dl=0 Since this is my backup drive, and the second time within a month that it had problems like this, *and* I've got both the btrfs-image dump and btrfs-check output, I'm going to go ahead and reformat, so that my three computers are finally backed up again. Oh, and for what it's worth, I did test against a 4.8 kernel, and pretty much immediately got the "forced RO" error, just like with 4.9. I didn't try anything older (or newer). As a last step, I'll probably collect all information I have and post it to bugzilla when I have a chance, since others might hit it, too (Kai did before me, after all). Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-02-02 12:01 ` Marc Joliet 2017-02-03 22:44 ` Kai Krakow @ 2017-03-02 0:43 ` Qu Wenruo 2017-03-02 9:43 ` Marc Joliet 1 sibling, 1 reply; 23+ messages in thread From: Qu Wenruo @ 2017-03-02 0:43 UTC (permalink / raw) To: Marc Joliet, linux-btrfs At 02/02/2017 08:01 PM, Marc Joliet wrote: > On Sunday 28 August 2016 15:29:08 Kai Krakow wrote: >> Hello list! > > Hi list [kernel message snipped] > >> Btrfs --repair refused to repair the filesystem telling me something >> about compressed extents and an unsupported case, wanting me to take an >> image and send it to the devs. *sigh* > > I haven't tried a repair yet; it's a big file system, and btrfs-check is still > running: > > # btrfs check -p /dev/sdd2 > Checking filesystem on /dev/sdd2 > UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > parent transid verify failed on 3829276291072 wanted 224274 found 283858 Normal transid error, can't say much about if it's harmless, but at least some thing went wrong. > Ignoring transid failure > leaf parent key incorrect 3829276291072 > bad block 3829276291072 That's some what a big problem for that tree block. If this tree block is extent tree block, no wonder why kernel output kernel warning and abort transaction. You could try "btrfs-debug-tree -b 3829276291072 <device>" to show the content of the tree block. If it's an extent tree block, then I'm afraid that's the problem. Not sure if repair can repair such problem, but at least from what I see in btrfs-progs fsck self testcases, it doesn't handle extent tree error well. > > ERROR: errors found in extent allocation tree or chunk allocation > block group 4722282987520 has wrong amount of free space > failed to load free space cache for block group 4722282987520 > checking free space cache [O] > root 32018 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32089 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32091 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32092 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32107 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32189 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32190 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32191 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32265 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32266 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32409 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32410 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32411 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32412 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32413 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32631 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32632 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32633 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32634 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32635 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32636 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32718 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 > root 32732 inode 95066 errors 100, file extent discount > Found file extent holes: > start: 413696, len: 4096 File extent holes are completely fine, one of the few problems we can fix safely in btrfs-check. But previous extent tree one is not. If lowmem mode also reports the same problems only (file extent discount + extent tree error), then there is a chance that --init-extent-tree may help. But it will be super time consuming though. Thanks, Qu > checking fs roots [o] > > I know that the "file extend discount" errors are fixable from my previous > email to this ML, but what about the rest? From looking through the ML > archives it seems that --repair won't be able to fix the transid failures. It > seems that one person had success with the "usebackuproot" mount option, which > I haven't tried yet. > >> System is kernel 4.7.2, Gentoo Linux, latest VirtualBox stable. >> VirtualBox was using VDI image format without nocow. I now reverted >> back to using nocow on VDI files and hope it doesn't strike again too >> soon. I didn't try again yet, first I need to refresh my backup which >> takes a while. >> >> The filesystem runs on 3x SATA 1TB mraid1 draid0 through bcache in >> writeback mode, backed by a 500GB 850 Evo - if that matters. >> >> The problem occurred during high IO on 4.7.2. I previously ran 4.6.6 >> which didn't show this problem. Part of the culprit may be that I was >> using bfq patches - I removed them for now and went back to deadline io >> scheduler. The bfq patches froze my system a few times when I booted >> 4.7.2 which may already have broken my btrfs (although it shouldn't, >> right? btrfs is transactional). Last time this happened (on an earlier >> kernel), bfq may have been part of the problem, too. So I think bfq >> does something to btrfs which may break the fs, or at least interferes >> badly with the transaction as otherwise it shouldn't break. You may >> want to run your test suites with bfq also (or different io schedulers >> in general). >> >> My home partition is mounted as a subvolume: >> /dev/bcache0 on /home type btrfs >> (rw,noatime,compress=lzo,nossd,space_cache,autodefrag,subvolid=261,subvol=/ >> home) > > The system the drive runs on is: > > % uname -a > Linux diefledermaus 4.9.7-gentoo #1 SMP Wed Feb 1 23:52:56 CET 2017 x86_64 > Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux > > However during the crash it was running 4.9.6-gentoo. The system uses the > standard CFQ scheduler, so perhaps BFQ is not at fault in Kai's case. > > The system I am running btrfs-check on is: > > % uname -a > Linux thetick 4.9.6-gentoo #1 SMP PREEMPT Fri Jan 27 00:50:02 CET 2017 x86_64 > AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux > > Both have btrfs-progs 4.9: > > % /sbin/btrfs --version > btrfs-progs v4.9 > > And the file system in question: > > % sudo /sbin/btrfs fi show /dev/sdd2 > Label: 'MARCEC_BACKUP' uuid: f97b3cda-15e8-418b-bb9b-235391ef2a38 > Total devices 1 FS bytes used 842.50GiB > devid 1 size 976.56GiB used 877.31GiB path /dev/sdd2 > > The file system is mounted with "noatime,compress,comment=systemd.automount". > > In my case the crash also happened during high I/O load (three btrfs- > send/receive backups running at the same time). If "usebackuproot" (now > called "recovery"?) fails, then I'll just wipe the FS and start the backups > from scratch. > > Since I would like to have that done by Saturday: is there any information I > can provide that might help fix whatever bug(s) caused this? Should I file a > bug if one doesn't exist yet (I haven't checked yet, sorry)? > > Greetings > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-02 0:43 ` Qu Wenruo @ 2017-03-02 9:43 ` Marc Joliet 2017-03-03 1:00 ` Qu Wenruo 0 siblings, 1 reply; 23+ messages in thread From: Marc Joliet @ 2017-03-02 9:43 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 13045 bytes --] On Thursday 02 March 2017 08:43:53 Qu Wenruo wrote: > At 02/02/2017 08:01 PM, Marc Joliet wrote: > > On Sunday 28 August 2016 15:29:08 Kai Krakow wrote: > >> Hello list! > > > > Hi list > > [kernel message snipped] > > >> Btrfs --repair refused to repair the filesystem telling me something > >> about compressed extents and an unsupported case, wanting me to take an > >> image and send it to the devs. *sigh* > > > > I haven't tried a repair yet; it's a big file system, and btrfs-check is > > still running: > > > > # btrfs check -p /dev/sdd2 > > Checking filesystem on /dev/sdd2 > > UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 > > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > > parent transid verify failed on 3829276291072 wanted 224274 found 283858 > > Normal transid error, can't say much about if it's harmless, but at > least some thing went wrong. > > > Ignoring transid failure > > leaf parent key incorrect 3829276291072 > > bad block 3829276291072 > > That's some what a big problem for that tree block. > > If this tree block is extent tree block, no wonder why kernel output > kernel warning and abort transaction. > > You could try "btrfs-debug-tree -b 3829276291072 <device>" to show the > content of the tree block. # btrfs-debug-tree -b 3829276291072 /dev/sdb2 btrfs-progs v4.9 node 3829276291072 level 1 items 70 free 51 generation 292525 owner 2 fs uuid f97b3cda-15e8-418b-bb9b-235391ef2a38 chunk uuid 1cee580c-3442-4717-9300-8514dd8ff297 key (3828594696192 METADATA_ITEM 0) block 3828933423104 (934798199) gen 292523 key (3828594925568 METADATA_ITEM 0) block 3829427818496 (934918901) gen 292525 key (3828595109888 METADATA_ITEM 0) block 3828895723520 (934788995) gen 292523 key (3828595232768 METADATA_ITEM 0) block 3829202751488 (934863953) gen 292524 key (3828595412992 METADATA_ITEM 0) block 3829097209856 (934838186) gen 292523 key (3828595572736 TREE_BLOCK_REF 33178) block 3829235073024 (934871844) gen 292524 key (3828595744768 METADATA_ITEM 0) block 3829128351744 (934845789) gen 292524 key (3828595982336 METADATA_ITEM 0) block 3829146484736 (934850216) gen 292524 key (3828596187136 METADATA_ITEM 1) block 3829097234432 (934838192) gen 292523 key (3828596387840 TREE_BLOCK_REF 33527) block 3829301653504 (934888099) gen 292525 key (3828596617216 METADATA_ITEM 0) block 3828885737472 (934786557) gen 292523 key (3828596838400 METADATA_ITEM 0) block 3828885741568 (934786558) gen 292523 key (3828597047296 METADATA_ITEM 0) block 3829320552448 (934892713) gen 292525 key (3828597231616 METADATA_ITEM 0) block 3828945653760 (934801185) gen 292523 key (3828597383168 METADATA_ITEM 0) block 3829276299264 (934881909) gen 292525 key (3828597641216 METADATA_ITEM 1) block 3829349351424 (934899744) gen 292525 key (3828597866496 METADATA_ITEM 0) block 3829364776960 (934903510) gen 292525 key (3828598067200 METADATA_ITEM 0) block 3828598321152 (934716387) gen 292522 key (3828598259712 METADATA_ITEM 0) block 3829422968832 (934917717) gen 292525 key (3828598415360 TREE_BLOCK_REF 33252) block 3828885803008 (934786573) gen 292523 key (3828598665216 METADATA_ITEM 0) block 3828937863168 (934799283) gen 292523 key (3828598829056 METADATA_ITEM 0) block 3828885811200 (934786575) gen 292523 key (3828599054336 METADATA_ITEM 0) block 3829363744768 (934903258) gen 292525 key (3828599246848 METADATA_ITEM 0) block 3828915838976 (934793906) gen 292523 key (3828599504896 METADATA_ITEM 0) block 3829436194816 (934920946) gen 292525 key (3828599672832 METADATA_ITEM 0) block 3828905140224 (934791294) gen 292523 key (3828599771136 METADATA_ITEM 0) block 3829333323776 (934895831) gen 292525 key (3828599988224 METADATA_ITEM 0) block 3829087199232 (934835742) gen 292523 key (3828600135680 METADATA_ITEM 0) block 3828885827584 (934786579) gen 292523 key (3828600389632 METADATA_ITEM 0) block 3829436284928 (934920968) gen 292525 key (3828600528896 METADATA_ITEM 0) block 3829316214784 (934891654) gen 292525 key (3828600729600 METADATA_ITEM 0) block 3828885905408 (934786598) gen 292523 key (3828600934400 METADATA_ITEM 0) block 3829384486912 (934908322) gen 292525 key (3828601143296 METADATA_ITEM 0) block 3829423611904 (934917874) gen 292525 key (3828601356288 METADATA_ITEM 0) block 3829113688064 (934842209) gen 292524 key (3828601556992 METADATA_ITEM 0) block 3829134540800 (934847300) gen 292524 key (3828601696256 METADATA_ITEM 0) block 3829181837312 (934858847) gen 292524 key (3828601823232 METADATA_ITEM 0) block 3829157421056 (934852886) gen 292524 key (3828602015744 TREE_BLOCK_REF 32943) block 3829316218880 (934891655) gen 292525 key (3828602245120 METADATA_ITEM 0) block 3829207601152 (934865137) gen 292524 key (3828602384384 METADATA_ITEM 0) block 3828933246976 (934798156) gen 292523 key (3828602531840 METADATA_ITEM 0) block 3829346013184 (934898929) gen 292525 key (3828602740736 METADATA_ITEM 0) block 3828916629504 (934794099) gen 292523 key (3828602990592 METADATA_ITEM 0) block 3829160525824 (934853644) gen 292524 key (3828603097088 METADATA_ITEM 0) block 3828933267456 (934798161) gen 292523 key (3828603195392 METADATA_ITEM 0) block 4632523673600 (1130987225) gen 292474 key (3828603293696 TREE_BLOCK_REF 32827) block 3829160529920 (934853645) gen 292524 key (3828603453440 METADATA_ITEM 0) block 3828886011904 (934786624) gen 292523 key (3828603711488 METADATA_ITEM 0) block 3829299806208 (934887648) gen 292525 key (3828603940864 METADATA_ITEM 0) block 3829325082624 (934893819) gen 292525 key (3828604145664 METADATA_ITEM 0) block 3828886040576 (934786631) gen 292523 key (3828604321792 METADATA_ITEM 0) block 3829207818240 (934865190) gen 292524 key (3828604538880 METADATA_ITEM 0) block 3829347282944 (934899239) gen 292525 key (3828604796928 METADATA_ITEM 1) block 3829304320000 (934888750) gen 292525 key (3828605042688 METADATA_ITEM 0) block 3829159145472 (934853307) gen 292524 key (3828605239296 METADATA_ITEM 0) block 3829170442240 (934856065) gen 292524 key (3828605505536 METADATA_ITEM 0) block 3829360103424 (934902369) gen 292525 key (3828605698048 METADATA_ITEM 0) block 3829362032640 (934902840) gen 292525 key (3828605915136 METADATA_ITEM 0) block 3829336616960 (934896635) gen 292525 key (3828606103552 METADATA_ITEM 0) block 3829360107520 (934902370) gen 292525 key (3828606291968 METADATA_ITEM 0) block 3829210537984 (934865854) gen 292524 key (3828606504960 METADATA_ITEM 0) block 3829406662656 (934913736) gen 292525 key (3828606754816 METADATA_ITEM 0) block 3829345562624 (934898819) gen 292525 key (3828606943232 METADATA_ITEM 0) block 3829342674944 (934898114) gen 292525 key (3828607156224 METADATA_ITEM 0) block 3829444628480 (934923005) gen 292525 key (3828607324160 METADATA_ITEM 0) block 3829087207424 (934835744) gen 292523 key (3828607561728 METADATA_ITEM 0) block 3829313220608 (934890923) gen 292525 key (3828607680512 METADATA_ITEM 0) block 3829370867712 (934904997) gen 292525 key (3828607930368 METADATA_ITEM 0) block 3829411094528 (934914818) gen 292525 key (3828608180224 METADATA_ITEM 0) block 3829299822592 (934887652) gen 292525 I wonder, though, whether that is still the same block or whether it has been replaced during the past two weeks of FS usage. > If it's an extent tree block, then I'm afraid that's the problem. > Not sure if repair can repair such problem, but at least from what I see > in btrfs-progs fsck self testcases, it doesn't handle extent tree error > well. OK > > ERROR: errors found in extent allocation tree or chunk allocation > > block group 4722282987520 has wrong amount of free space > > failed to load free space cache for block group 4722282987520 > > checking free space cache [O] > > root 32018 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32089 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32091 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32092 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32107 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32189 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32190 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32191 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32265 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32266 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32409 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32410 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32411 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32412 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32413 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32631 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32632 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32633 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32634 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32635 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32636 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32718 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > > > root 32732 inode 95066 errors 100, file extent discount > > > > Found file extent holes: > > start: 413696, len: 4096 > > File extent holes are completely fine, one of the few problems we can > fix safely in btrfs-check. Yeah, I found that out myself by searching my btrfs ML archive. > But previous extent tree one is not. OK > If lowmem mode also reports the same problems only (file extent discount > + extent tree error), then there is a chance that --init-extent-tree may > help. > > But it will be super time consuming though. Alright, in that case, I'll just reformat the partition and start my backups over from scratch once btrfs-check is done running (I'll post the current output in another Email). FWIW, as per my later messages, after mounting with clear_cache and letting btrfs-cleaner finish, btrfs-check did *not* print out those errors after running again. It's now about two weeks later that the file system is showing problems again. Oh, and just in case it's relevant, the file system was created with btrfs- convert (a long time, maybe 1.5 years ago, though; it was originally ext4). [...] Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-02 9:43 ` Marc Joliet @ 2017-03-03 1:00 ` Qu Wenruo 2017-03-03 11:54 ` Marc Joliet 0 siblings, 1 reply; 23+ messages in thread From: Qu Wenruo @ 2017-03-03 1:00 UTC (permalink / raw) To: Marc Joliet, linux-btrfs At 03/02/2017 05:43 PM, Marc Joliet wrote: > On Thursday 02 March 2017 08:43:53 Qu Wenruo wrote: >> At 02/02/2017 08:01 PM, Marc Joliet wrote: >>> On Sunday 28 August 2016 15:29:08 Kai Krakow wrote: >>>> Hello list! >>> >>> Hi list >> >> [kernel message snipped] >> >>>> Btrfs --repair refused to repair the filesystem telling me something >>>> about compressed extents and an unsupported case, wanting me to take an >>>> image and send it to the devs. *sigh* >>> >>> I haven't tried a repair yet; it's a big file system, and btrfs-check is >>> still running: >>> >>> # btrfs check -p /dev/sdd2 >>> Checking filesystem on /dev/sdd2 >>> UUID: f97b3cda-15e8-418b-bb9b-235391ef2a38 >>> parent transid verify failed on 3829276291072 wanted 224274 found 283858 >>> parent transid verify failed on 3829276291072 wanted 224274 found 283858 >>> parent transid verify failed on 3829276291072 wanted 224274 found 283858 >>> parent transid verify failed on 3829276291072 wanted 224274 found 283858 >> >> Normal transid error, can't say much about if it's harmless, but at >> least some thing went wrong. >> >>> Ignoring transid failure >>> leaf parent key incorrect 3829276291072 >>> bad block 3829276291072 >> >> That's some what a big problem for that tree block. >> >> If this tree block is extent tree block, no wonder why kernel output >> kernel warning and abort transaction. >> >> You could try "btrfs-debug-tree -b 3829276291072 <device>" to show the >> content of the tree block. > > # btrfs-debug-tree -b 3829276291072 /dev/sdb2 > btrfs-progs v4.9 > node 3829276291072 level 1 items 70 free 51 generation 292525 owner 2 > fs uuid f97b3cda-15e8-418b-bb9b-235391ef2a38 > chunk uuid 1cee580c-3442-4717-9300-8514dd8ff297 > key (3828594696192 METADATA_ITEM 0) block 3828933423104 (934798199) > gen 292523 > key (3828594925568 METADATA_ITEM 0) block 3829427818496 (934918901) > gen 292525 > key (3828595109888 METADATA_ITEM 0) block 3828895723520 (934788995) > gen 292523 > key (3828595232768 METADATA_ITEM 0) block 3829202751488 (934863953) > gen 292524 > key (3828595412992 METADATA_ITEM 0) block 3829097209856 (934838186) > gen 292523 > key (3828595572736 TREE_BLOCK_REF 33178) block 3829235073024 > (934871844) gen 292524 > key (3828595744768 METADATA_ITEM 0) block 3829128351744 (934845789) > gen 292524 > key (3828595982336 METADATA_ITEM 0) block 3829146484736 (934850216) > gen 292524 > key (3828596187136 METADATA_ITEM 1) block 3829097234432 (934838192) > gen 292523 > key (3828596387840 TREE_BLOCK_REF 33527) block 3829301653504 > (934888099) gen 292525 > key (3828596617216 METADATA_ITEM 0) block 3828885737472 (934786557) > gen 292523 > key (3828596838400 METADATA_ITEM 0) block 3828885741568 (934786558) > gen 292523 > key (3828597047296 METADATA_ITEM 0) block 3829320552448 (934892713) > gen 292525 > key (3828597231616 METADATA_ITEM 0) block 3828945653760 (934801185) > gen 292523 > key (3828597383168 METADATA_ITEM 0) block 3829276299264 (934881909) > gen 292525 > key (3828597641216 METADATA_ITEM 1) block 3829349351424 (934899744) > gen 292525 > key (3828597866496 METADATA_ITEM 0) block 3829364776960 (934903510) > gen 292525 > key (3828598067200 METADATA_ITEM 0) block 3828598321152 (934716387) > gen 292522 > key (3828598259712 METADATA_ITEM 0) block 3829422968832 (934917717) > gen 292525 > key (3828598415360 TREE_BLOCK_REF 33252) block 3828885803008 > (934786573) gen 292523 > key (3828598665216 METADATA_ITEM 0) block 3828937863168 (934799283) > gen 292523 > key (3828598829056 METADATA_ITEM 0) block 3828885811200 (934786575) > gen 292523 > key (3828599054336 METADATA_ITEM 0) block 3829363744768 (934903258) > gen 292525 > key (3828599246848 METADATA_ITEM 0) block 3828915838976 (934793906) > gen 292523 > key (3828599504896 METADATA_ITEM 0) block 3829436194816 (934920946) > gen 292525 > key (3828599672832 METADATA_ITEM 0) block 3828905140224 (934791294) > gen 292523 > key (3828599771136 METADATA_ITEM 0) block 3829333323776 (934895831) > gen 292525 > key (3828599988224 METADATA_ITEM 0) block 3829087199232 (934835742) > gen 292523 > key (3828600135680 METADATA_ITEM 0) block 3828885827584 (934786579) > gen 292523 > key (3828600389632 METADATA_ITEM 0) block 3829436284928 (934920968) > gen 292525 > key (3828600528896 METADATA_ITEM 0) block 3829316214784 (934891654) > gen 292525 > key (3828600729600 METADATA_ITEM 0) block 3828885905408 (934786598) > gen 292523 > key (3828600934400 METADATA_ITEM 0) block 3829384486912 (934908322) > gen 292525 > key (3828601143296 METADATA_ITEM 0) block 3829423611904 (934917874) > gen 292525 > key (3828601356288 METADATA_ITEM 0) block 3829113688064 (934842209) > gen 292524 > key (3828601556992 METADATA_ITEM 0) block 3829134540800 (934847300) > gen 292524 > key (3828601696256 METADATA_ITEM 0) block 3829181837312 (934858847) > gen 292524 > key (3828601823232 METADATA_ITEM 0) block 3829157421056 (934852886) > gen 292524 > key (3828602015744 TREE_BLOCK_REF 32943) block 3829316218880 > (934891655) gen 292525 > key (3828602245120 METADATA_ITEM 0) block 3829207601152 (934865137) > gen 292524 > key (3828602384384 METADATA_ITEM 0) block 3828933246976 (934798156) > gen 292523 > key (3828602531840 METADATA_ITEM 0) block 3829346013184 (934898929) > gen 292525 > key (3828602740736 METADATA_ITEM 0) block 3828916629504 (934794099) > gen 292523 > key (3828602990592 METADATA_ITEM 0) block 3829160525824 (934853644) > gen 292524 > key (3828603097088 METADATA_ITEM 0) block 3828933267456 (934798161) > gen 292523 > key (3828603195392 METADATA_ITEM 0) block 4632523673600 (1130987225) > gen 292474 > key (3828603293696 TREE_BLOCK_REF 32827) block 3829160529920 > (934853645) gen 292524 > key (3828603453440 METADATA_ITEM 0) block 3828886011904 (934786624) > gen 292523 > key (3828603711488 METADATA_ITEM 0) block 3829299806208 (934887648) > gen 292525 > key (3828603940864 METADATA_ITEM 0) block 3829325082624 (934893819) > gen 292525 > key (3828604145664 METADATA_ITEM 0) block 3828886040576 (934786631) > gen 292523 > key (3828604321792 METADATA_ITEM 0) block 3829207818240 (934865190) > gen 292524 > key (3828604538880 METADATA_ITEM 0) block 3829347282944 (934899239) > gen 292525 > key (3828604796928 METADATA_ITEM 1) block 3829304320000 (934888750) > gen 292525 > key (3828605042688 METADATA_ITEM 0) block 3829159145472 (934853307) > gen 292524 > key (3828605239296 METADATA_ITEM 0) block 3829170442240 (934856065) > gen 292524 > key (3828605505536 METADATA_ITEM 0) block 3829360103424 (934902369) > gen 292525 > key (3828605698048 METADATA_ITEM 0) block 3829362032640 (934902840) > gen 292525 > key (3828605915136 METADATA_ITEM 0) block 3829336616960 (934896635) > gen 292525 > key (3828606103552 METADATA_ITEM 0) block 3829360107520 (934902370) > gen 292525 > key (3828606291968 METADATA_ITEM 0) block 3829210537984 (934865854) > gen 292524 > key (3828606504960 METADATA_ITEM 0) block 3829406662656 (934913736) > gen 292525 > key (3828606754816 METADATA_ITEM 0) block 3829345562624 (934898819) > gen 292525 > key (3828606943232 METADATA_ITEM 0) block 3829342674944 (934898114) > gen 292525 > key (3828607156224 METADATA_ITEM 0) block 3829444628480 (934923005) > gen 292525 > key (3828607324160 METADATA_ITEM 0) block 3829087207424 (934835744) > gen 292523 > key (3828607561728 METADATA_ITEM 0) block 3829313220608 (934890923) > gen 292525 > key (3828607680512 METADATA_ITEM 0) block 3829370867712 (934904997) > gen 292525 > key (3828607930368 METADATA_ITEM 0) block 3829411094528 (934914818) > gen 292525 > key (3828608180224 METADATA_ITEM 0) block 3829299822592 (934887652) > gen 292525 > > I wonder, though, whether that is still the same block or whether it has been > replaced during the past two weeks of FS usage. It's a tree block of extent tree. So the problem is all about extent tree now. > >> If it's an extent tree block, then I'm afraid that's the problem. >> Not sure if repair can repair such problem, but at least from what I see >> in btrfs-progs fsck self testcases, it doesn't handle extent tree error >> well. > > OK > >>> ERROR: errors found in extent allocation tree or chunk allocation >>> block group 4722282987520 has wrong amount of free space >>> failed to load free space cache for block group 4722282987520 >>> checking free space cache [O] >>> root 32018 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32089 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32091 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32092 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32107 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32189 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32190 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32191 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32265 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32266 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32409 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32410 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32411 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32412 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32413 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32631 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32632 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32633 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32634 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32635 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32636 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32718 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >>> >>> root 32732 inode 95066 errors 100, file extent discount >>> >>> Found file extent holes: >>> start: 413696, len: 4096 >> >> File extent holes are completely fine, one of the few problems we can >> fix safely in btrfs-check. > > Yeah, I found that out myself by searching my btrfs ML archive. > >> But previous extent tree one is not. > > OK > >> If lowmem mode also reports the same problems only (file extent discount >> + extent tree error), then there is a chance that --init-extent-tree may >> help. >> >> But it will be super time consuming though. > > Alright, in that case, I'll just reformat the partition and start my backups > over from scratch once btrfs-check is done running (I'll post the current > output in another Email). > > FWIW, as per my later messages, after mounting with clear_cache and letting > btrfs-cleaner finish, btrfs-check did *not* print out those errors after > running again. It's now about two weeks later that the file system is showing > problems again. If btrfs-check didn't print out *any* error, then it should be mostly fine. (Unless there is some case we don't expose yet) The problem should be caused by kernel AFAIK. > > Oh, and just in case it's relevant, the file system was created with btrfs- > convert (a long time, maybe 1.5 years ago, though; it was originally ext4). Not sure if it's related. But at least for that old convert, it's chunk layout is somewhat rare and sometimes even bug-prone. Did you balance the btrfs after convert? If so, it should be more like a traditional btrfs then. Personally speaking I don't think it is relative for your bug, but much like a normal extent tree corruption seen in mail list. Thanks, Qu > > [...] > > Greetings > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists 2017-03-03 1:00 ` Qu Wenruo @ 2017-03-03 11:54 ` Marc Joliet 0 siblings, 0 replies; 23+ messages in thread From: Marc Joliet @ 2017-03-03 11:54 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1451 bytes --] On Friday 03 March 2017 09:00:10 Qu Wenruo wrote: > > FWIW, as per my later messages, after mounting with clear_cache and > > letting > > btrfs-cleaner finish, btrfs-check did *not* print out those errors after > > running again. It's now about two weeks later that the file system is > > showing problems again. > > If btrfs-check didn't print out *any* error, then it should be mostly > fine. (Unless there is some case we don't expose yet) > > The problem should be caused by kernel AFAIK. So you think it could be a regression in 4.9? Should I try 4.10? Or is it more likely just an undiscovered bug? > > Oh, and just in case it's relevant, the file system was created with > > btrfs- > > convert (a long time, maybe 1.5 years ago, though; it was originally > > ext4). > > Not sure if it's related. > But at least for that old convert, it's chunk layout is somewhat rare > and sometimes even bug-prone. > > Did you balance the btrfs after convert? If so, it should be more like a > traditional btrfs then. Yes, I'm fairly certain I did that, as that is what the btrfs wiki recommends. > Personally speaking I don't think it is relative for your bug, but much > like a normal extent tree corruption seen in mail list. OK, so is there anything else I can do? Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2017-03-06 11:20 UTC | newest] Thread overview: 23+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-08-28 13:29 [4.7.2] btrfs_run_delayed_refs:2963: errno=-17 Object already exists Kai Krakow 2017-02-02 12:01 ` Marc Joliet 2017-02-03 22:44 ` Kai Krakow 2017-02-10 22:15 ` Marc Joliet 2017-02-11 2:01 ` Kai Krakow 2017-02-11 10:18 ` Marc Joliet 2017-02-14 12:52 ` Marc Joliet 2017-02-17 8:19 ` Kai Krakow 2017-02-28 22:14 ` Marc Joliet 2017-03-01 8:23 ` Marc Joliet 2017-03-01 9:32 ` Qu Wenruo 2017-03-01 18:14 ` Marc Joliet 2017-03-01 18:27 ` Marc Joliet 2017-03-01 18:43 ` Marc Joliet 2017-03-02 9:44 ` Marc Joliet 2017-03-03 1:09 ` Qu Wenruo 2017-03-03 11:26 ` Marc Joliet 2017-03-05 23:53 ` Marc Joliet 2017-03-06 11:18 ` Marc Joliet 2017-03-02 0:43 ` Qu Wenruo 2017-03-02 9:43 ` Marc Joliet 2017-03-03 1:00 ` Qu Wenruo 2017-03-03 11:54 ` Marc Joliet
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).