* btrfs partition fails to mount after power outage
@ 2016-08-23 12:30 Malte Westerhoff
2016-08-23 14:52 ` Chris Murphy
2016-08-23 16:06 ` Roman Mamedov
0 siblings, 2 replies; 9+ messages in thread
From: Malte Westerhoff @ 2016-08-23 12:30 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
[-- Attachment #1: Type: text/plain, Size: 2978 bytes --]
Hi,
I hope this is the right list for this problem report.
After an unclean shutdown of a server due to power outage, one of the partitions fails to mount.
mount -o recovery ...,mount -o recovery,ro ..., mount -o ro do not help. They produce this error in the log:
[ 5464.525816] BTRFS info (device dm-0): enabling auto recovery
[ 5464.525823] BTRFS info (device dm-0): disk space caching is enabled
[ 5464.525825] BTRFS: has skinny extents
[ 5499.299686] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
[ 5499.308576] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
[ 5499.308587] BTRFS: Failed to read block groups: -5
[ 5499.391359] BTRFS: open_ctree failed
btrfs check --repair fails with an error message:
... (full output attached)
Ignoring transid failure
parent transid verify failed on 7375567323136 wanted 52059 found 52045
parent transid verify failed on 7375567323136 wanted 52059 found 52045
Error: could not find btree root extent for root 12974
This is kernel 4.1.27 and btrfs-progs 4.1.2. We also tried the btrfs check --repair with btrfs-progs 4.7 with the same result.
The partition with the problem is /dev/mapper/VGBIGRAID6-DATA1
(The partition is an LVM logical volume that is on top of an md raid6.)
Anything else that we can try in order to recover the volume?
Thanks
Malte
quickstore2:~/btrfs-progs # uname -a
Linux quickstore2 4.1.27-27-default #1 SMP PREEMPT Fri Jul 15 12:46:41 UTC 2016 (84ae57e) x86_64 x86_64 x86_64 GNU/Linux
quickstore2:~/btrfs-progs # btrfs --version
btrfs-progs v4.1.2+20151002
quickstore2:~/btrfs-progs # btrfs fi show
Label: none uuid: ef3fc810-d74e-4c8e-97e3-7c7e788795ae
Total devices 1 FS bytes used 12.14GiB
devid 1 size 80.00GiB used 15.04GiB path /dev/sda1
Label: none uuid: 9b4af403-e40c-4228-825a-20666aa0ec3c
Total devices 1 FS bytes used 581.48GiB
devid 1 size 8.00TiB used 616.03GiB path /dev/mapper/VGBIGRAID6-DATA2
Label: none uuid: 98330879-6f67-44f0-92ba-974c86a836d1
Total devices 1 FS bytes used 1015.34GiB
devid 1 size 8.00TiB used 1.08TiB path /dev/mapper/VGBIGRAID6-data3
Label: none uuid: 014d9f5c-48b2-4043-add3-5150ce418570
Total devices 1 FS bytes used 3.15TiB
devid 1 size 16.00TiB used 3.21TiB path /dev/mapper/VGBIGRAID6-data4
Label: none uuid: 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
Total devices 1 FS bytes used 6.72TiB
devid 1 size 16.00TiB used 6.97TiB path /dev/mapper/VGBIGRAID6-DATA1
btrfs-progs v4.1.2+20151002
quickstore2:~/btrfs-progs # btrfs fi show /dev/mapper/VGBIGRAID6-DATA1
Label: none uuid: 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
Total devices 1 FS bytes used 6.72TiB
devid 1 size 16.00TiB used 6.97TiB path /dev/mapper/VGBIGRAID6-DATA1
btrfs-progs v4.1.2+20151002
quickstore2:~/btrfs-progs #
[-- Attachment #2: btrfs-check.txt --]
[-- Type: text/plain, Size: 26360 bytes --]
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
Ignoring transid failure
leaf parent key incorrect 7375769206784
checking extents
parent transid verify failed on 7375715827712 wanted 52059 found 52045
parent transid verify failed on 7375715827712 wanted 52059 found 52045
parent transid verify failed on 7375715827712 wanted 52059 found 52045
parent transid verify failed on 7375715827712 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375712600064 wanted 52059 found 52045
parent transid verify failed on 7375712600064 wanted 52059 found 52045
parent transid verify failed on 7375716237312 wanted 52059 found 52045
parent transid verify failed on 7375716237312 wanted 52059 found 52045
parent transid verify failed on 7375716237312 wanted 52059 found 52045
parent transid verify failed on 7375716237312 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375712763904 wanted 52059 found 52045
parent transid verify failed on 7375712763904 wanted 52059 found 52045
parent transid verify failed on 7375712600064 wanted 52059 found 52045
parent transid verify failed on 7375712600064 wanted 52059 found 52045
parent transid verify failed on 7375712763904 wanted 52059 found 52045
parent transid verify failed on 7375712763904 wanted 52059 found 52045
parent transid verify failed on 7375715827712 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375716237312 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717466112 wanted 52059 found 52045
parent transid verify failed on 7375717466112 wanted 52059 found 52045
parent transid verify failed on 7375717466112 wanted 52059 found 52045
parent transid verify failed on 7375717466112 wanted 52059 found 52045
Ignoring transid failure
leaf parent key incorrect 7375717466112
parent transid verify failed on 7375773761536 wanted 52059 found 52028
parent transid verify failed on 7375773761536 wanted 52059 found 52028
parent transid verify failed on 7375773761536 wanted 52059 found 52028
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
leaf parent key incorrect 7375773761536
parent transid verify failed on 7375562375168 wanted 52059 found 52045
parent transid verify failed on 7375562375168 wanted 52059 found 52045
parent transid verify failed on 7375564832768 wanted 52059 found 52045
parent transid verify failed on 7375564832768 wanted 52059 found 52045
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567323136 wanted 52059 found 52045
parent transid verify failed on 7375567323136 wanted 52059 found 52045
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375717466112 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
leaf parent key incorrect 7375717466112
bad block 7375717466112
Errors found in extent allocation tree or chunk allocation
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375567568896 wanted 52059 found 52028
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375717089280 wanted 52059 found 52045
parent transid verify failed on 7375764078592 wanted 52059 found 52028
parent transid verify failed on 7375764078592 wanted 52059 found 52028
parent transid verify failed on 7375764094976 wanted 52059 found 52028
parent transid verify failed on 7375764094976 wanted 52059 found 52028
parent transid verify failed on 7375773761536 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375780462592 wanted 52059 found 52028
parent transid verify failed on 7375780462592 wanted 52059 found 52028
parent transid verify failed on 7375780462592 wanted 52059 found 52028
parent transid verify failed on 7375780462592 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375796436992 wanted 52059 found 52028
parent transid verify failed on 7375796436992 wanted 52059 found 52028
parent transid verify failed on 7375796436992 wanted 52059 found 52028
parent transid verify failed on 7375796436992 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375798697984 wanted 52059 found 52028
parent transid verify failed on 7375798697984 wanted 52059 found 52028
parent transid verify failed on 7375798697984 wanted 52059 found 52028
parent transid verify failed on 7375798697984 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375803990016 wanted 52059 found 52028
parent transid verify failed on 7375803990016 wanted 52059 found 52028
parent transid verify failed on 7375803990016 wanted 52059 found 52028
parent transid verify failed on 7375803990016 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375810117632 wanted 52059 found 52028
parent transid verify failed on 7375810117632 wanted 52059 found 52028
parent transid verify failed on 7375810117632 wanted 52059 found 52028
parent transid verify failed on 7375810117632 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375810232320 wanted 52059 found 52028
parent transid verify failed on 7375810232320 wanted 52059 found 52028
parent transid verify failed on 7375810232320 wanted 52059 found 52028
parent transid verify failed on 7375810232320 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375821520896 wanted 52059 found 52028
parent transid verify failed on 7375821520896 wanted 52059 found 52028
parent transid verify failed on 7375821520896 wanted 52059 found 52028
parent transid verify failed on 7375821520896 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375832285184 wanted 52059 found 52028
parent transid verify failed on 7375832285184 wanted 52059 found 52028
parent transid verify failed on 7375832285184 wanted 52059 found 52028
parent transid verify failed on 7375832285184 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375845064704 wanted 52059 found 52028
parent transid verify failed on 7375845064704 wanted 52059 found 52028
parent transid verify failed on 7375845064704 wanted 52059 found 52028
parent transid verify failed on 7375845064704 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375846965248 wanted 52059 found 52028
parent transid verify failed on 7375846965248 wanted 52059 found 52028
parent transid verify failed on 7375846965248 wanted 52059 found 52028
parent transid verify failed on 7375846965248 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375846981632 wanted 52059 found 52028
parent transid verify failed on 7375846981632 wanted 52059 found 52028
parent transid verify failed on 7375846981632 wanted 52059 found 52028
parent transid verify failed on 7375846981632 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375856844800 wanted 52059 found 52028
parent transid verify failed on 7375856844800 wanted 52059 found 52028
parent transid verify failed on 7375856844800 wanted 52059 found 52028
parent transid verify failed on 7375856844800 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375881895936 wanted 52059 found 52028
parent transid verify failed on 7375881895936 wanted 52059 found 52028
parent transid verify failed on 7375881895936 wanted 52059 found 52028
parent transid verify failed on 7375881895936 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375890317312 wanted 52059 found 52028
parent transid verify failed on 7375890317312 wanted 52059 found 52028
parent transid verify failed on 7375890317312 wanted 52059 found 52028
parent transid verify failed on 7375890317312 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375890333696 wanted 52059 found 51640
parent transid verify failed on 7375890333696 wanted 52059 found 51640
parent transid verify failed on 7375890333696 wanted 52059 found 51640
parent transid verify failed on 7375890333696 wanted 52059 found 51640
Ignoring transid failure
parent transid verify failed on 7375918333952 wanted 52059 found 52045
parent transid verify failed on 7375918333952 wanted 52059 found 52045
parent transid verify failed on 7375918333952 wanted 52059 found 52045
parent transid verify failed on 7375918333952 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375928475648 wanted 52059 found 52045
parent transid verify failed on 7375928475648 wanted 52059 found 52045
parent transid verify failed on 7375928475648 wanted 52059 found 52045
parent transid verify failed on 7375928475648 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375937912832 wanted 52059 found 52028
parent transid verify failed on 7375937912832 wanted 52059 found 52028
parent transid verify failed on 7375937912832 wanted 52059 found 52028
parent transid verify failed on 7375937912832 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375937929216 wanted 52059 found 52028
parent transid verify failed on 7375937929216 wanted 52059 found 52028
parent transid verify failed on 7375937929216 wanted 52059 found 52028
parent transid verify failed on 7375937929216 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375890350080 wanted 52059 found 52045
parent transid verify failed on 7375890350080 wanted 52059 found 52045
parent transid verify failed on 7375890350080 wanted 52059 found 52045
parent transid verify failed on 7375890350080 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375717466112 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375815049216 wanted 52059 found 52028
parent transid verify failed on 7375815049216 wanted 52059 found 52028
parent transid verify failed on 7375815049216 wanted 52059 found 52028
parent transid verify failed on 7375815049216 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375815344128 wanted 52059 found 52028
parent transid verify failed on 7375815344128 wanted 52059 found 52028
parent transid verify failed on 7375815344128 wanted 52059 found 52028
parent transid verify failed on 7375815344128 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375562375168 wanted 52059 found 52045
parent transid verify failed on 7375562375168 wanted 52059 found 52045
parent transid verify failed on 7375832842240 wanted 52059 found 52028
parent transid verify failed on 7375832842240 wanted 52059 found 52028
parent transid verify failed on 7375832842240 wanted 52059 found 52028
parent transid verify failed on 7375832842240 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7376078815232 wanted 52059 found 52028
parent transid verify failed on 7376078815232 wanted 52059 found 52028
parent transid verify failed on 7376078815232 wanted 52059 found 52028
parent transid verify failed on 7376078815232 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375848259584 wanted 52059 found 52028
parent transid verify failed on 7375848259584 wanted 52059 found 52028
parent transid verify failed on 7375848259584 wanted 52059 found 52028
parent transid verify failed on 7375848259584 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375564832768 wanted 52059 found 52045
parent transid verify failed on 7375564832768 wanted 52059 found 52045
parent transid verify failed on 7375769255936 wanted 52059 found 52028
parent transid verify failed on 7375769255936 wanted 52059 found 52028
parent transid verify failed on 7375769255936 wanted 52059 found 52028
parent transid verify failed on 7375769255936 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375833595904 wanted 52059 found 52028
parent transid verify failed on 7375833595904 wanted 52059 found 52028
parent transid verify failed on 7375833595904 wanted 52059 found 52028
parent transid verify failed on 7375833595904 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375769206784 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375775793152 wanted 52059 found 52028
parent transid verify failed on 7375775793152 wanted 52059 found 52028
parent transid verify failed on 7375775809536 wanted 52059 found 52028
parent transid verify failed on 7375775809536 wanted 52059 found 52028
parent transid verify failed on 7375782952960 wanted 52059 found 52028
parent transid verify failed on 7375782952960 wanted 52059 found 52028
parent transid verify failed on 7375782952960 wanted 52059 found 52028
parent transid verify failed on 7375782952960 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375799353344 wanted 52059 found 52028
parent transid verify failed on 7375799353344 wanted 52059 found 52028
parent transid verify failed on 7375831973888 wanted 52059 found 52028
parent transid verify failed on 7375831973888 wanted 52059 found 52028
parent transid verify failed on 7375831973888 wanted 52059 found 52028
parent transid verify failed on 7375831973888 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375833137152 wanted 52059 found 52028
parent transid verify failed on 7375833137152 wanted 52059 found 52028
parent transid verify failed on 7375833137152 wanted 52059 found 52028
parent transid verify failed on 7375833137152 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375833317376 wanted 52059 found 52028
parent transid verify failed on 7375833317376 wanted 52059 found 52028
parent transid verify failed on 7375833317376 wanted 52059 found 52028
parent transid verify failed on 7375833317376 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375833579520 wanted 52059 found 52028
parent transid verify failed on 7375833579520 wanted 52059 found 52028
parent transid verify failed on 7375833579520 wanted 52059 found 52028
parent transid verify failed on 7375833579520 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375849193472 wanted 52059 found 52028
parent transid verify failed on 7375849193472 wanted 52059 found 52028
parent transid verify failed on 7375849193472 wanted 52059 found 52028
parent transid verify failed on 7375849193472 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375856041984 wanted 52059 found 52028
parent transid verify failed on 7375856041984 wanted 52059 found 52028
parent transid verify failed on 7375856041984 wanted 52059 found 52028
parent transid verify failed on 7375856041984 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375864922112 wanted 52059 found 52028
parent transid verify failed on 7375864922112 wanted 52059 found 52028
parent transid verify failed on 7375864922112 wanted 52059 found 52028
parent transid verify failed on 7375864922112 wanted 52059 found 52028
Ignoring transid failure
parent transid verify failed on 7375894282240 wanted 52059 found 52045
parent transid verify failed on 7375894282240 wanted 52059 found 52045
parent transid verify failed on 7375894282240 wanted 52059 found 52045
parent transid verify failed on 7375894282240 wanted 52059 found 52045
Ignoring transid failure
parent transid verify failed on 7375567323136 wanted 52059 found 52045
parent transid verify failed on 7375567323136 wanted 52059 found 52045
Error: could not find btree root extent for root 12974
enabling repair mode
Checking filesystem on /dev/mapper/VGBIGRAID6-DATA1
UUID: 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 12:30 btrfs partition fails to mount after power outage Malte Westerhoff
@ 2016-08-23 14:52 ` Chris Murphy
2016-08-23 15:19 ` Malte Westerhoff
2016-08-23 16:06 ` Roman Mamedov
1 sibling, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2016-08-23 14:52 UTC (permalink / raw)
To: Malte Westerhoff; +Cc: linux-btrfs@vger.kernel.org
On Tue, Aug 23, 2016 at 6:30 AM, Malte Westerhoff
<MWesterhoff@visageimaging.com> wrote:
> Hi,
> I hope this is the right list for this problem report.
>
> After an unclean shutdown of a server due to power outage, one of the partitions fails to mount.
> mount -o recovery ...,mount -o recovery,ro ..., mount -o ro do not help. They produce this error in the log:
>
> [ 5464.525816] BTRFS info (device dm-0): enabling auto recovery
> [ 5464.525823] BTRFS info (device dm-0): disk space caching is enabled
> [ 5464.525825] BTRFS: has skinny extents
> [ 5499.299686] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
> [ 5499.308576] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
> [ 5499.308587] BTRFS: Failed to read block groups: -5
> [ 5499.391359] BTRFS: open_ctree failed
>
>
> btrfs check --repair fails with an error message:
> ... (full output attached)
> Ignoring transid failure
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> Error: could not find btree root extent for root 12974
>
> This is kernel 4.1.27 and btrfs-progs 4.1.2. We also tried the btrfs check --repair with btrfs-progs 4.7 with the same result.
>
> The partition with the problem is /dev/mapper/VGBIGRAID6-DATA1
> (The partition is an LVM logical volume that is on top of an md raid6.)
Sounds to me like the raid6 is not assembling correctly or is dirty.
The btrfs structures are small enough that it's possible for incorrect
or dirty raid assembly to present some pieces of the file system to
make it look like Btrfs (like the super blocks can be found) but then
not enough for mount. Especially if this is a default md chunk size of
512KiB.
What do you get for
mdadm -E /dev/mdX
btrfs-show-super -fa /dev/mapper/VGBIGRAID6-DATA1
btrfs-find-root -d /dev/mapper/VGBIGRAID6-DATA1
>
> Anything else that we can try in order to recover the volume?
>
> Thanks
> Malte
>
> quickstore2:~/btrfs-progs # uname -a
> Linux quickstore2 4.1.27-27-default #1 SMP PREEMPT Fri Jul 15 12:46:41 UTC 2016 (84ae57e) x86_64 x86_64 x86_64 GNU/Linux
> quickstore2:~/btrfs-progs # btrfs --version
> btrfs-progs v4.1.2+20151002
> quickstore2:~/btrfs-progs # btrfs fi show
> Label: none uuid: ef3fc810-d74e-4c8e-97e3-7c7e788795ae
> Total devices 1 FS bytes used 12.14GiB
> devid 1 size 80.00GiB used 15.04GiB path /dev/sda1
>
> Label: none uuid: 9b4af403-e40c-4228-825a-20666aa0ec3c
> Total devices 1 FS bytes used 581.48GiB
> devid 1 size 8.00TiB used 616.03GiB path /dev/mapper/VGBIGRAID6-DATA2
>
> Label: none uuid: 98330879-6f67-44f0-92ba-974c86a836d1
> Total devices 1 FS bytes used 1015.34GiB
> devid 1 size 8.00TiB used 1.08TiB path /dev/mapper/VGBIGRAID6-data3
>
> Label: none uuid: 014d9f5c-48b2-4043-add3-5150ce418570
> Total devices 1 FS bytes used 3.15TiB
> devid 1 size 16.00TiB used 3.21TiB path /dev/mapper/VGBIGRAID6-data4
>
> Label: none uuid: 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
> Total devices 1 FS bytes used 6.72TiB
> devid 1 size 16.00TiB used 6.97TiB path /dev/mapper/VGBIGRAID6-DATA1
Is it the same md array that these other dataX LV's are on? And they
are mountable?
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 14:52 ` Chris Murphy
@ 2016-08-23 15:19 ` Malte Westerhoff
2016-08-23 15:45 ` Chris Murphy
0 siblings, 1 reply; 9+ messages in thread
From: Malte Westerhoff @ 2016-08-23 15:19 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs@vger.kernel.org
[-- Attachment #1: Type: text/plain, Size: 11771 bytes --]
Hi Chris,
Thanks for the response.
Yes, there is one large raid6 on which there is one LVM physical volume, which has four logical volumes that each have a btrfs file system.
Only one of them fails to mount, the other three are fine (data2...4).
The RAID itself claims to be clean (see below).
Output of btrfs-show-super attached.
btrfs-find-root is running now for 30 minutes. It has produced the following output so far (but is still running – not sure whether it is hanging or will eventually terminate).
quickstore2:~/btrfs-progs # ./btrfs-find-root /dev/mapper/VGBIGRAID6-DATA1
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
Ignoring transid failure
leaf parent key incorrect 7375769206784
Superblock thinks the generation is 52059
Superblock thinks the level is 1
quickstore2:~ # mdadm --detail /dev/md/BIGRAID6
/dev/md/BIGRAID6:
Version : 1.0
Creation Time : Fri Jan 29 15:05:42 2016
Raid Level : raid6
Array Size : 109396349440 (104328.49 GiB 112021.86 GB)
Used Dev Size : 7814024960 (7452.04 GiB 8001.56 GB)
Raid Devices : 16
Total Devices : 16
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Tue Aug 23 17:03:29 2016
State : active
Active Devices : 16
Working Devices : 16
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 128K
Name : any:BIGRAID6
UUID : 7145b27b:cd5b7c9e:79d86f7b:46f54aeb
Events : 17908
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1
2 8 49 2 active sync /dev/sdd1
3 8 65 3 active sync /dev/sde1
4 8 81 4 active sync /dev/sdf1
5 8 97 5 active sync /dev/sdg1
6 8 113 6 active sync /dev/sdh1
7 8 129 7 active sync /dev/sdi1
8 8 145 8 active sync /dev/sdj1
9 8 161 9 active sync /dev/sdk1
10 8 177 10 active sync /dev/sdl1
11 8 193 11 active sync /dev/sdm1
12 8 209 12 active sync /dev/sdn1
13 8 225 13 active sync /dev/sdo1
14 8 241 14 active sync /dev/sdp1
15 65 1 15 active sync /dev/sdq1
quickstore2:~ # mdadm -E /dev/sdc1
/dev/sdc1:
Magic : a92b4efc
Version : 1.0
Feature Map : 0x1
Array UUID : 7145b27b:cd5b7c9e:79d86f7b:46f54aeb
Name : any:BIGRAID6
Creation Time : Fri Jan 29 15:05:42 2016
Raid Level : raid6
Raid Devices : 16
Avail Dev Size : 15628050152 (7452.04 GiB 8001.56 GB)
Array Size : 109396349440 (104328.49 GiB 112021.86 GB)
Used Dev Size : 15628049920 (7452.04 GiB 8001.56 GB)
Super Offset : 15628050416 sectors
Unused Space : before=0 sectors, after=456 sectors
State : clean
Device UUID : f03ba2d9:e25d52a2:616ee62e:316017ff
Internal Bitmap : -40 sectors from superblock
Update Time : Tue Aug 23 17:10:45 2016
Bad Block Log : 512 entries available at offset -8 sectors
Checksum : 3007a4b6 - correct
Events : 17907
Layout : left-symmetric
Chunk Size : 128K
Device Role : Active device 1
Array State : AAAAAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
Malte
On 8/23/16, 4:52 PM, "chris@colorremedies.com on behalf of Chris Murphy" <chris@colorremedies.com on behalf of lists@colorremedies.com> wrote:
On Tue, Aug 23, 2016 at 6:30 AM, Malte Westerhoff
<MWesterhoff@visageimaging.com> wrote:
> Hi,
> I hope this is the right list for this problem report.
>
> After an unclean shutdown of a server due to power outage, one of the partitions fails to mount.
> mount -o recovery ...,mount -o recovery,ro ..., mount -o ro do not help. They produce this error in the log:
>
> [ 5464.525816] BTRFS info (device dm-0): enabling auto recovery
> [ 5464.525823] BTRFS info (device dm-0): disk space caching is enabled
> [ 5464.525825] BTRFS: has skinny extents
> [ 5499.299686] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
> [ 5499.308576] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
> [ 5499.308587] BTRFS: Failed to read block groups: -5
> [ 5499.391359] BTRFS: open_ctree failed
>
>
> btrfs check --repair fails with an error message:
> ... (full output attached)
> Ignoring transid failure
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> Error: could not find btree root extent for root 12974
>
> This is kernel 4.1.27 and btrfs-progs 4.1.2. We also tried the btrfs check --repair with btrfs-progs 4.7 with the same result.
>
> The partition with the problem is /dev/mapper/VGBIGRAID6-DATA1
> (The partition is an LVM logical volume that is on top of an md raid6.)
Sounds to me like the raid6 is not assembling correctly or is dirty.
The btrfs structures are small enough that it's possible for incorrect
or dirty raid assembly to present some pieces of the file system to
make it look like Btrfs (like the super blocks can be found) but then
not enough for mount. Especially if this is a default md chunk size of
512KiB.
What do you get for
mdadm -E /dev/mdX
btrfs-show-super -fa /dev/mapper/VGBIGRAID6-DATA1
btrfs-find-root -d /dev/mapper/VGBIGRAID6-DATA1
>
> Anything else that we can try in order to recover the volume?
>
> Thanks
> Malte
>
> quickstore2:~/btrfs-progs # uname -a
> Linux quickstore2 4.1.27-27-default #1 SMP PREEMPT Fri Jul 15 12:46:41 UTC 2016 (84ae57e) x86_64 x86_64 x86_64 GNU/Linux
> quickstore2:~/btrfs-progs # btrfs --version
> btrfs-progs v4.1.2+20151002
> quickstore2:~/btrfs-progs # btrfs fi show
> Label: none uuid: ef3fc810-d74e-4c8e-97e3-7c7e788795ae
> Total devices 1 FS bytes used 12.14GiB
> devid 1 size 80.00GiB used 15.04GiB path /dev/sda1
>
> Label: none uuid: 9b4af403-e40c-4228-825a-20666aa0ec3c
> Total devices 1 FS bytes used 581.48GiB
> devid 1 size 8.00TiB used 616.03GiB path /dev/mapper/VGBIGRAID6-DATA2
>
> Label: none uuid: 98330879-6f67-44f0-92ba-974c86a836d1
> Total devices 1 FS bytes used 1015.34GiB
> devid 1 size 8.00TiB used 1.08TiB path /dev/mapper/VGBIGRAID6-data3
>
> Label: none uuid: 014d9f5c-48b2-4043-add3-5150ce418570
> Total devices 1 FS bytes used 3.15TiB
> devid 1 size 16.00TiB used 3.21TiB path /dev/mapper/VGBIGRAID6-data4
>
> Label: none uuid: 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
> Total devices 1 FS bytes used 6.72TiB
> devid 1 size 16.00TiB used 6.97TiB path /dev/mapper/VGBIGRAID6-DATA1
Is it the same md array that these other dataX LV's are on? And they
are mountable?
--
Chris Murphy
On 8/23/16, 4:52 PM, "chris@colorremedies.com on behalf of Chris Murphy" <chris@colorremedies.com on behalf of lists@colorremedies.com> wrote:
On Tue, Aug 23, 2016 at 6:30 AM, Malte Westerhoff
<MWesterhoff@visageimaging.com> wrote:
> Hi,
> I hope this is the right list for this problem report.
>
> After an unclean shutdown of a server due to power outage, one of the partitions fails to mount.
> mount -o recovery ...,mount -o recovery,ro ..., mount -o ro do not help. They produce this error in the log:
>
> [ 5464.525816] BTRFS info (device dm-0): enabling auto recovery
> [ 5464.525823] BTRFS info (device dm-0): disk space caching is enabled
> [ 5464.525825] BTRFS: has skinny extents
> [ 5499.299686] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
> [ 5499.308576] BTRFS (device dm-0): parent transid verify failed on 7375769206784 wanted 52059 found 52028
> [ 5499.308587] BTRFS: Failed to read block groups: -5
> [ 5499.391359] BTRFS: open_ctree failed
>
>
> btrfs check --repair fails with an error message:
> ... (full output attached)
> Ignoring transid failure
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> Error: could not find btree root extent for root 12974
>
> This is kernel 4.1.27 and btrfs-progs 4.1.2. We also tried the btrfs check --repair with btrfs-progs 4.7 with the same result.
>
> The partition with the problem is /dev/mapper/VGBIGRAID6-DATA1
> (The partition is an LVM logical volume that is on top of an md raid6.)
Sounds to me like the raid6 is not assembling correctly or is dirty.
The btrfs structures are small enough that it's possible for incorrect
or dirty raid assembly to present some pieces of the file system to
make it look like Btrfs (like the super blocks can be found) but then
not enough for mount. Especially if this is a default md chunk size of
512KiB.
What do you get for
mdadm -E /dev/mdX
btrfs-show-super -fa /dev/mapper/VGBIGRAID6-DATA1
btrfs-find-root -d /dev/mapper/VGBIGRAID6-DATA1
>
> Anything else that we can try in order to recover the volume?
>
> Thanks
> Malte
>
> quickstore2:~/btrfs-progs # uname -a
> Linux quickstore2 4.1.27-27-default #1 SMP PREEMPT Fri Jul 15 12:46:41 UTC 2016 (84ae57e) x86_64 x86_64 x86_64 GNU/Linux
> quickstore2:~/btrfs-progs # btrfs --version
> btrfs-progs v4.1.2+20151002
> quickstore2:~/btrfs-progs # btrfs fi show
> Label: none uuid: ef3fc810-d74e-4c8e-97e3-7c7e788795ae
> Total devices 1 FS bytes used 12.14GiB
> devid 1 size 80.00GiB used 15.04GiB path /dev/sda1
>
> Label: none uuid: 9b4af403-e40c-4228-825a-20666aa0ec3c
> Total devices 1 FS bytes used 581.48GiB
> devid 1 size 8.00TiB used 616.03GiB path /dev/mapper/VGBIGRAID6-DATA2
>
> Label: none uuid: 98330879-6f67-44f0-92ba-974c86a836d1
> Total devices 1 FS bytes used 1015.34GiB
> devid 1 size 8.00TiB used 1.08TiB path /dev/mapper/VGBIGRAID6-data3
>
> Label: none uuid: 014d9f5c-48b2-4043-add3-5150ce418570
> Total devices 1 FS bytes used 3.15TiB
> devid 1 size 16.00TiB used 3.21TiB path /dev/mapper/VGBIGRAID6-data4
>
> Label: none uuid: 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
> Total devices 1 FS bytes used 6.72TiB
> devid 1 size 16.00TiB used 6.97TiB path /dev/mapper/VGBIGRAID6-DATA1
Is it the same md array that these other dataX LV's are on? And they
are mountable?
--
Chris Murphy
[-- Attachment #2: btrfs-show-super.txt --]
[-- Type: text/plain, Size: 9779 bytes --]
superblock: bytenr=65536, device=/dev/mapper/VGBIGRAID6-DATA1
---------------------------------------------------------
csum 0xd6fc94ec [match]
bytenr 65536
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
label
generation 52059
root 3421559472128
sys_array_size 226
chunk_root_generation 52005
root_level 1
chunk_root 20971520
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 17592186044416
bytes_used 7386753196032
sectorsize 4096
nodesize 16384
leafsize 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
csum_type 0
csum_size 4
cache_generation 52059
uuid_tree_generation 52059
dev_item.uuid 4e582c98-ccfc-4740-8e59-a1cb1cbfca56
dev_item.fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2 [match]
dev_item.type 0
dev_item.total_bytes 17592186044416
dev_item.bytes_used 7661185662976
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
chunk length 4194304 owner 2 type SYSTEM num_stripes 1
stripe 0 devid 1 offset 0
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520)
chunk length 8388608 owner 2 type SYSTEM|DUP num_stripes 2
stripe 0 devid 1 offset 20971520
stripe 1 devid 1 offset 29360128
backup_roots[4]:
backup 0:
backup_tree_root: 3421559472128 gen: 52059 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 3421770170368 gen: 52059 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7304828502016 gen: 52059 level: 1
backup_csum_root: 7315970588672 gen: 52059 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753196032
backup_num_devices: 1
backup 1:
backup_tree_root: 7442297503744 gen: 52056 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442294161408 gen: 52056 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298175488 gen: 52056 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 2:
backup_tree_root: 7442289229824 gen: 52057 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442283692032 gen: 52057 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442294063104 gen: 52057 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
backup 3:
backup_tree_root: 7442297503744 gen: 52058 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442284920832 gen: 52058 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442299355136 gen: 52058 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
superblock: bytenr=67108864, device=/dev/mapper/VGBIGRAID6-DATA1
---------------------------------------------------------
csum 0x69cac6a6 [match]
bytenr 67108864
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
label
generation 52058
root 7442297503744
sys_array_size 226
chunk_root_generation 52005
root_level 1
chunk_root 20971520
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 17592186044416
bytes_used 7386753179648
sectorsize 4096
nodesize 16384
leafsize 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
csum_type 0
csum_size 4
cache_generation 52058
uuid_tree_generation 52058
dev_item.uuid 4e582c98-ccfc-4740-8e59-a1cb1cbfca56
dev_item.fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2 [match]
dev_item.type 0
dev_item.total_bytes 17592186044416
dev_item.bytes_used 7661185662976
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
chunk length 4194304 owner 2 type SYSTEM num_stripes 1
stripe 0 devid 1 offset 0
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520)
chunk length 8388608 owner 2 type SYSTEM|DUP num_stripes 2
stripe 0 devid 1 offset 20971520
stripe 1 devid 1 offset 29360128
backup_roots[4]:
backup 0:
backup_tree_root: 7442297978880 gen: 52055 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442297290752 gen: 52055 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298601472 gen: 52055 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 1:
backup_tree_root: 7442297503744 gen: 52056 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442294161408 gen: 52056 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298175488 gen: 52056 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 2:
backup_tree_root: 7442289229824 gen: 52057 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442283692032 gen: 52057 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442294063104 gen: 52057 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
backup 3:
backup_tree_root: 7442297503744 gen: 52058 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442284920832 gen: 52058 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442299355136 gen: 52058 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
superblock: bytenr=274877906944, device=/dev/mapper/VGBIGRAID6-DATA1
---------------------------------------------------------
csum 0x944d9097 [match]
bytenr 274877906944
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
label
generation 52058
root 7442297503744
sys_array_size 226
chunk_root_generation 52005
root_level 1
chunk_root 20971520
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 17592186044416
bytes_used 7386753179648
sectorsize 4096
nodesize 16384
leafsize 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
csum_type 0
csum_size 4
cache_generation 52058
uuid_tree_generation 52058
dev_item.uuid 4e582c98-ccfc-4740-8e59-a1cb1cbfca56
dev_item.fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2 [match]
dev_item.type 0
dev_item.total_bytes 17592186044416
dev_item.bytes_used 7661185662976
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
chunk length 4194304 owner 2 type SYSTEM num_stripes 1
stripe 0 devid 1 offset 0
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520)
chunk length 8388608 owner 2 type SYSTEM|DUP num_stripes 2
stripe 0 devid 1 offset 20971520
stripe 1 devid 1 offset 29360128
backup_roots[4]:
backup 0:
backup_tree_root: 7442297978880 gen: 52055 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442297290752 gen: 52055 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298601472 gen: 52055 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 1:
backup_tree_root: 7442297503744 gen: 52056 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442294161408 gen: 52056 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298175488 gen: 52056 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 2:
backup_tree_root: 7442289229824 gen: 52057 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442283692032 gen: 52057 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442294063104 gen: 52057 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
backup 3:
backup_tree_root: 7442297503744 gen: 52058 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442284920832 gen: 52058 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442299355136 gen: 52058 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 15:19 ` Malte Westerhoff
@ 2016-08-23 15:45 ` Chris Murphy
2016-08-23 15:51 ` Malte Westerhoff
0 siblings, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2016-08-23 15:45 UTC (permalink / raw)
To: Malte Westerhoff; +Cc: Chris Murphy, linux-btrfs@vger.kernel.org
On Tue, Aug 23, 2016 at 9:19 AM, Malte Westerhoff
<MWesterhoff@visageimaging.com> wrote:
> Hi Chris,
> Thanks for the response.
> Yes, there is one large raid6 on which there is one LVM physical volume, which has four logical volumes that each have a btrfs file system.
> Only one of them fails to mount, the other three are fine (data2...4).
> The RAID itself claims to be clean (see below).
>
> Output of btrfs-show-super attached.
>
> btrfs-find-root is running now for 30 minutes. It has produced the following output so far (but is still running – not sure whether it is hanging or will eventually terminate).
>
> quickstore2:~/btrfs-progs # ./btrfs-find-root /dev/mapper/VGBIGRAID6-DATA1
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> Ignoring transid failure
> leaf parent key incorrect 7375769206784
> Superblock thinks the generation is 52059
> Superblock thinks the level is 1
Huh. Well someone who knows more about Btrfs and devices doing the
wrong thing will have to speak up. To me this looks like something got
the commit order wrong, where the superblock has a more recent
generation than any tree generation, as if the metadata for generation
52059 (or even 52058, 52057, or 52056) is not written or not where
it's supposed to be, and yet the superblock was updated.
>> btrfs-show-super -fa /dev/mapper/VGBIGRAID6-DATA1
Where is this output?
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 15:45 ` Chris Murphy
@ 2016-08-23 15:51 ` Malte Westerhoff
2016-08-23 16:05 ` Chris Murphy
0 siblings, 1 reply; 9+ messages in thread
From: Malte Westerhoff @ 2016-08-23 15:51 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs@vger.kernel.org
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 12007 bytes --]
>> btrfs-show-super -fa /dev/mapper/VGBIGRAID6-DATA1
> Where is this output?
I had attached it as a separate file. Here it is again:
superblock: bytenr=65536, device=/dev/mapper/VGBIGRAID6-DATA1
---------------------------------------------------------
csum 0xd6fc94ec [match]
bytenr 65536
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
label
generation 52059
root 3421559472128
sys_array_size 226
chunk_root_generation 52005
root_level 1
chunk_root 20971520
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 17592186044416
bytes_used 7386753196032
sectorsize 4096
nodesize 16384
leafsize 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
csum_type 0
csum_size 4
cache_generation 52059
uuid_tree_generation 52059
dev_item.uuid 4e582c98-ccfc-4740-8e59-a1cb1cbfca56
dev_item.fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2 [match]
dev_item.type 0
dev_item.total_bytes 17592186044416
dev_item.bytes_used 7661185662976
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
chunk length 4194304 owner 2 type SYSTEM num_stripes 1
stripe 0 devid 1 offset 0
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520)
chunk length 8388608 owner 2 type SYSTEM|DUP num_stripes 2
stripe 0 devid 1 offset 20971520
stripe 1 devid 1 offset 29360128
backup_roots[4]:
backup 0:
backup_tree_root: 3421559472128 gen: 52059 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 3421770170368 gen: 52059 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7304828502016 gen: 52059 level: 1
backup_csum_root: 7315970588672 gen: 52059 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753196032
backup_num_devices: 1
backup 1:
backup_tree_root: 7442297503744 gen: 52056 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442294161408 gen: 52056 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298175488 gen: 52056 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 2:
backup_tree_root: 7442289229824 gen: 52057 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442283692032 gen: 52057 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442294063104 gen: 52057 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
backup 3:
backup_tree_root: 7442297503744 gen: 52058 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442284920832 gen: 52058 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442299355136 gen: 52058 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
superblock: bytenr=67108864, device=/dev/mapper/VGBIGRAID6-DATA1
---------------------------------------------------------
csum 0x69cac6a6 [match]
bytenr 67108864
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
label
generation 52058
root 7442297503744
sys_array_size 226
chunk_root_generation 52005
root_level 1
chunk_root 20971520
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 17592186044416
bytes_used 7386753179648
sectorsize 4096
nodesize 16384
leafsize 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
csum_type 0
csum_size 4
cache_generation 52058
uuid_tree_generation 52058
dev_item.uuid 4e582c98-ccfc-4740-8e59-a1cb1cbfca56
dev_item.fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2 [match]
dev_item.type 0
dev_item.total_bytes 17592186044416
dev_item.bytes_used 7661185662976
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
chunk length 4194304 owner 2 type SYSTEM num_stripes 1
stripe 0 devid 1 offset 0
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520)
chunk length 8388608 owner 2 type SYSTEM|DUP num_stripes 2
stripe 0 devid 1 offset 20971520
stripe 1 devid 1 offset 29360128
backup_roots[4]:
backup 0:
backup_tree_root: 7442297978880 gen: 52055 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442297290752 gen: 52055 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298601472 gen: 52055 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 1:
backup_tree_root: 7442297503744 gen: 52056 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442294161408 gen: 52056 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298175488 gen: 52056 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 2:
backup_tree_root: 7442289229824 gen: 52057 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442283692032 gen: 52057 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442294063104 gen: 52057 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
backup 3:
backup_tree_root: 7442297503744 gen: 52058 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442284920832 gen: 52058 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442299355136 gen: 52058 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
superblock: bytenr=274877906944, device=/dev/mapper/VGBIGRAID6-DATA1
---------------------------------------------------------
csum 0x944d9097 [match]
bytenr 274877906944
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
label
generation 52058
root 7442297503744
sys_array_size 226
chunk_root_generation 52005
root_level 1
chunk_root 20971520
chunk_root_level 1
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 17592186044416
bytes_used 7386753179648
sectorsize 4096
nodesize 16384
leafsize 16384
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x163
( MIXED_BACKREF |
DEFAULT_SUBVOL |
BIG_METADATA |
EXTENDED_IREF |
SKINNY_METADATA )
csum_type 0
csum_size 4
cache_generation 52058
uuid_tree_generation 52058
dev_item.uuid 4e582c98-ccfc-4740-8e59-a1cb1cbfca56
dev_item.fsid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2 [match]
dev_item.type 0
dev_item.total_bytes 17592186044416
dev_item.bytes_used 7661185662976
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
sys_chunk_array[2048]:
item 0 key (FIRST_CHUNK_TREE CHUNK_ITEM 0)
chunk length 4194304 owner 2 type SYSTEM num_stripes 1
stripe 0 devid 1 offset 0
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520)
chunk length 8388608 owner 2 type SYSTEM|DUP num_stripes 2
stripe 0 devid 1 offset 20971520
stripe 1 devid 1 offset 29360128
backup_roots[4]:
backup 0:
backup_tree_root: 7442297978880 gen: 52055 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442297290752 gen: 52055 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298601472 gen: 52055 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 1:
backup_tree_root: 7442297503744 gen: 52056 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442294161408 gen: 52056 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442298175488 gen: 52056 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753163264
backup_num_devices: 1
backup 2:
backup_tree_root: 7442289229824 gen: 52057 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442283692032 gen: 52057 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442294063104 gen: 52057 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
backup 3:
backup_tree_root: 7442297503744 gen: 52058 level: 1
backup_chunk_root: 20971520 gen: 52005 level: 1
backup_extent_root: 7442284920832 gen: 52058 level: 3
backup_fs_root: 29556736 gen: 6 level: 0
backup_dev_root: 7254075654144 gen: 52005 level: 1
backup_csum_root: 7442299355136 gen: 52058 level: 3
backup_total_bytes: 17592186044416
backup_bytes_used: 7386753179648
backup_num_devices: 1
On 8/23/16, 5:45 PM, "chris@colorremedies.com on behalf of Chris Murphy" <chris@colorremedies.com on behalf of lists@colorremedies.com> wrote:
On Tue, Aug 23, 2016 at 9:19 AM, Malte Westerhoff
<MWesterhoff@visageimaging.com> wrote:
> Hi Chris,
> Thanks for the response.
> Yes, there is one large raid6 on which there is one LVM physical volume, which has four logical volumes that each have a btrfs file system.
> Only one of them fails to mount, the other three are fine (data2...4).
> The RAID itself claims to be clean (see below).
>
> Output of btrfs-show-super attached.
>
> btrfs-find-root is running now for 30 minutes. It has produced the following output so far (but is still running – not sure whether it is hanging or will eventually terminate).
>
> quickstore2:~/btrfs-progs # ./btrfs-find-root /dev/mapper/VGBIGRAID6-DATA1
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> parent transid verify failed on 7375769206784 wanted 52059 found 52028
> Ignoring transid failure
> leaf parent key incorrect 7375769206784
> Superblock thinks the generation is 52059
> Superblock thinks the level is 1
Huh. Well someone who knows more about Btrfs and devices doing the
wrong thing will have to speak up. To me this looks like something got
the commit order wrong, where the superblock has a more recent
generation than any tree generation, as if the metadata for generation
52059 (or even 52058, 52057, or 52056) is not written or not where
it's supposed to be, and yet the superblock was updated.
>> btrfs-show-super -fa /dev/mapper/VGBIGRAID6-DATA1
Where is this output?
--
Chris Murphy
ÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨èÚ&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~††Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 15:51 ` Malte Westerhoff
@ 2016-08-23 16:05 ` Chris Murphy
2016-08-23 16:07 ` Chris Murphy
0 siblings, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2016-08-23 16:05 UTC (permalink / raw)
To: Malte Westerhoff; +Cc: Chris Murphy, linux-btrfs@vger.kernel.org
On Tue, Aug 23, 2016 at 9:51 AM, Malte Westerhoff
<MWesterhoff@visageimaging.com> wrote:
> >> btrfs-show-super -fa /d/dev/mapper/VGBIGRAID6-DATA1
>
>> Where is this output?
>
> I had attached it as a separate file. Here it is again:
Sorry, missed that there was an attachment.
> backup_roots[4]:
> backup 0:
> backup_tree_root: 3421559472128 gen: 52059 level: 1
> backup_chunk_root: 20971520 gen: 52005 level: 1
> backup_extent_root: 3421770170368 gen: 52059 level: 3
> backup_fs_root: 29556736 gen: 6 level: 0
> backup_dev_root: 7304828502016 gen: 52059 level: 1
> backup_csum_root: 7315970588672 gen: 52059 level: 3
> backup_total_bytes: 17592186044416
> backup_bytes_used: 7386753196032
> backup_num_devices: 1
>
> backup 1:
> backup_tree_root: 7442297503744 gen: 52056 level: 1
> backup_chunk_root: 20971520 gen: 52005 level: 1
> backup_extent_root: 7442294161408 gen: 52056 level: 3
> backup_fs_root: 29556736 gen: 6 level: 0
> backup_dev_root: 7254075654144 gen: 52005 level: 1
> backup_csum_root: 7442298175488 gen: 52056 level: 3
> backup_total_bytes: 17592186044416
> backup_bytes_used: 7386753163264
> backup_num_devices: 1
>
> backup 2:
> backup_tree_root: 7442289229824 gen: 52057 level: 1
> backup_chunk_root: 20971520 gen: 52005 level: 1
> backup_extent_root: 7442283692032 gen: 52057 level: 3
> backup_fs_root: 29556736 gen: 6 level: 0
> backup_dev_root: 7254075654144 gen: 52005 level: 1
> backup_csum_root: 7442294063104 gen: 52057 level: 3
> backup_total_bytes: 17592186044416
> backup_bytes_used: 7386753179648
> backup_num_devices: 1
>
> backup 3:
> backup_tree_root: 7442297503744 gen: 52058 level: 1
> backup_chunk_root: 20971520 gen: 52005 level: 1
> backup_extent_root: 7442284920832 gen: 52058 level: 3
> backup_fs_root: 29556736 gen: 6 level: 0
> backup_dev_root: 7254075654144 gen: 52005 level: 1
> backup_csum_root: 7442299355136 gen: 52058 level: 3
> backup_total_bytes: 17592186044416
> backup_bytes_used: 7386753179648
> backup_num_devices: 1
Uhhhh, is anyone else having a WTF moment? Why is the backup_fs_root
generation 6 on all of these backup roots? Doesn't that seem really
unlikely? On all of my filesystems, the fs_root has a lower
generation, but isn't separated by this many generations.
What do you get for:
btrfs-debug-tree -b 29556736 /dev/mapper/VGBIGRAID6-DATA1
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 12:30 btrfs partition fails to mount after power outage Malte Westerhoff
2016-08-23 14:52 ` Chris Murphy
@ 2016-08-23 16:06 ` Roman Mamedov
1 sibling, 0 replies; 9+ messages in thread
From: Roman Mamedov @ 2016-08-23 16:06 UTC (permalink / raw)
To: Malte Westerhoff; +Cc: linux-btrfs@vger.kernel.org
[-- Attachment #1: Type: text/plain, Size: 1920 bytes --]
On Tue, 23 Aug 2016 12:30:35 +0000
Malte Westerhoff <MWesterhoff@visageimaging.com> wrote:
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> parent transid verify failed on 7375567323136 wanted 52059 found 52045
> Error: could not find btree root extent for root 12974
>
> This is kernel 4.1.27 and btrfs-progs 4.1.2. We also tried the btrfs check --repair with btrfs-progs 4.7 with the same result.
>
> The partition with the problem is /dev/mapper/VGBIGRAID6-DATA1
> (The partition is an LVM logical volume that is on top of an md raid6.)
>
> Anything else that we can try in order to recover the volume?
In my experience the last resort solution to this is to simply kill the
problematic block with the "btrfs-corrupt-block" tool (compile from
btrfs-progs source), then run "btrfs check --repair" again, and this time it
should quite trivially set things right. See my success story about that:
http://www.spinics.net/lists/linux-btrfs/msg53061.html
and especially note the part about making COW snapshots of the entire block
device (should be easier since you already run LVM), which will allow you to
experiment while having an ability to undo.
Some files will be lost or corrupted (likely only an extremely tiny portion,
but not knowing which -- sucks), so ideally you should always have means to
verify the integrity of user data on any FS. At least for those which change
infrequently, e.g. media file libraries, always keep '*.sfv' files produced by
the 'cfv' tool beside everything. Also something along the lines of
cd /mnt/mydata && find > /somewhereelse/mydata-`date -Iseconds`.lst
in crontab.
Before starting you can also try checking what kind of data the problematic
block contains, IIRC it would be (but not guaranteed to be helpful):
btrfs-debug-tree -b 7375567323136 /dev/mapper/VGBIGRAID6-DATA1
--
With respect,
Roman
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 16:05 ` Chris Murphy
@ 2016-08-23 16:07 ` Chris Murphy
2016-08-23 16:29 ` Malte Westerhoff
0 siblings, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2016-08-23 16:07 UTC (permalink / raw)
To: Chris Murphy; +Cc: Malte Westerhoff, linux-btrfs@vger.kernel.org
On Tue, Aug 23, 2016 at 10:05 AM, Chris Murphy <lists@colorremedies.com> wrote:
>btrfs-find-root -d /dev/mapper/VGBIGRAID6-DATA1
I meant to ask for
btrfs-debug-tree -d /dev/...
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs partition fails to mount after power outage
2016-08-23 16:07 ` Chris Murphy
@ 2016-08-23 16:29 ` Malte Westerhoff
0 siblings, 0 replies; 9+ messages in thread
From: Malte Westerhoff @ 2016-08-23 16:29 UTC (permalink / raw)
To: Chris Murphy; +Cc: linux-btrfs@vger.kernel.org
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2176 bytes --]
The output for btrfs-debug-tree –d /dev/… is quite large – so I have placed it here:
https://cloud.visageimaging.com/index.php/s/Q7ILbVWFoGeckCI
The –b one is this:
quickstore2:~/btrfs-progs # ./btrfs-debug-tree -b 29556736 /dev/mapper/VGBIGRAID6-DATA1
btrfs-progs v4.7
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
parent transid verify failed on 7375769206784 wanted 52059 found 52028
Ignoring transid failure
leaf parent key incorrect 7375769206784
leaf 29556736 items 4 free space 15949 generation 6 owner 5
fs uuid 0397b3eb-fdc4-4d3f-9cc9-d38467dcb6c2
chunk uuid 74addbad-a6f1-4c82-9e70-97d9aab63103
item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
inode generation 3 transid 6 size 2 nbytes 16384
block group 0 mode 40755 links 1 uid 0 gid 0
rdev 0 flags 0x0
item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12
inode ref index 0 namelen 2 name: ..
item 2 key (256 DIR_ITEM 512897553) itemoff 16080 itemsize 31
location key (257 ROOT_ITEM -1) type DIR
namelen 1 datalen 0 name: @
item 3 key (256 DIR_INDEX 2) itemoff 16049 itemsize 31
location key (257 ROOT_ITEM -1) type DIR
namelen 1 datalen 0 name: @
On 8/23/16, 6:07 PM, "linux-btrfs-owner@vger.kernel.org on behalf of Chris Murphy" <linux-btrfs-owner@vger.kernel.org on behalf of lists@colorremedies.com> wrote:
On Tue, Aug 23, 2016 at 10:05 AM, Chris Murphy <lists@colorremedies.com> wrote:
>btrfs-find-root -d /dev/mapper/VGBIGRAID6-DATA1
I meant to ask for
btrfs-debug-tree -d /dev/...
--
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
ÿôèº{.nÇ+‰·Ÿ®‰†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±ý»k~ÏâžØ^n‡r¡ö¦zË\x1aëh™¨èÚ&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~††Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣøm
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-08-23 16:29 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-08-23 12:30 btrfs partition fails to mount after power outage Malte Westerhoff
2016-08-23 14:52 ` Chris Murphy
2016-08-23 15:19 ` Malte Westerhoff
2016-08-23 15:45 ` Chris Murphy
2016-08-23 15:51 ` Malte Westerhoff
2016-08-23 16:05 ` Chris Murphy
2016-08-23 16:07 ` Chris Murphy
2016-08-23 16:29 ` Malte Westerhoff
2016-08-23 16:06 ` Roman Mamedov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).