* btrfs unmountable: read block failed check_tree_block; Couldn't read tree root @ 2014-10-27 13:23 Ansgar Hockmann-Stolle 2014-10-27 23:03 ` Ansgar Hockmann-Stolle 2014-10-27 23:17 ` Duncan 0 siblings, 2 replies; 8+ messages in thread From: Ansgar Hockmann-Stolle @ 2014-10-27 13:23 UTC (permalink / raw) To: linux-btrfs Hi! My btrfs system partition went readonly. After reboot it doesnt mount anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 GB SSD to some USB file and tried some btrfs tools on another copy per loopback device. But everything failed with: kernel: BTRFS: failed to read tree root on dm-2 See http://pastebin.com/raw.php?i=dPnU6nzg. Any hints where to go from here? Ciao Ansgar -- Ansgar Hockmann-Stolle, Universität Osnabrück, Rechenzentrum Albrechtstraße 28, 49076 Osnabrück, Deutschland, Raum 31/E77B +49 541 969-2749 (fax -2470), http://www.home.uos.de/anshockm ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-27 13:23 btrfs unmountable: read block failed check_tree_block; Couldn't read tree root Ansgar Hockmann-Stolle @ 2014-10-27 23:03 ` Ansgar Hockmann-Stolle 2014-10-28 1:05 ` Qu Wenruo 2014-11-04 22:40 ` [SOLVED] " Ansgar Hockmann-Stolle 2014-10-27 23:17 ` Duncan 1 sibling, 2 replies; 8+ messages in thread From: Ansgar Hockmann-Stolle @ 2014-10-27 23:03 UTC (permalink / raw) To: linux-btrfs Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: > Hi! > > My btrfs system partition went readonly. After reboot it doesnt mount > anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm > on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 > GB SSD to some USB file and tried some btrfs tools on another copy per > loopback device. But everything failed with: > > kernel: BTRFS: failed to read tree root on dm-2 > > See http://pastebin.com/raw.php?i=dPnU6nzg. > > Any hints where to go from here? After an offlist hint (thanks Tom!) I compiled the latest btrfs-progs 3.17 and tried some more ... linux:~/bin # ./btrfs --version Btrfs v3.17 linux:~/bin # ./btrfs-find-root /dev/sda3 Super think's the tree root is at 1015238656, chunk root 20971520 Well block 239718400 seems great, but generation doesn't match, have=661931, want=663595 level 0 Well block 239722496 seems great, but generation doesn't match, have=661931, want=663595 level 0 Well block 320098304 seems great, but generation doesn't match, have=662233, want=663595 level 0 Well block 879341568 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 879345664 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 879382528 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 879398912 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 879403008 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 879423488 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 879435776 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 880095232 seems great, but generation doesn't match, have=663227, want=663595 level 0 Well block 881504256 seems great, but generation doesn't match, have=663228, want=663595 level 0 Well block 881512448 seems great, but generation doesn't match, have=663228, want=663595 level 0 Well block 936271872 seems great, but generation doesn't match, have=663397, want=663595 level 0 Well block 1004490752 seems great, but generation doesn't match, have=663571, want=663595 level 0 Well block 1007804416 seems great, but generation doesn't match, have=663572, want=663595 level 0 Well block 1012031488 seems great, but generation doesn't match, have=663575, want=663595 level 0 Well block 1012396032 seems great, but generation doesn't match, have=663575, want=663595 level 0 Well block 1012633600 seems great, but generation doesn't match, have=663586, want=663595 level 0 Well block 1012871168 seems great, but generation doesn't match, have=663585, want=663595 level 0 Well block 1015201792 seems great, but generation doesn't match, have=663588, want=663595 level 0 Well block 1015836672 seems great, but generation doesn't match, have=663596, want=663595 level 1 Well block 44132536320 seems great, but generation doesn't match, have=658774, want=663595 level 0 Well block 44178280448 seems great, but generation doesn't match, have=658774, want=663595 level 0 Well block 87443644416 seems great, but generation doesn't match, have=661349, want=663595 level 0 Well block 87514079232 seems great, but generation doesn't match, have=651051, want=663595 level 0 Well block 87517679616 seems great, but generation doesn't match, have=661349, want=663595 level 0 Well block 98697822208 seems great, but generation doesn't match, have=643548, want=663595 level 0 Well block 103285026816 seems great, but generation doesn't match, have=661672, want=663595 level 0 Well block 103309553664 seems great, but generation doesn't match, have=661674, want=663595 level 0 Well block 103523430400 seems great, but generation doesn't match, have=661767, want=663595 level 0 No more metdata to scan, exiting This line I found interesting because "have" is "want + 1": Well block 1015836672 seems great, but generation doesn't match, have=663596, want=663595 level 1 And here the tail of "btrfs rescue chunk-recover" (full output at http://pastebin.com/raw.php?i=1D5VgDxv) [..] Total Chunks: 234 Heathy: 231 Bad: 3 Orphan Block Groups: Orphan Device Extents: Couldn't map the block 1015238656 btrfs: volumes.c:1140: btrfs_num_copies: Assertion `!(ce->start > logical || ce->start + ce->size < logical)' failed. Aborted Sadly "btrfs check --repair" keep up refusing to do its job. linux:~ # btrfs check --repair /dev/sda3 enabling repair mode Check tree block failed, want=1015238656, have=0 Check tree block failed, want=1015238656, have=0 Check tree block failed, want=1015238656, have=0 Check tree block failed, want=1015238656, have=0 Check tree block failed, want=1015238656, have=0 read block failed check_tree_block Couldn't read tree root Checking filesystem on /dev/sda3 UUID: 1af256b5-b1ad-443b-aeee-a6853e70b7e2 Critical roots corrupted, unable to fsck the FS Segmentation fault Any more hints? Ciao Ansgar ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-27 23:03 ` Ansgar Hockmann-Stolle @ 2014-10-28 1:05 ` Qu Wenruo 2014-10-28 1:40 ` Qu Wenruo 2014-11-04 22:40 ` [SOLVED] " Ansgar Hockmann-Stolle 1 sibling, 1 reply; 8+ messages in thread From: Qu Wenruo @ 2014-10-28 1:05 UTC (permalink / raw) To: Ansgar Hockmann-Stolle, linux-btrfs -------- Original Message -------- Subject: Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root From: Ansgar Hockmann-Stolle <Ansgar.Hockmann-Stolle@uni-osnabrueck.de> To: <linux-btrfs@vger.kernel.org> Date: 2014年10月28日 07:03 > Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: >> Hi! >> >> My btrfs system partition went readonly. After reboot it doesnt mount >> anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm >> on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 >> GB SSD to some USB file and tried some btrfs tools on another copy per >> loopback device. But everything failed with: >> >> kernel: BTRFS: failed to read tree root on dm-2 >> >> See http://pastebin.com/raw.php?i=dPnU6nzg. >> >> Any hints where to go from here? > > After an offlist hint (thanks Tom!) I compiled the latest btrfs-progs > 3.17 and tried some more ... > > linux:~/bin # ./btrfs --version > Btrfs v3.17 > linux:~/bin # ./btrfs-find-root /dev/sda3 > Super think's the tree root is at 1015238656, chunk root 20971520 > Well block 239718400 seems great, but generation doesn't match, > have=661931, want=663595 level 0 > Well block 239722496 seems great, but generation doesn't match, > have=661931, want=663595 level 0 > Well block 320098304 seems great, but generation doesn't match, > have=662233, want=663595 level 0 > Well block 879341568 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879345664 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879382528 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879398912 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879403008 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879423488 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879435776 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 880095232 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 881504256 seems great, but generation doesn't match, > have=663228, want=663595 level 0 > Well block 881512448 seems great, but generation doesn't match, > have=663228, want=663595 level 0 > Well block 936271872 seems great, but generation doesn't match, > have=663397, want=663595 level 0 > Well block 1004490752 seems great, but generation doesn't match, > have=663571, want=663595 level 0 > Well block 1007804416 seems great, but generation doesn't match, > have=663572, want=663595 level 0 > Well block 1012031488 seems great, but generation doesn't match, > have=663575, want=663595 level 0 > Well block 1012396032 seems great, but generation doesn't match, > have=663575, want=663595 level 0 > Well block 1012633600 seems great, but generation doesn't match, > have=663586, want=663595 level 0 > Well block 1012871168 seems great, but generation doesn't match, > have=663585, want=663595 level 0 > Well block 1015201792 seems great, but generation doesn't match, > have=663588, want=663595 level 0 > Well block 1015836672 seems great, but generation doesn't match, > have=663596, want=663595 level 1 > Well block 44132536320 seems great, but generation doesn't match, > have=658774, want=663595 level 0 > Well block 44178280448 seems great, but generation doesn't match, > have=658774, want=663595 level 0 > Well block 87443644416 seems great, but generation doesn't match, > have=661349, want=663595 level 0 > Well block 87514079232 seems great, but generation doesn't match, > have=651051, want=663595 level 0 > Well block 87517679616 seems great, but generation doesn't match, > have=661349, want=663595 level 0 > Well block 98697822208 seems great, but generation doesn't match, > have=643548, want=663595 level 0 > Well block 103285026816 seems great, but generation doesn't match, > have=661672, want=663595 level 0 > Well block 103309553664 seems great, but generation doesn't match, > have=661674, want=663595 level 0 > Well block 103523430400 seems great, but generation doesn't match, > have=661767, want=663595 level 0 > No more metdata to scan, exiting > > This line I found interesting because "have" is "want + 1": > Well block 1015836672 seems great, but generation doesn't match, > have=663596, want=663595 level 1 > > And here the tail of "btrfs rescue chunk-recover" (full output at > http://pastebin.com/raw.php?i=1D5VgDxv) > > [..] > Total Chunks: 234 > Heathy: 231 > Bad: 3 > > Orphan Block Groups: > > Orphan Device Extents: > Couldn't map the block 1015238656 > btrfs: volumes.c:1140: btrfs_num_copies: Assertion `!(ce->start > > logical || ce->start + ce->size < logical)' failed. > Aborted > First, I think the assertion could be dealt with. Second, much like the other one who encounters chunk tree corruption, the chunk-recovery did quite well and salvaged most of the chunks. However the codes is somewhat too strict to rebuild the chunk tree if there is *ANY* orphan chunk. I prefer to make chunk-rescue less strict to rebuild chunk. Maybe it would help on your case but it may takes some time. Thanks, Qu > > Sadly "btrfs check --repair" keep up refusing to do its job. > > linux:~ # btrfs check --repair /dev/sda3 > enabling repair mode > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > read block failed check_tree_block > Couldn't read tree root > Checking filesystem on /dev/sda3 > UUID: 1af256b5-b1ad-443b-aeee-a6853e70b7e2 > Critical roots corrupted, unable to fsck the FS > Segmentation fault > > Any more hints? > > Ciao > Ansgar > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-28 1:05 ` Qu Wenruo @ 2014-10-28 1:40 ` Qu Wenruo 2014-10-28 21:56 ` Ansgar Hockmann-Stolle 0 siblings, 1 reply; 8+ messages in thread From: Qu Wenruo @ 2014-10-28 1:40 UTC (permalink / raw) To: Ansgar Hockmann-Stolle, linux-btrfs -------- Original Message -------- Subject: Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root From: Qu Wenruo <quwenruo@cn.fujitsu.com> To: Ansgar Hockmann-Stolle <Ansgar.Hockmann-Stolle@uni-osnabrueck.de>, <linux-btrfs@vger.kernel.org> Date: 2014年10月28日 09:05 > > -------- Original Message -------- > Subject: Re: btrfs unmountable: read block failed check_tree_block; > Couldn't read tree root > From: Ansgar Hockmann-Stolle <Ansgar.Hockmann-Stolle@uni-osnabrueck.de> > To: <linux-btrfs@vger.kernel.org> > Date: 2014年10月28日 07:03 >> Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: >>> Hi! >>> >>> My btrfs system partition went readonly. After reboot it doesnt mount >>> anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm >>> on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole >>> 250 >>> GB SSD to some USB file and tried some btrfs tools on another copy per >>> loopback device. But everything failed with: >>> >>> kernel: BTRFS: failed to read tree root on dm-2 >>> >>> See http://pastebin.com/raw.php?i=dPnU6nzg. >>> >>> Any hints where to go from here? >> >> After an offlist hint (thanks Tom!) I compiled the latest btrfs-progs >> 3.17 and tried some more ... >> >> linux:~/bin # ./btrfs --version >> Btrfs v3.17 >> linux:~/bin # ./btrfs-find-root /dev/sda3 >> Super think's the tree root is at 1015238656, chunk root 20971520 >> Well block 239718400 seems great, but generation doesn't match, >> have=661931, want=663595 level 0 >> Well block 239722496 seems great, but generation doesn't match, >> have=661931, want=663595 level 0 >> Well block 320098304 seems great, but generation doesn't match, >> have=662233, want=663595 level 0 >> Well block 879341568 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 879345664 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 879382528 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 879398912 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 879403008 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 879423488 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 879435776 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 880095232 seems great, but generation doesn't match, >> have=663227, want=663595 level 0 >> Well block 881504256 seems great, but generation doesn't match, >> have=663228, want=663595 level 0 >> Well block 881512448 seems great, but generation doesn't match, >> have=663228, want=663595 level 0 >> Well block 936271872 seems great, but generation doesn't match, >> have=663397, want=663595 level 0 >> Well block 1004490752 seems great, but generation doesn't match, >> have=663571, want=663595 level 0 >> Well block 1007804416 seems great, but generation doesn't match, >> have=663572, want=663595 level 0 >> Well block 1012031488 seems great, but generation doesn't match, >> have=663575, want=663595 level 0 >> Well block 1012396032 seems great, but generation doesn't match, >> have=663575, want=663595 level 0 >> Well block 1012633600 seems great, but generation doesn't match, >> have=663586, want=663595 level 0 >> Well block 1012871168 seems great, but generation doesn't match, >> have=663585, want=663595 level 0 >> Well block 1015201792 seems great, but generation doesn't match, >> have=663588, want=663595 level 0 >> Well block 1015836672 seems great, but generation doesn't match, >> have=663596, want=663595 level 1 >> Well block 44132536320 seems great, but generation doesn't match, >> have=658774, want=663595 level 0 >> Well block 44178280448 seems great, but generation doesn't match, >> have=658774, want=663595 level 0 >> Well block 87443644416 seems great, but generation doesn't match, >> have=661349, want=663595 level 0 >> Well block 87514079232 seems great, but generation doesn't match, >> have=651051, want=663595 level 0 >> Well block 87517679616 seems great, but generation doesn't match, >> have=661349, want=663595 level 0 >> Well block 98697822208 seems great, but generation doesn't match, >> have=643548, want=663595 level 0 >> Well block 103285026816 seems great, but generation doesn't match, >> have=661672, want=663595 level 0 >> Well block 103309553664 seems great, but generation doesn't match, >> have=661674, want=663595 level 0 >> Well block 103523430400 seems great, but generation doesn't match, >> have=661767, want=663595 level 0 >> No more metdata to scan, exiting >> >> This line I found interesting because "have" is "want + 1": >> Well block 1015836672 seems great, but generation doesn't match, >> have=663596, want=663595 level 1 >> >> And here the tail of "btrfs rescue chunk-recover" (full output at >> http://pastebin.com/raw.php?i=1D5VgDxv) >> >> [..] >> Total Chunks: 234 >> Heathy: 231 >> Bad: 3 >> >> Orphan Block Groups: >> >> Orphan Device Extents: >> Couldn't map the block 1015238656 >> btrfs: volumes.c:1140: btrfs_num_copies: Assertion `!(ce->start > >> logical || ce->start + ce->size < logical)' failed. >> Aborted >> After looking into the 3 bad chunks, it turns out that the logical 1015238656 is covered by the bad chunk: Chunk: start = 29360128, len = 1073741824, type = 24, num_stripes = 2 Stripes list: [ 0] Stripe: devid = 1, offset = 37748736 [ 1] Stripe: devid = 1, offset = 1111490560 No block group. Device extent list: [ 0]Device extent: devid = 1, start = 1111490560, len = 1073741824, chunk offset = 29360128 [ 1]Device extent: devid = 1, start = 37748736, len = 1073741824, chunk offset = 29360128 Which means these bad chunks are in fact good chunks. However current chunk-recovery can't rebuild block group since it doesn't know how to rebuild the 'used' member. So these needed chunks can't be rebuilt. I'll try to implement the block group rebuild function, but it may take some time. Thanks, Qu > First, I think the assertion could be dealt with. > > Second, much like the other one who encounters chunk tree corruption, > the chunk-recovery did quite well and salvaged most of the chunks. > However the codes is somewhat too strict to rebuild the chunk tree > if there is *ANY* orphan chunk. > > I prefer to make chunk-rescue less strict to rebuild chunk. > Maybe it would help on your case but it may takes some time. > > Thanks, > Qu >> >> Sadly "btrfs check --repair" keep up refusing to do its job. >> >> linux:~ # btrfs check --repair /dev/sda3 >> enabling repair mode >> Check tree block failed, want=1015238656, have=0 >> Check tree block failed, want=1015238656, have=0 >> Check tree block failed, want=1015238656, have=0 >> Check tree block failed, want=1015238656, have=0 >> Check tree block failed, want=1015238656, have=0 >> read block failed check_tree_block >> Couldn't read tree root >> Checking filesystem on /dev/sda3 >> UUID: 1af256b5-b1ad-443b-aeee-a6853e70b7e2 >> Critical roots corrupted, unable to fsck the FS >> Segmentation fault >> >> Any more hints? >> >> Ciao >> Ansgar >> -- >> To unsubscribe from this list: send the line "unsubscribe >> linux-btrfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-28 1:40 ` Qu Wenruo @ 2014-10-28 21:56 ` Ansgar Hockmann-Stolle 0 siblings, 0 replies; 8+ messages in thread From: Ansgar Hockmann-Stolle @ 2014-10-28 21:56 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs Am 28.10.14 um 02:40 schrieb Qu Wenruo: > > -------- Original Message -------- > Subject: Re: btrfs unmountable: read block failed check_tree_block; > Couldn't read tree root > From: Qu Wenruo <quwenruo@cn.fujitsu.com> > To: Ansgar Hockmann-Stolle <Ansgar.Hockmann-Stolle@uni-osnabrueck.de>, > <linux-btrfs@vger.kernel.org> > Date: 2014年10月28日 09:05 >> >> -------- Original Message -------- >> Subject: Re: btrfs unmountable: read block failed check_tree_block; >> Couldn't read tree root >> From: Ansgar Hockmann-Stolle <Ansgar.Hockmann-Stolle@uni-osnabrueck.de> >> To: <linux-btrfs@vger.kernel.org> >> Date: 2014年10月28日 07:03 >>> Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: >>>> Hi! >>>> >>>> My btrfs system partition went readonly. After reboot it doesnt mount >>>> anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm >>>> on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole >>>> 250 >>>> GB SSD to some USB file and tried some btrfs tools on another copy per >>>> loopback device. But everything failed with: >>>> >>>> kernel: BTRFS: failed to read tree root on dm-2 >>>> >>>> See http://pastebin.com/raw.php?i=dPnU6nzg. >>>> >>>> Any hints where to go from here? >>> >>> After an offlist hint (thanks Tom!) I compiled the latest btrfs-progs >>> 3.17 and tried some more ... >>> >>> linux:~/bin # ./btrfs --version >>> Btrfs v3.17 >>> linux:~/bin # ./btrfs-find-root /dev/sda3 >>> Super think's the tree root is at 1015238656, chunk root 20971520 >>> Well block 239718400 seems great, but generation doesn't match, >>> have=661931, want=663595 level 0 >>> Well block 239722496 seems great, but generation doesn't match, >>> have=661931, want=663595 level 0 >>> Well block 320098304 seems great, but generation doesn't match, >>> have=662233, want=663595 level 0 >>> Well block 879341568 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 879345664 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 879382528 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 879398912 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 879403008 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 879423488 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 879435776 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 880095232 seems great, but generation doesn't match, >>> have=663227, want=663595 level 0 >>> Well block 881504256 seems great, but generation doesn't match, >>> have=663228, want=663595 level 0 >>> Well block 881512448 seems great, but generation doesn't match, >>> have=663228, want=663595 level 0 >>> Well block 936271872 seems great, but generation doesn't match, >>> have=663397, want=663595 level 0 >>> Well block 1004490752 seems great, but generation doesn't match, >>> have=663571, want=663595 level 0 >>> Well block 1007804416 seems great, but generation doesn't match, >>> have=663572, want=663595 level 0 >>> Well block 1012031488 seems great, but generation doesn't match, >>> have=663575, want=663595 level 0 >>> Well block 1012396032 seems great, but generation doesn't match, >>> have=663575, want=663595 level 0 >>> Well block 1012633600 seems great, but generation doesn't match, >>> have=663586, want=663595 level 0 >>> Well block 1012871168 seems great, but generation doesn't match, >>> have=663585, want=663595 level 0 >>> Well block 1015201792 seems great, but generation doesn't match, >>> have=663588, want=663595 level 0 >>> Well block 1015836672 seems great, but generation doesn't match, >>> have=663596, want=663595 level 1 >>> Well block 44132536320 seems great, but generation doesn't match, >>> have=658774, want=663595 level 0 >>> Well block 44178280448 seems great, but generation doesn't match, >>> have=658774, want=663595 level 0 >>> Well block 87443644416 seems great, but generation doesn't match, >>> have=661349, want=663595 level 0 >>> Well block 87514079232 seems great, but generation doesn't match, >>> have=651051, want=663595 level 0 >>> Well block 87517679616 seems great, but generation doesn't match, >>> have=661349, want=663595 level 0 >>> Well block 98697822208 seems great, but generation doesn't match, >>> have=643548, want=663595 level 0 >>> Well block 103285026816 seems great, but generation doesn't match, >>> have=661672, want=663595 level 0 >>> Well block 103309553664 seems great, but generation doesn't match, >>> have=661674, want=663595 level 0 >>> Well block 103523430400 seems great, but generation doesn't match, >>> have=661767, want=663595 level 0 >>> No more metdata to scan, exiting >>> >>> This line I found interesting because "have" is "want + 1": >>> Well block 1015836672 seems great, but generation doesn't match, >>> have=663596, want=663595 level 1 >>> >>> And here the tail of "btrfs rescue chunk-recover" (full output at >>> http://pastebin.com/raw.php?i=1D5VgDxv) >>> >>> [..] >>> Total Chunks: 234 >>> Heathy: 231 >>> Bad: 3 >>> >>> Orphan Block Groups: >>> >>> Orphan Device Extents: >>> Couldn't map the block 1015238656 >>> btrfs: volumes.c:1140: btrfs_num_copies: Assertion `!(ce->start > >>> logical || ce->start + ce->size < logical)' failed. >>> Aborted >>> > After looking into the 3 bad chunks, it turns out that the logical > 1015238656 is covered by the bad chunk: > > Chunk: start = 29360128, len = 1073741824, type = 24, num_stripes = 2 > Stripes list: > [ 0] Stripe: devid = 1, offset = 37748736 > [ 1] Stripe: devid = 1, offset = 1111490560 > No block group. > Device extent list: > [ 0]Device extent: devid = 1, start = 1111490560, len = > 1073741824, chunk offset = 29360128 > [ 1]Device extent: devid = 1, start = 37748736, len = > 1073741824, chunk offset = 29360128 > > Which means these bad chunks are in fact good chunks. Great news! > However current chunk-recovery can't rebuild block group > since it doesn't know how to rebuild the 'used' member. > > So these needed chunks can't be rebuilt. > > I'll try to implement the block group rebuild function, > but it may take some time. Tell me if I can help anything. Thanks, Ansgar ^ permalink raw reply [flat|nested] 8+ messages in thread
* [SOLVED] btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-27 23:03 ` Ansgar Hockmann-Stolle 2014-10-28 1:05 ` Qu Wenruo @ 2014-11-04 22:40 ` Ansgar Hockmann-Stolle 1 sibling, 0 replies; 8+ messages in thread From: Ansgar Hockmann-Stolle @ 2014-11-04 22:40 UTC (permalink / raw) To: linux-btrfs For solution see http://article.gmane.org/gmane.comp.file-systems.btrfs/39974 Am 28.10.14 um 00:03 schrieb Ansgar Hockmann-Stolle: > Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: >> Hi! >> >> My btrfs system partition went readonly. After reboot it doesnt mount >> anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm >> on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 >> GB SSD to some USB file and tried some btrfs tools on another copy per >> loopback device. But everything failed with: >> >> kernel: BTRFS: failed to read tree root on dm-2 >> >> See http://pastebin.com/raw.php?i=dPnU6nzg. >> >> Any hints where to go from here? > > After an offlist hint (thanks Tom!) I compiled the latest btrfs-progs > 3.17 and tried some more ... > > linux:~/bin # ./btrfs --version > Btrfs v3.17 > linux:~/bin # ./btrfs-find-root /dev/sda3 > Super think's the tree root is at 1015238656, chunk root 20971520 > Well block 239718400 seems great, but generation doesn't match, > have=661931, want=663595 level 0 > Well block 239722496 seems great, but generation doesn't match, > have=661931, want=663595 level 0 > Well block 320098304 seems great, but generation doesn't match, > have=662233, want=663595 level 0 > Well block 879341568 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879345664 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879382528 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879398912 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879403008 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879423488 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 879435776 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 880095232 seems great, but generation doesn't match, > have=663227, want=663595 level 0 > Well block 881504256 seems great, but generation doesn't match, > have=663228, want=663595 level 0 > Well block 881512448 seems great, but generation doesn't match, > have=663228, want=663595 level 0 > Well block 936271872 seems great, but generation doesn't match, > have=663397, want=663595 level 0 > Well block 1004490752 seems great, but generation doesn't match, > have=663571, want=663595 level 0 > Well block 1007804416 seems great, but generation doesn't match, > have=663572, want=663595 level 0 > Well block 1012031488 seems great, but generation doesn't match, > have=663575, want=663595 level 0 > Well block 1012396032 seems great, but generation doesn't match, > have=663575, want=663595 level 0 > Well block 1012633600 seems great, but generation doesn't match, > have=663586, want=663595 level 0 > Well block 1012871168 seems great, but generation doesn't match, > have=663585, want=663595 level 0 > Well block 1015201792 seems great, but generation doesn't match, > have=663588, want=663595 level 0 > Well block 1015836672 seems great, but generation doesn't match, > have=663596, want=663595 level 1 > Well block 44132536320 seems great, but generation doesn't match, > have=658774, want=663595 level 0 > Well block 44178280448 seems great, but generation doesn't match, > have=658774, want=663595 level 0 > Well block 87443644416 seems great, but generation doesn't match, > have=661349, want=663595 level 0 > Well block 87514079232 seems great, but generation doesn't match, > have=651051, want=663595 level 0 > Well block 87517679616 seems great, but generation doesn't match, > have=661349, want=663595 level 0 > Well block 98697822208 seems great, but generation doesn't match, > have=643548, want=663595 level 0 > Well block 103285026816 seems great, but generation doesn't match, > have=661672, want=663595 level 0 > Well block 103309553664 seems great, but generation doesn't match, > have=661674, want=663595 level 0 > Well block 103523430400 seems great, but generation doesn't match, > have=661767, want=663595 level 0 > No more metdata to scan, exiting > > This line I found interesting because "have" is "want + 1": > Well block 1015836672 seems great, but generation doesn't match, > have=663596, want=663595 level 1 > > And here the tail of "btrfs rescue chunk-recover" (full output at > http://pastebin.com/raw.php?i=1D5VgDxv) > > [..] > Total Chunks: 234 > Heathy: 231 > Bad: 3 > > Orphan Block Groups: > > Orphan Device Extents: > Couldn't map the block 1015238656 > btrfs: volumes.c:1140: btrfs_num_copies: Assertion `!(ce->start > > logical || ce->start + ce->size < logical)' failed. > Aborted > > > Sadly "btrfs check --repair" keep up refusing to do its job. > > linux:~ # btrfs check --repair /dev/sda3 > enabling repair mode > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > Check tree block failed, want=1015238656, have=0 > read block failed check_tree_block > Couldn't read tree root > Checking filesystem on /dev/sda3 > UUID: 1af256b5-b1ad-443b-aeee-a6853e70b7e2 > Critical roots corrupted, unable to fsck the FS > Segmentation fault > > Any more hints? > > Ciao > Ansgar -- Ansgar Hockmann-Stolle, Universität Osnabrück, Rechenzentrum Albrechtstraße 28, 49076 Osnabrück, Deutschland, Raum 31/E77B +49 541 969-2749 (fax -2470), http://www.home.uos.de/anshockm ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-27 13:23 btrfs unmountable: read block failed check_tree_block; Couldn't read tree root Ansgar Hockmann-Stolle 2014-10-27 23:03 ` Ansgar Hockmann-Stolle @ 2014-10-27 23:17 ` Duncan 2014-10-28 22:42 ` Ansgar Hockmann-Stolle 1 sibling, 1 reply; 8+ messages in thread From: Duncan @ 2014-10-27 23:17 UTC (permalink / raw) To: linux-btrfs Ansgar Hockmann-Stolle posted on Mon, 27 Oct 2014 14:23:19 +0100 as excerpted: > Hi! > > My btrfs system partition went readonly. After reboot it doesnt mount > anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm > on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 > GB SSD to some USB file and tried some btrfs tools on another copy per > loopback device. But everything failed with: > > kernel: BTRFS: failed to read tree root on dm-2 > > See http://pastebin.com/raw.php?i=dPnU6nzg. > > Any hints where to go from here? Good job posting initial problem information. =:^) A lot of folks take 2-3 rounds of request and reply before that much info is available on the problem. While others may be able to assist you in restoring that filesystem to working condition, my focus is more on recovering what can be recovered from it and doing a fresh mkfs. System partition, 250 GB, looks to be just under 231 GiB based on the total bytes from btrfs-show-super. How recent is your backup, and/or being a system partition, is it simply the distro installation, possibly without too much customization, thus easily reinstalled? IOW, if you were to call that partition a total loss and simply mkfs it, would you lose anything real valuable that's not backed up? (Of course, the standard lecture at this point is that if it's not backed up, by definition you didn't consider it valuable enough to be worth the hassle, so by definition it's not valuable and you can simply blow it away, but...) If you're in good shape in that regard, that's what I'd probably do at this point, keeping the dd image you made in case someone's interested in tracking the problem down and making btrfs handle that case. If there's important files on there that you don't have backed up, or if you have a backup but it's older than you'd like and you want to try to recover current versions of what you can (the situation I was in a few months ago), then btrfs restore is what you're interested in. Restore works on an /unmounted/ (and potentially unmountable, as here) filesystem, letting you retrieve files from it and copy them to other filesystems. It does NOT write anything to the damaged filesystem itself, so no worries about making the problem worse. There's a page on the wiki describing how to use btrfs restore along with btrfs-find-root in some detail, definitely more than is in the manpages or that I want to do here. https://btrfs.wiki.kernel.org/index.php/Restore Some useful hints that weren't originally clear to me as I used that page here: * Generation and transid are the same thing, a sequentially increasing number that updates every time the root tree is written. The generation recorded in your superblocks (from btrfs-show-super) is 663595, so the idea would be that generation/transid, falling back one to 663594 if 95 isn't usable, then 93, then... etc. The lower the number the further back in history you're going, so obviously, you want the closest to 663595 that you can get, that still gives you access to a (nearly) whole filesystem, or at least the parts of it you are interested in. * That page was written before restore's -D/--dry-run option was available. This option can be quite helpful, and I recommend using it to see what will actually be restored at each generation and associated tree root (bytenr/byte-number). Tho (with -v/verbose) the list of files restored will normally be too long to go thru in detail, you can either scan it or pipe the output to wc -l to get a general idea of how many files would be restored. * Restore's -l/list-tree-roots option isn't listed on the page either. btrfs restore -l -t <bytenr> can be quite useful, giving you a nice list of trees available for the generation corresponding to that bytenr (as found using btrfs-find-root). This is where the page's advice to pick the latest tree root with all or as many as possible of the filesystem trees in it, comes in, since this lets you easily see which trees each root has available. * I don't use snapshots or subvolumes here, while I understand OpenSuSE uses them rather heavily (via snapper). Thus I have no direct experience with restore's snapshot-related options. Presumably you can either ignore the snapshots (the apparent default) or restore them either in general (using -s) or selectively (using -r, with the appropriate snapshot rootid). * It's worth noting that restore simply lets you retrieve files. It does *NOT* retrieve file ownership or permissions, with the restored files all being owned by the user you ran btrfs restore under (presumably root), with $UMASK permissions. You'll have to restore ownership and permissions manually. When I used restore here I had a backup, but the backup was old. So I hacked up a bash scriptlet with a for loop, that went thru all the restored files recursively, comparing them against the old backup. If the file existed in the old backup as well, the scriptlet did a chown and a chmod using the --reference option, setting the new file's ownership and permissions to that of the file in the backup. That took care of most files, but of course anything created since that (old) backup was still left as root owned with default UMASK perms. I then used find -user root to go thru the files again, giving me a list of those which were still root, so I could manually do the chown and chmod on them as appropriate. Of course if you don't have even an old backup, that could be quite a few files to have to update ownership and permissions for, but at least you do get the file data back! * Similarly, restore seems to flat-out ignore symlinks. It didn't restore any symlinks at all and I had to manually recreate them as necessary. * I didn't use it here, but restore's --path-regex option could be quite useful if you decide you only want to restore some subset of the files, either due to space constraints or because you have backups for the others or they simply aren't worth the trouble. You could use this for instance to restore all files in /etc to a safe location, then do a mkfs and a reinstall, before restoring your /etc files from that location, thereby recustomizing your system config. Using the -D (dry-run) and --patch-regex options together could be useful to either make sure your regex does what you expect, or to check what files in a particular location of interest within the tree might be restored, and which files aren't there and thus are likely to be lost. * I believe this is fixed in current btrfs-progs versions but when I ran restore a couple kernel cycles ago (3.15 I think, would have been progs 3.14 or 3.14.1 I believe), if there were too many files in the same subdir, restore would decide it was taking too long and would give up. I was able to overcome that by running restore repeatedly, pointing it at the same restore-to location each time, since it'll normally skip existing files. That let it restore more files each time, and eventually I didn't get any more of the taking too long errors, meaning it had restored all it could find to restore. IIRC the fix was to prompt the user whether they wanted to continue or not, with an override available to continually say yes, continue. Of course I've not used the newer version so I don't know how well that actually works, but I guess it should be easier than my having to repeatedly rerun the restore to get all the files. Hope that helps! =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root 2014-10-27 23:17 ` Duncan @ 2014-10-28 22:42 ` Ansgar Hockmann-Stolle 0 siblings, 0 replies; 8+ messages in thread From: Ansgar Hockmann-Stolle @ 2014-10-28 22:42 UTC (permalink / raw) To: linux-btrfs Duncan <1i5t5.duncan <at> cox.net> writes: [..] > > Hope that helps! =:^) > Thanks a lot for that many hints! Unfortunately, btrfs restore does not find the tree root and so it does not find anything. I will wait for Qu Wenruo to enhance chunk-recovering. And in the meantime I will test openSUSE 13.2-RC1 by installing a fresh copy of it. I have the dd dump and a working copy of it to test. And backups ... I told each friend I helped in the last years to make regular backups but me ... ;-) Thanks, Ansgar ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-11-04 22:40 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-10-27 13:23 btrfs unmountable: read block failed check_tree_block; Couldn't read tree root Ansgar Hockmann-Stolle 2014-10-27 23:03 ` Ansgar Hockmann-Stolle 2014-10-28 1:05 ` Qu Wenruo 2014-10-28 1:40 ` Qu Wenruo 2014-10-28 21:56 ` Ansgar Hockmann-Stolle 2014-11-04 22:40 ` [SOLVED] " Ansgar Hockmann-Stolle 2014-10-27 23:17 ` Duncan 2014-10-28 22:42 ` Ansgar Hockmann-Stolle
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.