If btrfs-find-root doesn't find tree roots is the filesystem lost?

If btrfs-find-root doesn't find tree roots is the filesystem lost?

[Sandy McArthur Jr]

I have a many-disk btrfs filesystem that I cannot access after a
crash. When I run btrfs-find-root I get lots of "Well block [#] seems
great [...]" lines but zero "Generation: [...]" lines.
Is this filesystem lost?


Context info below. I'll upgrade to a 3.17.4 kernel soon and try again
just in case.

mcplex src # uname -a
Linux mcplex 3.16.5-gentoo #1 SMP Mon Dec 1 22:03:39 EST 2014 x86_64
Intel(R) Core(TM) i7-2600S CPU @ 2.80GHz GenuineIntel GNU/Linux

mcplex src #   btrfs --version
Btrfs v3.17.2

mcplex src #   btrfs fi show
parent transid verify failed on 18948425080832 wanted 720141 found 720122
parent transid verify failed on 18948425080832 wanted 720141 found 720122
parent transid verify failed on 18948425080832 wanted 720141 found 720122
parent transid verify failed on 18948425080832 wanted 720141 found 720122
Ignoring transid failure
Couldn't setup extent tree
Couldn't setup device tree
Label: 'mcmedia'  uuid: 94b3345e-2589-423c-a228-d569bf94ab58
Total devices 10 FS bytes used 11.38TiB
devid    1 size 2.73TiB used 2.27TiB path /dev/sdb1
devid    2 size 2.73TiB used 2.27TiB path /dev/sdc1
devid    3 size 2.73TiB used 2.27TiB path /dev/sdd1
devid    5 size 2.73TiB used 2.27TiB path /dev/sde1
devid    6 size 2.73TiB used 2.27TiB path /dev/sdf1
devid    7 size 2.73TiB used 2.27TiB path /dev/sdg1
devid    8 size 3.64TiB used 3.18TiB path /dev/sdh1
devid    9 size 3.64TiB used 3.18TiB path /dev/sdi1
devid   10 size 3.64TiB used 1.40TiB path /dev/sdj1
devid   11 size 3.64TiB used 1.40TiB path /dev/sdk1

Btrfs v3.17.2

[Note: devid #4 isn't there because it was cleanly removed months ago.]

The relevent dmesg output from attempting to access the filesystem
[85870.630827] BTRFS info (device sdc1): enabling auto recovery
[85870.630832] BTRFS info (device sdc1): disk space caching is enabled
[85870.848351] parent transid verify failed on 18948425080832 wanted
720141 found 720122
[85870.848848] parent transid verify failed on 18948425080832 wanted
720141 found 720122
[85870.848852] BTRFS: failed to read tree root on sdc1
[85870.849365] parent transid verify failed on 18948425080832 wanted
720141 found 720122
[85870.849974] parent transid verify failed on 18948425080832 wanted
720141 found 720122
[85870.849976] BTRFS: failed to read tree root on sdc1
[85870.850602] parent transid verify failed on 18948423397376 wanted
720140 found 720113
[85870.851228] parent transid verify failed on 18948423397376 wanted
720140 found 720113
[85870.851230] BTRFS: failed to read tree root on sdc1
[85870.851857] parent transid verify failed on 18948466679808 wanted
720139 found 720091
[85870.852482] parent transid verify failed on 18948466679808 wanted
720139 found 720091
[85870.852485] BTRFS: failed to read tree root on sdc1
[85870.853120] parent transid verify failed on 18948421505024 wanted
720138 found 720122
[85870.853731] parent transid verify failed on 18948421505024 wanted
720138 found 720122
[85870.853734] BTRFS: failed to read tree root on sdc1
[85870.985258] BTRFS: open_ctree failed

mcplex src # btrfs-find-root /dev/sdh1
Super think's the tree root is at 18948425080832, chunk root 22179840
Well block 31789056 seems great, but generation doesn't match,
have=720011, want=720141 level 0
[many lines similar to the above repeated a lot but no Generation lines]

-- 
Sandy McArthur

Re: If btrfs-find-root doesn't find tree roots is the filesystem lost?

[Qu Wenruo]

-------- Original Message --------
Subject: If btrfs-find-root doesn't find tree roots is the filesystem lost?
From: Sandy McArthur Jr <sandymac@gmail.com>
To: linux-btrfs@vger.kernel.org <linux-btrfs@vger.kernel.org>
Date: 2014年12月04日 09:20
> I have a many-disk btrfs filesystem that I cannot access after a
> crash. When I run btrfs-find-root I get lots of "Well block [#] seems
> great [...]" lines but zero "Generation: [...]" lines.
> Is this filesystem lost?
>
>
> Context info below. I'll upgrade to a 3.17.4 kernel soon and try again
> just in case.
>
> mcplex src # uname -a
> Linux mcplex 3.16.5-gentoo #1 SMP Mon Dec 1 22:03:39 EST 2014 x86_64
> Intel(R) Core(TM) i7-2600S CPU @ 2.80GHz GenuineIntel GNU/Linux
>
> mcplex src #   btrfs --version
> Btrfs v3.17.2
>
> mcplex src #   btrfs fi show
> parent transid verify failed on 18948425080832 wanted 720141 found 720122
> parent transid verify failed on 18948425080832 wanted 720141 found 720122
> parent transid verify failed on 18948425080832 wanted 720141 found 720122
> parent transid verify failed on 18948425080832 wanted 720141 found 720122
> Ignoring transid failure
> Couldn't setup extent tree
> Couldn't setup device tree
> Label: 'mcmedia'  uuid: 94b3345e-2589-423c-a228-d569bf94ab58
> Total devices 10 FS bytes used 11.38TiB
> devid    1 size 2.73TiB used 2.27TiB path /dev/sdb1
> devid    2 size 2.73TiB used 2.27TiB path /dev/sdc1
> devid    3 size 2.73TiB used 2.27TiB path /dev/sdd1
> devid    5 size 2.73TiB used 2.27TiB path /dev/sde1
> devid    6 size 2.73TiB used 2.27TiB path /dev/sdf1
> devid    7 size 2.73TiB used 2.27TiB path /dev/sdg1
> devid    8 size 3.64TiB used 3.18TiB path /dev/sdh1
> devid    9 size 3.64TiB used 3.18TiB path /dev/sdi1
> devid   10 size 3.64TiB used 1.40TiB path /dev/sdj1
> devid   11 size 3.64TiB used 1.40TiB path /dev/sdk1
>
> Btrfs v3.17.2
>
> [Note: devid #4 isn't there because it was cleanly removed months ago.]
>
> The relevent dmesg output from attempting to access the filesystem
> [85870.630827] BTRFS info (device sdc1): enabling auto recovery
> [85870.630832] BTRFS info (device sdc1): disk space caching is enabled
> [85870.848351] parent transid verify failed on 18948425080832 wanted
> 720141 found 720122
> [85870.848848] parent transid verify failed on 18948425080832 wanted
> 720141 found 720122
> [85870.848852] BTRFS: failed to read tree root on sdc1
> [85870.849365] parent transid verify failed on 18948425080832 wanted
> 720141 found 720122
> [85870.849974] parent transid verify failed on 18948425080832 wanted
> 720141 found 720122
> [85870.849976] BTRFS: failed to read tree root on sdc1
> [85870.850602] parent transid verify failed on 18948423397376 wanted
> 720140 found 720113
> [85870.851228] parent transid verify failed on 18948423397376 wanted
> 720140 found 720113
> [85870.851230] BTRFS: failed to read tree root on sdc1
> [85870.851857] parent transid verify failed on 18948466679808 wanted
> 720139 found 720091
> [85870.852482] parent transid verify failed on 18948466679808 wanted
> 720139 found 720091
> [85870.852485] BTRFS: failed to read tree root on sdc1
> [85870.853120] parent transid verify failed on 18948421505024 wanted
> 720138 found 720122
> [85870.853731] parent transid verify failed on 18948421505024 wanted
> 720138 found 720122
> [85870.853734] BTRFS: failed to read tree root on sdc1
> [85870.985258] BTRFS: open_ctree failed
Only transid error,IMHO it should be recoverable.
>
> mcplex src # btrfs-find-root /dev/sdh1
> Super think's the tree root is at 18948425080832, chunk root 22179840
> Well block 31789056 seems great, but generation doesn't match,
> have=720011, want=720141 level 0
> [many lines similar to the above repeated a lot but no Generation lines]
>
Since it is a transid mismatch problem, it is common since 
btrfs-find-root can't find the root which completely
matches with the transid in btrfs super block.

In that case, btrfs-find-root is a very good help tool to find the real 
root but you need to find it by hand(in 3.17.2).
1. The root node should have the highest level
2. The higher generation, the higher chance the fs can be recovered 
using that root.

Or you can try my patch sent sometimes ago:
https://patchwork.kernel.org/patch/5285521/

With this patch, btrfs-find-root should only output the highest level 
node and sort them with generation,
which can help you find the best tree root.

After you got the possible root bytenr(something like "31789056" in your 
output), with the following patch:
https://patchwork.kernel.org/patch/5206201/

You can pass the bytenr to --tree-root option.
If btrfsck reports no or only small error, you can try btrfsck --repair 
--tree-root <bytenr> to fix it.

Thank,
Qu

Re: If btrfs-find-root doesn't find tree roots is the filesystem lost?

[Duncan]

Sandy McArthur Jr posted on Wed, 03 Dec 2014 20:20:38 -0500 as excerpted:

> mcplex src # btrfs-find-root /dev/sdh1
> Super think's the tree root is at 18948425080832, chunk root 22179840
> Well block 31789056 seems great, but generation doesn't match,
> have=720011, want=720141 level 0

> [many lines similar to the above repeated a lot but no Generation lines]

Actually the numbers in those "have 720011, want 720141 ..." lines refer 
to exactly that, generations, aka transids (the two labels are used 
interchangeably).

I take it you are referring to this on the wiki:

https://btrfs.wiki.kernel.org/index.php/Restore

That information gives you a general /idea/ of what to do, but it's 
/very/ dated at this point, and even healthy filesystems don't have the 
Generation: lines anymore, or at least I don't get them here.

Instead, use btrfs restore -l -t <bytenr> to get the tree-list.  -l gets 
you the tree-list; -t <bytenr> tells restore which bytenr/blockaddr to 
use.

Output from btrfs-find-root (repeat-quoting the above):

> Well block 31789056 seems great, but generation doesn't match,
> have=720011, want=720141 level 0

31789056 is the bytenr/blocknr.  720011 is the generation/transid of that 
root, 720141 is the generation/transid that the superblock says it wants.

On a healthy filesystem, after a bunch of "Well... but..." lines, it'll 
find the tree-root matching the one in the superblock.  Here's an example 
from a healthy filesystem without a lot of commits and thus a relatively 
low generation/transid of 41:

Super think's the tree root is at 112164864, chunk root 20971520
Well block 29360128 seems great, but generation doesn't match, have=27, 
want=41 level 0
Well block 61505536 seems great, but generation doesn't match, have=28, 
want=41 level 0
Well block 61734912 seems great, but generation doesn't match, have=29, 
want=41 level 0
Well block 61997056 seems great, but generation doesn't match, have=30, 
want=41 level 0
Well block 62308352 seems great, but generation doesn't match, have=31, 
want=41 level 0
Well block 62685184 seems great, but generation doesn't match, have=32, 
want=41 level 0
Well block 63012864 seems great, but generation doesn't match, have=33, 
want=41 level 0
Well block 67534848 seems great, but generation doesn't match, have=34, 
want=41 level 0
Well block 73875456 seems great, but generation doesn't match, have=35, 
want=41 level 0
Well block 74465280 seems great, but generation doesn't match, have=36, 
want=41 level 0
Well block 78938112 seems great, but generation doesn't match, have=37, 
want=41 level 0
Well block 111230976 seems great, but generation doesn't match, have=38, 
want=41 level 0
Well block 111575040 seems great, but generation doesn't match, have=39, 
want=41 level 0
Well block 111886336 seems great, but generation doesn't match, have=40, 
want=41 level 0
Found tree root at 112164864 gen 41 level 0

41 commits, with a history going back from 41 to 27.  See how that last 
"Found tree root" block-addr matches the one from the first "Super 
thinks" line?  That's what should happen if it's healthy.

Running btrfs restore -l without the -t will list the other trees 
available in the healthy tree root (gen 41, there's no subvolumes on this 
filesystem so all it has is the basic trees):

tree key (EXTENT_TREE ROOT_ITEM 0) 112197632 level 2
 tree key (DEV_TREE ROOT_ITEM 0) 111853568 level 0
 tree key (FS_TREE ROOT_ITEM 0) 927072256 level 2
 tree key (CSUM_TREE ROOT_ITEM 0) 112295936 level 2
 tree key (UUID_TREE ROOT_ITEM 0) 29458432 level 0
 tree key (DATA_RELOC_TREE ROOT_ITEM 0) 111837184 level 0

Now, picking the block from generation 27 just because...

btrfs restore -l -t 29360128 <device>

parent transid verify failed on 29360128 wanted 41 found 27
parent transid verify failed on 29360128 wanted 41 found 27
parent transid verify failed on 29360128 wanted 41 found 27
parent transid verify failed on 29360128 wanted 41 found 27
Ignoring transid failure
 tree key (EXTENT_TREE ROOT_ITEM 0) 29376512 level 2
 tree key (DEV_TREE ROOT_ITEM 0) 29474816 level 0
 tree key (FS_TREE ROOT_ITEM 0) 927072256 level 2
 tree key (CSUM_TREE ROOT_ITEM 0) 57573376 level 2
 tree key (UUID_TREE ROOT_ITEM 0) 29458432 level 0
 tree key (DATA_RELOC_TREE ROOT_ITEM 0) 29442048 level 0

See how the transids are all 27, but wanted 41?  That's because the 
superblock says generation/transid 41, so that's what's wanted, but I 
told restore to look at a block of a different generation, in this case 
27.


So supposing the generation 41 root got corrupted and couldn't be used.  
I could use find-root to see what else was available, then use restore -l 
with the blocks corresponding to generations going backward, 40, 39, 
38...  until I found one with what looked to be the full set of trees.

Then I'd try restore with the -D (dry run) option, feeding it the block 
corresponding to the generation/transid I had chosen, to see if it looked 
like I could restore most of what I needed.  If so, I'd run the command 
without the -D.  If not, I'd try a block corresponding to a different 
generation to see if it gave me better luck.


Additional notes on restore: 

* At least when I tried it a few userspace version cycles ago, it seemed 
to do a reasonable job at restoring file data.  However, all the restored 
files were written using the user (root) and umask I was working with, so 
file ownership and permissions (meta)data is lost (not restored).  
Additionally, no symlinks at all were restored so I had to recreate those 
manually.

* If restore runs into a large directory it'll think it's looping over 
the same thing over and over (apparently there's a filesystem error that 
will cause it to actually loop like that, thus the test), and prompt 
whether you want to continue or skip to the next directory.  Check and 
see that it's actually writing new files and not actually looping, and 
answer yes, or you'll only get a few of the files in the larger dirs.

Back when I ran it, it would just skip the rest of the directory without 
asking, and I had to run it over and over, pointing it at the same place 
to restore to each time (without overwrite enabled), so it would do a few 
more files each time, until it quit giving me the warning.  I haven't 
actually had to do it with the new prompting code, so I don't know 
exactly how it works, but you may find out. =:^)

Meanwhile, if the above works for you and you'd like, please feel free to 
update that wiki page.  For whatever reason I seem to work better on 
lists/newsgroups than on wiki pages, and I don't have an account on the 
wiki.  But it does need updated, and if you want to quote large parts of 
the above verbatim in ordered to do so I wouldn't mind at all. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: If btrfs-find-root doesn't find tree roots is the filesystem lost?

[Sandy McArthur Jr]

On Wed, Dec 3, 2014 at 10:28 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
> You can pass the bytenr to --tree-root option.
> If btrfsck reports no or only small error, you can try btrfsck --repair
> --tree-root <bytenr> to fix it.

Would the `btrfs check` output below indicate "only small error" and
that a --repair is likely to succeed? Alternatively, I can buy some
more disk try a real `btrfs restore` per Duncan's email.
Also, If I repair this filesystem should I migrate the data off of it anyway?

btrfs-progs # ./btrfs check --tree-root 18948467019776 /dev/sdb1
parent transid verify failed on 18948467019776 wanted 720141 found 720122
parent transid verify failed on 18948467019776 wanted 720141 found 720122
parent transid verify failed on 18948467019776 wanted 720141 found 720122
parent transid verify failed on 18948467019776 wanted 720141 found 720122
Ignoring transid failure
Checking filesystem on /dev/sdb1
UUID: 94b3345e-2589-423c-a228-d569bf94ab58
checking extents
checking free space cache
btrfs: space cache generation (720139) does not match inode (720096)
failed to load free space cache for block group 4324327424
btrfs: space cache generation (720141) does not match inode (720122)
failed to load free space cache for block group 5398069248
btrfs: space cache generation (720140) does not match inode (720122)
failed to load free space cache for block group 6471811072
btrfs: space cache generation (720141) does not match inode (720122)
failed to load free space cache for block group 8619294720
btrfs: space cache generation (720140) does not match inode (720122)
failed to load free space cache for block group 9693036544
btrfs: space cache generation (720141) does not match inode (720122)
failed to load free space cache for block group 10766778368
btrfs: space cache generation (720124) does not match inode (720122)
failed to load free space cache for block group 11840520192
btrfs: space cache generation (720140) does not match inode (720122)
failed to load free space cache for block group 13988003840
btrfs: space cache generation (720140) does not match inode (720109)
failed to load free space cache for block group 16135487488
btrfs: space cache generation (720139) does not match inode (720096)
failed to load free space cache for block group 15552105938944
btrfs: space cache generation (720141) does not match inode (720108)
failed to load free space cache for block group 16355264823296
btrfs: space cache generation (720124) does not match inode (720122)
failed to load free space cache for block group 18948351328256
checking fs roots
checking csums
checking root refs
Transid errors in file system
found 7236014922773 bytes used err is 1
total csum bytes: 11993511604
total tree bytes: 15886303232
total fs tree bytes: 1177964544
total extent tree bytes: 739074048
btree space waste bytes: 1543948503
file data blocks allocated: 173683188744192
 referenced 12257655894016
Btrfs v3.17.3-dirty




On Wed, Dec 3, 2014 at 10:28 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
>
> -------- Original Message --------
> Subject: If btrfs-find-root doesn't find tree roots is the filesystem lost?
> From: Sandy McArthur Jr <sandymac@gmail.com>
> To: linux-btrfs@vger.kernel.org <linux-btrfs@vger.kernel.org>
> Date: 2014年12月04日 09:20
>>
>> I have a many-disk btrfs filesystem that I cannot access after a
>> crash. When I run btrfs-find-root I get lots of "Well block [#] seems
>> great [...]" lines but zero "Generation: [...]" lines.
>> Is this filesystem lost?
>>
>>
>> Context info below. I'll upgrade to a 3.17.4 kernel soon and try again
>> just in case.
>>
>> mcplex src # uname -a
>> Linux mcplex 3.16.5-gentoo #1 SMP Mon Dec 1 22:03:39 EST 2014 x86_64
>> Intel(R) Core(TM) i7-2600S CPU @ 2.80GHz GenuineIntel GNU/Linux
>>
>> mcplex src #   btrfs --version
>> Btrfs v3.17.2
>>
>> mcplex src #   btrfs fi show
>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>> Ignoring transid failure
>> Couldn't setup extent tree
>> Couldn't setup device tree
>> Label: 'mcmedia'  uuid: 94b3345e-2589-423c-a228-d569bf94ab58
>> Total devices 10 FS bytes used 11.38TiB
>> devid    1 size 2.73TiB used 2.27TiB path /dev/sdb1
>> devid    2 size 2.73TiB used 2.27TiB path /dev/sdc1
>> devid    3 size 2.73TiB used 2.27TiB path /dev/sdd1
>> devid    5 size 2.73TiB used 2.27TiB path /dev/sde1
>> devid    6 size 2.73TiB used 2.27TiB path /dev/sdf1
>> devid    7 size 2.73TiB used 2.27TiB path /dev/sdg1
>> devid    8 size 3.64TiB used 3.18TiB path /dev/sdh1
>> devid    9 size 3.64TiB used 3.18TiB path /dev/sdi1
>> devid   10 size 3.64TiB used 1.40TiB path /dev/sdj1
>> devid   11 size 3.64TiB used 1.40TiB path /dev/sdk1
>>
>> Btrfs v3.17.2
>>
>> [Note: devid #4 isn't there because it was cleanly removed months ago.]
>>
>> The relevent dmesg output from attempting to access the filesystem
>> [85870.630827] BTRFS info (device sdc1): enabling auto recovery
>> [85870.630832] BTRFS info (device sdc1): disk space caching is enabled
>> [85870.848351] parent transid verify failed on 18948425080832 wanted
>> 720141 found 720122
>> [85870.848848] parent transid verify failed on 18948425080832 wanted
>> 720141 found 720122
>> [85870.848852] BTRFS: failed to read tree root on sdc1
>> [85870.849365] parent transid verify failed on 18948425080832 wanted
>> 720141 found 720122
>> [85870.849974] parent transid verify failed on 18948425080832 wanted
>> 720141 found 720122
>> [85870.849976] BTRFS: failed to read tree root on sdc1
>> [85870.850602] parent transid verify failed on 18948423397376 wanted
>> 720140 found 720113
>> [85870.851228] parent transid verify failed on 18948423397376 wanted
>> 720140 found 720113
>> [85870.851230] BTRFS: failed to read tree root on sdc1
>> [85870.851857] parent transid verify failed on 18948466679808 wanted
>> 720139 found 720091
>> [85870.852482] parent transid verify failed on 18948466679808 wanted
>> 720139 found 720091
>> [85870.852485] BTRFS: failed to read tree root on sdc1
>> [85870.853120] parent transid verify failed on 18948421505024 wanted
>> 720138 found 720122
>> [85870.853731] parent transid verify failed on 18948421505024 wanted
>> 720138 found 720122
>> [85870.853734] BTRFS: failed to read tree root on sdc1
>> [85870.985258] BTRFS: open_ctree failed
>
> Only transid error,IMHO it should be recoverable.
>>
>>
>> mcplex src # btrfs-find-root /dev/sdh1
>> Super think's the tree root is at 18948425080832, chunk root 22179840
>> Well block 31789056 seems great, but generation doesn't match,
>> have=720011, want=720141 level 0
>> [many lines similar to the above repeated a lot but no Generation lines]
>>
> Since it is a transid mismatch problem, it is common since btrfs-find-root
> can't find the root which completely
> matches with the transid in btrfs super block.
>
> In that case, btrfs-find-root is a very good help tool to find the real root
> but you need to find it by hand(in 3.17.2).
> 1. The root node should have the highest level
> 2. The higher generation, the higher chance the fs can be recovered using
> that root.
>
> Or you can try my patch sent sometimes ago:
> https://patchwork.kernel.org/patch/5285521/
>
> With this patch, btrfs-find-root should only output the highest level node
> and sort them with generation,
> which can help you find the best tree root.
>
> After you got the possible root bytenr(something like "31789056" in your
> output), with the following patch:
> https://patchwork.kernel.org/patch/5206201/
>
> You can pass the bytenr to --tree-root option.
> If btrfsck reports no or only small error, you can try btrfsck --repair
> --tree-root <bytenr> to fix it.
>
> Thank,
> Qu



-- 
Sandy McArthur

"He who dares not offend cannot be honest."
- Thomas Paine

Re: If btrfs-find-root doesn't find tree roots is the filesystem lost?

[Sandy McArthur Jr]

On Sun, Dec 7, 2014 at 12:50 AM, Sandy McArthur Jr <sandymac@gmail.com> wrote:
> Would the `btrfs check` output below indicate "only small error" and
> that a --repair is likely to succeed?

It did. Thanks Qu and Duncan.

dmesg suggests I have some scrubbing to do:
[220283.369150] BTRFS: bdev /dev/sdh1 errs: wr 0, rd 0, flush 0,
corrupt 5, gen 0
[220283.369159] BTRFS: bdev /dev/sdf1 errs: wr 0, rd 0, flush 0,
corrupt 5, gen 0
[220283.369163] BTRFS: bdev /dev/sdi1 errs: wr 0, rd 0, flush 0,
corrupt 5, gen 0



On Sun, Dec 7, 2014 at 12:50 AM, Sandy McArthur Jr <sandymac@gmail.com> wrote:
> On Wed, Dec 3, 2014 at 10:28 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
>> You can pass the bytenr to --tree-root option.
>> If btrfsck reports no or only small error, you can try btrfsck --repair
>> --tree-root <bytenr> to fix it.
>
> Would the `btrfs check` output below indicate "only small error" and
> that a --repair is likely to succeed? Alternatively, I can buy some
> more disk try a real `btrfs restore` per Duncan's email.
> Also, If I repair this filesystem should I migrate the data off of it anyway?
>
> btrfs-progs # ./btrfs check --tree-root 18948467019776 /dev/sdb1
> parent transid verify failed on 18948467019776 wanted 720141 found 720122
> parent transid verify failed on 18948467019776 wanted 720141 found 720122
> parent transid verify failed on 18948467019776 wanted 720141 found 720122
> parent transid verify failed on 18948467019776 wanted 720141 found 720122
> Ignoring transid failure
> Checking filesystem on /dev/sdb1
> UUID: 94b3345e-2589-423c-a228-d569bf94ab58
> checking extents
> checking free space cache
> btrfs: space cache generation (720139) does not match inode (720096)
> failed to load free space cache for block group 4324327424
> btrfs: space cache generation (720141) does not match inode (720122)
> failed to load free space cache for block group 5398069248
> btrfs: space cache generation (720140) does not match inode (720122)
> failed to load free space cache for block group 6471811072
> btrfs: space cache generation (720141) does not match inode (720122)
> failed to load free space cache for block group 8619294720
> btrfs: space cache generation (720140) does not match inode (720122)
> failed to load free space cache for block group 9693036544
> btrfs: space cache generation (720141) does not match inode (720122)
> failed to load free space cache for block group 10766778368
> btrfs: space cache generation (720124) does not match inode (720122)
> failed to load free space cache for block group 11840520192
> btrfs: space cache generation (720140) does not match inode (720122)
> failed to load free space cache for block group 13988003840
> btrfs: space cache generation (720140) does not match inode (720109)
> failed to load free space cache for block group 16135487488
> btrfs: space cache generation (720139) does not match inode (720096)
> failed to load free space cache for block group 15552105938944
> btrfs: space cache generation (720141) does not match inode (720108)
> failed to load free space cache for block group 16355264823296
> btrfs: space cache generation (720124) does not match inode (720122)
> failed to load free space cache for block group 18948351328256
> checking fs roots
> checking csums
> checking root refs
> Transid errors in file system
> found 7236014922773 bytes used err is 1
> total csum bytes: 11993511604
> total tree bytes: 15886303232
> total fs tree bytes: 1177964544
> total extent tree bytes: 739074048
> btree space waste bytes: 1543948503
> file data blocks allocated: 173683188744192
>  referenced 12257655894016
> Btrfs v3.17.3-dirty
>
>
>
>
> On Wed, Dec 3, 2014 at 10:28 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
>>
>> -------- Original Message --------
>> Subject: If btrfs-find-root doesn't find tree roots is the filesystem lost?
>> From: Sandy McArthur Jr <sandymac@gmail.com>
>> To: linux-btrfs@vger.kernel.org <linux-btrfs@vger.kernel.org>
>> Date: 2014年12月04日 09:20
>>>
>>> I have a many-disk btrfs filesystem that I cannot access after a
>>> crash. When I run btrfs-find-root I get lots of "Well block [#] seems
>>> great [...]" lines but zero "Generation: [...]" lines.
>>> Is this filesystem lost?
>>>
>>>
>>> Context info below. I'll upgrade to a 3.17.4 kernel soon and try again
>>> just in case.
>>>
>>> mcplex src # uname -a
>>> Linux mcplex 3.16.5-gentoo #1 SMP Mon Dec 1 22:03:39 EST 2014 x86_64
>>> Intel(R) Core(TM) i7-2600S CPU @ 2.80GHz GenuineIntel GNU/Linux
>>>
>>> mcplex src #   btrfs --version
>>> Btrfs v3.17.2
>>>
>>> mcplex src #   btrfs fi show
>>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>>> parent transid verify failed on 18948425080832 wanted 720141 found 720122
>>> Ignoring transid failure
>>> Couldn't setup extent tree
>>> Couldn't setup device tree
>>> Label: 'mcmedia'  uuid: 94b3345e-2589-423c-a228-d569bf94ab58
>>> Total devices 10 FS bytes used 11.38TiB
>>> devid    1 size 2.73TiB used 2.27TiB path /dev/sdb1
>>> devid    2 size 2.73TiB used 2.27TiB path /dev/sdc1
>>> devid    3 size 2.73TiB used 2.27TiB path /dev/sdd1
>>> devid    5 size 2.73TiB used 2.27TiB path /dev/sde1
>>> devid    6 size 2.73TiB used 2.27TiB path /dev/sdf1
>>> devid    7 size 2.73TiB used 2.27TiB path /dev/sdg1
>>> devid    8 size 3.64TiB used 3.18TiB path /dev/sdh1
>>> devid    9 size 3.64TiB used 3.18TiB path /dev/sdi1
>>> devid   10 size 3.64TiB used 1.40TiB path /dev/sdj1
>>> devid   11 size 3.64TiB used 1.40TiB path /dev/sdk1
>>>
>>> Btrfs v3.17.2
>>>
>>> [Note: devid #4 isn't there because it was cleanly removed months ago.]
>>>
>>> The relevent dmesg output from attempting to access the filesystem
>>> [85870.630827] BTRFS info (device sdc1): enabling auto recovery
>>> [85870.630832] BTRFS info (device sdc1): disk space caching is enabled
>>> [85870.848351] parent transid verify failed on 18948425080832 wanted
>>> 720141 found 720122
>>> [85870.848848] parent transid verify failed on 18948425080832 wanted
>>> 720141 found 720122
>>> [85870.848852] BTRFS: failed to read tree root on sdc1
>>> [85870.849365] parent transid verify failed on 18948425080832 wanted
>>> 720141 found 720122
>>> [85870.849974] parent transid verify failed on 18948425080832 wanted
>>> 720141 found 720122
>>> [85870.849976] BTRFS: failed to read tree root on sdc1
>>> [85870.850602] parent transid verify failed on 18948423397376 wanted
>>> 720140 found 720113
>>> [85870.851228] parent transid verify failed on 18948423397376 wanted
>>> 720140 found 720113
>>> [85870.851230] BTRFS: failed to read tree root on sdc1
>>> [85870.851857] parent transid verify failed on 18948466679808 wanted
>>> 720139 found 720091
>>> [85870.852482] parent transid verify failed on 18948466679808 wanted
>>> 720139 found 720091
>>> [85870.852485] BTRFS: failed to read tree root on sdc1
>>> [85870.853120] parent transid verify failed on 18948421505024 wanted
>>> 720138 found 720122
>>> [85870.853731] parent transid verify failed on 18948421505024 wanted
>>> 720138 found 720122
>>> [85870.853734] BTRFS: failed to read tree root on sdc1
>>> [85870.985258] BTRFS: open_ctree failed
>>
>> Only transid error,IMHO it should be recoverable.
>>>
>>>
>>> mcplex src # btrfs-find-root /dev/sdh1
>>> Super think's the tree root is at 18948425080832, chunk root 22179840
>>> Well block 31789056 seems great, but generation doesn't match,
>>> have=720011, want=720141 level 0
>>> [many lines similar to the above repeated a lot but no Generation lines]
>>>
>> Since it is a transid mismatch problem, it is common since btrfs-find-root
>> can't find the root which completely
>> matches with the transid in btrfs super block.
>>
>> In that case, btrfs-find-root is a very good help tool to find the real root
>> but you need to find it by hand(in 3.17.2).
>> 1. The root node should have the highest level
>> 2. The higher generation, the higher chance the fs can be recovered using
>> that root.
>>
>> Or you can try my patch sent sometimes ago:
>> https://patchwork.kernel.org/patch/5285521/
>>
>> With this patch, btrfs-find-root should only output the highest level node
>> and sort them with generation,
>> which can help you find the best tree root.
>>
>> After you got the possible root bytenr(something like "31789056" in your
>> output), with the following patch:
>> https://patchwork.kernel.org/patch/5206201/
>>
>> You can pass the bytenr to --tree-root option.
>> If btrfsck reports no or only small error, you can try btrfsck --repair
>> --tree-root <bytenr> to fix it.
>>
>> Thank,
>> Qu
>
>
>
> --
> Sandy McArthur
>
> "He who dares not offend cannot be honest."
> - Thomas Paine



-- 
Sandy McArthur

"He who dares not offend cannot be honest."
- Thomas Paine