linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* BTRFS error (device sda4): failed to read chunk tree: -5
@ 2017-08-16 22:25 Zirconium Hacker
  2017-08-16 22:52 ` Chris Murphy
  2017-08-17 20:51 ` Omar Sandoval
  0 siblings, 2 replies; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-16 22:25 UTC (permalink / raw)
  To: linux-btrfs

Hi,
This is my first time using a mailing list, and I hope I'm doing this right.

$ uname -a
Linux thinkpad 4.12.6-1-ARCH #1 SMP PREEMPT Sat Aug 12 09:16:22 CEST
2017 x86_64 GNU/Linux
$ btrfs --version
btrfs-progs v4.12
$ sudo mount -o ro,recovery /dev/sda4 /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4,
missing codepage or helper program, or other error.
$ dmesg | tail
<snip>
[ 1289.087439] BTRFS warning (device sda4): 'recovery' is deprecated,
use 'usebackuproot' instead
[ 1289.087440] BTRFS info (device sda4): trying to use backup root at mount time
[ 1289.087442] BTRFS info (device sda4): disk space caching is enabled
[ 1289.097757] BTRFS error (device sda4): failed to read chunk tree: -5
[ 1289.135222] BTRFS error (device sda4): open_ctree failed
<snip>
$ sudo btrfs check /dev/sda4
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
ERROR: cannot open file system
$ sudo btrfs restore -vvvv -D /dev/sda4 .
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
Could not open root, trying backup super
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
Could not open root, trying backup super
ERROR: superblock bytenr 274877906944 is larger than device size 58056507392
Could not open root, trying backup super

A script called btrfs-undelete
(https://gist.github.com/Changaco/45f8d171027ea2655d74) also fails
with similar errors.

I'd like to recover at least one folder, my desktop -- everything else
was backed up.
I'm using PhotoRec to try and recover some files, but I'd like a
better solution that keeps filenames and at least some folder
structure.

Thanks in advance!

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-16 22:25 BTRFS error (device sda4): failed to read chunk tree: -5 Zirconium Hacker
@ 2017-08-16 22:52 ` Chris Murphy
       [not found]   ` <CALsQ4_x13pBuZ0wDmO=28m6xAEXFjOkORJsxDeFASOeA5oDeLg@mail.gmail.com>
  2017-08-17 20:51 ` Omar Sandoval
  1 sibling, 1 reply; 24+ messages in thread
From: Chris Murphy @ 2017-08-16 22:52 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Btrfs BTRFS

On Wed, Aug 16, 2017 at 4:25 PM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:
> Hi,
> This is my first time using a mailing list, and I hope I'm doing this right.
>
> $ uname -a
> Linux thinkpad 4.12.6-1-ARCH #1 SMP PREEMPT Sat Aug 12 09:16:22 CEST
> 2017 x86_64 GNU/Linux
> $ btrfs --version
> btrfs-progs v4.12
> $ sudo mount -o ro,recovery /dev/sda4 /mnt
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4,
> missing codepage or helper program, or other error.
> $ dmesg | tail
> <snip>
> [ 1289.087439] BTRFS warning (device sda4): 'recovery' is deprecated,
> use 'usebackuproot' instead
> [ 1289.087440] BTRFS info (device sda4): trying to use backup root at mount time
> [ 1289.087442] BTRFS info (device sda4): disk space caching is enabled
> [ 1289.097757] BTRFS error (device sda4): failed to read chunk tree: -5
> [ 1289.135222] BTRFS error (device sda4): open_ctree failed
> <snip>
> $ sudo btrfs check /dev/sda4
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> ERROR: cannot open file system
> $ sudo btrfs restore -vvvv -D /dev/sda4 .
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> Could not open root, trying backup super
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> Could not open root, trying backup super
> ERROR: superblock bytenr 274877906944 is larger than device size 58056507392
> Could not open root, trying backup super
>

What happened before this?

What do you get for:

btrfs rescue super -v /dev/sda4




-- 
Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
       [not found]                   ` <CALsQ4_zTdxgfN25pkbhtXfj5EMVUbZ2=FrUHUocNxjynHhYhuA@mail.gmail.com>
@ 2017-08-17 16:53                     ` Chris Murphy
  2017-08-17 22:42                       ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Chris Murphy @ 2017-08-17 16:53 UTC (permalink / raw)
  To: Zirconium Hacker, Btrfs BTRFS; +Cc: Chris Murphy, Qu Wenruo

Readding Btrfslist, and adding Qu:


On Thu, Aug 17, 2017 at 12:48 AM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:
> Oh, sorry, I guess the output of the command I ran wasn't clear -- it
> was collecting the output of running the debug command on all 1,084
> and showing that it was the same.  Here's specifically what you asked
> for:
>
> $ sudo btrfs-debug-tree -b 61809344512 /dev/sda4
> btrfs-progs v4.12
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> ERROR: unable to open /dev/sda4
> $ sudo btrfs-debug-tree -b 1085440000 /dev/sda4
> btrfs-progs v4.12
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> ERROR: unable to open /dev/sda4
>
> I'm using GMail, and it's confusing me by trimming off quotes and
> stuff, so sorry if I miss something.
>

OK well now we're in the bad part of Btrfs repair where the error
messages don't help. It's one thing for it to complain about
1085440000 being invalid, because by now it might have been
overwritten, but to say it wants some other root that we already know
it can't read, and then fail reading that root is not helpful
information.
Maybe Qu has an idea. But it does sound like something really
catastrophic happened to blow away all of the backup root trees.

Going back to your first email, -o ro,usebackuproot failed with a
chunk tree error. I wonder if 'rescue chunk' might help.

Try 'btrfs rescue chunk -v' and see what you get.


-- 
Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-16 22:25 BTRFS error (device sda4): failed to read chunk tree: -5 Zirconium Hacker
  2017-08-16 22:52 ` Chris Murphy
@ 2017-08-17 20:51 ` Omar Sandoval
  1 sibling, 0 replies; 24+ messages in thread
From: Omar Sandoval @ 2017-08-17 20:51 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: linux-btrfs

On Wed, Aug 16, 2017 at 06:25:54PM -0400, Zirconium Hacker wrote:
> Hi,
> This is my first time using a mailing list, and I hope I'm doing this right.
> 
> $ uname -a
> Linux thinkpad 4.12.6-1-ARCH #1 SMP PREEMPT Sat Aug 12 09:16:22 CEST
> 2017 x86_64 GNU/Linux
> $ btrfs --version
> btrfs-progs v4.12
> $ sudo mount -o ro,recovery /dev/sda4 /mnt
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4,
> missing codepage or helper program, or other error.
> $ dmesg | tail
> <snip>
> [ 1289.087439] BTRFS warning (device sda4): 'recovery' is deprecated,
> use 'usebackuproot' instead
> [ 1289.087440] BTRFS info (device sda4): trying to use backup root at mount time
> [ 1289.087442] BTRFS info (device sda4): disk space caching is enabled
> [ 1289.097757] BTRFS error (device sda4): failed to read chunk tree: -5
> [ 1289.135222] BTRFS error (device sda4): open_ctree failed

Errno 5 is EIO, are there any I/O errors in your logs?

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-17 16:53                     ` Chris Murphy
@ 2017-08-17 22:42                       ` Qu Wenruo
  2017-08-18  3:13                         ` Zirconium Hacker
  2017-08-18  4:10                         ` Chris Murphy
  0 siblings, 2 replies; 24+ messages in thread
From: Qu Wenruo @ 2017-08-17 22:42 UTC (permalink / raw)
  To: Chris Murphy, Zirconium Hacker, Btrfs BTRFS



On 2017年08月18日 00:53, Chris Murphy wrote:
> Readding Btrfslist, and adding Qu:
> 
> 
> On Thu, Aug 17, 2017 at 12:48 AM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:
>> Oh, sorry, I guess the output of the command I ran wasn't clear -- it
>> was collecting the output of running the debug command on all 1,084
>> and showing that it was the same.  Here's specifically what you asked
>> for:
>>
>> $ sudo btrfs-debug-tree -b 61809344512 /dev/sda4
>> btrfs-progs v4.12
>> bytenr mismatch, want=61809344512, have=0
>> Couldn't read tree root
>> ERROR: unable to open /dev/sda4
>> $ sudo btrfs-debug-tree -b 1085440000 /dev/sda4
>> btrfs-progs v4.12
>> bytenr mismatch, want=61809344512, have=0

This means either chunk root is corrupted, or system chunk array in 
superblock is corrupted.
Bytenr mismatch is normally impossible for normal operation.

BTW are you using discard mount option? Sometimes it can cause problem.

And please also paste the following output:

# btrfs-show-super -fa /dev/sda4
# btrfs-find-root -o 5 /dev/sda4

The first is to output the full backup roots and current chunk root for 
us to debug.
The second one will try to iterate your whole disk to find a valid but 
old chunk root.
If we could find one (even a little old), it may make it possible to 
mount the fs.

Thanks,
Qu
>> Couldn't read tree root
>> ERROR: unable to open /dev/sda4
>>
>> I'm using GMail, and it's confusing me by trimming off quotes and
>> stuff, so sorry if I miss something.
>>
> 
> OK well now we're in the bad part of Btrfs repair where the error
> messages don't help. > It's one thing for it to complain about
> 1085440000 being invalid, because by now it might have been
> overwritten, but to say it wants some other root that we already know
> it can't read, and then fail reading that root is not helpful
> information.
> Maybe Qu has an idea. But it does sound like something really
> catastrophic happened to blow away all of the backup root trees.
> 
> Going back to your first email, -o ro,usebackuproot failed with a
> chunk tree error. I wonder if 'rescue chunk' might help.
> 
> Try 'btrfs rescue chunk -v' and see what you get.
> 
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-17 22:42                       ` Qu Wenruo
@ 2017-08-18  3:13                         ` Zirconium Hacker
  2017-08-18  3:49                           ` Qu Wenruo
  2017-08-18  4:10                         ` Chris Murphy
  1 sibling, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18  3:13 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

I hope "Reply All" is the right option here.  Again, first time
interacting with a mailing list.  Google said that was what to do.

I have found no I/O errors in dmesg -- at least, none mentioning
'I/O', 'IO', or anything triggered by mount besides BTRFS's
complaints.

$ sudo btrfs rescue chunk -v /dev/sda4
(See https://pastebin.com/YaRHuKeT -- the output hasn't visibly
changed since I tried this around a week ago, but this output is
recent)
$ man btrfs | grep show-super -A1
       btrfs-show-super
           moved to btrfs inspect-internal dump-super
$ sudo btrfs inspect-internal dump-super -fa /dev/sda4
(See https://pastebin.com/DbABqXGQ)
$ sudo btrfs-find-root -o 5 /dev/sda4
(See https://zerobin.net/?496ed00aed01ab0c#Kvp+FqrF6mfqQLZvUYJ1ODWYIzGayJbdyuMXc9RTauA=
   -- Pastebin wouldn't let me paste that much)

I hope the way I'm organizing the output is OK.

On Thu, Aug 17, 2017 at 6:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> On 2017年08月18日 00:53, Chris Murphy wrote:
>>
>> Readding Btrfslist, and adding Qu:
>>
>>
>> On Thu, Aug 17, 2017 at 12:48 AM, Zirconium Hacker <jared.e.vb@gmail.com>
>> wrote:
>>>
>>> Oh, sorry, I guess the output of the command I ran wasn't clear -- it
>>> was collecting the output of running the debug command on all 1,084
>>> and showing that it was the same.  Here's specifically what you asked
>>> for:
>>>
>>> $ sudo btrfs-debug-tree -b 61809344512 /dev/sda4
>>> btrfs-progs v4.12
>>> bytenr mismatch, want=61809344512, have=0
>>> Couldn't read tree root
>>> ERROR: unable to open /dev/sda4
>>> $ sudo btrfs-debug-tree -b 1085440000 /dev/sda4
>>> btrfs-progs v4.12
>>> bytenr mismatch, want=61809344512, have=0
>
>
> This means either chunk root is corrupted, or system chunk array in
> superblock is corrupted.
> Bytenr mismatch is normally impossible for normal operation.
>
> BTW are you using discard mount option? Sometimes it can cause problem.
>
> And please also paste the following output:
>
> # btrfs-show-super -fa /dev/sda4
> # btrfs-find-root -o 5 /dev/sda4
>
> The first is to output the full backup roots and current chunk root for us
> to debug.
> The second one will try to iterate your whole disk to find a valid but old
> chunk root.
> If we could find one (even a little old), it may make it possible to mount
> the fs.
>
> Thanks,
> Qu
>
>>> Couldn't read tree root
>>> ERROR: unable to open /dev/sda4
>>>
>>> I'm using GMail, and it's confusing me by trimming off quotes and
>>> stuff, so sorry if I miss something.
>>>
>>
>> OK well now we're in the bad part of Btrfs repair where the error
>> messages don't help. > It's one thing for it to complain about
>> 1085440000 being invalid, because by now it might have been
>> overwritten, but to say it wants some other root that we already know
>> it can't read, and then fail reading that root is not helpful
>> information.
>> Maybe Qu has an idea. But it does sound like something really
>> catastrophic happened to blow away all of the backup root trees.
>>
>> Going back to your first email, -o ro,usebackuproot failed with a
>> chunk tree error. I wonder if 'rescue chunk' might help.
>>
>> Try 'btrfs rescue chunk -v' and see what you get.
>>
>>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  3:13                         ` Zirconium Hacker
@ 2017-08-18  3:49                           ` Qu Wenruo
  2017-08-22  2:28                             ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18  3:49 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Chris Murphy, Btrfs BTRFS



On 2017年08月18日 11:13, Zirconium Hacker wrote:
> I hope "Reply All" is the right option here.  Again, first time
> interacting with a mailing list.  Google said that was what to do.

You're doing quite well, and yes, Reply All is the right option.

> 
> I have found no I/O errors in dmesg -- at least, none mentioning
> 'I/O', 'IO', or anything triggered by mount besides BTRFS's
> complaints.
> 
> $ sudo btrfs rescue chunk -v /dev/sda4
> (See https://pastebin.com/YaRHuKeT -- the output hasn't visibly
> changed since I tried this around a week ago, but this output is
> recent)
> $ man btrfs | grep show-super -A1
>         btrfs-show-super
>             moved to btrfs inspect-internal dump-super
> $ sudo btrfs inspect-internal dump-super -fa /dev/sda4
> (See https://pastebin.com/DbABqXGQ)

All your superblocks are saying that chunk root locates at 131072, which 
at least matches with your system chunk array.

But the problem is, your system chunk restored in superblock says that 
your system chunk is located in the very beginning of your first device.
Which is invalid as for each device, 0~1M is reserved and no chunk 
should be allocated to that range.

Quite strange to me.

Is this image a native btrfs? Or converted from ext*?

> $ sudo btrfs-find-root -o 5 /dev/sda4
> (See https://zerobin.net/?496ed00aed01ab0c#Kvp+FqrF6mfqQLZvUYJ1ODWYIzGayJbdyuMXc9RTauA=
>     -- Pastebin wouldn't let me paste that much)

I'm sorry that my previous comment has something wrong.
The magic number is not 5, but should be 3.

Please dump it again, and I think this time output will be much shorter.

Thanks,
Qu
> 
> I hope the way I'm organizing the output is OK.
> 
> On Thu, Aug 17, 2017 at 6:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>> On 2017年08月18日 00:53, Chris Murphy wrote:
>>>
>>> Readding Btrfslist, and adding Qu:
>>>
>>>
>>> On Thu, Aug 17, 2017 at 12:48 AM, Zirconium Hacker <jared.e.vb@gmail.com>
>>> wrote:
>>>>
>>>> Oh, sorry, I guess the output of the command I ran wasn't clear -- it
>>>> was collecting the output of running the debug command on all 1,084
>>>> and showing that it was the same.  Here's specifically what you asked
>>>> for:
>>>>
>>>> $ sudo btrfs-debug-tree -b 61809344512 /dev/sda4
>>>> btrfs-progs v4.12
>>>> bytenr mismatch, want=61809344512, have=0
>>>> Couldn't read tree root
>>>> ERROR: unable to open /dev/sda4
>>>> $ sudo btrfs-debug-tree -b 1085440000 /dev/sda4
>>>> btrfs-progs v4.12
>>>> bytenr mismatch, want=61809344512, have=0
>>
>>
>> This means either chunk root is corrupted, or system chunk array in
>> superblock is corrupted.
>> Bytenr mismatch is normally impossible for normal operation.
>>
>> BTW are you using discard mount option? Sometimes it can cause problem.
>>
>> And please also paste the following output:
>>
>> # btrfs-show-super -fa /dev/sda4
>> # btrfs-find-root -o 5 /dev/sda4
>>
>> The first is to output the full backup roots and current chunk root for us
>> to debug.
>> The second one will try to iterate your whole disk to find a valid but old
>> chunk root.
>> If we could find one (even a little old), it may make it possible to mount
>> the fs.
>>
>> Thanks,
>> Qu
>>
>>>> Couldn't read tree root
>>>> ERROR: unable to open /dev/sda4
>>>>
>>>> I'm using GMail, and it's confusing me by trimming off quotes and
>>>> stuff, so sorry if I miss something.
>>>>
>>>
>>> OK well now we're in the bad part of Btrfs repair where the error
>>> messages don't help. > It's one thing for it to complain about
>>> 1085440000 being invalid, because by now it might have been
>>> overwritten, but to say it wants some other root that we already know
>>> it can't read, and then fail reading that root is not helpful
>>> information.
>>> Maybe Qu has an idea. But it does sound like something really
>>> catastrophic happened to blow away all of the backup root trees.
>>>
>>> Going back to your first email, -o ro,usebackuproot failed with a
>>> chunk tree error. I wonder if 'rescue chunk' might help.
>>>
>>> Try 'btrfs rescue chunk -v' and see what you get.
>>>
>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-17 22:42                       ` Qu Wenruo
  2017-08-18  3:13                         ` Zirconium Hacker
@ 2017-08-18  4:10                         ` Chris Murphy
  2017-08-18  7:17                           ` Zirconium Hacker
  1 sibling, 1 reply; 24+ messages in thread
From: Chris Murphy @ 2017-08-18  4:10 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Zirconium Hacker, Btrfs BTRFS

On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:

> BTW are you using discard mount option? Sometimes it can cause problem.

OP did not say if it was using discard mount option; but did say some
time before this (I'm not sure how recent) he had used fstrim. The
firmware for this SSD model is current.


-- 
Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  4:10                         ` Chris Murphy
@ 2017-08-18  7:17                           ` Zirconium Hacker
  2017-08-18  8:15                             ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18  7:17 UTC (permalink / raw)
  To: Chris Murphy; +Cc: Qu Wenruo, Btrfs BTRFS

I checked my fstab, and my mount options for that partition are:
nodev,nosuid (so no discard).
As far as I remember, I had some issues converting from ext4 with
existing tools (I think that was on Debian so the tools were likely
older) so I did a manual conversion backup, wipe, copy files back).

$ sudo btrfs-find-root -o 3 /dev/sda4
Couldn't read tree root
Superblock thinks the generation is 311252
Superblock thinks the level is 0
ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
Found tree root at 131072 gen 311252 level 0

On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy <lists@colorremedies.com> wrote:
> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>> BTW are you using discard mount option? Sometimes it can cause problem.
>
> OP did not say if it was using discard mount option; but did say some
> time before this (I'm not sure how recent) he had used fstrim. The
> firmware for this SSD model is current.
>
>
> --
> Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  7:17                           ` Zirconium Hacker
@ 2017-08-18  8:15                             ` Qu Wenruo
  2017-08-18  8:47                               ` Zirconium Hacker
  0 siblings, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18  8:15 UTC (permalink / raw)
  To: Zirconium Hacker, Chris Murphy; +Cc: Btrfs BTRFS



On 2017年08月18日 15:17, Zirconium Hacker wrote:
> I checked my fstab, and my mount options for that partition are:
> nodev,nosuid (so no discard).
> As far as I remember, I had some issues converting from ext4 with
> existing tools (I think that was on Debian so the tools were likely
> older) so I did a manual conversion backup, wipe, copy files back).
> 
> $ sudo btrfs-find-root -o 3 /dev/sda4
> Couldn't read tree root
> Superblock thinks the generation is 311252
> Superblock thinks the level is 0
> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
> Found tree root at 131072 gen 311252 level 0

So chunk root (and since it's level 0, the whole chunk tree) seems good.

Could you please try the following command?
# btrfs-debug-tree -b 131072 /dev/sda4

I assume it may fail due to the fact that root tree is corrupted.
But maybe we are lucky?


And further investigating your super dump and the code, it's shows some 
clue, mostly related to your multi-device setup.

Your find-root output shows that, the only chunk leaf in /dev/sda4 seems 
good.
And in btrfs_read_chunk_tree(), which returned -EIO and caused the error 
message, will first search chunk root.

Since your chunk leaf is good, such search itself should not cause too 
much problem.

Then btrfs_read_chunk_tree() will try to read out each device, by 
calling read_one_dev().
Which can return -EIO if any device is missing and you're not using 
degraded mount option.

Is your 2nd device missing? If so, would you please try to mount with 
"degraded,ro" mount option?

BTW, if you didn't manually convert chunk profiles, did you first create 
btrfs on single device, and then added a new device to the btrfs?

Thanks,
Qu

> 
> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy <lists@colorremedies.com> wrote:
>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>> BTW are you using discard mount option? Sometimes it can cause problem.
>>
>> OP did not say if it was using discard mount option; but did say some
>> time before this (I'm not sure how recent) he had used fstrim. The
>> firmware for this SSD model is current.
>>
>>
>> --
>> Chris Murphy
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  8:15                             ` Qu Wenruo
@ 2017-08-18  8:47                               ` Zirconium Hacker
  2017-08-18  9:03                                 ` Qu Wenruo
  2017-08-18 15:00                                 ` Chris Murphy
  0 siblings, 2 replies; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18  8:47 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

$ sudo btrfs-debug-tree -b 131072 /dev/sda4
btrfs-progs v4.12
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
ERROR: unable to open /dev/sda4

Mounting with degraded,ro does not fix the multi-device issue.  The
system was never really intended to have a second device, though:

$ sudo btrfs fi show /dev/sda4
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
Total devices 2 FS bytes used 49.52GiB
devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
*** Some devices missing

I vaguely remember following this guide at some point:
http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
-- specifically the "Balance cannot run because the filesystem is
full" part.  This may have broken some things?

On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>
>> I checked my fstab, and my mount options for that partition are:
>> nodev,nosuid (so no discard).
>> As far as I remember, I had some issues converting from ext4 with
>> existing tools (I think that was on Debian so the tools were likely
>> older) so I did a manual conversion backup, wipe, copy files back).
>>
>> $ sudo btrfs-find-root -o 3 /dev/sda4
>> Couldn't read tree root
>> Superblock thinks the generation is 311252
>> Superblock thinks the level is 0
>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>> Found tree root at 131072 gen 311252 level 0
>
>
> So chunk root (and since it's level 0, the whole chunk tree) seems good.
>
> Could you please try the following command?
> # btrfs-debug-tree -b 131072 /dev/sda4
>
> I assume it may fail due to the fact that root tree is corrupted.
> But maybe we are lucky?
>
>
> And further investigating your super dump and the code, it's shows some
> clue, mostly related to your multi-device setup.
>
> Your find-root output shows that, the only chunk leaf in /dev/sda4 seems
> good.
> And in btrfs_read_chunk_tree(), which returned -EIO and caused the error
> message, will first search chunk root.
>
> Since your chunk leaf is good, such search itself should not cause too much
> problem.
>
> Then btrfs_read_chunk_tree() will try to read out each device, by calling
> read_one_dev().
> Which can return -EIO if any device is missing and you're not using degraded
> mount option.
>
> Is your 2nd device missing? If so, would you please try to mount with
> "degraded,ro" mount option?
>
> BTW, if you didn't manually convert chunk profiles, did you first create
> btrfs on single device, and then added a new device to the btrfs?
>
> Thanks,
> Qu
>
>>
>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy <lists@colorremedies.com>
>> wrote:
>>>
>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>> wrote:
>>>
>>>> BTW are you using discard mount option? Sometimes it can cause problem.
>>>
>>>
>>> OP did not say if it was using discard mount option; but did say some
>>> time before this (I'm not sure how recent) he had used fstrim. The
>>> firmware for this SSD model is current.
>>>
>>>
>>> --
>>> Chris Murphy
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  8:47                               ` Zirconium Hacker
@ 2017-08-18  9:03                                 ` Qu Wenruo
  2017-08-18  9:08                                   ` Zirconium Hacker
  2017-08-18 15:00                                 ` Chris Murphy
  1 sibling, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18  9:03 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Chris Murphy, Btrfs BTRFS



On 2017年08月18日 16:47, Zirconium Hacker wrote:
> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
> btrfs-progs v4.12
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> ERROR: unable to open /dev/sda4

I think this can be improved for case like this.
I'll try to submit a patch to enhance btrfs-debug-tree.

Would you please try "btrfs-find-root /dev/sda4"?
This will try to locate on-disk old tree root, and if we're lucky, old 
tree root can allow us to mount the fs.

> 
> Mounting with degraded,ro does not fix the multi-device issue.  The
> system was never really intended to have a second device, though:

Wait for a minute, did you mean this btrfs doesn't ever have a second 
device?
This seems quite weird now.

> 
> $ sudo btrfs fi show /dev/sda4
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
> Total devices 2 FS bytes used 49.52GiB
> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
> *** Some devices missing
> 
> I vaguely remember following this guide at some point:
> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
> -- specifically the "Balance cannot run because the filesystem is
> full" part.  This may have broken some things?

Not sure, at least from your superblock, too many things are in doubt.
 From the number of devices, to strange system chunk.

Thanks,
Qu
> 
> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>
>>> I checked my fstab, and my mount options for that partition are:
>>> nodev,nosuid (so no discard).
>>> As far as I remember, I had some issues converting from ext4 with
>>> existing tools (I think that was on Debian so the tools were likely
>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>
>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>> Couldn't read tree root
>>> Superblock thinks the generation is 311252
>>> Superblock thinks the level is 0
>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>> Found tree root at 131072 gen 311252 level 0
>>
>>
>> So chunk root (and since it's level 0, the whole chunk tree) seems good.
>>
>> Could you please try the following command?
>> # btrfs-debug-tree -b 131072 /dev/sda4
>>
>> I assume it may fail due to the fact that root tree is corrupted.
>> But maybe we are lucky?
>>
>>
>> And further investigating your super dump and the code, it's shows some
>> clue, mostly related to your multi-device setup.
>>
>> Your find-root output shows that, the only chunk leaf in /dev/sda4 seems
>> good.
>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the error
>> message, will first search chunk root.
>>
>> Since your chunk leaf is good, such search itself should not cause too much
>> problem.
>>
>> Then btrfs_read_chunk_tree() will try to read out each device, by calling
>> read_one_dev().
>> Which can return -EIO if any device is missing and you're not using degraded
>> mount option.
>>
>> Is your 2nd device missing? If so, would you please try to mount with
>> "degraded,ro" mount option?
>>
>> BTW, if you didn't manually convert chunk profiles, did you first create
>> btrfs on single device, and then added a new device to the btrfs?
>>
>> Thanks,
>> Qu
>>
>>>
>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy <lists@colorremedies.com>
>>> wrote:
>>>>
>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>> wrote:
>>>>
>>>>> BTW are you using discard mount option? Sometimes it can cause problem.
>>>>
>>>>
>>>> OP did not say if it was using discard mount option; but did say some
>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>> firmware for this SSD model is current.
>>>>
>>>>
>>>> --
>>>> Chris Murphy
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  9:03                                 ` Qu Wenruo
@ 2017-08-18  9:08                                   ` Zirconium Hacker
  2017-08-18  9:19                                     ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18  9:08 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

I already ran that earlier, here's the pastebin: https://pastebin.com/KGB8nVRA

Running debug-tree on all 1084 of them (I guess that was unnecessary)
gave the same errors every time:
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
ERROR: unable to open /dev/sda4

On Fri, Aug 18, 2017 at 5:03 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> On 2017年08月18日 16:47, Zirconium Hacker wrote:
>>
>> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
>> btrfs-progs v4.12
>> bytenr mismatch, want=61809344512, have=0
>> Couldn't read tree root
>> ERROR: unable to open /dev/sda4
>
>
> I think this can be improved for case like this.
> I'll try to submit a patch to enhance btrfs-debug-tree.
>
> Would you please try "btrfs-find-root /dev/sda4"?
> This will try to locate on-disk old tree root, and if we're lucky, old tree
> root can allow us to mount the fs.
>
>>
>> Mounting with degraded,ro does not fix the multi-device issue.  The
>> system was never really intended to have a second device, though:
>
>
> Wait for a minute, did you mean this btrfs doesn't ever have a second
> device?
> This seems quite weird now.
>
>>
>> $ sudo btrfs fi show /dev/sda4
>> bytenr mismatch, want=61809344512, have=0
>> Couldn't read tree root
>> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
>> Total devices 2 FS bytes used 49.52GiB
>> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
>> *** Some devices missing
>>
>> I vaguely remember following this guide at some point:
>>
>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>> -- specifically the "Balance cannot run because the filesystem is
>> full" part.  This may have broken some things?
>
>
> Not sure, at least from your superblock, too many things are in doubt.
> From the number of devices, to strange system chunk.
>
>
> Thanks,
> Qu
>>
>>
>> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>
>>>
>>>
>>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>>
>>>>
>>>> I checked my fstab, and my mount options for that partition are:
>>>> nodev,nosuid (so no discard).
>>>> As far as I remember, I had some issues converting from ext4 with
>>>> existing tools (I think that was on Debian so the tools were likely
>>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>>
>>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>>> Couldn't read tree root
>>>> Superblock thinks the generation is 311252
>>>> Superblock thinks the level is 0
>>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>>> Found tree root at 131072 gen 311252 level 0
>>>
>>>
>>>
>>> So chunk root (and since it's level 0, the whole chunk tree) seems good.
>>>
>>> Could you please try the following command?
>>> # btrfs-debug-tree -b 131072 /dev/sda4
>>>
>>> I assume it may fail due to the fact that root tree is corrupted.
>>> But maybe we are lucky?
>>>
>>>
>>> And further investigating your super dump and the code, it's shows some
>>> clue, mostly related to your multi-device setup.
>>>
>>> Your find-root output shows that, the only chunk leaf in /dev/sda4 seems
>>> good.
>>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the error
>>> message, will first search chunk root.
>>>
>>> Since your chunk leaf is good, such search itself should not cause too
>>> much
>>> problem.
>>>
>>> Then btrfs_read_chunk_tree() will try to read out each device, by calling
>>> read_one_dev().
>>> Which can return -EIO if any device is missing and you're not using
>>> degraded
>>> mount option.
>>>
>>> Is your 2nd device missing? If so, would you please try to mount with
>>> "degraded,ro" mount option?
>>>
>>> BTW, if you didn't manually convert chunk profiles, did you first create
>>> btrfs on single device, and then added a new device to the btrfs?
>>>
>>> Thanks,
>>> Qu
>>>
>>>>
>>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy <lists@colorremedies.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>> wrote:
>>>>>
>>>>>> BTW are you using discard mount option? Sometimes it can cause
>>>>>> problem.
>>>>>
>>>>>
>>>>>
>>>>> OP did not say if it was using discard mount option; but did say some
>>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>>> firmware for this SSD model is current.
>>>>>
>>>>>
>>>>> --
>>>>> Chris Murphy
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>>>> in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  9:08                                   ` Zirconium Hacker
@ 2017-08-18  9:19                                     ` Qu Wenruo
  2017-08-18  9:29                                       ` Zirconium Hacker
  0 siblings, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18  9:19 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Chris Murphy, Btrfs BTRFS



On 2017年08月18日 17:08, Zirconium Hacker wrote:
> I already ran that earlier, here's the pastebin: https://pastebin.com/KGB8nVRA
> 
> Running debug-tree on all 1084 of them (I guess that was unnecessary)
> gave the same errors every time:
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> ERROR: unable to open /dev/sda4
> 

Then try using btrfs check with new root:

# btrfs check -r 1085440000 /dev/sda4

Please note that, the generation in superblock differs quite a lot with 
find-root result.
So I'm afraid it will cause quite a lot of problems.

But least, it should help btrfs check to get over "Couldn't read tree 
root" error message.

And for btrfs-debug-tree error, I'll submit a patch soon to allow it to 
be run on such heavily damaged fs.

Thanks,
Qu

> On Fri, Aug 18, 2017 at 5:03 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>> On 2017年08月18日 16:47, Zirconium Hacker wrote:
>>>
>>> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
>>> btrfs-progs v4.12
>>> bytenr mismatch, want=61809344512, have=0
>>> Couldn't read tree root
>>> ERROR: unable to open /dev/sda4
>>
>>
>> I think this can be improved for case like this.
>> I'll try to submit a patch to enhance btrfs-debug-tree.
>>
>> Would you please try "btrfs-find-root /dev/sda4"?
>> This will try to locate on-disk old tree root, and if we're lucky, old tree
>> root can allow us to mount the fs.
>>
>>>
>>> Mounting with degraded,ro does not fix the multi-device issue.  The
>>> system was never really intended to have a second device, though:
>>
>>
>> Wait for a minute, did you mean this btrfs doesn't ever have a second
>> device?
>> This seems quite weird now.
>>
>>>
>>> $ sudo btrfs fi show /dev/sda4
>>> bytenr mismatch, want=61809344512, have=0
>>> Couldn't read tree root
>>> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
>>> Total devices 2 FS bytes used 49.52GiB
>>> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
>>> *** Some devices missing
>>>
>>> I vaguely remember following this guide at some point:
>>>
>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>> -- specifically the "Balance cannot run because the filesystem is
>>> full" part.  This may have broken some things?
>>
>>
>> Not sure, at least from your superblock, too many things are in doubt.
>>  From the number of devices, to strange system chunk.
>>
>>
>> Thanks,
>> Qu
>>>
>>>
>>> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>>>
>>>>>
>>>>> I checked my fstab, and my mount options for that partition are:
>>>>> nodev,nosuid (so no discard).
>>>>> As far as I remember, I had some issues converting from ext4 with
>>>>> existing tools (I think that was on Debian so the tools were likely
>>>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>>>
>>>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>>>> Couldn't read tree root
>>>>> Superblock thinks the generation is 311252
>>>>> Superblock thinks the level is 0
>>>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>>>> Found tree root at 131072 gen 311252 level 0
>>>>
>>>>
>>>>
>>>> So chunk root (and since it's level 0, the whole chunk tree) seems good.
>>>>
>>>> Could you please try the following command?
>>>> # btrfs-debug-tree -b 131072 /dev/sda4
>>>>
>>>> I assume it may fail due to the fact that root tree is corrupted.
>>>> But maybe we are lucky?
>>>>
>>>>
>>>> And further investigating your super dump and the code, it's shows some
>>>> clue, mostly related to your multi-device setup.
>>>>
>>>> Your find-root output shows that, the only chunk leaf in /dev/sda4 seems
>>>> good.
>>>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the error
>>>> message, will first search chunk root.
>>>>
>>>> Since your chunk leaf is good, such search itself should not cause too
>>>> much
>>>> problem.
>>>>
>>>> Then btrfs_read_chunk_tree() will try to read out each device, by calling
>>>> read_one_dev().
>>>> Which can return -EIO if any device is missing and you're not using
>>>> degraded
>>>> mount option.
>>>>
>>>> Is your 2nd device missing? If so, would you please try to mount with
>>>> "degraded,ro" mount option?
>>>>
>>>> BTW, if you didn't manually convert chunk profiles, did you first create
>>>> btrfs on single device, and then added a new device to the btrfs?
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>>
>>>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy <lists@colorremedies.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>> wrote:
>>>>>>
>>>>>>> BTW are you using discard mount option? Sometimes it can cause
>>>>>>> problem.
>>>>>>
>>>>>>
>>>>>>
>>>>>> OP did not say if it was using discard mount option; but did say some
>>>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>>>> firmware for this SSD model is current.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Chris Murphy
>>>>>
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>>>>> in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>
>>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  9:19                                     ` Qu Wenruo
@ 2017-08-18  9:29                                       ` Zirconium Hacker
  2017-08-18  9:46                                         ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18  9:29 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

$ sudo btrfs check -r 1085440000 /dev/sda4
parent transid verify failed on 1085440000 wanted 325966 found 325709
parent transid verify failed on 1085440000 wanted 325966 found 325709
Ignoring transid failure
bytenr mismatch, want=61352312832, have=0
Couldn't setup device tree
ERROR: cannot open file system

On Fri, Aug 18, 2017 at 5:19 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> On 2017年08月18日 17:08, Zirconium Hacker wrote:
>>
>> I already ran that earlier, here's the pastebin:
>> https://pastebin.com/KGB8nVRA
>>
>> Running debug-tree on all 1084 of them (I guess that was unnecessary)
>> gave the same errors every time:
>> bytenr mismatch, want=61809344512, have=0
>> Couldn't read tree root
>> ERROR: unable to open /dev/sda4
>>
>
> Then try using btrfs check with new root:
>
> # btrfs check -r 1085440000 /dev/sda4
>
> Please note that, the generation in superblock differs quite a lot with
> find-root result.
> So I'm afraid it will cause quite a lot of problems.
>
> But least, it should help btrfs check to get over "Couldn't read tree root"
> error message.
>
> And for btrfs-debug-tree error, I'll submit a patch soon to allow it to be
> run on such heavily damaged fs.
>
>
> Thanks,
> Qu
>
>> On Fri, Aug 18, 2017 at 5:03 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>
>>>
>>>
>>> On 2017年08月18日 16:47, Zirconium Hacker wrote:
>>>>
>>>>
>>>> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
>>>> btrfs-progs v4.12
>>>> bytenr mismatch, want=61809344512, have=0
>>>> Couldn't read tree root
>>>> ERROR: unable to open /dev/sda4
>>>
>>>
>>>
>>> I think this can be improved for case like this.
>>> I'll try to submit a patch to enhance btrfs-debug-tree.
>>>
>>> Would you please try "btrfs-find-root /dev/sda4"?
>>> This will try to locate on-disk old tree root, and if we're lucky, old
>>> tree
>>> root can allow us to mount the fs.
>>>
>>>>
>>>> Mounting with degraded,ro does not fix the multi-device issue.  The
>>>> system was never really intended to have a second device, though:
>>>
>>>
>>>
>>> Wait for a minute, did you mean this btrfs doesn't ever have a second
>>> device?
>>> This seems quite weird now.
>>>
>>>>
>>>> $ sudo btrfs fi show /dev/sda4
>>>> bytenr mismatch, want=61809344512, have=0
>>>> Couldn't read tree root
>>>> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
>>>> Total devices 2 FS bytes used 49.52GiB
>>>> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
>>>> *** Some devices missing
>>>>
>>>> I vaguely remember following this guide at some point:
>>>>
>>>>
>>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>>> -- specifically the "Balance cannot run because the filesystem is
>>>> full" part.  This may have broken some things?
>>>
>>>
>>>
>>> Not sure, at least from your superblock, too many things are in doubt.
>>>  From the number of devices, to strange system chunk.
>>>
>>>
>>> Thanks,
>>> Qu
>>>>
>>>>
>>>>
>>>> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> I checked my fstab, and my mount options for that partition are:
>>>>>> nodev,nosuid (so no discard).
>>>>>> As far as I remember, I had some issues converting from ext4 with
>>>>>> existing tools (I think that was on Debian so the tools were likely
>>>>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>>>>
>>>>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>>>>> Couldn't read tree root
>>>>>> Superblock thinks the generation is 311252
>>>>>> Superblock thinks the level is 0
>>>>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>>>>> Found tree root at 131072 gen 311252 level 0
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> So chunk root (and since it's level 0, the whole chunk tree) seems
>>>>> good.
>>>>>
>>>>> Could you please try the following command?
>>>>> # btrfs-debug-tree -b 131072 /dev/sda4
>>>>>
>>>>> I assume it may fail due to the fact that root tree is corrupted.
>>>>> But maybe we are lucky?
>>>>>
>>>>>
>>>>> And further investigating your super dump and the code, it's shows some
>>>>> clue, mostly related to your multi-device setup.
>>>>>
>>>>> Your find-root output shows that, the only chunk leaf in /dev/sda4
>>>>> seems
>>>>> good.
>>>>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the
>>>>> error
>>>>> message, will first search chunk root.
>>>>>
>>>>> Since your chunk leaf is good, such search itself should not cause too
>>>>> much
>>>>> problem.
>>>>>
>>>>> Then btrfs_read_chunk_tree() will try to read out each device, by
>>>>> calling
>>>>> read_one_dev().
>>>>> Which can return -EIO if any device is missing and you're not using
>>>>> degraded
>>>>> mount option.
>>>>>
>>>>> Is your 2nd device missing? If so, would you please try to mount with
>>>>> "degraded,ro" mount option?
>>>>>
>>>>> BTW, if you didn't manually convert chunk profiles, did you first
>>>>> create
>>>>> btrfs on single device, and then added a new device to the btrfs?
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>>
>>>>>>
>>>>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy
>>>>>> <lists@colorremedies.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> BTW are you using discard mount option? Sometimes it can cause
>>>>>>>> problem.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> OP did not say if it was using discard mount option; but did say some
>>>>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>>>>> firmware for this SSD model is current.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Chris Murphy
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>>>>>> in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>
>>>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  9:29                                       ` Zirconium Hacker
@ 2017-08-18  9:46                                         ` Qu Wenruo
  2017-08-18 10:20                                           ` Zirconium Hacker
  0 siblings, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18  9:46 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Chris Murphy, Btrfs BTRFS

Would you please try this patch?
https://patchwork.kernel.org/patch/9908173/

This should allow btrfs-debug-tree to output tree block even tree root 
is corrupted.
You could apply it on lasted master branch (tagged as v4.12).

Then re-execute the following command (with patched btrfs-progs):
# btrfs-debug-tree -b 131072 /dev/sda4

And some new output:
# btrfs-debug-tree -b 61809344512 /dev/sda4
# btrfs-debug-tree -b 61807755264 /dev/sda4
# btrfs-debug-tree -b 61809344512 /dev/sda4

Thanks,
Qu

On 2017年08月18日 17:29, Zirconium Hacker wrote:
> $ sudo btrfs check -r 1085440000 /dev/sda4
> parent transid verify failed on 1085440000 wanted 325966 found 325709
> parent transid verify failed on 1085440000 wanted 325966 found 325709
> Ignoring transid failure
> bytenr mismatch, want=61352312832, have=0
> Couldn't setup device tree
> ERROR: cannot open file system
> 
> On Fri, Aug 18, 2017 at 5:19 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>> On 2017年08月18日 17:08, Zirconium Hacker wrote:
>>>
>>> I already ran that earlier, here's the pastebin:
>>> https://pastebin.com/KGB8nVRA
>>>
>>> Running debug-tree on all 1084 of them (I guess that was unnecessary)
>>> gave the same errors every time:
>>> bytenr mismatch, want=61809344512, have=0
>>> Couldn't read tree root
>>> ERROR: unable to open /dev/sda4
>>>
>>
>> Then try using btrfs check with new root:
>>
>> # btrfs check -r 1085440000 /dev/sda4
>>
>> Please note that, the generation in superblock differs quite a lot with
>> find-root result.
>> So I'm afraid it will cause quite a lot of problems.
>>
>> But least, it should help btrfs check to get over "Couldn't read tree root"
>> error message.
>>
>> And for btrfs-debug-tree error, I'll submit a patch soon to allow it to be
>> run on such heavily damaged fs.
>>
>>
>> Thanks,
>> Qu
>>
>>> On Fri, Aug 18, 2017 at 5:03 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2017年08月18日 16:47, Zirconium Hacker wrote:
>>>>>
>>>>>
>>>>> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
>>>>> btrfs-progs v4.12
>>>>> bytenr mismatch, want=61809344512, have=0
>>>>> Couldn't read tree root
>>>>> ERROR: unable to open /dev/sda4
>>>>
>>>>
>>>>
>>>> I think this can be improved for case like this.
>>>> I'll try to submit a patch to enhance btrfs-debug-tree.
>>>>
>>>> Would you please try "btrfs-find-root /dev/sda4"?
>>>> This will try to locate on-disk old tree root, and if we're lucky, old
>>>> tree
>>>> root can allow us to mount the fs.
>>>>
>>>>>
>>>>> Mounting with degraded,ro does not fix the multi-device issue.  The
>>>>> system was never really intended to have a second device, though:
>>>>
>>>>
>>>>
>>>> Wait for a minute, did you mean this btrfs doesn't ever have a second
>>>> device?
>>>> This seems quite weird now.
>>>>
>>>>>
>>>>> $ sudo btrfs fi show /dev/sda4
>>>>> bytenr mismatch, want=61809344512, have=0
>>>>> Couldn't read tree root
>>>>> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
>>>>> Total devices 2 FS bytes used 49.52GiB
>>>>> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
>>>>> *** Some devices missing
>>>>>
>>>>> I vaguely remember following this guide at some point:
>>>>>
>>>>>
>>>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>>>> -- specifically the "Balance cannot run because the filesystem is
>>>>> full" part.  This may have broken some things?
>>>>
>>>>
>>>>
>>>> Not sure, at least from your superblock, too many things are in doubt.
>>>>   From the number of devices, to strange system chunk.
>>>>
>>>>
>>>> Thanks,
>>>> Qu
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I checked my fstab, and my mount options for that partition are:
>>>>>>> nodev,nosuid (so no discard).
>>>>>>> As far as I remember, I had some issues converting from ext4 with
>>>>>>> existing tools (I think that was on Debian so the tools were likely
>>>>>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>>>>>
>>>>>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>>>>>> Couldn't read tree root
>>>>>>> Superblock thinks the generation is 311252
>>>>>>> Superblock thinks the level is 0
>>>>>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>>>>>> Found tree root at 131072 gen 311252 level 0
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> So chunk root (and since it's level 0, the whole chunk tree) seems
>>>>>> good.
>>>>>>
>>>>>> Could you please try the following command?
>>>>>> # btrfs-debug-tree -b 131072 /dev/sda4
>>>>>>
>>>>>> I assume it may fail due to the fact that root tree is corrupted.
>>>>>> But maybe we are lucky?
>>>>>>
>>>>>>
>>>>>> And further investigating your super dump and the code, it's shows some
>>>>>> clue, mostly related to your multi-device setup.
>>>>>>
>>>>>> Your find-root output shows that, the only chunk leaf in /dev/sda4
>>>>>> seems
>>>>>> good.
>>>>>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the
>>>>>> error
>>>>>> message, will first search chunk root.
>>>>>>
>>>>>> Since your chunk leaf is good, such search itself should not cause too
>>>>>> much
>>>>>> problem.
>>>>>>
>>>>>> Then btrfs_read_chunk_tree() will try to read out each device, by
>>>>>> calling
>>>>>> read_one_dev().
>>>>>> Which can return -EIO if any device is missing and you're not using
>>>>>> degraded
>>>>>> mount option.
>>>>>>
>>>>>> Is your 2nd device missing? If so, would you please try to mount with
>>>>>> "degraded,ro" mount option?
>>>>>>
>>>>>> BTW, if you didn't manually convert chunk profiles, did you first
>>>>>> create
>>>>>> btrfs on single device, and then added a new device to the btrfs?
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy
>>>>>>> <lists@colorremedies.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> BTW are you using discard mount option? Sometimes it can cause
>>>>>>>>> problem.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> OP did not say if it was using discard mount option; but did say some
>>>>>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>>>>>> firmware for this SSD model is current.
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Chris Murphy
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>>>>>>> in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>>
>>>>>>
>>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  9:46                                         ` Qu Wenruo
@ 2017-08-18 10:20                                           ` Zirconium Hacker
  2017-08-18 11:32                                             ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18 10:20 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

# ./btrfs-debug-tree -b 131072 /dev/sda4
https://pastebin.com/TDa0GuqB
# ./btrfs-debug-tree -b 61809344512 /dev/sda4
btrfs-progs v4.12-dirty
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
bytenr mismatch, want=61809344512, have=0
ERROR: failed to read 61809344512
# ./btrfs-debug-tree -b 61807755264 /dev/sda4
btrfs-progs v4.12-dirty
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
bytenr mismatch, want=61807755264, have=0
ERROR: failed to read 61807755264

And that last one you wanted me to run debug-tree on was a duplicate.

Bonus:
# ./btrfs-debug-tree -b 1085440000 /dev/sda4
btrfs-progs v4.12-dirty
bytenr mismatch, want=61809344512, have=0
Couldn't read tree root
node 1085440000 level 1 items 2 free 491 generation 325709 owner 1
fs uuid 29889b3a-1c10-48e4-ad6d-21d03d06e90b
chunk uuid 33f664ec-d0bc-42f9-87f1-d2c0bbbb5046
key (EXTENT_TREE ROOT_ITEM 0) block 1085456384 (66251) gen 325709
key (286 INODE_ITEM 0) block 1085505536 (66254) gen 325709

BTW, thank you for your quick responses and help so far.

On Fri, Aug 18, 2017 at 5:46 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> Would you please try this patch?
> https://patchwork.kernel.org/patch/9908173/
>
> This should allow btrfs-debug-tree to output tree block even tree root is
> corrupted.
> You could apply it on lasted master branch (tagged as v4.12).
>
> Then re-execute the following command (with patched btrfs-progs):
> # btrfs-debug-tree -b 131072 /dev/sda4
>
> And some new output:
> # btrfs-debug-tree -b 61809344512 /dev/sda4
> # btrfs-debug-tree -b 61807755264 /dev/sda4
> # btrfs-debug-tree -b 61809344512 /dev/sda4
>
> Thanks,
> Qu
>
>
> On 2017年08月18日 17:29, Zirconium Hacker wrote:
>>
>> $ sudo btrfs check -r 1085440000 /dev/sda4
>> parent transid verify failed on 1085440000 wanted 325966 found 325709
>> parent transid verify failed on 1085440000 wanted 325966 found 325709
>> Ignoring transid failure
>> bytenr mismatch, want=61352312832, have=0
>> Couldn't setup device tree
>> ERROR: cannot open file system
>>
>> On Fri, Aug 18, 2017 at 5:19 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>
>>>
>>>
>>> On 2017年08月18日 17:08, Zirconium Hacker wrote:
>>>>
>>>>
>>>> I already ran that earlier, here's the pastebin:
>>>> https://pastebin.com/KGB8nVRA
>>>>
>>>> Running debug-tree on all 1084 of them (I guess that was unnecessary)
>>>> gave the same errors every time:
>>>> bytenr mismatch, want=61809344512, have=0
>>>> Couldn't read tree root
>>>> ERROR: unable to open /dev/sda4
>>>>
>>>
>>> Then try using btrfs check with new root:
>>>
>>> # btrfs check -r 1085440000 /dev/sda4
>>>
>>> Please note that, the generation in superblock differs quite a lot with
>>> find-root result.
>>> So I'm afraid it will cause quite a lot of problems.
>>>
>>> But least, it should help btrfs check to get over "Couldn't read tree
>>> root"
>>> error message.
>>>
>>> And for btrfs-debug-tree error, I'll submit a patch soon to allow it to
>>> be
>>> run on such heavily damaged fs.
>>>
>>>
>>> Thanks,
>>> Qu
>>>
>>>> On Fri, Aug 18, 2017 at 5:03 AM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 2017年08月18日 16:47, Zirconium Hacker wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
>>>>>> btrfs-progs v4.12
>>>>>> bytenr mismatch, want=61809344512, have=0
>>>>>> Couldn't read tree root
>>>>>> ERROR: unable to open /dev/sda4
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I think this can be improved for case like this.
>>>>> I'll try to submit a patch to enhance btrfs-debug-tree.
>>>>>
>>>>> Would you please try "btrfs-find-root /dev/sda4"?
>>>>> This will try to locate on-disk old tree root, and if we're lucky, old
>>>>> tree
>>>>> root can allow us to mount the fs.
>>>>>
>>>>>>
>>>>>> Mounting with degraded,ro does not fix the multi-device issue.  The
>>>>>> system was never really intended to have a second device, though:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Wait for a minute, did you mean this btrfs doesn't ever have a second
>>>>> device?
>>>>> This seems quite weird now.
>>>>>
>>>>>>
>>>>>> $ sudo btrfs fi show /dev/sda4
>>>>>> bytenr mismatch, want=61809344512, have=0
>>>>>> Couldn't read tree root
>>>>>> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
>>>>>> Total devices 2 FS bytes used 49.52GiB
>>>>>> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
>>>>>> *** Some devices missing
>>>>>>
>>>>>> I vaguely remember following this guide at some point:
>>>>>>
>>>>>>
>>>>>>
>>>>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>>>>> -- specifically the "Balance cannot run because the filesystem is
>>>>>> full" part.  This may have broken some things?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Not sure, at least from your superblock, too many things are in doubt.
>>>>>   From the number of devices, to strange system chunk.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I checked my fstab, and my mount options for that partition are:
>>>>>>>> nodev,nosuid (so no discard).
>>>>>>>> As far as I remember, I had some issues converting from ext4 with
>>>>>>>> existing tools (I think that was on Debian so the tools were likely
>>>>>>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>>>>>>
>>>>>>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>>>>>>> Couldn't read tree root
>>>>>>>> Superblock thinks the generation is 311252
>>>>>>>> Superblock thinks the level is 0
>>>>>>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>>>>>>> Found tree root at 131072 gen 311252 level 0
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So chunk root (and since it's level 0, the whole chunk tree) seems
>>>>>>> good.
>>>>>>>
>>>>>>> Could you please try the following command?
>>>>>>> # btrfs-debug-tree -b 131072 /dev/sda4
>>>>>>>
>>>>>>> I assume it may fail due to the fact that root tree is corrupted.
>>>>>>> But maybe we are lucky?
>>>>>>>
>>>>>>>
>>>>>>> And further investigating your super dump and the code, it's shows
>>>>>>> some
>>>>>>> clue, mostly related to your multi-device setup.
>>>>>>>
>>>>>>> Your find-root output shows that, the only chunk leaf in /dev/sda4
>>>>>>> seems
>>>>>>> good.
>>>>>>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the
>>>>>>> error
>>>>>>> message, will first search chunk root.
>>>>>>>
>>>>>>> Since your chunk leaf is good, such search itself should not cause
>>>>>>> too
>>>>>>> much
>>>>>>> problem.
>>>>>>>
>>>>>>> Then btrfs_read_chunk_tree() will try to read out each device, by
>>>>>>> calling
>>>>>>> read_one_dev().
>>>>>>> Which can return -EIO if any device is missing and you're not using
>>>>>>> degraded
>>>>>>> mount option.
>>>>>>>
>>>>>>> Is your 2nd device missing? If so, would you please try to mount with
>>>>>>> "degraded,ro" mount option?
>>>>>>>
>>>>>>> BTW, if you didn't manually convert chunk profiles, did you first
>>>>>>> create
>>>>>>> btrfs on single device, and then added a new device to the btrfs?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Qu
>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy
>>>>>>>> <lists@colorremedies.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> BTW are you using discard mount option? Sometimes it can cause
>>>>>>>>>> problem.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> OP did not say if it was using discard mount option; but did say
>>>>>>>>> some
>>>>>>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>>>>>>> firmware for this SSD model is current.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Chris Murphy
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>>> linux-btrfs"
>>>>>>>> in
>>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>>>
>>>>>>>
>>>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18 10:20                                           ` Zirconium Hacker
@ 2017-08-18 11:32                                             ` Qu Wenruo
  0 siblings, 0 replies; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18 11:32 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Chris Murphy, Btrfs BTRFS



On 2017年08月18日 18:20, Zirconium Hacker wrote:
> # ./btrfs-debug-tree -b 131072 /dev/sda4
> https://pastebin.com/TDa0GuqB

At least this output explains everything.
( although the result may not make you happy )

Check this:

     item 59 key (FIRST_CHUNK_TREE CHUNK_ITEM 62351474688) itemoff 11447 
itemsize 80
         length 1073741824 owner 2 stripe_len 65536 type DATA
         io_align 65536 io_width 65536 sector_size 4096
         num_stripes 1 sub_stripes 1
             stripe 0 devid 2 offset 1074790400
             dev_uuid 039224e8-dd3a-4e95-af1d-660a2021ac55

This means that, for logical address in range [62351474688, 
62351474688+1073741824), all these very important metadata are in your 
*2nd* device!!
( And 4G data is also on that device)

That's why all tree backup roots and tree roots read fails.
Because there is no such device for btrfs to read, and that's why all we 
get is 0.

And considering all your tree root and backup roots are in this range, 
that's to say, without finding your 2nd device (whose dev uuid is 
039224e8-dd3a-4e95-af1d-660a2021ac55) you lost all your possibility to 
recovery the fs.
(This reminds me to enhance btrfs kernel message about missing device)

Since both super block and dump-tree result point to a missing device, I 
really recommend you to double check the history of the fs.
( Maybe you added a usb device to do balance but forget to run "btrfs 
device remove"? )

BTW, to check the dev uuid, you could use the following command to get 
dev_uuid:
# btrfs inspect dump-super <dev> | grep dev_item.uuid

Good luck.

Thanks,
Qu

> # ./btrfs-debug-tree -b 61809344512 /dev/sda4
> btrfs-progs v4.12-dirty
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> bytenr mismatch, want=61809344512, have=0
> ERROR: failed to read 61809344512
> # ./btrfs-debug-tree -b 61807755264 /dev/sda4
> btrfs-progs v4.12-dirty
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> bytenr mismatch, want=61807755264, have=0
> ERROR: failed to read 61807755264
> 
> And that last one you wanted me to run debug-tree on was a duplicate.
> 
> Bonus:
> # ./btrfs-debug-tree -b 1085440000 /dev/sda4
> btrfs-progs v4.12-dirty
> bytenr mismatch, want=61809344512, have=0
> Couldn't read tree root
> node 1085440000 level 1 items 2 free 491 generation 325709 owner 1
> fs uuid 29889b3a-1c10-48e4-ad6d-21d03d06e90b
> chunk uuid 33f664ec-d0bc-42f9-87f1-d2c0bbbb5046
> key (EXTENT_TREE ROOT_ITEM 0) block 1085456384 (66251) gen 325709
> key (286 INODE_ITEM 0) block 1085505536 (66254) gen 325709
> 
> BTW, thank you for your quick responses and help so far.
> 
> On Fri, Aug 18, 2017 at 5:46 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>> Would you please try this patch?
>> https://patchwork.kernel.org/patch/9908173/
>>
>> This should allow btrfs-debug-tree to output tree block even tree root is
>> corrupted.
>> You could apply it on lasted master branch (tagged as v4.12).
>>
>> Then re-execute the following command (with patched btrfs-progs):
>> # btrfs-debug-tree -b 131072 /dev/sda4
>>
>> And some new output:
>> # btrfs-debug-tree -b 61809344512 /dev/sda4
>> # btrfs-debug-tree -b 61807755264 /dev/sda4
>> # btrfs-debug-tree -b 61809344512 /dev/sda4
>>
>> Thanks,
>> Qu
>>
>>
>> On 2017年08月18日 17:29, Zirconium Hacker wrote:
>>>
>>> $ sudo btrfs check -r 1085440000 /dev/sda4
>>> parent transid verify failed on 1085440000 wanted 325966 found 325709
>>> parent transid verify failed on 1085440000 wanted 325966 found 325709
>>> Ignoring transid failure
>>> bytenr mismatch, want=61352312832, have=0
>>> Couldn't setup device tree
>>> ERROR: cannot open file system
>>>
>>> On Fri, Aug 18, 2017 at 5:19 AM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2017年08月18日 17:08, Zirconium Hacker wrote:
>>>>>
>>>>>
>>>>> I already ran that earlier, here's the pastebin:
>>>>> https://pastebin.com/KGB8nVRA
>>>>>
>>>>> Running debug-tree on all 1084 of them (I guess that was unnecessary)
>>>>> gave the same errors every time:
>>>>> bytenr mismatch, want=61809344512, have=0
>>>>> Couldn't read tree root
>>>>> ERROR: unable to open /dev/sda4
>>>>>
>>>>
>>>> Then try using btrfs check with new root:
>>>>
>>>> # btrfs check -r 1085440000 /dev/sda4
>>>>
>>>> Please note that, the generation in superblock differs quite a lot with
>>>> find-root result.
>>>> So I'm afraid it will cause quite a lot of problems.
>>>>
>>>> But least, it should help btrfs check to get over "Couldn't read tree
>>>> root"
>>>> error message.
>>>>
>>>> And for btrfs-debug-tree error, I'll submit a patch soon to allow it to
>>>> be
>>>> run on such heavily damaged fs.
>>>>
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>> On Fri, Aug 18, 2017 at 5:03 AM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2017年08月18日 16:47, Zirconium Hacker wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> $ sudo btrfs-debug-tree -b 131072 /dev/sda4
>>>>>>> btrfs-progs v4.12
>>>>>>> bytenr mismatch, want=61809344512, have=0
>>>>>>> Couldn't read tree root
>>>>>>> ERROR: unable to open /dev/sda4
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I think this can be improved for case like this.
>>>>>> I'll try to submit a patch to enhance btrfs-debug-tree.
>>>>>>
>>>>>> Would you please try "btrfs-find-root /dev/sda4"?
>>>>>> This will try to locate on-disk old tree root, and if we're lucky, old
>>>>>> tree
>>>>>> root can allow us to mount the fs.
>>>>>>
>>>>>>>
>>>>>>> Mounting with degraded,ro does not fix the multi-device issue.  The
>>>>>>> system was never really intended to have a second device, though:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Wait for a minute, did you mean this btrfs doesn't ever have a second
>>>>>> device?
>>>>>> This seems quite weird now.
>>>>>>
>>>>>>>
>>>>>>> $ sudo btrfs fi show /dev/sda4
>>>>>>> bytenr mismatch, want=61809344512, have=0
>>>>>>> Couldn't read tree root
>>>>>>> Label: none  uuid: 29889b3a-1c10-48e4-ad6d-21d03d06e90b
>>>>>>> Total devices 2 FS bytes used 49.52GiB
>>>>>>> devid    1 size 54.07GiB used 54.07GiB path /dev/sda4
>>>>>>> *** Some devices missing
>>>>>>>
>>>>>>> I vaguely remember following this guide at some point:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>>>>>> -- specifically the "Balance cannot run because the filesystem is
>>>>>>> full" part.  This may have broken some things?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Not sure, at least from your superblock, too many things are in doubt.
>>>>>>    From the number of devices, to strange system chunk.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 18, 2017 at 4:15 AM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2017年08月18日 15:17, Zirconium Hacker wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I checked my fstab, and my mount options for that partition are:
>>>>>>>>> nodev,nosuid (so no discard).
>>>>>>>>> As far as I remember, I had some issues converting from ext4 with
>>>>>>>>> existing tools (I think that was on Debian so the tools were likely
>>>>>>>>> older) so I did a manual conversion backup, wipe, copy files back).
>>>>>>>>>
>>>>>>>>> $ sudo btrfs-find-root -o 3 /dev/sda4
>>>>>>>>> Couldn't read tree root
>>>>>>>>> Superblock thinks the generation is 311252
>>>>>>>>> Superblock thinks the level is 0
>>>>>>>>> ERROR: tree block bytenr 0 is not aligned to sectorsize 4096
>>>>>>>>> Found tree root at 131072 gen 311252 level 0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> So chunk root (and since it's level 0, the whole chunk tree) seems
>>>>>>>> good.
>>>>>>>>
>>>>>>>> Could you please try the following command?
>>>>>>>> # btrfs-debug-tree -b 131072 /dev/sda4
>>>>>>>>
>>>>>>>> I assume it may fail due to the fact that root tree is corrupted.
>>>>>>>> But maybe we are lucky?
>>>>>>>>
>>>>>>>>
>>>>>>>> And further investigating your super dump and the code, it's shows
>>>>>>>> some
>>>>>>>> clue, mostly related to your multi-device setup.
>>>>>>>>
>>>>>>>> Your find-root output shows that, the only chunk leaf in /dev/sda4
>>>>>>>> seems
>>>>>>>> good.
>>>>>>>> And in btrfs_read_chunk_tree(), which returned -EIO and caused the
>>>>>>>> error
>>>>>>>> message, will first search chunk root.
>>>>>>>>
>>>>>>>> Since your chunk leaf is good, such search itself should not cause
>>>>>>>> too
>>>>>>>> much
>>>>>>>> problem.
>>>>>>>>
>>>>>>>> Then btrfs_read_chunk_tree() will try to read out each device, by
>>>>>>>> calling
>>>>>>>> read_one_dev().
>>>>>>>> Which can return -EIO if any device is missing and you're not using
>>>>>>>> degraded
>>>>>>>> mount option.
>>>>>>>>
>>>>>>>> Is your 2nd device missing? If so, would you please try to mount with
>>>>>>>> "degraded,ro" mount option?
>>>>>>>>
>>>>>>>> BTW, if you didn't manually convert chunk profiles, did you first
>>>>>>>> create
>>>>>>>> btrfs on single device, and then added a new device to the btrfs?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Qu
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Aug 18, 2017 at 12:10 AM, Chris Murphy
>>>>>>>>> <lists@colorremedies.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Aug 17, 2017 at 4:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> BTW are you using discard mount option? Sometimes it can cause
>>>>>>>>>>> problem.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OP did not say if it was using discard mount option; but did say
>>>>>>>>>> some
>>>>>>>>>> time before this (I'm not sure how recent) he had used fstrim. The
>>>>>>>>>> firmware for this SSD model is current.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Chris Murphy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>>>> linux-btrfs"
>>>>>>>>> in
>>>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  8:47                               ` Zirconium Hacker
  2017-08-18  9:03                                 ` Qu Wenruo
@ 2017-08-18 15:00                                 ` Chris Murphy
  2017-08-18 21:52                                   ` Zirconium Hacker
  1 sibling, 1 reply; 24+ messages in thread
From: Chris Murphy @ 2017-08-18 15:00 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Qu Wenruo, Chris Murphy, Btrfs BTRFS

On Fri, Aug 18, 2017 at 2:47 AM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:

> I vaguely remember following this guide at some point:
> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
> -- specifically the "Balance cannot run because the filesystem is
> full" part.  This may have broken some things?
>

If you don't do 'btrfs device delete /dev/loop0' or if that command
does not complete, then it's possible to get into the situation you're
in.

Have you ever mounted this file system with -o degraded?

I'm going to guess the history is something like:
1. enospc
2. btrfs dev add
3. some kind of filtered balance, which only causes data block groups
to be moved to the 2nd device
4. 2nd device is physically removed without first 'btrfs dev del'

Zirco's superblock very clearly says num_devices  2, so I'd expect
normal mount to always fail unless both devices are present. Is there
some weird edge case where Btrfs might permit non-degraded mount when
only data bg's are on a 2nd device? And then trouble only happens
later when a balance is done and it goes looking for these bg's? And
then, boom!

-- 
Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18 15:00                                 ` Chris Murphy
@ 2017-08-18 21:52                                   ` Zirconium Hacker
  2017-08-18 23:21                                     ` Qu Wenruo
  0 siblings, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-18 21:52 UTC (permalink / raw)
  To: Chris Murphy; +Cc: Qu Wenruo, Btrfs BTRFS

Ok, so since it's clear now that I need that 5 GB device to be
present... I found the image file.  But how do I get BTRFS to
recognize the image as a device?  I have zero experience with
multi-device systems.  Setting it up as a loop device doesn't fix
mounting, and wipefs doesn't detect the BTRFS magic number, but
printing some of it to console shows it does have real data.  Writing
the magic number onto it (it's a copy of the original to be safe)
shows in dump-super, but all other values are zero.

I tried sending the above on my phone earlier but it was detected as a
"virus" because it contained HTML.  Whoops.

On Fri, Aug 18, 2017 at 11:00 AM, Chris Murphy <lists@colorremedies.com> wrote:
> On Fri, Aug 18, 2017 at 2:47 AM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:
>
>> I vaguely remember following this guide at some point:
>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>> -- specifically the "Balance cannot run because the filesystem is
>> full" part.  This may have broken some things?
>>
>
> If you don't do 'btrfs device delete /dev/loop0' or if that command
> does not complete, then it's possible to get into the situation you're
> in.
>
> Have you ever mounted this file system with -o degraded?
>
> I'm going to guess the history is something like:
> 1. enospc
> 2. btrfs dev add
> 3. some kind of filtered balance, which only causes data block groups
> to be moved to the 2nd device
> 4. 2nd device is physically removed without first 'btrfs dev del'
>
> Zirco's superblock very clearly says num_devices  2, so I'd expect
> normal mount to always fail unless both devices are present. Is there
> some weird edge case where Btrfs might permit non-degraded mount when
> only data bg's are on a 2nd device? And then trouble only happens
> later when a balance is done and it goes looking for these bg's? And
> then, boom!
>
> --
> Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18 21:52                                   ` Zirconium Hacker
@ 2017-08-18 23:21                                     ` Qu Wenruo
  2017-08-19  2:28                                       ` Zirconium Hacker
  0 siblings, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-08-18 23:21 UTC (permalink / raw)
  To: Zirconium Hacker, Chris Murphy; +Cc: Btrfs BTRFS



On 2017年08月19日 05:52, Zirconium Hacker wrote:
> Ok, so since it's clear now that I need that 5 GB device to be
> present... I found the image file.  But how do I get BTRFS to
> recognize the image as a device?

# losetup -f
Remember the loop*, here use /dev/loop1 as example.

# losetup /dev/loop1 <your image>
# partprobe /dev/loop1
Then you should have /dev/loop1p1

# btrfs dev rescan
If nothing wrong happened, you should be good to go.

Thanks,
Qu

>  I have zero experience with
> multi-device systems.  Setting it up as a loop device doesn't fix
> mounting, and wipefs doesn't detect the BTRFS magic number, but
> printing some of it to console shows it does have real data.  Writing
> the magic number onto it (it's a copy of the original to be safe)
> shows in dump-super, but all other values are zero.
> 
> I tried sending the above on my phone earlier but it was detected as a
> "virus" because it contained HTML.  Whoops.
> 
> On Fri, Aug 18, 2017 at 11:00 AM, Chris Murphy <lists@colorremedies.com> wrote:
>> On Fri, Aug 18, 2017 at 2:47 AM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:
>>
>>> I vaguely remember following this guide at some point:
>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>> -- specifically the "Balance cannot run because the filesystem is
>>> full" part.  This may have broken some things?
>>>
>>
>> If you don't do 'btrfs device delete /dev/loop0' or if that command
>> does not complete, then it's possible to get into the situation you're
>> in.
>>
>> Have you ever mounted this file system with -o degraded?
>>
>> I'm going to guess the history is something like:
>> 1. enospc
>> 2. btrfs dev add
>> 3. some kind of filtered balance, which only causes data block groups
>> to be moved to the 2nd device
>> 4. 2nd device is physically removed without first 'btrfs dev del'
>>
>> Zirco's superblock very clearly says num_devices  2, so I'd expect
>> normal mount to always fail unless both devices are present. Is there
>> some weird edge case where Btrfs might permit non-degraded mount when
>> only data bg's are on a 2nd device? And then trouble only happens
>> later when a balance is done and it goes looking for these bg's? And
>> then, boom!
>>
>> --
>> Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18 23:21                                     ` Qu Wenruo
@ 2017-08-19  2:28                                       ` Zirconium Hacker
  2017-08-19  3:45                                         ` Zirconium Hacker
  0 siblings, 1 reply; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-19  2:28 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

The image doesn't have a valid superblock.  I'm really confused as to
how that could've happened.

On Fri, Aug 18, 2017 at 7:21 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> On 2017年08月19日 05:52, Zirconium Hacker wrote:
>>
>> Ok, so since it's clear now that I need that 5 GB device to be
>> present... I found the image file.  But how do I get BTRFS to
>> recognize the image as a device?
>
>
> # losetup -f
> Remember the loop*, here use /dev/loop1 as example.
>
> # losetup /dev/loop1 <your image>
> # partprobe /dev/loop1
> Then you should have /dev/loop1p1
>
> # btrfs dev rescan
> If nothing wrong happened, you should be good to go.
>
> Thanks,
> Qu
>
>
>>  I have zero experience with
>> multi-device systems.  Setting it up as a loop device doesn't fix
>> mounting, and wipefs doesn't detect the BTRFS magic number, but
>> printing some of it to console shows it does have real data.  Writing
>> the magic number onto it (it's a copy of the original to be safe)
>> shows in dump-super, but all other values are zero.
>>
>> I tried sending the above on my phone earlier but it was detected as a
>> "virus" because it contained HTML.  Whoops.
>>
>> On Fri, Aug 18, 2017 at 11:00 AM, Chris Murphy <lists@colorremedies.com>
>> wrote:
>>>
>>> On Fri, Aug 18, 2017 at 2:47 AM, Zirconium Hacker <jared.e.vb@gmail.com>
>>> wrote:
>>>
>>>> I vaguely remember following this guide at some point:
>>>>
>>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>>> -- specifically the "Balance cannot run because the filesystem is
>>>> full" part.  This may have broken some things?
>>>>
>>>
>>> If you don't do 'btrfs device delete /dev/loop0' or if that command
>>> does not complete, then it's possible to get into the situation you're
>>> in.
>>>
>>> Have you ever mounted this file system with -o degraded?
>>>
>>> I'm going to guess the history is something like:
>>> 1. enospc
>>> 2. btrfs dev add
>>> 3. some kind of filtered balance, which only causes data block groups
>>> to be moved to the 2nd device
>>> 4. 2nd device is physically removed without first 'btrfs dev del'
>>>
>>> Zirco's superblock very clearly says num_devices  2, so I'd expect
>>> normal mount to always fail unless both devices are present. Is there
>>> some weird edge case where Btrfs might permit non-degraded mount when
>>> only data bg's are on a 2nd device? And then trouble only happens
>>> later when a balance is done and it goes looking for these bg's? And
>>> then, boom!
>>>
>>> --
>>> Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-19  2:28                                       ` Zirconium Hacker
@ 2017-08-19  3:45                                         ` Zirconium Hacker
  0 siblings, 0 replies; 24+ messages in thread
From: Zirconium Hacker @ 2017-08-19  3:45 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Chris Murphy, Btrfs BTRFS

THANK YOU ALL!  I just had to truncate the first 1.5 KiB of the image
file to get the offsets right (God knows why), but I could then get
the btrfs driver to recognize it, and I could MOUNT THE FILESYSTEM!
I'm going to run btrfs check on it, free some space on it, PROPERLY
remove the extra device, and then boot the system and set up both more
cloud backups and probably throw in a 500GB disk I have laying around
so I can set up snapshots.
Everyone has been very helpful, and sorry for the real issue being my
own stupidity.  :)

On Fri, Aug 18, 2017 at 10:28 PM, Zirconium Hacker <jared.e.vb@gmail.com> wrote:
> The image doesn't have a valid superblock.  I'm really confused as to
> how that could've happened.
>
> On Fri, Aug 18, 2017 at 7:21 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>> On 2017年08月19日 05:52, Zirconium Hacker wrote:
>>>
>>> Ok, so since it's clear now that I need that 5 GB device to be
>>> present... I found the image file.  But how do I get BTRFS to
>>> recognize the image as a device?
>>
>>
>> # losetup -f
>> Remember the loop*, here use /dev/loop1 as example.
>>
>> # losetup /dev/loop1 <your image>
>> # partprobe /dev/loop1
>> Then you should have /dev/loop1p1
>>
>> # btrfs dev rescan
>> If nothing wrong happened, you should be good to go.
>>
>> Thanks,
>> Qu
>>
>>
>>>  I have zero experience with
>>> multi-device systems.  Setting it up as a loop device doesn't fix
>>> mounting, and wipefs doesn't detect the BTRFS magic number, but
>>> printing some of it to console shows it does have real data.  Writing
>>> the magic number onto it (it's a copy of the original to be safe)
>>> shows in dump-super, but all other values are zero.
>>>
>>> I tried sending the above on my phone earlier but it was detected as a
>>> "virus" because it contained HTML.  Whoops.
>>>
>>> On Fri, Aug 18, 2017 at 11:00 AM, Chris Murphy <lists@colorremedies.com>
>>> wrote:
>>>>
>>>> On Fri, Aug 18, 2017 at 2:47 AM, Zirconium Hacker <jared.e.vb@gmail.com>
>>>> wrote:
>>>>
>>>>> I vaguely remember following this guide at some point:
>>>>>
>>>>> http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html
>>>>> -- specifically the "Balance cannot run because the filesystem is
>>>>> full" part.  This may have broken some things?
>>>>>
>>>>
>>>> If you don't do 'btrfs device delete /dev/loop0' or if that command
>>>> does not complete, then it's possible to get into the situation you're
>>>> in.
>>>>
>>>> Have you ever mounted this file system with -o degraded?
>>>>
>>>> I'm going to guess the history is something like:
>>>> 1. enospc
>>>> 2. btrfs dev add
>>>> 3. some kind of filtered balance, which only causes data block groups
>>>> to be moved to the 2nd device
>>>> 4. 2nd device is physically removed without first 'btrfs dev del'
>>>>
>>>> Zirco's superblock very clearly says num_devices  2, so I'd expect
>>>> normal mount to always fail unless both devices are present. Is there
>>>> some weird edge case where Btrfs might permit non-degraded mount when
>>>> only data bg's are on a 2nd device? And then trouble only happens
>>>> later when a balance is done and it goes looking for these bg's? And
>>>> then, boom!
>>>>
>>>> --
>>>> Chris Murphy

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: BTRFS error (device sda4): failed to read chunk tree: -5
  2017-08-18  3:49                           ` Qu Wenruo
@ 2017-08-22  2:28                             ` Qu Wenruo
  0 siblings, 0 replies; 24+ messages in thread
From: Qu Wenruo @ 2017-08-22  2:28 UTC (permalink / raw)
  To: Zirconium Hacker; +Cc: Chris Murphy, Btrfs BTRFS



On 2017年08月18日 11:49, Qu Wenruo wrote:
> 
> 
> On 2017年08月18日 11:13, Zirconium Hacker wrote:
>> I hope "Reply All" is the right option here.  Again, first time
>> interacting with a mailing list.  Google said that was what to do.
> 
> You're doing quite well, and yes, Reply All is the right option.
> 
>>
>> I have found no I/O errors in dmesg -- at least, none mentioning
>> 'I/O', 'IO', or anything triggered by mount besides BTRFS's
>> complaints.
>>
>> $ sudo btrfs rescue chunk -v /dev/sda4
>> (See https://pastebin.com/YaRHuKeT -- the output hasn't visibly
>> changed since I tried this around a week ago, but this output is
>> recent)
>> $ man btrfs | grep show-super -A1
>>         btrfs-show-super
>>             moved to btrfs inspect-internal dump-super
>> $ sudo btrfs inspect-internal dump-super -fa /dev/sda4
>> (See https://pastebin.com/DbABqXGQ)
> 
> All your superblocks are saying that chunk root locates at 131072, which 
> at least matches with your system chunk array.
> 
> But the problem is, your system chunk restored in superblock says that 
> your system chunk is located in the very beginning of your first device.
> Which is invalid as for each device, 0~1M is reserved and no chunk 
> should be allocated to that range.
> 
> Quite strange to me.
> 
> Is this image a native btrfs? Or converted from ext*?

Finally I get answer why there will be such strange chunk layout.

I reproduced it by making a btrfs with "-r" option.
Then it will create such strange chunk layout.

I'll try to fix that option, to make it follow the correct method to 
create file system.

Thanks,
Qu

> 
>> $ sudo btrfs-find-root -o 5 /dev/sda4
>> (See 
>> https://zerobin.net/?496ed00aed01ab0c#Kvp+FqrF6mfqQLZvUYJ1ODWYIzGayJbdyuMXc9RTauA= 
>>
>>     -- Pastebin wouldn't let me paste that much)
> 
> I'm sorry that my previous comment has something wrong.
> The magic number is not 5, but should be 3.
> 
> Please dump it again, and I think this time output will be much shorter.
> 
> Thanks,
> Qu
>>
>> I hope the way I'm organizing the output is OK.
>>
>> On Thu, Aug 17, 2017 at 6:42 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> 
>> wrote:
>>>
>>>
>>> On 2017年08月18日 00:53, Chris Murphy wrote:
>>>>
>>>> Readding Btrfslist, and adding Qu:
>>>>
>>>>
>>>> On Thu, Aug 17, 2017 at 12:48 AM, Zirconium Hacker 
>>>> <jared.e.vb@gmail.com>
>>>> wrote:
>>>>>
>>>>> Oh, sorry, I guess the output of the command I ran wasn't clear -- it
>>>>> was collecting the output of running the debug command on all 1,084
>>>>> and showing that it was the same.  Here's specifically what you asked
>>>>> for:
>>>>>
>>>>> $ sudo btrfs-debug-tree -b 61809344512 /dev/sda4
>>>>> btrfs-progs v4.12
>>>>> bytenr mismatch, want=61809344512, have=0
>>>>> Couldn't read tree root
>>>>> ERROR: unable to open /dev/sda4
>>>>> $ sudo btrfs-debug-tree -b 1085440000 /dev/sda4
>>>>> btrfs-progs v4.12
>>>>> bytenr mismatch, want=61809344512, have=0
>>>
>>>
>>> This means either chunk root is corrupted, or system chunk array in
>>> superblock is corrupted.
>>> Bytenr mismatch is normally impossible for normal operation.
>>>
>>> BTW are you using discard mount option? Sometimes it can cause problem.
>>>
>>> And please also paste the following output:
>>>
>>> # btrfs-show-super -fa /dev/sda4
>>> # btrfs-find-root -o 5 /dev/sda4
>>>
>>> The first is to output the full backup roots and current chunk root 
>>> for us
>>> to debug.
>>> The second one will try to iterate your whole disk to find a valid 
>>> but old
>>> chunk root.
>>> If we could find one (even a little old), it may make it possible to 
>>> mount
>>> the fs.
>>>
>>> Thanks,
>>> Qu
>>>
>>>>> Couldn't read tree root
>>>>> ERROR: unable to open /dev/sda4
>>>>>
>>>>> I'm using GMail, and it's confusing me by trimming off quotes and
>>>>> stuff, so sorry if I miss something.
>>>>>
>>>>
>>>> OK well now we're in the bad part of Btrfs repair where the error
>>>> messages don't help. > It's one thing for it to complain about
>>>> 1085440000 being invalid, because by now it might have been
>>>> overwritten, but to say it wants some other root that we already know
>>>> it can't read, and then fail reading that root is not helpful
>>>> information.
>>>> Maybe Qu has an idea. But it does sound like something really
>>>> catastrophic happened to blow away all of the backup root trees.
>>>>
>>>> Going back to your first email, -o ro,usebackuproot failed with a
>>>> chunk tree error. I wonder if 'rescue chunk' might help.
>>>>
>>>> Try 'btrfs rescue chunk -v' and see what you get.
>>>>
>>>>
>>>
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2017-08-22  2:28 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-08-16 22:25 BTRFS error (device sda4): failed to read chunk tree: -5 Zirconium Hacker
2017-08-16 22:52 ` Chris Murphy
     [not found]   ` <CALsQ4_x13pBuZ0wDmO=28m6xAEXFjOkORJsxDeFASOeA5oDeLg@mail.gmail.com>
     [not found]     ` <CAJCQCtRTiZy2YNZNLLx_KWuLMuNxPK26u1Orz91MqtDRFNRVNg@mail.gmail.com>
     [not found]       ` <CALsQ4_xkiyWuwJ4mVOV7QhWx6RFD80Y23fDNdPam2UFpRi1Gxw@mail.gmail.com>
     [not found]         ` <CAJCQCtTx_i_PrqgRbA-P9+zKX926KKeT1nwAJT=dkr-qcOpCCg@mail.gmail.com>
     [not found]           ` <CALsQ4_wZLvdqHdKa9DyjRXTmf1CV47+11-O_SqLRQJoQDT+gqQ@mail.gmail.com>
     [not found]             ` <CAJCQCtRYxb4QdN__WcUX_Jhs=GyguwJV79pKUp4PYhC9h_H-xA@mail.gmail.com>
     [not found]               ` <CALsQ4_xWte-vOJD-ppOPkvfHpLtu_-24KZPj2H9Tp99S7jDGcw@mail.gmail.com>
     [not found]                 ` <CAJCQCtR-+o_GA7fCfSe8Lfwp8h6FzfiSR=qz8ekc4=dy98mtiQ@mail.gmail.com>
     [not found]                   ` <CALsQ4_zTdxgfN25pkbhtXfj5EMVUbZ2=FrUHUocNxjynHhYhuA@mail.gmail.com>
2017-08-17 16:53                     ` Chris Murphy
2017-08-17 22:42                       ` Qu Wenruo
2017-08-18  3:13                         ` Zirconium Hacker
2017-08-18  3:49                           ` Qu Wenruo
2017-08-22  2:28                             ` Qu Wenruo
2017-08-18  4:10                         ` Chris Murphy
2017-08-18  7:17                           ` Zirconium Hacker
2017-08-18  8:15                             ` Qu Wenruo
2017-08-18  8:47                               ` Zirconium Hacker
2017-08-18  9:03                                 ` Qu Wenruo
2017-08-18  9:08                                   ` Zirconium Hacker
2017-08-18  9:19                                     ` Qu Wenruo
2017-08-18  9:29                                       ` Zirconium Hacker
2017-08-18  9:46                                         ` Qu Wenruo
2017-08-18 10:20                                           ` Zirconium Hacker
2017-08-18 11:32                                             ` Qu Wenruo
2017-08-18 15:00                                 ` Chris Murphy
2017-08-18 21:52                                   ` Zirconium Hacker
2017-08-18 23:21                                     ` Qu Wenruo
2017-08-19  2:28                                       ` Zirconium Hacker
2017-08-19  3:45                                         ` Zirconium Hacker
2017-08-17 20:51 ` Omar Sandoval

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).