Question about a specific error.

Question about a specific error.

[Austin S. Hemmelgarn]

In the process of trying to debug issues I'm having on one of my
systems with a new kernel version, I decided to do a dry run check on
the root filesystem.  'btrfs check' returned a bunch of lines like:

root 257 inode XXXXXX errors 2000, link count wrong
        unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index

I got about 20 messages like this with varying values for everything
except the filetype and error counts.  Based on what I can tell, these
look like orphaned inodex, but I'm not certain.
Is it safe to tell BTRFS to try and fix these errors?

I'm running btrfs-progs 4.3.1 on kernel 4.3.3 (at least, that's the version that
works right now, I see the same errors on 4.4.1, but I have other issues
there that are (hopefully) unrelated to BtRFS).

Re: Question about a specific error.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
> In the process of trying to debug issues I'm having on one of my
> systems with a new kernel version, I decided to do a dry run check on
> the root filesystem.  'btrfs check' returned a bunch of lines like:
> 
> root 257 inode XXXXXX errors 2000, link count wrong
>         unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index
> 
> I got about 20 messages like this with varying values for everything
> except the filetype and error counts.  Based on what I can tell, these
> look like orphaned inodex, but I'm not certain.
> Is it safe to tell BTRFS to try and fix these errors?

   Yes, those are errors I'd expect btrfs check --repair to handle
properly.

   Hugo.

> I'm running btrfs-progs 4.3.1 on kernel 4.3.3 (at least, that's the version that
> works right now, I see the same errors on 4.4.1, but I have other issues
> there that are (hopefully) unrelated to BtRFS).

-- 
Hugo Mills             | Charting the inexorable advance of Western
hugo@... carfax.org.uk | syphilisation.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Question about a specific error.

[Austin S. Hemmelgarn]

On 2016-02-01 15:21, Hugo Mills wrote:
> On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
>> In the process of trying to debug issues I'm having on one of my
>> systems with a new kernel version, I decided to do a dry run check on
>> the root filesystem.  'btrfs check' returned a bunch of lines like:
>>
>> root 257 inode XXXXXX errors 2000, link count wrong
>>          unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index
>>
>> I got about 20 messages like this with varying values for everything
>> except the filetype and error counts.  Based on what I can tell, these
>> look like orphaned inodex, but I'm not certain.
>> Is it safe to tell BTRFS to try and fix these errors?
>
>     Yes, those are errors I'd expect btrfs check --repair to handle
> properly.
>
Out of curiosity, do you happen to know if this is how btrfs check 
reports orphaned inodes, or is this something else entirely?

Re: Question about a specific error.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]

On Mon, Feb 01, 2016 at 03:37:39PM -0500, Austin S. Hemmelgarn wrote:
> On 2016-02-01 15:21, Hugo Mills wrote:
> >On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
> >>In the process of trying to debug issues I'm having on one of my
> >>systems with a new kernel version, I decided to do a dry run check on
> >>the root filesystem.  'btrfs check' returned a bunch of lines like:
> >>
> >>root 257 inode XXXXXX errors 2000, link count wrong
> >>         unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index
> >>
> >>I got about 20 messages like this with varying values for everything
> >>except the filetype and error counts.  Based on what I can tell, these
> >>look like orphaned inodex, but I'm not certain.
> >>Is it safe to tell BTRFS to try and fix these errors?
> >
> >    Yes, those are errors I'd expect btrfs check --repair to handle
> >properly.
> >
> Out of curiosity, do you happen to know if this is how btrfs check
> reports orphaned inodes, or is this something else entirely?

   I don't know, sorry.

   Hugo.

-- 
Hugo Mills             | We teach people management skills by examining
hugo@... carfax.org.uk | characters in Shakespeare. You could look at
http://carfax.org.uk/  | Claudius's crisis management techniques, for
PGP: E2AB1DE4          | example.       Richard Smith-Jones, Slings and Arrows

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Question about a specific error.

[Austin S. Hemmelgarn]

On 2016-02-01 15:21, Hugo Mills wrote:
> On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
>> In the process of trying to debug issues I'm having on one of my
>> systems with a new kernel version, I decided to do a dry run check on
>> the root filesystem.  'btrfs check' returned a bunch of lines like:
>>
>> root 257 inode XXXXXX errors 2000, link count wrong
>>          unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index
>>
>> I got about 20 messages like this with varying values for everything
>> except the filetype and error counts.  Based on what I can tell, these
>> look like orphaned inodex, but I'm not certain.
>> Is it safe to tell BTRFS to try and fix these errors?
>
>     Yes, those are errors I'd expect btrfs check --repair to handle
> properly.
>
OK, it looks like things were fixed safely, but I'm not 100% certain 
that it fixed things the way it should have.  All of the files it 
reported got moved to /lost+found (which makes me think it thought they 
were orphaned items), but none of the files themselves showed any issues 
in regular usage (they were all perfectly visible beforehand in the 
regular directory structure, and there were no errors accessing them). 
On top of that, it pulled out two different versions of each one, one 
from more than a year ago, and one current version.  I think btrfs check 
may have gotten either confused or over-zealous and just decided it 
needed to pull out the current, perfectly fine versions of the files as 
well.

Re: Question about a specific error.

[Chris Murphy]

On Tue, Feb 2, 2016 at 6:00 AM, Austin S. Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 2016-02-01 15:21, Hugo Mills wrote:
>>
>> On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
>>>
>>> In the process of trying to debug issues I'm having on one of my
>>> systems with a new kernel version, I decided to do a dry run check on
>>> the root filesystem.  'btrfs check' returned a bunch of lines like:
>>>
>>> root 257 inode XXXXXX errors 2000, link count wrong
>>>          unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0
>>> errors 3, no dir item, no dir index
>>>
>>> I got about 20 messages like this with varying values for everything
>>> except the filetype and error counts.  Based on what I can tell, these
>>> look like orphaned inodex, but I'm not certain.
>>> Is it safe to tell BTRFS to try and fix these errors?
>>
>>
>>     Yes, those are errors I'd expect btrfs check --repair to handle
>> properly.
>>
> OK, it looks like things were fixed safely, but I'm not 100% certain that it
> fixed things the way it should have.  All of the files it reported got moved
> to /lost+found (which makes me think it thought they were orphaned items),
> but none of the files themselves showed any issues in regular usage (they
> were all perfectly visible beforehand in the regular directory structure,
> and there were no errors accessing them). On top of that, it pulled out two
> different versions of each one, one from more than a year ago, and one
> current version.  I think btrfs check may have gotten either confused or
> over-zealous and just decided it needed to pull out the current, perfectly
> fine versions of the files as well.

The problems look different. You're reporting errors 2000. I'm seeing
errors 2001. I'm not sure what the distinction is; but in my case,
cancelling btrfs check and just rerunning it gives different results.
https://bugzilla.kernel.org/show_bug.cgi?id=111841

-- 
Chris Murphy

Re: Question about a specific error.

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2563 bytes --]

On Wed, Feb 03, 2016 at 02:17:06PM -0700, Chris Murphy wrote:
> On Tue, Feb 2, 2016 at 6:00 AM, Austin S. Hemmelgarn
> <ahferroin7@gmail.com> wrote:
> > On 2016-02-01 15:21, Hugo Mills wrote:
> >>
> >> On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
> >>>
> >>> In the process of trying to debug issues I'm having on one of my
> >>> systems with a new kernel version, I decided to do a dry run check on
> >>> the root filesystem.  'btrfs check' returned a bunch of lines like:
> >>>
> >>> root 257 inode XXXXXX errors 2000, link count wrong
> >>>          unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0
> >>> errors 3, no dir item, no dir index
> >>>
> >>> I got about 20 messages like this with varying values for everything
> >>> except the filetype and error counts.  Based on what I can tell, these
> >>> look like orphaned inodex, but I'm not certain.
> >>> Is it safe to tell BTRFS to try and fix these errors?
> >>
> >>
> >>     Yes, those are errors I'd expect btrfs check --repair to handle
> >> properly.
> >>
> > OK, it looks like things were fixed safely, but I'm not 100% certain that it
> > fixed things the way it should have.  All of the files it reported got moved
> > to /lost+found (which makes me think it thought they were orphaned items),
> > but none of the files themselves showed any issues in regular usage (they
> > were all perfectly visible beforehand in the regular directory structure,
> > and there were no errors accessing them). On top of that, it pulled out two
> > different versions of each one, one from more than a year ago, and one
> > current version.  I think btrfs check may have gotten either confused or
> > over-zealous and just decided it needed to pull out the current, perfectly
> > fine versions of the files as well.
> 
> The problems look different. You're reporting errors 2000. I'm seeing
> errors 2001. I'm not sure what the distinction is; but in my case,
> cancelling btrfs check and just rerunning it gives different results.
> https://bugzilla.kernel.org/show_bug.cgi?id=111841

   The "errors" field in the output of btrfs check is a hex
(strange... I thought it was octal...) bit-field indicating the set of
error types encountered. They're defined in #defines at the top of
cmds-check.c. 2000 is I_ERR_SOME_CSUM_MISSING, 1 is I_ERR_NO_INODE_ITEM.

   Hugo.

-- 
Hugo Mills             | You are not stuck in traffic: you are traffic
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                    German ad campaign

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Question about a specific error.

[Austin S. Hemmelgarn]

On 2016-02-03 16:27, Hugo Mills wrote:
> On Wed, Feb 03, 2016 at 02:17:06PM -0700, Chris Murphy wrote:
>> On Tue, Feb 2, 2016 at 6:00 AM, Austin S. Hemmelgarn
>> <ahferroin7@gmail.com> wrote:
>>> On 2016-02-01 15:21, Hugo Mills wrote:
>>>>
>>>> On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:
>>>>>
>>>>> In the process of trying to debug issues I'm having on one of my
>>>>> systems with a new kernel version, I decided to do a dry run check on
>>>>> the root filesystem.  'btrfs check' returned a bunch of lines like:
>>>>>
>>>>> root 257 inode XXXXXX errors 2000, link count wrong
>>>>>           unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0
>>>>> errors 3, no dir item, no dir index
>>>>>
>>>>> I got about 20 messages like this with varying values for everything
>>>>> except the filetype and error counts.  Based on what I can tell, these
>>>>> look like orphaned inodex, but I'm not certain.
>>>>> Is it safe to tell BTRFS to try and fix these errors?
>>>>
>>>>
>>>>      Yes, those are errors I'd expect btrfs check --repair to handle
>>>> properly.
>>>>
>>> OK, it looks like things were fixed safely, but I'm not 100% certain that it
>>> fixed things the way it should have.  All of the files it reported got moved
>>> to /lost+found (which makes me think it thought they were orphaned items),
>>> but none of the files themselves showed any issues in regular usage (they
>>> were all perfectly visible beforehand in the regular directory structure,
>>> and there were no errors accessing them). On top of that, it pulled out two
>>> different versions of each one, one from more than a year ago, and one
>>> current version.  I think btrfs check may have gotten either confused or
>>> over-zealous and just decided it needed to pull out the current, perfectly
>>> fine versions of the files as well.
>>
>> The problems look different. You're reporting errors 2000. I'm seeing
>> errors 2001. I'm not sure what the distinction is; but in my case,
>> cancelling btrfs check and just rerunning it gives different results.
>> https://bugzilla.kernel.org/show_bug.cgi?id=111841
>
>     The "errors" field in the output of btrfs check is a hex
> (strange... I thought it was octal...) bit-field indicating the set of
> error types encountered. They're defined in #defines at the top of
> cmds-check.c. 2000 is I_ERR_SOME_CSUM_MISSING, 1 is I_ERR_NO_INODE_ITEM.

OK, that makes some sense.  Although I'm still curious why a file with 
some of the checksums missing would:
1) Not have any apparent errors during regular usage (no errors in dmesg 
or syslog, and access worked just fine) when it's not set to nodatasum 
or nocow.
2) Be reported in a manner that seems to imply it's orphaned (which I 
would expect for Chris' case, as I_ERR_NO_INODE_ITEM sounds like a type 
of orphan to me).

Re: Question about a specific error.

[Duncan]

Chris Murphy posted on Wed, 03 Feb 2016 14:17:06 -0700 as excerpted:

> The problems look different. You're reporting errors 2000. I'm seeing
> errors 2001. I'm not sure what the distinction is; but in my case,
> cancelling btrfs check and just rerunning it gives different results.
> https://bugzilla.kernel.org/show_bug.cgi?id=111841

You likely know this but didn't explicitly state it, and for others...

AFAIK, errors is a bitmask, so errors 2001 is the same bitmask error as 
2000, plus another, errors 0001.

But I'm not sure what errors those actually are, either.  The errors 
bitmask is obviously targeted at devs who can read the code and see what 
each bit means, and to my knowledge there's no table posted on the wiki 
or wherever for non-devs to look it up.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Question about a specific error.

[Chris Murphy]

On Mon, Feb 1, 2016 at 12:44 PM, Austin S. Hemmelgarn
<ahferroin7@gmail.com> wrote:
> In the process of trying to debug issues I'm having on one of my
> systems with a new kernel version, I decided to do a dry run check on
> the root filesystem.  'btrfs check' returned a bunch of lines like:
>
> root 257 inode XXXXXX errors 2000, link count wrong
>         unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index
>
> I got about 20 messages like this with varying values for everything
> except the filetype and error counts.  Based on what I can tell, these
> look like orphaned inodex, but I'm not certain.
> Is it safe to tell BTRFS to try and fix these errors?

Is it consistent between boots? I have a system with SSD where if I
boot with rd.break=pre-mount, and run btrfs check, I sometimes get a
long string of similar messages. But upon reboot they don't happen
anymore.


-- 
Chris Murphy

Re: Question about a specific error.

[Austin S. Hemmelgarn]

On 2016-02-01 15:27, Chris Murphy wrote:
> On Mon, Feb 1, 2016 at 12:44 PM, Austin S. Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>> In the process of trying to debug issues I'm having on one of my
>> systems with a new kernel version, I decided to do a dry run check on
>> the root filesystem.  'btrfs check' returned a bunch of lines like:
>>
>> root 257 inode XXXXXX errors 2000, link count wrong
>>          unresolved ref dir YYYYY index 53 namelen 3 name LOG filetype 0 errors 3, no dir item, no dir index
>>
>> I got about 20 messages like this with varying values for everything
>> except the filetype and error counts.  Based on what I can tell, these
>> look like orphaned inodex, but I'm not certain.
>> Is it safe to tell BTRFS to try and fix these errors?
>
> Is it consistent between boots? I have a system with SSD where if I
> boot with rd.break=pre-mount, and run btrfs check, I sometimes get a
> long string of similar messages. But upon reboot they don't happen
> anymore.
>
I've only checked twice, but the output is identical, so they appear to 
be consistent.  What seems interesting about this to me is that while 
most of them are in the same directory, about 1/4 are just scattered 
around the FS.  None of it is anything critical though, and the system 
runs just fine (or at least, it appears to, most of the issues I'm 
having appear to be GPU related, and don't seem to have anything to do 
with BTRFS).