Help repairing corrupt btrfs -- btrfsck --repair doesn't change anything

Help repairing corrupt btrfs -- btrfsck --repair doesn't change anything

[Zack Weinberg]

I have a btrfs partition with what *sounds* like minor damage; btrfsck
--repair prints

| enabling repair mode
| Checking filesystem on /dev/sda
| UUID: ec93d2c2-7937-40f8-aaa6-
c20c9775d93a
| checking extents
| checking free space cache
| cache and super generation don't match, space cache will be invalidated
| checking fs roots
| root 258 inode 4493802 errors 400, nbytes wrong
| root 258 inode 4509858 errors 400, nbytes wrong
| root 258 inode 4510014 errors 400, nbytes wrong
| root 258 inode 4838894 errors 400, nbytes wrong
| root 258 inode 4838895 errors 400, nbytes wrong
| found 41852229430 bytes used err is 1
| total csum bytes: 619630328
| total tree bytes: 3216027648
| total fs tree bytes: 2342981632
| total extent tree bytes: 135536640
| btree space waste bytes: 767795634
| file data blocks allocated: 1744289230848
|  referenced 631766474752
| Btrfs v3.12

The trouble is, these errors do not go away if I run btrfsck --repair
a second time, which implies that the problems have not actually been
corrected.  As the corruption has already caused one kernel oops (see
https://bugzilla.kernel.org/show_bug.cgi?id=68411 ) I am reluctant to
remount the file system until I am sure no corruption remains.  I did
try mounting it (and immediately unmounting it again) and that did
seem to do some sort of additional check ("checking UUID tree") but a
subsequent btrfsck run still prints the same errors.

Any advice on how to fix this so it stays fixed would be appreciated.

zw

errors 400, nbytes wrong, was: Help repairing corrupt btrfs -- btrfsck --repair doesn't change anything

[Chris Murphy]

On Jan 9, 2014, at 3:47 PM, Zack Weinberg <zackw@panix.com> wrote:

> I have a btrfs partition with what *sounds* like minor damage; btrfsck
> --repair prints

For what it's worth, repair occasionally makes things worse so it's better to just run 'btrfs check' and post the results and get some advice before resorting to --repair.

> 
> | enabling repair mode
> | Checking filesystem on /dev/sda
> | UUID: ec93d2c2-7937-40f8-aaa6-
> c20c9775d93a
> | checking extents
> | checking free space cache
> | cache and super generation don't match, space cache will be invalidated
> | checking fs roots
> | root 258 inode 4493802 errors 400, nbytes wrong
> | root 258 inode 4509858 errors 400, nbytes wrong
> | root 258 inode 4510014 errors 400, nbytes wrong
> | root 258 inode 4838894 errors 400, nbytes wrong
> | root 258 inode 4838895 errors 400, nbytes wrong
> | found 41852229430 bytes used err is 1
> | total csum bytes: 619630328
> | total tree bytes: 3216027648
> | total fs tree bytes: 2342981632
> | total extent tree bytes: 135536640
> | btree space waste bytes: 767795634
> | file data blocks allocated: 1744289230848
> |  referenced 631766474752
> | Btrfs v3.12

I think this is benign, but I'm not sure. I have three Btrfs file systems and one of them also has these same kinds of errors. In some cases mounting with clear_cache option and leaving the system alone to rebuild the space cache will fix it. In my case it's not.

I also get only the normal messages when mounting this Btrfs volume, no errors. And no apparent problems. So the btrfs check is a bit confusing that it says there are errors but then apparently doesn't fix them.

# btrfs check /dev/sdc
Checking filesystem on /dev/sdc
UUID: e7cd9159-9c4e-4ddd-9887-85758dbc97af
checking extents
checking free space cache
checking fs roots
root 256 inode 37895 errors 400, nbytes wrong
root 266 inode 37895 errors 400, nbytes wrong
found 1201615306 bytes used err is 1
total csum bytes: 4344024
total tree bytes: 328122368
total fs tree bytes: 306774016
total extent tree bytes: 15695872
btree space waste bytes: 61398268
file data blocks allocated: 9117638656
 referenced 9929928704
Btrfs v3.12

The file system was created with kernel 3.11.10 running, and has variably had 3.12.x, 3.13.x, and 3.14-rc2 running and the results are the same. I'm not sure when these errors first started as I don't often run btrfs check, but it's quite a young file system, maybe only two weeks old. There have been some unclean shutdowns, with the only reported mount errors being about unlinked orphans but those messages didn't occur again on subsequent mounts.

In any case, I'm confused also if there's a real problem or not. I guess confusion about the state of a file system itself isn't good. But I figure it's probably not a serious error or there'd be mount error messages, and would maybe only mount ro.

Chris Murphy

Re: errors 400, nbytes wrong, was: Help repairing corrupt btrfs -- btrfsck --repair doesn't change anything

[Chris Murphy]

On Feb 13, 2014, at 9:48 PM, Chris Murphy <lists@colorremedies.com> wrote:
> 
> 
> In any case, I'm confused also if there's a real problem or not. I guess confusion about the state of a file system itself isn't good. But I figure it's probably not a serious error or there'd be mount error messages, and would maybe only mount ro.

Hmm, another case on a fairly new file system. The commonality with my case and this one, is the file system is created with kernel 3.11 and btrfs-progs-0.20.rc1.20131114git9f0c53f-1.fc20 - but the problem materializes at some later point, inconsistently.

https://bugzilla.redhat.com/show_bug.cgi?id=1037963


Chris Murphy

Re: errors 400, nbytes wrong, was: Help repairing corrupt btrfs -- btrfsck --repair doesn't change anything

[Chris Murphy]

On Feb 13, 2014, at 10:10 PM, Chris Murphy <lists@colorremedies.com> wrote:

> 
> On Feb 13, 2014, at 9:48 PM, Chris Murphy <lists@colorremedies.com> wrote:
>> 
>> 
>> In any case, I'm confused also if there's a real problem or not. I guess confusion about the state of a file system itself isn't good. But I figure it's probably not a serious error or there'd be mount error messages, and would maybe only mount ro.
> 
> Hmm, another case on a fairly new file system. The commonality with my case and this one, is the file system is created with kernel 3.11 and btrfs-progs-0.20.rc1.20131114git9f0c53f-1.fc20 - but the problem materializes at some later point, inconsistently.
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1037963

Aha, I think this was fairly thoroughly explained here:
https://bugzilla.kernel.org/show_bug.cgi?id=68411


Chris Murphy