btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.

btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.

[Richard Cooper]

Hi All,

I'm trying to repair a broken fs using btrfsck and am hitting a failed assertion. I'd appreciate any suggestions for what to do next. Is there any thing I can do to help fix this bug? Any other information from my FS which would help? If the FS could be salvaged that would be a bonus, but I'm more interested in providing a useful bug report before wiping the disk.

My versions are:
- OS - CentOS 6.3
- Kernel - 3.7.1-2 from http://elrepo.org/tiki/kernel-ml 
- btrfs-progs - v0.20-rc1-37-g91d9eec. Built today from git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git

The failure is:

# ./btrfsck --repair /dev/md4 
enabling repair mode
checking extents
leaf parent key incorrect 183603200
bad block 183603200
leaf parent key incorrect 183640064
bad block 183640064
warning, start mismatch 152387469312 762175488
block 152387469312 rec extent_item_refs 1, passed 1
warning, start mismatch 449606139904 427217858560
block 427217858560 rec extent_item_refs 1, passed 1
ref mismatch on [32215040 4096] extent item 30, found 31
btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.
Aborted

If i run btrfsck in non-repair mode I get:

# ./btrfsck /dev/md4 
checking extents
leaf parent key incorrect 183603200
bad block 183603200
leaf parent key incorrect 183640064
bad block 183640064
warning, start mismatch 152387469312 762175488
checksum verify failed on 34390753280 wanted 26D779EB found 40
checksum verify failed on 34390753280 wanted 26D779EB found 40
block 152387469312 rec extent_item_refs 1, passed 1
warning, start mismatch 449606139904 427217858560
block 427217858560 rec extent_item_refs 1, passed 1
ref mismatch on [32215040 4096] extent item 30, found 31
Backref 32215040 parent 427255582720 root 427255582720 not found in extent tree
backpointer mismatch on [32215040 4096]

... [snipped several thousand lines similar to the previous three] ...

ref mismatch on [477808889856 4096] extent item 11, found 12
Backref 477808889856 parent 427202011136 root 427202011136 not found in extent tree
backpointer mismatch on [477808889856 4096]
Errors found in extent allocation tree
checking fs roots
root 256 inode 140337 errors 400
root 256 inode 169441 errors 400
root 256 inode 169442 errors 400
root 256 inode 1843202 errors 400
warning line 1789
found 651533594626 bytes used err is 1
total csum bytes: 624739028
total tree bytes: 11639873536
total fs tree bytes: 10394214400
btree space waste bytes: 2925577458
file data blocks allocated: 741854191616
 referenced 741832200192
Btrfs v0.20-rc1-37-g91d9eec]

Re: btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.

[Josef Bacik]

On Thu, Jan 03, 2013 at 05:26:38AM -0700, Richard Cooper wrote:
> Hi All,
> 
> I'm trying to repair a broken fs using btrfsck and am hitting a failed assertion. I'd appreciate any suggestions for what to do next. Is there any thing I can do to help fix this bug? Any other information from my FS which would help? If the FS could be salvaged that would be a bonus, but I'm more interested in providing a useful bug report before wiping the disk.
> 

Well good news is that its the allocator failing to find space for a new block,
and the allocator in btrfs-progs is under-tested, so it's likely just an
internal bug and something we can fix.  Can you do btrfs fi show /dev/md4 (not
mounted) and post that so we can be sure there's actually enough space.  And
I'll look at the allocator code in the meantime and see if theres something
obvious wrong.  Thanks,

Josef

Re: btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.

[Richard Cooper]

On 3 Jan 2013, at 15:06, Josef Bacik wrote:

> On Thu, Jan 03, 2013 at 05:26:38AM -0700, Richard Cooper wrote:
>> Hi All,
>> 
>> I'm trying to repair a broken fs using btrfsck and am hitting a failed assertion. I'd appreciate any suggestions for what to do next. Is there anything I can do to help fix this bug? Any other information from my FS which would help? If the FS could be salvaged that would be a bonus, but I'm more interested in providing a useful bug report before wiping the disk.
>> 
> 
> Well good news is that its the allocator failing to find space for a new block,
> and the allocator in btrfs-progs is under-tested, so it's likely just an
> internal bug and something we can fix.  Can you do btrfs fi show /dev/md4 (not
> mounted) and post that so we can be sure there's actually enough space. 

# ./btrfs fi show /dev/md4 
Label: none  uuid: 5be10dea-64c1-474e-b640-987b25af3c27
	Total devices 1 FS bytes used 606.79GB
	devid    1 size 16.36TB used 627.04GB path /dev/md4

Thanks for looking at this.

Re: btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.

[Richard Cooper]

On 3 Jan 2013, at 16:43, Richard Cooper wrote:

> On 3 Jan 2013, at 15:06, Josef Bacik wrote:
> 
>> On Thu, Jan 03, 2013 at 05:26:38AM -0700, Richard Cooper wrote:
>>> Hi All,
>>> 
>>> I'm trying to repair a broken fs using btrfsck and am hitting a failed assertion. I'd appreciate any suggestions for what to do next. Is there anything I can do to help fix this bug? Any other information from my FS which would help? If the FS could be salvaged that would be a bonus, but I'm more interested in providing a useful bug report before wiping the disk.
>>> 
>> 
>> Well good news is that its the allocator failing to find space for a new block,
>> and the allocator in btrfs-progs is under-tested, so it's likely just an
>> internal bug and something we can fix.  Can you do btrfs fi show /dev/md4 (not
>> mounted) and post that so we can be sure there's actually enough space. 
> 
> # ./btrfs fi show /dev/md4 
> Label: none  uuid: 5be10dea-64c1-474e-b640-987b25af3c27
> 	Total devices 1 FS bytes used 606.79GB
> 	devid    1 size 16.36TB used 627.04GB path /dev/md4

Is this all the information you need? Is there a bug tracker I should report this to, to stop it getting lost in the mailing list archives?

Re: btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.

[Richard Cooper]

On 3 Jan 2013, at 12:26, Richard Cooper wrote:
> Hi All,
> 
> I'm trying to repair a broken fs using btrfsck and am hitting a failed assertion. I'd appreciate any suggestions for what to do next. Is there any thing I can do to help fix this bug? Any other information from my FS which would help? If the FS could be salvaged that would be a bonus, but I'm more interested in providing a useful bug report before wiping the disk.

After this error I reformatted and started again. A few days ago I hit exactly the same error in btrfsck again. Is it useful for me to report the same errors again? Would it make more sense for me to submit them to https://bugzilla.kernel.org? Is that tracker appropriate for btrfs-progs related bugs?

My versions are:
- OS - CentOS 6.3
- Kernel -  3.7.8-1 from http://elrepo.org/tiki/kernel-ml 
- btrfs-progs - v0.20-rc1-56-g6cd836d. Built from git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git

# ./btrfs-progs/btrfs fi show /dev/md4
Label: none  uuid: fecad98e-63e8-47b7-9bc1-5d9351f15e76
	Total devices 1 FS bytes used 702.22GB
	devid    1 size 16.36TB used 723.04GB path /dev/md4

Btrfs v0.20-rc1-56-g6cd836d

# ./btrfsck --repair /dev/md4 
enabling repair mode
checking extents
incorrect offsets 2539 133611
bad block 694804008960
bad key ordering 26 27
bad block 710147256320
bad key ordering 29 30
bad block 710793940992
ref mismatch on [506097786880 8192] extent item 1, found 0
btrfsck: extent-tree.c:2549: btrfs_reserve_extent: Assertion `!(ret)' failed.
Aborted
# echo $?
134

If i run btrfsck in non-repair mode I get:

# ./btrfsck /dev/md4 
checking extents
incorrect offsets 2539 133611
bad block 694804008960
bad key ordering 26 27
bad block 710147256320
bad key ordering 29 30
bad block 710793940992
ref mismatch on [506097786880 8192] extent item 1, found 0
Incorrect local backref count on 506097786880 root 257 owner 6201018 offset 0 found 0 wanted 1 back 0x72966380
backpointer mismatch on [506097786880 8192]
owner ref check failed [506097786880 8192]
ref mismatch on [506097795072 8192] extent item 1, found 0
Incorrect local backref count on 506097795072 root 257 owner 6201019 offset 0 found 0 wanted 1 back 0x72966460
backpointer mismatch on [506097795072 8192]
owner ref check failed [506097795072 8192]
ref mismatch on [506097803264 8192] extent item 1, found 0
Incorrect local backref count on 506097803264 root 257 owner 6201021 offset 0 found 0 wanted 1 back 0x72966540
backpointer mismatch on [506097803264 8192]
owner ref check failed [506097803264 8192]
ref mismatch on [506097811456 8192] extent item 1, found 0
Incorrect local backref count on 506097811456 root 257 owner 6201022 offset 0 found 0 wanted 1 back 0x72966620
backpointer mismatch on [506097811456 8192]
owner ref check failed [506097811456 8192]
ref mismatch on [686802194432 20480] extent item 1, found 0
Incorrect local backref count on 686802194432 root 257 owner 8037960 offset 0 found 0 wanted 1 back 0x82c1e2a0
backpointer mismatch on [686802194432 20480]
owner ref check failed [686802194432 20480]
ref mismatch on [686802214912 20480] extent item 1, found 0
Incorrect local backref count on 686802214912 root 257 owner 8037961 offset 0 found 0 wanted 1 back 0x82c1e380
backpointer mismatch on [686802214912 20480]
owner ref check failed [686802214912 20480]
ref mismatch on [686802235392 20480] extent item 1, found 0
Incorrect local backref count on 686802235392 root 257 owner 8037962 offset 0 found 0 wanted 1 back 0x82c1e460
backpointer mismatch on [686802235392 20480]
owner ref check failed [686802235392 20480]
ref mismatch on [686802255872 20480] extent item 1, found 0
Incorrect local backref count on 686802255872 root 257 owner 8037963 offset 0 found 0 wanted 1 back 0x82c1e540
backpointer mismatch on [686802255872 20480]
owner ref check failed [686802255872 20480]
ref mismatch on [686802276352 20480] extent item 1, found 0
Incorrect local backref count on 686802276352 root 257 owner 8037964 offset 0 found 0 wanted 1 back 0x82c1e620
backpointer mismatch on [686802276352 20480]
owner ref check failed [686802276352 20480]
ref mismatch on [686802296832 20480] extent item 1, found 0
Incorrect local backref count on 686802296832 root 257 owner 8037965 offset 0 found 0 wanted 1 back 0x82c1e700
backpointer mismatch on [686802296832 20480]
owner ref check failed [686802296832 20480]
ref mismatch on [686802317312 20480] extent item 1, found 0
Incorrect local backref count on 686802317312 root 257 owner 8037966 offset 0 found 0 wanted 1 back 0x82c1e7e0
backpointer mismatch on [686802317312 20480]
owner ref check failed [686802317312 20480]
owner ref check failed [694804008960 4096]
owner ref check failed [710147256320 4096]
owner ref check failed [710793940992 4096]
Errors found in extent allocation tree
checking fs roots
root 257 inode 6292865 errors 400
root 257 inode 6292867 errors 400
found 754001297408 bytes used err is 1
total csum bytes: 718173168
total tree bytes: 18407149568
total fs tree bytes: 16537788416
btree space waste bytes: 4527688532
file data blocks allocated: 735593959424
 referenced 735590412288
Btrfs v0.20-rc1-56-g6cd836d