Re: corrupt lead, bad key order & no csum found for inode & csum failed ino & input/output error

[Hugo Mills <hugo@carfax.org.uk>]

[-- Attachment #1: Type: text/plain, Size: 3251 bytes --]

On Sun, Dec 27, 2015 at 08:03:16PM -0500, james harvey wrote:
> Have gotten about 300 (mostly duplicate) BTRFS errors in the last
> hours.  No signs of disk problem.  Non-SSD SATA.  Was able to dd the
> drive into an image file on another drive without errors.  Smartctl
> reports no issues.  It is a single disk though.
> 
> Been running btrfs fine since 7/9/15, installing arch linux with linux
> 4.0.7-2 and btrfs-progs 4.1-1.
> 
> Now running linux 4.2.5-1 (installed 10/27, 4.3 isn't in arch core
> repo yet) and btrfs-progs 4.3.1-2 (installed 11/24.)
> 
> corrupt leaf, bad key order

   You have bad RAM. You need to run a memtest, find the bad hardware,
and replace it.

   After that, btrfs check may be able to fix this -- there was
certainly a patch to do so, and I think David was going to apply it.
Not sure where that's got to, though.

   Hugo.

> ====================
> 
> 219 of these lines.  All duplicates.
> 
> [ 836.639911] BTRFS critical (device sda2): corrupt leaf, bad key
> order: block=241468833792,root=1, slot=292
> 
> no csum found / csum failed
> =====================
> 
> After two of those corrupt lead criticals, have gotten 82 of these
> lines.  More of the corrupt leaf criticals between these.  Mostly
> duplicates, 5 unique inodes given.  All of these files happen to be
> .cache/chromium/Default/Cache/data_* files and
> .local/share/baloo/index, which makes sense because that's mostly what
> this machine is used for.
> 
> [ 836.640132] BTRFS info (device sda2): no csum found for inode
> 1809716 start 8011776
> [ 836.644547] BTRFS warning (device sda2): csum failed ino 1809716 off
> 8011776 csum 571345089 expected csum 0
> 
> Input/output error
> =============
> 
> Started to run a "btrfs scrub start -B -r /dev/sda2" (didn't see the
> messages above on the list recently, so wasn't going to write changes
> until I had made a dd disk image on another drive.  Got:
> 
> ERROR: scrubbing /dev/sda2 failed for device id 1: ret=-1, errno=5
> (Input/output error)
> scrub canceled for 535e3d41-989e-4c14-a837-536d79e4ff51
> scrub started at Sun Dec 27 18:33:31 2015 and was aborted after 00:18:22
> total bytes scrubbed: 156.31GiB with 0 errors
> 
> dmesg doesn't show any entries saying something like input/output
> error, just the corrupt leaf bad key order, no csum found csum failed
> errors.
> 
> sda2 is what I'm booted off of, and I'm still running the system
> without rebooting, typing this email on it, so the drive didn't stop
> being usable after btrfs scrub reported an Input/output error, and
> isn't giving more issues besides the above errors.
> 
> 
> Obviously looking for comments on all of this, but most interested in
> if the corrupt leaf, bad key order message risks a bunch of files or
> not.  Familiar with leafs as a computing term, but don't even know
> what they are in the context of btrfs.  If it's a single file, how do
> I use the corrupt leaf, bad key order block number to see what it
> corresponds to, like I did with btrfs inspect-internal with the
> inodes?

-- 
Hugo Mills             | Talking about music is like dancing about
hugo@... carfax.org.uk | architecture
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                           Frank Zappa

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]