Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

[Duncan <1i5t5.duncan@cox.net>]

Nils Steinger posted on Mon, 23 Nov 2015 22:10:12 +0100 as excerpted:

> Do we anything about what might cause a filesystem to enter a state
> which `send` chokes on?
> I've only seen a small sample of the corrupted files before growing
> tired of the process and just recreating the whole thing, but all of
> them were database files (presumably SQLite). Could it be that the files
> were being written to during an unclean shutdown, leading to some kind
> of corruption of the FS? Unfortunately, I was a little triggerhappy when
> cleaning up old snapshots, so there aren't any left to aid in
> troubleshooting this problem further…

Austin's the one attempting to trace down the problem, so he'd have the 
most direct answer there.  (My use-case doesn't involve snapshotting or 
send/receive at all.)

But if any type of files would be likely to create issues, it'd be 
something like database or VM image files, since the random-file-rewrite-
pattern they typically have is in general the most problematic for copy-
on-write (COW) filesystems such as btrfs.  Without some sort of 
additional fragmentation management (like the autodefrag mount option), 
these files will end up _highly_ fragmented on btrfs, often thousands of 
fragments, tens of thousands when the files in question are multi-gig.

For the typically smallish sqlite database files, autodefrag can help 
with the fragmentation such a rewrite pattern generally triggers with 
COW, and it'd be recommended in general if all such files on the 
filesystem are smallish (quarter to a half gig or smaller), but if you're 
running large VM images, etc, autodefrag doesn't scale so well to them, 
and much more complex fragmentation management will be needed.  But 
that'd be for a different post as I don't yet know if it applies here, 
and I'm trying to keep this one short.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman