btrfsck failures on old backup volumes

btrfsck failures on old backup volumes

[Simon Kirby]

Hi!

We have some btrfs rsync-snapshot backup servers which have been running
since about mid-2009, with a pretty good record so far. We've been
following the development kernels and hitting some bugs here an there,
but we still haven't managed to lose anything yet. The main problems we
have ran in to relate to ENOSPC and reporting full when "df" only shows
74% used, etc.

Recently, with some newer kernels (currently 3.4-rc6), we have some cases
where we can no longer write to some volumes -- they report out of space
even when trying to rm, or hang forever. Most volumes are 3TB carved from
some LVM'd attached storage.

Since btrfsck seems to exist now in git btrfs-progs, and this data is
already elsewhere, I figured it'd be worthwhile to try. On this one FS,
btrfsck in check mode reported thousands of errors. The --repair mode
did as well, and then exited with "failed to repair damaged filesystem,
aborting". Output logs (huge) are here: http://0x.ca/sim/ref/3.4-rc6/

I'm not sure how long these consistency issues have been on disk, since
these file systems have been around for many years. Is this useful data?
Would it be useful to try to reproduce these issues on smaller devices,
or are the current results helpful in any way for improving btrfsck?

Mount options for these are "noacl,noatime,nodiratime,compress-force",
and they were mkfs'd with no special options.

Cheers,

Simon-