Re: btrfs raid1 and btrfs raid10 arrays NOT REDUNDANT

Jim Salter
2014-01-04T21:16:50Z

On 01/04/2014 02:18 PM, Chris Murphy wrote:
> I'm not sure what else you're referring to?(working on boot
> environment of btrfs)

Just the string of caveats regarding mounting at boot time - needing to
monkeypatch 00_header to avoid the bogus sparse file error (which,
worse, tells you to press a key when pressing a key does nothing)
followed by this, in my opinion completely unexpected, behavior when
missing a disk in a fault-tolerant array, which also requires
monkey-patching in fstab and now elsewhere in GRUB to avoid.

Please keep in mind - I think we got off on the wrong foot here, and I'm
sorry for my part in that, it was unintentional. I *love* btrfs, and
think the devs are doing incredible work. I'm excited about it. I'm
aware it's not intended for production yet. However, it's just on the
cusp, with distributions not only including it in their installers but a
couple teetering on the fence with declaring it their next default FS
(Oracle Unbreakable, OpenSuse, hell even RedHat was flirting with the
idea) that it seems to me some extra testing with an eye towards
production isn't a bad thing. That's why I'm here. Not to crap on
anybody, but to get involved, hopefully helpfully.

> fs_passno is 1 which doesn't apply to Btrfs.
Again, that's the distribution's default, so the argument should be with
them, not me... with that said, I'd respectfully argue that fs_passno 1
is correct for any root file system; if the file system itself declines
to run an fsck that's up to the filesystem, but it's correct to specify
fs_passno 1 if the filesystem is to be mounted as root in the first place.

I'm open to hearing why that's a bad idea, if you have a specific reason?

> Well actually LVM thinp does have fast snapshots without requiring
> preallocation, and uses COW.

LVM's snapshots aren't very useful for me - there's a performance
penalty while you have them in place, so they're best used as a
transient use-then-immediately-delete feature, for instance for
rsync'ing off a database binary. Until recently, there also wasn't a
good way to roll back an LV to a snapshot, and even now, that can be
pretty problematic. Finally, there's no way to get a partial copy of an
LV snapshot out of the snapshot and back into production, so if eg you
have virtual machines of significant size, you could be looking at
*hours* of file copy operations to restore an individual VM out of a
snapshot (if you even have the drive space available for it), as
compared to btrfs' cp --reflink=always operation, which allows you to do
the same thing instantaneously.

FWIW, I think the ability to do cp --reflink=always is one of the big
killer features that makes btrfs more attractive than zfs (which, again
FWIW, I have 5+ years of experience with, and is my current primary
storage system).

> I'm not sure what you mean by self-correcting, but if the drive
> reports a read error md, lvm, and Btrfs raid1+ all will get missing
> data from mirror/parity reconstruction, and write corrected data back
> to the bad sector.

You're assuming that the drive will actually *report* a read error,
which is frequently not the case. I have a production ZFS array right
now that I need to replace an Intel SSD on - the SSD has thrown > 10K
checksum errors in six months. Zero read or write errors. Neither
hardware RAID nor mdraid nor LVM would have helped me there.

Since running filesystems that do block-level checksumming, I have
become aware that bitrot happens without hardware errors getting thrown
FAR more frequently than I would have thought before having the tools to
spot it. ZFS, and now btrfs, are the only tools at hand that can
actually prevent it.