Re: [IDEA RFC] Forward error correction (FEC) / Error correction code (ECC) for BTRFS

[Phillip Susi <phill@thesusis.net>]

Zhang Boyang <zhangboyang.id@gmail.com> writes:

> Here are some detailed approaches:
>
> 1) Zero-cost approach

<snip>

> 2) Low-cost approach:

Both of these home brew methods rely on guess and check to repair, which
has a high computational cost and the possibility of false repair.  It
also requires that only a minescule amount of silent corruption has
occured.  Disk drives already have highly robust error correction and
detection in place, so it is almost unheard of for them to silently
corrupt data, but rather they refuse to read the whole sector.  If
corrupt data somehow managed to pass the error detection code in the
drive, it is highly likely that a lot more than one or two bytes will be wrong.


> 3) Reed-Solomon approach:

If you are going to do error correction, this or another real FEC
altorithm is the way to go.  Also since the drive reports corrupt
sectors, you can use RS in erasure mode where it can correct T errors
instead of T/2.  There is a handy tool called par2 that lets you create
a small RS FEC archive of a file that you can later use to repair damaged
portions of it.

Another common behavior of drives is for them to fail outright with 100%
data loss rather than just have a few bad sectors.  For that reason, I
think that anyone who really cares about their data should be using a
raid and making regular backups rather than relying on an automatic on
the fly FEC in the filesystem.  If they don't care enough to do that
then they probably don't care enough to think the cost of online FEC is
worth it either.