Re: Redundancy check using "echo check > sync_action": error reporting?

[Bas van Schaik <bas@tuxes.nl>]

Theodore Tso wrote:
> On Fri, Mar 21, 2008 at 06:35:43PM +0100, Peter Rabbitson wrote:
>   
>> Of course it would be possible to instruct md to always read all 
>> data+parity chunks and make a comparison on every read. The performance 
>> would not be much to write home about though.
>>     
>
> (...)
>
> Does it happen as much as ZFS's marketing literature implies?
> Probably not.  But as you start making bigger and bigger filesystems,
> the chances that even relatively improbable errors happen start
> increasing significantly.  Of course, the flip side of the argument is
> that if you are using the huge arrays to store things like music and
> video, maybe you don't care about a small amount of data corruption,
> since it might not be noticeable to the human eye/ear.  That's a
> pretty weak argument though, and it sends shivers up the spins of
> people who are storing, for example, medical images of X-ray or CAT
> scans.
>   
I totally agree with you, Ted, although I think your idea of a
filesystem communicating with RAID in an sophisticated way kind of
conflicts with the "layered approach" which is chosen in the world of
Linux. Should that be a reason not to implement this feature? I don't
think so.

Although most of you sketch scenarios in which it is very rare that
corruptions occur, I think you should also take into account that
storage is booming and growing like never before. This trend has caused
people (like me) to use other media to transfer and store data, using
the network for example. The assumption that data corruption is rare
because the bus and the disk are very reliable doesn't hold anymore:
other ways of communication are much more sensitive to corruption.

Of course, protection against these types of corruption should be
implemented in the appropriate layer (using checksums over packets, like
TCP does), but I think it is a little bit naive to assume that this will
succeed in all cases. On the other hand it would not make sense to read
every block after writing it (to check its consistency), but it might be
a nice feature to extend the monthly consistency check with advanced
error reporting features. Users who don't care (storing music and video,
using Ted's example) would disable this check, administrators like me
(storing large amounts of medical data) could run this check every week
or so.

Regards,

  -- Bas