Re: RFC: log record CRC validation

Andi Kleen
2007-07-28T13:09:08Z

"William J. Earl" <earl@agami.com> writes:

> It is usually
> cost-effective, however, to simply select a reliable disk subsystem.

In practice people don't run fully reliable disk subsystems
unfortunately. And also a previously reliable disk subsystem
might degrade over time. It's better to handle this case.

Also CPUs have gotten a lot faster recently with more cache
misses dominating than actual operations; CRCs on relatively small data
items that are already manipulated by the CPU (like file system
metadata) should be essentially free.

> With SATA, SAS, and FC, which have link-level integrity checks, silent
> data corruption on the link is unlikely.

For TCP/IP networking that is in theory true too -- with L2 networks
usually having CRCs. In practice the TCP/UDP checksum finds a lot of
corruptions wherever they come from and there are even some that are
only detected by stronger checksums (like in SSL) or go undetected.

While disk subsystems are probably doing a bit better than IP networks
in terms of error rates they are far from perfect either.

-Andi