Re: End to end data integrity checking in Ceph?

[Craig Dunwoody <cdunwoody@graphstream.com>]

Hello Sage,

sage writes:
>The ceph transport layer does a crc32c over all data that passes over the 
>wire to catch bit flips from the network (TCP's checksumming isn't very 
>strong).  This isn't truly end-to-end protection, though, as bit flips on 
>the client after the applicate write(2) but before writeback starts, or on 
>the server after receiving the message won't be detected.
>
>Btrfs does do it's own checksumming, so in theory if we match the function 
>on the client we can do better.  There is also some end-to-end data 
>integrity infrastructure in the kernel that IIRC Martin Peterson was 
>working on.  Much that is in the block layer, though; the only parts that 
>would be useful to ceph would relate to the userspace interface and page 
>cache.  I'm not sure what the current state of that work is.
>
>It would be nice to see end-to-end protection (complete with some sort of 
>userspace api) in action on a local file system (probably btrfs, which 
>actually stores checksums) as a model before trying to build it into a 
>more complicated distributed file system...

Thanks very much for this info.  Good to know that Ceph currently does
checking at network-transport level beyond what TCP does, and makes sense
to me that a local-FS implementation of end to end protection facilities
could be a next-step preceding distributed-FS implementation.

Craig Dunwoody
GraphStream Incorporated

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev