Crc32 Challenge

[chris holcombe <chris.holcombe@canonical.com>]

Hello Ceph Devs,

I'm almost certain at this point that I have discovered a major bug in 
ceph's crc32c mechanism.  http://tracker.ceph.com/issues/13713 I'm 
totally open to be proven wrong and that's what this email is about.  
Can someone out there write a piece of code using an outside library 
that produces the same crc32c checksums that Ceph does?  If they can 
I'll close my bug and stand corrected :).  I've tried 3 python libraries 
and 1 rust library so far and my conclusions are 1) they are all in 
agreement and 2) they all produce different checksums than ceph's 
checksums 
https://github.com/ceph/ceph/blob/83e10f7e2df0a71bd59e6ef2aa06b52b186fddaa/src/test/common/test_crc32c.cc#L21

Start small and see if you can verify the "foo bar baz" checksum and 
then try some of the others.

For a known good checksum to test your program against use this: 
http://www.pdl.cmu.edu/mailinglists/ips/mail/msg04970.html  In there 
Mark Bakke talks about a 32 byte array of all 00h should produce a 
checksum of 8A9136AA.  Printing that with python in decimal: 2324772522

The implications of this are unfortunately tricky.  If I'm right and we 
fix ceph's algorithm then it won't be able to talk to any previous 
version of ceph past the beginning protocol handshake. There would have 
to be a mechanism introduced so that any x and older version would speak 
the previous crc and anything y and newer would speak the new version.  
Another option is we could break ceph's crc code out into a library and 
make that available to everyone and call it ceph-crc32c.

Thanks!
Chris