Re: Suggestion on hashing

[Bill Zaumen <bill.zaumen@gmail.com>]

On Fri, 2011-12-02 at 21:22 +0700, Nguyen Thai Ngoc Duy wrote:
> (I'm not sure why you dropped git@vger. I see nothing private here so
> I bring git@vger back)

Oh, I just didn't want to flood the mailing list with too much on
one topic and figured we could summarize a discussion at some point
and post that, but if you'd rather keep it all on the list, that's 
fine with me.

I can split the code into a series of smaller patches - smaller than
the set of three I sent, but I'm not sure if the test scripts will work
with all of the intermediate patches if I do that.

I can also make the digest (current a CRC) pluggable.  Then you can try
different digests as an experiment and see how that affects performance.
My implementation uses the CRC or new digests only when
the object database is being modified or explicitly verified. Basically
the code provides memoization for an additional hash function, used
for whatever purpose you desire.

If you want to put a digest of message digests into a commit message,
you can do that fairly quickly as one level of digests has been
precomputed. I think Jeff's or your suggestion of putting an additional
digest in the commit message is a good idea.  If you want to experiment
with such changes, the code would provide a reasonable start on that.

So, I guess I should make those changes - pluggable digest and 
splitting the patches further.

> You'd need to convince git maintainer this is worth doing first,
> before talking how big the changes are ;-)

I'd guess there are several issues: the amount of code, how complex
the changes are, what the performance impacts are, whether the changes
are backwards compatible, and what you get for the effort.

As a start on the last question, "what you get," aside from some extra
checking to detect problems, if you modify commit messages and signed
tags to use better digests, you can make a stronger argument regarding
authentication.  For example, suppose you have a project in which your
code is dual-licensed - GPL for free use but a separate license if the
code is used in a proprietary product and there is a legal dispute,
using a better digest than SHA-1 would have some advantages - when they
start calling in expert witnesses, one side will bring in a security
expert who will testify that SHA-1 is too weak to be used for
authentication, citing government publications such
as http://csrc.nist.gov/groups/ST/hash/statement.html as evidence. The
jury is not going to consist of people who can fully understand the
details, so being able to say that git's authentication matches current
best practices would be an additional reason to use git.