Re: [PATCH 2/4] read-cache: add index.skipHash config option

[Derrick Stolee <derrickstolee@github.com>]

On 12/7/2022 6:06 PM, Ævar Arnfjörð Bjarmason wrote:
> 
> On Wed, Dec 07 2022, Derrick Stolee via GitGitGadget wrote:
> 
>> From: Derrick Stolee <derrickstolee@github.com>
>> [...]
>> While older Git versions will not recognize the null hash as a special
>> case, the file format itself is still being met in terms of its
>> structure. Using this null hash will still allow Git operations to
>> function across older versions.
> 
> That's good news, but...
> 
>> The one exception is 'git fsck' which checks the hash of the index file.
>> This used to be a check on every index read, but was split out to just
>> the index in a33fc72fe91 (read-cache: force_verify_index_checksum,
>> 2017-04-14).
> 
> ...uh, what?
> 
> Is there an implied claim here that versions before v2.13.0 don't count
> as "older versions"?
> 
> I.e. doesn't v2.12.0 hard fail the verification for all index writing?
> It's only after v2.13.0 that we do it only for the fsck.
> 
> That seems like a rather significant caveat that we should be noting
> prominently in the docs added in 4/4.

I can add those details.
 
>> As a quick comparison, I tested 'git update-index --force-write' with
>> and without index.computeHash=false on a copy of the Linux kernel
>> repository.
> 
> It took me a bit to see why I was failing to reproduce this, before
> finding that it's because you mention index.computeHash here, but it's
> index.skipHash now.
>>
>> Benchmark 1: with hash
>>   Time (mean ± σ):      46.3 ms ±  13.8 ms    [User: 34.3 ms, System: 11.9 ms]
>>   Range (min … max):    34.3 ms …  79.1 ms    82 runs
>>
>> Benchmark 2: without hash
>>   Time (mean ± σ):      26.0 ms ±   7.9 ms    [User: 11.8 ms, System: 14.2 ms]
>>   Range (min … max):    16.3 ms …  42.0 ms    69 runs
>>
>> Summary
>>   'without hash' ran
>>     1.78 ± 0.76 times faster than 'with hash'
> 
> I suggested in
> https://lore.kernel.org/git/221207.868rjiam86.gmgdl@evledraar.gmail.com/
> earlier to benchmark this against not-sha1collisiondetection.

Generally, I'm avoiding that benchmark because sha1dc is here to stay.

If users want to go through the trouble of compiling to use the non-dc
version, then I would expect the difference to be less noticeable, but
still significant. However, I would strongly avoid considering compiling
both into the client by default, letting certain paths use sha1dc and
others using non-dc. Certain secure environments currently only use Git
under exceptions that allow SHA1 for "non-cryptographic" reasons, but
also with the understanding that sha1dc is used as a safety measure.
Adding the non-dc version back in would put that understanding at risk.

Thanks,
-Stolee