Re: [PATCH 2/4] read-cache: add index.skipHash config option

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Wed, Dec 07 2022, Derrick Stolee via GitGitGadget wrote:

> From: Derrick Stolee <derrickstolee@github.com>
> [...]
> While older Git versions will not recognize the null hash as a special
> case, the file format itself is still being met in terms of its
> structure. Using this null hash will still allow Git operations to
> function across older versions.

That's good news, but...

> The one exception is 'git fsck' which checks the hash of the index file.
> This used to be a check on every index read, but was split out to just
> the index in a33fc72fe91 (read-cache: force_verify_index_checksum,
> 2017-04-14).

...uh, what?

Is there an implied claim here that versions before v2.13.0 don't count
as "older versions"?

I.e. doesn't v2.12.0 hard fail the verification for all index writing?
It's only after v2.13.0 that we do it only for the fsck.

That seems like a rather significant caveat that we should be noting
prominently in the docs added in 4/4.

> As a quick comparison, I tested 'git update-index --force-write' with
> and without index.computeHash=false on a copy of the Linux kernel
> repository.

It took me a bit to see why I was failing to reproduce this, before
finding that it's because you mention index.computeHash here, but it's
index.skipHash now.
>
> Benchmark 1: with hash
>   Time (mean ± σ):      46.3 ms ±  13.8 ms    [User: 34.3 ms, System: 11.9 ms]
>   Range (min … max):    34.3 ms …  79.1 ms    82 runs
>
> Benchmark 2: without hash
>   Time (mean ± σ):      26.0 ms ±   7.9 ms    [User: 11.8 ms, System: 14.2 ms]
>   Range (min … max):    16.3 ms …  42.0 ms    69 runs
>
> Summary
>   'without hash' ran
>     1.78 ± 0.76 times faster than 'with hash'

I suggested in
https://lore.kernel.org/git/221207.868rjiam86.gmgdl@evledraar.gmail.com/
earlier to benchmark this against not-sha1collisiondetection.

I don't think I get HW-accelerated SHA-1 on that box with OPENSSL (how
do I check...?). The results on my main development box are:
	
	$ hyperfine -L g sha1dc,openssl -L v false,true -n '{g} with index.skipHash={v}' './git.{g} -c index.skipHash={v} -C /dev/shm/linux-mem update-index --force-write' -w 5 -r 10
	Benchmark 1: sha1dc with index.skipHash=false
	  Time (mean ± σ):      37.0 ms ±   2.3 ms    [User: 30.8 ms, System: 6.0 ms]
	  Range (min … max):    35.1 ms …  41.4 ms    10 runs
	 
	Benchmark 2: openssl with index.skipHash=false
	  Time (mean ± σ):      21.5 ms ±   0.4 ms    [User: 15.0 ms, System: 6.3 ms]
	  Range (min … max):    20.7 ms …  22.0 ms    10 runs
	 
	Benchmark 3: sha1dc with index.skipHash=true
	  Time (mean ± σ):      13.5 ms ±   0.4 ms    [User: 7.9 ms, System: 5.4 ms]
	  Range (min … max):    13.0 ms …  14.2 ms    10 runs
	 
	Benchmark 4: openssl with index.skipHash=true
	  Time (mean ± σ):      14.2 ms ±   0.3 ms    [User: 9.5 ms, System: 4.6 ms]
	  Range (min … max):    13.6 ms …  14.6 ms    10 runs
	 
	Summary
	  'sha1dc with index.skipHash=true' ran
	    1.05 ± 0.04 times faster than 'openssl with index.skipHash=true'
	    1.60 ± 0.05 times faster than 'openssl with index.skipHash=false'
	    2.75 ± 0.19 times faster than 'sha1dc with index.skipHash=false'

So, curiously it's proportionally much slower for me with the hash
checking, and skipping it with openssl is on the order of the results
you see.