Re: [PATCH 0/4] faster SHA-1 collision detection

[Junio C Hamano <gitster@pobox.com>]

Scott Chacon <scott@gitbutler.net> writes:

> So, spoiler alert, the code in this patch series is mainly AI generated.
> I would try to fool you, but too many of you are far too aware of my
> actual C skills. That being said, I thought maybe someone here (especially
> those of you working on server optimization stuff) would be interested
> in the speed increases for both the server and client in making sha1dc
> quite a bit faster.
>
> This series ports the approach of Sam Reis's sha1dc Rust crate [1], 
> which gitoxide recently switched to [2], to C. 

Which means license-wise the original is compatible with us, I
presume, as they are "Apache2 or MIT, your choice".

How can you/we be sure, with respect to the current AI policy in
SubmittingPatches (which by the way was vetted by SFC lawyers), that
your "AI generated" code did not "borrow" from places that gets
you/us into trouble?

> The end result hashes roughly 2.7x faster on the Xeon and 2.85x faster
> on the M5 Max. Single-threaded index-pack of git.git goes from 24.3s to
> 12.7s on the Xeon, and from 16.1s to 8.7s on the M5 Max.
>
> Hashing throughput on the Xeon, in MiB/s:
>
>                                 16KiB    1MiB   vs OpenSSL
>   OpenSSL SHA-1 (no detection)   1234    1129      1.00x
>   sha1dc/ (today)                 435     450      2.67x
>   shani+avx2 (default here)      1002     901      1.24x
>   shani+sse2                     1075    1008      1.13x
>   portable+avx2                   553     654      1.96x
>   portable+sse2                   603     681      1.84x
>   portable                        466     565      2.29x
>
> In other words, currently collision detection costs about 1.5–2.5x on
> top of the hashing itself today, but only about 0.2x with the series. 

Thanks for these numbers.

> [1] https://sam.dev/blog/faster-sha1-collision-detection
> [2] https://github.com/GitoxideLabs/gitoxide/pull/3008

And the pointers to the original sources.