Re: x86 SHA1: Faster than OpenSSL

[Andy Polyakov <appro@fy.chalmers.se>]

> On Mon, 3 Aug 2009, Linus Torvalds wrote:
>> The thing that I'd prefer is simply
>>
>> 	git fsck --full
>>
>> on the Linux kernel archive. For me (with a fast machine), it takes about 
>> 4m30s with the OpenSSL SHA1, and takes 6m40s with the Mozilla SHA1 (ie 
>> using a NO_OPENSSL=1 build).
>>
>> So that's an example of a load that is actually very sensitive to SHA1 
>> performance (more so than _most_ git loads, I suspect), and at the same 
>> time is a real git load rather than some SHA1-only microbenchmark.

I couldn't agree more that real-life benchmarks are of greater value
than specific algorithm micro-benchmark. And given the provided
profiling data one can argue that +17% (or my +12%) improvement on
micro-benchmark aren't really worth bothering about. But it's kind of
sport [at least for me], so don't judge too harshly:-)

>> It also 
>> shows very clearly why we default to the OpenSSL version over the Mozilla 
>> one.

As George implicitly mentioned most OpenSSL assembler modules are
available under more permissive license and if there is interest I'm
ready to assist...

> "perf report --sort comm,dso,symbol" profiling shows the following for 
> 'git fsck --full' on the kernel repo, using the Mozilla SHA1:
> 
>     47.69%               git  /home/torvalds/git/git     [.] moz_SHA1_Update
>     22.98%               git  /lib64/libz.so.1.2.3       [.] inflate_fast
>      7.32%               git  /lib64/libc-2.10.1.so      [.] __GI_memcpy
>      4.66%               git  /lib64/libz.so.1.2.3       [.] inflate
>      3.76%               git  /lib64/libz.so.1.2.3       [.] adler32
>      2.86%               git  /lib64/libz.so.1.2.3       [.] inflate_table
>      2.41%               git  /home/torvalds/git/git     [.] lookup_object
>      1.31%               git  /lib64/libc-2.10.1.so      [.] _int_malloc
>      0.84%               git  /home/torvalds/git/git     [.] patch_delta
>      0.78%               git  [kernel]                   [k] hpet_next_event
> 
> so yeah, SHA1 performance matters. Judging by the OpenSSL numbers, the 
> OpenSSL SHA1 implementation must be about twice as fast as the C version 
> we use.

And given /lib64 path this is 64-bit C compiler-generated code compared
to 32-bit assembler? Either way in this context I have extra comment
addressing previous subscriber, Mark Lodato, who effectively wondered
how would 64-bit assembler compare to 32-bit one. First of all there
*is* even 64-bit assembler version. But as SHA1 is essentially 32-bit
algorithm, 64-bit implementation is only nominally faster, +20% at most.
Faster thanks to larger register bank facilitating more efficient
instruction scheduling.

Cheers. A.