Re: x86 SHA1: Faster than OpenSSL

[Jon Smirl <jonsmirl@gmail.com>]

On Mon, Aug 3, 2009 at 10:51 PM, Linus
Torvalds<torvalds@linux-foundation.org> wrote:
>
>
> On Mon, 3 Aug 2009, Linus Torvalds wrote:
>>
>> The thing that I'd prefer is simply
>>
>>       git fsck --full
>>
>> on the Linux kernel archive. For me (with a fast machine), it takes about
>> 4m30s with the OpenSSL SHA1, and takes 6m40s with the Mozilla SHA1 (ie
>> using a NO_OPENSSL=1 build).
>>
>> So that's an example of a load that is actually very sensitive to SHA1
>> performance (more so than _most_ git loads, I suspect), and at the same
>> time is a real git load rather than some SHA1-only microbenchmark. It also
>> shows very clearly why we default to the OpenSSL version over the Mozilla
>> one.
>
> "perf report --sort comm,dso,symbol" profiling shows the following for
> 'git fsck --full' on the kernel repo, using the Mozilla SHA1:
>
>    47.69%               git  /home/torvalds/git/git     [.] moz_SHA1_Update
>    22.98%               git  /lib64/libz.so.1.2.3       [.] inflate_fast
>     7.32%               git  /lib64/libc-2.10.1.so      [.] __GI_memcpy
>     4.66%               git  /lib64/libz.so.1.2.3       [.] inflate
>     3.76%               git  /lib64/libz.so.1.2.3       [.] adler32
>     2.86%               git  /lib64/libz.so.1.2.3       [.] inflate_table
>     2.41%               git  /home/torvalds/git/git     [.] lookup_object
>     1.31%               git  /lib64/libc-2.10.1.so      [.] _int_malloc
>     0.84%               git  /home/torvalds/git/git     [.] patch_delta
>     0.78%               git  [kernel]                   [k] hpet_next_event
>
> so yeah, SHA1 performance matters. Judging by the OpenSSL numbers, the
> OpenSSL SHA1 implementation must be about twice as fast as the C version
> we use.

Would there happen to be a SHA1 implementation around that can compute
the SHA1 without first decompressing the data? Databases gain a lot of
speed by using special algorithms that can directly operate on the
compressed data.

>
> That said, under "normal" git usage models, the SHA1 costs are almost
> invisible. So git-fsck is definitely a fairly unusual case that stresses
> the SHA1 performance more than most git lods.
>
>                Linus
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Jon Smirl
jonsmirl@gmail.com