Re: [PATCH] Disable OpenSSL SHA1 implementation by default

[Jeff King <peff@peff.net>]

On Thu, Feb 25, 2010 at 11:11:59PM -0500, Mark Lodato wrote:

> For what it's worth, here are my numbers from running git-fsck on my
> x86-64 machine.  It appears that BLK-SHA1 is slightly faster.  If you
> know a better benchmark to run, let me know.  Perhaps it would be good
> to write some sort of suite to test this, and let people post their
> results to some website.
> 
> make git-fsck
> ./git-fsck  66.75s user 0.28s system 99% cpu 1:07.17 total
> ./git-fsck  66.70s user 1.28s system 99% cpu 1:08.06 total
> ./git-fsck  66.77s user 0.63s system 99% cpu 1:07.42 total
> 
> make BLK_SHA1=1 git-fsck
> ./git-fsck  65.60s user 0.65s system 99% cpu 1:06.26 total
> ./git-fsck  65.39s user 0.65s system 99% cpu 1:06.06 total
> ./git-fsck  65.24s user 1.36s system 100% cpu 1:06.60 total
> 
> Core 2 Duo E6300 1.86 GHz (2MB L2 cache), 2GB memory
> Ubuntu 9.10 x86-64, gcc 4.4.1, git v1.7.0-90-g251a495

Thanks for the numbers. I get the opposite results:

  openssl, ./git-fsck (best of 3):
  62.29user 2.08system 1:04.43elapsed 99%CPU

  blk-sha1, ./git-fsck (best of 3):
  65.57user 1.33system 1:07.19elapsed 99%CPU

So about a 5% slow-down to use blk-sha1. But that is only half the
story. Dropping the openssl link dependency gives:

  stock, make -j4 test (best of 3):
  57.43user 74.60system 1:09.48elapsed 190%CPU

  NO_OPENSSL=TooSlow, make -j4 test (best of 3):
  50.64user 68.74system 1:03.48elapsed 188%CPU

Which is almost a 12% speedup by dropping openssl.

Now, a few qualifiers on what this means.

git-fsck is not a particularly good test of what users experience. They
just don't do it very often. We use it here because it's sha1-heavy. The
other heavy sha1 operation would be "git add" on large files. But it
means we are singling out pure sha1 performance, which is a relatively
small part of what users do.

The "make -j4 test" is also skewed a bit. Notice how we kept the dual
CPUs almost pegged by running a bunch of tests in parallel. In terms of
actual latency to the user, any small git speedup in the couple of git
commands somebody is running may be lost in the noise (e.g., just
invoking perl once destroys any git startup cost). The place that gets
the most benefit is going to be shell scripts which invoke git many
times.

And on top of that, there is the question of where users perceive
slowness. Even a 1000% speedup in the startup code of git may not be
noticeable to me if the different is .002 seconds to .02 seconds. But if
my gigantic "git add" takes 4 seconds instead of 4.5, I might notice it.

So I don't think there is any objective winner. But, I am willing to
assume that:

  1. As many people actually see blk-sha1 _faster_, let's assume my
     numbers for its slowdown are among the worst case (i.e., we do not
     need to do a massive campaign to collect more speed results).

  2. Most people don't care about large "git add" or "fsck". The sha-1
     performance for them falls into the "if it's not .02 instead of .002,
     I don't care" realm.

which argues for dropping openssl as the default.

In other words, the reverse of what I said in my previous message (well,
not the reverse; what I really wanted was for somebody to generate
actual numbers so we could base the decision on data).

-Peff