From: Linus Torvalds <torvalds@linux-foundation.org>
To: Artur Skawina <art.08.09@gmail.com>
Cc: Nicolas Pitre <nico@cam.org>, George Spelvin <linux@horizon.com>,
Junio C Hamano <gitster@pobox.com>,
git@vger.kernel.org
Subject: Re: x86 SHA1: Faster than OpenSSL
Date: Wed, 5 Aug 2009 20:48:20 -0700 (PDT) [thread overview]
Message-ID: <alpine.LFD.2.01.0908052043082.3390@localhost.localdomain> (raw)
In-Reply-To: <alpine.LFD.2.01.0908052024081.3390@localhost.localdomain>
On Wed, 5 Aug 2009, Linus Torvalds wrote:
>
> However, right now my biggest profile hit is on this irritating loop:
>
> /* Unroll it? */
> for (t = 16; t <= 79; t++)
> W[t] = SHA_ROL(W[t-3] ^ W[t-8] ^ W[t-14] ^ W[t-16], 1);
>
> and I haven't been able to move _that_ into the other iterations yet.
Oh yes I have.
Here's the patch that gets me sub-28s git-fsck times. In fact, it gives me
sub-27s times. In fact, it's really close to the OpenSSL times.
And all using plain C.
Again - this is all on x86-64. I suspect 32-bit code ends up having
spills due to register pressure. That said, I did get rid of that big
temporary array, and it now basically only uses that 512-bit array as one
circular queue.
Linus
PS. Ok, so my definition of "plain C" is a bit odd. There's nothing plain
about it. It's disgusting C preprocessor misuse. But dang, it's kind of
fun to abuse the compiler this way.
---
block-sha1/sha1.c | 28 ++++++++++++++++------------
1 files changed, 16 insertions(+), 12 deletions(-)
diff --git a/block-sha1/sha1.c b/block-sha1/sha1.c
index 39a5bbb..80193d4 100644
--- a/block-sha1/sha1.c
+++ b/block-sha1/sha1.c
@@ -96,9 +96,8 @@ void blk_SHA1_Final(unsigned char hashout[20], blk_SHA_CTX *ctx)
static void blk_SHA1Block(blk_SHA_CTX *ctx, const unsigned int *data)
{
- int t;
unsigned int A,B,C,D,E,TEMP;
- unsigned int W[80];
+ unsigned int array[16];
A = ctx->H[0];
B = ctx->H[1];
@@ -107,8 +106,8 @@ static void blk_SHA1Block(blk_SHA_CTX *ctx, const unsigned int *data)
E = ctx->H[4];
#define T_0_15(t) \
- TEMP = htonl(data[t]); W[t] = TEMP; \
- TEMP += SHA_ROL(A,5) + (((C^D)&B)^D) + E + 0x5a827999; \
+ TEMP = htonl(data[t]); array[t] = TEMP; \
+ TEMP += SHA_ROL(A,5) + (((C^D)&B)^D) + E + 0x5a827999; \
E = D; D = C; C = SHA_ROR(B, 2); B = A; A = TEMP; \
T_0_15( 0); T_0_15( 1); T_0_15( 2); T_0_15( 3); T_0_15( 4);
@@ -116,18 +115,21 @@ static void blk_SHA1Block(blk_SHA_CTX *ctx, const unsigned int *data)
T_0_15(10); T_0_15(11); T_0_15(12); T_0_15(13); T_0_15(14);
T_0_15(15);
- /* Unroll it? */
- for (t = 16; t <= 79; t++)
- W[t] = SHA_ROL(W[t-3] ^ W[t-8] ^ W[t-14] ^ W[t-16], 1);
+/* This "rolls" over the 512-bit array */
+#define W(x) (array[(x)&15])
+#define SHA_XOR(t) \
+ TEMP = SHA_ROL(W(t+13) ^ W(t+8) ^ W(t+2) ^ W(t), 1); W(t) = TEMP;
#define T_16_19(t) \
- TEMP = SHA_ROL(A,5) + (((C^D)&B)^D) + E + W[t] + 0x5a827999; \
- E = D; D = C; C = SHA_ROR(B, 2); B = A; A = TEMP;
+ SHA_XOR(t); \
+ TEMP += SHA_ROL(A,5) + (((C^D)&B)^D) + E + 0x5a827999; \
+ E = D; D = C; C = SHA_ROR(B, 2); B = A; A = TEMP; \
T_16_19(16); T_16_19(17); T_16_19(18); T_16_19(19);
#define T_20_39(t) \
- TEMP = SHA_ROL(A,5) + (B^C^D) + E + W[t] + 0x6ed9eba1; \
+ SHA_XOR(t); \
+ TEMP += SHA_ROL(A,5) + (B^C^D) + E + 0x6ed9eba1; \
E = D; D = C; C = SHA_ROR(B, 2); B = A; A = TEMP;
T_20_39(20); T_20_39(21); T_20_39(22); T_20_39(23); T_20_39(24);
@@ -136,7 +138,8 @@ static void blk_SHA1Block(blk_SHA_CTX *ctx, const unsigned int *data)
T_20_39(35); T_20_39(36); T_20_39(37); T_20_39(38); T_20_39(39);
#define T_40_59(t) \
- TEMP = SHA_ROL(A,5) + ((B&C)|(D&(B|C))) + E + W[t] + 0x8f1bbcdc; \
+ SHA_XOR(t); \
+ TEMP += SHA_ROL(A,5) + ((B&C)|(D&(B|C))) + E + 0x8f1bbcdc; \
E = D; D = C; C = SHA_ROR(B, 2); B = A; A = TEMP;
T_40_59(40); T_40_59(41); T_40_59(42); T_40_59(43); T_40_59(44);
@@ -145,7 +148,8 @@ static void blk_SHA1Block(blk_SHA_CTX *ctx, const unsigned int *data)
T_40_59(55); T_40_59(56); T_40_59(57); T_40_59(58); T_40_59(59);
#define T_60_79(t) \
- TEMP = SHA_ROL(A,5) + (B^C^D) + E + W[t] + 0xca62c1d6; \
+ SHA_XOR(t); \
+ TEMP += SHA_ROL(A,5) + (B^C^D) + E + 0xca62c1d6; \
E = D; D = C; C = SHA_ROR(B, 2); B = A; A = TEMP;
T_60_79(60); T_60_79(61); T_60_79(62); T_60_79(63); T_60_79(64);
next prev parent reply other threads:[~2009-08-06 3:49 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-07-26 23:21 Performance issue of 'git branch' George Spelvin
2009-07-31 10:46 ` Request for benchmarking: x86 SHA1 code George Spelvin
2009-07-31 11:11 ` Erik Faye-Lund
2009-07-31 11:31 ` George Spelvin
2009-07-31 11:37 ` Michael J Gruber
2009-07-31 12:24 ` Erik Faye-Lund
2009-07-31 12:29 ` Johannes Schindelin
2009-07-31 12:32 ` George Spelvin
2009-07-31 12:45 ` Erik Faye-Lund
2009-07-31 13:02 ` George Spelvin
2009-07-31 11:21 ` Michael J Gruber
2009-07-31 11:26 ` Michael J Gruber
2009-07-31 12:31 ` Carlos R. Mafra
2009-07-31 13:27 ` Brian Ristuccia
2009-07-31 14:05 ` George Spelvin
2009-07-31 13:27 ` Jakub Narebski
2009-07-31 15:05 ` Peter Harris
2009-07-31 15:22 ` Peter Harris
2009-08-03 3:47 ` x86 SHA1: Faster than OpenSSL George Spelvin
2009-08-03 7:36 ` Jonathan del Strother
2009-08-04 1:40 ` Mark Lodato
2009-08-04 2:30 ` Linus Torvalds
2009-08-04 2:51 ` Linus Torvalds
2009-08-04 3:07 ` Jon Smirl
2009-08-04 5:01 ` George Spelvin
2009-08-04 12:56 ` Jon Smirl
2009-08-04 14:29 ` Dmitry Potapov
2009-08-18 21:50 ` Andy Polyakov
2009-08-04 4:48 ` George Spelvin
2009-08-04 6:30 ` Linus Torvalds
2009-08-04 8:01 ` George Spelvin
2009-08-04 20:41 ` Junio C Hamano
2009-08-05 18:17 ` George Spelvin
2009-08-05 20:36 ` Johannes Schindelin
2009-08-05 20:44 ` Junio C Hamano
2009-08-05 20:55 ` Linus Torvalds
2009-08-05 23:13 ` Linus Torvalds
2009-08-06 1:18 ` Linus Torvalds
2009-08-06 1:52 ` Nicolas Pitre
2009-08-06 2:04 ` Junio C Hamano
2009-08-06 2:10 ` Linus Torvalds
2009-08-06 2:20 ` Nicolas Pitre
2009-08-06 2:08 ` Linus Torvalds
2009-08-06 3:19 ` Artur Skawina
2009-08-06 3:31 ` Linus Torvalds
2009-08-06 3:48 ` Linus Torvalds [this message]
2009-08-06 4:01 ` Linus Torvalds
2009-08-06 4:28 ` Artur Skawina
2009-08-06 4:50 ` Linus Torvalds
2009-08-06 5:19 ` Artur Skawina
2009-08-06 7:03 ` George Spelvin
2009-08-06 4:52 ` George Spelvin
2009-08-06 4:08 ` Artur Skawina
2009-08-06 4:27 ` Linus Torvalds
2009-08-06 5:44 ` Artur Skawina
2009-08-06 5:56 ` Artur Skawina
2009-08-06 7:45 ` Artur Skawina
2009-08-06 18:49 ` Erik Faye-Lund
2009-08-04 6:40 ` Linus Torvalds
2009-08-18 21:26 ` Andy Polyakov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.LFD.2.01.0908052043082.3390@localhost.localdomain \
--to=torvalds@linux-foundation.org \
--cc=art.08.09@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=linux@horizon.com \
--cc=nico@cam.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).