From: Eric Biggers <ebiggers@kernel.org>
To: David Laight <david.laight.linux@gmail.com>
Cc: linux-crypto@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>,
"Jason A . Donenfeld" <Jason@zx2c4.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge
Date: Sun, 22 Nov 2026 11:55:51 -0800 [thread overview]
Message-ID: <20261122195551.GB2122@sol> (raw)
In-Reply-To: <20261122113934.48a9ac42@pumpkin>
On Sun, Nov 22, 2026 at 11:39:34AM +0000, David Laight wrote:
> On Sun, 15 Nov 2026 00:09:00 -0800
> Eric Biggers <ebiggers@kernel.org> wrote:
>
> > [ This is proposed to supersede my patch
> > "lib/crypto: x86/sha512: Partially roll up sha512_transform_avx()"
> > which was starting to clean up one of the files being dropped. ]
> >
> > Currently, the kernel has SSSE3, AVX, and AVX2 implementations of each
> > of SHA-256 and SHA-512.
> >
> > The AVX implementations of SHA-256 and SHA-512 run only on CPUs that
> > have AVX but do *not* have AVX2. These are all the ones I could find:
> >
> > - Intel Sandy Bridge and Ivy Bridge
> > - AMD Bulldozer, Piledriver, Steamroller, Jaguar, and Puma
> > - Zhaoxin KX-5000 and KX-6000 (possibly SHA-512 only, as these CPUs
> > might be able to run the PHE optimized SHA-256 code)
> > - Maybe some oddly configured VMs
> >
> > Oddly enough, this "AVX" code doesn't actually have much to do with AVX.
> > The only AVX capability it uses is support for 3-operand VEX-coded
> > instructions, which is unlikely to make a huge difference.
>
> Especially if they are the only VEX encoded instructions.
> Switching back and forth is horribly expensive and the change isn't
> 'sticky' on some later Intel cpu and you get the 'hit' for every
> instruction.
> (kernel_fpu_begin() might need a vzeroupper to migitage this.)
>
> >
> > Its actual purpose is mostly unrelated to AVX: it replaces the 'ror'
> > instruction with 'shld'. This is entirely to work around a quirk
> > specific to Sandy Bridge and Ivy Bridge where the 'ror' instruction was
> > very slow for microarchitectural reasons, and 'shld' worked better.
> >
> > This is documented in the contemporaneous paper "Speeding up SHA-1,
> > SHA-256 and SHA-512 on the 2nd Generation Intel Core Processors"
> > (https://ieeexplore.ieee.org/document/6209073). There, the 'shld'
> > workaround was claimed to speed up SHA-256 and SHA-512 by about 20% on
> > Sandy Bridge.
> >
> > However, that was a one-off quirk. And even the paper itself warns
> > about using the 'shld' approach on other microarchitectures.
> >
> > On AMD CPUs, and on Intel CPUs after Sandy and Ivy Bridge, the exact
> > opposite applies: ror is a fast single-cycle and single µop instruction,
> > while shld is either a slow microcoded or multi-µop instruction.
> >
> > From what I can find, this is true even on the older AMD CPUs mentioned
> > above that run this code. Thus this code has likely always been harmful
> > on AMD. (I don't have such a CPU available to test, but for what it's
> > worth, on Zen 5 the AVX SHA-512 code is 58% as fast as the SSSE3 code.
> > Clearly, the 'shld' hack can be quite slow on CPUs it wasn't meant for.)
>
> I've got a bulldozer (or similar) I can run a test on.
> I might just to a quick one for shld v ror.
> I've rewritten my 'rep movs/stos' benchmark and plan to run it on the
> older cpu I have.
> (I need to get something newer than an i7-7 and older than a zen5.)
Thanks. If you do want to benchmark it, you can use sha512_kunit with
CONFIG_CRYPTO_LIB_BENCHMARK=y enabled.
After that, you could also check the performance with the
sha512-ssse3-asm.S optimization patchset applied on top of it
(https://lore.kernel.org/linux-crypto/20261122022030.296468-1-ebiggers@kernel.org/)
My prediction is that performance on Bulldozer goes up slightly when
sha512-avx-asm.S is removed (due to sha512-ssse3-asm.S being faster),
then up slightly more due to the sha512-ssse3-asm.S optimizations.
(Of course, even if the results are more mixed, both patches seem like
the clear way to go anyway for other reasons: reducing the code size,
deleting an entire asm file that we'll no longer have to maintain,
eliminating the huge on-stack buffer that was never cleared, etc...)
- Eric
prev parent reply other threads:[~2026-11-22 19:55 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-15 8:09 [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge Eric Biggers
2026-11-21 19:51 ` Eric Biggers
2026-11-22 11:39 ` David Laight
2026-11-22 19:55 ` Eric Biggers [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122195551.GB2122@sol \
--to=ebiggers@kernel.org \
--cc=Jason@zx2c4.com \
--cc=ardb@kernel.org \
--cc=david.laight.linux@gmail.com \
--cc=herbert@gondor.apana.org.au \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox