From: Eric Biggers <ebiggers@kernel.org>
To: linux-crypto@vger.kernel.org
Cc: Ard Biesheuvel <ardb@kernel.org>,
"Jason A . Donenfeld" <Jason@zx2c4.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge
Date: Sat, 21 Nov 2026 11:51:48 -0800 [thread overview]
Message-ID: <20261121195148.GC1975@quark> (raw)
In-Reply-To: <20261115080900.216434-1-ebiggers@kernel.org>
On Sun, Nov 15, 2026 at 12:09:00AM -0800, Eric Biggers wrote:
> [ This is proposed to supersede my patch
> "lib/crypto: x86/sha512: Partially roll up sha512_transform_avx()"
> which was starting to clean up one of the files being dropped. ]
>
> Currently, the kernel has SSSE3, AVX, and AVX2 implementations of each
> of SHA-256 and SHA-512.
>
> The AVX implementations of SHA-256 and SHA-512 run only on CPUs that
> have AVX but do *not* have AVX2. These are all the ones I could find:
>
> - Intel Sandy Bridge and Ivy Bridge
> - AMD Bulldozer, Piledriver, Steamroller, Jaguar, and Puma
> - Zhaoxin KX-5000 and KX-6000 (possibly SHA-512 only, as these CPUs
> might be able to run the PHE optimized SHA-256 code)
> - Maybe some oddly configured VMs
>
> Oddly enough, this "AVX" code doesn't actually have much to do with AVX.
> The only AVX capability it uses is support for 3-operand VEX-coded
> instructions, which is unlikely to make a huge difference.
>
> Its actual purpose is mostly unrelated to AVX: it replaces the 'ror'
> instruction with 'shld'. This is entirely to work around a quirk
> specific to Sandy Bridge and Ivy Bridge where the 'ror' instruction was
> very slow for microarchitectural reasons, and 'shld' worked better.
>
> This is documented in the contemporaneous paper "Speeding up SHA-1,
> SHA-256 and SHA-512 on the 2nd Generation Intel Core Processors"
> (https://ieeexplore.ieee.org/document/6209073). There, the 'shld'
> workaround was claimed to speed up SHA-256 and SHA-512 by about 20% on
> Sandy Bridge.
>
> However, that was a one-off quirk. And even the paper itself warns
> about using the 'shld' approach on other microarchitectures.
>
> On AMD CPUs, and on Intel CPUs after Sandy and Ivy Bridge, the exact
> opposite applies: ror is a fast single-cycle and single µop instruction,
> while shld is either a slow microcoded or multi-µop instruction.
>
> From what I can find, this is true even on the older AMD CPUs mentioned
> above that run this code. Thus this code has likely always been harmful
> on AMD. (I don't have such a CPU available to test, but for what it's
> worth, on Zen 5 the AVX SHA-512 code is 58% as fast as the SSSE3 code.
> Clearly, the 'shld' hack can be quite slow on CPUs it wasn't meant for.)
>
> We generally shouldn't be carrying asm files that are one-offs for
> single microarchitectures, as it's just not worth it. This is
> especially true when the code has been mistakenly targeted to other CPUs
> on which it is likely harmful, and when it has other issues that would
> require maintenance, such as excessive loop unrolling and spilling the
> entire 640-byte SHA-512 message schedule to the stack. (The SSSE3 code
> has the same issues, but it's easier to fix it in one place.)
>
> Therefore, let's drop sha256-avx-asm.S and sha512-avx-asm.S.
>
> The CPUs that could have run this code will instead run
> sha256-ssse3-asm.S and sha512-ssse3-asm.S. SHA-256 and SHA-512 may get
> around 20% slower on Sandy and Ivy Bridge, but they likely get
> significantly faster on AMD CPUs from around the same time (2011-2014).
>
> Everything with AVX2 continues to run the newer code and is entirely
> unaffected by this cleanup.
>
> Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> ---
> lib/crypto/Makefile | 2 -
> lib/crypto/x86/sha256-avx-asm.S | 493 --------------------------------
> lib/crypto/x86/sha256.h | 12 +-
> lib/crypto/x86/sha512-avx-asm.S | 421 ---------------------------
> lib/crypto/x86/sha512.h | 14 +-
> 5 files changed, 6 insertions(+), 936 deletions(-)
> delete mode 100644 lib/crypto/x86/sha256-avx-asm.S
> delete mode 100644 lib/crypto/x86/sha512-avx-asm.S
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next
Acks appreciated, but I think this is very well justified.
- Eric
prev parent reply other threads:[~2026-11-21 19:51 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-15 8:09 [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge Eric Biggers
2026-11-21 19:51 ` Eric Biggers [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121195148.GC1975@quark \
--to=ebiggers@kernel.org \
--cc=Jason@zx2c4.com \
--cc=ardb@kernel.org \
--cc=herbert@gondor.apana.org.au \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox