linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Eric Biggers <ebiggers@kernel.org>
To: linux-crypto@vger.kernel.org
Cc: Ard Biesheuvel <ardb@kernel.org>,
	"Jason A . Donenfeld" <Jason@zx2c4.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	x86@kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge
Date: Sat, 21 Nov 2026 11:51:48 -0800	[thread overview]
Message-ID: <20261121195148.GC1975@quark> (raw)
In-Reply-To: <20261115080900.216434-1-ebiggers@kernel.org>

On Sun, Nov 15, 2026 at 12:09:00AM -0800, Eric Biggers wrote:
> [ This is proposed to supersede my patch
>   "lib/crypto: x86/sha512: Partially roll up sha512_transform_avx()"
>   which was starting to clean up one of the files being dropped. ]
> 
> Currently, the kernel has SSSE3, AVX, and AVX2 implementations of each
> of SHA-256 and SHA-512.
> 
> The AVX implementations of SHA-256 and SHA-512 run only on CPUs that
> have AVX but do *not* have AVX2.  These are all the ones I could find:
> 
>     - Intel Sandy Bridge and Ivy Bridge
>     - AMD Bulldozer, Piledriver, Steamroller, Jaguar, and Puma
>     - Zhaoxin KX-5000 and KX-6000 (possibly SHA-512 only, as these CPUs
>       might be able to run the PHE optimized SHA-256 code)
>     - Maybe some oddly configured VMs
> 
> Oddly enough, this "AVX" code doesn't actually have much to do with AVX.
> The only AVX capability it uses is support for 3-operand VEX-coded
> instructions, which is unlikely to make a huge difference.
> 
> Its actual purpose is mostly unrelated to AVX: it replaces the 'ror'
> instruction with 'shld'.  This is entirely to work around a quirk
> specific to Sandy Bridge and Ivy Bridge where the 'ror' instruction was
> very slow for microarchitectural reasons, and 'shld' worked better.
> 
> This is documented in the contemporaneous paper "Speeding up SHA-1,
> SHA-256 and SHA-512 on the 2nd Generation Intel Core Processors"
> (https://ieeexplore.ieee.org/document/6209073).  There, the 'shld'
> workaround was claimed to speed up SHA-256 and SHA-512 by about 20% on
> Sandy Bridge.
> 
> However, that was a one-off quirk.  And even the paper itself warns
> about using the 'shld' approach on other microarchitectures.
> 
> On AMD CPUs, and on Intel CPUs after Sandy and Ivy Bridge, the exact
> opposite applies: ror is a fast single-cycle and single µop instruction,
> while shld is either a slow microcoded or multi-µop instruction.
> 
> From what I can find, this is true even on the older AMD CPUs mentioned
> above that run this code.  Thus this code has likely always been harmful
> on AMD.  (I don't have such a CPU available to test, but for what it's
> worth, on Zen 5 the AVX SHA-512 code is 58% as fast as the SSSE3 code.
> Clearly, the 'shld' hack can be quite slow on CPUs it wasn't meant for.)
> 
> We generally shouldn't be carrying asm files that are one-offs for
> single microarchitectures, as it's just not worth it.  This is
> especially true when the code has been mistakenly targeted to other CPUs
> on which it is likely harmful, and when it has other issues that would
> require maintenance, such as excessive loop unrolling and spilling the
> entire 640-byte SHA-512 message schedule to the stack.  (The SSSE3 code
> has the same issues, but it's easier to fix it in one place.)
> 
> Therefore, let's drop sha256-avx-asm.S and sha512-avx-asm.S.
> 
> The CPUs that could have run this code will instead run
> sha256-ssse3-asm.S and sha512-ssse3-asm.S.  SHA-256 and SHA-512 may get
> around 20% slower on Sandy and Ivy Bridge, but they likely get
> significantly faster on AMD CPUs from around the same time (2011-2014).
> 
> Everything with AVX2 continues to run the newer code and is entirely
> unaffected by this cleanup.
> 
> Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> ---
>  lib/crypto/Makefile             |   2 -
>  lib/crypto/x86/sha256-avx-asm.S | 493 --------------------------------
>  lib/crypto/x86/sha256.h         |  12 +-
>  lib/crypto/x86/sha512-avx-asm.S | 421 ---------------------------
>  lib/crypto/x86/sha512.h         |  14 +-
>  5 files changed, 6 insertions(+), 936 deletions(-)
>  delete mode 100644 lib/crypto/x86/sha256-avx-asm.S
>  delete mode 100644 lib/crypto/x86/sha512-avx-asm.S

Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next

Acks appreciated, but I think this is very well justified.

- Eric

      reply	other threads:[~2026-11-21 19:51 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-15  8:09 [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge Eric Biggers
2026-11-21 19:51 ` Eric Biggers [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121195148.GC1975@quark \
    --to=ebiggers@kernel.org \
    --cc=Jason@zx2c4.com \
    --cc=ardb@kernel.org \
    --cc=herbert@gondor.apana.org.au \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).