[PATCH 00/17] SHA-3 library

[PATCH 00/17] SHA-3 library

[Eric Biggers]

This series is targeting libcrypto-next.  It can also be retrieved from:

    git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha3-lib-v1

This series builds on the first 5 patches of David's v6 series
(https://lore.kernel.org/r/20251017144311.817771-1-dhowells@redhat.com),
which I've included unmodified as the initial patches.  The remaining
patches improve the SHA-3 library, for example by simplifying the API
and migrating the existing arm64 and s390-optimized SHA-3 code into the
library.  The last patch reimplements the sha3-* crypto_shash algorithms
on top of the library.

If the s390 folks could re-test the s390 optimized SHA-3 code, that
would be helpful.  QEMU doesn't support the instructions it uses.

David Howells (5):
  s390/sha3: Rename conflicting functions
  arm64/sha3: Rename conflicting functions
  lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128,
    SHAKE256
  lib/crypto: Move the SHA3 Iota transform into the single round
    function
  lib/crypto: Add SHA3 kunit tests

Eric Biggers (12):
  lib/crypto: sha3: Fix libsha3 build condition
  lib/crypto: sha3: Use appropriate conversions in
    sha3_keccakf_generic()
  lib/crypto: sha3: Drop unfinished SHAKE support from
    gen-hash-testvecs.py
  lib/crypto: sha3: Consistently use EXPORT_SYMBOL_GPL
  lib/crypto: sha3: Replace redundant ad-hoc test with FIPS test
  lib/crypto: sha3: Simplify the API
  lib/crypto: sha3: Document one-shot functions in header and improve
    docs
  crypto: arm64/sha3 - Update sha3_ce_transform() to prepare for library
  lib/crypto: arm64/sha3: Migrate optimized code into library
  lib/crypto: s390/sha3: Migrate optimized code into library
  crypto: jitterentropy - use default sha3 implementation
  crypto: sha3 - Reimplement using library API

 Documentation/crypto/index.rst                |   1 +
 Documentation/crypto/sha3.rst                 | 147 +++++++
 arch/arm64/configs/defconfig                  |   2 +-
 arch/arm64/crypto/Kconfig                     |  11 -
 arch/arm64/crypto/Makefile                    |   3 -
 arch/arm64/crypto/sha3-ce-glue.c              | 151 -------
 arch/s390/configs/debug_defconfig             |   3 +-
 arch/s390/configs/defconfig                   |   3 +-
 arch/s390/crypto/Kconfig                      |  20 -
 arch/s390/crypto/Makefile                     |   2 -
 arch/s390/crypto/sha.h                        |  51 ---
 arch/s390/crypto/sha3_256_s390.c              | 157 --------
 arch/s390/crypto/sha3_512_s390.c              | 157 --------
 arch/s390/crypto/sha_common.c                 | 117 ------
 crypto/Makefile                               |   2 +-
 crypto/jitterentropy-kcapi.c                  |  12 +-
 crypto/sha3.c                                 | 172 ++++++++
 crypto/sha3_generic.c                         | 290 -------------
 crypto/testmgr.c                              |   8 +
 include/crypto/sha3.h                         | 314 ++++++++++++++-
 lib/crypto/Kconfig                            |  13 +
 lib/crypto/Makefile                           |  10 +
 .../crypto/arm64}/sha3-ce-core.S              |  66 +--
 lib/crypto/arm64/sha3.h                       |  64 +++
 lib/crypto/fips.h                             |   7 +
 lib/crypto/s390/sha3.h                        |  76 ++++
 lib/crypto/sha3.c                             | 380 ++++++++++++++++++
 lib/crypto/tests/Kconfig                      |  11 +
 lib/crypto/tests/Makefile                     |   1 +
 lib/crypto/tests/sha3_kunit.c                 | 344 ++++++++++++++++
 lib/crypto/tests/sha3_testvecs.h              | 231 +++++++++++
 scripts/crypto/gen-fips-testvecs.py           |   4 +
 scripts/crypto/gen-hash-testvecs.py           |   6 +-
 33 files changed, 1822 insertions(+), 1014 deletions(-)
 create mode 100644 Documentation/crypto/sha3.rst
 delete mode 100644 arch/arm64/crypto/sha3-ce-glue.c
 delete mode 100644 arch/s390/crypto/sha.h
 delete mode 100644 arch/s390/crypto/sha3_256_s390.c
 delete mode 100644 arch/s390/crypto/sha3_512_s390.c
 delete mode 100644 arch/s390/crypto/sha_common.c
 create mode 100644 crypto/sha3.c
 delete mode 100644 crypto/sha3_generic.c
 rename {arch/arm64/crypto => lib/crypto/arm64}/sha3-ce-core.S (84%)
 create mode 100644 lib/crypto/arm64/sha3.h
 create mode 100644 lib/crypto/s390/sha3.h
 create mode 100644 lib/crypto/sha3.c
 create mode 100644 lib/crypto/tests/sha3_kunit.c
 create mode 100644 lib/crypto/tests/sha3_testvecs.h

base-commit: 123fa1574bccee87da735d13e89c931e88288b40
-- 
2.51.1.dirty

[PATCH 01/17] s390/sha3: Rename conflicting functions

[Eric Biggers]

From: David Howells <dhowells@redhat.com>

Rename the s390 sha3_* functions to have an "s390_" prefix to avoid
conflict with generic code.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-By: Harald Freudenberger <freude@linux.ibm.com>
cc: Eric Biggers <ebiggers@kernel.org>
cc: "Jason A. Donenfeld" <Jason@zx2c4.com>
cc: Ard Biesheuvel <ardb@kernel.org>
cc: Holger Dengler <dengler@linux.ibm.com>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: Stephan Mueller <smueller@chronox.de>
cc: linux-crypto@vger.kernel.org
cc: linux-s390@vger.kernel.org
Link: https://lore.kernel.org/r/20251017144311.817771-2-dhowells@redhat.com
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 arch/s390/crypto/sha3_256_s390.c | 26 +++++++++++++-------------
 arch/s390/crypto/sha3_512_s390.c | 26 +++++++++++++-------------
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/arch/s390/crypto/sha3_256_s390.c b/arch/s390/crypto/sha3_256_s390.c
index 03bb4f4bab701..fd5ecae60a572 100644
--- a/arch/s390/crypto/sha3_256_s390.c
+++ b/arch/s390/crypto/sha3_256_s390.c
@@ -17,11 +17,11 @@
 #include <linux/module.h>
 #include <linux/string.h>
 
 #include "sha.h"
 
-static int sha3_256_init(struct shash_desc *desc)
+static int s390_sha3_256_init(struct shash_desc *desc)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 
 	sctx->first_message_part = test_facility(86);
 	if (!sctx->first_message_part)
@@ -30,11 +30,11 @@ static int sha3_256_init(struct shash_desc *desc)
 	sctx->func = CPACF_KIMD_SHA3_256;
 
 	return 0;
 }
 
-static int sha3_256_export(struct shash_desc *desc, void *out)
+static int s390_sha3_256_export(struct shash_desc *desc, void *out)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 	union {
 		u8 *u8;
 		u64 *u64;
@@ -48,11 +48,11 @@ static int sha3_256_export(struct shash_desc *desc, void *out)
 	for (i = 0; i < SHA3_STATE_SIZE / 8; i++)
 		put_unaligned(le64_to_cpu(sctx->sha3.state[i]), p.u64++);
 	return 0;
 }
 
-static int sha3_256_import(struct shash_desc *desc, const void *in)
+static int s390_sha3_256_import(struct shash_desc *desc, const void *in)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 	union {
 		const u8 *u8;
 		const u64 *u64;
@@ -66,26 +66,26 @@ static int sha3_256_import(struct shash_desc *desc, const void *in)
 	sctx->func = CPACF_KIMD_SHA3_256;
 
 	return 0;
 }
 
-static int sha3_224_import(struct shash_desc *desc, const void *in)
+static int s390_sha3_224_import(struct shash_desc *desc, const void *in)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 
-	sha3_256_import(desc, in);
+	s390_sha3_256_import(desc, in);
 	sctx->func = CPACF_KIMD_SHA3_224;
 	return 0;
 }
 
 static struct shash_alg sha3_256_alg = {
 	.digestsize	=	SHA3_256_DIGEST_SIZE,	   /* = 32 */
-	.init		=	sha3_256_init,
+	.init		=	s390_sha3_256_init,
 	.update		=	s390_sha_update_blocks,
 	.finup		=	s390_sha_finup,
-	.export		=	sha3_256_export,
-	.import		=	sha3_256_import,
+	.export		=	s390_sha3_256_export,
+	.import		=	s390_sha3_256_import,
 	.descsize	=	S390_SHA_CTX_SIZE,
 	.statesize	=	SHA3_STATE_SIZE,
 	.base		=	{
 		.cra_name	 =	"sha3-256",
 		.cra_driver_name =	"sha3-256-s390",
@@ -94,26 +94,26 @@ static struct shash_alg sha3_256_alg = {
 		.cra_blocksize	 =	SHA3_256_BLOCK_SIZE,
 		.cra_module	 =	THIS_MODULE,
 	}
 };
 
-static int sha3_224_init(struct shash_desc *desc)
+static int s390_sha3_224_init(struct shash_desc *desc)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 
-	sha3_256_init(desc);
+	s390_sha3_256_init(desc);
 	sctx->func = CPACF_KIMD_SHA3_224;
 	return 0;
 }
 
 static struct shash_alg sha3_224_alg = {
 	.digestsize	=	SHA3_224_DIGEST_SIZE,
-	.init		=	sha3_224_init,
+	.init		=	s390_sha3_224_init,
 	.update		=	s390_sha_update_blocks,
 	.finup		=	s390_sha_finup,
-	.export		=	sha3_256_export, /* same as for 256 */
-	.import		=	sha3_224_import, /* function code different! */
+	.export		=	s390_sha3_256_export, /* same as for 256 */
+	.import		=	s390_sha3_224_import, /* function code different! */
 	.descsize	=	S390_SHA_CTX_SIZE,
 	.statesize	=	SHA3_STATE_SIZE,
 	.base		=	{
 		.cra_name	 =	"sha3-224",
 		.cra_driver_name =	"sha3-224-s390",
diff --git a/arch/s390/crypto/sha3_512_s390.c b/arch/s390/crypto/sha3_512_s390.c
index a5c9690eecb19..f4b52a3a04339 100644
--- a/arch/s390/crypto/sha3_512_s390.c
+++ b/arch/s390/crypto/sha3_512_s390.c
@@ -16,11 +16,11 @@
 #include <linux/module.h>
 #include <linux/string.h>
 
 #include "sha.h"
 
-static int sha3_512_init(struct shash_desc *desc)
+static int s390_sha3_512_init(struct shash_desc *desc)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 
 	sctx->first_message_part = test_facility(86);
 	if (!sctx->first_message_part)
@@ -29,11 +29,11 @@ static int sha3_512_init(struct shash_desc *desc)
 	sctx->func = CPACF_KIMD_SHA3_512;
 
 	return 0;
 }
 
-static int sha3_512_export(struct shash_desc *desc, void *out)
+static int s390_sha3_512_export(struct shash_desc *desc, void *out)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 	union {
 		u8 *u8;
 		u64 *u64;
@@ -47,11 +47,11 @@ static int sha3_512_export(struct shash_desc *desc, void *out)
 	for (i = 0; i < SHA3_STATE_SIZE / 8; i++)
 		put_unaligned(le64_to_cpu(sctx->sha3.state[i]), p.u64++);
 	return 0;
 }
 
-static int sha3_512_import(struct shash_desc *desc, const void *in)
+static int s390_sha3_512_import(struct shash_desc *desc, const void *in)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 	union {
 		const u8 *u8;
 		const u64 *u64;
@@ -65,26 +65,26 @@ static int sha3_512_import(struct shash_desc *desc, const void *in)
 	sctx->func = CPACF_KIMD_SHA3_512;
 
 	return 0;
 }
 
-static int sha3_384_import(struct shash_desc *desc, const void *in)
+static int s390_sha3_384_import(struct shash_desc *desc, const void *in)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 
-	sha3_512_import(desc, in);
+	s390_sha3_512_import(desc, in);
 	sctx->func = CPACF_KIMD_SHA3_384;
 	return 0;
 }
 
 static struct shash_alg sha3_512_alg = {
 	.digestsize	=	SHA3_512_DIGEST_SIZE,
-	.init		=	sha3_512_init,
+	.init		=	s390_sha3_512_init,
 	.update		=	s390_sha_update_blocks,
 	.finup		=	s390_sha_finup,
-	.export		=	sha3_512_export,
-	.import		=	sha3_512_import,
+	.export		=	s390_sha3_512_export,
+	.import		=	s390_sha3_512_import,
 	.descsize	=	S390_SHA_CTX_SIZE,
 	.statesize	=	SHA3_STATE_SIZE,
 	.base		=	{
 		.cra_name	 =	"sha3-512",
 		.cra_driver_name =	"sha3-512-s390",
@@ -95,26 +95,26 @@ static struct shash_alg sha3_512_alg = {
 	}
 };
 
 MODULE_ALIAS_CRYPTO("sha3-512");
 
-static int sha3_384_init(struct shash_desc *desc)
+static int s390_sha3_384_init(struct shash_desc *desc)
 {
 	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
 
-	sha3_512_init(desc);
+	s390_sha3_512_init(desc);
 	sctx->func = CPACF_KIMD_SHA3_384;
 	return 0;
 }
 
 static struct shash_alg sha3_384_alg = {
 	.digestsize	=	SHA3_384_DIGEST_SIZE,
-	.init		=	sha3_384_init,
+	.init		=	s390_sha3_384_init,
 	.update		=	s390_sha_update_blocks,
 	.finup		=	s390_sha_finup,
-	.export		=	sha3_512_export, /* same as for 512 */
-	.import		=	sha3_384_import, /* function code different! */
+	.export		=	s390_sha3_512_export, /* same as for 512 */
+	.import		=	s390_sha3_384_import, /* function code different! */
 	.descsize	=	S390_SHA_CTX_SIZE,
 	.statesize	=	SHA3_STATE_SIZE,
 	.base		=	{
 		.cra_name	 =	"sha3-384",
 		.cra_driver_name =	"sha3-384-s390",
-- 
2.51.1.dirty

[PATCH 02/17] arm64/sha3: Rename conflicting functions

[Eric Biggers]

From: David Howells <dhowells@redhat.com>

Rename the arm64 sha3_* functions to have an "arm64_" prefix to avoid
conflict with generic code.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: Eric Biggers <ebiggers@kernel.org>
cc: "Jason A. Donenfeld" <Jason@zx2c4.com>
cc: Ard Biesheuvel <ardb@kernel.org>
cc: Catalin Marinas <catalin.marinas@arm.com>
cc: Will Deacon <will@kernel.org>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: Stephan Mueller <smueller@chronox.de>
cc: linux-crypto@vger.kernel.org
cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20251017144311.817771-3-dhowells@redhat.com
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 arch/arm64/crypto/sha3-ce-glue.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/crypto/sha3-ce-glue.c b/arch/arm64/crypto/sha3-ce-glue.c
index b4f1001046c9a..426d8044535ab 100644
--- a/arch/arm64/crypto/sha3-ce-glue.c
+++ b/arch/arm64/crypto/sha3-ce-glue.c
@@ -29,11 +29,11 @@ MODULE_ALIAS_CRYPTO("sha3-384");
 MODULE_ALIAS_CRYPTO("sha3-512");
 
 asmlinkage int sha3_ce_transform(u64 *st, const u8 *data, int blocks,
 				 int md_len);
 
-static int sha3_update(struct shash_desc *desc, const u8 *data,
+static int arm64_sha3_update(struct shash_desc *desc, const u8 *data,
 		       unsigned int len)
 {
 	struct sha3_state *sctx = shash_desc_ctx(desc);
 	struct crypto_shash *tfm = desc->tfm;
 	unsigned int bs, ds;
@@ -53,12 +53,12 @@ static int sha3_update(struct shash_desc *desc, const u8 *data,
 		blocks = rem;
 	} while (blocks);
 	return len;
 }
 
-static int sha3_finup(struct shash_desc *desc, const u8 *src, unsigned int len,
-		      u8 *out)
+static int arm64_sha3_finup(struct shash_desc *desc, const u8 *src, unsigned int len,
+			    u8 *out)
 {
 	struct sha3_state *sctx = shash_desc_ctx(desc);
 	struct crypto_shash *tfm = desc->tfm;
 	__le64 *digest = (__le64 *)out;
 	u8 block[SHA3_224_BLOCK_SIZE];
@@ -88,48 +88,48 @@ static int sha3_finup(struct shash_desc *desc, const u8 *src, unsigned int len,
 }
 
 static struct shash_alg algs[] = { {
 	.digestsize		= SHA3_224_DIGEST_SIZE,
 	.init			= crypto_sha3_init,
-	.update			= sha3_update,
-	.finup			= sha3_finup,
+	.update			= arm64_sha3_update,
+	.finup			= arm64_sha3_finup,
 	.descsize		= SHA3_STATE_SIZE,
 	.base.cra_name		= "sha3-224",
 	.base.cra_driver_name	= "sha3-224-ce",
 	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
 	.base.cra_blocksize	= SHA3_224_BLOCK_SIZE,
 	.base.cra_module	= THIS_MODULE,
 	.base.cra_priority	= 200,
 }, {
 	.digestsize		= SHA3_256_DIGEST_SIZE,
 	.init			= crypto_sha3_init,
-	.update			= sha3_update,
-	.finup			= sha3_finup,
+	.update			= arm64_sha3_update,
+	.finup			= arm64_sha3_finup,
 	.descsize		= SHA3_STATE_SIZE,
 	.base.cra_name		= "sha3-256",
 	.base.cra_driver_name	= "sha3-256-ce",
 	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
 	.base.cra_blocksize	= SHA3_256_BLOCK_SIZE,
 	.base.cra_module	= THIS_MODULE,
 	.base.cra_priority	= 200,
 }, {
 	.digestsize		= SHA3_384_DIGEST_SIZE,
 	.init			= crypto_sha3_init,
-	.update			= sha3_update,
-	.finup			= sha3_finup,
+	.update			= arm64_sha3_update,
+	.finup			= arm64_sha3_finup,
 	.descsize		= SHA3_STATE_SIZE,
 	.base.cra_name		= "sha3-384",
 	.base.cra_driver_name	= "sha3-384-ce",
 	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
 	.base.cra_blocksize	= SHA3_384_BLOCK_SIZE,
 	.base.cra_module	= THIS_MODULE,
 	.base.cra_priority	= 200,
 }, {
 	.digestsize		= SHA3_512_DIGEST_SIZE,
 	.init			= crypto_sha3_init,
-	.update			= sha3_update,
-	.finup			= sha3_finup,
+	.update			= arm64_sha3_update,
+	.finup			= arm64_sha3_finup,
 	.descsize		= SHA3_STATE_SIZE,
 	.base.cra_name		= "sha3-512",
 	.base.cra_driver_name	= "sha3-512-ce",
 	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
 	.base.cra_blocksize	= SHA3_512_BLOCK_SIZE,
-- 
2.51.1.dirty

[PATCH 03/17] lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256

[Eric Biggers]

From: David Howells <dhowells@redhat.com>

Add SHA3, providing SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128 and
SHAKE256 to lib/crypto.

Notes:

 (1) I've left hooks in sha3.c for asm-optimised variants, but as I don't
     entirely know what those might look like, not having implemented any,
     the hooks' usability is uncertain.

 (2) The SHAKE algorithms will be required for ML-DSA.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: Eric Biggers <ebiggers@kernel.org>
cc: Jason A. Donenfeld <Jason@zx2c4.com>
cc: Ard Biesheuvel <ardb@kernel.org>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: Stephan Mueller <smueller@chronox.de>
cc: linux-crypto@vger.kernel.org

Changes
=======
v4)
 - Doc fixes:
   - Fix the description of the algorithm to be closer to the NIST spec's
     terminology.
   - Don't talk of finialising the context for XOFs.
   - Don't say "Return: None".
   - Declare the "Context" to be "Any context" and make no mention of the
     fact that it might use the FPU.
   - Change "initialise" to "initialize".
   - Don't warn that the context is relatively large for stack use.
 - Use size_t for size parameters/variables.
 - Make the module_exit unconditional.

v3)
 - Split the s390 function rename out to a separate patch.

v2)
 - Simplify things by keeping the state array in LE form and byteswapping
   all the 64-bit words before and after applying the keccak function on a
   BE system.  This means no byteswapping is required when XOR'ing data
   into the state array or when extracting the digest.  Further, this is a
   no-op on LE systems.

 - Rename sha3_final() to sha3_squeeze() and don't clear the context at the
   end as it's permitted to continue calling sha3_final() to extract
   continuations of the digest (needed by ML-DSA).

 - Don't reapply the end marker to the hash state in continuation
   sha3_squeeze() unless sha3_update() gets called again (needed by
   ML-DSA).

 - Give sha3_squeeze() the amount of digest to produce as a parameter
   rather than using ctx->digest_size and don't return the amount digested.

 - Reimplement sha3_final() as a wrapper around sha3_squeeze() that
   extracts ctx->digest_size amount of digest and then zeroes out the
   context.  The latter is necessary to avoid upsetting
   hash-test-template.h.

 - Provide a sha3_reinit() function to clear the state, but to leave the
   parameters that indicate the hash properties unaffected, allowing for
   reuse.

 - Provide a sha3_set_digestsize() function to change the size of the
   digest to be extracted by sha3_final().  sha3_squeeze() takes a
   parameter for this instead.

 - Don't pass the digest size as a parameter to shake128/256_init() but
   rather default to 128/256 bits as per the function name.

- Provide a sha3_clear() function to zero out the context.

Link: https://lore.kernel.org/r/20251017144311.817771-4-dhowells@redhat.com
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 Documentation/crypto/index.rst |   1 +
 Documentation/crypto/sha3.rst  | 241 ++++++++++++++++
 include/crypto/sha3.h          | 433 +++++++++++++++++++++++++++-
 lib/crypto/Kconfig             |   7 +
 lib/crypto/Makefile            |   6 +
 lib/crypto/sha3.c              | 511 +++++++++++++++++++++++++++++++++
 6 files changed, 1198 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/crypto/sha3.rst
 create mode 100644 lib/crypto/sha3.c

diff --git a/Documentation/crypto/index.rst b/Documentation/crypto/index.rst
index 100b47d049c04..4ee667c446f99 100644
--- a/Documentation/crypto/index.rst
+++ b/Documentation/crypto/index.rst
@@ -25,5 +25,6 @@ for cryptographic use cases, as well as programming examples.
    api
    api-samples
    descore-readme
    device_drivers/index
    krb5
+   sha3
diff --git a/Documentation/crypto/sha3.rst b/Documentation/crypto/sha3.rst
new file mode 100644
index 0000000000000..3255618ea2ec3
--- /dev/null
+++ b/Documentation/crypto/sha3.rst
@@ -0,0 +1,241 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+==========================
+SHA-3 Algorithm collection
+==========================
+
+.. Contents:
+
+  - Overview
+  - Basic API
+    - Extendable-Output Functions
+  - Convenience API
+  - Internal API
+    - Testing
+  - References
+  - API Function Reference
+
+
+Overview
+========
+
+The SHA-3 algorithm base, as specified in NIST FIPS-202[1], provides a number
+of specific variants all based on the same basic algorithm (the Keccak sponge
+function and permutation).  The differences between them are: the "rate" (how
+much of the common state buffer gets updated with new data between invocations
+of the Keccak function and analogous to the "block size"), what domain
+separation suffix/padding gets appended to the message and how much data is
+extracted at the end.  The Keccak sponge function is designed such that
+arbitrary amounts of output can be obtained for certain algorithms.
+
+Four standard digest algorithms are provided:
+
+ - SHA3-224
+ - SHA3-256
+ - SHA3-384
+ - SHA3-512
+
+and two Extendable-Output Functions (XOF):
+
+ - SHAKE128
+ - SHAKE256
+
+If selectable algorithms are required then the crypto_hash API may be used
+instead as this binds each algorithm to a specific C type.
+
+
+Basic API
+=========
+
+The basic API has a separate context struct for each algorithm in the SHA3
+suite, none of the contents of which are expected to be accessed directly::
+
+	struct sha3_224_ctx { ... };
+	struct sha3_256_ctx { ... };
+	struct sha3_384_ctx { ... };
+	struct sha3_512_ctx { ... };
+	struct shake128_ctx { ... };
+	struct shake256_ctx { ... };
+
+There are a collection of initialisation functions, one for each algorithm
+supported, that initialise the context appropriately for that algorithm::
+
+	void sha3_224_init(struct sha3_224_ctx *ctx);
+	void sha3_256_init(struct sha3_256_ctx *ctx);
+	void sha3_384_init(struct sha3_384_ctx *ctx);
+	void sha3_512_init(struct sha3_512_ctx *ctx);
+	void shake128_init(struct shake128_ctx *ctx);
+	void shake256_init(struct shake256_ctx *ctx);
+
+Data is then added with the appropriate update function, again one per
+algorithm::
+
+	void sha3_224_update(struct sha3_224_ctx *ctx,
+			     const u8 *data, size_t len);
+	void sha3_256_update(struct sha3_256_ctx *ctx,
+			     const u8 *data, size_t len);
+	void sha3_384_update(struct sha3_384_ctx *ctx,
+			     const u8 *data, size_t len);
+	void sha3_512_update(struct sha3_512_ctx *ctx,
+			     const u8 *data, size_t len);
+	void shake128_update(struct shake128_ctx *ctx,
+			     const u8 *data, size_t len);
+	void shake256_update(struct shake256_ctx *ctx,
+			     const u8 *data, size_t len);
+
+The update function may be called multiple times if need be to add
+non-contiguous data.
+
+For digest algorithms, the digest is finalised and extracted with the
+algorithm-specific function::
+
+	void sha3_224_final(struct sha3_224_ctx *ctx,
+			    u8 out[SHA3_224_DIGEST_SIZE]);
+	void sha3_256_final(struct sha3_256_ctx *ctx,
+			    u8 out[SHA3_256_DIGEST_SIZE]);
+	void sha3_384_final(struct sha3_384_ctx *ctx,
+			    u8 out[SHA3_384_DIGEST_SIZE]);
+	void sha3_512_final(struct sha3_512_ctx *ctx,
+			    u8 out[SHA3_512_DIGEST_SIZE]);
+
+which also explicitly clears the context.  The amount of data extracted is
+determined by the type.
+
+
+Extendable-Output Functions
+---------------------------
+
+For XOFs, once the data has been added to a context, a variable amount of data
+may be extracted.  This can be done by calling the appropriate squeeze
+function::
+
+	void shake128_squeeze(struct shake128_ctx *ctx, u8 *out, size_t out_len);
+	void shake256_squeeze(struct shake256_ctx *ctx, u8 *out, size_t out_len);
+
+and telling it how much data should be extracted.  The squeeze function may be
+called multiple times but it will only append the domain separation suffix on
+the first invocation.
+
+Note that performing a number of squeezes, with the output laid consequitively
+in a buffer, gets exactly the same output as doing a single squeeze for the
+combined amount over the same buffer.
+
+Once all the desired output has been extracted, the context should be cleared
+with the clear function appropriate to the algorithm::
+
+	void shake128_clear(struct shake128_ctx *ctx);
+	void shake256_clear(struct shake256_ctx *ctx);
+
+
+Convenience API
+===============
+
+It only a single contiguous buffer of input needs to be added and only a single
+buffer of digest or XOF output is required, then a convenience API is provided
+that wraps all the required steps into a single function.  There is one
+function for each algorithm supported::
+
+	void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE]);
+	void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE]);
+	void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE]);
+	void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE]);
+	void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len);
+	void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len);
+
+
+Internal API
+============
+
+There is a common internal API underlying all of this that may be used to build
+further algorithms or APIs as the engine in the same in all cases.  The
+algorithm APIs all wrap the common context structure::
+
+	struct sha3_ctx {
+		struct sha3_state	state;
+		u8			block_size;
+		u8			padding;
+		u8			absorb_offset;
+		u8			squeeze_offset;
+		bool			end_marked;
+	};
+
+	struct sha3_state {
+		u64			st[SHA3_STATE_SIZE / 8];
+	};
+
+The fields are as follows:
+
+ * ``state.st``
+
+   An array of 25 64-bit state buckets that are used to hold the mathematical
+   state of the Keccak engine.  Data is XOR'd onto part of this, the engine is
+   cranked and then the output is copied from this.
+
+   For the convenience of adding input and extract output from it, the array is
+   kept in little-endian order most of the time, but is byteswapped to
+   host-endian in order to perform the Keccak function and then byteswapped
+   back again.  On an LE machine, the byteswapping is a no-op.
+
+ * ``block_size``
+
+   The size of the block of state that can be updated or extracted at a time.
+   This is related to the algorithm size and is analogous to the "rate" in the
+   algorithm definition.
+
+ * ``padding``
+
+   The terminating byte to add when finalising the stat.  This may differ
+   between algorithms.
+
+ * ``absorb_offset``
+
+   This tracks which is the next byte of state to be updated; when it hits
+   ``block_size``, the engine is cranked and this is reset to 0.
+
+ * ``squeeze_offset``
+
+   This tracks which is the next byte of state to be extracted; similar to
+   ``partial``, when it hits ``block_size``, if more output is requested, the
+   engine is cranked to generate more and this is reset to 0.
+
+ * ``end_marked``
+
+   This is set to true when the domain separation suffix and any padding have
+   been appended to the state to prevent multiple squeezings from XOF
+   algorithms from re-appending this.
+
+Note that the size of the digest is *not* included here as that's only needed
+at finalisation time for digest algorithms and can be supplied then.  It is not
+relevant to XOFs.
+
+To make use of the context, the following internal functions are provided::
+
+	void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len);
+	void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_len);
+	void sha3_clear(struct sha3_ctx *ctx);
+
+These allow data to be appended to/absorbed into the state, output to be
+extracted/squeezed from the state and for the state to be cleared.  Note that
+there is no "final" function, per se, but that can be constructed by squeezing
+and clearing.
+
+
+Testing
+-------
+
+The sha3 module does a basic sanity test on initialisation, but there is also a
+kunit test module available.
+
+
+References
+==========
+
+[1] https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
+
+
+
+API Function Reference
+======================
+
+.. kernel-doc:: crypto/lib/sha3.c
+.. kernel-doc:: include/crypto/sha3.h
diff --git a/include/crypto/sha3.h b/include/crypto/sha3.h
index 41e1b83a6d918..a54117cb1546b 100644
--- a/include/crypto/sha3.h
+++ b/include/crypto/sha3.h
@@ -1,13 +1,16 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 /*
  * Common values for SHA-3 algorithms
+ *
+ * See also Documentation/crypto/sha3.rst
  */
 #ifndef __CRYPTO_SHA3_H__
 #define __CRYPTO_SHA3_H__
 
 #include <linux/types.h>
+#include <linux/string.h>
 
 #define SHA3_224_DIGEST_SIZE	(224 / 8)
 #define SHA3_224_BLOCK_SIZE	(200 - 2 * SHA3_224_DIGEST_SIZE)
 #define SHA3_224_EXPORT_SIZE	SHA3_STATE_SIZE + SHA3_224_BLOCK_SIZE + 1
 
@@ -21,16 +24,444 @@
 
 #define SHA3_512_DIGEST_SIZE	(512 / 8)
 #define SHA3_512_BLOCK_SIZE	(200 - 2 * SHA3_512_DIGEST_SIZE)
 #define SHA3_512_EXPORT_SIZE	SHA3_STATE_SIZE + SHA3_512_BLOCK_SIZE + 1
 
+/* SHAKE128 and SHAKE256 actually have variable output size, but this is
+ * used to calculate the block size/rate analogously to the above..
+ */
+#define SHAKE128_DEFAULT_SIZE	(128 / 8)
+#define SHAKE128_BLOCK_SIZE	(200 - 2 * SHAKE128_DEFAULT_SIZE)
+#define SHAKE256_DEFAULT_SIZE	(256 / 8)
+#define SHAKE256_BLOCK_SIZE	(200 - 2 * SHAKE256_DEFAULT_SIZE)
+
 #define SHA3_STATE_SIZE		200
 
 struct shash_desc;
 
 struct sha3_state {
 	u64		st[SHA3_STATE_SIZE / 8];
 };
 
+/*
+ * The SHA3 context structure and state buffer.
+ *
+ * To avoid the need to byteswap when adding input and extracting output from
+ * the state array, the state array is kept in little-endian order most of the
+ * time, but is byteswapped to host-endian in order to perform the Keccak
+ * function and then byteswapped back again after.  On a LE machine, the
+ * byteswap step is a no-op.
+ */
+struct sha3_ctx {
+	struct sha3_state	state;
+	u8			block_size;	/* Block size in bytes */
+	u8			padding;	/* Padding byte */
+	u8			absorb_offset;	/* Next state byte to absorb into */
+	u8			squeeze_offset;	/* Next state byte to extract */
+	bool			end_marked;	/* T if end marker inserted */
+};
+
 int crypto_sha3_init(struct shash_desc *desc);
 
-#endif
+/**
+ * sha3_clear() - Explicitly clear the entire context
+ * @ctx: the context to clear
+ *
+ * Explicitly clear the entire context, including the type parameters; after
+ * this, the context must be fully initialized again.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_clear(struct sha3_ctx *ctx)
+{
+	memzero_explicit(ctx, sizeof(*ctx));
+}
+
+void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len);
+void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_len);
+
+/*
+ * Context wrapper for SHA3-224.
+ */
+struct sha3_224_ctx {
+	struct sha3_ctx ctx;
+};
+
+/**
+ * sha3_224_init() - Set a SHA3 context for SHA3-224
+ * @ctx: the context to initialize
+ *
+ * Initialize a SHA3 context for the production of a SHA3-224 digest of a
+ * message.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_224_init(struct sha3_224_ctx *ctx)
+{
+	*ctx = (struct sha3_224_ctx){
+		.ctx.block_size	= SHA3_224_BLOCK_SIZE,
+		.ctx.padding	= 0x06,
+	};
+}
+
+/**
+ * sha3_224_update() - Update a SHA3-224 hash with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to add data to the hash, performing
+ * the "keccak sponge absorbing" phase.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_224_update(struct sha3_224_ctx *ctx, const u8 *data, size_t len)
+{
+	return sha3_update(&ctx->ctx, data, len);
+}
+
+/**
+ * sha3_224_final() - Finalise a SHA3-224 hash and extract the digest
+ * @ctx: The context to finalise; must have been initialized
+ * @out: Where to write the resulting message digest
+ *
+ * Finish the computation of a SHA3-224 hash and perform the "Keccak sponge
+ * squeezing" phase.  The digest is written to @out buffer and the context will
+ * be completely zeroed out.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_224_final(struct sha3_224_ctx *ctx, u8 out[SHA3_224_DIGEST_SIZE])
+{
+	sha3_squeeze(&ctx->ctx, out, SHA3_224_DIGEST_SIZE);
+	sha3_clear(&ctx->ctx);
+}
+
+/*
+ * Context wrapper for SHA3-256.
+ */
+struct sha3_256_ctx {
+	struct sha3_ctx ctx;
+};
+
+/**
+ * sha3_256_init() - Set a SHA3 context for SHA3-256
+ * @ctx: the context to initialize
+ *
+ * Initialize a SHA3 context for the production of a SHA3-256 digest of a
+ * message.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_256_init(struct sha3_256_ctx *ctx)
+{
+	*ctx = (struct sha3_256_ctx){
+		.ctx.block_size	= SHA3_256_BLOCK_SIZE,
+		.ctx.padding	= 0x06,
+	};
+}
+
+/**
+ * sha3_256_update() - Update a SHA3-256 hash with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to add data to the hash, performing
+ * the "keccak sponge absorbing" phase.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_256_update(struct sha3_256_ctx *ctx, const u8 *data, size_t len)
+{
+	return sha3_update(&ctx->ctx, data, len);
+}
+
+/**
+ * sha3_256_final() - Finalise a SHA3-256 hash and extract the digest
+ * @ctx: The context to finalise; must have been initialized
+ * @out: Where to write the resulting message digest
+ *
+ * Finish the computation of a SHA3-256 hash and perform the "Keccak sponge
+ * squeezing" phase.  The digest is written to @out buffer and the context will
+ * be completely zeroed out.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_256_final(struct sha3_256_ctx *ctx, u8 out[SHA3_256_DIGEST_SIZE])
+{
+	sha3_squeeze(&ctx->ctx, out, SHA3_256_DIGEST_SIZE);
+	sha3_clear(&ctx->ctx);
+}
+
+/*
+ * Context wrapper for SHA3-384.
+ */
+struct sha3_384_ctx {
+	struct sha3_ctx ctx;
+};
+
+/**
+ * sha3_384_init() - Set a SHA3 context for SHA3-384
+ * @ctx: the context to initialize
+ *
+ * Initialize a SHA3 context for the production of a SHA3-384 digest of a
+ * message.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_384_init(struct sha3_384_ctx *ctx)
+{
+	*ctx = (struct sha3_384_ctx){
+		.ctx.block_size	= SHA3_384_BLOCK_SIZE,
+		.ctx.padding	= 0x06,
+	};
+}
+
+/**
+ * sha3_384_update() - Update a SHA3-384 hash with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to add data to the hash, performing
+ * the "keccak sponge absorbing" phase.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_384_update(struct sha3_384_ctx *ctx, const u8 *data, size_t len)
+{
+	return sha3_update(&ctx->ctx, data, len);
+}
+
+/**
+ * sha3_384_final() - Finalise a SHA3-384 hash and extract the digest
+ * @ctx: The context to finalise; must have been initialized
+ * @out: Where to write the resulting message digest
+ *
+ * Finish the computation of a SHA3-384 hash and perform the "Keccak sponge
+ * squeezing" phase.  The digest is written to @out buffer and the context will
+ * be completely zeroed out.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_384_final(struct sha3_384_ctx *ctx, u8 out[SHA3_384_DIGEST_SIZE])
+{
+	sha3_squeeze(&ctx->ctx, out, SHA3_384_DIGEST_SIZE);
+	sha3_clear(&ctx->ctx);
+}
+
+/*
+ * Context wrapper for SHA3-512.
+ */
+struct sha3_512_ctx {
+	struct sha3_ctx ctx;
+};
+
+/**
+ * sha3_512_init() - Set a SHA3 context for SHA3-512
+ * @ctx: the context to initialize
+ *
+ * Initialize a SHA3 context for the production of a SHA3-512 digest of a
+ * message.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_512_init(struct sha3_512_ctx *ctx)
+{
+	*ctx = (struct sha3_512_ctx){
+		.ctx.block_size	= SHA3_512_BLOCK_SIZE,
+		.ctx.padding	= 0x06,
+	};
+}
+
+/**
+ * sha3_512_update() - Update a SHA3-512 hash with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to add data to the hash, performing
+ * the "keccak sponge absorbing" phase.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_512_update(struct sha3_512_ctx *ctx, const u8 *data, size_t len)
+{
+	return sha3_update(&ctx->ctx, data, len);
+}
+
+/**
+ * sha3_512_final() - Finalise a SHA3-512 hash and extract the digest
+ * @ctx: The context to finalise; must have been initialized
+ * @out: Where to write the resulting message digest
+ *
+ * Finish the computation of a SHA3-512 hash and perform the "Keccak sponge
+ * squeezing" phase.  The digest is written to @out buffer and the context will
+ * be completely zeroed out.
+ *
+ * Context: Any context.
+ */
+static inline void sha3_512_final(struct sha3_512_ctx *ctx, u8 out[SHA3_512_DIGEST_SIZE])
+{
+	sha3_squeeze(&ctx->ctx, out, SHA3_512_DIGEST_SIZE);
+	sha3_clear(&ctx->ctx);
+}
+
+/*
+ * Context wrapper for SHAKE128.
+ */
+struct shake128_ctx {
+	struct sha3_ctx ctx;
+};
+
+/**
+ * shake128_init() - Set a SHA3 context for SHAKE128
+ * @ctx: The context to initialize
+ *
+ * Initialize a SHA3 context for the production of SHAKE128 output generation
+ * from a message.  The sha3_squeeze() function can be used to extract an
+ * arbitrary amount of data from the context.
+ *
+ * Context: Any context.
+ */
+static inline void shake128_init(struct shake128_ctx *ctx)
+{
+	*ctx = (struct shake128_ctx){
+		.ctx.block_size	= SHAKE128_BLOCK_SIZE,
+		.ctx.padding	= 0x1f,
+	};
+}
+
+/**
+ * shake128_update() - Update a SHAKE128 hash with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to add data to the XOF state,
+ * performing the "keccak sponge absorbing" phase.
+ *
+ * Context: Any context.
+ */
+static inline void shake128_update(struct shake128_ctx *ctx, const u8 *data, size_t len)
+{
+	return sha3_update(&ctx->ctx, data, len);
+}
+
+/**
+ * shake128_squeeze() - Finalize a SHAKE128 digest of any type and extract the digest
+ * @ctx: the context to finalize; must have been initialized
+ * @out: Where to write the resulting message digest
+ * @out_size: The amount of digest to extract to @out in bytes
+ *
+ * Finish the computation of a SHAKE128 XOF and perform the "Keccak sponge
+ * squeezing" phase.  @out_size amount of digest is written to @out buffer.
+ *
+ * This may be called multiple times to extract continuations of the digest.
+ * Note that, a number of consecutive squeezes laid end-to-end will yield the
+ * same output as one big squeeze generating the same total amount of output.
+ *
+ * Context: Any context.
+ */
+static inline void shake128_squeeze(struct shake128_ctx *ctx, u8 *out, size_t out_size)
+{
+	return sha3_squeeze(&ctx->ctx, out, out_size);
+}
+
+/**
+ * shake128_clear() - Explicitly clear the entire SHAKE128 context
+ * @ctx: the context to clear
+ *
+ * Explicitly clear the entire context; after this, the context must be
+ * initialized again.
+ *
+ * Context: Any context.
+ */
+static inline void shake128_clear(struct shake128_ctx *ctx)
+{
+	sha3_clear(&ctx->ctx);
+}
+
+/*
+ * Context wrapper for SHAKE256.
+ */
+struct shake256_ctx {
+	struct sha3_ctx ctx;
+};
+
+/**
+ * shake256_init() - Set a SHA3 context for SHAKE256
+ * @ctx: The context to initialize
+ *
+ * Initialize a SHA3 context for the production of SHAKE256 output generation
+ * from a message.  The sha3_squeeze() function can be used to extract an
+ * arbitrary amount of data from the context.
+ *
+ * Context: Any context.
+ */
+static inline void shake256_init(struct shake256_ctx *ctx)
+{
+	*ctx = (struct shake256_ctx){
+		.ctx.block_size	= SHAKE256_BLOCK_SIZE,
+		.ctx.padding	= 0x1f,
+	};
+}
+
+/**
+ * shake256_update() - Update a SHAKE256 hash with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to add data to the XOF state,
+ * performing the "keccak sponge absorbing" phase.
+ *
+ * Context: Any context.
+ */
+static inline void shake256_update(struct shake256_ctx *ctx, const u8 *data, size_t len)
+{
+	return sha3_update(&ctx->ctx, data, len);
+}
+
+/**
+ * shake256_squeeze() - Finalize a SHAKE256 digest of any type and extract the digest
+ * @ctx: the context to finalize; must have been initialized
+ * @out: Where to write the resulting message digest
+ * @out_size: The amount of digest to extract to @out in bytes
+ *
+ * Finish the computation of a SHAKE256 XOF and perform the "Keccak sponge
+ * squeezing" phase.  @out_size amount of digest is written to @out buffer.
+ *
+ * This may be called multiple times to extract continuations of the digest.
+ * Note that, a number of consecutive squeezes laid end-to-end will yield the
+ * same output as one big squeeze generating the same total amount of output.
+ *
+ * Context: Any context.
+ */
+static inline void shake256_squeeze(struct shake256_ctx *ctx, u8 *out, size_t out_size)
+{
+	return sha3_squeeze(&ctx->ctx, out, out_size);
+}
+
+/**
+ * shake256_clear() - Explicitly clear the entire SHAKE256 context
+ * @ctx: the context to clear
+ *
+ * Explicitly clear the entire context; after this, the context must be
+ * initialized again.
+ *
+ * Context: Any context.
+ */
+static inline void shake256_clear(struct shake256_ctx *ctx)
+{
+	sha3_clear(&ctx->ctx);
+}
+
+void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE]);
+void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE]);
+void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE]);
+void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE]);
+void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len);
+void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len);
+
+#endif /* __CRYPTO_SHA3_H__ */
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index eea17e36a22be..1caf3fbe24bff 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -182,10 +182,17 @@ config CRYPTO_LIB_SHA512_ARCH
 	default y if RISCV && 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO
 	default y if S390
 	default y if SPARC64
 	default y if X86_64
 
+config CRYPTO_LIB_SHA3
+	tristate
+	help
+	  The SHA3 library functions.
+	  Select this if your module uses any of these functions from
+	  <crypto/sha3.h>.
+
 config CRYPTO_LIB_SM3
 	tristate
 
 source "lib/crypto/tests/Kconfig"
 
diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index bded351aeacef..b7fa51bfeebb5 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -266,10 +266,16 @@ libsha512-$(CONFIG_X86) += x86/sha512-ssse3-asm.o \
 			   x86/sha512-avx2-asm.o
 endif # CONFIG_CRYPTO_LIB_SHA512_ARCH
 
 ################################################################################
 
+#obj-$(CONFIG_CRYPTO_LIB_SHA3) += libsha3.o
+#libsha3-y := sha3.o
+obj-$(CONFIG_CRYPTO_SHA3) += sha3.o
+
+################################################################################
+
 obj-$(CONFIG_MPILIB) += mpi/
 
 obj-$(CONFIG_CRYPTO_SELFTESTS_FULL)		+= simd.o
 
 obj-$(CONFIG_CRYPTO_LIB_SM3)			+= libsm3.o
diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
new file mode 100644
index 0000000000000..5f847a5eecdc7
--- /dev/null
+++ b/lib/crypto/sha3.c
@@ -0,0 +1,511 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Cryptographic API.
+ *
+ * SHA-3, as specified in
+ * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
+ *
+ * SHA-3 code by Jeff Garzik <jeff@garzik.org>
+ *               Ard Biesheuvel <ard.biesheuvel@linaro.org>
+ *		 David Howells <dhowells@redhat.com>
+ *
+ * See also Documentation/crypto/sha3.rst
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <crypto/sha3.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/unaligned.h>
+
+/*
+ * On some 32-bit architectures, such as h8300, GCC ends up using over 1 KB of
+ * stack if the round calculation gets inlined into the loop in
+ * sha3_keccakf_rounds_generic().  On the other hand, on 64-bit architectures
+ * with plenty of [64-bit wide] general purpose registers, not inlining it
+ * severely hurts performance.  So let's use 64-bitness as a heuristic to
+ * decide whether to inline or not.
+ */
+#ifdef CONFIG_64BIT
+#define SHA3_INLINE	inline
+#else
+#define SHA3_INLINE	noinline
+#endif
+
+#define SHA3_KECCAK_ROUNDS 24
+
+static const u64 sha3_keccakf_rndc[24] = {
+	0x0000000000000001ULL, 0x0000000000008082ULL, 0x800000000000808aULL,
+	0x8000000080008000ULL, 0x000000000000808bULL, 0x0000000080000001ULL,
+	0x8000000080008081ULL, 0x8000000000008009ULL, 0x000000000000008aULL,
+	0x0000000000000088ULL, 0x0000000080008009ULL, 0x000000008000000aULL,
+	0x000000008000808bULL, 0x800000000000008bULL, 0x8000000000008089ULL,
+	0x8000000000008003ULL, 0x8000000000008002ULL, 0x8000000000000080ULL,
+	0x000000000000800aULL, 0x800000008000000aULL, 0x8000000080008081ULL,
+	0x8000000000008080ULL, 0x0000000080000001ULL, 0x8000000080008008ULL
+};
+
+/*
+ * Perform a single round of Keccak mixing.
+ */
+static SHA3_INLINE void sha3_keccakf_one_round_generic(struct sha3_state *state)
+{
+	u64 *st = state->st;
+	u64 t[5], tt, bc[5];
+
+	/* Theta */
+	bc[0] = st[0] ^ st[5] ^ st[10] ^ st[15] ^ st[20];
+	bc[1] = st[1] ^ st[6] ^ st[11] ^ st[16] ^ st[21];
+	bc[2] = st[2] ^ st[7] ^ st[12] ^ st[17] ^ st[22];
+	bc[3] = st[3] ^ st[8] ^ st[13] ^ st[18] ^ st[23];
+	bc[4] = st[4] ^ st[9] ^ st[14] ^ st[19] ^ st[24];
+
+	t[0] = bc[4] ^ rol64(bc[1], 1);
+	t[1] = bc[0] ^ rol64(bc[2], 1);
+	t[2] = bc[1] ^ rol64(bc[3], 1);
+	t[3] = bc[2] ^ rol64(bc[4], 1);
+	t[4] = bc[3] ^ rol64(bc[0], 1);
+
+	st[0] ^= t[0];
+
+	/* Rho Pi */
+	tt = st[1];
+	st[ 1] = rol64(st[ 6] ^ t[1], 44);
+	st[ 6] = rol64(st[ 9] ^ t[4], 20);
+	st[ 9] = rol64(st[22] ^ t[2], 61);
+	st[22] = rol64(st[14] ^ t[4], 39);
+	st[14] = rol64(st[20] ^ t[0], 18);
+	st[20] = rol64(st[ 2] ^ t[2], 62);
+	st[ 2] = rol64(st[12] ^ t[2], 43);
+	st[12] = rol64(st[13] ^ t[3], 25);
+	st[13] = rol64(st[19] ^ t[4],  8);
+	st[19] = rol64(st[23] ^ t[3], 56);
+	st[23] = rol64(st[15] ^ t[0], 41);
+	st[15] = rol64(st[ 4] ^ t[4], 27);
+	st[ 4] = rol64(st[24] ^ t[4], 14);
+	st[24] = rol64(st[21] ^ t[1],  2);
+	st[21] = rol64(st[ 8] ^ t[3], 55);
+	st[ 8] = rol64(st[16] ^ t[1], 45);
+	st[16] = rol64(st[ 5] ^ t[0], 36);
+	st[ 5] = rol64(st[ 3] ^ t[3], 28);
+	st[ 3] = rol64(st[18] ^ t[3], 21);
+	st[18] = rol64(st[17] ^ t[2], 15);
+	st[17] = rol64(st[11] ^ t[1], 10);
+	st[11] = rol64(st[ 7] ^ t[2],  6);
+	st[ 7] = rol64(st[10] ^ t[0],  3);
+	st[10] = rol64(    tt ^ t[1],  1);
+
+	/* Chi */
+	bc[ 0] = ~st[ 1] & st[ 2];
+	bc[ 1] = ~st[ 2] & st[ 3];
+	bc[ 2] = ~st[ 3] & st[ 4];
+	bc[ 3] = ~st[ 4] & st[ 0];
+	bc[ 4] = ~st[ 0] & st[ 1];
+	st[ 0] ^= bc[ 0];
+	st[ 1] ^= bc[ 1];
+	st[ 2] ^= bc[ 2];
+	st[ 3] ^= bc[ 3];
+	st[ 4] ^= bc[ 4];
+
+	bc[ 0] = ~st[ 6] & st[ 7];
+	bc[ 1] = ~st[ 7] & st[ 8];
+	bc[ 2] = ~st[ 8] & st[ 9];
+	bc[ 3] = ~st[ 9] & st[ 5];
+	bc[ 4] = ~st[ 5] & st[ 6];
+	st[ 5] ^= bc[ 0];
+	st[ 6] ^= bc[ 1];
+	st[ 7] ^= bc[ 2];
+	st[ 8] ^= bc[ 3];
+	st[ 9] ^= bc[ 4];
+
+	bc[ 0] = ~st[11] & st[12];
+	bc[ 1] = ~st[12] & st[13];
+	bc[ 2] = ~st[13] & st[14];
+	bc[ 3] = ~st[14] & st[10];
+	bc[ 4] = ~st[10] & st[11];
+	st[10] ^= bc[ 0];
+	st[11] ^= bc[ 1];
+	st[12] ^= bc[ 2];
+	st[13] ^= bc[ 3];
+	st[14] ^= bc[ 4];
+
+	bc[ 0] = ~st[16] & st[17];
+	bc[ 1] = ~st[17] & st[18];
+	bc[ 2] = ~st[18] & st[19];
+	bc[ 3] = ~st[19] & st[15];
+	bc[ 4] = ~st[15] & st[16];
+	st[15] ^= bc[ 0];
+	st[16] ^= bc[ 1];
+	st[17] ^= bc[ 2];
+	st[18] ^= bc[ 3];
+	st[19] ^= bc[ 4];
+
+	bc[ 0] = ~st[21] & st[22];
+	bc[ 1] = ~st[22] & st[23];
+	bc[ 2] = ~st[23] & st[24];
+	bc[ 3] = ~st[24] & st[20];
+	bc[ 4] = ~st[20] & st[21];
+	st[20] ^= bc[ 0];
+	st[21] ^= bc[ 1];
+	st[22] ^= bc[ 2];
+	st[23] ^= bc[ 3];
+	st[24] ^= bc[ 4];
+}
+
+static void sha3_keccakf_rounds_generic(struct sha3_state *state)
+{
+	for (int round = 0; round < SHA3_KECCAK_ROUNDS; round++) {
+		sha3_keccakf_one_round_generic(state);
+		/* Iota */
+		state->st[0] ^= sha3_keccakf_rndc[round];
+	}
+}
+
+/*
+ * Byteswap the state buckets to CPU-endian if we're not on a little-endian
+ * machine for the duration of the Keccak mixing function.  Note that these
+ * loops are no-ops on LE machines and will be optimised away.
+ */
+static void sha3_keccakf_generic(struct sha3_state *state)
+{
+	for (int  i = 0; i < ARRAY_SIZE(state->st); i++)
+		cpu_to_le64s(&state->st[i]);
+
+	sha3_keccakf_rounds_generic(state);
+
+	for (int  i = 0; i < ARRAY_SIZE(state->st); i++)
+		le64_to_cpus(&state->st[i]);
+}
+
+static void sha3_absorb_block_generic(struct sha3_ctx *ctx, const u8 *data)
+{
+	struct sha3_state *state = &ctx->state;
+	size_t bsize = ctx->block_size;
+
+	for (size_t i = 0; i < bsize / 8; i++)
+		state->st[i] ^= get_unaligned((u64 *)(data + 8 * i));
+	sha3_keccakf_generic(state);
+}
+
+/*
+ * Perform rounds of XOR'ing whole blocks of data into the state buffer and
+ * then performing a keccak mix step.
+ */
+static void sha3_absorb_blocks_generic(struct sha3_ctx *ctx,
+				       const u8 *data, size_t nblocks)
+{
+	do {
+		sha3_absorb_block_generic(ctx, data);
+		data += ctx->block_size;
+	} while (--nblocks);
+}
+
+#ifdef CONFIG_CRYPTO_LIB_SHA3_ARCH
+#include "sha3.h" /* $(SRCARCH)/sha3.h */
+#else
+#define sha3_keccakf		sha3_keccakf_generic
+#define sha3_absorb_blocks	sha3_absorb_blocks_generic
+#endif
+
+/*
+ * XOR in partial data that's insufficient to fill a whole block.
+ */
+static void sha3_absorb_xorle(struct sha3_ctx *ctx, const u8 *data,
+			      size_t partial, size_t len)
+{
+	u8 *buf = (u8 *)ctx->state.st;
+
+	buf += partial;
+	for (size_t i = 0; i < len; i++)
+		*buf++ ^= *data++;
+}
+
+/**
+ * sha3_update() - Update a SHA3 context of any type with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times to perform the "keccak sponge
+ * absorbing" phase.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len)
+{
+	size_t absorb_offset = ctx->absorb_offset;
+	size_t bsize = ctx->block_size;
+
+	WARN_ON_ONCE(ctx->end_marked);
+
+	if (absorb_offset && absorb_offset + len >= bsize) {
+		sha3_absorb_xorle(ctx, data, absorb_offset, bsize - absorb_offset);
+		len  -= bsize - absorb_offset;
+		data += bsize - absorb_offset;
+		sha3_keccakf(&ctx->state);
+		ctx->absorb_offset = 0;
+	}
+
+	if (len >= bsize) {
+		size_t nblocks = len / bsize;
+
+		sha3_absorb_blocks(ctx, data, nblocks);
+		data += nblocks * bsize;
+		len  -= nblocks * bsize;
+	}
+
+	if (len) {
+		sha3_absorb_xorle(ctx, data, ctx->absorb_offset, len);
+		ctx->absorb_offset += len;
+	}
+}
+EXPORT_SYMBOL_GPL(sha3_update);
+
+/**
+ * sha3_squeeze() - Finalize a SHA3 digest of any type and extract the digest
+ * @ctx: the context to finalize; must have been initialized
+ * @out: Where to write the resulting message digest
+ * @out_size: The amount of digest to extract to @out
+ *
+ * Finish the computation of a SHA3 message digest of any type and perform the
+ * "Keccak sponge squeezing" phase.  @out_size amount of digest is written to
+ * @out buffer.
+ *
+ * This may be called multiple times to extract continuations of the digest.
+ * Note that, for example, two consecutive 16-byte squeezes laid end-to-end
+ * will yield the same as one 32-byte squeeze.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_size)
+{
+	size_t squeeze_offset = ctx->squeeze_offset;
+	size_t bsize = ctx->block_size;
+	u8 *p = (u8 *)ctx->state.st, end_marker = 0x80;
+
+	if (!ctx->end_marked) {
+		sha3_absorb_xorle(ctx, &ctx->padding, ctx->absorb_offset, 1);
+		sha3_absorb_xorle(ctx, &end_marker, bsize - 1, 1);
+		ctx->end_marked = true;
+	}
+
+	for (;;) {
+		if (squeeze_offset == 0)
+			sha3_keccakf(&ctx->state);
+
+		size_t part = umin(out_size, bsize - squeeze_offset);
+
+		if (part > 0) {
+			memcpy(out, p + squeeze_offset, part);
+			out_size -= part;
+			out += part;
+			squeeze_offset += part;
+		}
+		if (!out_size)
+			break;
+		if (squeeze_offset >= bsize)
+			squeeze_offset = 0;
+	}
+
+	ctx->squeeze_offset = squeeze_offset;
+}
+EXPORT_SYMBOL_GPL(sha3_squeeze);
+
+/**
+ * sha3_224() - Convenience wrapper to digest a simple buffer as SHA3-224
+ * @in: The data to be digested
+ * @in_len: The amount of data to be digested in bytes
+ * @out: The buffer into which the digest will be stored (size not checked)
+ *
+ * Convenience wrapper to initialise a SHA3 context for SHA-224, add the input
+ * data to it, finalise it, extract 28 bytes of digest and clear the context.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE])
+{
+	struct sha3_224_ctx ctx;
+
+	sha3_224_init(&ctx);
+	sha3_224_update(&ctx, in, in_len);
+	sha3_224_final(&ctx, out);
+}
+EXPORT_SYMBOL(sha3_224);
+
+/**
+ * sha3_256() - Convenience wrapper to digest a simple buffer as SHA3-256
+ * @in: The data to be digested
+ * @in_len: The amount of data to be digested in bytes
+ * @out: The buffer into which the digest will be stored (size not checked)
+ *
+ * Convenience wrapper to initialise a SHA3 context for SHA-256, add the input
+ * data to it, finalise it, extract 32 bytes of digest and clear the context.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE])
+{
+	struct sha3_256_ctx ctx;
+
+	sha3_256_init(&ctx);
+	sha3_256_update(&ctx, in, in_len);
+	sha3_256_final(&ctx, out);
+}
+EXPORT_SYMBOL(sha3_256);
+
+/**
+ * sha3_384() - Convenience wrapper to digest a simple buffer as SHA3-384
+ * @in: The data to be digested
+ * @in_len: The amount of data to be digested in bytes
+ * @out: The buffer into which the digest will be stored (size not checked)
+ *
+ * Convenience wrapper to initialise a SHA3 context for SHA-384, add the input
+ * data to it, finalise it, extract 48 bytes of digest and clear the context.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE])
+{
+	struct sha3_384_ctx ctx;
+
+	sha3_384_init(&ctx);
+	sha3_384_update(&ctx, in, in_len);
+	sha3_384_final(&ctx, out);
+}
+EXPORT_SYMBOL(sha3_384);
+
+/**
+ * sha3_512() - Convenience wrapper to digest a simple buffer as SHA3-512
+ * @in: The data to be digested in bytes
+ * @in_len: The amount of data to be digested in bytes
+ * @out: The buffer into which the digest will be stored (size not checked)
+ *
+ * Convenience wrapper to initialise a SHA3 context for SHA-512, add the input
+ * data to it, finalise it, extract 64 bytes of digest and clear the context.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE])
+{
+	struct sha3_512_ctx ctx;
+
+	sha3_512_init(&ctx);
+	sha3_512_update(&ctx, in, in_len);
+	sha3_512_final(&ctx, out);
+}
+EXPORT_SYMBOL(sha3_512);
+
+/**
+ * shake128() - Convenience wrapper to apply SHAKE128 to a simple buffer
+ * @in: The input data to be used
+ * @in_len: The amount of input data in bytes
+ * @out: The buffer in which to store the output
+ * @out_len: The amount of output to store in bytes (variable length)
+ *
+ * Convenience wrapper to initialise a SHA3 context for SHAKE128, add the input
+ * data to it, finalise it, extract the requested amount of output and clear
+ * the context.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len)
+{
+	struct shake128_ctx ctx;
+
+	shake128_init(&ctx);
+	shake128_update(&ctx, in, in_len);
+	shake128_squeeze(&ctx, out, out_len);
+	shake128_clear(&ctx);
+}
+EXPORT_SYMBOL(shake128);
+
+/**
+ * shake256() - Convenience wrapper to apply SHAKE256 to a simple buffer
+ * @in: The input data to be used
+ * @in_len: The amount of input data in bytes
+ * @out: The buffer in which to store the output
+ * @out_len: The amount of output to store in bytes (variable length)
+ *
+ * Convenience wrapper to initialise a SHA3 context for SHAKE128, add the input
+ * data to it, finalise it, extract the requested amount of output and clear
+ * the context.
+ *
+ * Context: May use the FPU/Vector unit registers.
+ */
+void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len)
+{
+	struct shake256_ctx ctx;
+
+	shake256_init(&ctx);
+	shake256_update(&ctx, in, in_len);
+	shake256_squeeze(&ctx, out, out_len);
+	shake256_clear(&ctx);
+}
+EXPORT_SYMBOL(shake256);
+
+/*
+ * Do a quick test using SHAKE256 and a 200 byte digest.
+ */
+static const u8 sha3_sample[] __initconst =
+	"The quick red fox jumped over the lazy brown dog!\n"
+	"The quick red fox jumped over the lazy brown dog!\n"
+	"The quick red fox jumped over the lazy brown dog!\n"
+	"The quick red fox jumped over the lazy brown dog!\n";
+
+static const u8 sha3_sample_shake256_200[] __initconst = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0xab, 0x06, 0xd4, 0xf9, 0x8b, 0xfd, 0xb2, 0xc4, 0xfe, 0xf1, 0xcc, 0xe2,
+	0x40, 0x45, 0xdd, 0x15, 0xcb, 0xdd, 0x02, 0x8d, 0xb7, 0x9f, 0x1e, 0x67,
+	0xd6, 0x7f, 0x98, 0x5e, 0x1b, 0x19, 0xf8, 0x01, 0x43, 0x82, 0xcb, 0xd8,
+	0x5d, 0x21, 0x64, 0xa8, 0x80, 0xc9, 0x22, 0xe5, 0x07, 0xaf, 0xe2, 0x5d,
+	0xcd, 0xc6, 0x23, 0x36, 0x2b, 0xc7, 0xc7, 0x7d, 0x09, 0x9d, 0x68, 0x05,
+	0xe4, 0x62, 0x63, 0x1b, 0x67, 0xbc, 0xf8, 0x95, 0x07, 0xd2, 0xe4, 0xd0,
+	0xba, 0xa2, 0x67, 0xf5, 0xe3, 0x15, 0xbc, 0x85, 0xa1, 0x50, 0xd6, 0x6f,
+	0x6f, 0xd4, 0x54, 0x4c, 0x3f, 0x4f, 0xe5, 0x1f, 0xb7, 0x00, 0x27, 0xfc,
+	0x15, 0x33, 0xc2, 0xf9, 0xb3, 0x4b, 0x9e, 0x81, 0xe5, 0x96, 0xbe, 0x05,
+	0x6c, 0xac, 0xf9, 0x9f, 0x65, 0x36, 0xbb, 0x11, 0x47, 0x6d, 0xf6, 0x8f,
+	0x9f, 0xa2, 0x77, 0x37, 0x3b, 0x18, 0x77, 0xcf, 0x65, 0xc5, 0xa1, 0x7e,
+	0x2c, 0x0e, 0x71, 0xf0, 0x4d, 0x18, 0x67, 0xb9, 0xc4, 0x8c, 0x64, 0x3b,
+	0x4b, 0x45, 0xea, 0x16, 0xb2, 0x4a, 0xc5, 0xf5, 0x85, 0xdc, 0xd2, 0xd9,
+	0x13, 0x77, 0xb3, 0x19, 0xd9, 0x8c, 0x9f, 0x28, 0xe7, 0x64, 0x91, 0x0f,
+	0x6f, 0x32, 0xbf, 0xa8, 0xa8, 0xa3, 0xff, 0x99, 0x0e, 0x0b, 0x62, 0x50,
+	0xf8, 0x3a, 0xc2, 0xf5, 0x98, 0x21, 0xeb, 0x9d, 0xe8, 0x45, 0xf4, 0x46,
+	0x1e, 0x8b, 0xbd, 0x10, 0x59, 0x2c, 0x87, 0xe2,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static int __init sha3_mod_init(void)
+{
+#define out_len 200
+	u8 out[8 + out_len + 8] = {};
+
+#ifdef sha3_mod_init_arch
+	sha3_mod_init_arch();
+#endif
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(sha3_sample_shake256_200));
+
+	shake256(sha3_sample, sizeof(sha3_sample) - 1, out + 8, out_len);
+
+	if (memcmp(out, sha3_sample_shake256_200,
+		   sizeof(sha3_sample_shake256_200)) != 0) {
+		pr_err("SHAKE256(200) failed\n");
+		for (size_t i = 0; i < out_len;) {
+			size_t part = min(out_len - i, 32);
+
+			pr_err("%*phN\n", (int)part, out + i);
+			i += part;
+		}
+		return -EBADMSG;
+	}
+	return 0;
+}
+subsys_initcall(sha3_mod_init);
+
+static void __exit sha3_mod_exit(void)
+{
+}
+module_exit(sha3_mod_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SHA-3 Secure Hash Algorithm");
-- 
2.51.1.dirty

[PATCH 04/17] lib/crypto: Move the SHA3 Iota transform into the single round function

[Eric Biggers]

From: David Howells <dhowells@redhat.com>

In crypto/sha3_generic.c, the keccakf() function calls keccakf_round() to
do most of the transforms, but not the Iota transform - presumably because
that is dependent on round number, whereas the Theta, Rho, Pi and Chi
transforms are not.

Note that the keccakf_round() function needs to be explicitly non-inlined
on certain architectures as gcc's produced output will (or used to) use
over 1KiB of stack space if inlined.

Now, this code was copied more or less verbatim into lib/crypto/sha3.c, so
that has the same aesthetic issue.  Fix this there by passing the round
number into sha3_keccakf_one_round_generic() and doing the Iota transform
there.

crypto/sha3_generic.c is left untouched as that will be converted to use
lib/crypto/sha3.c at some point.

Suggested-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Eric Biggers <ebiggers@kernel.org>
cc: "Jason A. Donenfeld" <Jason@zx2c4.com>
cc: Ard Biesheuvel <ardb@kernel.org>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: Stephan Mueller <smueller@chronox.de>
cc: linux-crypto@vger.kernel.org
Link: https://lore.kernel.org/r/20251017144311.817771-5-dhowells@redhat.com
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 lib/crypto/sha3.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
index 5f847a5eecdc7..2c292b0b3db34 100644
--- a/lib/crypto/sha3.c
+++ b/lib/crypto/sha3.c
@@ -46,11 +46,12 @@ static const u64 sha3_keccakf_rndc[24] = {
 };
 
 /*
  * Perform a single round of Keccak mixing.
  */
-static SHA3_INLINE void sha3_keccakf_one_round_generic(struct sha3_state *state)
+static SHA3_INLINE void sha3_keccakf_one_round_generic(struct sha3_state *state,
+						       int round)
 {
 	u64 *st = state->st;
 	u64 t[5], tt, bc[5];
 
 	/* Theta */
@@ -148,19 +149,19 @@ static SHA3_INLINE void sha3_keccakf_one_round_generic(struct sha3_state *state)
 	st[20] ^= bc[ 0];
 	st[21] ^= bc[ 1];
 	st[22] ^= bc[ 2];
 	st[23] ^= bc[ 3];
 	st[24] ^= bc[ 4];
+
+	/* Iota */
+	state->st[0] ^= sha3_keccakf_rndc[round];
 }
 
 static void sha3_keccakf_rounds_generic(struct sha3_state *state)
 {
-	for (int round = 0; round < SHA3_KECCAK_ROUNDS; round++) {
-		sha3_keccakf_one_round_generic(state);
-		/* Iota */
-		state->st[0] ^= sha3_keccakf_rndc[round];
-	}
+	for (int round = 0; round < SHA3_KECCAK_ROUNDS; round++)
+		sha3_keccakf_one_round_generic(state, round);
 }
 
 /*
  * Byteswap the state buckets to CPU-endian if we're not on a little-endian
  * machine for the duration of the Keccak mixing function.  Note that these
-- 
2.51.1.dirty

[PATCH 05/17] lib/crypto: Add SHA3 kunit tests

[Eric Biggers]

From: David Howells <dhowells@redhat.com>

Add a SHA3 kunit test, providing the following:

 (*) A simple test of each of SHA3-224, SHA3-256, SHA3-384, SHA-512,
     SHAKE128 and SHAKE256.

 (*) NIST 0- and 1600-bit test vectors for SHAKE128 and SHAKE256.

 (*) Output tiling (multiple squeezing) tests for SHAKE256.

 (*) Standard hash template test for SHA3-256.

 (*) Standard benchmark test for SHA3-256.

gen-hash-testvecs.py is also modified in a number of ways:

 (1) To be able to generate SHAKE* testvecs because Python's hashlib
     requires the output digest size supplying for those two algorithms as
     they produce arbitrary length digests.

 (2) To change '-' in the algo name into '_' when generating symbols from
     it.

 (3) To not generate HMAC for SHA3 or SHAKE as support for that isn't
     currently implemented.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: Eric Biggers <ebiggers@kernel.org>
cc: Jason A. Donenfeld <Jason@zx2c4.com>
cc: Ard Biesheuvel <ardb@kernel.org>
cc: Herbert Xu <herbert@gondor.apana.org.au>
cc: Stephan Mueller <smueller@chronox.de>
cc: linux-crypto@vger.kernel.org

Changes
=======
v5)
 - Fix gen-hash-testvecs.py to correctly handle algo names that contain a
   dash.
 - Fix gen-hash-testvecs.py to not generate HMAC for SHA3-* or SHAKE* as
   these don't currently have HMAC variants.
 - Fix algo names to be correct.
 - Fix kunit module description as it now tests all SHA3 variants.

Link: https://lore.kernel.org/r/20251017144311.817771-6-dhowells@redhat.com
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 lib/crypto/tests/Kconfig            |  11 +
 lib/crypto/tests/Makefile           |   1 +
 lib/crypto/tests/sha3_kunit.c       | 342 ++++++++++++++++++++++++++++
 lib/crypto/tests/sha3_testvecs.h    | 231 +++++++++++++++++++
 scripts/crypto/gen-hash-testvecs.py |  10 +-
 5 files changed, 593 insertions(+), 2 deletions(-)
 create mode 100644 lib/crypto/tests/sha3_kunit.c
 create mode 100644 lib/crypto/tests/sha3_testvecs.h

diff --git a/lib/crypto/tests/Kconfig b/lib/crypto/tests/Kconfig
index 578af717e13a7..67538968edabb 100644
--- a/lib/crypto/tests/Kconfig
+++ b/lib/crypto/tests/Kconfig
@@ -70,10 +70,21 @@ config CRYPTO_LIB_SHA512_KUNIT_TEST
 	select CRYPTO_LIB_SHA512
 	help
 	  KUnit tests for the SHA-384 and SHA-512 cryptographic hash functions
 	  and their corresponding HMACs.
 
+config CRYPTO_LIB_SHA3_KUNIT_TEST
+	tristate "KUnit tests for SHA-3" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS
+	select CRYPTO_LIB_BENCHMARK_VISIBLE
+	select CRYPTO_LIB_SHA3
+	help
+	  KUnit tests for the SHA3 cryptographic hash and XOF functions,
+	  including SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128 and
+	  SHAKE256.
+
 config CRYPTO_LIB_BENCHMARK_VISIBLE
 	bool
 
 config CRYPTO_LIB_BENCHMARK
 	bool "Include benchmarks in KUnit tests for cryptographic functions"
diff --git a/lib/crypto/tests/Makefile b/lib/crypto/tests/Makefile
index a71fad19922ba..9c61b29ac4ca1 100644
--- a/lib/crypto/tests/Makefile
+++ b/lib/crypto/tests/Makefile
@@ -5,5 +5,6 @@ obj-$(CONFIG_CRYPTO_LIB_CURVE25519_KUNIT_TEST) += curve25519_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_MD5_KUNIT_TEST) += md5_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST) += poly1305_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST) += sha1_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST) += sha224_kunit.o sha256_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST) += sha384_kunit.o sha512_kunit.o
+obj-$(CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST) += sha3_kunit.o
diff --git a/lib/crypto/tests/sha3_kunit.c b/lib/crypto/tests/sha3_kunit.c
new file mode 100644
index 0000000000000..032fa3950afe4
--- /dev/null
+++ b/lib/crypto/tests/sha3_kunit.c
@@ -0,0 +1,342 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+#include <crypto/sha3.h>
+#include "sha3_testvecs.h"
+
+#define HASH		sha3_256
+#define HASH_CTX	sha3_256_ctx
+#define HASH_SIZE	SHA3_256_DIGEST_SIZE
+#define HASH_INIT	sha3_256_init
+#define HASH_UPDATE	sha3_256_update
+#define HASH_FINAL	sha3_256_final
+#include "hash-test-template.h"
+
+/*
+ * Sample message and the output generated for various algorithms by passing it
+ * into "openssl sha3-224" etc..
+ */
+static const u8 test_sha3_sample[] =
+	"The quick red fox jumped over the lazy brown dog!\n"
+	"The quick red fox jumped over the lazy brown dog!\n"
+	"The quick red fox jumped over the lazy brown dog!\n"
+	"The quick red fox jumped over the lazy brown dog!\n";
+
+static const u8 test_sha3_224[8 + SHA3_224_DIGEST_SIZE + 8] = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0xd6, 0xe8, 0xd8, 0x80, 0xfa, 0x42, 0x80, 0x70,
+	0x7e, 0x7f, 0xd7, 0xd2, 0xd7, 0x7a, 0x35, 0x65,
+	0xf0, 0x0b, 0x4f, 0x9f, 0x2a, 0x33, 0xca, 0x0a,
+	0xef, 0xa6, 0x4c, 0xb8,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static const u8 test_sha3_256[8 + SHA3_256_DIGEST_SIZE + 8] = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0xdb, 0x3b, 0xb0, 0xb8, 0x8d, 0x15, 0x78, 0xe5,
+	0x78, 0x76, 0x8e, 0x39, 0x7e, 0x89, 0x86, 0xb9,
+	0x14, 0x3a, 0x1e, 0xe7, 0x96, 0x7c, 0xf3, 0x25,
+	0x70, 0xbd, 0xc3, 0xa9, 0xae, 0x63, 0x71, 0x1d,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static const u8 test_sha3_384[8 + SHA3_384_DIGEST_SIZE + 8] = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0x2d, 0x4b, 0x29, 0x85, 0x19, 0x94, 0xaa, 0x31,
+	0x9b, 0x04, 0x9d, 0x6e, 0x79, 0x66, 0xc7, 0x56,
+	0x8a, 0x2e, 0x99, 0x84, 0x06, 0xcf, 0x10, 0x2d,
+	0xec, 0xf0, 0x03, 0x04, 0x1f, 0xd5, 0x99, 0x63,
+	0x2f, 0xc3, 0x2b, 0x0d, 0xd9, 0x45, 0xf7, 0xbb,
+	0x0a, 0xc3, 0x46, 0xab, 0xfe, 0x4d, 0x94, 0xc2,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static const u8 test_sha3_512[8 + SHA3_512_DIGEST_SIZE + 8] = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0xdd, 0x71, 0x3b, 0x44, 0xb6, 0x6c, 0xd7, 0x78,
+	0xe7, 0x93, 0xa1, 0x4c, 0xd7, 0x24, 0x16, 0xf1,
+	0xfd, 0xa2, 0x82, 0x4e, 0xed, 0x59, 0xe9, 0x83,
+	0x15, 0x38, 0x89, 0x7d, 0x39, 0x17, 0x0c, 0xb2,
+	0xcf, 0x12, 0x80, 0x78, 0xa1, 0x78, 0x41, 0xeb,
+	0xed, 0x21, 0x4c, 0xa4, 0x4a, 0x5f, 0x30, 0x1a,
+	0x70, 0x98, 0x4f, 0x14, 0xa2, 0xd1, 0x64, 0x1b,
+	0xc2, 0x0a, 0xff, 0x3b, 0xe8, 0x26, 0x41, 0x8f,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static const u8 test_shake128[8 + SHAKE128_DEFAULT_SIZE + 8] = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0x41, 0xd6, 0xb8, 0x9c, 0xf8, 0xe8, 0x54, 0xf2,
+	0x5c, 0xde, 0x51, 0x12, 0xaf, 0x9e, 0x0d, 0x91,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static const u8 test_shake256[8 + SHAKE256_DEFAULT_SIZE + 8] = {
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
+	0xab, 0x06, 0xd4, 0xf9, 0x8b, 0xfd, 0xb2, 0xc4,
+	0xfe, 0xf1, 0xcc, 0xe2, 0x40, 0x45, 0xdd, 0x15,
+	0xcb, 0xdd, 0x02, 0x8d, 0xb7, 0x9f, 0x1e, 0x67,
+	0xd6, 0x7f, 0x98, 0x5e, 0x1b, 0x19, 0xf8, 0x01,
+	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
+};
+
+static void test_sha3_224_basic(struct kunit *test)
+{
+	u8 out[8 + SHA3_224_DIGEST_SIZE + 8];
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(test_sha3_224));
+
+	memset(out, 0, sizeof(out));
+	sha3_224(test_sha3_sample, sizeof(test_sha3_sample) - 1, out + 8);
+
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_sha3_224, sizeof(test_sha3_224),
+			       "SHA3-224 gives wrong output");
+}
+
+static void test_sha3_256_basic(struct kunit *test)
+{
+	u8 out[8 + SHA3_256_DIGEST_SIZE + 8];
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(test_sha3_256));
+
+	memset(out, 0, sizeof(out));
+	sha3_256(test_sha3_sample, sizeof(test_sha3_sample) - 1, out + 8);
+
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_sha3_256, sizeof(test_sha3_256),
+			       "SHA3-256 gives wrong output");
+}
+
+static void test_sha3_384_basic(struct kunit *test)
+{
+	u8 out[8 + SHA3_384_DIGEST_SIZE + 8];
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(test_sha3_384));
+
+	memset(out, 0, sizeof(out));
+	sha3_384(test_sha3_sample, sizeof(test_sha3_sample) - 1, out + 8);
+
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_sha3_384, sizeof(test_sha3_384),
+			       "SHA3-384 gives wrong output");
+}
+
+static void test_sha3_512_basic(struct kunit *test)
+{
+	u8 out[8 + SHA3_512_DIGEST_SIZE + 8];
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(test_sha3_512));
+
+	memset(out, 0, sizeof(out));
+	sha3_512(test_sha3_sample, sizeof(test_sha3_sample) - 1, out + 8);
+
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_sha3_512, sizeof(test_sha3_512),
+			       "SHA3-512 gives wrong output");
+}
+
+static void test_shake128_basic(struct kunit *test)
+{
+	u8 out[8 + SHAKE128_DEFAULT_SIZE + 8];
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(test_shake128));
+
+	memset(out, 0, sizeof(out));
+	shake128(test_sha3_sample, sizeof(test_sha3_sample) - 1,
+		 out + 8, sizeof(out) - 16);
+
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake128, sizeof(test_shake128),
+			       "SHAKE128 gives wrong output");
+}
+
+static void test_shake256_basic(struct kunit *test)
+{
+	u8 out[8 + SHAKE256_DEFAULT_SIZE + 8];
+
+	BUILD_BUG_ON(sizeof(out) != sizeof(test_shake256));
+
+	memset(out, 0, sizeof(out));
+	shake256(test_sha3_sample, sizeof(test_sha3_sample) - 1,
+		 out + 8, sizeof(out) - 16);
+
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake256, sizeof(test_shake256),
+			       "SHAKE256 gives wrong output");
+}
+
+/*
+ * Usable NIST tests.
+ *
+ * From: https://csrc.nist.gov/projects/cryptographic-standards-and-guidelines/example-values
+ */
+static const u8 test_nist_1600_sample[] = {
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3,
+	0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3, 0xa3
+};
+
+static const u8 test_shake128_nist_0[] = {
+	0x7f, 0x9c, 0x2b, 0xa4, 0xe8, 0x8f, 0x82, 0x7d,
+	0x61, 0x60, 0x45, 0x50, 0x76, 0x05, 0x85, 0x3e
+};
+
+static const u8 test_shake128_nist_1600[] = {
+	0x13, 0x1a, 0xb8, 0xd2, 0xb5, 0x94, 0x94, 0x6b,
+	0x9c, 0x81, 0x33, 0x3f, 0x9b, 0xb6, 0xe0, 0xce,
+};
+
+static const u8 test_shake256_nist_0[] = {
+	0x46, 0xb9, 0xdd, 0x2b, 0x0b, 0xa8, 0x8d, 0x13,
+	0x23, 0x3b, 0x3f, 0xeb, 0x74, 0x3e, 0xeb, 0x24,
+	0x3f, 0xcd, 0x52, 0xea, 0x62, 0xb8, 0x1b, 0x82,
+	0xb5, 0x0c, 0x27, 0x64, 0x6e, 0xd5, 0x76, 0x2f
+};
+
+static const u8 test_shake256_nist_1600[] = {
+	0xcd, 0x8a, 0x92, 0x0e, 0xd1, 0x41, 0xaa, 0x04,
+	0x07, 0xa2, 0x2d, 0x59, 0x28, 0x86, 0x52, 0xe9,
+	0xd9, 0xf1, 0xa7, 0xee, 0x0c, 0x1e, 0x7c, 0x1c,
+	0xa6, 0x99, 0x42, 0x4d, 0xa8, 0x4a, 0x90, 0x4d,
+};
+
+static void test_shake128_nist(struct kunit *test)
+{
+	u8 out[SHAKE128_DEFAULT_SIZE];
+
+	shake128("", 0, out, sizeof(out));
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake128_nist_0, sizeof(out),
+			       "SHAKE128 gives wrong output for NIST.0");
+
+	shake128(test_nist_1600_sample, sizeof(test_nist_1600_sample),
+		 out, sizeof(out));
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake128_nist_1600, sizeof(out),
+			       "SHAKE128 gives wrong output for NIST.1600");
+}
+
+static void test_shake256_nist(struct kunit *test)
+{
+	u8 out[SHAKE256_DEFAULT_SIZE];
+
+	shake256("", 0, out, sizeof(out));
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake256_nist_0, sizeof(out),
+			       "SHAKE256 gives wrong output for NIST.0");
+
+	shake256(test_nist_1600_sample, sizeof(test_nist_1600_sample),
+		 out, sizeof(out));
+	KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake256_nist_1600, sizeof(out),
+			       "SHAKE256 gives wrong output for NIST.1600");
+}
+
+/*
+ * Output tiling test of SHAKE256; equal output tiles barring the last.  A
+ * series of squeezings of the same context should, if laid end-to-end, match a
+ * single squeezing of the combined size.
+ */
+static void test_shake256_tiling(struct kunit *test)
+{
+	struct shake256_ctx ctx;
+	u8 out[8 + SHA3_512_DIGEST_SIZE + 8];
+
+	for (int tile_size = 1; tile_size < SHAKE256_DEFAULT_SIZE; tile_size++) {
+		int left = SHAKE256_DEFAULT_SIZE;
+		u8 *p = out + 8;
+
+		memset(out, 0, sizeof(out));
+		shake256_init(&ctx);
+		shake256_update(&ctx, test_sha3_sample, sizeof(test_sha3_sample) - 1);
+		while (left > 0) {
+			int part = umin(tile_size, left);
+
+			shake256_squeeze(&ctx, p, part);
+			p += part;
+			left -= part;
+		}
+
+		KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake256, sizeof(test_shake256),
+				       "SHAKE tile %u gives wrong output", tile_size);
+	}
+}
+
+/*
+ * Output tiling test of SHAKE256; output tiles getting gradually smaller and
+ * then cycling round to medium sized ones.  A series of squeezings of the same
+ * context should, if laid end-to-end, match a single squeezing of the combined
+ * size.
+ */
+static void test_shake256_tiling2(struct kunit *test)
+{
+	struct shake256_ctx ctx;
+	u8 out[8 + SHA3_512_DIGEST_SIZE + 8];
+
+	for (int first_tile_size = 3;
+	     first_tile_size < SHAKE256_DEFAULT_SIZE;
+	     first_tile_size++) {
+		int tile_size = first_tile_size;
+		int left = SHAKE256_DEFAULT_SIZE;
+		u8 *p = out + 8;
+
+		memset(out, 0, sizeof(out));
+		shake256_init(&ctx);
+		shake256_update(&ctx, test_sha3_sample, sizeof(test_sha3_sample) - 1);
+		while (left > 0) {
+			int part = umin(tile_size, left);
+
+			shake256_squeeze(&ctx, p, part);
+			p += part;
+			left -= part;
+			tile_size--;
+			if (tile_size < 1)
+				tile_size = 5;
+		}
+
+		KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake256, sizeof(test_shake256),
+				       "SHAKE tile %u gives wrong output", tile_size);
+	}
+}
+
+static struct kunit_case hash_test_cases[] = {
+	KUNIT_CASE(test_sha3_224_basic),
+	KUNIT_CASE(test_sha3_256_basic),
+	KUNIT_CASE(test_sha3_384_basic),
+	KUNIT_CASE(test_sha3_512_basic),
+	KUNIT_CASE(test_shake128_basic),
+	KUNIT_CASE(test_shake256_basic),
+	KUNIT_CASE(test_shake128_nist),
+	KUNIT_CASE(test_shake256_nist),
+	KUNIT_CASE(test_shake256_tiling),
+	KUNIT_CASE(test_shake256_tiling2),
+	HASH_KUNIT_CASES,
+	KUNIT_CASE(benchmark_hash),
+	{},
+};
+
+static struct kunit_suite hash_test_suite = {
+	.name = "sha3",
+	.test_cases = hash_test_cases,
+	.suite_init = hash_suite_init,
+	.suite_exit = hash_suite_exit,
+};
+kunit_test_suite(hash_test_suite);
+
+MODULE_DESCRIPTION("KUnit tests and benchmark for SHA3");
+MODULE_LICENSE("GPL");
diff --git a/lib/crypto/tests/sha3_testvecs.h b/lib/crypto/tests/sha3_testvecs.h
new file mode 100644
index 0000000000000..9c4c403cc6e06
--- /dev/null
+++ b/lib/crypto/tests/sha3_testvecs.h
@@ -0,0 +1,231 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/* This file was generated by: ./scripts/crypto/gen-hash-testvecs.py sha3-256 */
+
+static const struct {
+	size_t data_len;
+	u8 digest[SHA3_256_DIGEST_SIZE];
+} hash_testvecs[] = {
+	{
+		.data_len = 0,
+		.digest = {
+			0xa7, 0xff, 0xc6, 0xf8, 0xbf, 0x1e, 0xd7, 0x66,
+			0x51, 0xc1, 0x47, 0x56, 0xa0, 0x61, 0xd6, 0x62,
+			0xf5, 0x80, 0xff, 0x4d, 0xe4, 0x3b, 0x49, 0xfa,
+			0x82, 0xd8, 0x0a, 0x4b, 0x80, 0xf8, 0x43, 0x4a,
+		},
+	},
+	{
+		.data_len = 1,
+		.digest = {
+			0x11, 0x03, 0xe7, 0x84, 0x51, 0x50, 0x86, 0x35,
+			0x71, 0x8a, 0x70, 0xe3, 0xc4, 0x26, 0x7b, 0x21,
+			0x02, 0x13, 0xa0, 0x81, 0xe8, 0xe6, 0x14, 0x25,
+			0x07, 0x34, 0xe5, 0xc5, 0x40, 0x06, 0xf2, 0x8b,
+		},
+	},
+	{
+		.data_len = 2,
+		.digest = {
+			0x2f, 0x6f, 0x6d, 0x47, 0x48, 0x52, 0x11, 0xb9,
+			0xe4, 0x3d, 0xc8, 0x71, 0xcf, 0xb2, 0xee, 0xae,
+			0x5b, 0xf4, 0x12, 0x84, 0x5b, 0x1c, 0xec, 0x6c,
+			0xc1, 0x66, 0x88, 0xaa, 0xc3, 0x40, 0xbd, 0x7e,
+		},
+	},
+	{
+		.data_len = 3,
+		.digest = {
+			0xec, 0x02, 0xe8, 0x81, 0x4f, 0x84, 0x41, 0x69,
+			0x06, 0xd8, 0xdc, 0x1d, 0x01, 0x78, 0xd7, 0xcb,
+			0x39, 0xdf, 0xd3, 0x12, 0x1c, 0x99, 0xfd, 0xf3,
+			0x5c, 0x83, 0xc9, 0xc2, 0x7a, 0x7b, 0x6a, 0x05,
+		},
+	},
+	{
+		.data_len = 16,
+		.digest = {
+			0xff, 0x6f, 0xc3, 0x41, 0xc3, 0x5f, 0x34, 0x6d,
+			0xa7, 0xdf, 0x3e, 0xc2, 0x8b, 0x29, 0xb6, 0xf1,
+			0xf8, 0x67, 0xfd, 0xcd, 0xb1, 0x9f, 0x38, 0x08,
+			0x1d, 0x8d, 0xd9, 0xc2, 0x43, 0x66, 0x18, 0x6c,
+		},
+	},
+	{
+		.data_len = 32,
+		.digest = {
+			0xe4, 0xb1, 0x06, 0x17, 0xf8, 0x8b, 0x91, 0x95,
+			0xe7, 0x57, 0x66, 0xac, 0x08, 0xb2, 0x03, 0x3e,
+			0xf7, 0x84, 0x1f, 0xe3, 0x25, 0xa3, 0x11, 0xd2,
+			0x11, 0xa4, 0x78, 0x74, 0x2a, 0x43, 0x20, 0xa5,
+		},
+	},
+	{
+		.data_len = 48,
+		.digest = {
+			0xeb, 0x57, 0x5f, 0x20, 0xa3, 0x6b, 0xc7, 0xb4,
+			0x66, 0x2a, 0xa0, 0x30, 0x3b, 0x52, 0x00, 0xc9,
+			0xce, 0x6a, 0xd8, 0x1e, 0xbe, 0xed, 0xa1, 0xd1,
+			0xbe, 0x63, 0xc7, 0xe1, 0xe2, 0x66, 0x67, 0x0c,
+		},
+	},
+	{
+		.data_len = 49,
+		.digest = {
+			0xf0, 0x67, 0xad, 0x66, 0xbe, 0xec, 0x5a, 0xfd,
+			0x29, 0xd2, 0x4f, 0x1d, 0xb2, 0x24, 0xb8, 0x90,
+			0x05, 0x28, 0x0e, 0x66, 0x67, 0x74, 0x2d, 0xee,
+			0x66, 0x25, 0x11, 0xd1, 0x76, 0xa2, 0xfc, 0x3a,
+		},
+	},
+	{
+		.data_len = 63,
+		.digest = {
+			0x57, 0x56, 0x21, 0xb3, 0x2d, 0x2d, 0xe1, 0x9d,
+			0xbf, 0x2c, 0x82, 0xa8, 0xad, 0x7e, 0x6c, 0x46,
+			0xfb, 0x30, 0xeb, 0xce, 0xcf, 0xed, 0x2d, 0x65,
+			0xe7, 0xe4, 0x96, 0x69, 0xe0, 0x48, 0xd2, 0xb6,
+		},
+	},
+	{
+		.data_len = 64,
+		.digest = {
+			0x7b, 0xba, 0x67, 0x15, 0xe5, 0x21, 0xc4, 0x69,
+			0xd3, 0xef, 0x5c, 0x97, 0x9f, 0x5b, 0xba, 0x9c,
+			0xfa, 0x55, 0x64, 0xec, 0xb5, 0x37, 0x53, 0x1b,
+			0x3f, 0x4c, 0x0a, 0xed, 0x51, 0x98, 0x2b, 0x52,
+		},
+	},
+	{
+		.data_len = 65,
+		.digest = {
+			0x44, 0xb6, 0x6b, 0x83, 0x09, 0x83, 0x55, 0x83,
+			0xde, 0x1f, 0xcc, 0x33, 0xef, 0xdc, 0x05, 0xbb,
+			0x3b, 0x63, 0x76, 0x45, 0xe4, 0x8e, 0x14, 0x7a,
+			0x2d, 0xae, 0x90, 0xce, 0x68, 0xc3, 0xa4, 0xf2,
+		},
+	},
+	{
+		.data_len = 127,
+		.digest = {
+			0x50, 0x3e, 0x99, 0x4e, 0x28, 0x2b, 0xc9, 0xf4,
+			0xf5, 0xeb, 0x2b, 0x16, 0x04, 0x2d, 0xf5, 0xbe,
+			0xc0, 0x91, 0x41, 0x2a, 0x8e, 0x69, 0x5e, 0x39,
+			0x53, 0x2c, 0xc1, 0x18, 0xa5, 0xeb, 0xd8, 0xda,
+		},
+	},
+	{
+		.data_len = 128,
+		.digest = {
+			0x90, 0x0b, 0xa6, 0x92, 0x84, 0x30, 0xaf, 0xee,
+			0x38, 0x59, 0x83, 0x83, 0xe9, 0xfe, 0xab, 0x86,
+			0x79, 0x1b, 0xcd, 0xe7, 0x0a, 0x0f, 0x58, 0x53,
+			0x36, 0xab, 0x12, 0xe1, 0x5c, 0x97, 0xc1, 0xfb,
+		},
+	},
+	{
+		.data_len = 129,
+		.digest = {
+			0x2b, 0x52, 0x1e, 0x54, 0xbe, 0x38, 0x4c, 0x3e,
+			0x73, 0x37, 0x18, 0xf5, 0x25, 0x2c, 0xc8, 0xc7,
+			0xda, 0x7e, 0xb6, 0x47, 0x9d, 0xf4, 0x46, 0xce,
+			0xfa, 0x80, 0x20, 0x6b, 0xbd, 0xfd, 0x2a, 0xd8,
+		},
+	},
+	{
+		.data_len = 256,
+		.digest = {
+			0x45, 0xf0, 0xf5, 0x9b, 0xd9, 0x91, 0x26, 0xd5,
+			0x91, 0x3b, 0xf8, 0x87, 0x8b, 0x34, 0x02, 0x31,
+			0x64, 0xab, 0xf4, 0x1c, 0x6e, 0x34, 0x72, 0xdf,
+			0x32, 0x6d, 0xe5, 0xd2, 0x67, 0x5e, 0x86, 0x93,
+		},
+	},
+	{
+		.data_len = 511,
+		.digest = {
+			0xb3, 0xaf, 0x71, 0x64, 0xfa, 0xd4, 0xf1, 0x07,
+			0x38, 0xef, 0x04, 0x8e, 0x89, 0xf4, 0x02, 0xd2,
+			0xa5, 0xaf, 0x3b, 0xf5, 0x67, 0x56, 0xcf, 0xa9,
+			0x8e, 0x43, 0xf5, 0xb5, 0xe3, 0x91, 0x8e, 0xe7,
+		},
+	},
+	{
+		.data_len = 513,
+		.digest = {
+			0x51, 0xac, 0x0a, 0x65, 0xb7, 0x96, 0x20, 0xcf,
+			0x88, 0xf6, 0x97, 0x35, 0x89, 0x0d, 0x31, 0x0f,
+			0xbe, 0x17, 0xbe, 0x62, 0x03, 0x67, 0xc0, 0xee,
+			0x4f, 0xc1, 0xe3, 0x7f, 0x6f, 0xab, 0xac, 0xb4,
+		},
+	},
+	{
+		.data_len = 1000,
+		.digest = {
+			0x7e, 0xea, 0xa8, 0xd7, 0xde, 0x20, 0x1b, 0x58,
+			0x24, 0xd8, 0x26, 0x40, 0x36, 0x5f, 0x3f, 0xaa,
+			0xe5, 0x5a, 0xea, 0x98, 0x58, 0xd4, 0xd6, 0xfc,
+			0x20, 0x4c, 0x5c, 0x4f, 0xaf, 0x56, 0xc7, 0xc3,
+		},
+	},
+	{
+		.data_len = 3333,
+		.digest = {
+			0x61, 0xb1, 0xb1, 0x3e, 0x0e, 0x7e, 0x90, 0x3d,
+			0x31, 0x54, 0xbd, 0xc9, 0x0d, 0x53, 0x62, 0xf1,
+			0xcd, 0x18, 0x80, 0xf9, 0x91, 0x75, 0x41, 0xb3,
+			0x51, 0x39, 0x57, 0xa7, 0xa8, 0x1e, 0xfb, 0xc9,
+		},
+	},
+	{
+		.data_len = 4096,
+		.digest = {
+			0xab, 0x29, 0xda, 0x10, 0xc4, 0x11, 0x2d, 0x5c,
+			0xd1, 0xce, 0x1c, 0x95, 0xfa, 0xc6, 0xc7, 0xb0,
+			0x1b, 0xd1, 0xdc, 0x6f, 0xa0, 0x9d, 0x1b, 0x23,
+			0xfb, 0x6e, 0x90, 0x97, 0xd0, 0x75, 0x44, 0x7a,
+		},
+	},
+	{
+		.data_len = 4128,
+		.digest = {
+			0x02, 0x45, 0x95, 0xf4, 0x19, 0xb5, 0x93, 0x29,
+			0x90, 0xf2, 0x63, 0x3f, 0x89, 0xe8, 0xa5, 0x31,
+			0x76, 0xf2, 0x89, 0x79, 0x66, 0xd3, 0x96, 0xdf,
+			0x33, 0xd1, 0xa6, 0x17, 0x73, 0xb1, 0xd0, 0x45,
+		},
+	},
+	{
+		.data_len = 4160,
+		.digest = {
+			0xd1, 0x8e, 0x22, 0xea, 0x44, 0x87, 0x6e, 0x9d,
+			0xfb, 0x36, 0x02, 0x20, 0x63, 0xb7, 0x69, 0x45,
+			0x25, 0x41, 0x69, 0xe0, 0x9b, 0x87, 0xcf, 0xa3,
+			0x51, 0xbb, 0xfc, 0x8d, 0xf7, 0x29, 0xa7, 0xea,
+		},
+	},
+	{
+		.data_len = 4224,
+		.digest = {
+			0x11, 0x86, 0x7d, 0x84, 0xf9, 0x8c, 0x6e, 0xc4,
+			0x64, 0x36, 0xc6, 0xf3, 0x42, 0x92, 0x31, 0x2b,
+			0x1e, 0x12, 0xe6, 0x4d, 0xbe, 0xfa, 0x77, 0x3f,
+			0x89, 0x41, 0x33, 0x58, 0x1c, 0x98, 0x16, 0x0a,
+		},
+	},
+	{
+		.data_len = 16384,
+		.digest = {
+			0xb2, 0xba, 0x0c, 0x8c, 0x9d, 0xbb, 0x1e, 0xb0,
+			0x03, 0xb5, 0xdf, 0x4f, 0xf5, 0x35, 0xdb, 0xec,
+			0x60, 0xf2, 0x5b, 0xb6, 0xd0, 0x49, 0xd3, 0xed,
+			0x55, 0xc0, 0x7a, 0xd7, 0xaf, 0xa1, 0xea, 0x53,
+		},
+	},
+};
+
+static const u8 hash_testvec_consolidated[SHA3_256_DIGEST_SIZE] = {
+	0x3b, 0x33, 0x67, 0xf8, 0xea, 0x92, 0x78, 0x62,
+	0xdd, 0xbe, 0x72, 0x15, 0xbd, 0x6f, 0xfa, 0xe5,
+	0x5e, 0xab, 0x9f, 0xb1, 0xe4, 0x23, 0x7c, 0x2c,
+	0x80, 0xcf, 0x09, 0x75, 0xf8, 0xe2, 0xfa, 0x30,
+};
diff --git a/scripts/crypto/gen-hash-testvecs.py b/scripts/crypto/gen-hash-testvecs.py
index fc063f2ee95f1..cb6f6cfbedeb2 100755
--- a/scripts/crypto/gen-hash-testvecs.py
+++ b/scripts/crypto/gen-hash-testvecs.py
@@ -59,10 +59,14 @@ def hash_init(alg):
 
 def hash_update(ctx, data):
     ctx.update(data)
 
 def hash_final(ctx):
+    if ctx.name == "shake_128":
+        return ctx.digest(16)
+    if ctx.name == "shake_256":
+        return ctx.digest(32)
     return ctx.digest()
 
 def compute_hash(alg, data):
     ctx = hash_init(alg)
     hash_update(ctx, data)
@@ -85,11 +89,11 @@ def print_c_struct_u8_array_field(name, value):
     print('\t\t},')
 
 def alg_digest_size_const(alg):
     if alg == 'blake2s':
         return 'BLAKE2S_HASH_SIZE'
-    return f'{alg.upper()}_DIGEST_SIZE'
+    return f'{alg.upper().replace('-', '_')}_DIGEST_SIZE'
 
 def gen_unkeyed_testvecs(alg):
     print('')
     print('static const struct {')
     print('\tsize_t data_len;')
@@ -120,11 +124,11 @@ def gen_hmac_testvecs(alg):
         key = rand_bytes(key_len)
         mac = hmac.digest(key, data[:data_len], alg)
         ctx.update(mac)
     print_static_u8_array_definition(
             f'hmac_testvec_consolidated[{alg.upper()}_DIGEST_SIZE]',
-            ctx.digest())
+            hash_final(ctx))
 
 BLAKE2S_KEY_SIZE = 32
 BLAKE2S_HASH_SIZE = 32
 
 def gen_additional_blake2s_testvecs():
@@ -162,7 +166,9 @@ print(f'/* This file was generated by: {sys.argv[0]} {" ".join(sys.argv[1:])} */
 gen_unkeyed_testvecs(alg)
 if alg == 'blake2s':
     gen_additional_blake2s_testvecs()
 elif alg == 'poly1305':
     gen_additional_poly1305_testvecs()
+elif alg.startswith('sha3-') or alg.startswith('shake'):
+    pass # no HMAC
 else:
     gen_hmac_testvecs(alg)
-- 
2.51.1.dirty

[PATCH 06/17] lib/crypto: sha3: Fix libsha3 build condition

[Eric Biggers]

The correct condition is there but is commented out.  Fix it.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 lib/crypto/Makefile | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index b7fa51bfeebb5..2f15cef850507 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -266,13 +266,12 @@ libsha512-$(CONFIG_X86) += x86/sha512-ssse3-asm.o \
 			   x86/sha512-avx2-asm.o
 endif # CONFIG_CRYPTO_LIB_SHA512_ARCH
 
 ################################################################################
 
-#obj-$(CONFIG_CRYPTO_LIB_SHA3) += libsha3.o
-#libsha3-y := sha3.o
-obj-$(CONFIG_CRYPTO_SHA3) += sha3.o
+obj-$(CONFIG_CRYPTO_LIB_SHA3) += libsha3.o
+libsha3-y := sha3.o
 
 ################################################################################
 
 obj-$(CONFIG_MPILIB) += mpi/
 
-- 
2.51.1.dirty

[PATCH 07/17] lib/crypto: sha3: Use appropriate conversions in sha3_keccakf_generic()

[Eric Biggers]

For converting from little endian to CPU endian, use le64_to_cpus().
For converting from CPU endian to little endian, use cpu_to_le64s().

No functional change, but this makes the code clearer.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 lib/crypto/sha3.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
index 2c292b0b3db34..8f08e7b8f4521 100644
--- a/lib/crypto/sha3.c
+++ b/lib/crypto/sha3.c
@@ -168,16 +168,16 @@ static void sha3_keccakf_rounds_generic(struct sha3_state *state)
  * loops are no-ops on LE machines and will be optimised away.
  */
 static void sha3_keccakf_generic(struct sha3_state *state)
 {
 	for (int  i = 0; i < ARRAY_SIZE(state->st); i++)
-		cpu_to_le64s(&state->st[i]);
+		le64_to_cpus(&state->st[i]);
 
 	sha3_keccakf_rounds_generic(state);
 
 	for (int  i = 0; i < ARRAY_SIZE(state->st); i++)
-		le64_to_cpus(&state->st[i]);
+		cpu_to_le64s(&state->st[i]);
 }
 
 static void sha3_absorb_block_generic(struct sha3_ctx *ctx, const u8 *data)
 {
 	struct sha3_state *state = &ctx->state;
-- 
2.51.1.dirty

[PATCH 08/17] lib/crypto: sha3: Drop unfinished SHAKE support from gen-hash-testvecs.py

[Eric Biggers]

The SHAKE testing doesn't actually use gen-hash-testvecs.py, so remove
the unfinished SHAKE support from it.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 scripts/crypto/gen-hash-testvecs.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/scripts/crypto/gen-hash-testvecs.py b/scripts/crypto/gen-hash-testvecs.py
index cb6f6cfbedeb2..ba241cb1ed2fd 100755
--- a/scripts/crypto/gen-hash-testvecs.py
+++ b/scripts/crypto/gen-hash-testvecs.py
@@ -59,14 +59,10 @@ def hash_init(alg):
 
 def hash_update(ctx, data):
     ctx.update(data)
 
 def hash_final(ctx):
-    if ctx.name == "shake_128":
-        return ctx.digest(16)
-    if ctx.name == "shake_256":
-        return ctx.digest(32)
     return ctx.digest()
 
 def compute_hash(alg, data):
     ctx = hash_init(alg)
     hash_update(ctx, data)
@@ -166,9 +162,9 @@ print(f'/* This file was generated by: {sys.argv[0]} {" ".join(sys.argv[1:])} */
 gen_unkeyed_testvecs(alg)
 if alg == 'blake2s':
     gen_additional_blake2s_testvecs()
 elif alg == 'poly1305':
     gen_additional_poly1305_testvecs()
-elif alg.startswith('sha3-') or alg.startswith('shake'):
+elif alg.startswith('sha3-'):
     pass # no HMAC
 else:
     gen_hmac_testvecs(alg)
-- 
2.51.1.dirty

[PATCH 09/17] lib/crypto: sha3: Consistently use EXPORT_SYMBOL_GPL

[Eric Biggers]

Consistently use EXPORT_SYMBOL_GPL in sha3.c, instead of a mix of
EXPORT_SYMBOL and EXPORT_SYMBOL_GPL.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 lib/crypto/sha3.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
index 8f08e7b8f4521..f8167037138c7 100644
--- a/lib/crypto/sha3.c
+++ b/lib/crypto/sha3.c
@@ -329,11 +329,11 @@ void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE])
 
 	sha3_224_init(&ctx);
 	sha3_224_update(&ctx, in, in_len);
 	sha3_224_final(&ctx, out);
 }
-EXPORT_SYMBOL(sha3_224);
+EXPORT_SYMBOL_GPL(sha3_224);
 
 /**
  * sha3_256() - Convenience wrapper to digest a simple buffer as SHA3-256
  * @in: The data to be digested
  * @in_len: The amount of data to be digested in bytes
@@ -350,11 +350,11 @@ void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE])
 
 	sha3_256_init(&ctx);
 	sha3_256_update(&ctx, in, in_len);
 	sha3_256_final(&ctx, out);
 }
-EXPORT_SYMBOL(sha3_256);
+EXPORT_SYMBOL_GPL(sha3_256);
 
 /**
  * sha3_384() - Convenience wrapper to digest a simple buffer as SHA3-384
  * @in: The data to be digested
  * @in_len: The amount of data to be digested in bytes
@@ -371,11 +371,11 @@ void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE])
 
 	sha3_384_init(&ctx);
 	sha3_384_update(&ctx, in, in_len);
 	sha3_384_final(&ctx, out);
 }
-EXPORT_SYMBOL(sha3_384);
+EXPORT_SYMBOL_GPL(sha3_384);
 
 /**
  * sha3_512() - Convenience wrapper to digest a simple buffer as SHA3-512
  * @in: The data to be digested in bytes
  * @in_len: The amount of data to be digested in bytes
@@ -392,11 +392,11 @@ void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE])
 
 	sha3_512_init(&ctx);
 	sha3_512_update(&ctx, in, in_len);
 	sha3_512_final(&ctx, out);
 }
-EXPORT_SYMBOL(sha3_512);
+EXPORT_SYMBOL_GPL(sha3_512);
 
 /**
  * shake128() - Convenience wrapper to apply SHAKE128 to a simple buffer
  * @in: The input data to be used
  * @in_len: The amount of input data in bytes
@@ -416,11 +416,11 @@ void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 	shake128_init(&ctx);
 	shake128_update(&ctx, in, in_len);
 	shake128_squeeze(&ctx, out, out_len);
 	shake128_clear(&ctx);
 }
-EXPORT_SYMBOL(shake128);
+EXPORT_SYMBOL_GPL(shake128);
 
 /**
  * shake256() - Convenience wrapper to apply SHAKE256 to a simple buffer
  * @in: The input data to be used
  * @in_len: The amount of input data in bytes
@@ -440,11 +440,11 @@ void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 	shake256_init(&ctx);
 	shake256_update(&ctx, in, in_len);
 	shake256_squeeze(&ctx, out, out_len);
 	shake256_clear(&ctx);
 }
-EXPORT_SYMBOL(shake256);
+EXPORT_SYMBOL_GPL(shake256);
 
 /*
  * Do a quick test using SHAKE256 and a 200 byte digest.
  */
 static const u8 sha3_sample[] __initconst =
-- 
2.51.1.dirty

[PATCH 10/17] lib/crypto: sha3: Replace redundant ad-hoc test with FIPS test

[Eric Biggers]

Since SHA-3 is a FIPS-approved algorithm, add a FIPS cryptographic
algorithm self-test to it, following the example of SHA-1 and SHA-2.
Make it replace the existing ad-hoc test.  The actual correctness tests
are in the KUnit test suite; the boot-time testing is just for FIPS.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 Documentation/crypto/sha3.rst       | 12 ++++--
 lib/crypto/fips.h                   |  7 ++++
 lib/crypto/sha3.c                   | 63 +++++++----------------------
 scripts/crypto/gen-fips-testvecs.py |  4 ++
 4 files changed, 33 insertions(+), 53 deletions(-)

diff --git a/Documentation/crypto/sha3.rst b/Documentation/crypto/sha3.rst
index 3255618ea2ec3..6d7a07e088e9a 100644
--- a/Documentation/crypto/sha3.rst
+++ b/Documentation/crypto/sha3.rst
@@ -9,11 +9,11 @@ SHA-3 Algorithm collection
   - Overview
   - Basic API
     - Extendable-Output Functions
   - Convenience API
   - Internal API
-    - Testing
+  - Testing
   - References
   - API Function Reference
 
 
 Overview
@@ -219,14 +219,18 @@ extracted/squeezed from the state and for the state to be cleared.  Note that
 there is no "final" function, per se, but that can be constructed by squeezing
 and clearing.
 
 
 Testing
--------
+=======
 
-The sha3 module does a basic sanity test on initialisation, but there is also a
-kunit test module available.
+To test the SHA-3 code, use sha3_kunit.
+
+Since the SHA-3 algorithms are FIPS-approved, when the kernel is booted in FIPS
+mode the SHA-3 library also performs a simple self-test.  This is purely to meet
+a FIPS requirement.  Normal testing done by kernel developers and integrators
+should use the much more comprehensive KUnit test suite instead.
 
 
 References
 ==========
 
diff --git a/lib/crypto/fips.h b/lib/crypto/fips.h
index 78a1bdd33a151..023410c2e0dbc 100644
--- a/lib/crypto/fips.h
+++ b/lib/crypto/fips.h
@@ -34,5 +34,12 @@ static const u8 fips_test_hmac_sha512_value[] __initconst __maybe_unused = {
 	0x92, 0x7e, 0x3c, 0xbd, 0xb1, 0x3c, 0x49, 0x98,
 	0x44, 0x9c, 0x8f, 0xee, 0x3f, 0x02, 0x71, 0x51,
 	0x57, 0x0b, 0x15, 0x38, 0x95, 0xd8, 0xa3, 0x81,
 	0xba, 0xb3, 0x15, 0x37, 0x5c, 0x6d, 0x57, 0x2b,
 };
+
+static const u8 fips_test_sha3_256_value[] __initconst __maybe_unused = {
+	0x77, 0xc4, 0x8b, 0x69, 0x70, 0x5f, 0x0a, 0xb1,
+	0xb1, 0xa5, 0x82, 0x0a, 0x22, 0x2b, 0x49, 0x31,
+	0xba, 0x9b, 0xb6, 0xaa, 0x32, 0xa7, 0x97, 0x00,
+	0x98, 0xdb, 0xff, 0xe7, 0xc6, 0xde, 0xb5, 0x82,
+};
diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
index f8167037138c7..83fbfa1169cdb 100644
--- a/lib/crypto/sha3.c
+++ b/lib/crypto/sha3.c
@@ -15,10 +15,11 @@
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 #include <crypto/sha3.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/unaligned.h>
+#include "fips.h"
 
 /*
  * On some 32-bit architectures, such as h8300, GCC ends up using over 1 KB of
  * stack if the round calculation gets inlined into the loop in
  * sha3_keccakf_rounds_generic().  On the other hand, on 64-bit architectures
@@ -442,71 +443,35 @@ void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 	shake256_squeeze(&ctx, out, out_len);
 	shake256_clear(&ctx);
 }
 EXPORT_SYMBOL_GPL(shake256);
 
-/*
- * Do a quick test using SHAKE256 and a 200 byte digest.
- */
-static const u8 sha3_sample[] __initconst =
-	"The quick red fox jumped over the lazy brown dog!\n"
-	"The quick red fox jumped over the lazy brown dog!\n"
-	"The quick red fox jumped over the lazy brown dog!\n"
-	"The quick red fox jumped over the lazy brown dog!\n";
-
-static const u8 sha3_sample_shake256_200[] __initconst = {
-	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-before guard */
-	0xab, 0x06, 0xd4, 0xf9, 0x8b, 0xfd, 0xb2, 0xc4, 0xfe, 0xf1, 0xcc, 0xe2,
-	0x40, 0x45, 0xdd, 0x15, 0xcb, 0xdd, 0x02, 0x8d, 0xb7, 0x9f, 0x1e, 0x67,
-	0xd6, 0x7f, 0x98, 0x5e, 0x1b, 0x19, 0xf8, 0x01, 0x43, 0x82, 0xcb, 0xd8,
-	0x5d, 0x21, 0x64, 0xa8, 0x80, 0xc9, 0x22, 0xe5, 0x07, 0xaf, 0xe2, 0x5d,
-	0xcd, 0xc6, 0x23, 0x36, 0x2b, 0xc7, 0xc7, 0x7d, 0x09, 0x9d, 0x68, 0x05,
-	0xe4, 0x62, 0x63, 0x1b, 0x67, 0xbc, 0xf8, 0x95, 0x07, 0xd2, 0xe4, 0xd0,
-	0xba, 0xa2, 0x67, 0xf5, 0xe3, 0x15, 0xbc, 0x85, 0xa1, 0x50, 0xd6, 0x6f,
-	0x6f, 0xd4, 0x54, 0x4c, 0x3f, 0x4f, 0xe5, 0x1f, 0xb7, 0x00, 0x27, 0xfc,
-	0x15, 0x33, 0xc2, 0xf9, 0xb3, 0x4b, 0x9e, 0x81, 0xe5, 0x96, 0xbe, 0x05,
-	0x6c, 0xac, 0xf9, 0x9f, 0x65, 0x36, 0xbb, 0x11, 0x47, 0x6d, 0xf6, 0x8f,
-	0x9f, 0xa2, 0x77, 0x37, 0x3b, 0x18, 0x77, 0xcf, 0x65, 0xc5, 0xa1, 0x7e,
-	0x2c, 0x0e, 0x71, 0xf0, 0x4d, 0x18, 0x67, 0xb9, 0xc4, 0x8c, 0x64, 0x3b,
-	0x4b, 0x45, 0xea, 0x16, 0xb2, 0x4a, 0xc5, 0xf5, 0x85, 0xdc, 0xd2, 0xd9,
-	0x13, 0x77, 0xb3, 0x19, 0xd9, 0x8c, 0x9f, 0x28, 0xe7, 0x64, 0x91, 0x0f,
-	0x6f, 0x32, 0xbf, 0xa8, 0xa8, 0xa3, 0xff, 0x99, 0x0e, 0x0b, 0x62, 0x50,
-	0xf8, 0x3a, 0xc2, 0xf5, 0x98, 0x21, 0xeb, 0x9d, 0xe8, 0x45, 0xf4, 0x46,
-	0x1e, 0x8b, 0xbd, 0x10, 0x59, 0x2c, 0x87, 0xe2,
-	0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* Write-after guard */
-};
-
+#if defined(sha3_mod_init_arch) || defined(CONFIG_CRYPTO_FIPS)
 static int __init sha3_mod_init(void)
 {
-#define out_len 200
-	u8 out[8 + out_len + 8] = {};
-
 #ifdef sha3_mod_init_arch
 	sha3_mod_init_arch();
 #endif
-
-	BUILD_BUG_ON(sizeof(out) != sizeof(sha3_sample_shake256_200));
-
-	shake256(sha3_sample, sizeof(sha3_sample) - 1, out + 8, out_len);
-
-	if (memcmp(out, sha3_sample_shake256_200,
-		   sizeof(sha3_sample_shake256_200)) != 0) {
-		pr_err("SHAKE256(200) failed\n");
-		for (size_t i = 0; i < out_len;) {
-			size_t part = min(out_len - i, 32);
-
-			pr_err("%*phN\n", (int)part, out + i);
-			i += part;
-		}
-		return -EBADMSG;
+	if (fips_enabled) {
+		/*
+		 * FIPS cryptographic algorithm self-test.  As per the FIPS
+		 * Implementation Guidance, testing any SHA-3 algorithm
+		 * satisfies the test requirement for all of them.
+		 */
+		u8 hash[SHA3_256_DIGEST_SIZE];
+
+		sha3_256(fips_test_data, sizeof(fips_test_data), hash);
+		if (memcmp(fips_test_sha3_256_value, hash, sizeof(hash)) != 0)
+			panic("sha3: FIPS self-test failed\n");
 	}
 	return 0;
 }
 subsys_initcall(sha3_mod_init);
 
 static void __exit sha3_mod_exit(void)
 {
 }
 module_exit(sha3_mod_exit);
+#endif
 
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("SHA-3 Secure Hash Algorithm");
diff --git a/scripts/crypto/gen-fips-testvecs.py b/scripts/crypto/gen-fips-testvecs.py
index 2956f88b764ae..db873f88619ac 100755
--- a/scripts/crypto/gen-fips-testvecs.py
+++ b/scripts/crypto/gen-fips-testvecs.py
@@ -3,10 +3,11 @@
 #
 # Script that generates lib/crypto/fips.h
 #
 # Copyright 2025 Google LLC
 
+import hashlib
 import hmac
 
 fips_test_data = b"fips test data\0\0"
 fips_test_key = b"fips test key\0\0\0"
 
@@ -28,5 +29,8 @@ print_static_u8_array_definition("fips_test_key", fips_test_key)
 
 for alg in 'sha1', 'sha256', 'sha512':
     ctx = hmac.new(fips_test_key, digestmod=alg)
     ctx.update(fips_test_data)
     print_static_u8_array_definition(f'fips_test_hmac_{alg}_value', ctx.digest())
+
+print_static_u8_array_definition(f'fips_test_sha3_256_value',
+                                 hashlib.sha3_256(fips_test_data).digest())
-- 
2.51.1.dirty

[PATCH 11/17] lib/crypto: sha3: Simplify the API

[Eric Biggers]

Instead of having separate types and functions for each of the six SHA-3
algorithms, instead divide them into two groups: the digests and the
XOFs.  The digests use sha3_ctx and the XOFs use shake_ctx.  The
internal context is now called __sha3_ctx.

The result is a simpler, smaller API that is more convenient for users
that need to support multiple SHA-3 variants, e.g. crypto/sha3.c.

It still preserves the property that the digests can't be incorrectly
treated as XOFs and vice versa.

It does result in the explicit array bounds for the output digests being
lost from the incremental API.  However, explicit array bounds don't
really work properly in C anyway.  They're nice to have as hints, but
not worth duplicating all the update and final functions for.  Note that
the BLAKE2s and BLAKE2b code uses this same solution.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 Documentation/crypto/sha3.rst | 194 +++++------------
 include/crypto/sha3.h         | 386 +++++++++-------------------------
 lib/crypto/sha3.c             |  87 +++-----
 lib/crypto/tests/sha3_kunit.c |  20 +-
 4 files changed, 184 insertions(+), 503 deletions(-)

diff --git a/Documentation/crypto/sha3.rst b/Documentation/crypto/sha3.rst
index 6d7a07e088e9a..c27da98c89b7f 100644
--- a/Documentation/crypto/sha3.rst
+++ b/Documentation/crypto/sha3.rst
@@ -5,14 +5,13 @@ SHA-3 Algorithm collection
 ==========================
 
 .. Contents:
 
   - Overview
-  - Basic API
-    - Extendable-Output Functions
+  - Digests
+  - Extendable-Output Functions
   - Convenience API
-  - Internal API
   - Testing
   - References
   - API Function Reference
 
 
@@ -39,95 +38,75 @@ and two Extendable-Output Functions (XOF):
 
  - SHAKE128
  - SHAKE256
 
 If selectable algorithms are required then the crypto_hash API may be used
-instead as this binds each algorithm to a specific C type.
-
-
-Basic API
-=========
-
-The basic API has a separate context struct for each algorithm in the SHA3
-suite, none of the contents of which are expected to be accessed directly::
-
-	struct sha3_224_ctx { ... };
-	struct sha3_256_ctx { ... };
-	struct sha3_384_ctx { ... };
-	struct sha3_512_ctx { ... };
-	struct shake128_ctx { ... };
-	struct shake256_ctx { ... };
-
-There are a collection of initialisation functions, one for each algorithm
-supported, that initialise the context appropriately for that algorithm::
-
-	void sha3_224_init(struct sha3_224_ctx *ctx);
-	void sha3_256_init(struct sha3_256_ctx *ctx);
-	void sha3_384_init(struct sha3_384_ctx *ctx);
-	void sha3_512_init(struct sha3_512_ctx *ctx);
-	void shake128_init(struct shake128_ctx *ctx);
-	void shake256_init(struct shake256_ctx *ctx);
-
-Data is then added with the appropriate update function, again one per
-algorithm::
-
-	void sha3_224_update(struct sha3_224_ctx *ctx,
-			     const u8 *data, size_t len);
-	void sha3_256_update(struct sha3_256_ctx *ctx,
-			     const u8 *data, size_t len);
-	void sha3_384_update(struct sha3_384_ctx *ctx,
-			     const u8 *data, size_t len);
-	void sha3_512_update(struct sha3_512_ctx *ctx,
-			     const u8 *data, size_t len);
-	void shake128_update(struct shake128_ctx *ctx,
-			     const u8 *data, size_t len);
-	void shake256_update(struct shake256_ctx *ctx,
-			     const u8 *data, size_t len);
+instead.
+
+
+Digests
+=======
+
+The SHA-3 digest API uses the following struct::
+
+	struct sha3_ctx { ... };
+
+There are a collection of initialization functions, one for each algorithm::
+
+	void sha3_224_init(struct sha3_ctx *ctx);
+	void sha3_256_init(struct sha3_ctx *ctx);
+	void sha3_384_init(struct sha3_ctx *ctx);
+	void sha3_512_init(struct sha3_ctx *ctx);
+
+Data is then added with::
+
+	void sha3_update(struct sha3_ctx *ctx, const u8 *in, size_t in_len);
 
 The update function may be called multiple times if need be to add
 non-contiguous data.
 
-For digest algorithms, the digest is finalised and extracted with the
-algorithm-specific function::
+Finally, the digest is generated using::
 
-	void sha3_224_final(struct sha3_224_ctx *ctx,
-			    u8 out[SHA3_224_DIGEST_SIZE]);
-	void sha3_256_final(struct sha3_256_ctx *ctx,
-			    u8 out[SHA3_256_DIGEST_SIZE]);
-	void sha3_384_final(struct sha3_384_ctx *ctx,
-			    u8 out[SHA3_384_DIGEST_SIZE]);
-	void sha3_512_final(struct sha3_512_ctx *ctx,
-			    u8 out[SHA3_512_DIGEST_SIZE]);
-
-which also explicitly clears the context.  The amount of data extracted is
-determined by the type.
+	void sha3_final(struct sha3_ctx *ctx, u8 *out);
 
+which also zeroizes the context.  The length of the digest is determined by the
+initialization function that was called.
 
 Extendable-Output Functions
----------------------------
+===========================
+
+The SHA-3 extendable-output function (XOF) API uses the following struct::
 
-For XOFs, once the data has been added to a context, a variable amount of data
-may be extracted.  This can be done by calling the appropriate squeeze
-function::
+	struct shake_ctx { ... };
 
-	void shake128_squeeze(struct shake128_ctx *ctx, u8 *out, size_t out_len);
-	void shake256_squeeze(struct shake256_ctx *ctx, u8 *out, size_t out_len);
+Initialization is done with one of::
+
+	void shake128_init(struct shake_ctx *ctx);
+	void shake256_init(struct shake_ctx *ctx);
+
+Data is then added with::
+
+	void shake_update(struct shake_ctx *ctx, const u8 *in, size_t in_len);
+
+The update function may be called multiple times if need be to add
+non-contiguous data.
+
+Finally, the output is extracted using::
+
+	void shake_squeeze(struct shake_ctx *ctx, u8 *out, size_t out_len);
 
 and telling it how much data should be extracted.  The squeeze function may be
 called multiple times but it will only append the domain separation suffix on
 the first invocation.
 
 Note that performing a number of squeezes, with the output laid consequitively
 in a buffer, gets exactly the same output as doing a single squeeze for the
 combined amount over the same buffer.
 
-Once all the desired output has been extracted, the context should be cleared
-with the clear function appropriate to the algorithm::
-
-	void shake128_clear(struct shake128_ctx *ctx);
-	void shake256_clear(struct shake256_ctx *ctx);
+Once all the desired output has been extracted, zeroize the context::
 
+	void shake_zeroize_ctx(struct shake_ctx *ctx);
 
 Convenience API
 ===============
 
 It only a single contiguous buffer of input needs to be added and only a single
@@ -141,87 +120,10 @@ function for each algorithm supported::
 	void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE]);
 	void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len);
 	void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len);
 
 
-Internal API
-============
-
-There is a common internal API underlying all of this that may be used to build
-further algorithms or APIs as the engine in the same in all cases.  The
-algorithm APIs all wrap the common context structure::
-
-	struct sha3_ctx {
-		struct sha3_state	state;
-		u8			block_size;
-		u8			padding;
-		u8			absorb_offset;
-		u8			squeeze_offset;
-		bool			end_marked;
-	};
-
-	struct sha3_state {
-		u64			st[SHA3_STATE_SIZE / 8];
-	};
-
-The fields are as follows:
-
- * ``state.st``
-
-   An array of 25 64-bit state buckets that are used to hold the mathematical
-   state of the Keccak engine.  Data is XOR'd onto part of this, the engine is
-   cranked and then the output is copied from this.
-
-   For the convenience of adding input and extract output from it, the array is
-   kept in little-endian order most of the time, but is byteswapped to
-   host-endian in order to perform the Keccak function and then byteswapped
-   back again.  On an LE machine, the byteswapping is a no-op.
-
- * ``block_size``
-
-   The size of the block of state that can be updated or extracted at a time.
-   This is related to the algorithm size and is analogous to the "rate" in the
-   algorithm definition.
-
- * ``padding``
-
-   The terminating byte to add when finalising the stat.  This may differ
-   between algorithms.
-
- * ``absorb_offset``
-
-   This tracks which is the next byte of state to be updated; when it hits
-   ``block_size``, the engine is cranked and this is reset to 0.
-
- * ``squeeze_offset``
-
-   This tracks which is the next byte of state to be extracted; similar to
-   ``partial``, when it hits ``block_size``, if more output is requested, the
-   engine is cranked to generate more and this is reset to 0.
-
- * ``end_marked``
-
-   This is set to true when the domain separation suffix and any padding have
-   been appended to the state to prevent multiple squeezings from XOF
-   algorithms from re-appending this.
-
-Note that the size of the digest is *not* included here as that's only needed
-at finalisation time for digest algorithms and can be supplied then.  It is not
-relevant to XOFs.
-
-To make use of the context, the following internal functions are provided::
-
-	void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len);
-	void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_len);
-	void sha3_clear(struct sha3_ctx *ctx);
-
-These allow data to be appended to/absorbed into the state, output to be
-extracted/squeezed from the state and for the state to be cleared.  Note that
-there is no "final" function, per se, but that can be constructed by squeezing
-and clearing.
-
-
 Testing
 =======
 
 To test the SHA-3 code, use sha3_kunit.
 
@@ -239,7 +141,7 @@ References
 
 
 API Function Reference
 ======================
 
-.. kernel-doc:: crypto/lib/sha3.c
+.. kernel-doc:: lib/crypto/sha3.c
 .. kernel-doc:: include/crypto/sha3.h
diff --git a/include/crypto/sha3.h b/include/crypto/sha3.h
index a54117cb1546b..58d2b3e0663e8 100644
--- a/include/crypto/sha3.h
+++ b/include/crypto/sha3.h
@@ -36,427 +36,231 @@
 
 #define SHA3_STATE_SIZE		200
 
 struct shash_desc;
 
+int crypto_sha3_init(struct shash_desc *desc);
+
 struct sha3_state {
 	u64		st[SHA3_STATE_SIZE / 8];
 };
 
-/*
- * The SHA3 context structure and state buffer.
+/**
+ * Internal context, shared by the digests (SHA3-*) and the XOFs (SHAKE*).
  *
  * To avoid the need to byteswap when adding input and extracting output from
  * the state array, the state array is kept in little-endian order most of the
  * time, but is byteswapped to host-endian in order to perform the Keccak
  * function and then byteswapped back again after.  On a LE machine, the
  * byteswap step is a no-op.
  */
-struct sha3_ctx {
+struct __sha3_ctx {
 	struct sha3_state	state;
 	u8			block_size;	/* Block size in bytes */
 	u8			padding;	/* Padding byte */
 	u8			absorb_offset;	/* Next state byte to absorb into */
 	u8			squeeze_offset;	/* Next state byte to extract */
 	bool			end_marked;	/* T if end marker inserted */
 };
 
-int crypto_sha3_init(struct shash_desc *desc);
+void __sha3_update(struct __sha3_ctx *ctx, const u8 *in, size_t in_len);
+void __sha3_squeeze(struct __sha3_ctx *ctx, u8 *out, size_t out_len);
+
+/** Context for SHA3-224, SHA3-256, SHA3-384, or SHA3-512 */
+struct sha3_ctx {
+	struct __sha3_ctx	ctx;
+	u8			digest_size;	/* Digest size in bytes */
+};
 
 /**
- * sha3_clear() - Explicitly clear the entire context
- * @ctx: the context to clear
- *
- * Explicitly clear the entire context, including the type parameters; after
- * this, the context must be fully initialized again.
- *
- * Context: Any context.
+ * Zeroize a sha3_ctx.  This is already called by sha3_final().  Call this
+ * explicitly when abandoning a context without calling sha3_final().
  */
-static inline void sha3_clear(struct sha3_ctx *ctx)
+static inline void sha3_zeroize_ctx(struct sha3_ctx *ctx)
 {
 	memzero_explicit(ctx, sizeof(*ctx));
 }
 
-void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len);
-void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_len);
-
-/*
- * Context wrapper for SHA3-224.
- */
-struct sha3_224_ctx {
-	struct sha3_ctx ctx;
+/** Context for SHAKE128 or SHAKE256 */
+struct shake_ctx {
+	struct __sha3_ctx ctx;
 };
 
-/**
- * sha3_224_init() - Set a SHA3 context for SHA3-224
- * @ctx: the context to initialize
- *
- * Initialize a SHA3 context for the production of a SHA3-224 digest of a
- * message.
- *
- * Context: Any context.
- */
-static inline void sha3_224_init(struct sha3_224_ctx *ctx)
-{
-	*ctx = (struct sha3_224_ctx){
-		.ctx.block_size	= SHA3_224_BLOCK_SIZE,
-		.ctx.padding	= 0x06,
-	};
-}
-
-/**
- * sha3_224_update() - Update a SHA3-224 hash with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
- *
- * This can be called any number of times to add data to the hash, performing
- * the "keccak sponge absorbing" phase.
- *
- * Context: Any context.
- */
-static inline void sha3_224_update(struct sha3_224_ctx *ctx, const u8 *data, size_t len)
+/** Zeroize a shake_ctx.  Call this after the last squeeze. */
+static inline void shake_zeroize_ctx(struct shake_ctx *ctx)
 {
-	return sha3_update(&ctx->ctx, data, len);
+	memzero_explicit(ctx, sizeof(*ctx));
 }
 
 /**
- * sha3_224_final() - Finalise a SHA3-224 hash and extract the digest
- * @ctx: The context to finalise; must have been initialized
- * @out: Where to write the resulting message digest
+ * sha3_224_init() - Initialize a context for SHA3-224
+ * @ctx: The context to initialize
  *
- * Finish the computation of a SHA3-224 hash and perform the "Keccak sponge
- * squeezing" phase.  The digest is written to @out buffer and the context will
- * be completely zeroed out.
+ * This begins a new SHA3-224 message digest computation.
  *
  * Context: Any context.
  */
-static inline void sha3_224_final(struct sha3_224_ctx *ctx, u8 out[SHA3_224_DIGEST_SIZE])
+static inline void sha3_224_init(struct sha3_ctx *ctx)
 {
-	sha3_squeeze(&ctx->ctx, out, SHA3_224_DIGEST_SIZE);
-	sha3_clear(&ctx->ctx);
+	*ctx = (struct sha3_ctx){
+		.digest_size	= SHA3_224_DIGEST_SIZE,
+		.ctx.block_size	= SHA3_224_BLOCK_SIZE,
+		.ctx.padding	= 0x06,
+	};
 }
 
-/*
- * Context wrapper for SHA3-256.
- */
-struct sha3_256_ctx {
-	struct sha3_ctx ctx;
-};
-
 /**
- * sha3_256_init() - Set a SHA3 context for SHA3-256
- * @ctx: the context to initialize
+ * sha3_256_init() - Initialize a context for SHA3-256
+ * @ctx: The context to initialize
  *
- * Initialize a SHA3 context for the production of a SHA3-256 digest of a
- * message.
+ * This begins a new SHA3-256 message digest computation.
  *
  * Context: Any context.
  */
-static inline void sha3_256_init(struct sha3_256_ctx *ctx)
+static inline void sha3_256_init(struct sha3_ctx *ctx)
 {
-	*ctx = (struct sha3_256_ctx){
+	*ctx = (struct sha3_ctx){
+		.digest_size	= SHA3_256_DIGEST_SIZE,
 		.ctx.block_size	= SHA3_256_BLOCK_SIZE,
 		.ctx.padding	= 0x06,
 	};
 }
 
 /**
- * sha3_256_update() - Update a SHA3-256 hash with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
- *
- * This can be called any number of times to add data to the hash, performing
- * the "keccak sponge absorbing" phase.
- *
- * Context: Any context.
- */
-static inline void sha3_256_update(struct sha3_256_ctx *ctx, const u8 *data, size_t len)
-{
-	return sha3_update(&ctx->ctx, data, len);
-}
-
-/**
- * sha3_256_final() - Finalise a SHA3-256 hash and extract the digest
- * @ctx: The context to finalise; must have been initialized
- * @out: Where to write the resulting message digest
- *
- * Finish the computation of a SHA3-256 hash and perform the "Keccak sponge
- * squeezing" phase.  The digest is written to @out buffer and the context will
- * be completely zeroed out.
- *
- * Context: Any context.
- */
-static inline void sha3_256_final(struct sha3_256_ctx *ctx, u8 out[SHA3_256_DIGEST_SIZE])
-{
-	sha3_squeeze(&ctx->ctx, out, SHA3_256_DIGEST_SIZE);
-	sha3_clear(&ctx->ctx);
-}
-
-/*
- * Context wrapper for SHA3-384.
- */
-struct sha3_384_ctx {
-	struct sha3_ctx ctx;
-};
-
-/**
- * sha3_384_init() - Set a SHA3 context for SHA3-384
- * @ctx: the context to initialize
+ * sha3_384_init() - Initialize a context for SHA3-384
+ * @ctx: The context to initialize
  *
- * Initialize a SHA3 context for the production of a SHA3-384 digest of a
- * message.
+ * This begins a new SHA3-384 message digest computation.
  *
  * Context: Any context.
  */
-static inline void sha3_384_init(struct sha3_384_ctx *ctx)
+static inline void sha3_384_init(struct sha3_ctx *ctx)
 {
-	*ctx = (struct sha3_384_ctx){
+	*ctx = (struct sha3_ctx){
+		.digest_size	= SHA3_384_DIGEST_SIZE,
 		.ctx.block_size	= SHA3_384_BLOCK_SIZE,
 		.ctx.padding	= 0x06,
 	};
 }
 
 /**
- * sha3_384_update() - Update a SHA3-384 hash with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
- *
- * This can be called any number of times to add data to the hash, performing
- * the "keccak sponge absorbing" phase.
- *
- * Context: Any context.
- */
-static inline void sha3_384_update(struct sha3_384_ctx *ctx, const u8 *data, size_t len)
-{
-	return sha3_update(&ctx->ctx, data, len);
-}
-
-/**
- * sha3_384_final() - Finalise a SHA3-384 hash and extract the digest
- * @ctx: The context to finalise; must have been initialized
- * @out: Where to write the resulting message digest
- *
- * Finish the computation of a SHA3-384 hash and perform the "Keccak sponge
- * squeezing" phase.  The digest is written to @out buffer and the context will
- * be completely zeroed out.
- *
- * Context: Any context.
- */
-static inline void sha3_384_final(struct sha3_384_ctx *ctx, u8 out[SHA3_384_DIGEST_SIZE])
-{
-	sha3_squeeze(&ctx->ctx, out, SHA3_384_DIGEST_SIZE);
-	sha3_clear(&ctx->ctx);
-}
-
-/*
- * Context wrapper for SHA3-512.
- */
-struct sha3_512_ctx {
-	struct sha3_ctx ctx;
-};
-
-/**
- * sha3_512_init() - Set a SHA3 context for SHA3-512
- * @ctx: the context to initialize
+ * sha3_512_init() - Initialize a context for SHA3-512
+ * @ctx: The context to initialize
  *
- * Initialize a SHA3 context for the production of a SHA3-512 digest of a
- * message.
+ * This begins a new SHA3-512 message digest computation.
  *
  * Context: Any context.
  */
-static inline void sha3_512_init(struct sha3_512_ctx *ctx)
+static inline void sha3_512_init(struct sha3_ctx *ctx)
 {
-	*ctx = (struct sha3_512_ctx){
+	*ctx = (struct sha3_ctx){
+		.digest_size	= SHA3_512_DIGEST_SIZE,
 		.ctx.block_size	= SHA3_512_BLOCK_SIZE,
 		.ctx.padding	= 0x06,
 	};
 }
 
 /**
- * sha3_512_update() - Update a SHA3-512 hash with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
+ * sha3_update() - Update a SHA-3 digest context with input data
+ * @ctx: The context to update; must have been initialized
+ * @in: The input data
+ * @in_len: Length of the input data in bytes
  *
- * This can be called any number of times to add data to the hash, performing
- * the "keccak sponge absorbing" phase.
+ * This can be called any number of times to add data to a SHA3-224, SHA3-256,
+ * SHA3-384, or SHA3-512 digest (depending on which init function was called).
  *
  * Context: Any context.
  */
-static inline void sha3_512_update(struct sha3_512_ctx *ctx, const u8 *data, size_t len)
+static inline void sha3_update(struct sha3_ctx *ctx,
+			       const u8 *in, size_t in_len)
 {
-	return sha3_update(&ctx->ctx, data, len);
+	__sha3_update(&ctx->ctx, in, in_len);
 }
 
 /**
- * sha3_512_final() - Finalise a SHA3-512 hash and extract the digest
- * @ctx: The context to finalise; must have been initialized
- * @out: Where to write the resulting message digest
+ * sha3_final() - Finish computing a SHA-3 message digest
+ * @ctx: The context to finalize; must have been initialized
+ * @out: (output) The resulting SHA3-224, SHA3-256, SHA3-384, or SHA3-512
+ *	 message digest, matching the init function that was called.  Note that
+ *	 the size differs for each one; see SHA3_*_DIGEST_SIZE.
  *
- * Finish the computation of a SHA3-512 hash and perform the "Keccak sponge
- * squeezing" phase.  The digest is written to @out buffer and the context will
- * be completely zeroed out.
+ * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
  *
  * Context: Any context.
  */
-static inline void sha3_512_final(struct sha3_512_ctx *ctx, u8 out[SHA3_512_DIGEST_SIZE])
+static inline void sha3_final(struct sha3_ctx *ctx, u8 *out)
 {
-	sha3_squeeze(&ctx->ctx, out, SHA3_512_DIGEST_SIZE);
-	sha3_clear(&ctx->ctx);
+	__sha3_squeeze(&ctx->ctx, out, ctx->digest_size);
+	sha3_zeroize_ctx(ctx);
 }
 
-/*
- * Context wrapper for SHAKE128.
- */
-struct shake128_ctx {
-	struct sha3_ctx ctx;
-};
-
 /**
- * shake128_init() - Set a SHA3 context for SHAKE128
+ * shake128_init() - Initialize a context for SHAKE128
  * @ctx: The context to initialize
  *
- * Initialize a SHA3 context for the production of SHAKE128 output generation
- * from a message.  The sha3_squeeze() function can be used to extract an
- * arbitrary amount of data from the context.
+ * This begins a new SHAKE128 extendable-output function (XOF) computation.
  *
  * Context: Any context.
  */
-static inline void shake128_init(struct shake128_ctx *ctx)
+static inline void shake128_init(struct shake_ctx *ctx)
 {
-	*ctx = (struct shake128_ctx){
+	*ctx = (struct shake_ctx){
 		.ctx.block_size	= SHAKE128_BLOCK_SIZE,
 		.ctx.padding	= 0x1f,
 	};
 }
 
 /**
- * shake128_update() - Update a SHAKE128 hash with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
- *
- * This can be called any number of times to add data to the XOF state,
- * performing the "keccak sponge absorbing" phase.
- *
- * Context: Any context.
- */
-static inline void shake128_update(struct shake128_ctx *ctx, const u8 *data, size_t len)
-{
-	return sha3_update(&ctx->ctx, data, len);
-}
-
-/**
- * shake128_squeeze() - Finalize a SHAKE128 digest of any type and extract the digest
- * @ctx: the context to finalize; must have been initialized
- * @out: Where to write the resulting message digest
- * @out_size: The amount of digest to extract to @out in bytes
- *
- * Finish the computation of a SHAKE128 XOF and perform the "Keccak sponge
- * squeezing" phase.  @out_size amount of digest is written to @out buffer.
- *
- * This may be called multiple times to extract continuations of the digest.
- * Note that, a number of consecutive squeezes laid end-to-end will yield the
- * same output as one big squeeze generating the same total amount of output.
- *
- * Context: Any context.
- */
-static inline void shake128_squeeze(struct shake128_ctx *ctx, u8 *out, size_t out_size)
-{
-	return sha3_squeeze(&ctx->ctx, out, out_size);
-}
-
-/**
- * shake128_clear() - Explicitly clear the entire SHAKE128 context
- * @ctx: the context to clear
- *
- * Explicitly clear the entire context; after this, the context must be
- * initialized again.
- *
- * Context: Any context.
- */
-static inline void shake128_clear(struct shake128_ctx *ctx)
-{
-	sha3_clear(&ctx->ctx);
-}
-
-/*
- * Context wrapper for SHAKE256.
- */
-struct shake256_ctx {
-	struct sha3_ctx ctx;
-};
-
-/**
- * shake256_init() - Set a SHA3 context for SHAKE256
+ * shake256_init() - Initialize a context for SHAKE256
  * @ctx: The context to initialize
  *
- * Initialize a SHA3 context for the production of SHAKE256 output generation
- * from a message.  The sha3_squeeze() function can be used to extract an
- * arbitrary amount of data from the context.
+ * This begins a new SHAKE256 extendable-output function (XOF) computation.
  *
  * Context: Any context.
  */
-static inline void shake256_init(struct shake256_ctx *ctx)
+static inline void shake256_init(struct shake_ctx *ctx)
 {
-	*ctx = (struct shake256_ctx){
+	*ctx = (struct shake_ctx){
 		.ctx.block_size	= SHAKE256_BLOCK_SIZE,
 		.ctx.padding	= 0x1f,
 	};
 }
 
 /**
- * shake256_update() - Update a SHAKE256 hash with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
- *
- * This can be called any number of times to add data to the XOF state,
- * performing the "keccak sponge absorbing" phase.
- *
- * Context: Any context.
- */
-static inline void shake256_update(struct shake256_ctx *ctx, const u8 *data, size_t len)
-{
-	return sha3_update(&ctx->ctx, data, len);
-}
-
-/**
- * shake256_squeeze() - Finalize a SHAKE256 digest of any type and extract the digest
- * @ctx: the context to finalize; must have been initialized
- * @out: Where to write the resulting message digest
- * @out_size: The amount of digest to extract to @out in bytes
- *
- * Finish the computation of a SHAKE256 XOF and perform the "Keccak sponge
- * squeezing" phase.  @out_size amount of digest is written to @out buffer.
+ * shake_update() - Update a SHAKE context with input data
+ * @ctx: The context to update; must have been initialized
+ * @in: The input data
+ * @in_len: Length of the input data in bytes
  *
- * This may be called multiple times to extract continuations of the digest.
- * Note that, a number of consecutive squeezes laid end-to-end will yield the
- * same output as one big squeeze generating the same total amount of output.
+ * This can be called any number of times to add more input data to SHAKE128 or
+ * SHAKE256.  This cannot be called after squeezing has begun.
  *
  * Context: Any context.
  */
-static inline void shake256_squeeze(struct shake256_ctx *ctx, u8 *out, size_t out_size)
+static inline void shake_update(struct shake_ctx *ctx,
+				const u8 *in, size_t in_len)
 {
-	return sha3_squeeze(&ctx->ctx, out, out_size);
+	__sha3_update(&ctx->ctx, in, in_len);
 }
 
 /**
- * shake256_clear() - Explicitly clear the entire SHAKE256 context
- * @ctx: the context to clear
+ * shake_squeeze() - Generate output from SHAKE128 or SHAKE256
+ * @ctx: The context to squeeze; must have been initialized
+ * @out: Where to write the resulting output data
+ * @out_len: The amount of data to extract to @out in bytes
  *
- * Explicitly clear the entire context; after this, the context must be
- * initialized again.
+ * This may be called multiple times.  A number of consecutive squeezes laid
+ * end-to-end will yield the same output as one big squeeze generating the same
+ * total amount of output.  More input cannot be provided after squeezing has
+ * begun.  After all squeezing has been completed, call shake_zeroize_ctx().
  *
  * Context: Any context.
  */
-static inline void shake256_clear(struct shake256_ctx *ctx)
+static inline void shake_squeeze(struct shake_ctx *ctx,
+				 u8 *out, size_t out_len)
 {
-	sha3_clear(&ctx->ctx);
+	return __sha3_squeeze(&ctx->ctx, out, out_len);
 }
 
 void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE]);
 void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE]);
 void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE]);
diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
index 83fbfa1169cdb..6c13a43b0f868 100644
--- a/lib/crypto/sha3.c
+++ b/lib/crypto/sha3.c
@@ -177,11 +177,11 @@ static void sha3_keccakf_generic(struct sha3_state *state)
 
 	for (int  i = 0; i < ARRAY_SIZE(state->st); i++)
 		cpu_to_le64s(&state->st[i]);
 }
 
-static void sha3_absorb_block_generic(struct sha3_ctx *ctx, const u8 *data)
+static void sha3_absorb_block_generic(struct __sha3_ctx *ctx, const u8 *data)
 {
 	struct sha3_state *state = &ctx->state;
 	size_t bsize = ctx->block_size;
 
 	for (size_t i = 0; i < bsize / 8; i++)
@@ -191,11 +191,11 @@ static void sha3_absorb_block_generic(struct sha3_ctx *ctx, const u8 *data)
 
 /*
  * Perform rounds of XOR'ing whole blocks of data into the state buffer and
  * then performing a keccak mix step.
  */
-static void sha3_absorb_blocks_generic(struct sha3_ctx *ctx,
+static void sha3_absorb_blocks_generic(struct __sha3_ctx *ctx,
 				       const u8 *data, size_t nblocks)
 {
 	do {
 		sha3_absorb_block_generic(ctx, data);
 		data += ctx->block_size;
@@ -210,32 +210,21 @@ static void sha3_absorb_blocks_generic(struct sha3_ctx *ctx,
 #endif
 
 /*
  * XOR in partial data that's insufficient to fill a whole block.
  */
-static void sha3_absorb_xorle(struct sha3_ctx *ctx, const u8 *data,
+static void sha3_absorb_xorle(struct __sha3_ctx *ctx, const u8 *data,
 			      size_t partial, size_t len)
 {
 	u8 *buf = (u8 *)ctx->state.st;
 
 	buf += partial;
 	for (size_t i = 0; i < len; i++)
 		*buf++ ^= *data++;
 }
 
-/**
- * sha3_update() - Update a SHA3 context of any type with message data
- * @ctx: the context to update; must have been initialized
- * @data: the message data
- * @len: the data length in bytes
- *
- * This can be called any number of times to perform the "keccak sponge
- * absorbing" phase.
- *
- * Context: May use the FPU/Vector unit registers.
- */
-void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len)
+void __sha3_update(struct __sha3_ctx *ctx, const u8 *data, size_t len)
 {
 	size_t absorb_offset = ctx->absorb_offset;
 	size_t bsize = ctx->block_size;
 
 	WARN_ON_ONCE(ctx->end_marked);
@@ -259,29 +248,13 @@ void sha3_update(struct sha3_ctx *ctx, const u8 *data, size_t len)
 	if (len) {
 		sha3_absorb_xorle(ctx, data, ctx->absorb_offset, len);
 		ctx->absorb_offset += len;
 	}
 }
-EXPORT_SYMBOL_GPL(sha3_update);
+EXPORT_SYMBOL_GPL(__sha3_update);
 
-/**
- * sha3_squeeze() - Finalize a SHA3 digest of any type and extract the digest
- * @ctx: the context to finalize; must have been initialized
- * @out: Where to write the resulting message digest
- * @out_size: The amount of digest to extract to @out
- *
- * Finish the computation of a SHA3 message digest of any type and perform the
- * "Keccak sponge squeezing" phase.  @out_size amount of digest is written to
- * @out buffer.
- *
- * This may be called multiple times to extract continuations of the digest.
- * Note that, for example, two consecutive 16-byte squeezes laid end-to-end
- * will yield the same as one 32-byte squeeze.
- *
- * Context: May use the FPU/Vector unit registers.
- */
-void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_size)
+void __sha3_squeeze(struct __sha3_ctx *ctx, u8 *out, size_t out_len)
 {
 	size_t squeeze_offset = ctx->squeeze_offset;
 	size_t bsize = ctx->block_size;
 	u8 *p = (u8 *)ctx->state.st, end_marker = 0x80;
 
@@ -293,27 +266,27 @@ void sha3_squeeze(struct sha3_ctx *ctx, u8 *out, size_t out_size)
 
 	for (;;) {
 		if (squeeze_offset == 0)
 			sha3_keccakf(&ctx->state);
 
-		size_t part = umin(out_size, bsize - squeeze_offset);
+		size_t part = umin(out_len, bsize - squeeze_offset);
 
 		if (part > 0) {
 			memcpy(out, p + squeeze_offset, part);
-			out_size -= part;
+			out_len -= part;
 			out += part;
 			squeeze_offset += part;
 		}
-		if (!out_size)
+		if (!out_len)
 			break;
 		if (squeeze_offset >= bsize)
 			squeeze_offset = 0;
 	}
 
 	ctx->squeeze_offset = squeeze_offset;
 }
-EXPORT_SYMBOL_GPL(sha3_squeeze);
+EXPORT_SYMBOL_GPL(__sha3_squeeze);
 
 /**
  * sha3_224() - Convenience wrapper to digest a simple buffer as SHA3-224
  * @in: The data to be digested
  * @in_len: The amount of data to be digested in bytes
@@ -324,15 +297,15 @@ EXPORT_SYMBOL_GPL(sha3_squeeze);
  *
  * Context: May use the FPU/Vector unit registers.
  */
 void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE])
 {
-	struct sha3_224_ctx ctx;
+	struct sha3_ctx ctx;
 
 	sha3_224_init(&ctx);
-	sha3_224_update(&ctx, in, in_len);
-	sha3_224_final(&ctx, out);
+	sha3_update(&ctx, in, in_len);
+	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_224);
 
 /**
  * sha3_256() - Convenience wrapper to digest a simple buffer as SHA3-256
@@ -345,15 +318,15 @@ EXPORT_SYMBOL_GPL(sha3_224);
  *
  * Context: May use the FPU/Vector unit registers.
  */
 void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE])
 {
-	struct sha3_256_ctx ctx;
+	struct sha3_ctx ctx;
 
 	sha3_256_init(&ctx);
-	sha3_256_update(&ctx, in, in_len);
-	sha3_256_final(&ctx, out);
+	sha3_update(&ctx, in, in_len);
+	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_256);
 
 /**
  * sha3_384() - Convenience wrapper to digest a simple buffer as SHA3-384
@@ -366,15 +339,15 @@ EXPORT_SYMBOL_GPL(sha3_256);
  *
  * Context: May use the FPU/Vector unit registers.
  */
 void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE])
 {
-	struct sha3_384_ctx ctx;
+	struct sha3_ctx ctx;
 
 	sha3_384_init(&ctx);
-	sha3_384_update(&ctx, in, in_len);
-	sha3_384_final(&ctx, out);
+	sha3_update(&ctx, in, in_len);
+	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_384);
 
 /**
  * sha3_512() - Convenience wrapper to digest a simple buffer as SHA3-512
@@ -387,15 +360,15 @@ EXPORT_SYMBOL_GPL(sha3_384);
  *
  * Context: May use the FPU/Vector unit registers.
  */
 void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE])
 {
-	struct sha3_512_ctx ctx;
+	struct sha3_ctx ctx;
 
 	sha3_512_init(&ctx);
-	sha3_512_update(&ctx, in, in_len);
-	sha3_512_final(&ctx, out);
+	sha3_update(&ctx, in, in_len);
+	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_512);
 
 /**
  * shake128() - Convenience wrapper to apply SHAKE128 to a simple buffer
@@ -410,16 +383,16 @@ EXPORT_SYMBOL_GPL(sha3_512);
  *
  * Context: May use the FPU/Vector unit registers.
  */
 void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 {
-	struct shake128_ctx ctx;
+	struct shake_ctx ctx;
 
 	shake128_init(&ctx);
-	shake128_update(&ctx, in, in_len);
-	shake128_squeeze(&ctx, out, out_len);
-	shake128_clear(&ctx);
+	shake_update(&ctx, in, in_len);
+	shake_squeeze(&ctx, out, out_len);
+	shake_zeroize_ctx(&ctx);
 }
 EXPORT_SYMBOL_GPL(shake128);
 
 /**
  * shake256() - Convenience wrapper to apply SHAKE256 to a simple buffer
@@ -434,16 +407,16 @@ EXPORT_SYMBOL_GPL(shake128);
  *
  * Context: May use the FPU/Vector unit registers.
  */
 void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 {
-	struct shake256_ctx ctx;
+	struct shake_ctx ctx;
 
 	shake256_init(&ctx);
-	shake256_update(&ctx, in, in_len);
-	shake256_squeeze(&ctx, out, out_len);
-	shake256_clear(&ctx);
+	shake_update(&ctx, in, in_len);
+	shake_squeeze(&ctx, out, out_len);
+	shake_zeroize_ctx(&ctx);
 }
 EXPORT_SYMBOL_GPL(shake256);
 
 #if defined(sha3_mod_init_arch) || defined(CONFIG_CRYPTO_FIPS)
 static int __init sha3_mod_init(void)
diff --git a/lib/crypto/tests/sha3_kunit.c b/lib/crypto/tests/sha3_kunit.c
index 032fa3950afe4..c2e0c37d8a543 100644
--- a/lib/crypto/tests/sha3_kunit.c
+++ b/lib/crypto/tests/sha3_kunit.c
@@ -5,15 +5,15 @@
  */
 #include <crypto/sha3.h>
 #include "sha3_testvecs.h"
 
 #define HASH		sha3_256
-#define HASH_CTX	sha3_256_ctx
+#define HASH_CTX	sha3_ctx
 #define HASH_SIZE	SHA3_256_DIGEST_SIZE
 #define HASH_INIT	sha3_256_init
-#define HASH_UPDATE	sha3_256_update
-#define HASH_FINAL	sha3_256_final
+#define HASH_UPDATE	sha3_update
+#define HASH_FINAL	sha3_final
 #include "hash-test-template.h"
 
 /*
  * Sample message and the output generated for various algorithms by passing it
  * into "openssl sha3-224" etc..
@@ -252,24 +252,25 @@ static void test_shake256_nist(struct kunit *test)
  * series of squeezings of the same context should, if laid end-to-end, match a
  * single squeezing of the combined size.
  */
 static void test_shake256_tiling(struct kunit *test)
 {
-	struct shake256_ctx ctx;
+	struct shake_ctx ctx;
 	u8 out[8 + SHA3_512_DIGEST_SIZE + 8];
 
 	for (int tile_size = 1; tile_size < SHAKE256_DEFAULT_SIZE; tile_size++) {
 		int left = SHAKE256_DEFAULT_SIZE;
 		u8 *p = out + 8;
 
 		memset(out, 0, sizeof(out));
 		shake256_init(&ctx);
-		shake256_update(&ctx, test_sha3_sample, sizeof(test_sha3_sample) - 1);
+		shake_update(&ctx, test_sha3_sample,
+			     sizeof(test_sha3_sample) - 1);
 		while (left > 0) {
 			int part = umin(tile_size, left);
 
-			shake256_squeeze(&ctx, p, part);
+			shake_squeeze(&ctx, p, part);
 			p += part;
 			left -= part;
 		}
 
 		KUNIT_ASSERT_MEMEQ_MSG(test, out, test_shake256, sizeof(test_shake256),
@@ -283,11 +284,11 @@ static void test_shake256_tiling(struct kunit *test)
  * context should, if laid end-to-end, match a single squeezing of the combined
  * size.
  */
 static void test_shake256_tiling2(struct kunit *test)
 {
-	struct shake256_ctx ctx;
+	struct shake_ctx ctx;
 	u8 out[8 + SHA3_512_DIGEST_SIZE + 8];
 
 	for (int first_tile_size = 3;
 	     first_tile_size < SHAKE256_DEFAULT_SIZE;
 	     first_tile_size++) {
@@ -295,15 +296,16 @@ static void test_shake256_tiling2(struct kunit *test)
 		int left = SHAKE256_DEFAULT_SIZE;
 		u8 *p = out + 8;
 
 		memset(out, 0, sizeof(out));
 		shake256_init(&ctx);
-		shake256_update(&ctx, test_sha3_sample, sizeof(test_sha3_sample) - 1);
+		shake_update(&ctx, test_sha3_sample,
+			     sizeof(test_sha3_sample) - 1);
 		while (left > 0) {
 			int part = umin(tile_size, left);
 
-			shake256_squeeze(&ctx, p, part);
+			shake_squeeze(&ctx, p, part);
 			p += part;
 			left -= part;
 			tile_size--;
 			if (tile_size < 1)
 				tile_size = 5;
-- 
2.51.1.dirty

[PATCH 12/17] lib/crypto: sha3: Document one-shot functions in header and improve docs

[Eric Biggers]

Move kerneldoc for the one-shot functions to the header for the
following reasons:

- This style is already used in sha1.h, sha2.h, and crypto_shash.
  And all the inline functions in sha3.h already.

- The header is where people who want to call the code usually look.

- This way the documentation won't have to be moved each time a function
  is changed to or from an inline function.

Also revise this documentation to fix some errors, be more user focused
(rather than focusing on implementation details like whether a context
is allocated and zeroized), and be a bit more consistent with sha2.h.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 include/crypto/sha3.h | 75 +++++++++++++++++++++++++++++++++++++++++++
 lib/crypto/sha3.c     | 70 ----------------------------------------
 2 files changed, 75 insertions(+), 70 deletions(-)

diff --git a/include/crypto/sha3.h b/include/crypto/sha3.h
index 58d2b3e0663e8..80acdb266b8e3 100644
--- a/include/crypto/sha3.h
+++ b/include/crypto/sha3.h
@@ -259,13 +259,88 @@ static inline void shake_squeeze(struct shake_ctx *ctx,
 				 u8 *out, size_t out_len)
 {
 	return __sha3_squeeze(&ctx->ctx, out, out_len);
 }
 
+/**
+ * sha3_224() - Compute SHA3-224 digest in one shot
+ * @in: The input data to be digested
+ * @in_len: Length of the input data in bytes
+ * @out: The buffer into which the digest will be stored
+ *
+ * Convenience function that computes a SHA3-224 digest.  Use this instead of
+ * the incremental API if you're able to provide all the input at once.
+ *
+ * Context: Any context.
+ */
 void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE]);
+
+/**
+ * sha3_256() - Compute SHA3-256 digest in one shot
+ * @in: The input data to be digested
+ * @in_len: Length of the input data in bytes
+ * @out: The buffer into which the digest will be stored
+ *
+ * Convenience function that computes a SHA3-256 digest.  Use this instead of
+ * the incremental API if you're able to provide all the input at once.
+ *
+ * Context: Any context.
+ */
 void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE]);
+
+/**
+ * sha3_384() - Compute SHA3-384 digest in one shot
+ * @in: The input data to be digested
+ * @in_len: Length of the input data in bytes
+ * @out: The buffer into which the digest will be stored
+ *
+ * Convenience function that computes a SHA3-384 digest.  Use this instead of
+ * the incremental API if you're able to provide all the input at once.
+ *
+ * Context: Any context.
+ */
 void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE]);
+
+/**
+ * sha3_512() - Compute SHA3-512 digest in one shot
+ * @in: The input data to be digested
+ * @in_len: Length of the input data in bytes
+ * @out: The buffer into which the digest will be stored
+ *
+ * Convenience function that computes a SHA3-512 digest.  Use this instead of
+ * the incremental API if you're able to provide all the input at once.
+ *
+ * Context: Any context.
+ */
 void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE]);
+
+/**
+ * shake128() - Compute SHAKE128 in one shot
+ * @in: The input data to be used
+ * @in_len: Length of the input data in bytes
+ * @out: The buffer into which the output will be stored
+ * @out_len: Length of the output to produce in bytes
+ *
+ * Convenience function that computes SHAKE128 in one shot.  Use this instead of
+ * the incremental API if you're able to provide all the input at once as well
+ * as receive all the output at once.  All output lengths are supported.
+ *
+ * Context: Any context.
+ */
 void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len);
+
+/**
+ * shake256() - Compute SHAKE256 in one shot
+ * @in: The input data to be used
+ * @in_len: Length of the input data in bytes
+ * @out: The buffer into which the output will be stored
+ * @out_len: Length of the output to produce in bytes
+ *
+ * Convenience function that computes SHAKE256 in one shot.  Use this instead of
+ * the incremental API if you're able to provide all the input at once as well
+ * as receive all the output at once.  All output lengths are supported.
+ *
+ * Context: Any context.
+ */
 void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len);
 
 #endif /* __CRYPTO_SHA3_H__ */
diff --git a/lib/crypto/sha3.c b/lib/crypto/sha3.c
index 6c13a43b0f868..6705c1b48da48 100644
--- a/lib/crypto/sha3.c
+++ b/lib/crypto/sha3.c
@@ -284,107 +284,50 @@ void __sha3_squeeze(struct __sha3_ctx *ctx, u8 *out, size_t out_len)
 
 	ctx->squeeze_offset = squeeze_offset;
 }
 EXPORT_SYMBOL_GPL(__sha3_squeeze);
 
-/**
- * sha3_224() - Convenience wrapper to digest a simple buffer as SHA3-224
- * @in: The data to be digested
- * @in_len: The amount of data to be digested in bytes
- * @out: The buffer into which the digest will be stored (size not checked)
- *
- * Convenience wrapper to initialise a SHA3 context for SHA-224, add the input
- * data to it, finalise it, extract 28 bytes of digest and clear the context.
- *
- * Context: May use the FPU/Vector unit registers.
- */
 void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE])
 {
 	struct sha3_ctx ctx;
 
 	sha3_224_init(&ctx);
 	sha3_update(&ctx, in, in_len);
 	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_224);
 
-/**
- * sha3_256() - Convenience wrapper to digest a simple buffer as SHA3-256
- * @in: The data to be digested
- * @in_len: The amount of data to be digested in bytes
- * @out: The buffer into which the digest will be stored (size not checked)
- *
- * Convenience wrapper to initialise a SHA3 context for SHA-256, add the input
- * data to it, finalise it, extract 32 bytes of digest and clear the context.
- *
- * Context: May use the FPU/Vector unit registers.
- */
 void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE])
 {
 	struct sha3_ctx ctx;
 
 	sha3_256_init(&ctx);
 	sha3_update(&ctx, in, in_len);
 	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_256);
 
-/**
- * sha3_384() - Convenience wrapper to digest a simple buffer as SHA3-384
- * @in: The data to be digested
- * @in_len: The amount of data to be digested in bytes
- * @out: The buffer into which the digest will be stored (size not checked)
- *
- * Convenience wrapper to initialise a SHA3 context for SHA-384, add the input
- * data to it, finalise it, extract 48 bytes of digest and clear the context.
- *
- * Context: May use the FPU/Vector unit registers.
- */
 void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE])
 {
 	struct sha3_ctx ctx;
 
 	sha3_384_init(&ctx);
 	sha3_update(&ctx, in, in_len);
 	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_384);
 
-/**
- * sha3_512() - Convenience wrapper to digest a simple buffer as SHA3-512
- * @in: The data to be digested in bytes
- * @in_len: The amount of data to be digested in bytes
- * @out: The buffer into which the digest will be stored (size not checked)
- *
- * Convenience wrapper to initialise a SHA3 context for SHA-512, add the input
- * data to it, finalise it, extract 64 bytes of digest and clear the context.
- *
- * Context: May use the FPU/Vector unit registers.
- */
 void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE])
 {
 	struct sha3_ctx ctx;
 
 	sha3_512_init(&ctx);
 	sha3_update(&ctx, in, in_len);
 	sha3_final(&ctx, out);
 }
 EXPORT_SYMBOL_GPL(sha3_512);
 
-/**
- * shake128() - Convenience wrapper to apply SHAKE128 to a simple buffer
- * @in: The input data to be used
- * @in_len: The amount of input data in bytes
- * @out: The buffer in which to store the output
- * @out_len: The amount of output to store in bytes (variable length)
- *
- * Convenience wrapper to initialise a SHA3 context for SHAKE128, add the input
- * data to it, finalise it, extract the requested amount of output and clear
- * the context.
- *
- * Context: May use the FPU/Vector unit registers.
- */
 void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 {
 	struct shake_ctx ctx;
 
 	shake128_init(&ctx);
@@ -392,23 +335,10 @@ void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 	shake_squeeze(&ctx, out, out_len);
 	shake_zeroize_ctx(&ctx);
 }
 EXPORT_SYMBOL_GPL(shake128);
 
-/**
- * shake256() - Convenience wrapper to apply SHAKE256 to a simple buffer
- * @in: The input data to be used
- * @in_len: The amount of input data in bytes
- * @out: The buffer in which to store the output
- * @out_len: The amount of output to store in bytes (variable length)
- *
- * Convenience wrapper to initialise a SHA3 context for SHAKE128, add the input
- * data to it, finalise it, extract the requested amount of output and clear
- * the context.
- *
- * Context: May use the FPU/Vector unit registers.
- */
 void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len)
 {
 	struct shake_ctx ctx;
 
 	shake256_init(&ctx);
-- 
2.51.1.dirty

[PATCH 13/17] crypto: arm64/sha3 - Update sha3_ce_transform() to prepare for library

[Eric Biggers]

- Use size_t lengths, to match the library.

- Pass the block size instead of digest size, and add support for the
  block size that SHAKE128 uses.  This allows the code to be used with
  SHAKE128 and SHAKE256, which don't have the concept of a digest size.
  SHAKE256 has the same block size as SHA3-256, but SHAKE128 has a
  unique block size.  Thus, there are now 5 supported block sizes.

Don't bother changing the "glue" code arm64_ce_transform() too much, as
it gets deleted when the SHA-3 code is migrated into lib/crypto/ anyway.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 arch/arm64/crypto/sha3-ce-core.S | 66 ++++++++++++++++----------------
 arch/arm64/crypto/sha3-ce-glue.c | 11 +++---
 2 files changed, 38 insertions(+), 39 deletions(-)

diff --git a/arch/arm64/crypto/sha3-ce-core.S b/arch/arm64/crypto/sha3-ce-core.S
index 9c77313f5a608..62477848821d9 100644
--- a/arch/arm64/crypto/sha3-ce-core.S
+++ b/arch/arm64/crypto/sha3-ce-core.S
@@ -35,11 +35,15 @@
 	.macro	xar, rd, rn, rm, imm6
 	.inst	0xce800000 | .L\rd | (.L\rn << 5) | ((\imm6) << 10) | (.L\rm << 16)
 	.endm
 
 	/*
-	 * int sha3_ce_transform(u64 *st, const u8 *data, int blocks, int dg_size)
+	 * size_t sha3_ce_transform(struct sha3_state *state, const u8 *data,
+	 *			    size_t nblocks, size_t block_size)
+	 *
+	 * block_size is assumed to be one of 72 (SHA3-512), 104 (SHA3-384), 136
+	 * (SHA3-256 and SHAKE256), 144 (SHA3-224), or 168 (SHAKE128).
 	 */
 	.text
 SYM_FUNC_START(sha3_ce_transform)
 	/* load state */
 	add	x8, x0, #32
@@ -49,62 +53,58 @@ SYM_FUNC_START(sha3_ce_transform)
 	ld1	{v12.1d-v15.1d}, [x8], #32
 	ld1	{v16.1d-v19.1d}, [x8], #32
 	ld1	{v20.1d-v23.1d}, [x8], #32
 	ld1	{v24.1d}, [x8]
 
-0:	sub	w2, w2, #1
+0:	sub	x2, x2, #1
 	mov	w8, #24
 	adr_l	x9, .Lsha3_rcon
 
 	/* load input */
 	ld1	{v25.8b-v28.8b}, [x1], #32
-	ld1	{v29.8b-v31.8b}, [x1], #24
+	ld1	{v29.8b}, [x1], #8
 	eor	v0.8b, v0.8b, v25.8b
 	eor	v1.8b, v1.8b, v26.8b
 	eor	v2.8b, v2.8b, v27.8b
 	eor	v3.8b, v3.8b, v28.8b
 	eor	v4.8b, v4.8b, v29.8b
-	eor	v5.8b, v5.8b, v30.8b
-	eor	v6.8b, v6.8b, v31.8b
-
-	tbnz	x3, #6, 2f		// SHA3-512
 
 	ld1	{v25.8b-v28.8b}, [x1], #32
-	ld1	{v29.8b-v30.8b}, [x1], #16
-	eor	 v7.8b,  v7.8b, v25.8b
-	eor	 v8.8b,  v8.8b, v26.8b
-	eor	 v9.8b,  v9.8b, v27.8b
-	eor	v10.8b, v10.8b, v28.8b
-	eor	v11.8b, v11.8b, v29.8b
-	eor	v12.8b, v12.8b, v30.8b
+	eor	v5.8b, v5.8b, v25.8b
+	eor	v6.8b, v6.8b, v26.8b
+	eor	v7.8b, v7.8b, v27.8b
+	eor	v8.8b, v8.8b, v28.8b
+	cmp	x3, #72
+	b.eq	3f	/* SHA3-512 (block_size=72)? */
 
-	tbnz	x3, #4, 1f		// SHA3-384 or SHA3-224
+	ld1	{v25.8b-v28.8b}, [x1], #32
+	eor	v9.8b, v9.8b, v25.8b
+	eor	v10.8b, v10.8b, v26.8b
+	eor	v11.8b, v11.8b, v27.8b
+	eor	v12.8b, v12.8b, v28.8b
+	cmp	x3, #104
+	b.eq	3f	/* SHA3-384 (block_size=104)? */
 
-	// SHA3-256
 	ld1	{v25.8b-v28.8b}, [x1], #32
 	eor	v13.8b, v13.8b, v25.8b
 	eor	v14.8b, v14.8b, v26.8b
 	eor	v15.8b, v15.8b, v27.8b
 	eor	v16.8b, v16.8b, v28.8b
-	b	3f
-
-1:	tbz	x3, #2, 3f		// bit 2 cleared? SHA-384
+	cmp	x3, #144
+	b.lt	3f	/* SHA3-256 or SHAKE256 (block_size=136)? */
+	b.eq	2f	/* SHA3-224 (block_size=144)? */
 
-	// SHA3-224
+	/* SHAKE128 (block_size=168) */
 	ld1	{v25.8b-v28.8b}, [x1], #32
-	ld1	{v29.8b}, [x1], #8
-	eor	v13.8b, v13.8b, v25.8b
-	eor	v14.8b, v14.8b, v26.8b
-	eor	v15.8b, v15.8b, v27.8b
-	eor	v16.8b, v16.8b, v28.8b
-	eor	v17.8b, v17.8b, v29.8b
+	eor	v17.8b, v17.8b, v25.8b
+	eor	v18.8b, v18.8b, v26.8b
+	eor	v19.8b, v19.8b, v27.8b
+	eor	v20.8b, v20.8b, v28.8b
 	b	3f
-
-	// SHA3-512
-2:	ld1	{v25.8b-v26.8b}, [x1], #16
-	eor	 v7.8b,  v7.8b, v25.8b
-	eor	 v8.8b,  v8.8b, v26.8b
+2:
+	ld1	{v25.8b}, [x1], #8	/* SHA3-224 (block_size=144) */
+	eor	v17.8b, v17.8b, v25.8b
 
 3:	sub	w8, w8, #1
 
 	eor3	v29.16b,  v4.16b,  v9.16b, v14.16b
 	eor3	v26.16b,  v1.16b,  v6.16b, v11.16b
@@ -183,21 +183,21 @@ SYM_FUNC_START(sha3_ce_transform)
 
 	eor	 v0.16b,  v0.16b, v31.16b
 
 	cbnz	w8, 3b
 	cond_yield 4f, x8, x9
-	cbnz	w2, 0b
+	cbnz	x2, 0b
 
 	/* save state */
 4:	st1	{ v0.1d- v3.1d}, [x0], #32
 	st1	{ v4.1d- v7.1d}, [x0], #32
 	st1	{ v8.1d-v11.1d}, [x0], #32
 	st1	{v12.1d-v15.1d}, [x0], #32
 	st1	{v16.1d-v19.1d}, [x0], #32
 	st1	{v20.1d-v23.1d}, [x0], #32
 	st1	{v24.1d}, [x0]
-	mov	w0, w2
+	mov	x0, x2
 	ret
 SYM_FUNC_END(sha3_ce_transform)
 
 	.section	".rodata", "a"
 	.align		8
diff --git a/arch/arm64/crypto/sha3-ce-glue.c b/arch/arm64/crypto/sha3-ce-glue.c
index 426d8044535ab..f4eae798371ef 100644
--- a/arch/arm64/crypto/sha3-ce-glue.c
+++ b/arch/arm64/crypto/sha3-ce-glue.c
@@ -26,30 +26,29 @@ MODULE_LICENSE("GPL v2");
 MODULE_ALIAS_CRYPTO("sha3-224");
 MODULE_ALIAS_CRYPTO("sha3-256");
 MODULE_ALIAS_CRYPTO("sha3-384");
 MODULE_ALIAS_CRYPTO("sha3-512");
 
-asmlinkage int sha3_ce_transform(u64 *st, const u8 *data, int blocks,
-				 int md_len);
+asmlinkage size_t sha3_ce_transform(struct sha3_state *state, const u8 *data,
+				    size_t nblocks, size_t block_size);
 
 static int arm64_sha3_update(struct shash_desc *desc, const u8 *data,
 		       unsigned int len)
 {
 	struct sha3_state *sctx = shash_desc_ctx(desc);
 	struct crypto_shash *tfm = desc->tfm;
-	unsigned int bs, ds;
+	unsigned int bs;
 	int blocks;
 
-	ds = crypto_shash_digestsize(tfm);
 	bs = crypto_shash_blocksize(tfm);
 	blocks = len / bs;
 	len -= blocks * bs;
 	do {
 		int rem;
 
 		kernel_neon_begin();
-		rem = sha3_ce_transform(sctx->st, data, blocks, ds);
+		rem = sha3_ce_transform(sctx, data, blocks, bs);
 		kernel_neon_end();
 		data += (blocks - rem) * bs;
 		blocks = rem;
 	} while (blocks);
 	return len;
@@ -72,11 +71,11 @@ static int arm64_sha3_finup(struct shash_desc *desc, const u8 *src, unsigned int
 	block[len++] = 0x06;
 	memset(block + len, 0, bs - len);
 	block[bs - 1] |= 0x80;
 
 	kernel_neon_begin();
-	sha3_ce_transform(sctx->st, block, 1, ds);
+	sha3_ce_transform(sctx, block, 1, bs);
 	kernel_neon_end();
 	memzero_explicit(block , sizeof(block));
 
 	for (i = 0; i < ds / 8; i++)
 		put_unaligned_le64(sctx->st[i], digest++);
-- 
2.51.1.dirty

[PATCH 14/17] lib/crypto: arm64/sha3: Migrate optimized code into library

[Eric Biggers]

Instead of exposing the arm64-optimized SHA-3 code via arm64-specific
crypto_shash algorithms, instead just implement the sha3_absorb_blocks()
and sha3_keccakf() library functions.  This is much simpler, it makes
the SHA-3 library functions be arm64-optimized, and it fixes the
longstanding issue where the arm64-optimized SHA-3 code was disabled by
default.  SHA-3 still remains available through crypto_shash, but
individual architectures no longer need to handle it.

Note: to see the diff from arch/arm64/crypto/sha3-ce-glue.c to
lib/crypto/arm64/sha3.h, view this commit with 'git show -M10'.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 arch/arm64/configs/defconfig                  |   2 +-
 arch/arm64/crypto/Kconfig                     |  11 --
 arch/arm64/crypto/Makefile                    |   3 -
 arch/arm64/crypto/sha3-ce-glue.c              | 150 ------------------
 lib/crypto/Kconfig                            |   5 +
 lib/crypto/Makefile                           |   5 +
 .../crypto/arm64}/sha3-ce-core.S              |   0
 lib/crypto/arm64/sha3.h                       |  64 ++++++++
 8 files changed, 75 insertions(+), 165 deletions(-)
 delete mode 100644 arch/arm64/crypto/sha3-ce-glue.c
 rename {arch/arm64/crypto => lib/crypto/arm64}/sha3-ce-core.S (100%)
 create mode 100644 lib/crypto/arm64/sha3.h

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index e3a2d37bd1042..20dd3a39faead 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -1781,14 +1781,14 @@ CONFIG_SECURITY=y
 CONFIG_CRYPTO_USER=y
 CONFIG_CRYPTO_CHACHA20=m
 CONFIG_CRYPTO_BENCHMARK=m
 CONFIG_CRYPTO_ECHAINIV=y
 CONFIG_CRYPTO_MICHAEL_MIC=m
+CONFIG_CRYPTO_SHA3=m
 CONFIG_CRYPTO_ANSI_CPRNG=y
 CONFIG_CRYPTO_USER_API_RNG=m
 CONFIG_CRYPTO_GHASH_ARM64_CE=y
-CONFIG_CRYPTO_SHA3_ARM64=m
 CONFIG_CRYPTO_SM3_ARM64_CE=m
 CONFIG_CRYPTO_AES_ARM64_CE_BLK=y
 CONFIG_CRYPTO_AES_ARM64_BS=m
 CONFIG_CRYPTO_AES_ARM64_CE_CCM=y
 CONFIG_CRYPTO_DEV_SUN8I_CE=m
diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig
index 91f3093eee6ab..376d6b50743ff 100644
--- a/arch/arm64/crypto/Kconfig
+++ b/arch/arm64/crypto/Kconfig
@@ -23,21 +23,10 @@ config CRYPTO_NHPOLY1305_NEON
 	  NHPoly1305 hash function (Adiantum)
 
 	  Architecture: arm64 using:
 	  - NEON (Advanced SIMD) extensions
 
-config CRYPTO_SHA3_ARM64
-	tristate "Hash functions: SHA-3 (ARMv8.2 Crypto Extensions)"
-	depends on KERNEL_MODE_NEON
-	select CRYPTO_HASH
-	select CRYPTO_SHA3
-	help
-	  SHA-3 secure hash algorithms (FIPS 202)
-
-	  Architecture: arm64 using:
-	  - ARMv8.2 Crypto Extensions
-
 config CRYPTO_SM3_NEON
 	tristate "Hash functions: SM3 (NEON)"
 	depends on KERNEL_MODE_NEON
 	select CRYPTO_HASH
 	select CRYPTO_LIB_SM3
diff --git a/arch/arm64/crypto/Makefile b/arch/arm64/crypto/Makefile
index a8b2cdbe202c1..fd3d590fa1137 100644
--- a/arch/arm64/crypto/Makefile
+++ b/arch/arm64/crypto/Makefile
@@ -3,13 +3,10 @@
 # linux/arch/arm64/crypto/Makefile
 #
 # Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
 #
 
-obj-$(CONFIG_CRYPTO_SHA3_ARM64) += sha3-ce.o
-sha3-ce-y := sha3-ce-glue.o sha3-ce-core.o
-
 obj-$(CONFIG_CRYPTO_SM3_NEON) += sm3-neon.o
 sm3-neon-y := sm3-neon-glue.o sm3-neon-core.o
 
 obj-$(CONFIG_CRYPTO_SM3_ARM64_CE) += sm3-ce.o
 sm3-ce-y := sm3-ce-glue.o sm3-ce-core.o
diff --git a/arch/arm64/crypto/sha3-ce-glue.c b/arch/arm64/crypto/sha3-ce-glue.c
deleted file mode 100644
index f4eae798371ef..0000000000000
--- a/arch/arm64/crypto/sha3-ce-glue.c
+++ /dev/null
@@ -1,150 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/*
- * sha3-ce-glue.c - core SHA-3 transform using v8.2 Crypto Extensions
- *
- * Copyright (C) 2018 Linaro Ltd <ard.biesheuvel@linaro.org>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#include <asm/hwcap.h>
-#include <asm/neon.h>
-#include <asm/simd.h>
-#include <crypto/internal/hash.h>
-#include <crypto/sha3.h>
-#include <linux/cpufeature.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
-#include <linux/unaligned.h>
-
-MODULE_DESCRIPTION("SHA3 secure hash using ARMv8 Crypto Extensions");
-MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
-MODULE_LICENSE("GPL v2");
-MODULE_ALIAS_CRYPTO("sha3-224");
-MODULE_ALIAS_CRYPTO("sha3-256");
-MODULE_ALIAS_CRYPTO("sha3-384");
-MODULE_ALIAS_CRYPTO("sha3-512");
-
-asmlinkage size_t sha3_ce_transform(struct sha3_state *state, const u8 *data,
-				    size_t nblocks, size_t block_size);
-
-static int arm64_sha3_update(struct shash_desc *desc, const u8 *data,
-		       unsigned int len)
-{
-	struct sha3_state *sctx = shash_desc_ctx(desc);
-	struct crypto_shash *tfm = desc->tfm;
-	unsigned int bs;
-	int blocks;
-
-	bs = crypto_shash_blocksize(tfm);
-	blocks = len / bs;
-	len -= blocks * bs;
-	do {
-		int rem;
-
-		kernel_neon_begin();
-		rem = sha3_ce_transform(sctx, data, blocks, bs);
-		kernel_neon_end();
-		data += (blocks - rem) * bs;
-		blocks = rem;
-	} while (blocks);
-	return len;
-}
-
-static int arm64_sha3_finup(struct shash_desc *desc, const u8 *src, unsigned int len,
-			    u8 *out)
-{
-	struct sha3_state *sctx = shash_desc_ctx(desc);
-	struct crypto_shash *tfm = desc->tfm;
-	__le64 *digest = (__le64 *)out;
-	u8 block[SHA3_224_BLOCK_SIZE];
-	unsigned int bs, ds;
-	int i;
-
-	ds = crypto_shash_digestsize(tfm);
-	bs = crypto_shash_blocksize(tfm);
-	memcpy(block, src, len);
-
-	block[len++] = 0x06;
-	memset(block + len, 0, bs - len);
-	block[bs - 1] |= 0x80;
-
-	kernel_neon_begin();
-	sha3_ce_transform(sctx, block, 1, bs);
-	kernel_neon_end();
-	memzero_explicit(block , sizeof(block));
-
-	for (i = 0; i < ds / 8; i++)
-		put_unaligned_le64(sctx->st[i], digest++);
-
-	if (ds & 4)
-		put_unaligned_le32(sctx->st[i], (__le32 *)digest);
-
-	return 0;
-}
-
-static struct shash_alg algs[] = { {
-	.digestsize		= SHA3_224_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= arm64_sha3_update,
-	.finup			= arm64_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-224",
-	.base.cra_driver_name	= "sha3-224-ce",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_224_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-	.base.cra_priority	= 200,
-}, {
-	.digestsize		= SHA3_256_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= arm64_sha3_update,
-	.finup			= arm64_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-256",
-	.base.cra_driver_name	= "sha3-256-ce",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_256_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-	.base.cra_priority	= 200,
-}, {
-	.digestsize		= SHA3_384_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= arm64_sha3_update,
-	.finup			= arm64_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-384",
-	.base.cra_driver_name	= "sha3-384-ce",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_384_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-	.base.cra_priority	= 200,
-}, {
-	.digestsize		= SHA3_512_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= arm64_sha3_update,
-	.finup			= arm64_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-512",
-	.base.cra_driver_name	= "sha3-512-ce",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_512_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-	.base.cra_priority	= 200,
-} };
-
-static int __init sha3_neon_mod_init(void)
-{
-	return crypto_register_shashes(algs, ARRAY_SIZE(algs));
-}
-
-static void __exit sha3_neon_mod_fini(void)
-{
-	crypto_unregister_shashes(algs, ARRAY_SIZE(algs));
-}
-
-module_cpu_feature_match(SHA3, sha3_neon_mod_init);
-module_exit(sha3_neon_mod_fini);
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index 1caf3fbe24bff..544eb67fe70f4 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -189,10 +189,15 @@ config CRYPTO_LIB_SHA3
 	help
 	  The SHA3 library functions.
 	  Select this if your module uses any of these functions from
 	  <crypto/sha3.h>.
 
+config CRYPTO_LIB_SHA3_ARCH
+	bool
+	depends on CRYPTO_LIB_SHA3 && !UML
+	default y if ARM64 && KERNEL_MODE_NEON
+
 config CRYPTO_LIB_SM3
 	tristate
 
 source "lib/crypto/tests/Kconfig"
 
diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index 2f15cef850507..f638bff0fb081 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -269,10 +269,15 @@ endif # CONFIG_CRYPTO_LIB_SHA512_ARCH
 ################################################################################
 
 obj-$(CONFIG_CRYPTO_LIB_SHA3) += libsha3.o
 libsha3-y := sha3.o
 
+ifeq ($(CONFIG_CRYPTO_LIB_SHA3_ARCH),y)
+CFLAGS_sha3.o += -I$(src)/$(SRCARCH)
+libsha3-$(CONFIG_ARM64) += arm64/sha3-ce-core.o
+endif # CONFIG_CRYPTO_LIB_SHA3_ARCH
+
 ################################################################################
 
 obj-$(CONFIG_MPILIB) += mpi/
 
 obj-$(CONFIG_CRYPTO_SELFTESTS_FULL)		+= simd.o
diff --git a/arch/arm64/crypto/sha3-ce-core.S b/lib/crypto/arm64/sha3-ce-core.S
similarity index 100%
rename from arch/arm64/crypto/sha3-ce-core.S
rename to lib/crypto/arm64/sha3-ce-core.S
diff --git a/lib/crypto/arm64/sha3.h b/lib/crypto/arm64/sha3.h
new file mode 100644
index 0000000000000..286c519eecd5a
--- /dev/null
+++ b/lib/crypto/arm64/sha3.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/neon.h>
+#include <asm/simd.h>
+#include <linux/cpufeature.h>
+
+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_sha3);
+
+asmlinkage size_t sha3_ce_transform(struct sha3_state *state, const u8 *data,
+				    size_t nblocks, size_t block_size);
+
+static void sha3_absorb_blocks(struct __sha3_ctx *ctx,
+			       const u8 *data, size_t nblocks)
+{
+	size_t block_size = ctx->block_size;
+
+	if (static_branch_likely(&have_sha3) && likely(may_use_simd())) {
+		do {
+			size_t rem;
+
+			kernel_neon_begin();
+			rem = sha3_ce_transform(&ctx->state, data, nblocks,
+						block_size);
+			kernel_neon_end();
+			data += (nblocks - rem) * block_size;
+			nblocks = rem;
+		} while (nblocks);
+	} else {
+		sha3_absorb_blocks_generic(ctx, data, nblocks);
+	}
+}
+
+static void sha3_keccakf(struct sha3_state *state)
+{
+	if (static_branch_likely(&have_sha3) && likely(may_use_simd())) {
+		/*
+		 * Passing zeroes into sha3_ce_transform() gives the plain
+		 * Keccak-f permutation, which is what we want here.  Any
+		 * supported block size may be used.  Use SHA3_512_BLOCK_SIZE
+		 * since it's the shortest.
+		 */
+		static const u8 zeroes[SHA3_512_BLOCK_SIZE];
+
+		kernel_neon_begin();
+		sha3_ce_transform(state, zeroes, 1, sizeof(zeroes));
+		kernel_neon_end();
+	} else {
+		sha3_keccakf_generic(state);
+	}
+}
+
+#define sha3_mod_init_arch sha3_mod_init_arch
+static void sha3_mod_init_arch(void)
+{
+	if (cpu_have_named_feature(SHA3))
+		static_branch_enable(&have_sha3);
+}
-- 
2.51.1.dirty

[PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Eric Biggers]

Instead of exposing the s390-optimized SHA-3 code via s390-specific
crypto_shash algorithms, instead just implement the sha3_absorb_blocks()
and sha3_keccakf() library functions.  This is much simpler, it makes
the SHA-3 library functions be s390-optimized, and it fixes the
longstanding issue where the s390-optimized SHA-3 code was disabled by
default.  SHA-3 still remains available through crypto_shash, but
individual architectures no longer need to handle it.

Note that the existing code used both CPACF_KIMD_SHA3_224 and
CPACF_KIMD_SHA3_256 after checking for just CPACF_KIMD_SHA3_256, and
similarly for 384 and 512.  I've preserved that behavior.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 arch/s390/configs/debug_defconfig |   3 +-
 arch/s390/configs/defconfig       |   3 +-
 arch/s390/crypto/Kconfig          |  20 ----
 arch/s390/crypto/Makefile         |   2 -
 arch/s390/crypto/sha.h            |  51 ----------
 arch/s390/crypto/sha3_256_s390.c  | 157 ------------------------------
 arch/s390/crypto/sha3_512_s390.c  | 157 ------------------------------
 arch/s390/crypto/sha_common.c     | 117 ----------------------
 lib/crypto/Kconfig                |   1 +
 lib/crypto/s390/sha3.h            |  76 +++++++++++++++
 10 files changed, 79 insertions(+), 508 deletions(-)
 delete mode 100644 arch/s390/crypto/sha.h
 delete mode 100644 arch/s390/crypto/sha3_256_s390.c
 delete mode 100644 arch/s390/crypto/sha3_512_s390.c
 delete mode 100644 arch/s390/crypto/sha_common.c
 create mode 100644 lib/crypto/s390/sha3.h

diff --git a/arch/s390/configs/debug_defconfig b/arch/s390/configs/debug_defconfig
index b31c1df902577..5fdfebcfd50f2 100644
--- a/arch/s390/configs/debug_defconfig
+++ b/arch/s390/configs/debug_defconfig
@@ -790,10 +790,11 @@ CONFIG_CRYPTO_GCM=y
 CONFIG_CRYPTO_SEQIV=y
 CONFIG_CRYPTO_MD4=m
 CONFIG_CRYPTO_MD5=y
 CONFIG_CRYPTO_MICHAEL_MIC=m
 CONFIG_CRYPTO_RMD160=m
+CONFIG_CRYPTO_SHA3=m
 CONFIG_CRYPTO_SM3_GENERIC=m
 CONFIG_CRYPTO_WP512=m
 CONFIG_CRYPTO_XCBC=m
 CONFIG_CRYPTO_CRC32=m
 CONFIG_CRYPTO_842=m
@@ -803,12 +804,10 @@ CONFIG_CRYPTO_ZSTD=m
 CONFIG_CRYPTO_ANSI_CPRNG=m
 CONFIG_CRYPTO_USER_API_HASH=m
 CONFIG_CRYPTO_USER_API_SKCIPHER=m
 CONFIG_CRYPTO_USER_API_RNG=m
 CONFIG_CRYPTO_USER_API_AEAD=m
-CONFIG_CRYPTO_SHA3_256_S390=m
-CONFIG_CRYPTO_SHA3_512_S390=m
 CONFIG_CRYPTO_GHASH_S390=m
 CONFIG_CRYPTO_AES_S390=m
 CONFIG_CRYPTO_DES_S390=m
 CONFIG_CRYPTO_HMAC_S390=m
 CONFIG_ZCRYPT=m
diff --git a/arch/s390/configs/defconfig b/arch/s390/configs/defconfig
index 161dad7ef211a..7bac3f53a95b0 100644
--- a/arch/s390/configs/defconfig
+++ b/arch/s390/configs/defconfig
@@ -774,10 +774,11 @@ CONFIG_CRYPTO_GCM=y
 CONFIG_CRYPTO_SEQIV=y
 CONFIG_CRYPTO_MD4=m
 CONFIG_CRYPTO_MD5=y
 CONFIG_CRYPTO_MICHAEL_MIC=m
 CONFIG_CRYPTO_RMD160=m
+CONFIG_CRYPTO_SHA3=m
 CONFIG_CRYPTO_SM3_GENERIC=m
 CONFIG_CRYPTO_WP512=m
 CONFIG_CRYPTO_XCBC=m
 CONFIG_CRYPTO_CRC32=m
 CONFIG_CRYPTO_842=m
@@ -788,12 +789,10 @@ CONFIG_CRYPTO_ANSI_CPRNG=m
 CONFIG_CRYPTO_JITTERENTROPY_OSR=1
 CONFIG_CRYPTO_USER_API_HASH=m
 CONFIG_CRYPTO_USER_API_SKCIPHER=m
 CONFIG_CRYPTO_USER_API_RNG=m
 CONFIG_CRYPTO_USER_API_AEAD=m
-CONFIG_CRYPTO_SHA3_256_S390=m
-CONFIG_CRYPTO_SHA3_512_S390=m
 CONFIG_CRYPTO_GHASH_S390=m
 CONFIG_CRYPTO_AES_S390=m
 CONFIG_CRYPTO_DES_S390=m
 CONFIG_CRYPTO_HMAC_S390=m
 CONFIG_ZCRYPT=m
diff --git a/arch/s390/crypto/Kconfig b/arch/s390/crypto/Kconfig
index 03f73fbd38b62..f838ca055f6d7 100644
--- a/arch/s390/crypto/Kconfig
+++ b/arch/s390/crypto/Kconfig
@@ -1,29 +1,9 @@
 # SPDX-License-Identifier: GPL-2.0
 
 menu "Accelerated Cryptographic Algorithms for CPU (s390)"
 
-config CRYPTO_SHA3_256_S390
-	tristate "Hash functions: SHA3-224 and SHA3-256"
-	select CRYPTO_HASH
-	help
-	  SHA3-224 and SHA3-256 secure hash algorithms (FIPS 202)
-
-	  Architecture: s390
-
-	  It is available as of z14.
-
-config CRYPTO_SHA3_512_S390
-	tristate "Hash functions: SHA3-384 and SHA3-512"
-	select CRYPTO_HASH
-	help
-	  SHA3-384 and SHA3-512 secure hash algorithms (FIPS 202)
-
-	  Architecture: s390
-
-	  It is available as of z14.
-
 config CRYPTO_GHASH_S390
 	tristate "Hash functions: GHASH"
 	select CRYPTO_HASH
 	help
 	  GCM GHASH hash function (NIST SP800-38D)
diff --git a/arch/s390/crypto/Makefile b/arch/s390/crypto/Makefile
index 998f4b656b18e..387a229e10381 100644
--- a/arch/s390/crypto/Makefile
+++ b/arch/s390/crypto/Makefile
@@ -1,12 +1,10 @@
 # SPDX-License-Identifier: GPL-2.0
 #
 # Cryptographic API
 #
 
-obj-$(CONFIG_CRYPTO_SHA3_256_S390) += sha3_256_s390.o sha_common.o
-obj-$(CONFIG_CRYPTO_SHA3_512_S390) += sha3_512_s390.o sha_common.o
 obj-$(CONFIG_CRYPTO_DES_S390) += des_s390.o
 obj-$(CONFIG_CRYPTO_AES_S390) += aes_s390.o
 obj-$(CONFIG_CRYPTO_PAES_S390) += paes_s390.o
 obj-$(CONFIG_S390_PRNG) += prng.o
 obj-$(CONFIG_CRYPTO_GHASH_S390) += ghash_s390.o
diff --git a/arch/s390/crypto/sha.h b/arch/s390/crypto/sha.h
deleted file mode 100644
index b9cd9572dd35c..0000000000000
--- a/arch/s390/crypto/sha.h
+++ /dev/null
@@ -1,51 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0+ */
-/*
- * Cryptographic API.
- *
- * s390 generic implementation of the SHA Secure Hash Algorithms.
- *
- * Copyright IBM Corp. 2007
- * Author(s): Jan Glauber (jang@de.ibm.com)
- */
-#ifndef _CRYPTO_ARCH_S390_SHA_H
-#define _CRYPTO_ARCH_S390_SHA_H
-
-#include <crypto/hash.h>
-#include <crypto/sha2.h>
-#include <crypto/sha3.h>
-#include <linux/build_bug.h>
-#include <linux/types.h>
-
-/* must be big enough for the largest SHA variant */
-#define CPACF_MAX_PARMBLOCK_SIZE	SHA3_STATE_SIZE
-#define SHA_MAX_BLOCK_SIZE		SHA3_224_BLOCK_SIZE
-
-struct s390_sha_ctx {
-	u64 count;		/* message length in bytes */
-	union {
-		u32 state[CPACF_MAX_PARMBLOCK_SIZE / sizeof(u32)];
-		struct {
-			u64 state[SHA512_DIGEST_SIZE / sizeof(u64)];
-			u64 count_hi;
-		} sha512;
-		struct {
-			__le64 state[SHA3_STATE_SIZE / sizeof(u64)];
-		} sha3;
-	};
-	int func;		/* KIMD function to use */
-	bool first_message_part;
-};
-
-struct shash_desc;
-
-int s390_sha_update_blocks(struct shash_desc *desc, const u8 *data,
-			   unsigned int len);
-int s390_sha_finup(struct shash_desc *desc, const u8 *src, unsigned int len,
-		   u8 *out);
-
-static inline void __check_s390_sha_ctx_size(void)
-{
-	BUILD_BUG_ON(S390_SHA_CTX_SIZE != sizeof(struct s390_sha_ctx));
-}
-
-#endif
diff --git a/arch/s390/crypto/sha3_256_s390.c b/arch/s390/crypto/sha3_256_s390.c
deleted file mode 100644
index fd5ecae60a572..0000000000000
--- a/arch/s390/crypto/sha3_256_s390.c
+++ /dev/null
@@ -1,157 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0+
-/*
- * Cryptographic API.
- *
- * s390 implementation of the SHA256 and SHA224 Secure Hash Algorithm.
- *
- * s390 Version:
- *   Copyright IBM Corp. 2019
- *   Author(s): Joerg Schmidbauer (jschmidb@de.ibm.com)
- */
-#include <asm/cpacf.h>
-#include <crypto/internal/hash.h>
-#include <crypto/sha3.h>
-#include <linux/cpufeature.h>
-#include <linux/errno.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
-
-#include "sha.h"
-
-static int s390_sha3_256_init(struct shash_desc *desc)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-
-	sctx->first_message_part = test_facility(86);
-	if (!sctx->first_message_part)
-		memset(sctx->state, 0, sizeof(sctx->state));
-	sctx->count = 0;
-	sctx->func = CPACF_KIMD_SHA3_256;
-
-	return 0;
-}
-
-static int s390_sha3_256_export(struct shash_desc *desc, void *out)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-	union {
-		u8 *u8;
-		u64 *u64;
-	} p = { .u8 = out };
-	int i;
-
-	if (sctx->first_message_part) {
-		memset(out, 0, SHA3_STATE_SIZE);
-		return 0;
-	}
-	for (i = 0; i < SHA3_STATE_SIZE / 8; i++)
-		put_unaligned(le64_to_cpu(sctx->sha3.state[i]), p.u64++);
-	return 0;
-}
-
-static int s390_sha3_256_import(struct shash_desc *desc, const void *in)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-	union {
-		const u8 *u8;
-		const u64 *u64;
-	} p = { .u8 = in };
-	int i;
-
-	for (i = 0; i < SHA3_STATE_SIZE / 8; i++)
-		sctx->sha3.state[i] = cpu_to_le64(get_unaligned(p.u64++));
-	sctx->count = 0;
-	sctx->first_message_part = 0;
-	sctx->func = CPACF_KIMD_SHA3_256;
-
-	return 0;
-}
-
-static int s390_sha3_224_import(struct shash_desc *desc, const void *in)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-
-	s390_sha3_256_import(desc, in);
-	sctx->func = CPACF_KIMD_SHA3_224;
-	return 0;
-}
-
-static struct shash_alg sha3_256_alg = {
-	.digestsize	=	SHA3_256_DIGEST_SIZE,	   /* = 32 */
-	.init		=	s390_sha3_256_init,
-	.update		=	s390_sha_update_blocks,
-	.finup		=	s390_sha_finup,
-	.export		=	s390_sha3_256_export,
-	.import		=	s390_sha3_256_import,
-	.descsize	=	S390_SHA_CTX_SIZE,
-	.statesize	=	SHA3_STATE_SIZE,
-	.base		=	{
-		.cra_name	 =	"sha3-256",
-		.cra_driver_name =	"sha3-256-s390",
-		.cra_priority	 =	300,
-		.cra_flags	 =	CRYPTO_AHASH_ALG_BLOCK_ONLY,
-		.cra_blocksize	 =	SHA3_256_BLOCK_SIZE,
-		.cra_module	 =	THIS_MODULE,
-	}
-};
-
-static int s390_sha3_224_init(struct shash_desc *desc)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-
-	s390_sha3_256_init(desc);
-	sctx->func = CPACF_KIMD_SHA3_224;
-	return 0;
-}
-
-static struct shash_alg sha3_224_alg = {
-	.digestsize	=	SHA3_224_DIGEST_SIZE,
-	.init		=	s390_sha3_224_init,
-	.update		=	s390_sha_update_blocks,
-	.finup		=	s390_sha_finup,
-	.export		=	s390_sha3_256_export, /* same as for 256 */
-	.import		=	s390_sha3_224_import, /* function code different! */
-	.descsize	=	S390_SHA_CTX_SIZE,
-	.statesize	=	SHA3_STATE_SIZE,
-	.base		=	{
-		.cra_name	 =	"sha3-224",
-		.cra_driver_name =	"sha3-224-s390",
-		.cra_priority	 =	300,
-		.cra_flags	 =	CRYPTO_AHASH_ALG_BLOCK_ONLY,
-		.cra_blocksize	 =	SHA3_224_BLOCK_SIZE,
-		.cra_module	 =	THIS_MODULE,
-	}
-};
-
-static int __init sha3_256_s390_init(void)
-{
-	int ret;
-
-	if (!cpacf_query_func(CPACF_KIMD, CPACF_KIMD_SHA3_256))
-		return -ENODEV;
-
-	ret = crypto_register_shash(&sha3_256_alg);
-	if (ret < 0)
-		goto out;
-
-	ret = crypto_register_shash(&sha3_224_alg);
-	if (ret < 0)
-		crypto_unregister_shash(&sha3_256_alg);
-out:
-	return ret;
-}
-
-static void __exit sha3_256_s390_fini(void)
-{
-	crypto_unregister_shash(&sha3_224_alg);
-	crypto_unregister_shash(&sha3_256_alg);
-}
-
-module_cpu_feature_match(S390_CPU_FEATURE_MSA, sha3_256_s390_init);
-module_exit(sha3_256_s390_fini);
-
-MODULE_ALIAS_CRYPTO("sha3-256");
-MODULE_ALIAS_CRYPTO("sha3-224");
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("SHA3-256 and SHA3-224 Secure Hash Algorithm");
diff --git a/arch/s390/crypto/sha3_512_s390.c b/arch/s390/crypto/sha3_512_s390.c
deleted file mode 100644
index f4b52a3a04339..0000000000000
--- a/arch/s390/crypto/sha3_512_s390.c
+++ /dev/null
@@ -1,157 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0+
-/*
- * Cryptographic API.
- *
- * s390 implementation of the SHA512 and SHA384 Secure Hash Algorithm.
- *
- * Copyright IBM Corp. 2019
- * Author(s): Joerg Schmidbauer (jschmidb@de.ibm.com)
- */
-#include <asm/cpacf.h>
-#include <crypto/internal/hash.h>
-#include <crypto/sha3.h>
-#include <linux/cpufeature.h>
-#include <linux/errno.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
-
-#include "sha.h"
-
-static int s390_sha3_512_init(struct shash_desc *desc)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-
-	sctx->first_message_part = test_facility(86);
-	if (!sctx->first_message_part)
-		memset(sctx->state, 0, sizeof(sctx->state));
-	sctx->count = 0;
-	sctx->func = CPACF_KIMD_SHA3_512;
-
-	return 0;
-}
-
-static int s390_sha3_512_export(struct shash_desc *desc, void *out)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-	union {
-		u8 *u8;
-		u64 *u64;
-	} p = { .u8 = out };
-	int i;
-
-	if (sctx->first_message_part) {
-		memset(out, 0, SHA3_STATE_SIZE);
-		return 0;
-	}
-	for (i = 0; i < SHA3_STATE_SIZE / 8; i++)
-		put_unaligned(le64_to_cpu(sctx->sha3.state[i]), p.u64++);
-	return 0;
-}
-
-static int s390_sha3_512_import(struct shash_desc *desc, const void *in)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-	union {
-		const u8 *u8;
-		const u64 *u64;
-	} p = { .u8 = in };
-	int i;
-
-	for (i = 0; i < SHA3_STATE_SIZE / 8; i++)
-		sctx->sha3.state[i] = cpu_to_le64(get_unaligned(p.u64++));
-	sctx->count = 0;
-	sctx->first_message_part = 0;
-	sctx->func = CPACF_KIMD_SHA3_512;
-
-	return 0;
-}
-
-static int s390_sha3_384_import(struct shash_desc *desc, const void *in)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-
-	s390_sha3_512_import(desc, in);
-	sctx->func = CPACF_KIMD_SHA3_384;
-	return 0;
-}
-
-static struct shash_alg sha3_512_alg = {
-	.digestsize	=	SHA3_512_DIGEST_SIZE,
-	.init		=	s390_sha3_512_init,
-	.update		=	s390_sha_update_blocks,
-	.finup		=	s390_sha_finup,
-	.export		=	s390_sha3_512_export,
-	.import		=	s390_sha3_512_import,
-	.descsize	=	S390_SHA_CTX_SIZE,
-	.statesize	=	SHA3_STATE_SIZE,
-	.base		=	{
-		.cra_name	 =	"sha3-512",
-		.cra_driver_name =	"sha3-512-s390",
-		.cra_priority	 =	300,
-		.cra_flags	 =	CRYPTO_AHASH_ALG_BLOCK_ONLY,
-		.cra_blocksize	 =	SHA3_512_BLOCK_SIZE,
-		.cra_module	 =	THIS_MODULE,
-	}
-};
-
-MODULE_ALIAS_CRYPTO("sha3-512");
-
-static int s390_sha3_384_init(struct shash_desc *desc)
-{
-	struct s390_sha_ctx *sctx = shash_desc_ctx(desc);
-
-	s390_sha3_512_init(desc);
-	sctx->func = CPACF_KIMD_SHA3_384;
-	return 0;
-}
-
-static struct shash_alg sha3_384_alg = {
-	.digestsize	=	SHA3_384_DIGEST_SIZE,
-	.init		=	s390_sha3_384_init,
-	.update		=	s390_sha_update_blocks,
-	.finup		=	s390_sha_finup,
-	.export		=	s390_sha3_512_export, /* same as for 512 */
-	.import		=	s390_sha3_384_import, /* function code different! */
-	.descsize	=	S390_SHA_CTX_SIZE,
-	.statesize	=	SHA3_STATE_SIZE,
-	.base		=	{
-		.cra_name	 =	"sha3-384",
-		.cra_driver_name =	"sha3-384-s390",
-		.cra_priority	 =	300,
-		.cra_flags	 =	CRYPTO_AHASH_ALG_BLOCK_ONLY,
-		.cra_blocksize	 =	SHA3_384_BLOCK_SIZE,
-		.cra_ctxsize	 =	sizeof(struct s390_sha_ctx),
-		.cra_module	 =	THIS_MODULE,
-	}
-};
-
-MODULE_ALIAS_CRYPTO("sha3-384");
-
-static int __init init(void)
-{
-	int ret;
-
-	if (!cpacf_query_func(CPACF_KIMD, CPACF_KIMD_SHA3_512))
-		return -ENODEV;
-	ret = crypto_register_shash(&sha3_512_alg);
-	if (ret < 0)
-		goto out;
-	ret = crypto_register_shash(&sha3_384_alg);
-	if (ret < 0)
-		crypto_unregister_shash(&sha3_512_alg);
-out:
-	return ret;
-}
-
-static void __exit fini(void)
-{
-	crypto_unregister_shash(&sha3_512_alg);
-	crypto_unregister_shash(&sha3_384_alg);
-}
-
-module_cpu_feature_match(S390_CPU_FEATURE_MSA, init);
-module_exit(fini);
-
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("SHA3-512 and SHA3-384 Secure Hash Algorithm");
diff --git a/arch/s390/crypto/sha_common.c b/arch/s390/crypto/sha_common.c
deleted file mode 100644
index d6f8396187946..0000000000000
--- a/arch/s390/crypto/sha_common.c
+++ /dev/null
@@ -1,117 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0+
-/*
- * Cryptographic API.
- *
- * s390 generic implementation of the SHA Secure Hash Algorithms.
- *
- * Copyright IBM Corp. 2007
- * Author(s): Jan Glauber (jang@de.ibm.com)
- */
-
-#include <crypto/internal/hash.h>
-#include <linux/export.h>
-#include <linux/module.h>
-#include <asm/cpacf.h>
-#include "sha.h"
-
-int s390_sha_update_blocks(struct shash_desc *desc, const u8 *data,
-			   unsigned int len)
-{
-	unsigned int bsize = crypto_shash_blocksize(desc->tfm);
-	struct s390_sha_ctx *ctx = shash_desc_ctx(desc);
-	unsigned int n;
-	int fc;
-
-	fc = ctx->func;
-	if (ctx->first_message_part)
-		fc |= CPACF_KIMD_NIP;
-
-	/* process as many blocks as possible */
-	n = (len / bsize) * bsize;
-	ctx->count += n;
-	switch (ctx->func) {
-	case CPACF_KLMD_SHA_512:
-	case CPACF_KLMD_SHA3_384:
-		if (ctx->count < n)
-			ctx->sha512.count_hi++;
-		break;
-	}
-	cpacf_kimd(fc, ctx->state, data, n);
-	ctx->first_message_part = 0;
-	return len - n;
-}
-EXPORT_SYMBOL_GPL(s390_sha_update_blocks);
-
-static int s390_crypto_shash_parmsize(int func)
-{
-	switch (func) {
-	case CPACF_KLMD_SHA_1:
-		return 20;
-	case CPACF_KLMD_SHA_256:
-		return 32;
-	case CPACF_KLMD_SHA_512:
-		return 64;
-	case CPACF_KLMD_SHA3_224:
-	case CPACF_KLMD_SHA3_256:
-	case CPACF_KLMD_SHA3_384:
-	case CPACF_KLMD_SHA3_512:
-		return 200;
-	default:
-		return -EINVAL;
-	}
-}
-
-int s390_sha_finup(struct shash_desc *desc, const u8 *src, unsigned int len,
-		   u8 *out)
-{
-	struct s390_sha_ctx *ctx = shash_desc_ctx(desc);
-	int mbl_offset, fc;
-	u64 bits;
-
-	ctx->count += len;
-
-	bits = ctx->count * 8;
-	mbl_offset = s390_crypto_shash_parmsize(ctx->func);
-	if (mbl_offset < 0)
-		return -EINVAL;
-
-	mbl_offset = mbl_offset / sizeof(u32);
-
-	/* set total msg bit length (mbl) in CPACF parmblock */
-	switch (ctx->func) {
-	case CPACF_KLMD_SHA_512:
-		/* The SHA512 parmblock has a 128-bit mbl field. */
-		if (ctx->count < len)
-			ctx->sha512.count_hi++;
-		ctx->sha512.count_hi <<= 3;
-		ctx->sha512.count_hi |= ctx->count >> 61;
-		mbl_offset += sizeof(u64) / sizeof(u32);
-		fallthrough;
-	case CPACF_KLMD_SHA_1:
-	case CPACF_KLMD_SHA_256:
-		memcpy(ctx->state + mbl_offset, &bits, sizeof(bits));
-		break;
-	case CPACF_KLMD_SHA3_224:
-	case CPACF_KLMD_SHA3_256:
-	case CPACF_KLMD_SHA3_384:
-	case CPACF_KLMD_SHA3_512:
-		break;
-	default:
-		return -EINVAL;
-	}
-
-	fc = ctx->func;
-	fc |= test_facility(86) ? CPACF_KLMD_DUFOP : 0;
-	if (ctx->first_message_part)
-		fc |= CPACF_KLMD_NIP;
-	cpacf_klmd(fc, ctx->state, src, len);
-
-	/* copy digest to out */
-	memcpy(out, ctx->state, crypto_shash_digestsize(desc->tfm));
-
-	return 0;
-}
-EXPORT_SYMBOL_GPL(s390_sha_finup);
-
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("s390 SHA cipher common functions");
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index 544eb67fe70f4..36bb4b94d0123 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -193,10 +193,11 @@ config CRYPTO_LIB_SHA3
 
 config CRYPTO_LIB_SHA3_ARCH
 	bool
 	depends on CRYPTO_LIB_SHA3 && !UML
 	default y if ARM64 && KERNEL_MODE_NEON
+	default y if S390
 
 config CRYPTO_LIB_SM3
 	tristate
 
 source "lib/crypto/tests/Kconfig"
diff --git a/lib/crypto/s390/sha3.h b/lib/crypto/s390/sha3.h
new file mode 100644
index 0000000000000..c08a192cc01a0
--- /dev/null
+++ b/lib/crypto/s390/sha3.h
@@ -0,0 +1,76 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * SHA-3 optimized using the CP Assist for Cryptographic Functions (CPACF)
+ *
+ * Copyright 2025 Google LLC
+ */
+#include <asm/cpacf.h>
+#include <linux/cpufeature.h>
+
+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_cpacf_sha3_256);
+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_cpacf_sha3_512);
+
+static void sha3_absorb_blocks(struct __sha3_ctx *ctx,
+			       const u8 *data, size_t nblocks)
+{
+	/*
+	 * Note that since the library functions keep the sha3_state in little
+	 * endian order, there is no need to convert it to little endian before
+	 * invoking CPACF_KIMD_SHA3_*, which also assume little endian order.
+	 */
+	if (static_branch_likely(&have_cpacf_sha3_256)) {
+		if (ctx->block_size == SHA3_224_BLOCK_SIZE) {
+			cpacf_kimd(CPACF_KIMD_SHA3_224, &ctx->state, data,
+				   nblocks * SHA3_224_BLOCK_SIZE);
+			return;
+		}
+		if (ctx->block_size == SHA3_256_BLOCK_SIZE) {
+			cpacf_kimd(CPACF_KIMD_SHA3_256, &ctx->state, data,
+				   nblocks * SHA3_256_BLOCK_SIZE);
+			return;
+		}
+	}
+	if (static_branch_likely(&have_cpacf_sha3_512)) {
+		if (ctx->block_size == SHA3_384_BLOCK_SIZE) {
+			cpacf_kimd(CPACF_KIMD_SHA3_384, &ctx->state, data,
+				   nblocks * SHA3_384_BLOCK_SIZE);
+			return;
+		}
+		if (ctx->block_size == SHA3_512_BLOCK_SIZE) {
+			cpacf_kimd(CPACF_KIMD_SHA3_512, &ctx->state, data,
+				   nblocks * SHA3_512_BLOCK_SIZE);
+			return;
+		}
+	}
+	sha3_absorb_blocks_generic(ctx, data, nblocks);
+}
+
+static void sha3_keccakf(struct sha3_state *state)
+{
+	if (static_branch_likely(&have_cpacf_sha3_512)) {
+		/*
+		 * Passing zeroes into any of CPACF_KIMD_SHA3_* gives the plain
+		 * Keccak-f permutation, which is what we want here.  Use
+		 * SHA3-512 since it has the smallest block size.
+		 *
+		 * Also, as in sha3_absorb_blocks(), the state needn't be
+		 * converted to little endian.  It already is little endian.
+		 */
+		static const u8 zeroes[SHA3_512_BLOCK_SIZE];
+
+		cpacf_kimd(CPACF_KIMD_SHA3_512, state, zeroes, sizeof(zeroes));
+	} else {
+		sha3_keccakf_generic(state);
+	}
+}
+
+#define sha3_mod_init_arch sha3_mod_init_arch
+static void sha3_mod_init_arch(void)
+{
+	if (cpu_have_feature(S390_CPU_FEATURE_MSA)) {
+		if (cpacf_query_func(CPACF_KIMD, CPACF_KIMD_SHA3_256))
+			static_branch_enable(&have_cpacf_sha3_256);
+		if (cpacf_query_func(CPACF_KIMD, CPACF_KIMD_SHA3_512))
+			static_branch_enable(&have_cpacf_sha3_512);
+	}
+}
-- 
2.51.1.dirty

[PATCH 16/17] crypto: jitterentropy - use default sha3 implementation

[Eric Biggers]

Make jitterentropy use "sha3-256" instead of "sha3-256-generic", as the
ability to explicitly request the generic code is going away.  It's not
worth providing a special generic API just for jitterentropy.  There are
many other solutions available to it, such as doing more iterations or
using a more effective jitter collection method.

Moreover, the status quo is that SHA-3 is quite slow anyway.  Currently
only arm64 and s390 have architecture-optimized SHA-3 code.  I'm not
familiar with the performance of the s390 one, but the arm64 one isn't
actually that much faster than the generic code anyway.

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 crypto/jitterentropy-kcapi.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/crypto/jitterentropy-kcapi.c b/crypto/jitterentropy-kcapi.c
index a53de7affe8d1..7c880cf34c523 100644
--- a/crypto/jitterentropy-kcapi.c
+++ b/crypto/jitterentropy-kcapi.c
@@ -46,11 +46,11 @@
 #include <linux/time.h>
 #include <crypto/internal/rng.h>
 
 #include "jitterentropy.h"
 
-#define JENT_CONDITIONING_HASH	"sha3-256-generic"
+#define JENT_CONDITIONING_HASH	"sha3-256"
 
 /***************************************************************************
  * Helper function
  ***************************************************************************/
 
@@ -228,19 +228,11 @@ static int jent_kcapi_init(struct crypto_tfm *tfm)
 	struct shash_desc *sdesc;
 	int size, ret = 0;
 
 	spin_lock_init(&rng->jent_lock);
 
-	/*
-	 * Use SHA3-256 as conditioner. We allocate only the generic
-	 * implementation as we are not interested in high-performance. The
-	 * execution time of the SHA3 operation is measured and adds to the
-	 * Jitter RNG's unpredictable behavior. If we have a slower hash
-	 * implementation, the execution timing variations are larger. When
-	 * using a fast implementation, we would need to call it more often
-	 * as its variations are lower.
-	 */
+	/* Use SHA3-256 as conditioner */
 	hash = crypto_alloc_shash(JENT_CONDITIONING_HASH, 0, 0);
 	if (IS_ERR(hash)) {
 		pr_err("Cannot allocate conditioning digest\n");
 		return PTR_ERR(hash);
 	}
-- 
2.51.1.dirty

[PATCH 17/17] crypto: sha3 - Reimplement using library API

[Eric Biggers]

Replace sha3_generic.c with a new file sha3.c which implements the SHA-3
crypto_shash algorithms on top of the SHA-3 library API.

Change the driver name suffix from "-generic" to "-lib" to reflect that
these algorithms now just use the (possibly arch-optimized) library.

This closely mirrors crypto/{md5,sha1,sha256,sha512,blake2b}.c.

Implement export_core and import_core since crypto/hmac.c expects these
to be present.  (Not like HMAC is needed with SHA-3 anyway, but dropping
support for it could be viewed as a regression.)

Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
 crypto/Makefile       |   2 +-
 crypto/sha3.c         | 172 +++++++++++++++++++++++++
 crypto/sha3_generic.c | 290 ------------------------------------------
 crypto/testmgr.c      |   8 ++
 include/crypto/sha3.h |   4 -
 5 files changed, 181 insertions(+), 295 deletions(-)
 create mode 100644 crypto/sha3.c
 delete mode 100644 crypto/sha3_generic.c

diff --git a/crypto/Makefile b/crypto/Makefile
index e430e6e99b6a2..657e15ba1fd7d 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -76,11 +76,11 @@ obj-$(CONFIG_CRYPTO_MD4) += md4.o
 obj-$(CONFIG_CRYPTO_MD5) += md5.o
 obj-$(CONFIG_CRYPTO_RMD160) += rmd160.o
 obj-$(CONFIG_CRYPTO_SHA1) += sha1.o
 obj-$(CONFIG_CRYPTO_SHA256) += sha256.o
 obj-$(CONFIG_CRYPTO_SHA512) += sha512.o
-obj-$(CONFIG_CRYPTO_SHA3) += sha3_generic.o
+obj-$(CONFIG_CRYPTO_SHA3) += sha3.o
 obj-$(CONFIG_CRYPTO_SM3_GENERIC) += sm3_generic.o
 obj-$(CONFIG_CRYPTO_STREEBOG) += streebog_generic.o
 obj-$(CONFIG_CRYPTO_WP512) += wp512.o
 CFLAGS_wp512.o := $(call cc-option,-fno-schedule-insns)  # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79149
 obj-$(CONFIG_CRYPTO_BLAKE2B) += blake2b_generic.o
diff --git a/crypto/sha3.c b/crypto/sha3.c
new file mode 100644
index 0000000000000..90c14841cb6b6
--- /dev/null
+++ b/crypto/sha3.c
@@ -0,0 +1,172 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Cryptographic API.
+ *
+ * SHA-3, as specified in
+ * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
+ *
+ * SHA-3 code by Jeff Garzik <jeff@garzik.org>
+ *               Ard Biesheuvel <ard.biesheuvel@linaro.org>
+ */
+#include <crypto/internal/hash.h>
+#include <crypto/sha3.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+
+#define SHA3_CTX(desc) ((struct sha3_ctx *)shash_desc_ctx(desc))
+
+static int crypto_sha3_224_init(struct shash_desc *desc)
+{
+	sha3_224_init(SHA3_CTX(desc));
+	return 0;
+}
+
+static int crypto_sha3_256_init(struct shash_desc *desc)
+{
+	sha3_256_init(SHA3_CTX(desc));
+	return 0;
+}
+
+static int crypto_sha3_384_init(struct shash_desc *desc)
+{
+	sha3_384_init(SHA3_CTX(desc));
+	return 0;
+}
+
+static int crypto_sha3_512_init(struct shash_desc *desc)
+{
+	sha3_512_init(SHA3_CTX(desc));
+	return 0;
+}
+
+static int crypto_sha3_update(struct shash_desc *desc, const u8 *data,
+			      unsigned int len)
+{
+	sha3_update(SHA3_CTX(desc), data, len);
+	return 0;
+}
+
+static int crypto_sha3_final(struct shash_desc *desc, u8 *out)
+{
+	sha3_final(SHA3_CTX(desc), out);
+	return 0;
+}
+
+static int crypto_sha3_224_digest(struct shash_desc *desc,
+				  const u8 *data, unsigned int len, u8 *out)
+{
+	sha3_224(data, len, out);
+	return 0;
+}
+
+static int crypto_sha3_256_digest(struct shash_desc *desc,
+				  const u8 *data, unsigned int len, u8 *out)
+{
+	sha3_256(data, len, out);
+	return 0;
+}
+
+static int crypto_sha3_384_digest(struct shash_desc *desc,
+				  const u8 *data, unsigned int len, u8 *out)
+{
+	sha3_384(data, len, out);
+	return 0;
+}
+
+static int crypto_sha3_512_digest(struct shash_desc *desc,
+				  const u8 *data, unsigned int len, u8 *out)
+{
+	sha3_512(data, len, out);
+	return 0;
+}
+
+static int crypto_sha3_export_core(struct shash_desc *desc, void *out)
+{
+	memcpy(out, SHA3_CTX(desc), sizeof(struct sha3_ctx));
+	return 0;
+}
+
+static int crypto_sha3_import_core(struct shash_desc *desc, const void *in)
+{
+	memcpy(SHA3_CTX(desc), in, sizeof(struct sha3_ctx));
+	return 0;
+}
+
+static struct shash_alg algs[] = { {
+	.digestsize		= SHA3_224_DIGEST_SIZE,
+	.init			= crypto_sha3_224_init,
+	.update			= crypto_sha3_update,
+	.final			= crypto_sha3_final,
+	.digest			= crypto_sha3_224_digest,
+	.export_core		= crypto_sha3_export_core,
+	.import_core		= crypto_sha3_import_core,
+	.descsize		= sizeof(struct sha3_ctx),
+	.base.cra_name		= "sha3-224",
+	.base.cra_driver_name	= "sha3-224-lib",
+	.base.cra_blocksize	= SHA3_224_BLOCK_SIZE,
+	.base.cra_module	= THIS_MODULE,
+}, {
+	.digestsize		= SHA3_256_DIGEST_SIZE,
+	.init			= crypto_sha3_256_init,
+	.update			= crypto_sha3_update,
+	.final			= crypto_sha3_final,
+	.digest			= crypto_sha3_256_digest,
+	.export_core		= crypto_sha3_export_core,
+	.import_core		= crypto_sha3_import_core,
+	.descsize		= sizeof(struct sha3_ctx),
+	.base.cra_name		= "sha3-256",
+	.base.cra_driver_name	= "sha3-256-lib",
+	.base.cra_blocksize	= SHA3_256_BLOCK_SIZE,
+	.base.cra_module	= THIS_MODULE,
+}, {
+	.digestsize		= SHA3_384_DIGEST_SIZE,
+	.init			= crypto_sha3_384_init,
+	.update			= crypto_sha3_update,
+	.final			= crypto_sha3_final,
+	.digest			= crypto_sha3_384_digest,
+	.export_core		= crypto_sha3_export_core,
+	.import_core		= crypto_sha3_import_core,
+	.descsize		= sizeof(struct sha3_ctx),
+	.base.cra_name		= "sha3-384",
+	.base.cra_driver_name	= "sha3-384-lib",
+	.base.cra_blocksize	= SHA3_384_BLOCK_SIZE,
+	.base.cra_module	= THIS_MODULE,
+}, {
+	.digestsize		= SHA3_512_DIGEST_SIZE,
+	.init			= crypto_sha3_512_init,
+	.update			= crypto_sha3_update,
+	.final			= crypto_sha3_final,
+	.digest			= crypto_sha3_512_digest,
+	.export_core		= crypto_sha3_export_core,
+	.import_core		= crypto_sha3_import_core,
+	.descsize		= sizeof(struct sha3_ctx),
+	.base.cra_name		= "sha3-512",
+	.base.cra_driver_name	= "sha3-512-lib",
+	.base.cra_blocksize	= SHA3_512_BLOCK_SIZE,
+	.base.cra_module	= THIS_MODULE,
+} };
+
+static int __init crypto_sha3_mod_init(void)
+{
+	return crypto_register_shashes(algs, ARRAY_SIZE(algs));
+}
+module_init(crypto_sha3_mod_init);
+
+static void __exit crypto_sha3_mod_exit(void)
+{
+	crypto_unregister_shashes(algs, ARRAY_SIZE(algs));
+}
+module_exit(crypto_sha3_mod_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("SHA-3 Secure Hash Algorithm");
+
+MODULE_ALIAS_CRYPTO("sha3-224");
+MODULE_ALIAS_CRYPTO("sha3-224-lib");
+MODULE_ALIAS_CRYPTO("sha3-256");
+MODULE_ALIAS_CRYPTO("sha3-256-lib");
+MODULE_ALIAS_CRYPTO("sha3-384");
+MODULE_ALIAS_CRYPTO("sha3-384-lib");
+MODULE_ALIAS_CRYPTO("sha3-512");
+MODULE_ALIAS_CRYPTO("sha3-512-lib");
diff --git a/crypto/sha3_generic.c b/crypto/sha3_generic.c
deleted file mode 100644
index 41d1e506e6dea..0000000000000
--- a/crypto/sha3_generic.c
+++ /dev/null
@@ -1,290 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Cryptographic API.
- *
- * SHA-3, as specified in
- * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
- *
- * SHA-3 code by Jeff Garzik <jeff@garzik.org>
- *               Ard Biesheuvel <ard.biesheuvel@linaro.org>
- */
-#include <crypto/internal/hash.h>
-#include <crypto/sha3.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/string.h>
-#include <linux/unaligned.h>
-
-/*
- * On some 32-bit architectures (h8300), GCC ends up using
- * over 1 KB of stack if we inline the round calculation into the loop
- * in keccakf(). On the other hand, on 64-bit architectures with plenty
- * of [64-bit wide] general purpose registers, not inlining it severely
- * hurts performance. So let's use 64-bitness as a heuristic to decide
- * whether to inline or not.
- */
-#ifdef CONFIG_64BIT
-#define SHA3_INLINE	inline
-#else
-#define SHA3_INLINE	noinline
-#endif
-
-#define KECCAK_ROUNDS 24
-
-static const u64 keccakf_rndc[24] = {
-	0x0000000000000001ULL, 0x0000000000008082ULL, 0x800000000000808aULL,
-	0x8000000080008000ULL, 0x000000000000808bULL, 0x0000000080000001ULL,
-	0x8000000080008081ULL, 0x8000000000008009ULL, 0x000000000000008aULL,
-	0x0000000000000088ULL, 0x0000000080008009ULL, 0x000000008000000aULL,
-	0x000000008000808bULL, 0x800000000000008bULL, 0x8000000000008089ULL,
-	0x8000000000008003ULL, 0x8000000000008002ULL, 0x8000000000000080ULL,
-	0x000000000000800aULL, 0x800000008000000aULL, 0x8000000080008081ULL,
-	0x8000000000008080ULL, 0x0000000080000001ULL, 0x8000000080008008ULL
-};
-
-/* update the state with given number of rounds */
-
-static SHA3_INLINE void keccakf_round(u64 st[25])
-{
-	u64 t[5], tt, bc[5];
-
-	/* Theta */
-	bc[0] = st[0] ^ st[5] ^ st[10] ^ st[15] ^ st[20];
-	bc[1] = st[1] ^ st[6] ^ st[11] ^ st[16] ^ st[21];
-	bc[2] = st[2] ^ st[7] ^ st[12] ^ st[17] ^ st[22];
-	bc[3] = st[3] ^ st[8] ^ st[13] ^ st[18] ^ st[23];
-	bc[4] = st[4] ^ st[9] ^ st[14] ^ st[19] ^ st[24];
-
-	t[0] = bc[4] ^ rol64(bc[1], 1);
-	t[1] = bc[0] ^ rol64(bc[2], 1);
-	t[2] = bc[1] ^ rol64(bc[3], 1);
-	t[3] = bc[2] ^ rol64(bc[4], 1);
-	t[4] = bc[3] ^ rol64(bc[0], 1);
-
-	st[0] ^= t[0];
-
-	/* Rho Pi */
-	tt = st[1];
-	st[ 1] = rol64(st[ 6] ^ t[1], 44);
-	st[ 6] = rol64(st[ 9] ^ t[4], 20);
-	st[ 9] = rol64(st[22] ^ t[2], 61);
-	st[22] = rol64(st[14] ^ t[4], 39);
-	st[14] = rol64(st[20] ^ t[0], 18);
-	st[20] = rol64(st[ 2] ^ t[2], 62);
-	st[ 2] = rol64(st[12] ^ t[2], 43);
-	st[12] = rol64(st[13] ^ t[3], 25);
-	st[13] = rol64(st[19] ^ t[4],  8);
-	st[19] = rol64(st[23] ^ t[3], 56);
-	st[23] = rol64(st[15] ^ t[0], 41);
-	st[15] = rol64(st[ 4] ^ t[4], 27);
-	st[ 4] = rol64(st[24] ^ t[4], 14);
-	st[24] = rol64(st[21] ^ t[1],  2);
-	st[21] = rol64(st[ 8] ^ t[3], 55);
-	st[ 8] = rol64(st[16] ^ t[1], 45);
-	st[16] = rol64(st[ 5] ^ t[0], 36);
-	st[ 5] = rol64(st[ 3] ^ t[3], 28);
-	st[ 3] = rol64(st[18] ^ t[3], 21);
-	st[18] = rol64(st[17] ^ t[2], 15);
-	st[17] = rol64(st[11] ^ t[1], 10);
-	st[11] = rol64(st[ 7] ^ t[2],  6);
-	st[ 7] = rol64(st[10] ^ t[0],  3);
-	st[10] = rol64(    tt ^ t[1],  1);
-
-	/* Chi */
-	bc[ 0] = ~st[ 1] & st[ 2];
-	bc[ 1] = ~st[ 2] & st[ 3];
-	bc[ 2] = ~st[ 3] & st[ 4];
-	bc[ 3] = ~st[ 4] & st[ 0];
-	bc[ 4] = ~st[ 0] & st[ 1];
-	st[ 0] ^= bc[ 0];
-	st[ 1] ^= bc[ 1];
-	st[ 2] ^= bc[ 2];
-	st[ 3] ^= bc[ 3];
-	st[ 4] ^= bc[ 4];
-
-	bc[ 0] = ~st[ 6] & st[ 7];
-	bc[ 1] = ~st[ 7] & st[ 8];
-	bc[ 2] = ~st[ 8] & st[ 9];
-	bc[ 3] = ~st[ 9] & st[ 5];
-	bc[ 4] = ~st[ 5] & st[ 6];
-	st[ 5] ^= bc[ 0];
-	st[ 6] ^= bc[ 1];
-	st[ 7] ^= bc[ 2];
-	st[ 8] ^= bc[ 3];
-	st[ 9] ^= bc[ 4];
-
-	bc[ 0] = ~st[11] & st[12];
-	bc[ 1] = ~st[12] & st[13];
-	bc[ 2] = ~st[13] & st[14];
-	bc[ 3] = ~st[14] & st[10];
-	bc[ 4] = ~st[10] & st[11];
-	st[10] ^= bc[ 0];
-	st[11] ^= bc[ 1];
-	st[12] ^= bc[ 2];
-	st[13] ^= bc[ 3];
-	st[14] ^= bc[ 4];
-
-	bc[ 0] = ~st[16] & st[17];
-	bc[ 1] = ~st[17] & st[18];
-	bc[ 2] = ~st[18] & st[19];
-	bc[ 3] = ~st[19] & st[15];
-	bc[ 4] = ~st[15] & st[16];
-	st[15] ^= bc[ 0];
-	st[16] ^= bc[ 1];
-	st[17] ^= bc[ 2];
-	st[18] ^= bc[ 3];
-	st[19] ^= bc[ 4];
-
-	bc[ 0] = ~st[21] & st[22];
-	bc[ 1] = ~st[22] & st[23];
-	bc[ 2] = ~st[23] & st[24];
-	bc[ 3] = ~st[24] & st[20];
-	bc[ 4] = ~st[20] & st[21];
-	st[20] ^= bc[ 0];
-	st[21] ^= bc[ 1];
-	st[22] ^= bc[ 2];
-	st[23] ^= bc[ 3];
-	st[24] ^= bc[ 4];
-}
-
-static void keccakf(u64 st[25])
-{
-	int round;
-
-	for (round = 0; round < KECCAK_ROUNDS; round++) {
-		keccakf_round(st);
-		/* Iota */
-		st[0] ^= keccakf_rndc[round];
-	}
-}
-
-int crypto_sha3_init(struct shash_desc *desc)
-{
-	struct sha3_state *sctx = shash_desc_ctx(desc);
-
-	memset(sctx->st, 0, sizeof(sctx->st));
-	return 0;
-}
-EXPORT_SYMBOL(crypto_sha3_init);
-
-static int crypto_sha3_update(struct shash_desc *desc, const u8 *data,
-			      unsigned int len)
-{
-	unsigned int rsiz = crypto_shash_blocksize(desc->tfm);
-	struct sha3_state *sctx = shash_desc_ctx(desc);
-	unsigned int rsizw = rsiz / 8;
-
-	do {
-		int i;
-
-		for (i = 0; i < rsizw; i++)
-			sctx->st[i] ^= get_unaligned_le64(data + 8 * i);
-		keccakf(sctx->st);
-
-		data += rsiz;
-		len -= rsiz;
-	} while (len >= rsiz);
-	return len;
-}
-
-static int crypto_sha3_finup(struct shash_desc *desc, const u8 *src,
-			     unsigned int len, u8 *out)
-{
-	unsigned int digest_size = crypto_shash_digestsize(desc->tfm);
-	unsigned int rsiz = crypto_shash_blocksize(desc->tfm);
-	struct sha3_state *sctx = shash_desc_ctx(desc);
-	__le64 block[SHA3_224_BLOCK_SIZE / 8] = {};
-	__le64 *digest = (__le64 *)out;
-	unsigned int rsizw = rsiz / 8;
-	u8 *p;
-	int i;
-
-	p = memcpy(block, src, len);
-	p[len++] = 0x06;
-	p[rsiz - 1] |= 0x80;
-
-	for (i = 0; i < rsizw; i++)
-		sctx->st[i] ^= le64_to_cpu(block[i]);
-	memzero_explicit(block, sizeof(block));
-
-	keccakf(sctx->st);
-
-	for (i = 0; i < digest_size / 8; i++)
-		put_unaligned_le64(sctx->st[i], digest++);
-
-	if (digest_size & 4)
-		put_unaligned_le32(sctx->st[i], (__le32 *)digest);
-
-	return 0;
-}
-
-static struct shash_alg algs[] = { {
-	.digestsize		= SHA3_224_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= crypto_sha3_update,
-	.finup			= crypto_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-224",
-	.base.cra_driver_name	= "sha3-224-generic",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_224_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-}, {
-	.digestsize		= SHA3_256_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= crypto_sha3_update,
-	.finup			= crypto_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-256",
-	.base.cra_driver_name	= "sha3-256-generic",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_256_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-}, {
-	.digestsize		= SHA3_384_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= crypto_sha3_update,
-	.finup			= crypto_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-384",
-	.base.cra_driver_name	= "sha3-384-generic",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_384_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-}, {
-	.digestsize		= SHA3_512_DIGEST_SIZE,
-	.init			= crypto_sha3_init,
-	.update			= crypto_sha3_update,
-	.finup			= crypto_sha3_finup,
-	.descsize		= SHA3_STATE_SIZE,
-	.base.cra_name		= "sha3-512",
-	.base.cra_driver_name	= "sha3-512-generic",
-	.base.cra_flags		= CRYPTO_AHASH_ALG_BLOCK_ONLY,
-	.base.cra_blocksize	= SHA3_512_BLOCK_SIZE,
-	.base.cra_module	= THIS_MODULE,
-} };
-
-static int __init sha3_generic_mod_init(void)
-{
-	return crypto_register_shashes(algs, ARRAY_SIZE(algs));
-}
-
-static void __exit sha3_generic_mod_fini(void)
-{
-	crypto_unregister_shashes(algs, ARRAY_SIZE(algs));
-}
-
-module_init(sha3_generic_mod_init);
-module_exit(sha3_generic_mod_fini);
-
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("SHA-3 Secure Hash Algorithm");
-
-MODULE_ALIAS_CRYPTO("sha3-224");
-MODULE_ALIAS_CRYPTO("sha3-224-generic");
-MODULE_ALIAS_CRYPTO("sha3-256");
-MODULE_ALIAS_CRYPTO("sha3-256-generic");
-MODULE_ALIAS_CRYPTO("sha3-384");
-MODULE_ALIAS_CRYPTO("sha3-384-generic");
-MODULE_ALIAS_CRYPTO("sha3-512");
-MODULE_ALIAS_CRYPTO("sha3-512-generic");
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 6a490aaa71b9a..bf56877ff179d 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -5098,31 +5098,35 @@ static const struct alg_test_desc alg_test_descs[] = {
 		.suite = {
 			.hash = __VECS(hmac_sha256_tv_template)
 		}
 	}, {
 		.alg = "hmac(sha3-224)",
+		.generic_driver = "hmac(sha3-224-lib)",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(hmac_sha3_224_tv_template)
 		}
 	}, {
 		.alg = "hmac(sha3-256)",
+		.generic_driver = "hmac(sha3-256-lib)",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(hmac_sha3_256_tv_template)
 		}
 	}, {
 		.alg = "hmac(sha3-384)",
+		.generic_driver = "hmac(sha3-384-lib)",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(hmac_sha3_384_tv_template)
 		}
 	}, {
 		.alg = "hmac(sha3-512)",
+		.generic_driver = "hmac(sha3-512-lib)",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(hmac_sha3_512_tv_template)
 		}
@@ -5472,31 +5476,35 @@ static const struct alg_test_desc alg_test_descs[] = {
 		.suite = {
 			.hash = __VECS(sha256_tv_template)
 		}
 	}, {
 		.alg = "sha3-224",
+		.generic_driver = "sha3-224-lib",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(sha3_224_tv_template)
 		}
 	}, {
 		.alg = "sha3-256",
+		.generic_driver = "sha3-256-lib",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(sha3_256_tv_template)
 		}
 	}, {
 		.alg = "sha3-384",
+		.generic_driver = "sha3-384-lib",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(sha3_384_tv_template)
 		}
 	}, {
 		.alg = "sha3-512",
+		.generic_driver = "sha3-512-lib",
 		.test = alg_test_hash,
 		.fips_allowed = 1,
 		.suite = {
 			.hash = __VECS(sha3_512_tv_template)
 		}
diff --git a/include/crypto/sha3.h b/include/crypto/sha3.h
index 80acdb266b8e3..d562a25332b92 100644
--- a/include/crypto/sha3.h
+++ b/include/crypto/sha3.h
@@ -34,14 +34,10 @@
 #define SHAKE256_DEFAULT_SIZE	(256 / 8)
 #define SHAKE256_BLOCK_SIZE	(200 - 2 * SHAKE256_DEFAULT_SIZE)
 
 #define SHA3_STATE_SIZE		200
 
-struct shash_desc;
-
-int crypto_sha3_init(struct shash_desc *desc);
-
 struct sha3_state {
 	u64		st[SHA3_STATE_SIZE / 8];
 };
 
 /**
-- 
2.51.1.dirty

Re: [PATCH 03/17] lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

On Sun, Oct 19, 2025 at 05:50:24PM -0700, Eric Biggers wrote:
> +The SHA-3 algorithm base, as specified in NIST FIPS-202[1], provides a number
> +of specific variants all based on the same basic algorithm (the Keccak sponge
> +function and permutation).  The differences between them are: the "rate" (how
> +much of the common state buffer gets updated with new data between invocations

Use reST footnotes, like:

---- >8 ----
diff --git a/Documentation/crypto/sha3.rst b/Documentation/crypto/sha3.rst
index c27da98c89b7f8..ae1fd3e01e34c2 100644
--- a/Documentation/crypto/sha3.rst
+++ b/Documentation/crypto/sha3.rst
@@ -18,7 +18,7 @@ SHA-3 Algorithm collection
 Overview
 ========
 
-The SHA-3 algorithm base, as specified in NIST FIPS-202[1], provides a number
+The SHA-3 algorithm base, as specified in NIST FIPS-202 [1]_, provides a number
 of specific variants all based on the same basic algorithm (the Keccak sponge
 function and permutation).  The differences between them are: the "rate" (how
 much of the common state buffer gets updated with new data between invocations
@@ -136,7 +136,7 @@ should use the much more comprehensive KUnit test suite instead.
 References
 ==========
 
-[1] https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
+.. [1] https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
 
 
 
> +If selectable algorithms are required then the crypto_hash API may be used
> +instead as this binds each algorithm to a specific C type.

What is crypto_hash API then? I can't find any of its documentation beside
being mentioned here.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH 11/17] lib/crypto: sha3: Simplify the API

[David Howells]

Eric Biggers <ebiggers@kernel.org> wrote:

> Instead of having separate types and functions for each of the six SHA-3
> algorithms, instead divide them into two groups: the digests and the
> XOFs.  The digests use sha3_ctx and the XOFs use shake_ctx.  The
> internal context is now called __sha3_ctx.

Please roll changes into the original patches rather than posting them with a
set of "fixes" and add a Co-developed-by tag for yourself.  Or if you want
your authorship on your changes, just switch the Author to yourself and put a
note in the changelog noting that you modified it from what I posted.

> +/** Context for SHA3-224, SHA3-256, SHA3-384, or SHA3-512 */
> +struct sha3_ctx {
> +	struct __sha3_ctx	ctx;
> +	u8			digest_size;	/* Digest size in bytes */
> +};

Don't do that.  That expands the context by an extra word when there's spare
space in __sha3_ctx.  If you go with the separate types, then this field is
redundant.  Actually, I lean slightly towards passing in the desired digest
length to sha3_*final() and doing a WARN if it doesn't match.

> +static inline void sha3_zeroize_ctx(struct sha3_ctx *ctx)

sha3_zero_ctx() please if you don't like "sha3_clear_ctx".  "zero" is a
perfectly usable as verb in itself.

> +/** Zeroize a shake_ctx.  Call this after the last squeeze. */

/**
 * shake_zero_ctx - Clear a shake_ctx.
 * @ctx: The context to clear.
 * 
 * Clear the context for a shake XOF.  Call after the last squeeze.
 */

Something like this, please.

David

Re: [PATCH 16/17] crypto: jitterentropy - use default sha3 implementation

[David Howells]

Why don't you take my approach and just call lib/crypto/sha3 directly rather
than using a crypto/ object as an intermediary if that crypto/ object is just
going to wrap lib/crypto?

David

Re: [PATCH 03/17] lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256

[David Howells]

Bagas Sanjaya <bagasdotme@gmail.com> wrote:

> > +If selectable algorithms are required then the crypto_hash API may be used
> > +instead as this binds each algorithm to a specific C type.
> 
> What is crypto_hash API then? I can't find any of its documentation beside
> being mentioned here.

crypto_shash, then.

David

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Holger Dengler]

On 20/10/2025 02:50, Eric Biggers wrote:
> Instead of exposing the s390-optimized SHA-3 code via s390-specific
> crypto_shash algorithms, instead just implement the sha3_absorb_blocks()
> and sha3_keccakf() library functions.  This is much simpler, it makes
> the SHA-3 library functions be s390-optimized, and it fixes the
> longstanding issue where the s390-optimized SHA-3 code was disabled by
> default.  SHA-3 still remains available through crypto_shash, but
> individual architectures no longer need to handle it.
> 
> Note that the existing code used both CPACF_KIMD_SHA3_224 and
> CPACF_KIMD_SHA3_256 after checking for just CPACF_KIMD_SHA3_256, and
> similarly for 384 and 512.  I've preserved that behavior.
> 
> Signed-off-by: Eric Biggers <ebiggers@kernel.org>
The current code also cover a performance feature, which allows (on supported hardware, e.g. z17) to skip the ICV initialization. The support has been introduced with 88c02b3f79a6 ("s390/sha3: Support sha3 performance enhancements"). Unfortunately, this patch removes this support. Was this intended?

The get this feature back, we need to hook also into the init() function, set the according bit for the first message block and skip the initialization of the ICV.

Please also add me and Harald Freudenberger to the cc: list for this patch.

-- 
Mit freundlichen Grüßen / Kind regards
Holger Dengler
--
IBM Systems, Linux on IBM Z Development
dengler@linux.ibm.com

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Holger Dengler]

On 20/10/2025 16:00, Holger Dengler wrote:
> On 20/10/2025 02:50, Eric Biggers wrote:
>> Instead of exposing the s390-optimized SHA-3 code via s390-specific
>> crypto_shash algorithms, instead just implement the sha3_absorb_blocks()
>> and sha3_keccakf() library functions.  This is much simpler, it makes
>> the SHA-3 library functions be s390-optimized, and it fixes the
>> longstanding issue where the s390-optimized SHA-3 code was disabled by
>> default.  SHA-3 still remains available through crypto_shash, but
>> individual architectures no longer need to handle it.
>>
>> Note that the existing code used both CPACF_KIMD_SHA3_224 and
>> CPACF_KIMD_SHA3_256 after checking for just CPACF_KIMD_SHA3_256, and
>> similarly for 384 and 512.  I've preserved that behavior.
>>
>> Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> The current code also cover a performance feature, which allows (on supported hardware, e.g. z17) to skip the ICV initialization. The support has been introduced with 88c02b3f79a6 ("s390/sha3: Support sha3 performance enhancements"). Unfortunately, this patch removes this support. Was this intended?
> 
> The get this feature back, we need to hook also into the init() function, set the according bit for the first message block and skip the initialization of the ICV.

And it would - performance wise - make some sense on s390 to replace also the sha3_xxx() functions. The init()/update()/final() sequence is not necessary on s390, as it can all be processed by one single KLMD instruction call.

> 
> Please also add me and Harald Freudenberger to the cc: list for this patch.
> 

-- 
Mit freundlichen Grüßen / Kind regards
Holger Dengler
--
IBM Systems, Linux on IBM Z Development
dengler@linux.ibm.com

Re: [PATCH 11/17] lib/crypto: sha3: Simplify the API

[Eric Biggers]

On Mon, Oct 20, 2025 at 11:33:36AM +0100, David Howells wrote:
> Eric Biggers <ebiggers@kernel.org> wrote:
> 
> > Instead of having separate types and functions for each of the six SHA-3
> > algorithms, instead divide them into two groups: the digests and the
> > XOFs.  The digests use sha3_ctx and the XOFs use shake_ctx.  The
> > internal context is now called __sha3_ctx.
> 
> Please roll changes into the original patches rather than posting them with a
> set of "fixes" and add a Co-developed-by tag for yourself.

Sure, that sounds good to me.  I'll do that on the next version.  For
this version I wanted to make it clear what I had changed.  Also, some
people don't like it when their patches are changed, so I wanted to make
sure you were okay with it first.

> > +/** Context for SHA3-224, SHA3-256, SHA3-384, or SHA3-512 */
> > +struct sha3_ctx {
> > +	struct __sha3_ctx	ctx;
> > +	u8			digest_size;	/* Digest size in bytes */
> > +};
> 
> Don't do that.  That expands the context by an extra word when there's spare
> space in __sha3_ctx.  If you go with the separate types, then this field is
> redundant.

Right, packing the struct tightly is nice.  On the other hand, having
the digest_size be in the digest-specific struct prevents it from
accidentally be used during XOF operations.

Consider that the arm64 SHA-3 assembly function took a digest_size
argument.  It was tempting to just use __sha3_ctx::digest_size, but of
course that would have been wrong.

But it looks like we could tighten sha3_absorb_blocks() to take a
(sha3_state, block_size) pair.  That would reduce the number of places
in which __sha3_ctx is used.  We should do that anyway.  So I'll
tentatively plan to do that and also move the digest_size to __sha3_ctx.

> Actually, I lean slightly towards passing in the desired digest length
> to sha3_*final() and doing a WARN if it doesn't match.

That would be redundant though.  It would make the API more difficult to
use, especially in the case where the caller is supporting multiple
SHA-3 variants, e.g. crypto/sha3.c.

Note that BLAKE2s has a variable-length digest too, and the BLAKE2s API
only requires that the digest size be passed to init.

I prefer the simpler version that's consistent with the BLAKE2s API.

> > +static inline void sha3_zeroize_ctx(struct sha3_ctx *ctx)
> 
> sha3_zero_ctx() please if you don't like "sha3_clear_ctx".  "zero" is a
> perfectly usable as verb in itself.

In cryptography it's normally zeroize.  See
https://en.wikipedia.org/wiki/Zeroisation

And also try 'git grep zeroize include/crypto/':

    include/crypto/acompress.h: * acomp_request_free() -- zeroize and free asynchronous (de)compression
    include/crypto/aead.h: * crypto_free_aead() - zeroize and free aead handle
    include/crypto/aead.h: * aead_request_free() - zeroize and free request data structure
    include/crypto/akcipher.h: * akcipher_request_free() - zeroize and free public key request
    include/crypto/chacha.h:static inline void chacha_zeroize_state(struct chacha_state *state)
    include/crypto/hash.h: * crypto_free_ahash() - zeroize and free the ahash handle
    include/crypto/hash.h: * ahash_request_free() - zeroize and free the request data structure
    include/crypto/hash.h: * crypto_free_shash() - zeroize and free the message digest handle
    include/crypto/internal/cipher.h: * crypto_free_cipher() - zeroize and free the single block cipher handle
    include/crypto/kpp.h: * kpp_request_free() - zeroize and free kpp request
    include/crypto/md5.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/md5.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/rng.h: * crypto_free_rng() - zeroize and free RNG handle
    include/crypto/sha1.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha1.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/sha2.h: * After finishing, this zeroizes @ctx.  So the caller does not need to do it.
    include/crypto/skcipher.h: * crypto_free_skcipher() - zeroize and free cipher handle
    include/crypto/skcipher.h: * crypto_free_lskcipher() - zeroize and free cipher handle
    include/crypto/skcipher.h: * skcipher_request_free() - zeroize and free request data structure

Point is, we should prefer standard terminology that is already used in
the code and the wider problem domain, instead of coming up with new
terminology each time a new contributor hops in.

- Eric

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Eric Biggers]

On Mon, Oct 20, 2025 at 04:00:42PM +0200, Holger Dengler wrote:
> On 20/10/2025 02:50, Eric Biggers wrote:
> > Instead of exposing the s390-optimized SHA-3 code via s390-specific
> > crypto_shash algorithms, instead just implement the sha3_absorb_blocks()
> > and sha3_keccakf() library functions.  This is much simpler, it makes
> > the SHA-3 library functions be s390-optimized, and it fixes the
> > longstanding issue where the s390-optimized SHA-3 code was disabled by
> > default.  SHA-3 still remains available through crypto_shash, but
> > individual architectures no longer need to handle it.
> > 
> > Note that the existing code used both CPACF_KIMD_SHA3_224 and
> > CPACF_KIMD_SHA3_256 after checking for just CPACF_KIMD_SHA3_256, and
> > similarly for 384 and 512.  I've preserved that behavior.
> > 
> > Signed-off-by: Eric Biggers <ebiggers@kernel.org>
> The current code also cover a performance feature, which allows (on
> supported hardware, e.g. z17) to skip the ICV initialization.

I'm not sure if by "ICV" you mean "Integrity Check Value" or "Initial
Chaining Value", but SHA-3 doesn't have either of those.  It just starts
with a state of all zeroes.  I assume that skipping the
zero-initialization of the state is what you're referring to?

> support has been introduced with 88c02b3f79a6 ("s390/sha3: Support
> sha3 performance enhancements"). Unfortunately, this patch removes
> this support. Was this intended?

For now, yes.  I should have explained more in the patch, sorry.

As currently proposed, lib/crypto/sha3.c supports arch-specific
overrides of sha3_absorb_blocks() and sha3_keccakf().  Those cover the
Keccak-f permutation which is by far the most performance critical part.
This strategy is working well in the SHA-2, SHA-1, and MD5 libraries,
which support the same level of arch overrides.

We could update lib/crypto/sha3.c to allow architectures to override
more of the code.  But we need to consider the tradeoffs:

- Risk of bugs.  QEMU doesn't support the s390 SHA-3 instructions, so no
  one except the s390 folks can test the code.  I can try to write code
  for you, but I can't test it.  And the s390 SHA-3 code has had bugs;
  see commits 992b7066800f, 68279380266a5, 73c2437109c3.

  The first priority should be correctness.

- The proposed change to the init functions would cause the format of
  'struct __sha3_ctx' to be architecture-dependent.  While we can do
  that if really needed, it's something that's best avoided for
  simplicity.  It opens up more opportunity for error.

- As I mentioned, Keccak-f is by far the most performance critical part
  anyway.  The initial state is just all zeroes, and initializing it is
  very lightweight.  Also consider that these contexts are often on the
  stack, and people increasingly set the "init all stack variables to
  zero" kernel hardening option anyway.

  I'll also note that commit 88c02b3f79a6 has no performance data in it.
  So it's not clear that it actually helped much.

- The library has an optimization to greatly reduce the size of the
  context: instead of buffering data separately, it just XOR's data into
  the state.  So, if there's a sha3_*_init() followed by a sha3_update()
  of less than 1 block, it will have to initialize the state anyway.  We
  can delay it until that point on s390.  But again: complexity.

- These potential additional s390 optimizations would presumably help
  the most on short messages.  However, on short messages, merely
  switching to the library often gives a large performance improvement
  due to eliminating the very slow call to crypto_alloc_shash().  That's
  actually a lot more important.

I would suggest that we drop the sha3_*_init() optimization from
consideration for now.  Providing overrides for the one-shot functions
sha3_{224,256,384,512}() should be simpler as well as possibly a bit
more useful, and I would suggest exploring that.

I guess I can try to write the code for you again.  But again, without
QEMU support I cannot test it.  The first priority in cryptography code
is correctness, so that's not a great position to be in.

Note that for new optimized code I'm requiring QEMU support for the
instructions it uses.  This one would only be allowed because code that
used these instructions already existed in arch/s390/crypto/.

> Please also add me and Harald Freudenberger to the cc: list for this patch.

Will do, thanks.

- Eric

Re: [PATCH 16/17] crypto: jitterentropy - use default sha3 implementation

[Eric Biggers]

On Mon, Oct 20, 2025 at 11:35:30AM +0100, David Howells wrote:
> Why don't you take my approach and just call lib/crypto/sha3 directly rather
> than using a crypto/ object as an intermediary if that crypto/ object is just
> going to wrap lib/crypto?

We'll do that, and thanks for writing the patch already!  But that's
something to do in a later patch after adding the library API.  Your
patch description kind of raised a red flag:

    Make the jitterentropy RNG use lib/crypto/sha3 rather than
    crypto/sha3.

    For some reason it goes absolutely wild if crypto/sha3 is
    reimplemented to use lib/crypto/sha3, but it's fine if it uses lib
    directly.

That implies that your changes broke crypto_shash, so you *had* to
convert to the library right away as a workaround for that.

That shouldn't be necessary.  We should generally keep crypto_shash
working for now.  In which case, all jitterentropy should need for now
is a simple substitution s/sha3-256-generic/sha3-256/.

We'll convert jitterentropy to use the library API too.  It's better,
after all.  But it should be done later and because the library API is
better -- not as some sort of workaround.

- Eric

Re: [PATCH 03/17] lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256

[Bagas Sanjaya]

On 10/20/25 17:39, David Howells wrote:
> Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> 
>>> +If selectable algorithms are required then the crypto_hash API may be used
>>> +instead as this binds each algorithm to a specific C type.
>>
>> What is crypto_hash API then? I can't find any of its documentation beside
>> being mentioned here.
> 
> crypto_shash, then.

OK, thanks!

-- 
An old man doll... just what I always wanted! - Clara

Re: [PATCH 17/17] crypto: sha3 - Reimplement using library API

[David Howells]

This needs to modify Kconfig too.

 config CRYPTO_SHA3
 	tristate "SHA-3"
 	select CRYPTO_HASH
+	select CRYPTO_LIB_SHA3
 	help
 	  SHA-3 secure hash algorithms (FIPS 202, ISO/IEC 10118-3)

David

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Holger Dengler]

On 20/10/2025 19:57, Eric Biggers wrote:
> On Mon, Oct 20, 2025 at 04:00:42PM +0200, Holger Dengler wrote:
>> On 20/10/2025 02:50, Eric Biggers wrote:
>>> Instead of exposing the s390-optimized SHA-3 code via s390-specific
>>> crypto_shash algorithms, instead just implement the sha3_absorb_blocks()
>>> and sha3_keccakf() library functions.  This is much simpler, it makes
>>> the SHA-3 library functions be s390-optimized, and it fixes the
>>> longstanding issue where the s390-optimized SHA-3 code was disabled by
>>> default.  SHA-3 still remains available through crypto_shash, but
>>> individual architectures no longer need to handle it.
>>>
>>> Note that the existing code used both CPACF_KIMD_SHA3_224 and
>>> CPACF_KIMD_SHA3_256 after checking for just CPACF_KIMD_SHA3_256, and
>>> similarly for 384 and 512.  I've preserved that behavior.
>>>
>>> Signed-off-by: Eric Biggers <ebiggers@kernel.org>
>> The current code also cover a performance feature, which allows (on
>> supported hardware, e.g. z17) to skip the ICV initialization.
> 
> I'm not sure if by "ICV" you mean "Integrity Check Value" or "Initial
> Chaining Value", but SHA-3 doesn't have either of those.  It just starts
> with a state of all zeroes.  I assume that skipping the
> zero-initialization of the state is what you're referring to?

I meant "Initial Chaining Value". On s390, this memory is set to zero by the
KIMD/KLMD instructions so that it can be skipped in the init() calls.

>> support has been introduced with 88c02b3f79a6 ("s390/sha3: Support
>> sha3 performance enhancements"). Unfortunately, this patch removes
>> this support. Was this intended?
> 
> For now, yes.  I should have explained more in the patch, sorry.
> 
> As currently proposed, lib/crypto/sha3.c supports arch-specific
> overrides of sha3_absorb_blocks() and sha3_keccakf().  Those cover the
> Keccak-f permutation which is by far the most performance critical part.
> This strategy is working well in the SHA-2, SHA-1, and MD5 libraries,
> which support the same level of arch overrides.
> 
> We could update lib/crypto/sha3.c to allow architectures to override
> more of the code.  But we need to consider the tradeoffs:
> 
> - Risk of bugs.  QEMU doesn't support the s390 SHA-3 instructions, so no
>   one except the s390 folks can test the code.  I can try to write code
>   for you, but I can't test it.  And the s390 SHA-3 code has had bugs;
>   see commits 992b7066800f, 68279380266a5, 73c2437109c3.
> 
>   The first priority should be correctness.

Let me figure out, if me and my colleagues can do the testing for you.
Unfortunately, I'll be unavailable for the next two weeks. But I'll come back
with a solution for the testing.

> - The proposed change to the init functions would cause the format of
>   'struct __sha3_ctx' to be architecture-dependent.  While we can do
>   that if really needed, it's something that's best avoided for
>   simplicity.  It opens up more opportunity for error.

Would it make sense to provide a delayed zeroize mechanism of the initial
chaining value (ICV), and architectures may implement it or not? The feature
on s390 is exactly that: a delayed zeroize of the ICV, done by the instruction.

> - As I mentioned, Keccak-f is by far the most performance critical part
>   anyway.  The initial state is just all zeroes, and initializing it is
>   very lightweight.  Also consider that these contexts are often on the
>   stack, and people increasingly set the "init all stack variables to
>   zero" kernel hardening option anyway.
> 
>   I'll also note that commit 88c02b3f79a6 has no performance data in it.
>   So it's not clear that it actually helped much.
> 
> - The library has an optimization to greatly reduce the size of the
>   context: instead of buffering data separately, it just XOR's data into
>   the state.  So, if there's a sha3_*_init() followed by a sha3_update()
>   of less than 1 block, it will have to initialize the state anyway.  We
>   can delay it until that point on s390.  But again: complexity.
> 
> - These potential additional s390 optimizations would presumably help
>   the most on short messages.  However, on short messages, merely
>   switching to the library often gives a large performance improvement
>   due to eliminating the very slow call to crypto_alloc_shash().  That's
>   actually a lot more important.
> 
> I would suggest that we drop the sha3_*_init() optimization from
> consideration for now.  Providing overrides for the one-shot functions
> sha3_{224,256,384,512}() should be simpler as well as possibly a bit
> more useful, and I would suggest exploring that.

Ok, sounds reasonable.

> I guess I can try to write the code for you again.  But again, without
> QEMU support I cannot test it.  The first priority in cryptography code
> is correctness, so that's not a great position to be in.
> 
> Note that for new optimized code I'm requiring QEMU support for the
> instructions it uses.  This one would only be allowed because code that
> used these instructions already existed in arch/s390/crypto/.

Ok, I'll come back to you on that.

>> Please also add me and Harald Freudenberger to the cc: list for this patch.
> 
> Will do, thanks.
> 
> - Eric

-- 
Mit freundlichen Grüßen / Kind regards
Holger Dengler
--
IBM Systems, Linux on IBM Z Development
dengler@linux.ibm.com

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Holger Dengler]

Hi Eric,

On 21/10/2025 09:24, Holger Dengler wrote:
> On 20/10/2025 19:57, Eric Biggers wrote:
[...]>> - Risk of bugs.  QEMU doesn't support the s390 SHA-3 instructions, so no
>>   one except the s390 folks can test the code.  I can try to write code
>>   for you, but I can't test it.  And the s390 SHA-3 code has had bugs;
>>   see commits 992b7066800f, 68279380266a5, 73c2437109c3.
>>
>>   The first priority should be correctness.
> 
> Let me figure out, if me and my colleagues can do the testing for you.
> Unfortunately, I'll be unavailable for the next two weeks. But I'll come back
> with a solution for the testing.

I talked to Harald: we can do the testing for you on our development machines.
Please send new series to us or provide them in your git repo.

-- 
Mit freundlichen Grüßen / Kind regards
Holger Dengler
--
IBM Systems, Linux on IBM Z Development
dengler@linux.ibm.com

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Eric Biggers]

On Tue, Oct 21, 2025 at 10:43:00AM +0200, Holger Dengler wrote:
> Hi Eric,
> 
> On 21/10/2025 09:24, Holger Dengler wrote:
> > On 20/10/2025 19:57, Eric Biggers wrote:
> [...]>> - Risk of bugs.  QEMU doesn't support the s390 SHA-3 instructions, so no
> >>   one except the s390 folks can test the code.  I can try to write code
> >>   for you, but I can't test it.  And the s390 SHA-3 code has had bugs;
> >>   see commits 992b7066800f, 68279380266a5, 73c2437109c3.
> >>
> >>   The first priority should be correctness.
> > 
> > Let me figure out, if me and my colleagues can do the testing for you.
> > Unfortunately, I'll be unavailable for the next two weeks. But I'll come back
> > with a solution for the testing.
> 
> I talked to Harald: we can do the testing for you on our development machines.
> Please send new series to us or provide them in your git repo.

Thanks!  I'll Cc both of you on v2 when I send it later.  For now, this
series (v1) can be found in lore at
https://lore.kernel.org/linux-crypto/20251020005038.661542-1-ebiggers@kernel.org/T/#u
And as mentioned in the cover letter it's also retrievable from git:

    git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha3-lib-v1

v1 already has the s390 optimized implementations of
sha3_absorb_blocks() and sha3_keccakf().  If you could enable the
following:

    CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST=y
    CONFIG_CRYPTO_LIB_BENCHMARK=y

... and then show the results for sha3_kunit before and after the commit
"lib/crypto: s390/sha3: Migrate optimized code into library", that would
be helpful.

In v2, I'll look into providing overrides for the one-shot functions
sha3_{224,256,384,512}() too.  If it works out, I'll ask you to re-test
with that additional change as well.

- Eric

Re: [PATCH 00/17] SHA-3 library

[Ard Biesheuvel]

On Mon, 20 Oct 2025 at 02:53, Eric Biggers <ebiggers@kernel.org> wrote:
>
> This series is targeting libcrypto-next.  It can also be retrieved from:
>
>     git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sha3-lib-v1
>
> This series builds on the first 5 patches of David's v6 series
> (https://lore.kernel.org/r/20251017144311.817771-1-dhowells@redhat.com),
> which I've included unmodified as the initial patches.  The remaining
> patches improve the SHA-3 library, for example by simplifying the API
> and migrating the existing arm64 and s390-optimized SHA-3 code into the
> library.  The last patch reimplements the sha3-* crypto_shash algorithms
> on top of the library.
>
> If the s390 folks could re-test the s390 optimized SHA-3 code, that
> would be helpful.  QEMU doesn't support the instructions it uses.
>
> David Howells (5):
>   s390/sha3: Rename conflicting functions
>   arm64/sha3: Rename conflicting functions
>   lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128,
>     SHAKE256
>   lib/crypto: Move the SHA3 Iota transform into the single round
>     function
>   lib/crypto: Add SHA3 kunit tests
>
> Eric Biggers (12):
>   lib/crypto: sha3: Fix libsha3 build condition
>   lib/crypto: sha3: Use appropriate conversions in
>     sha3_keccakf_generic()
>   lib/crypto: sha3: Drop unfinished SHAKE support from
>     gen-hash-testvecs.py
>   lib/crypto: sha3: Consistently use EXPORT_SYMBOL_GPL
>   lib/crypto: sha3: Replace redundant ad-hoc test with FIPS test
>   lib/crypto: sha3: Simplify the API
>   lib/crypto: sha3: Document one-shot functions in header and improve
>     docs
>   crypto: arm64/sha3 - Update sha3_ce_transform() to prepare for library
>   lib/crypto: arm64/sha3: Migrate optimized code into library
>   lib/crypto: s390/sha3: Migrate optimized code into library
>   crypto: jitterentropy - use default sha3 implementation
>   crypto: sha3 - Reimplement using library API
>

Reviewed-by: Ard Biesheuvel <ardb@kernel.org>

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Harald Freudenberger]

On 2025-10-21 17:49, Eric Biggers wrote:
> On Tue, Oct 21, 2025 at 10:43:00AM +0200, Holger Dengler wrote:
>> Hi Eric,
>> 
>> On 21/10/2025 09:24, Holger Dengler wrote:
>> > On 20/10/2025 19:57, Eric Biggers wrote:
>> [...]>> - Risk of bugs.  QEMU doesn't support the s390 SHA-3 
>> instructions, so no
>> >>   one except the s390 folks can test the code.  I can try to write code
>> >>   for you, but I can't test it.  And the s390 SHA-3 code has had bugs;
>> >>   see commits 992b7066800f, 68279380266a5, 73c2437109c3.
>> >>
>> >>   The first priority should be correctness.
>> >
>> > Let me figure out, if me and my colleagues can do the testing for you.
>> > Unfortunately, I'll be unavailable for the next two weeks. But I'll come back
>> > with a solution for the testing.
>> 
>> I talked to Harald: we can do the testing for you on our development 
>> machines.
>> Please send new series to us or provide them in your git repo.
> 
> Thanks!  I'll Cc both of you on v2 when I send it later.  For now, this
> series (v1) can be found in lore at
> https://lore.kernel.org/linux-crypto/20251020005038.661542-1-ebiggers@kernel.org/T/#u
> And as mentioned in the cover letter it's also retrievable from git:
> 
>     git fetch
> https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git
> sha3-lib-v1
> 
> v1 already has the s390 optimized implementations of
> sha3_absorb_blocks() and sha3_keccakf().  If you could enable the
> following:
> 
>     CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST=y
>     CONFIG_CRYPTO_LIB_BENCHMARK=y
> 
> ... and then show the results for sha3_kunit before and after the 
> commit
> "lib/crypto: s390/sha3: Migrate optimized code into library", that 
> would
> be helpful.
> 
> In v2, I'll look into providing overrides for the one-shot functions
> sha3_{224,256,384,512}() too.  If it works out, I'll ask you to re-test
> with that additional change as well.
> 
> - Eric

I pulled your repository and checked out the branch sha3-lib-v1 and
while the kernel build runs I get link errors:

ld: crypto/sha3.o: in function `crypto_sha3_512_digest':
/root/ebiggers-linux/crypto/sha3.c:80:(.text+0xaa): undefined reference 
to `sha3_512'
ld: crypto/sha3.o: in function `crypto_sha3_384_digest':
/root/ebiggers-linux/crypto/sha3.c:73:(.text+0xea): undefined reference 
to `sha3_384'
ld: crypto/sha3.o: in function `crypto_sha3_256_digest':
/root/ebiggers-linux/crypto/sha3.c:66:(.text+0x12a): undefined reference 
to `sha3_256'
ld: crypto/sha3.o: in function `crypto_sha3_224_digest':
/root/ebiggers-linux/crypto/sha3.c:59:(.text+0x1aa): undefined reference 
to `sha3_224'
ld: crypto/sha3.o: in function `sha3_final':
/root/ebiggers-linux/./include/crypto/sha3.h:188:(.text+0x1f0): 
undefined reference to `__sha3_squeeze'
ld: crypto/sha3.o: in function `sha3_update':
/root/ebiggers-linux/./include/crypto/sha3.h:172:(.text+0x232): 
undefined reference to `__sha3_update'

with a s390 defconfig kernel configuration an a s390 debug_defconfig 
kernel configuration.

Re: [PATCH 15/17] lib/crypto: s390/sha3: Migrate optimized code into library

[Eric Biggers]

On Fri, Oct 24, 2025 at 04:24:00PM +0200, Harald Freudenberger wrote:
> On 2025-10-21 17:49, Eric Biggers wrote:
> > On Tue, Oct 21, 2025 at 10:43:00AM +0200, Holger Dengler wrote:
> > > Hi Eric,
> > > 
> > > On 21/10/2025 09:24, Holger Dengler wrote:
> > > > On 20/10/2025 19:57, Eric Biggers wrote:
> > > [...]>> - Risk of bugs.  QEMU doesn't support the s390 SHA-3
> > > instructions, so no
> > > >>   one except the s390 folks can test the code.  I can try to write code
> > > >>   for you, but I can't test it.  And the s390 SHA-3 code has had bugs;
> > > >>   see commits 992b7066800f, 68279380266a5, 73c2437109c3.
> > > >>
> > > >>   The first priority should be correctness.
> > > >
> > > > Let me figure out, if me and my colleagues can do the testing for you.
> > > > Unfortunately, I'll be unavailable for the next two weeks. But I'll come back
> > > > with a solution for the testing.
> > > 
> > > I talked to Harald: we can do the testing for you on our development
> > > machines.
> > > Please send new series to us or provide them in your git repo.
> > 
> > Thanks!  I'll Cc both of you on v2 when I send it later.  For now, this
> > series (v1) can be found in lore at
> > https://lore.kernel.org/linux-crypto/20251020005038.661542-1-ebiggers@kernel.org/T/#u
> > And as mentioned in the cover letter it's also retrievable from git:
> > 
> >     git fetch
> > https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git
> > sha3-lib-v1
> > 
> > v1 already has the s390 optimized implementations of
> > sha3_absorb_blocks() and sha3_keccakf().  If you could enable the
> > following:
> > 
> >     CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST=y
> >     CONFIG_CRYPTO_LIB_BENCHMARK=y
> > 
> > ... and then show the results for sha3_kunit before and after the commit
> > "lib/crypto: s390/sha3: Migrate optimized code into library", that would
> > be helpful.
> > 
> > In v2, I'll look into providing overrides for the one-shot functions
> > sha3_{224,256,384,512}() too.  If it works out, I'll ask you to re-test
> > with that additional change as well.
> > 
> > - Eric
> 
> I pulled your repository and checked out the branch sha3-lib-v1 and
> while the kernel build runs I get link errors:
> 
> ld: crypto/sha3.o: in function `crypto_sha3_512_digest':
> /root/ebiggers-linux/crypto/sha3.c:80:(.text+0xaa): undefined reference to
> `sha3_512'
> ld: crypto/sha3.o: in function `crypto_sha3_384_digest':
> /root/ebiggers-linux/crypto/sha3.c:73:(.text+0xea): undefined reference to
> `sha3_384'
> ld: crypto/sha3.o: in function `crypto_sha3_256_digest':
> /root/ebiggers-linux/crypto/sha3.c:66:(.text+0x12a): undefined reference to
> `sha3_256'
> ld: crypto/sha3.o: in function `crypto_sha3_224_digest':
> /root/ebiggers-linux/crypto/sha3.c:59:(.text+0x1aa): undefined reference to
> `sha3_224'
> ld: crypto/sha3.o: in function `sha3_final':
> /root/ebiggers-linux/./include/crypto/sha3.h:188:(.text+0x1f0): undefined
> reference to `__sha3_squeeze'
> ld: crypto/sha3.o: in function `sha3_update':
> /root/ebiggers-linux/./include/crypto/sha3.h:172:(.text+0x232): undefined
> reference to `__sha3_update'
> 
> with a s390 defconfig kernel configuration an a s390 debug_defconfig kernel
> configuration.

Yes, as mentioned elsewhere in the thread the following fixup is needed:

diff --git a/crypto/Kconfig b/crypto/Kconfig
index a04595f9d0ca4..0ff68212cb20a 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -1005,6 +1005,7 @@ config CRYPTO_SHA512
 config CRYPTO_SHA3
 	tristate "SHA-3"
 	select CRYPTO_HASH
+	select CRYPTO_LIB_SHA3
 	help
 	  SHA-3 secure hash algorithms (FIPS 202, ISO/IEC 10118-3)
 
I'll fix that in v2.

But also note that if you enable the KUnit test as I suggested then this
error gets avoided too, since it selects the library.  (That's why I
didn't notice it before sending -- I always had the KUnit test enabled.)

- Eric