* [PATCH 00/12] SM3 library
@ 2026-03-21 4:09 Eric Biggers
2026-03-21 4:09 ` [PATCH 01/12] crypto: sm3 - Fold sm3_init() into its caller Eric Biggers
` (13 more replies)
0 siblings, 14 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
This series is targeting libcrypto-next. It can also be retrieved from:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sm3-lib-v1
This series cleans up the kernel's existing SM3 hashing code:
- First, it updates lib/crypto/sm3.c to implement the full SM3 instead
of just SM3's compression function.
- Next, it adds a KUnit test suite for the new library API.
- Next, it replaces the "sm3-generic" crypto_shash with a wrapper around
the new library API.
- Finally, it accelerates the API using the existing SM3 assembly code
for arm64, riscv, and x86. The architecture-specific crypto_shash
glue code for SM3 is no longer needed and is removed.
This should look quite boring. It's the same cleanup that I've already
done for the other hash functions.
Note: I don't recommend using SM3. There also don't appear to be any
immediate candidate users of the SM3 library other than crypto_shash.
Still, this seems like the clear way to go. It's simpler, and it gets
the hash algorithms integrated in a consistent way. We won't have to
keep track of two quite different ways of doing things. With KUnit the
code becomes much easier to test and benchmark, as well.
Eric Biggers (12):
crypto: sm3 - Fold sm3_init() into its caller
crypto: sm3 - Remove sm3_zero_message_hash and SM3_T[1-2]
crypto: sm3 - Rename CRYPTO_SM3_GENERIC to CRYPTO_SM3
lib/crypto: sm3: Add SM3 library API
lib/crypto: tests: Add KUnit tests for SM3
crypto: sm3 - Replace with wrapper around library
lib/crypto: arm64/sm3: Migrate optimized code into library
lib/crypto: riscv/sm3: Migrate optimized code into library
lib/crypto: x86/sm3: Migrate optimized code into library
crypto: sm3 - Remove sm3_base.h
crypto: sm3 - Remove the original "sm3_block_generic()"
crypto: sm3 - Remove 'struct sm3_state'
arch/arm64/configs/defconfig | 2 +-
arch/arm64/crypto/Kconfig | 22 --
arch/arm64/crypto/Makefile | 6 -
arch/arm64/crypto/sm3-ce-glue.c | 70 ------
arch/arm64/crypto/sm3-neon-glue.c | 67 -----
arch/loongarch/configs/loongson32_defconfig | 2 +-
arch/loongarch/configs/loongson64_defconfig | 2 +-
arch/m68k/configs/amiga_defconfig | 2 +-
arch/m68k/configs/apollo_defconfig | 2 +-
arch/m68k/configs/atari_defconfig | 2 +-
arch/m68k/configs/bvme6000_defconfig | 2 +-
arch/m68k/configs/hp300_defconfig | 2 +-
arch/m68k/configs/mac_defconfig | 2 +-
arch/m68k/configs/multi_defconfig | 2 +-
arch/m68k/configs/mvme147_defconfig | 2 +-
arch/m68k/configs/mvme16x_defconfig | 2 +-
arch/m68k/configs/q40_defconfig | 2 +-
arch/m68k/configs/sun3_defconfig | 2 +-
arch/m68k/configs/sun3x_defconfig | 2 +-
arch/riscv/crypto/Kconfig | 13 -
arch/riscv/crypto/Makefile | 3 -
arch/s390/configs/debug_defconfig | 2 +-
arch/s390/configs/defconfig | 2 +-
arch/x86/crypto/Kconfig | 13 -
arch/x86/crypto/Makefile | 3 -
arch/x86/crypto/sm3_avx_glue.c | 100 --------
crypto/Kconfig | 2 +-
crypto/Makefile | 2 +-
crypto/{sm3_generic.c => sm3.c} | 89 ++++---
crypto/testmgr.c | 2 +
drivers/crypto/Kconfig | 2 +-
drivers/crypto/starfive/Kconfig | 2 +-
drivers/crypto/starfive/jh7110-hash.c | 8 +-
include/crypto/sm3.h | 85 ++++---
include/crypto/sm3_base.h | 82 -------
lib/crypto/.kunitconfig | 1 +
lib/crypto/Kconfig | 11 +
lib/crypto/Makefile | 15 +-
.../crypto => lib/crypto/arm64}/sm3-ce-core.S | 11 +-
.../crypto/arm64}/sm3-neon-core.S | 9 +-
lib/crypto/arm64/sm3.h | 41 ++++
.../crypto/riscv}/sm3-riscv64-zvksh-zvkb.S | 3 +-
.../crypto/riscv/sm3.h | 84 +------
lib/crypto/sm3.c | 148 +++++++++--
lib/crypto/tests/Kconfig | 9 +
lib/crypto/tests/Makefile | 1 +
lib/crypto/tests/sm3-testvecs.h | 231 ++++++++++++++++++
lib/crypto/tests/sm3_kunit.c | 31 +++
.../crypto/x86}/sm3-avx-asm_64.S | 13 +-
lib/crypto/x86/sm3.h | 39 +++
scripts/crypto/gen-hash-testvecs.py | 3 +
security/integrity/ima/Kconfig | 2 +-
52 files changed, 670 insertions(+), 587 deletions(-)
delete mode 100644 arch/arm64/crypto/sm3-ce-glue.c
delete mode 100644 arch/arm64/crypto/sm3-neon-glue.c
delete mode 100644 arch/x86/crypto/sm3_avx_glue.c
rename crypto/{sm3_generic.c => sm3.c} (30%)
delete mode 100644 include/crypto/sm3_base.h
rename {arch/arm64/crypto => lib/crypto/arm64}/sm3-ce-core.S (93%)
rename {arch/arm64/crypto => lib/crypto/arm64}/sm3-neon-core.S (98%)
create mode 100644 lib/crypto/arm64/sm3.h
rename {arch/riscv/crypto => lib/crypto/riscv}/sm3-riscv64-zvksh-zvkb.S (97%)
rename arch/riscv/crypto/sm3-riscv64-glue.c => lib/crypto/riscv/sm3.h (18%)
create mode 100644 lib/crypto/tests/sm3-testvecs.h
create mode 100644 lib/crypto/tests/sm3_kunit.c
rename {arch/x86/crypto => lib/crypto/x86}/sm3-avx-asm_64.S (98%)
create mode 100644 lib/crypto/x86/sm3.h
base-commit: 6bc9effb4cbf9b6eba0f51aba1c8893dfd4c8100
--
2.53.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 01/12] crypto: sm3 - Fold sm3_init() into its caller
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 02/12] crypto: sm3 - Remove sm3_zero_message_hash and SM3_T[1-2] Eric Biggers
` (12 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Fold sm3_init() into its caller to free up the name for use in a library
API mirroring the other hash function APIs.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
include/crypto/sm3.h | 13 -------------
include/crypto/sm3_base.h | 12 +++++++++++-
2 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/include/crypto/sm3.h b/include/crypto/sm3.h
index c8d02c86c298..c09f6bf4c0bf 100644
--- a/include/crypto/sm3.h
+++ b/include/crypto/sm3.h
@@ -44,21 +44,8 @@ struct sm3_state {
* amounts of data as those APIs may be hw-accelerated.
*
* For details see lib/crypto/sm3.c
*/
-static inline void sm3_init(struct sm3_state *sctx)
-{
- sctx->state[0] = SM3_IVA;
- sctx->state[1] = SM3_IVB;
- sctx->state[2] = SM3_IVC;
- sctx->state[3] = SM3_IVD;
- sctx->state[4] = SM3_IVE;
- sctx->state[5] = SM3_IVF;
- sctx->state[6] = SM3_IVG;
- sctx->state[7] = SM3_IVH;
- sctx->count = 0;
-}
-
void sm3_block_generic(struct sm3_state *sctx, u8 const *data, int blocks);
#endif
diff --git a/include/crypto/sm3_base.h b/include/crypto/sm3_base.h
index 7c53570bc05e..9fa995617495 100644
--- a/include/crypto/sm3_base.h
+++ b/include/crypto/sm3_base.h
@@ -19,11 +19,21 @@
typedef void (sm3_block_fn)(struct sm3_state *sst, u8 const *src, int blocks);
static inline int sm3_base_init(struct shash_desc *desc)
{
- sm3_init(shash_desc_ctx(desc));
+ struct sm3_state *sctx = shash_desc_ctx(desc);
+
+ sctx->state[0] = SM3_IVA;
+ sctx->state[1] = SM3_IVB;
+ sctx->state[2] = SM3_IVC;
+ sctx->state[3] = SM3_IVD;
+ sctx->state[4] = SM3_IVE;
+ sctx->state[5] = SM3_IVF;
+ sctx->state[6] = SM3_IVG;
+ sctx->state[7] = SM3_IVH;
+ sctx->count = 0;
return 0;
}
static inline int sm3_base_do_update_blocks(struct shash_desc *desc,
const u8 *data, unsigned int len,
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 02/12] crypto: sm3 - Remove sm3_zero_message_hash and SM3_T[1-2]
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
2026-03-21 4:09 ` [PATCH 01/12] crypto: sm3 - Fold sm3_init() into its caller Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 03/12] crypto: sm3 - Rename CRYPTO_SM3_GENERIC to CRYPTO_SM3 Eric Biggers
` (11 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Remove these, since they are unused.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
crypto/sm3_generic.c | 8 --------
include/crypto/sm3.h | 5 -----
2 files changed, 13 deletions(-)
diff --git a/crypto/sm3_generic.c b/crypto/sm3_generic.c
index 7529139fcc96..0c606f526347 100644
--- a/crypto/sm3_generic.c
+++ b/crypto/sm3_generic.c
@@ -12,18 +12,10 @@
#include <crypto/sm3.h>
#include <crypto/sm3_base.h>
#include <linux/kernel.h>
#include <linux/module.h>
-const u8 sm3_zero_message_hash[SM3_DIGEST_SIZE] = {
- 0x1A, 0xB2, 0x1D, 0x83, 0x55, 0xCF, 0xA1, 0x7F,
- 0x8e, 0x61, 0x19, 0x48, 0x31, 0xE8, 0x1A, 0x8F,
- 0x22, 0xBE, 0xC8, 0xC7, 0x28, 0xFE, 0xFB, 0x74,
- 0x7E, 0xD0, 0x35, 0xEB, 0x50, 0x82, 0xAA, 0x2B
-};
-EXPORT_SYMBOL_GPL(sm3_zero_message_hash);
-
static int crypto_sm3_update(struct shash_desc *desc, const u8 *data,
unsigned int len)
{
return sm3_base_do_update_blocks(desc, data, len, sm3_block_generic);
}
diff --git a/include/crypto/sm3.h b/include/crypto/sm3.h
index c09f6bf4c0bf..918d318795ef 100644
--- a/include/crypto/sm3.h
+++ b/include/crypto/sm3.h
@@ -14,24 +14,19 @@
#define SM3_DIGEST_SIZE 32
#define SM3_BLOCK_SIZE 64
#define SM3_STATE_SIZE 40
-#define SM3_T1 0x79CC4519
-#define SM3_T2 0x7A879D8A
-
#define SM3_IVA 0x7380166f
#define SM3_IVB 0x4914b2b9
#define SM3_IVC 0x172442d7
#define SM3_IVD 0xda8a0600
#define SM3_IVE 0xa96f30bc
#define SM3_IVF 0x163138aa
#define SM3_IVG 0xe38dee4d
#define SM3_IVH 0xb0fb0e4e
-extern const u8 sm3_zero_message_hash[SM3_DIGEST_SIZE];
-
struct sm3_state {
u32 state[SM3_DIGEST_SIZE / 4];
u64 count;
u8 buffer[SM3_BLOCK_SIZE];
};
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 03/12] crypto: sm3 - Rename CRYPTO_SM3_GENERIC to CRYPTO_SM3
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
2026-03-21 4:09 ` [PATCH 01/12] crypto: sm3 - Fold sm3_init() into its caller Eric Biggers
2026-03-21 4:09 ` [PATCH 02/12] crypto: sm3 - Remove sm3_zero_message_hash and SM3_T[1-2] Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 04/12] lib/crypto: sm3: Add SM3 library API Eric Biggers
` (10 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
The kconfig options for generic crypto API modules have traditionally
*not* had a "_GENERIC" suffix. Also, the "_GENERIC" suffix will make
even less sense once the architecture-optimized SM3 code is moved into
lib/crypto/ and the "sm3" crypto_shash is reimplemented on top of that.
Thus, rename CRYPTO_SM3_GENERIC to CRYPTO_SM3.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/loongarch/configs/loongson32_defconfig | 2 +-
arch/loongarch/configs/loongson64_defconfig | 2 +-
arch/m68k/configs/amiga_defconfig | 2 +-
arch/m68k/configs/apollo_defconfig | 2 +-
arch/m68k/configs/atari_defconfig | 2 +-
arch/m68k/configs/bvme6000_defconfig | 2 +-
arch/m68k/configs/hp300_defconfig | 2 +-
arch/m68k/configs/mac_defconfig | 2 +-
arch/m68k/configs/multi_defconfig | 2 +-
arch/m68k/configs/mvme147_defconfig | 2 +-
arch/m68k/configs/mvme16x_defconfig | 2 +-
arch/m68k/configs/q40_defconfig | 2 +-
arch/m68k/configs/sun3_defconfig | 2 +-
arch/m68k/configs/sun3x_defconfig | 2 +-
arch/s390/configs/debug_defconfig | 2 +-
arch/s390/configs/defconfig | 2 +-
crypto/Kconfig | 2 +-
crypto/Makefile | 2 +-
drivers/crypto/Kconfig | 2 +-
drivers/crypto/starfive/Kconfig | 2 +-
security/integrity/ima/Kconfig | 2 +-
21 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/arch/loongarch/configs/loongson32_defconfig b/arch/loongarch/configs/loongson32_defconfig
index 276b1577e0be..7abbb21f4f8f 100644
--- a/arch/loongarch/configs/loongson32_defconfig
+++ b/arch/loongarch/configs/loongson32_defconfig
@@ -1078,11 +1078,11 @@ CONFIG_CRYPTO_SEED=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_SM4_GENERIC=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_CHACHA20POLY1305=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/loongarch/configs/loongson64_defconfig b/arch/loongarch/configs/loongson64_defconfig
index a14db1a95e7e..51ccd18ecdae 100644
--- a/arch/loongarch/configs/loongson64_defconfig
+++ b/arch/loongarch/configs/loongson64_defconfig
@@ -1111,11 +1111,11 @@ CONFIG_CRYPTO_SEED=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_SM4_GENERIC=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_CHACHA20POLY1305=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/amiga_defconfig b/arch/m68k/configs/amiga_defconfig
index 31d16cba9879..03a8c192a7a3 100644
--- a/arch/m68k/configs/amiga_defconfig
+++ b/arch/m68k/configs/amiga_defconfig
@@ -579,11 +579,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/apollo_defconfig b/arch/m68k/configs/apollo_defconfig
index c0c419ec9a9e..0aee1939ac7a 100644
--- a/arch/m68k/configs/apollo_defconfig
+++ b/arch/m68k/configs/apollo_defconfig
@@ -536,11 +536,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/atari_defconfig b/arch/m68k/configs/atari_defconfig
index 2b7547ecc4c4..756256770afc 100644
--- a/arch/m68k/configs/atari_defconfig
+++ b/arch/m68k/configs/atari_defconfig
@@ -556,11 +556,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/bvme6000_defconfig b/arch/m68k/configs/bvme6000_defconfig
index 0b63787cff0d..8cfb75bb0add 100644
--- a/arch/m68k/configs/bvme6000_defconfig
+++ b/arch/m68k/configs/bvme6000_defconfig
@@ -528,11 +528,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/hp300_defconfig b/arch/m68k/configs/hp300_defconfig
index 308836b60bba..b2f5c9749e9b 100644
--- a/arch/m68k/configs/hp300_defconfig
+++ b/arch/m68k/configs/hp300_defconfig
@@ -538,11 +538,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/mac_defconfig b/arch/m68k/configs/mac_defconfig
index 97e108c0d24f..c4fddaaa6a86 100644
--- a/arch/m68k/configs/mac_defconfig
+++ b/arch/m68k/configs/mac_defconfig
@@ -555,11 +555,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/multi_defconfig b/arch/m68k/configs/multi_defconfig
index 7e9f83af9af4..926f12bc3d1d 100644
--- a/arch/m68k/configs/multi_defconfig
+++ b/arch/m68k/configs/multi_defconfig
@@ -642,11 +642,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/mvme147_defconfig b/arch/m68k/configs/mvme147_defconfig
index 2fe33271d249..e507012dbbc1 100644
--- a/arch/m68k/configs/mvme147_defconfig
+++ b/arch/m68k/configs/mvme147_defconfig
@@ -528,11 +528,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/mvme16x_defconfig b/arch/m68k/configs/mvme16x_defconfig
index 4308daaa7f74..6195cedd914b 100644
--- a/arch/m68k/configs/mvme16x_defconfig
+++ b/arch/m68k/configs/mvme16x_defconfig
@@ -529,11 +529,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/q40_defconfig b/arch/m68k/configs/q40_defconfig
index 36eb29ec54ee..9087bd9e3c35 100644
--- a/arch/m68k/configs/q40_defconfig
+++ b/arch/m68k/configs/q40_defconfig
@@ -545,11 +545,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/sun3_defconfig b/arch/m68k/configs/sun3_defconfig
index 524a89fa6953..25115bda7c8a 100644
--- a/arch/m68k/configs/sun3_defconfig
+++ b/arch/m68k/configs/sun3_defconfig
@@ -526,11 +526,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/m68k/configs/sun3x_defconfig b/arch/m68k/configs/sun3x_defconfig
index f4fbc65c52d9..15a086634ba5 100644
--- a/arch/m68k/configs/sun3x_defconfig
+++ b/arch/m68k/configs/sun3x_defconfig
@@ -526,11 +526,11 @@ CONFIG_CRYPTO_XTS=m
CONFIG_CRYPTO_AEGIS128=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA1=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/s390/configs/debug_defconfig b/arch/s390/configs/debug_defconfig
index 98fd0a2f51c6..271d683e7959 100644
--- a/arch/s390/configs/debug_defconfig
+++ b/arch/s390/configs/debug_defconfig
@@ -795,11 +795,11 @@ CONFIG_CRYPTO_SEQIV=y
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA3=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/arch/s390/configs/defconfig b/arch/s390/configs/defconfig
index 0f4cedcab3ce..e9b64c0d4bcc 100644
--- a/arch/s390/configs/defconfig
+++ b/arch/s390/configs/defconfig
@@ -779,11 +779,11 @@ CONFIG_CRYPTO_SEQIV=y
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_SHA3=m
-CONFIG_CRYPTO_SM3_GENERIC=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_CRC32=m
CONFIG_CRYPTO_842=m
CONFIG_CRYPTO_LZ4=m
diff --git a/crypto/Kconfig b/crypto/Kconfig
index b8608ef6823b..79234fd42eb4 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -979,11 +979,11 @@ config CRYPTO_SHA3
select CRYPTO_HASH
select CRYPTO_LIB_SHA3
help
SHA-3 secure hash algorithms (FIPS 202, ISO/IEC 10118-3)
-config CRYPTO_SM3_GENERIC
+config CRYPTO_SM3
tristate "SM3 (ShangMi 3)"
select CRYPTO_HASH
select CRYPTO_LIB_SM3
help
SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012, ISO/IEC 10118-3)
diff --git a/crypto/Makefile b/crypto/Makefile
index 04e269117589..28467f900c9a 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -81,11 +81,11 @@ obj-$(CONFIG_CRYPTO_MD5) += md5.o
obj-$(CONFIG_CRYPTO_RMD160) += rmd160.o
obj-$(CONFIG_CRYPTO_SHA1) += sha1.o
obj-$(CONFIG_CRYPTO_SHA256) += sha256.o
obj-$(CONFIG_CRYPTO_SHA512) += sha512.o
obj-$(CONFIG_CRYPTO_SHA3) += sha3.o
-obj-$(CONFIG_CRYPTO_SM3_GENERIC) += sm3_generic.o
+obj-$(CONFIG_CRYPTO_SM3) += sm3_generic.o
obj-$(CONFIG_CRYPTO_STREEBOG) += streebog_generic.o
obj-$(CONFIG_CRYPTO_WP512) += wp512.o
CFLAGS_wp512.o := $(call cc-option,-fno-schedule-insns) # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79149
obj-$(CONFIG_CRYPTO_BLAKE2B) += blake2b.o
obj-$(CONFIG_CRYPTO_ECB) += ecb.o
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 8d3b5d2890f8..9960100e6066 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -841,11 +841,11 @@ config CRYPTO_DEV_CCREE
select CRYPTO_CBC
select CRYPTO_ECB
select CRYPTO_CTR
select CRYPTO_XTS
select CRYPTO_SM4_GENERIC
- select CRYPTO_SM3_GENERIC
+ select CRYPTO_SM3
help
Say 'Y' to enable a driver for the REE interface of the Arm
TrustZone CryptoCell family of processors. Currently the
CryptoCell 713, 703, 712, 710 and 630 are supported.
Choose this if you wish to use hardware acceleration of
diff --git a/drivers/crypto/starfive/Kconfig b/drivers/crypto/starfive/Kconfig
index 0fe389e9f932..11518ca3eea1 100644
--- a/drivers/crypto/starfive/Kconfig
+++ b/drivers/crypto/starfive/Kconfig
@@ -8,11 +8,11 @@ config CRYPTO_DEV_JH7110
depends on HAS_DMA
select CRYPTO_ENGINE
select CRYPTO_HMAC
select CRYPTO_SHA256
select CRYPTO_SHA512
- select CRYPTO_SM3_GENERIC
+ select CRYPTO_SM3
select CRYPTO_RSA
select CRYPTO_AES
select CRYPTO_CCM
select CRYPTO_GCM
select CRYPTO_ECB
diff --git a/security/integrity/ima/Kconfig b/security/integrity/ima/Kconfig
index 976e75f9b9ba..862fbee2b174 100644
--- a/security/integrity/ima/Kconfig
+++ b/security/integrity/ima/Kconfig
@@ -109,11 +109,11 @@ choice
bool "WP512"
depends on CRYPTO_WP512=y
config IMA_DEFAULT_HASH_SM3
bool "SM3"
- depends on CRYPTO_SM3_GENERIC=y
+ depends on CRYPTO_SM3=y
endchoice
config IMA_DEFAULT_HASH
string
default "sha1" if IMA_DEFAULT_HASH_SHA1
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 04/12] lib/crypto: sm3: Add SM3 library API
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (2 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 03/12] crypto: sm3 - Rename CRYPTO_SM3_GENERIC to CRYPTO_SM3 Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 05/12] lib/crypto: tests: Add KUnit tests for SM3 Eric Biggers
` (9 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Add a straightforward library API for SM3, mirroring the ones for the
other hash algorithms. It uses the existing generic implementation of
SM3's compression function in lib/crypto/sm3.c. Hooks are added for
architecture-optimized implementations, which later commits will wire up
to the existing optimized SM3 code for arm64, riscv, and x86.
Note that the rationale for this is *not* that SM3 should be used, or
that any kernel subsystem currently seems like a candidate for switching
from the sm3 crypto_shash to SM3 library. (SM3, in fact, shouldn't be
used. Likewise you shouldn't use MD5, SHA-1, RC4, etc...)
Rather, it's just that this will simplify how the kernel's existing SM3
code is integrated and make it much easier to maintain and test. SM3 is
one of the only hash algorithms with arch-optimized code that is still
integrated in the old way. By converting it to the new lib/crypto/ code
organization, we'll only have to keep track of one way of doing things.
The library will also get a KUnit test suite (as usual for lib/crypto/),
so it will become more easily and comprehensively tested as well.
Skip adding functions for HMAC-SM3 for now, though. There's not as much
point in adding those right now.
Note: similar to the other hash algorithms, the library API uses
'struct sm3_ctx', not 'struct sm3_state'. The existing 'struct
sm3_state' and the sm3_block_generic() function which uses it are
temporarily kept around until their users are updated by later commits.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
include/crypto/sm3.h | 70 ++++++++++++++++---
lib/crypto/Kconfig | 7 ++
lib/crypto/sm3.c | 155 +++++++++++++++++++++++++++++++++++++------
3 files changed, 203 insertions(+), 29 deletions(-)
diff --git a/include/crypto/sm3.h b/include/crypto/sm3.h
index 918d318795ef..702c5326b4be 100644
--- a/include/crypto/sm3.h
+++ b/include/crypto/sm3.h
@@ -1,8 +1,8 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
- * Common values for SM3 algorithm
+ * SM3 hash algorithm
*
* Copyright (C) 2017 ARM Limited or its affiliates.
* Copyright (C) 2017 Gilad Ben-Yossef <gilad@benyossef.com>
* Copyright (C) 2021 Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
*/
@@ -29,18 +29,68 @@ struct sm3_state {
u32 state[SM3_DIGEST_SIZE / 4];
u64 count;
u8 buffer[SM3_BLOCK_SIZE];
};
-/*
- * Stand-alone implementation of the SM3 algorithm. It is designed to
- * have as little dependencies as possible so it can be used in the
- * kexec_file purgatory. In other cases you should generally use the
- * hash APIs from include/crypto/hash.h. Especially when hashing large
- * amounts of data as those APIs may be hw-accelerated.
+void sm3_block_generic(struct sm3_state *sctx, u8 const *data, int blocks);
+
+/* State for the SM3 compression function */
+struct sm3_block_state {
+ u32 h[SM3_DIGEST_SIZE / 4];
+};
+
+/**
+ * struct sm3_ctx - Context for hashing a message with SM3
+ * @state: the compression function state
+ * @bytecount: number of bytes processed so far
+ * @buf: partial block buffer; bytecount % SM3_BLOCK_SIZE bytes are valid
+ */
+struct sm3_ctx {
+ struct sm3_block_state state;
+ u64 bytecount;
+ u8 buf[SM3_BLOCK_SIZE] __aligned(__alignof__(__be64));
+};
+
+/**
+ * sm3_init() - Initialize an SM3 context for a new message
+ * @ctx: the context to initialize
*
- * For details see lib/crypto/sm3.c
+ * If you don't need incremental computation, consider sm3() instead.
+ *
+ * Context: Any context.
*/
+void sm3_init(struct sm3_ctx *ctx);
-void sm3_block_generic(struct sm3_state *sctx, u8 const *data, int blocks);
+/**
+ * sm3_update() - Update an SM3 context with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
+ *
+ * This can be called any number of times.
+ *
+ * Context: Any context.
+ */
+void sm3_update(struct sm3_ctx *ctx, const u8 *data, size_t len);
+
+/**
+ * sm3_final() - Finish computing an SM3 message digest
+ * @ctx: the context to finalize; must have been initialized
+ * @out: (output) the resulting SM3 message digest
+ *
+ * After finishing, this zeroizes @ctx. So the caller does not need to do it.
+ *
+ * Context: Any context.
+ */
+void sm3_final(struct sm3_ctx *ctx, u8 out[at_least SM3_DIGEST_SIZE]);
+
+/**
+ * sm3() - Compute SM3 message digest in one shot
+ * @data: the message data
+ * @len: the data length in bytes
+ * @out: (output) the resulting SM3 message digest
+ *
+ * Context: Any context.
+ */
+void sm3(const u8 *data, size_t len, u8 out[at_least SM3_DIGEST_SIZE]);
-#endif
+#endif /* _CRYPTO_SM3_H */
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index 4910fe20e42a..c5819e2518f6 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -265,9 +265,16 @@ config CRYPTO_LIB_SHA3_ARCH
default y if ARM64
default y if S390
config CRYPTO_LIB_SM3
tristate
+ help
+ The SM3 library functions. Select this if your module uses any of the
+ functions from <crypto/sm3.h>.
+
+config CRYPTO_LIB_SM3_ARCH
+ bool
+ depends on CRYPTO_LIB_SM3 && !UML
source "lib/crypto/tests/Kconfig"
endmenu
diff --git a/lib/crypto/sm3.c b/lib/crypto/sm3.c
index c6b9ad8a3ac6..20500cf4b8c0 100644
--- a/lib/crypto/sm3.c
+++ b/lib/crypto/sm3.c
@@ -13,10 +13,17 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/string.h>
#include <linux/unaligned.h>
+static const struct sm3_block_state sm3_iv = {
+ .h = {
+ SM3_IVA, SM3_IVB, SM3_IVC, SM3_IVD,
+ SM3_IVE, SM3_IVF, SM3_IVG, SM3_IVH,
+ },
+};
+
static const u32 ____cacheline_aligned K[64] = {
0x79cc4519, 0xf3988a32, 0xe7311465, 0xce6228cb,
0x9cc45197, 0x3988a32f, 0x7311465e, 0xe6228cbc,
0xcc451979, 0x988a32f3, 0x311465e7, 0x6228cbce,
0xc451979c, 0x88a32f39, 0x11465e73, 0x228cbce6,
@@ -70,22 +77,23 @@ static const u32 ____cacheline_aligned K[64] = {
^ W[(i-9) & 0x0f] \
^ rol32(W[(i-3) & 0x0f], 15)) \
^ rol32(W[(i-13) & 0x0f], 7) \
^ W[(i-6) & 0x0f])
-static void sm3_transform(struct sm3_state *sctx, u8 const *data, u32 W[16])
+static void sm3_transform(struct sm3_block_state *state,
+ const u8 data[SM3_BLOCK_SIZE], u32 W[16])
{
u32 a, b, c, d, e, f, g, h, ss1, ss2;
- a = sctx->state[0];
- b = sctx->state[1];
- c = sctx->state[2];
- d = sctx->state[3];
- e = sctx->state[4];
- f = sctx->state[5];
- g = sctx->state[6];
- h = sctx->state[7];
+ a = state->h[0];
+ b = state->h[1];
+ c = state->h[2];
+ d = state->h[3];
+ e = state->h[4];
+ f = state->h[5];
+ g = state->h[6];
+ h = state->h[7];
R1(a, b, c, d, e, f, g, h, K[0], I(0), I(4));
R1(d, a, b, c, h, e, f, g, K[1], I(1), I(5));
R1(c, d, a, b, g, h, e, f, K[2], I(2), I(6));
R1(b, c, d, a, f, g, h, e, K[3], I(3), I(7));
@@ -151,18 +159,18 @@ static void sm3_transform(struct sm3_state *sctx, u8 const *data, u32 W[16])
R2(a, b, c, d, e, f, g, h, K[60], W1(60), W2(64));
R2(d, a, b, c, h, e, f, g, K[61], W1(61), W2(65));
R2(c, d, a, b, g, h, e, f, K[62], W1(62), W2(66));
R2(b, c, d, a, f, g, h, e, K[63], W1(63), W2(67));
- sctx->state[0] ^= a;
- sctx->state[1] ^= b;
- sctx->state[2] ^= c;
- sctx->state[3] ^= d;
- sctx->state[4] ^= e;
- sctx->state[5] ^= f;
- sctx->state[6] ^= g;
- sctx->state[7] ^= h;
+ state->h[0] ^= a;
+ state->h[1] ^= b;
+ state->h[2] ^= c;
+ state->h[3] ^= d;
+ state->h[4] ^= e;
+ state->h[5] ^= f;
+ state->h[6] ^= g;
+ state->h[7] ^= h;
}
#undef R
#undef R1
#undef R2
#undef I
@@ -172,15 +180,124 @@ static void sm3_transform(struct sm3_state *sctx, u8 const *data, u32 W[16])
void sm3_block_generic(struct sm3_state *sctx, u8 const *data, int blocks)
{
u32 W[16];
do {
- sm3_transform(sctx, data, W);
+ sm3_transform((struct sm3_block_state *)sctx->state, data, W);
data += SM3_BLOCK_SIZE;
} while (--blocks);
memzero_explicit(W, sizeof(W));
}
EXPORT_SYMBOL_GPL(sm3_block_generic);
-MODULE_DESCRIPTION("Generic SM3 library");
+static void __maybe_unused sm3_blocks_generic(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks)
+{
+ u32 W[16];
+
+ do {
+ sm3_transform(state, data, W);
+ data += SM3_BLOCK_SIZE;
+ } while (--nblocks);
+
+ memzero_explicit(W, sizeof(W));
+}
+
+#ifdef CONFIG_CRYPTO_LIB_SM3_ARCH
+#include "sm3.h" /* $(SRCARCH)/sm3.h */
+#else
+#define sm3_blocks sm3_blocks_generic
+#endif
+
+void sm3_init(struct sm3_ctx *ctx)
+{
+ ctx->state = sm3_iv;
+ ctx->bytecount = 0;
+}
+EXPORT_SYMBOL_GPL(sm3_init);
+
+void sm3_update(struct sm3_ctx *ctx, const u8 *data, size_t len)
+{
+ size_t partial = ctx->bytecount % SM3_BLOCK_SIZE;
+
+ ctx->bytecount += len;
+
+ if (partial + len >= SM3_BLOCK_SIZE) {
+ size_t nblocks;
+
+ if (partial) {
+ size_t l = SM3_BLOCK_SIZE - partial;
+
+ memcpy(&ctx->buf[partial], data, l);
+ data += l;
+ len -= l;
+
+ sm3_blocks(&ctx->state, ctx->buf, 1);
+ }
+
+ nblocks = len / SM3_BLOCK_SIZE;
+ len %= SM3_BLOCK_SIZE;
+
+ if (nblocks) {
+ sm3_blocks(&ctx->state, data, nblocks);
+ data += nblocks * SM3_BLOCK_SIZE;
+ }
+ partial = 0;
+ }
+ if (len)
+ memcpy(&ctx->buf[partial], data, len);
+}
+EXPORT_SYMBOL_GPL(sm3_update);
+
+static void __sm3_final(struct sm3_ctx *ctx, u8 out[SM3_DIGEST_SIZE])
+{
+ u64 bitcount = ctx->bytecount << 3;
+ size_t partial = ctx->bytecount % SM3_BLOCK_SIZE;
+
+ ctx->buf[partial++] = 0x80;
+ if (partial > SM3_BLOCK_SIZE - 8) {
+ memset(&ctx->buf[partial], 0, SM3_BLOCK_SIZE - partial);
+ sm3_blocks(&ctx->state, ctx->buf, 1);
+ partial = 0;
+ }
+ memset(&ctx->buf[partial], 0, SM3_BLOCK_SIZE - 8 - partial);
+ *(__be64 *)&ctx->buf[SM3_BLOCK_SIZE - 8] = cpu_to_be64(bitcount);
+ sm3_blocks(&ctx->state, ctx->buf, 1);
+
+ for (size_t i = 0; i < SM3_DIGEST_SIZE; i += 4)
+ put_unaligned_be32(ctx->state.h[i / 4], out + i);
+}
+
+void sm3_final(struct sm3_ctx *ctx, u8 out[SM3_DIGEST_SIZE])
+{
+ __sm3_final(ctx, out);
+ memzero_explicit(ctx, sizeof(*ctx));
+}
+EXPORT_SYMBOL_GPL(sm3_final);
+
+void sm3(const u8 *data, size_t len, u8 out[SM3_DIGEST_SIZE])
+{
+ struct sm3_ctx ctx;
+
+ sm3_init(&ctx);
+ sm3_update(&ctx, data, len);
+ sm3_final(&ctx, out);
+}
+EXPORT_SYMBOL_GPL(sm3);
+
+#ifdef sm3_mod_init_arch
+static int __init sm3_mod_init(void)
+{
+ sm3_mod_init_arch();
+ return 0;
+}
+subsys_initcall(sm3_mod_init);
+
+static void __exit sm3_mod_exit(void)
+{
+}
+module_exit(sm3_mod_exit);
+#endif
+
+MODULE_DESCRIPTION("SM3 library functions");
MODULE_LICENSE("GPL v2");
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 05/12] lib/crypto: tests: Add KUnit tests for SM3
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (3 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 04/12] lib/crypto: sm3: Add SM3 library API Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 06/12] crypto: sm3 - Replace with wrapper around library Eric Biggers
` (8 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Add a KUnit test suite for the SM3 library. It closely mirrors the test
suites for the other cryptographic hash functions. The actual test and
benchmark logic is already in hash-test-template.h; this just wires it
up for SM3 in the usual way.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
lib/crypto/.kunitconfig | 1 +
lib/crypto/tests/Kconfig | 9 ++
lib/crypto/tests/Makefile | 1 +
lib/crypto/tests/sm3-testvecs.h | 231 ++++++++++++++++++++++++++++
lib/crypto/tests/sm3_kunit.c | 31 ++++
scripts/crypto/gen-hash-testvecs.py | 3 +
6 files changed, 276 insertions(+)
create mode 100644 lib/crypto/tests/sm3-testvecs.h
create mode 100644 lib/crypto/tests/sm3_kunit.c
diff --git a/lib/crypto/.kunitconfig b/lib/crypto/.kunitconfig
index 63a592731d1d..61f880859526 100644
--- a/lib/crypto/.kunitconfig
+++ b/lib/crypto/.kunitconfig
@@ -13,5 +13,6 @@ CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST=y
CONFIG_CRYPTO_LIB_POLYVAL_KUNIT_TEST=y
CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST=y
CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST=y
CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST=y
CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST=y
+CONFIG_CRYPTO_LIB_SM3_KUNIT_TEST=y
diff --git a/lib/crypto/tests/Kconfig b/lib/crypto/tests/Kconfig
index 42e1770e1883..72d445a7eac5 100644
--- a/lib/crypto/tests/Kconfig
+++ b/lib/crypto/tests/Kconfig
@@ -114,10 +114,18 @@ config CRYPTO_LIB_SHA3_KUNIT_TEST
help
KUnit tests for the SHA3 cryptographic hash and XOF functions,
including SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128 and
SHAKE256.
+config CRYPTO_LIB_SM3_KUNIT_TEST
+ tristate "KUnit tests for SM3" if !KUNIT_ALL_TESTS
+ depends on KUNIT && CRYPTO_LIB_SM3
+ default KUNIT_ALL_TESTS
+ select CRYPTO_LIB_BENCHMARK_VISIBLE
+ help
+ KUnit tests for the SM3 cryptographic hash function.
+
config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
tristate "Enable all crypto library code for KUnit tests"
depends on KUNIT
select CRYPTO_LIB_AES_CBC_MACS
select CRYPTO_LIB_BLAKE2B
@@ -129,10 +137,11 @@ config CRYPTO_LIB_ENABLE_ALL_FOR_KUNIT
select CRYPTO_LIB_POLYVAL
select CRYPTO_LIB_SHA1
select CRYPTO_LIB_SHA256
select CRYPTO_LIB_SHA512
select CRYPTO_LIB_SHA3
+ select CRYPTO_LIB_SM3
help
Enable all the crypto library code that has KUnit tests.
Enable this only if you'd like to test all the crypto library code,
even code that wouldn't otherwise need to be built.
diff --git a/lib/crypto/tests/Makefile b/lib/crypto/tests/Makefile
index f864e0ffbee4..88920fbc4324 100644
--- a/lib/crypto/tests/Makefile
+++ b/lib/crypto/tests/Makefile
@@ -11,5 +11,6 @@ obj-$(CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST) += poly1305_kunit.o
obj-$(CONFIG_CRYPTO_LIB_POLYVAL_KUNIT_TEST) += polyval_kunit.o
obj-$(CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST) += sha1_kunit.o
obj-$(CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST) += sha224_kunit.o sha256_kunit.o
obj-$(CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST) += sha384_kunit.o sha512_kunit.o
obj-$(CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST) += sha3_kunit.o
+obj-$(CONFIG_CRYPTO_LIB_SM3_KUNIT_TEST) += sm3_kunit.o
diff --git a/lib/crypto/tests/sm3-testvecs.h b/lib/crypto/tests/sm3-testvecs.h
new file mode 100644
index 000000000000..5e788c29f487
--- /dev/null
+++ b/lib/crypto/tests/sm3-testvecs.h
@@ -0,0 +1,231 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/* This file was generated by: ./scripts/crypto/gen-hash-testvecs.py sm3 */
+
+static const struct {
+ size_t data_len;
+ u8 digest[SM3_DIGEST_SIZE];
+} hash_testvecs[] = {
+ {
+ .data_len = 0,
+ .digest = {
+ 0x1a, 0xb2, 0x1d, 0x83, 0x55, 0xcf, 0xa1, 0x7f,
+ 0x8e, 0x61, 0x19, 0x48, 0x31, 0xe8, 0x1a, 0x8f,
+ 0x22, 0xbe, 0xc8, 0xc7, 0x28, 0xfe, 0xfb, 0x74,
+ 0x7e, 0xd0, 0x35, 0xeb, 0x50, 0x82, 0xaa, 0x2b,
+ },
+ },
+ {
+ .data_len = 1,
+ .digest = {
+ 0xb6, 0x22, 0x2c, 0x39, 0xdc, 0x14, 0x9a, 0xee,
+ 0x01, 0x9a, 0xcb, 0x0d, 0xe6, 0xc6, 0x75, 0x6e,
+ 0x8f, 0x18, 0x7b, 0x0e, 0xe8, 0x98, 0x61, 0x71,
+ 0x2b, 0xd8, 0x38, 0xa9, 0xee, 0x2c, 0x1e, 0x93,
+ },
+ },
+ {
+ .data_len = 2,
+ .digest = {
+ 0x62, 0x0c, 0x66, 0x77, 0x67, 0x28, 0x74, 0x8a,
+ 0xe3, 0x64, 0xea, 0x44, 0x6a, 0x3f, 0x34, 0x61,
+ 0x55, 0xc5, 0xaa, 0xb2, 0x6c, 0x67, 0x97, 0x68,
+ 0x68, 0xae, 0x4d, 0x64, 0xa8, 0xb6, 0x72, 0x3e,
+ },
+ },
+ {
+ .data_len = 3,
+ .digest = {
+ 0x71, 0xd4, 0x63, 0xb1, 0xfa, 0x27, 0xc7, 0xae,
+ 0x65, 0xed, 0x5c, 0x93, 0x70, 0xe0, 0x09, 0x34,
+ 0x2f, 0x42, 0xe6, 0x71, 0x16, 0x8e, 0x90, 0x90,
+ 0x9a, 0xdd, 0xa6, 0x44, 0x66, 0x71, 0x18, 0xf9,
+ },
+ },
+ {
+ .data_len = 16,
+ .digest = {
+ 0x79, 0x0b, 0x68, 0xb5, 0x41, 0xc1, 0x97, 0xa0,
+ 0x50, 0xe6, 0x93, 0x70, 0xf6, 0x98, 0x49, 0xea,
+ 0x92, 0xc9, 0xd0, 0xb1, 0x46, 0xbd, 0x4a, 0x0c,
+ 0x8e, 0xe8, 0xf3, 0xe4, 0x8f, 0x90, 0x33, 0x3c,
+ },
+ },
+ {
+ .data_len = 32,
+ .digest = {
+ 0x32, 0x9f, 0xa3, 0x18, 0x18, 0x45, 0xe0, 0x28,
+ 0xd3, 0xa4, 0x41, 0x3a, 0x25, 0x62, 0x9c, 0x95,
+ 0xab, 0xfe, 0x02, 0xe0, 0x37, 0x7d, 0x3c, 0xc4,
+ 0xce, 0x69, 0x57, 0x5a, 0x07, 0x0e, 0xb9, 0xf5,
+ },
+ },
+ {
+ .data_len = 48,
+ .digest = {
+ 0x0c, 0xcf, 0x7c, 0x48, 0x44, 0xa0, 0xb0, 0x8d,
+ 0xdf, 0xbe, 0x22, 0x14, 0x7e, 0xd4, 0xc3, 0x8d,
+ 0x6a, 0x23, 0xfc, 0x44, 0x0e, 0x0f, 0xde, 0xa5,
+ 0x7c, 0x8b, 0xc4, 0x8b, 0xab, 0x8c, 0x87, 0x41,
+ },
+ },
+ {
+ .data_len = 49,
+ .digest = {
+ 0xb3, 0x76, 0x8b, 0x19, 0xf9, 0x10, 0xa9, 0x56,
+ 0x4f, 0xce, 0x27, 0xaa, 0x65, 0x96, 0xe5, 0xdb,
+ 0x90, 0x9b, 0x92, 0xcd, 0x32, 0x0d, 0x16, 0xac,
+ 0xf8, 0xd0, 0x66, 0x62, 0x10, 0xf0, 0x44, 0xdf,
+ },
+ },
+ {
+ .data_len = 63,
+ .digest = {
+ 0x07, 0xc9, 0x45, 0x65, 0x9f, 0x68, 0x75, 0xc3,
+ 0x74, 0xb2, 0x3b, 0x0c, 0x97, 0x05, 0xd3, 0x13,
+ 0xc0, 0xb6, 0x21, 0xed, 0xf6, 0x10, 0x7a, 0xed,
+ 0xec, 0xd8, 0x10, 0x29, 0xbf, 0x7a, 0x78, 0x37,
+ },
+ },
+ {
+ .data_len = 64,
+ .digest = {
+ 0x3e, 0x69, 0x18, 0x45, 0xd8, 0x25, 0x6f, 0x44,
+ 0xc0, 0x02, 0xe5, 0xcf, 0xcd, 0x94, 0x42, 0xa9,
+ 0xd5, 0x12, 0x62, 0x10, 0x15, 0xa0, 0xf9, 0x16,
+ 0x19, 0x2d, 0x8d, 0x63, 0x31, 0xf2, 0x2f, 0x36,
+ },
+ },
+ {
+ .data_len = 65,
+ .digest = {
+ 0x6b, 0x3e, 0xc0, 0x20, 0xb7, 0x74, 0x30, 0xa0,
+ 0xc6, 0x5c, 0xee, 0xbe, 0xdc, 0xe6, 0xe5, 0x4f,
+ 0x3c, 0x61, 0x8d, 0x91, 0xac, 0x31, 0x4b, 0x2a,
+ 0xdf, 0x1c, 0xef, 0x24, 0xdc, 0x0a, 0x10, 0xe8,
+ },
+ },
+ {
+ .data_len = 127,
+ .digest = {
+ 0xab, 0xd6, 0xa1, 0xbf, 0x39, 0x43, 0x75, 0xda,
+ 0xbf, 0xc7, 0x22, 0xcc, 0x4e, 0xfc, 0xe4, 0x42,
+ 0x6d, 0x1b, 0x87, 0x25, 0x64, 0x7f, 0x88, 0xf7,
+ 0xc3, 0x0a, 0x0a, 0x4c, 0xd6, 0xa7, 0x68, 0xae,
+ },
+ },
+ {
+ .data_len = 128,
+ .digest = {
+ 0x1b, 0x70, 0xd4, 0x5f, 0x6c, 0xe4, 0x2d, 0x58,
+ 0x2d, 0x0f, 0x9a, 0x12, 0x34, 0xbb, 0x5e, 0x38,
+ 0xd8, 0x1f, 0x6a, 0x46, 0x8a, 0xef, 0xdb, 0x68,
+ 0x18, 0x62, 0xbb, 0x85, 0xfc, 0xc4, 0x6e, 0x2e,
+ },
+ },
+ {
+ .data_len = 129,
+ .digest = {
+ 0x33, 0x62, 0xba, 0xa7, 0x4a, 0xbc, 0xd7, 0x7b,
+ 0xd4, 0x67, 0x6d, 0x3e, 0xea, 0xe8, 0xb0, 0x64,
+ 0x0d, 0xf3, 0xae, 0x1d, 0x52, 0x24, 0x11, 0x9f,
+ 0xda, 0xa9, 0x7f, 0xd5, 0x22, 0x1a, 0xde, 0x8a,
+ },
+ },
+ {
+ .data_len = 256,
+ .digest = {
+ 0x70, 0xa8, 0xa6, 0x2b, 0xfb, 0x1f, 0x3b, 0x5a,
+ 0xcc, 0x71, 0x76, 0x9e, 0x25, 0x4c, 0xfa, 0x8f,
+ 0x39, 0x4a, 0x21, 0x8a, 0x9d, 0x74, 0x8d, 0x2c,
+ 0x31, 0xa5, 0xb5, 0xff, 0x30, 0xc1, 0x14, 0xc4,
+ },
+ },
+ {
+ .data_len = 511,
+ .digest = {
+ 0x39, 0xd0, 0x8c, 0x5f, 0xfc, 0x36, 0xc2, 0x7c,
+ 0xdb, 0x8b, 0x2e, 0xdc, 0x9d, 0x1b, 0xd1, 0xba,
+ 0x9b, 0x52, 0x6b, 0x35, 0x46, 0x46, 0x75, 0x73,
+ 0xe5, 0x62, 0x96, 0x6e, 0xf3, 0xba, 0xd9, 0x19,
+ },
+ },
+ {
+ .data_len = 513,
+ .digest = {
+ 0x76, 0xa0, 0x3d, 0xa2, 0x5f, 0xd4, 0xa6, 0xbe,
+ 0x6b, 0xdb, 0xed, 0x14, 0x9e, 0xa8, 0x15, 0x77,
+ 0xa9, 0x38, 0x30, 0x6b, 0x68, 0xfa, 0xb6, 0xe2,
+ 0x93, 0x19, 0x24, 0x72, 0x67, 0x20, 0x72, 0xc3,
+ },
+ },
+ {
+ .data_len = 1000,
+ .digest = {
+ 0x16, 0xbc, 0x33, 0x77, 0x0b, 0xcf, 0x93, 0x5e,
+ 0xec, 0x7d, 0x8d, 0x3c, 0xae, 0xd9, 0x75, 0xdf,
+ 0x46, 0x24, 0x17, 0x7e, 0x03, 0x88, 0xf2, 0x75,
+ 0xa9, 0x18, 0xa6, 0x1c, 0x7a, 0x74, 0x0d, 0xf3,
+ },
+ },
+ {
+ .data_len = 3333,
+ .digest = {
+ 0xdb, 0x54, 0x89, 0xe7, 0x1c, 0x50, 0xf2, 0xbf,
+ 0xde, 0x3a, 0xbf, 0x5b, 0xee, 0x5a, 0x46, 0x62,
+ 0x20, 0xb1, 0x80, 0x48, 0xac, 0x56, 0x33, 0xb3,
+ 0xbb, 0x3f, 0xfa, 0x02, 0xc6, 0x43, 0xb5, 0x8c,
+ },
+ },
+ {
+ .data_len = 4096,
+ .digest = {
+ 0xdf, 0x0d, 0xed, 0x3b, 0x8f, 0xea, 0x81, 0xfd,
+ 0xd6, 0x34, 0xae, 0x74, 0x24, 0x3a, 0x15, 0x38,
+ 0xe7, 0xcf, 0x45, 0x7e, 0x8f, 0xf5, 0x50, 0x6c,
+ 0xaa, 0x27, 0x23, 0x92, 0x6d, 0xab, 0x3b, 0xde,
+ },
+ },
+ {
+ .data_len = 4128,
+ .digest = {
+ 0x6a, 0xbd, 0x56, 0x5a, 0xf1, 0xc6, 0x40, 0x4d,
+ 0xf3, 0x50, 0x77, 0x87, 0x86, 0x63, 0x1b, 0x4d,
+ 0x21, 0x99, 0x96, 0xad, 0x24, 0x62, 0xce, 0xc0,
+ 0x3e, 0xb7, 0x35, 0x52, 0x56, 0x0e, 0x55, 0x85,
+ },
+ },
+ {
+ .data_len = 4160,
+ .digest = {
+ 0x5b, 0xc1, 0x1f, 0x25, 0x43, 0xa3, 0x1c, 0xa0,
+ 0x8c, 0xfc, 0x41, 0xf1, 0xcc, 0xb3, 0x95, 0x95,
+ 0xe0, 0xb9, 0xd3, 0x29, 0xf4, 0x08, 0x31, 0x47,
+ 0x6d, 0x09, 0xa8, 0x2e, 0xa5, 0xf4, 0xf1, 0x8d,
+ },
+ },
+ {
+ .data_len = 4224,
+ .digest = {
+ 0xec, 0x56, 0x1e, 0xa6, 0x1f, 0xb2, 0x87, 0xb2,
+ 0x7e, 0x15, 0xd6, 0x30, 0x08, 0x74, 0xb0, 0x48,
+ 0x90, 0x2a, 0xbe, 0x2f, 0x80, 0x3a, 0x88, 0xcc,
+ 0xd7, 0xc5, 0x87, 0x8c, 0x04, 0xef, 0x78, 0x71,
+ },
+ },
+ {
+ .data_len = 16384,
+ .digest = {
+ 0xe7, 0xb8, 0x84, 0x20, 0xff, 0xd5, 0x53, 0xe6,
+ 0x14, 0x31, 0x12, 0x75, 0xb7, 0x9a, 0x4f, 0x63,
+ 0x63, 0x00, 0xfe, 0x2c, 0x54, 0xee, 0x06, 0xfc,
+ 0x12, 0x16, 0xe5, 0xdc, 0xa4, 0x40, 0x62, 0x12,
+ },
+ },
+};
+
+static const u8 hash_testvec_consolidated[SM3_DIGEST_SIZE] = {
+ 0xe6, 0x58, 0xd4, 0x8e, 0x74, 0x92, 0xdf, 0xfe,
+ 0x58, 0x05, 0xe5, 0x29, 0x83, 0xfb, 0xb7, 0x51,
+ 0x7e, 0x66, 0x0c, 0x49, 0x3e, 0x11, 0x7e, 0x9b,
+ 0xb1, 0x83, 0x3a, 0xa6, 0xb0, 0x3c, 0xf5, 0xe0,
+};
diff --git a/lib/crypto/tests/sm3_kunit.c b/lib/crypto/tests/sm3_kunit.c
new file mode 100644
index 000000000000..dc8136acdff6
--- /dev/null
+++ b/lib/crypto/tests/sm3_kunit.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright 2026 Google LLC
+ */
+#include <crypto/sm3.h>
+#include "sm3-testvecs.h"
+
+#define HASH sm3
+#define HASH_CTX sm3_ctx
+#define HASH_SIZE SM3_DIGEST_SIZE
+#define HASH_INIT sm3_init
+#define HASH_UPDATE sm3_update
+#define HASH_FINAL sm3_final
+#include "hash-test-template.h"
+
+static struct kunit_case sm3_test_cases[] = {
+ HASH_KUNIT_CASES,
+ KUNIT_CASE(benchmark_hash),
+ {},
+};
+
+static struct kunit_suite sm3_test_suite = {
+ .name = "sm3",
+ .test_cases = sm3_test_cases,
+ .suite_init = hash_suite_init,
+ .suite_exit = hash_suite_exit,
+};
+kunit_test_suite(sm3_test_suite);
+
+MODULE_DESCRIPTION("KUnit tests and benchmark for SM3");
+MODULE_LICENSE("GPL");
diff --git a/scripts/crypto/gen-hash-testvecs.py b/scripts/crypto/gen-hash-testvecs.py
index 34b7c48f3456..37fdbc52b2c1 100755
--- a/scripts/crypto/gen-hash-testvecs.py
+++ b/scripts/crypto/gen-hash-testvecs.py
@@ -293,8 +293,11 @@ elif alg == 'sha3':
print('/* SHA3-256 test vectors */')
gen_unkeyed_testvecs('sha3-256')
print()
print('/* SHAKE test vectors */')
gen_additional_sha3_testvecs()
+elif alg == 'sm3':
+ gen_unkeyed_testvecs(alg)
+ # Kernel doesn't implement HMAC-SM3 library functions yet.
else:
gen_unkeyed_testvecs(alg)
gen_hmac_testvecs(alg)
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 06/12] crypto: sm3 - Replace with wrapper around library
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (4 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 05/12] lib/crypto: tests: Add KUnit tests for SM3 Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 07/12] lib/crypto: arm64/sm3: Migrate optimized code into library Eric Biggers
` (7 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Reimplement the "sm3" crypto_shash on top of the SM3 library, closely
mirroring the other hash algorithms (e.g. SHA-*).
The result, after later commits migrate the architecture-optimized SM3
code into the library as well, is that crypto/sm3.c will be the single
point of integration between crypto_shash and the actual SM3
implementations, simplifying the code.
Note: to see the diff from crypto/sm3_generic.c to crypto/sm3.c, view
this commit with 'git show -M10'.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
crypto/Makefile | 2 +-
crypto/{sm3_generic.c => sm3.c} | 83 +++++++++++++++++----------
crypto/testmgr.c | 2 +
drivers/crypto/starfive/jh7110-hash.c | 4 +-
4 files changed, 59 insertions(+), 32 deletions(-)
rename crypto/{sm3_generic.c => sm3.c} (30%)
diff --git a/crypto/Makefile b/crypto/Makefile
index 28467f900c9a..842dbc459e4b 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -81,11 +81,11 @@ obj-$(CONFIG_CRYPTO_MD5) += md5.o
obj-$(CONFIG_CRYPTO_RMD160) += rmd160.o
obj-$(CONFIG_CRYPTO_SHA1) += sha1.o
obj-$(CONFIG_CRYPTO_SHA256) += sha256.o
obj-$(CONFIG_CRYPTO_SHA512) += sha512.o
obj-$(CONFIG_CRYPTO_SHA3) += sha3.o
-obj-$(CONFIG_CRYPTO_SM3) += sm3_generic.o
+obj-$(CONFIG_CRYPTO_SM3) += sm3.o
obj-$(CONFIG_CRYPTO_STREEBOG) += streebog_generic.o
obj-$(CONFIG_CRYPTO_WP512) += wp512.o
CFLAGS_wp512.o := $(call cc-option,-fno-schedule-insns) # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79149
obj-$(CONFIG_CRYPTO_BLAKE2B) += blake2b.o
obj-$(CONFIG_CRYPTO_ECB) += ecb.o
diff --git a/crypto/sm3_generic.c b/crypto/sm3.c
similarity index 30%
rename from crypto/sm3_generic.c
rename to crypto/sm3.c
index 0c606f526347..05111a99b851 100644
--- a/crypto/sm3_generic.c
+++ b/crypto/sm3.c
@@ -4,61 +4,86 @@
* described at https://tools.ietf.org/html/draft-shen-sm3-hash-01
*
* Copyright (C) 2017 ARM Limited or its affiliates.
* Written by Gilad Ben-Yossef <gilad@benyossef.com>
* Copyright (C) 2021 Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
+ * Copyright 2026 Google LLC
*/
#include <crypto/internal/hash.h>
#include <crypto/sm3.h>
-#include <crypto/sm3_base.h>
#include <linux/kernel.h>
#include <linux/module.h>
-static int crypto_sm3_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
+#define SM3_CTX(desc) ((struct sm3_ctx *)shash_desc_ctx(desc))
+
+static int crypto_sm3_init(struct shash_desc *desc)
+{
+ sm3_init(SM3_CTX(desc));
+ return 0;
+}
+
+static int crypto_sm3_update(struct shash_desc *desc,
+ const u8 *data, unsigned int len)
+{
+ sm3_update(SM3_CTX(desc), data, len);
+ return 0;
+}
+
+static int crypto_sm3_final(struct shash_desc *desc, u8 *out)
{
- return sm3_base_do_update_blocks(desc, data, len, sm3_block_generic);
+ sm3_final(SM3_CTX(desc), out);
+ return 0;
}
-static int crypto_sm3_finup(struct shash_desc *desc, const u8 *data,
- unsigned int len, u8 *hash)
+static int crypto_sm3_digest(struct shash_desc *desc,
+ const u8 *data, unsigned int len, u8 *out)
{
- sm3_base_do_finup(desc, data, len, sm3_block_generic);
- return sm3_base_finish(desc, hash);
+ sm3(data, len, out);
+ return 0;
+}
+
+static int crypto_sm3_export_core(struct shash_desc *desc, void *out)
+{
+ memcpy(out, SM3_CTX(desc), sizeof(struct sm3_ctx));
+ return 0;
+}
+
+static int crypto_sm3_import_core(struct shash_desc *desc, const void *in)
+{
+ memcpy(SM3_CTX(desc), in, sizeof(struct sm3_ctx));
+ return 0;
}
static struct shash_alg sm3_alg = {
- .digestsize = SM3_DIGEST_SIZE,
- .init = sm3_base_init,
- .update = crypto_sm3_update,
- .finup = crypto_sm3_finup,
- .descsize = SM3_STATE_SIZE,
- .base = {
- .cra_name = "sm3",
- .cra_driver_name = "sm3-generic",
- .cra_priority = 100,
- .cra_flags = CRYPTO_AHASH_ALG_BLOCK_ONLY |
- CRYPTO_AHASH_ALG_FINUP_MAX,
- .cra_blocksize = SM3_BLOCK_SIZE,
- .cra_module = THIS_MODULE,
- }
+ .base.cra_name = "sm3",
+ .base.cra_driver_name = "sm3-lib",
+ .base.cra_priority = 300,
+ .base.cra_blocksize = SM3_BLOCK_SIZE,
+ .base.cra_module = THIS_MODULE,
+ .digestsize = SM3_DIGEST_SIZE,
+ .init = crypto_sm3_init,
+ .update = crypto_sm3_update,
+ .final = crypto_sm3_final,
+ .digest = crypto_sm3_digest,
+ .export_core = crypto_sm3_export_core,
+ .import_core = crypto_sm3_import_core,
+ .descsize = sizeof(struct sm3_ctx),
};
-static int __init sm3_generic_mod_init(void)
+static int __init crypto_sm3_mod_init(void)
{
return crypto_register_shash(&sm3_alg);
}
+module_init(crypto_sm3_mod_init);
-static void __exit sm3_generic_mod_fini(void)
+static void __exit crypto_sm3_mod_exit(void)
{
crypto_unregister_shash(&sm3_alg);
}
-
-module_init(sm3_generic_mod_init);
-module_exit(sm3_generic_mod_fini);
+module_exit(crypto_sm3_mod_exit);
MODULE_LICENSE("GPL v2");
-MODULE_DESCRIPTION("SM3 Secure Hash Algorithm");
+MODULE_DESCRIPTION("Crypto API support for SM3");
MODULE_ALIAS_CRYPTO("sm3");
-MODULE_ALIAS_CRYPTO("sm3-generic");
+MODULE_ALIAS_CRYPTO("sm3-lib");
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index fec950f1628b..8089e9f04218 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -5083,10 +5083,11 @@ static const struct alg_test_desc alg_test_descs[] = {
.suite = {
.hash = __VECS(hmac_sha512_tv_template)
}
}, {
.alg = "hmac(sm3)",
+ .generic_driver = "hmac(sm3-lib)",
.test = alg_test_hash,
.suite = {
.hash = __VECS(hmac_sm3_tv_template)
}
}, {
@@ -5450,10 +5451,11 @@ static const struct alg_test_desc alg_test_descs[] = {
.suite = {
.hash = __VECS(sha512_tv_template)
}
}, {
.alg = "sm3",
+ .generic_driver = "sm3-lib",
.test = alg_test_hash,
.suite = {
.hash = __VECS(sm3_tv_template)
}
}, {
diff --git a/drivers/crypto/starfive/jh7110-hash.c b/drivers/crypto/starfive/jh7110-hash.c
index 54b7af4a7aee..742038a5201a 100644
--- a/drivers/crypto/starfive/jh7110-hash.c
+++ b/drivers/crypto/starfive/jh7110-hash.c
@@ -518,11 +518,11 @@ static int starfive_sha512_init_tfm(struct crypto_ahash *hash)
STARFIVE_HASH_SHA512, 0);
}
static int starfive_sm3_init_tfm(struct crypto_ahash *hash)
{
- return starfive_hash_init_tfm(hash, "sm3-generic",
+ return starfive_hash_init_tfm(hash, "sm3-lib",
STARFIVE_HASH_SM3, 0);
}
static int starfive_hmac_sha224_init_tfm(struct crypto_ahash *hash)
{
@@ -548,11 +548,11 @@ static int starfive_hmac_sha512_init_tfm(struct crypto_ahash *hash)
STARFIVE_HASH_SHA512, 1);
}
static int starfive_hmac_sm3_init_tfm(struct crypto_ahash *hash)
{
- return starfive_hash_init_tfm(hash, "hmac(sm3-generic)",
+ return starfive_hash_init_tfm(hash, "hmac(sm3-lib)",
STARFIVE_HASH_SM3, 1);
}
static struct ahash_engine_alg algs_sha2_sm3[] = {
{
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 07/12] lib/crypto: arm64/sm3: Migrate optimized code into library
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (5 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 06/12] crypto: sm3 - Replace with wrapper around library Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 08/12] lib/crypto: riscv/sm3: " Eric Biggers
` (6 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Instead of exposing the arm64-optimized SM3 code via arm64-specific
crypto_shash algorithms, instead just implement the sm3_blocks() library
function. This is much simpler, it makes the SM3 library functions be
arm64-optimized, and it fixes the longstanding issue where the
arm64-optimized SM3 code was disabled by default. SM3 still remains
available through crypto_shash, but individual architectures no longer
need to handle it.
Tweak the SM3 assembly function prototypes to match what the library
expects, including changing the block count from 'int' to 'size_t'.
sm3_ce_transform() had to be updated to access 'x2' instead of 'w2',
while sm3_neon_transform() already used 'x2'.
Remove the CFI stubs which are no longer needed because the SM3 assembly
functions are no longer ever indirectly called.
Remove the dependency on KERNEL_MODE_NEON. It was unnecessary, because
KERNEL_MODE_NEON is always enabled on arm64.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/arm64/configs/defconfig | 2 +-
arch/arm64/crypto/Kconfig | 22 ------
arch/arm64/crypto/Makefile | 6 --
arch/arm64/crypto/sm3-ce-glue.c | 70 -------------------
arch/arm64/crypto/sm3-neon-glue.c | 67 ------------------
lib/crypto/Kconfig | 1 +
lib/crypto/Makefile | 13 +++-
.../crypto => lib/crypto/arm64}/sm3-ce-core.S | 11 ++-
.../crypto/arm64}/sm3-neon-core.S | 9 ++-
lib/crypto/arm64/sm3.h | 41 +++++++++++
10 files changed, 62 insertions(+), 180 deletions(-)
delete mode 100644 arch/arm64/crypto/sm3-ce-glue.c
delete mode 100644 arch/arm64/crypto/sm3-neon-glue.c
rename {arch/arm64/crypto => lib/crypto/arm64}/sm3-ce-core.S (93%)
rename {arch/arm64/crypto => lib/crypto/arm64}/sm3-neon-core.S (98%)
create mode 100644 lib/crypto/arm64/sm3.h
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index b67d5b1fc45b..b4458bee767a 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -1914,13 +1914,13 @@ CONFIG_CRYPTO_USER=y
CONFIG_CRYPTO_CHACHA20=m
CONFIG_CRYPTO_BENCHMARK=m
CONFIG_CRYPTO_ECHAINIV=y
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_SHA3=m
+CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_USER_API_RNG=m
CONFIG_CRYPTO_GHASH_ARM64_CE=y
-CONFIG_CRYPTO_SM3_ARM64_CE=m
CONFIG_CRYPTO_AES_ARM64_CE_BLK=y
CONFIG_CRYPTO_AES_ARM64_BS=m
CONFIG_CRYPTO_AES_ARM64_CE_CCM=y
CONFIG_CRYPTO_DEV_SUN8I_CE=m
CONFIG_CRYPTO_DEV_FSL_CAAM=m
diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig
index 82794afaffc9..b595062fd842 100644
--- a/arch/arm64/crypto/Kconfig
+++ b/arch/arm64/crypto/Kconfig
@@ -13,32 +13,10 @@ config CRYPTO_GHASH_ARM64_CE
GCM GHASH function (NIST SP800-38D)
Architecture: arm64 using:
- ARMv8 Crypto Extensions
-config CRYPTO_SM3_NEON
- tristate "Hash functions: SM3 (NEON)"
- depends on KERNEL_MODE_NEON
- select CRYPTO_HASH
- select CRYPTO_LIB_SM3
- help
- SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012)
-
- Architecture: arm64 using:
- - NEON (Advanced SIMD) extensions
-
-config CRYPTO_SM3_ARM64_CE
- tristate "Hash functions: SM3 (ARMv8.2 Crypto Extensions)"
- depends on KERNEL_MODE_NEON
- select CRYPTO_HASH
- select CRYPTO_LIB_SM3
- help
- SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012)
-
- Architecture: arm64 using:
- - ARMv8.2 Crypto Extensions
-
config CRYPTO_AES_ARM64_CE_BLK
tristate "Ciphers: AES, modes: ECB/CBC/CTR/XTS (ARMv8 Crypto Extensions)"
depends on KERNEL_MODE_NEON
select CRYPTO_SKCIPHER
select CRYPTO_LIB_AES
diff --git a/arch/arm64/crypto/Makefile b/arch/arm64/crypto/Makefile
index 8a8e3e551ed3..a169f9033401 100644
--- a/arch/arm64/crypto/Makefile
+++ b/arch/arm64/crypto/Makefile
@@ -3,16 +3,10 @@
# linux/arch/arm64/crypto/Makefile
#
# Copyright (C) 2014 Linaro Ltd <ard.biesheuvel@linaro.org>
#
-obj-$(CONFIG_CRYPTO_SM3_NEON) += sm3-neon.o
-sm3-neon-y := sm3-neon-glue.o sm3-neon-core.o
-
-obj-$(CONFIG_CRYPTO_SM3_ARM64_CE) += sm3-ce.o
-sm3-ce-y := sm3-ce-glue.o sm3-ce-core.o
-
obj-$(CONFIG_CRYPTO_SM4_ARM64_CE) += sm4-ce-cipher.o
sm4-ce-cipher-y := sm4-ce-cipher-glue.o sm4-ce-cipher-core.o
obj-$(CONFIG_CRYPTO_SM4_ARM64_CE_BLK) += sm4-ce.o
sm4-ce-y := sm4-ce-glue.o sm4-ce-core.o
diff --git a/arch/arm64/crypto/sm3-ce-glue.c b/arch/arm64/crypto/sm3-ce-glue.c
deleted file mode 100644
index 24c1fcfae072..000000000000
--- a/arch/arm64/crypto/sm3-ce-glue.c
+++ /dev/null
@@ -1,70 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * sm3-ce-glue.c - SM3 secure hash using ARMv8.2 Crypto Extensions
- *
- * Copyright (C) 2018 Linaro Ltd <ard.biesheuvel@linaro.org>
- */
-
-#include <crypto/internal/hash.h>
-#include <crypto/sm3.h>
-#include <crypto/sm3_base.h>
-#include <linux/cpufeature.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-
-#include <asm/simd.h>
-
-MODULE_DESCRIPTION("SM3 secure hash using ARMv8 Crypto Extensions");
-MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>");
-MODULE_LICENSE("GPL v2");
-
-asmlinkage void sm3_ce_transform(struct sm3_state *sst, u8 const *src,
- int blocks);
-
-static int sm3_ce_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
-{
- int remain;
-
- scoped_ksimd() {
- remain = sm3_base_do_update_blocks(desc, data, len, sm3_ce_transform);
- }
- return remain;
-}
-
-static int sm3_ce_finup(struct shash_desc *desc, const u8 *data,
- unsigned int len, u8 *out)
-{
- scoped_ksimd() {
- sm3_base_do_finup(desc, data, len, sm3_ce_transform);
- }
- return sm3_base_finish(desc, out);
-}
-
-static struct shash_alg sm3_alg = {
- .digestsize = SM3_DIGEST_SIZE,
- .init = sm3_base_init,
- .update = sm3_ce_update,
- .finup = sm3_ce_finup,
- .descsize = SM3_STATE_SIZE,
- .base.cra_name = "sm3",
- .base.cra_driver_name = "sm3-ce",
- .base.cra_flags = CRYPTO_AHASH_ALG_BLOCK_ONLY |
- CRYPTO_AHASH_ALG_FINUP_MAX,
- .base.cra_blocksize = SM3_BLOCK_SIZE,
- .base.cra_module = THIS_MODULE,
- .base.cra_priority = 400,
-};
-
-static int __init sm3_ce_mod_init(void)
-{
- return crypto_register_shash(&sm3_alg);
-}
-
-static void __exit sm3_ce_mod_fini(void)
-{
- crypto_unregister_shash(&sm3_alg);
-}
-
-module_cpu_feature_match(SM3, sm3_ce_mod_init);
-module_exit(sm3_ce_mod_fini);
diff --git a/arch/arm64/crypto/sm3-neon-glue.c b/arch/arm64/crypto/sm3-neon-glue.c
deleted file mode 100644
index 15f30cc24f32..000000000000
--- a/arch/arm64/crypto/sm3-neon-glue.c
+++ /dev/null
@@ -1,67 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * sm3-neon-glue.c - SM3 secure hash using NEON instructions
- *
- * Copyright (C) 2022 Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
- */
-
-#include <asm/simd.h>
-#include <crypto/internal/hash.h>
-#include <crypto/sm3.h>
-#include <crypto/sm3_base.h>
-#include <linux/cpufeature.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-
-
-asmlinkage void sm3_neon_transform(struct sm3_state *sst, u8 const *src,
- int blocks);
-
-static int sm3_neon_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
-{
- scoped_ksimd()
- return sm3_base_do_update_blocks(desc, data, len,
- sm3_neon_transform);
-}
-
-static int sm3_neon_finup(struct shash_desc *desc, const u8 *data,
- unsigned int len, u8 *out)
-{
- scoped_ksimd()
- sm3_base_do_finup(desc, data, len, sm3_neon_transform);
- return sm3_base_finish(desc, out);
-}
-
-static struct shash_alg sm3_alg = {
- .digestsize = SM3_DIGEST_SIZE,
- .init = sm3_base_init,
- .update = sm3_neon_update,
- .finup = sm3_neon_finup,
- .descsize = SM3_STATE_SIZE,
- .base.cra_name = "sm3",
- .base.cra_driver_name = "sm3-neon",
- .base.cra_flags = CRYPTO_AHASH_ALG_BLOCK_ONLY |
- CRYPTO_AHASH_ALG_FINUP_MAX,
- .base.cra_blocksize = SM3_BLOCK_SIZE,
- .base.cra_module = THIS_MODULE,
- .base.cra_priority = 200,
-};
-
-static int __init sm3_neon_init(void)
-{
- return crypto_register_shash(&sm3_alg);
-}
-
-static void __exit sm3_neon_fini(void)
-{
- crypto_unregister_shash(&sm3_alg);
-}
-
-module_init(sm3_neon_init);
-module_exit(sm3_neon_fini);
-
-MODULE_DESCRIPTION("SM3 secure hash using NEON instructions");
-MODULE_AUTHOR("Jussi Kivilinna <jussi.kivilinna@iki.fi>");
-MODULE_AUTHOR("Tianjia Zhang <tianjia.zhang@linux.alibaba.com>");
-MODULE_LICENSE("GPL v2");
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index c5819e2518f6..a4e55b6a03af 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -272,9 +272,10 @@ config CRYPTO_LIB_SM3
functions from <crypto/sm3.h>.
config CRYPTO_LIB_SM3_ARCH
bool
depends on CRYPTO_LIB_SM3 && !UML
+ default y if ARM64
source "lib/crypto/tests/Kconfig"
endmenu
diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index a961615c8c7f..48ed6ee5e3c9 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -347,15 +347,22 @@ CFLAGS_sha3.o += -I$(src)/$(SRCARCH)
libsha3-$(CONFIG_ARM64) += arm64/sha3-ce-core.o
endif # CONFIG_CRYPTO_LIB_SHA3_ARCH
################################################################################
+obj-$(CONFIG_CRYPTO_LIB_SM3) += libsm3.o
+libsm3-y := sm3.o
+ifeq ($(CONFIG_CRYPTO_LIB_SM3_ARCH),y)
+CFLAGS_sm3.o += -I$(src)/$(SRCARCH)
+libsm3-$(CONFIG_ARM64) += arm64/sm3-ce-core.o \
+ arm64/sm3-neon-core.o
+endif # CONFIG_CRYPTO_LIB_SM3_ARCH
+
+################################################################################
+
obj-$(CONFIG_MPILIB) += mpi/
obj-$(CONFIG_CRYPTO_SELFTESTS_FULL) += simd.o
-obj-$(CONFIG_CRYPTO_LIB_SM3) += libsm3.o
-libsm3-y := sm3.o
-
# clean-files must be defined unconditionally
clean-files += arm/sha256-core.S arm/sha512-core.S
clean-files += arm64/sha256-core.S arm64/sha512-core.S
diff --git a/arch/arm64/crypto/sm3-ce-core.S b/lib/crypto/arm64/sm3-ce-core.S
similarity index 93%
rename from arch/arm64/crypto/sm3-ce-core.S
rename to lib/crypto/arm64/sm3-ce-core.S
index ca70cfacd0d0..9cef7ea7f34f 100644
--- a/arch/arm64/crypto/sm3-ce-core.S
+++ b/lib/crypto/arm64/sm3-ce-core.S
@@ -4,11 +4,10 @@
*
* Copyright (C) 2018 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
#include <linux/linkage.h>
-#include <linux/cfi_types.h>
#include <asm/assembler.h>
.irp b, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
.set .Lv\b\().4s, \b
.endr
@@ -68,15 +67,15 @@
sm3partw2 \s4\().4s, v7.4s, v6.4s
.endif
.endm
/*
- * void sm3_ce_transform(struct sm3_state *sst, u8 const *src,
- * int blocks)
+ * void sm3_ce_transform(struct sm3_block_state *state,
+ * const u8 *data, size_t nblocks)
*/
.text
-SYM_TYPED_FUNC_START(sm3_ce_transform)
+SYM_FUNC_START(sm3_ce_transform)
/* load state */
ld1 {v8.4s-v9.4s}, [x0]
rev64 v8.4s, v8.4s
rev64 v9.4s, v9.4s
ext v8.16b, v8.16b, v8.16b, #8
@@ -85,11 +84,11 @@ SYM_TYPED_FUNC_START(sm3_ce_transform)
adr_l x8, .Lt
ldp s13, s14, [x8]
/* load input */
0: ld1 {v0.16b-v3.16b}, [x1], #64
- sub w2, w2, #1
+ sub x2, x2, #1
mov v15.16b, v8.16b
mov v16.16b, v9.16b
CPU_LE( rev32 v0.16b, v0.16b )
@@ -121,11 +120,11 @@ CPU_LE( rev32 v3.16b, v3.16b )
eor v8.16b, v8.16b, v15.16b
eor v9.16b, v9.16b, v16.16b
/* handled all input blocks? */
- cbnz w2, 0b
+ cbnz x2, 0b
/* save state */
rev64 v8.4s, v8.4s
rev64 v9.4s, v9.4s
ext v8.16b, v8.16b, v8.16b, #8
diff --git a/arch/arm64/crypto/sm3-neon-core.S b/lib/crypto/arm64/sm3-neon-core.S
similarity index 98%
rename from arch/arm64/crypto/sm3-neon-core.S
rename to lib/crypto/arm64/sm3-neon-core.S
index 4357e0e51be3..ad874af13802 100644
--- a/arch/arm64/crypto/sm3-neon-core.S
+++ b/lib/crypto/arm64/sm3-neon-core.S
@@ -7,11 +7,10 @@
* Copyright (C) 2021 Jussi Kivilinna <jussi.kivilinna@iki.fi>
* Copyright (c) 2022 Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
*/
#include <linux/linkage.h>
-#include <linux/cfi_types.h>
#include <asm/assembler.h>
/* Context structure */
#define state_h0 0
@@ -343,18 +342,18 @@
#define SCHED_W_W5W0W1W2W3W4_3(iop_num, round) \
SCHED_W_3_##iop_num(round, W5, W0, W1, W2, W3, W4)
/*
- * Transform blocks*64 bytes (blocks*16 32-bit words) at 'src'.
+ * Transform nblocks*64 bytes (nblocks*16 32-bit words) at 'data'.
*
- * void sm3_neon_transform(struct sm3_state *sst, u8 const *src,
- * int blocks)
+ * void sm3_neon_transform(struct sm3_block_state *state,
+ * const u8 *data, size_t nblocks)
*/
.text
.align 3
-SYM_TYPED_FUNC_START(sm3_neon_transform)
+SYM_FUNC_START(sm3_neon_transform)
ldp ra, rb, [RSTATE, #0]
ldp rc, rd, [RSTATE, #8]
ldp re, rf, [RSTATE, #16]
ldp rg, rh, [RSTATE, #24]
diff --git a/lib/crypto/arm64/sm3.h b/lib/crypto/arm64/sm3.h
new file mode 100644
index 000000000000..beb9cd82bb7d
--- /dev/null
+++ b/lib/crypto/arm64/sm3.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * SM3 optimized for ARM64
+ *
+ * Copyright 2026 Google LLC
+ */
+#include <asm/simd.h>
+#include <linux/cpufeature.h>
+
+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_neon);
+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_ce);
+
+asmlinkage void sm3_neon_transform(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks);
+asmlinkage void sm3_ce_transform(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks);
+
+static void sm3_blocks(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks)
+{
+ if (static_branch_likely(&have_neon) && likely(may_use_simd())) {
+ scoped_ksimd() {
+ if (static_branch_likely(&have_ce))
+ sm3_ce_transform(state, data, nblocks);
+ else
+ sm3_neon_transform(state, data, nblocks);
+ }
+ } else {
+ sm3_blocks_generic(state, data, nblocks);
+ }
+}
+
+#define sm3_mod_init_arch sm3_mod_init_arch
+static void sm3_mod_init_arch(void)
+{
+ if (cpu_have_named_feature(ASIMD)) {
+ static_branch_enable(&have_neon);
+ if (cpu_have_named_feature(SM3))
+ static_branch_enable(&have_ce);
+ }
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 08/12] lib/crypto: riscv/sm3: Migrate optimized code into library
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (6 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 07/12] lib/crypto: arm64/sm3: Migrate optimized code into library Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 09/12] lib/crypto: x86/sm3: " Eric Biggers
` (5 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Instead of exposing the riscv-optimized SM3 code via a riscv-specific
crypto_shash algorithm, instead just implement the sm3_blocks() library
function. This is much simpler, it makes the SM3 library functions be
riscv-optimized, and it fixes the longstanding issue where the
riscv-optimized SM3 code was disabled by default. SM3 still remains
available through crypto_shash, but individual architectures no longer
need to handle it.
Tweak the prototype of sm3_transform_zvksh_zvkb() to match what the
library expects, including changing the block count to size_t.
Note that the assembly code already treated it as size_t.
Note: to see the diff from arch/riscv/crypto/sm3-riscv64-glue.c to
lib/crypto/riscv/sm3.h, view this commit with 'git show -M10'.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/riscv/crypto/Kconfig | 13 ---
arch/riscv/crypto/Makefile | 3 -
lib/crypto/Kconfig | 2 +
lib/crypto/Makefile | 1 +
.../crypto/riscv}/sm3-riscv64-zvksh-zvkb.S | 3 +-
.../crypto/riscv/sm3.h | 84 +++----------------
6 files changed, 18 insertions(+), 88 deletions(-)
rename {arch/riscv/crypto => lib/crypto/riscv}/sm3-riscv64-zvksh-zvkb.S (97%)
rename arch/riscv/crypto/sm3-riscv64-glue.c => lib/crypto/riscv/sm3.h (18%)
diff --git a/arch/riscv/crypto/Kconfig b/arch/riscv/crypto/Kconfig
index 22d4eaab15f3..eefba937b015 100644
--- a/arch/riscv/crypto/Kconfig
+++ b/arch/riscv/crypto/Kconfig
@@ -26,23 +26,10 @@ config CRYPTO_GHASH_RISCV64
GCM GHASH function (NIST SP 800-38D)
Architecture: riscv64 using:
- Zvkg vector crypto extension
-config CRYPTO_SM3_RISCV64
- tristate "Hash functions: SM3 (ShangMi 3)"
- depends on 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
- RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
- select CRYPTO_HASH
- select CRYPTO_LIB_SM3
- help
- SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012)
-
- Architecture: riscv64 using:
- - Zvksh vector crypto extension
- - Zvkb vector crypto extension
-
config CRYPTO_SM4_RISCV64
tristate "Ciphers: SM4 (ShangMi 4)"
depends on 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
select CRYPTO_ALGAPI
diff --git a/arch/riscv/crypto/Makefile b/arch/riscv/crypto/Makefile
index 183495a95cc0..ca9a6c33ad53 100644
--- a/arch/riscv/crypto/Makefile
+++ b/arch/riscv/crypto/Makefile
@@ -5,10 +5,7 @@ aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \
aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o
obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o
ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o
-obj-$(CONFIG_CRYPTO_SM3_RISCV64) += sm3-riscv64.o
-sm3-riscv64-y := sm3-riscv64-glue.o sm3-riscv64-zvksh-zvkb.o
-
obj-$(CONFIG_CRYPTO_SM4_RISCV64) += sm4-riscv64.o
sm4-riscv64-y := sm4-riscv64-glue.o sm4-riscv64-zvksed-zvkb.o
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index a4e55b6a03af..b714f9cbd368 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -273,9 +273,11 @@ config CRYPTO_LIB_SM3
config CRYPTO_LIB_SM3_ARCH
bool
depends on CRYPTO_LIB_SM3 && !UML
default y if ARM64
+ default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
+ RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
source "lib/crypto/tests/Kconfig"
endmenu
diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index 48ed6ee5e3c9..3019e6cbb10d 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -353,10 +353,11 @@ obj-$(CONFIG_CRYPTO_LIB_SM3) += libsm3.o
libsm3-y := sm3.o
ifeq ($(CONFIG_CRYPTO_LIB_SM3_ARCH),y)
CFLAGS_sm3.o += -I$(src)/$(SRCARCH)
libsm3-$(CONFIG_ARM64) += arm64/sm3-ce-core.o \
arm64/sm3-neon-core.o
+libsm3-$(CONFIG_RISCV) += riscv/sm3-riscv64-zvksh-zvkb.o
endif # CONFIG_CRYPTO_LIB_SM3_ARCH
################################################################################
obj-$(CONFIG_MPILIB) += mpi/
diff --git a/arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S b/lib/crypto/riscv/sm3-riscv64-zvksh-zvkb.S
similarity index 97%
rename from arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S
rename to lib/crypto/riscv/sm3-riscv64-zvksh-zvkb.S
index 4fe754846f65..a1d4468b0485 100644
--- a/arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S
+++ b/lib/crypto/riscv/sm3-riscv64-zvksh-zvkb.S
@@ -78,11 +78,12 @@
vsm3me.vv \w0, \w1, \w0
.endif
// For the next 8 rounds, w0 and w1 are swapped.
.endm
-// void sm3_transform_zvksh_zvkb(u32 state[8], const u8 *data, int num_blocks);
+// void sm3_transform_zvksh_zvkb(struct sm3_block_state *state,
+// const u8 *data, size_t nblocks);
SYM_FUNC_START(sm3_transform_zvksh_zvkb)
// Load the state and endian-swap each 32-bit word.
vsetivli zero, 8, e32, m2, ta, ma
vle32.v STATE, (STATEP)
diff --git a/arch/riscv/crypto/sm3-riscv64-glue.c b/lib/crypto/riscv/sm3.h
similarity index 18%
rename from arch/riscv/crypto/sm3-riscv64-glue.c
rename to lib/crypto/riscv/sm3.h
index abdfe4a63a27..c1fbee7094e6 100644
--- a/arch/riscv/crypto/sm3-riscv64-glue.c
+++ b/lib/crypto/riscv/sm3.h
@@ -1,6 +1,6 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
+/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* SM3 using the RISC-V vector crypto extensions
*
* Copyright (C) 2023 VRULL GmbH
* Author: Heiko Stuebner <heiko.stuebner@vrull.eu>
@@ -9,89 +9,31 @@
* Author: Jerry Shih <jerry.shih@sifive.com>
*/
#include <asm/simd.h>
#include <asm/vector.h>
-#include <crypto/internal/hash.h>
-#include <crypto/internal/simd.h>
-#include <crypto/sm3.h>
-#include <crypto/sm3_base.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-/*
- * Note: the asm function only uses the 'state' field of struct sm3_state.
- * It is assumed to be the first field.
- */
-asmlinkage void sm3_transform_zvksh_zvkb(
- struct sm3_state *state, const u8 *data, int num_blocks);
+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_extensions);
-static void sm3_block(struct sm3_state *state, const u8 *data,
- int num_blocks)
-{
- /*
- * Ensure struct sm3_state begins directly with the SM3
- * 256-bit internal state, as this is what the asm function expects.
- */
- BUILD_BUG_ON(offsetof(struct sm3_state, state) != 0);
+asmlinkage void sm3_transform_zvksh_zvkb(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks);
- if (crypto_simd_usable()) {
+static void sm3_blocks(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks)
+{
+ if (static_branch_likely(&have_extensions) && likely(may_use_simd())) {
kernel_vector_begin();
- sm3_transform_zvksh_zvkb(state, data, num_blocks);
+ sm3_transform_zvksh_zvkb(state, data, nblocks);
kernel_vector_end();
} else {
- sm3_block_generic(state, data, num_blocks);
+ sm3_blocks_generic(state, data, nblocks);
}
}
-static int riscv64_sm3_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
-{
- return sm3_base_do_update_blocks(desc, data, len, sm3_block);
-}
-
-static int riscv64_sm3_finup(struct shash_desc *desc, const u8 *data,
- unsigned int len, u8 *out)
-{
- sm3_base_do_finup(desc, data, len, sm3_block);
- return sm3_base_finish(desc, out);
-}
-
-static struct shash_alg riscv64_sm3_alg = {
- .init = sm3_base_init,
- .update = riscv64_sm3_update,
- .finup = riscv64_sm3_finup,
- .descsize = SM3_STATE_SIZE,
- .digestsize = SM3_DIGEST_SIZE,
- .base = {
- .cra_blocksize = SM3_BLOCK_SIZE,
- .cra_flags = CRYPTO_AHASH_ALG_BLOCK_ONLY |
- CRYPTO_AHASH_ALG_FINUP_MAX,
- .cra_priority = 300,
- .cra_name = "sm3",
- .cra_driver_name = "sm3-riscv64-zvksh-zvkb",
- .cra_module = THIS_MODULE,
- },
-};
-
-static int __init riscv64_sm3_mod_init(void)
+#define sm3_mod_init_arch sm3_mod_init_arch
+static void sm3_mod_init_arch(void)
{
if (riscv_isa_extension_available(NULL, ZVKSH) &&
riscv_isa_extension_available(NULL, ZVKB) &&
riscv_vector_vlen() >= 128)
- return crypto_register_shash(&riscv64_sm3_alg);
-
- return -ENODEV;
-}
-
-static void __exit riscv64_sm3_mod_exit(void)
-{
- crypto_unregister_shash(&riscv64_sm3_alg);
+ static_branch_enable(&have_extensions);
}
-
-module_init(riscv64_sm3_mod_init);
-module_exit(riscv64_sm3_mod_exit);
-
-MODULE_DESCRIPTION("SM3 (RISC-V accelerated)");
-MODULE_AUTHOR("Heiko Stuebner <heiko.stuebner@vrull.eu>");
-MODULE_LICENSE("GPL");
-MODULE_ALIAS_CRYPTO("sm3");
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 09/12] lib/crypto: x86/sm3: Migrate optimized code into library
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (7 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 08/12] lib/crypto: riscv/sm3: " Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 10/12] crypto: sm3 - Remove sm3_base.h Eric Biggers
` (4 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Instead of exposing the x86-optimized SM3 code via an x86-specific
crypto_shash algorithm, instead just implement the sm3_blocks() library
function. This is much simpler, it makes the SM3 library functions be
x86-optimized, and it fixes the longstanding issue where the
x86-optimized SM3 code was disabled by default. SM3 still remains
available through crypto_shash, but individual architectures no longer
need to handle it.
Tweak the prototype of sm3_transform_avx() to match what the library
expects, including changing the block count to size_t. Note that the
assembly code actually already treated this argument as size_t.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
arch/x86/crypto/Kconfig | 13 ---
arch/x86/crypto/Makefile | 3 -
arch/x86/crypto/sm3_avx_glue.c | 100 ------------------
lib/crypto/Kconfig | 1 +
lib/crypto/Makefile | 1 +
.../crypto/x86}/sm3-avx-asm_64.S | 13 ++-
lib/crypto/x86/sm3.h | 39 +++++++
7 files changed, 47 insertions(+), 123 deletions(-)
delete mode 100644 arch/x86/crypto/sm3_avx_glue.c
rename {arch/x86/crypto => lib/crypto/x86}/sm3-avx-asm_64.S (98%)
create mode 100644 lib/crypto/x86/sm3.h
diff --git a/arch/x86/crypto/Kconfig b/arch/x86/crypto/Kconfig
index 7fb2319a0916..617494bd019f 100644
--- a/arch/x86/crypto/Kconfig
+++ b/arch/x86/crypto/Kconfig
@@ -329,23 +329,10 @@ config CRYPTO_AEGIS128_AESNI_SSE2
Architecture: x86_64 using:
- AES-NI (AES New Instructions)
- SSE4.1 (Streaming SIMD Extensions 4.1)
-config CRYPTO_SM3_AVX_X86_64
- tristate "Hash functions: SM3 (AVX)"
- depends on 64BIT
- select CRYPTO_HASH
- select CRYPTO_LIB_SM3
- help
- SM3 secure hash function as defined by OSCCA GM/T 0004-2012 SM3
-
- Architecture: x86_64 using:
- - AVX (Advanced Vector Extensions)
-
- If unsure, say N.
-
config CRYPTO_GHASH_CLMUL_NI_INTEL
tristate "Hash functions: GHASH (CLMUL-NI)"
depends on 64BIT
select CRYPTO_CRYPTD
help
diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile
index b21ad0978c52..9420b9ff51da 100644
--- a/arch/x86/crypto/Makefile
+++ b/arch/x86/crypto/Makefile
@@ -51,13 +51,10 @@ aesni-intel-$(CONFIG_64BIT) += aes-ctr-avx-x86_64.o \
aes-xts-avx-x86_64.o
obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o
ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o
-obj-$(CONFIG_CRYPTO_SM3_AVX_X86_64) += sm3-avx-x86_64.o
-sm3-avx-x86_64-y := sm3-avx-asm_64.o sm3_avx_glue.o
-
obj-$(CONFIG_CRYPTO_SM4_AESNI_AVX_X86_64) += sm4-aesni-avx-x86_64.o
sm4-aesni-avx-x86_64-y := sm4-aesni-avx-asm_64.o sm4_aesni_avx_glue.o
obj-$(CONFIG_CRYPTO_SM4_AESNI_AVX2_X86_64) += sm4-aesni-avx2-x86_64.o
sm4-aesni-avx2-x86_64-y := sm4-aesni-avx2-asm_64.o sm4_aesni_avx2_glue.o
diff --git a/arch/x86/crypto/sm3_avx_glue.c b/arch/x86/crypto/sm3_avx_glue.c
deleted file mode 100644
index 6e8c42b9dc8e..000000000000
--- a/arch/x86/crypto/sm3_avx_glue.c
+++ /dev/null
@@ -1,100 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * SM3 Secure Hash Algorithm, AVX assembler accelerated.
- * specified in: https://datatracker.ietf.org/doc/html/draft-sca-cfrg-sm3-02
- *
- * Copyright (C) 2021 Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <crypto/internal/hash.h>
-#include <crypto/internal/simd.h>
-#include <crypto/sm3.h>
-#include <crypto/sm3_base.h>
-#include <linux/cpufeature.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-
-asmlinkage void sm3_transform_avx(struct sm3_state *state,
- const u8 *data, int nblocks);
-
-static int sm3_avx_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
-{
- int remain;
-
- /*
- * Make sure struct sm3_state begins directly with the SM3
- * 256-bit internal state, as this is what the asm functions expect.
- */
- BUILD_BUG_ON(offsetof(struct sm3_state, state) != 0);
-
- kernel_fpu_begin();
- remain = sm3_base_do_update_blocks(desc, data, len, sm3_transform_avx);
- kernel_fpu_end();
- return remain;
-}
-
-static int sm3_avx_finup(struct shash_desc *desc, const u8 *data,
- unsigned int len, u8 *out)
-{
- kernel_fpu_begin();
- sm3_base_do_finup(desc, data, len, sm3_transform_avx);
- kernel_fpu_end();
- return sm3_base_finish(desc, out);
-}
-
-static struct shash_alg sm3_avx_alg = {
- .digestsize = SM3_DIGEST_SIZE,
- .init = sm3_base_init,
- .update = sm3_avx_update,
- .finup = sm3_avx_finup,
- .descsize = SM3_STATE_SIZE,
- .base = {
- .cra_name = "sm3",
- .cra_driver_name = "sm3-avx",
- .cra_priority = 300,
- .cra_flags = CRYPTO_AHASH_ALG_BLOCK_ONLY |
- CRYPTO_AHASH_ALG_FINUP_MAX,
- .cra_blocksize = SM3_BLOCK_SIZE,
- .cra_module = THIS_MODULE,
- }
-};
-
-static int __init sm3_avx_mod_init(void)
-{
- const char *feature_name;
-
- if (!boot_cpu_has(X86_FEATURE_AVX)) {
- pr_info("AVX instruction are not detected.\n");
- return -ENODEV;
- }
-
- if (!boot_cpu_has(X86_FEATURE_BMI2)) {
- pr_info("BMI2 instruction are not detected.\n");
- return -ENODEV;
- }
-
- if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM,
- &feature_name)) {
- pr_info("CPU feature '%s' is not supported.\n", feature_name);
- return -ENODEV;
- }
-
- return crypto_register_shash(&sm3_avx_alg);
-}
-
-static void __exit sm3_avx_mod_exit(void)
-{
- crypto_unregister_shash(&sm3_avx_alg);
-}
-
-module_init(sm3_avx_mod_init);
-module_exit(sm3_avx_mod_exit);
-
-MODULE_LICENSE("GPL v2");
-MODULE_AUTHOR("Tianjia Zhang <tianjia.zhang@linux.alibaba.com>");
-MODULE_DESCRIPTION("SM3 Secure Hash Algorithm, AVX assembler accelerated");
-MODULE_ALIAS_CRYPTO("sm3");
-MODULE_ALIAS_CRYPTO("sm3-avx");
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index b714f9cbd368..2824bfb0e30d 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -275,9 +275,10 @@ config CRYPTO_LIB_SM3_ARCH
bool
depends on CRYPTO_LIB_SM3 && !UML
default y if ARM64
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
+ default y if X86_64
source "lib/crypto/tests/Kconfig"
endmenu
diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile
index 3019e6cbb10d..308ec3e93b54 100644
--- a/lib/crypto/Makefile
+++ b/lib/crypto/Makefile
@@ -354,10 +354,11 @@ libsm3-y := sm3.o
ifeq ($(CONFIG_CRYPTO_LIB_SM3_ARCH),y)
CFLAGS_sm3.o += -I$(src)/$(SRCARCH)
libsm3-$(CONFIG_ARM64) += arm64/sm3-ce-core.o \
arm64/sm3-neon-core.o
libsm3-$(CONFIG_RISCV) += riscv/sm3-riscv64-zvksh-zvkb.o
+libsm3-$(CONFIG_X86) += x86/sm3-avx-asm_64.o
endif # CONFIG_CRYPTO_LIB_SM3_ARCH
################################################################################
obj-$(CONFIG_MPILIB) += mpi/
diff --git a/arch/x86/crypto/sm3-avx-asm_64.S b/lib/crypto/x86/sm3-avx-asm_64.S
similarity index 98%
rename from arch/x86/crypto/sm3-avx-asm_64.S
rename to lib/crypto/x86/sm3-avx-asm_64.S
index 503bab450a91..a1925b136010 100644
--- a/arch/x86/crypto/sm3-avx-asm_64.S
+++ b/lib/crypto/x86/sm3-avx-asm_64.S
@@ -10,14 +10,13 @@
/* Based on SM3 AES/BMI2 accelerated work by libgcrypt at:
* https://gnupg.org/software/libgcrypt/index.html
*/
#include <linux/linkage.h>
-#include <linux/cfi_types.h>
#include <asm/frame.h>
-/* Context structure */
+/* State structure */
#define state_h0 0
#define state_h1 4
#define state_h2 8
#define state_h3 12
@@ -323,17 +322,17 @@
.text
/*
* Transform nblocks*64 bytes (nblocks*16 32-bit words) at DATA.
*
- * void sm3_transform_avx(struct sm3_state *state,
- * const u8 *data, int nblocks);
+ * void sm3_transform_avx(struct sm3_block_state *state,
+ * const u8 *data, size_t nblocks);
*/
-SYM_TYPED_FUNC_START(sm3_transform_avx)
+SYM_FUNC_START(sm3_transform_avx)
/* input:
- * %rdi: ctx, CTX
- * %rsi: data (64*nblks bytes)
+ * %rdi: state
+ * %rsi: data
* %rdx: nblocks
*/
vzeroupper;
pushq %rbp;
diff --git a/lib/crypto/x86/sm3.h b/lib/crypto/x86/sm3.h
new file mode 100644
index 000000000000..3834780f2f6a
--- /dev/null
+++ b/lib/crypto/x86/sm3.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * SM3 optimized for x86_64
+ *
+ * Copyright 2026 Google LLC
+ */
+#include <asm/fpu/api.h>
+#include <linux/static_call.h>
+
+asmlinkage void sm3_transform_avx(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks);
+
+static void sm3_blocks_avx(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks)
+{
+ if (likely(irq_fpu_usable())) {
+ kernel_fpu_begin();
+ sm3_transform_avx(state, data, nblocks);
+ kernel_fpu_end();
+ } else {
+ sm3_blocks_generic(state, data, nblocks);
+ }
+}
+
+DEFINE_STATIC_CALL(sm3_blocks_x86, sm3_blocks_generic);
+
+static void sm3_blocks(struct sm3_block_state *state,
+ const u8 *data, size_t nblocks)
+{
+ static_call(sm3_blocks_x86)(state, data, nblocks);
+}
+
+#define sm3_mod_init_arch sm3_mod_init_arch
+static void sm3_mod_init_arch(void)
+{
+ if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2) &&
+ cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL))
+ static_call_update(sm3_blocks_x86, sm3_blocks_avx);
+}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 10/12] crypto: sm3 - Remove sm3_base.h
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (8 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 09/12] lib/crypto: x86/sm3: " Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 11/12] crypto: sm3 - Remove the original "sm3_block_generic()" Eric Biggers
` (3 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Remove include/crypto/sm3_base.h, since it's no longer used. The
corresponding logic was reimplemented in a central place in lib/crypto/.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
include/crypto/sm3_base.h | 92 ---------------------------------------
1 file changed, 92 deletions(-)
delete mode 100644 include/crypto/sm3_base.h
diff --git a/include/crypto/sm3_base.h b/include/crypto/sm3_base.h
deleted file mode 100644
index 9fa995617495..000000000000
--- a/include/crypto/sm3_base.h
+++ /dev/null
@@ -1,92 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * sm3_base.h - core logic for SM3 implementations
- *
- * Copyright (C) 2017 ARM Limited or its affiliates.
- * Written by Gilad Ben-Yossef <gilad@benyossef.com>
- */
-
-#ifndef _CRYPTO_SM3_BASE_H
-#define _CRYPTO_SM3_BASE_H
-
-#include <crypto/internal/hash.h>
-#include <crypto/sm3.h>
-#include <linux/math.h>
-#include <linux/module.h>
-#include <linux/string.h>
-#include <linux/types.h>
-#include <linux/unaligned.h>
-
-typedef void (sm3_block_fn)(struct sm3_state *sst, u8 const *src, int blocks);
-
-static inline int sm3_base_init(struct shash_desc *desc)
-{
- struct sm3_state *sctx = shash_desc_ctx(desc);
-
- sctx->state[0] = SM3_IVA;
- sctx->state[1] = SM3_IVB;
- sctx->state[2] = SM3_IVC;
- sctx->state[3] = SM3_IVD;
- sctx->state[4] = SM3_IVE;
- sctx->state[5] = SM3_IVF;
- sctx->state[6] = SM3_IVG;
- sctx->state[7] = SM3_IVH;
- sctx->count = 0;
- return 0;
-}
-
-static inline int sm3_base_do_update_blocks(struct shash_desc *desc,
- const u8 *data, unsigned int len,
- sm3_block_fn *block_fn)
-{
- unsigned int remain = len - round_down(len, SM3_BLOCK_SIZE);
- struct sm3_state *sctx = shash_desc_ctx(desc);
-
- sctx->count += len - remain;
- block_fn(sctx, data, len / SM3_BLOCK_SIZE);
- return remain;
-}
-
-static inline int sm3_base_do_finup(struct shash_desc *desc,
- const u8 *src, unsigned int len,
- sm3_block_fn *block_fn)
-{
- unsigned int bit_offset = SM3_BLOCK_SIZE / 8 - 1;
- struct sm3_state *sctx = shash_desc_ctx(desc);
- union {
- __be64 b64[SM3_BLOCK_SIZE / 4];
- u8 u8[SM3_BLOCK_SIZE * 2];
- } block = {};
-
- if (len >= SM3_BLOCK_SIZE) {
- int remain;
-
- remain = sm3_base_do_update_blocks(desc, src, len, block_fn);
- src += len - remain;
- len = remain;
- }
-
- if (len >= bit_offset * 8)
- bit_offset += SM3_BLOCK_SIZE / 8;
- memcpy(&block, src, len);
- block.u8[len] = 0x80;
- sctx->count += len;
- block.b64[bit_offset] = cpu_to_be64(sctx->count << 3);
- block_fn(sctx, block.u8, (bit_offset + 1) * 8 / SM3_BLOCK_SIZE);
- memzero_explicit(&block, sizeof(block));
-
- return 0;
-}
-
-static inline int sm3_base_finish(struct shash_desc *desc, u8 *out)
-{
- struct sm3_state *sctx = shash_desc_ctx(desc);
- __be32 *digest = (__be32 *)out;
- int i;
-
- for (i = 0; i < SM3_DIGEST_SIZE / sizeof(__be32); i++)
- put_unaligned_be32(sctx->state[i], digest++);
- return 0;
-}
-
-#endif /* _CRYPTO_SM3_BASE_H */
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 11/12] crypto: sm3 - Remove the original "sm3_block_generic()"
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (9 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 10/12] crypto: sm3 - Remove sm3_base.h Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-21 4:09 ` [PATCH 12/12] crypto: sm3 - Remove 'struct sm3_state' Eric Biggers
` (2 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Since the architecture-optimized SM3 code was migrated into lib/crypto/,
sm3_block_generic() is no longer called. Remove it. Then, since this
frees up the name, rename sm3_transform() to sm3_block_generic()
(matching the naming convention used in other hash algorithms).
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
include/crypto/sm3.h | 2 --
lib/crypto/sm3.c | 19 +++----------------
2 files changed, 3 insertions(+), 18 deletions(-)
diff --git a/include/crypto/sm3.h b/include/crypto/sm3.h
index 702c5326b4be..34d7eb32b7db 100644
--- a/include/crypto/sm3.h
+++ b/include/crypto/sm3.h
@@ -29,12 +29,10 @@ struct sm3_state {
u32 state[SM3_DIGEST_SIZE / 4];
u64 count;
u8 buffer[SM3_BLOCK_SIZE];
};
-void sm3_block_generic(struct sm3_state *sctx, u8 const *data, int blocks);
-
/* State for the SM3 compression function */
struct sm3_block_state {
u32 h[SM3_DIGEST_SIZE / 4];
};
diff --git a/lib/crypto/sm3.c b/lib/crypto/sm3.c
index 20500cf4b8c0..b02b8a247adf 100644
--- a/lib/crypto/sm3.c
+++ b/lib/crypto/sm3.c
@@ -77,12 +77,12 @@ static const u32 ____cacheline_aligned K[64] = {
^ W[(i-9) & 0x0f] \
^ rol32(W[(i-3) & 0x0f], 15)) \
^ rol32(W[(i-13) & 0x0f], 7) \
^ W[(i-6) & 0x0f])
-static void sm3_transform(struct sm3_block_state *state,
- const u8 data[SM3_BLOCK_SIZE], u32 W[16])
+static void sm3_block_generic(struct sm3_block_state *state,
+ const u8 data[SM3_BLOCK_SIZE], u32 W[16])
{
u32 a, b, c, d, e, f, g, h, ss1, ss2;
a = state->h[0];
b = state->h[1];
@@ -175,30 +175,17 @@ static void sm3_transform(struct sm3_block_state *state,
#undef R2
#undef I
#undef W1
#undef W2
-void sm3_block_generic(struct sm3_state *sctx, u8 const *data, int blocks)
-{
- u32 W[16];
-
- do {
- sm3_transform((struct sm3_block_state *)sctx->state, data, W);
- data += SM3_BLOCK_SIZE;
- } while (--blocks);
-
- memzero_explicit(W, sizeof(W));
-}
-EXPORT_SYMBOL_GPL(sm3_block_generic);
-
static void __maybe_unused sm3_blocks_generic(struct sm3_block_state *state,
const u8 *data, size_t nblocks)
{
u32 W[16];
do {
- sm3_transform(state, data, W);
+ sm3_block_generic(state, data, W);
data += SM3_BLOCK_SIZE;
} while (--nblocks);
memzero_explicit(W, sizeof(W));
}
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 12/12] crypto: sm3 - Remove 'struct sm3_state'
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (10 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 11/12] crypto: sm3 - Remove the original "sm3_block_generic()" Eric Biggers
@ 2026-03-21 4:09 ` Eric Biggers
2026-03-23 14:13 ` [PATCH 00/12] SM3 library Ard Biesheuvel
2026-03-24 23:27 ` Eric Biggers
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-21 4:09 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86, Eric Biggers
Update one driver that used sizeof(struct sm3_state) to use
sizeof(struct sm3_ctx) instead. Then, remove struct sm3_state and
SM3_STATE_SIZE.
This completes the replacement of struct sm3_state with struct sm3_ctx.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---
drivers/crypto/starfive/jh7110-hash.c | 4 ++--
include/crypto/sm3.h | 7 -------
2 files changed, 2 insertions(+), 9 deletions(-)
diff --git a/drivers/crypto/starfive/jh7110-hash.c b/drivers/crypto/starfive/jh7110-hash.c
index 742038a5201a..008a47baa165 100644
--- a/drivers/crypto/starfive/jh7110-hash.c
+++ b/drivers/crypto/starfive/jh7110-hash.c
@@ -793,11 +793,11 @@ static struct ahash_engine_alg algs_sha2_sm3[] = {
.base.import = starfive_hash_import,
.base.init_tfm = starfive_sm3_init_tfm,
.base.exit_tfm = starfive_hash_exit_tfm,
.base.halg = {
.digestsize = SM3_DIGEST_SIZE,
- .statesize = sizeof(struct sm3_state),
+ .statesize = sizeof(struct sm3_ctx),
.base = {
.cra_name = "sm3",
.cra_driver_name = "sm3-starfive",
.cra_priority = 200,
.cra_flags = CRYPTO_ALG_ASYNC |
@@ -822,11 +822,11 @@ static struct ahash_engine_alg algs_sha2_sm3[] = {
.base.init_tfm = starfive_hmac_sm3_init_tfm,
.base.exit_tfm = starfive_hash_exit_tfm,
.base.setkey = starfive_hash_setkey,
.base.halg = {
.digestsize = SM3_DIGEST_SIZE,
- .statesize = sizeof(struct sm3_state),
+ .statesize = sizeof(struct sm3_ctx),
.base = {
.cra_name = "hmac(sm3)",
.cra_driver_name = "sm3-hmac-starfive",
.cra_priority = 200,
.cra_flags = CRYPTO_ALG_ASYNC |
diff --git a/include/crypto/sm3.h b/include/crypto/sm3.h
index 34d7eb32b7db..371e8a661705 100644
--- a/include/crypto/sm3.h
+++ b/include/crypto/sm3.h
@@ -12,27 +12,20 @@
#include <linux/types.h>
#define SM3_DIGEST_SIZE 32
#define SM3_BLOCK_SIZE 64
-#define SM3_STATE_SIZE 40
#define SM3_IVA 0x7380166f
#define SM3_IVB 0x4914b2b9
#define SM3_IVC 0x172442d7
#define SM3_IVD 0xda8a0600
#define SM3_IVE 0xa96f30bc
#define SM3_IVF 0x163138aa
#define SM3_IVG 0xe38dee4d
#define SM3_IVH 0xb0fb0e4e
-struct sm3_state {
- u32 state[SM3_DIGEST_SIZE / 4];
- u64 count;
- u8 buffer[SM3_BLOCK_SIZE];
-};
-
/* State for the SM3 compression function */
struct sm3_block_state {
u32 h[SM3_DIGEST_SIZE / 4];
};
--
2.53.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 00/12] SM3 library
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (11 preceding siblings ...)
2026-03-21 4:09 ` [PATCH 12/12] crypto: sm3 - Remove 'struct sm3_state' Eric Biggers
@ 2026-03-23 14:13 ` Ard Biesheuvel
2026-03-24 23:27 ` Eric Biggers
13 siblings, 0 replies; 15+ messages in thread
From: Ard Biesheuvel @ 2026-03-23 14:13 UTC (permalink / raw)
To: Eric Biggers, linux-crypto
Cc: linux-kernel, Jason A . Donenfeld, Herbert Xu, Tianjia Zhang,
linux-arm-kernel, linux-riscv, x86
On Sat, 21 Mar 2026, at 05:09, Eric Biggers wrote:
> This series is targeting libcrypto-next. It can also be retrieved from:
>
> git fetch
> https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git
> sm3-lib-v1
>
> This series cleans up the kernel's existing SM3 hashing code:
>
> - First, it updates lib/crypto/sm3.c to implement the full SM3 instead
> of just SM3's compression function.
>
> - Next, it adds a KUnit test suite for the new library API.
>
> - Next, it replaces the "sm3-generic" crypto_shash with a wrapper around
> the new library API.
>
> - Finally, it accelerates the API using the existing SM3 assembly code
> for arm64, riscv, and x86. The architecture-specific crypto_shash
> glue code for SM3 is no longer needed and is removed.
>
> This should look quite boring. It's the same cleanup that I've already
> done for the other hash functions.
>
> Note: I don't recommend using SM3. There also don't appear to be any
> immediate candidate users of the SM3 library other than crypto_shash.
>
> Still, this seems like the clear way to go. It's simpler, and it gets
> the hash algorithms integrated in a consistent way. We won't have to
> keep track of two quite different ways of doing things. With KUnit the
> code becomes much easier to test and benchmark, as well.
>
> Eric Biggers (12):
> crypto: sm3 - Fold sm3_init() into its caller
> crypto: sm3 - Remove sm3_zero_message_hash and SM3_T[1-2]
> crypto: sm3 - Rename CRYPTO_SM3_GENERIC to CRYPTO_SM3
> lib/crypto: sm3: Add SM3 library API
> lib/crypto: tests: Add KUnit tests for SM3
> crypto: sm3 - Replace with wrapper around library
> lib/crypto: arm64/sm3: Migrate optimized code into library
> lib/crypto: riscv/sm3: Migrate optimized code into library
> lib/crypto: x86/sm3: Migrate optimized code into library
> crypto: sm3 - Remove sm3_base.h
> crypto: sm3 - Remove the original "sm3_block_generic()"
> crypto: sm3 - Remove 'struct sm3_state'
>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 00/12] SM3 library
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
` (12 preceding siblings ...)
2026-03-23 14:13 ` [PATCH 00/12] SM3 library Ard Biesheuvel
@ 2026-03-24 23:27 ` Eric Biggers
13 siblings, 0 replies; 15+ messages in thread
From: Eric Biggers @ 2026-03-24 23:27 UTC (permalink / raw)
To: linux-crypto
Cc: linux-kernel, Ard Biesheuvel, Jason A . Donenfeld, Herbert Xu,
Tianjia Zhang, linux-arm-kernel, linux-riscv, x86
On Fri, Mar 20, 2026 at 09:09:23PM -0700, Eric Biggers wrote:
> This series is targeting libcrypto-next. It can also be retrieved from:
>
> git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git sm3-lib-v1
>
> This series cleans up the kernel's existing SM3 hashing code:
>
> - First, it updates lib/crypto/sm3.c to implement the full SM3 instead
> of just SM3's compression function.
>
> - Next, it adds a KUnit test suite for the new library API.
>
> - Next, it replaces the "sm3-generic" crypto_shash with a wrapper around
> the new library API.
>
> - Finally, it accelerates the API using the existing SM3 assembly code
> for arm64, riscv, and x86. The architecture-specific crypto_shash
> glue code for SM3 is no longer needed and is removed.
>
> This should look quite boring. It's the same cleanup that I've already
> done for the other hash functions.
>
> Note: I don't recommend using SM3. There also don't appear to be any
> immediate candidate users of the SM3 library other than crypto_shash.
>
> Still, this seems like the clear way to go. It's simpler, and it gets
> the hash algorithms integrated in a consistent way. We won't have to
> keep track of two quite different ways of doing things. With KUnit the
> code becomes much easier to test and benchmark, as well.
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next
- Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-03-24 23:27 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-03-21 4:09 [PATCH 00/12] SM3 library Eric Biggers
2026-03-21 4:09 ` [PATCH 01/12] crypto: sm3 - Fold sm3_init() into its caller Eric Biggers
2026-03-21 4:09 ` [PATCH 02/12] crypto: sm3 - Remove sm3_zero_message_hash and SM3_T[1-2] Eric Biggers
2026-03-21 4:09 ` [PATCH 03/12] crypto: sm3 - Rename CRYPTO_SM3_GENERIC to CRYPTO_SM3 Eric Biggers
2026-03-21 4:09 ` [PATCH 04/12] lib/crypto: sm3: Add SM3 library API Eric Biggers
2026-03-21 4:09 ` [PATCH 05/12] lib/crypto: tests: Add KUnit tests for SM3 Eric Biggers
2026-03-21 4:09 ` [PATCH 06/12] crypto: sm3 - Replace with wrapper around library Eric Biggers
2026-03-21 4:09 ` [PATCH 07/12] lib/crypto: arm64/sm3: Migrate optimized code into library Eric Biggers
2026-03-21 4:09 ` [PATCH 08/12] lib/crypto: riscv/sm3: " Eric Biggers
2026-03-21 4:09 ` [PATCH 09/12] lib/crypto: x86/sm3: " Eric Biggers
2026-03-21 4:09 ` [PATCH 10/12] crypto: sm3 - Remove sm3_base.h Eric Biggers
2026-03-21 4:09 ` [PATCH 11/12] crypto: sm3 - Remove the original "sm3_block_generic()" Eric Biggers
2026-03-21 4:09 ` [PATCH 12/12] crypto: sm3 - Remove 'struct sm3_state' Eric Biggers
2026-03-23 14:13 ` [PATCH 00/12] SM3 library Ard Biesheuvel
2026-03-24 23:27 ` Eric Biggers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox