* [PATCH 0/2] crypto: implement SM4 for arm64 using special instructions
@ 2018-04-25 12:20 Ard Biesheuvel
2018-04-25 12:20 ` [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers Ard Biesheuvel
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Ard Biesheuvel @ 2018-04-25 12:20 UTC (permalink / raw)
To: linux-arm-kernel
Patch #1 makes some preparatory changes so the C routines can be used as
a fallback by other drivers.
Patch #2 implements the SM4 core cipher using the special instructions
introduced as an optional extension by revision 8.2 of the ARM architecture.
Note that this does not implement cipher+chaining mode combinations as we
do for AES. This can be added later if desiresd.
Ard Biesheuvel (2):
crypto: sm4 - export encrypt/decrypt routines to other drivers
crypto: arm64 - add support for SM4 encryption using special
instructions
arch/arm64/crypto/Kconfig | 6 ++
arch/arm64/crypto/Makefile | 3 +
arch/arm64/crypto/sm4-ce-core.S | 36 ++++++++++
arch/arm64/crypto/sm4-ce-glue.c | 73 ++++++++++++++++++++
crypto/sm4_generic.c | 10 +--
include/crypto/sm4.h | 3 +
6 files changed, 127 insertions(+), 4 deletions(-)
create mode 100644 arch/arm64/crypto/sm4-ce-core.S
create mode 100644 arch/arm64/crypto/sm4-ce-glue.c
--
2.17.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers 2018-04-25 12:20 [PATCH 0/2] crypto: implement SM4 for arm64 using special instructions Ard Biesheuvel @ 2018-04-25 12:20 ` Ard Biesheuvel 2018-04-25 12:23 ` Ard Biesheuvel 2018-04-25 12:47 ` Gilad Ben-Yossef 2018-04-25 12:20 ` [PATCH 2/2] crypto: arm64 - add support for SM4 encryption using special instructions Ard Biesheuvel 2018-05-05 7:17 ` [PATCH 0/2] crypto: implement SM4 for arm64 " Herbert Xu 2 siblings, 2 replies; 6+ messages in thread From: Ard Biesheuvel @ 2018-04-25 12:20 UTC (permalink / raw) To: linux-arm-kernel In preparation of adding support for the SIMD based arm64 implementation of arm64, which requires a fallback to non-SIMD code when invoked in certain contexts, expose the generic SM4 encrypt and decrypt routines to other drivers. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- crypto/sm4_generic.c | 10 ++++++---- include/crypto/sm4.h | 3 +++ 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/crypto/sm4_generic.c b/crypto/sm4_generic.c index f537a2766c55..c18eebfd5edd 100644 --- a/crypto/sm4_generic.c +++ b/crypto/sm4_generic.c @@ -190,21 +190,23 @@ static void sm4_do_crypt(const u32 *rk, u32 *out, const u32 *in) /* encrypt a block of text */ -static void sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +void crypto_sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); sm4_do_crypt(ctx->rkey_enc, (u32 *)out, (u32 *)in); } +EXPORT_SYMBOL_GPL(crypto_sm4_encrypt); /* decrypt a block of text */ -static void sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +void crypto_sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); sm4_do_crypt(ctx->rkey_dec, (u32 *)out, (u32 *)in); } +EXPORT_SYMBOL_GPL(crypto_sm4_decrypt); static struct crypto_alg sm4_alg = { .cra_name = "sm4", @@ -219,8 +221,8 @@ static struct crypto_alg sm4_alg = { .cia_min_keysize = SM4_KEY_SIZE, .cia_max_keysize = SM4_KEY_SIZE, .cia_setkey = crypto_sm4_set_key, - .cia_encrypt = sm4_encrypt, - .cia_decrypt = sm4_decrypt + .cia_encrypt = crypto_sm4_encrypt, + .cia_decrypt = crypto_sm4_decrypt } } }; diff --git a/include/crypto/sm4.h b/include/crypto/sm4.h index b64e64d20b28..7afd730d16ff 100644 --- a/include/crypto/sm4.h +++ b/include/crypto/sm4.h @@ -25,4 +25,7 @@ int crypto_sm4_set_key(struct crypto_tfm *tfm, const u8 *in_key, int crypto_sm4_expand_key(struct crypto_sm4_ctx *ctx, const u8 *in_key, unsigned int key_len); +void crypto_sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in); +void crypto_sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in); + #endif -- 2.17.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers 2018-04-25 12:20 ` [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers Ard Biesheuvel @ 2018-04-25 12:23 ` Ard Biesheuvel 2018-04-25 12:47 ` Gilad Ben-Yossef 1 sibling, 0 replies; 6+ messages in thread From: Ard Biesheuvel @ 2018-04-25 12:23 UTC (permalink / raw) To: linux-arm-kernel On 25 April 2018 at 14:20, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > In preparation of adding support for the SIMD based arm64 implementation > of arm64, SM4 ^^^ > which requires a fallback to non-SIMD code when invoked in > certain contexts, expose the generic SM4 encrypt and decrypt routines > to other drivers. > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > --- > crypto/sm4_generic.c | 10 ++++++---- > include/crypto/sm4.h | 3 +++ > 2 files changed, 9 insertions(+), 4 deletions(-) > > diff --git a/crypto/sm4_generic.c b/crypto/sm4_generic.c > index f537a2766c55..c18eebfd5edd 100644 > --- a/crypto/sm4_generic.c > +++ b/crypto/sm4_generic.c > @@ -190,21 +190,23 @@ static void sm4_do_crypt(const u32 *rk, u32 *out, const u32 *in) > > /* encrypt a block of text */ > > -static void sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > +void crypto_sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > { > const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); > > sm4_do_crypt(ctx->rkey_enc, (u32 *)out, (u32 *)in); > } > +EXPORT_SYMBOL_GPL(crypto_sm4_encrypt); > > /* decrypt a block of text */ > > -static void sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > +void crypto_sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > { > const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); > > sm4_do_crypt(ctx->rkey_dec, (u32 *)out, (u32 *)in); > } > +EXPORT_SYMBOL_GPL(crypto_sm4_decrypt); > > static struct crypto_alg sm4_alg = { > .cra_name = "sm4", > @@ -219,8 +221,8 @@ static struct crypto_alg sm4_alg = { > .cia_min_keysize = SM4_KEY_SIZE, > .cia_max_keysize = SM4_KEY_SIZE, > .cia_setkey = crypto_sm4_set_key, > - .cia_encrypt = sm4_encrypt, > - .cia_decrypt = sm4_decrypt > + .cia_encrypt = crypto_sm4_encrypt, > + .cia_decrypt = crypto_sm4_decrypt > } > } > }; > diff --git a/include/crypto/sm4.h b/include/crypto/sm4.h > index b64e64d20b28..7afd730d16ff 100644 > --- a/include/crypto/sm4.h > +++ b/include/crypto/sm4.h > @@ -25,4 +25,7 @@ int crypto_sm4_set_key(struct crypto_tfm *tfm, const u8 *in_key, > int crypto_sm4_expand_key(struct crypto_sm4_ctx *ctx, const u8 *in_key, > unsigned int key_len); > > +void crypto_sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in); > +void crypto_sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in); > + > #endif > -- > 2.17.0 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers 2018-04-25 12:20 ` [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers Ard Biesheuvel 2018-04-25 12:23 ` Ard Biesheuvel @ 2018-04-25 12:47 ` Gilad Ben-Yossef 1 sibling, 0 replies; 6+ messages in thread From: Gilad Ben-Yossef @ 2018-04-25 12:47 UTC (permalink / raw) To: linux-arm-kernel On Wed, Apr 25, 2018 at 3:20 PM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > In preparation of adding support for the SIMD based arm64 implementation > of arm64, which requires a fallback to non-SIMD code when invoked in > certain contexts, expose the generic SM4 encrypt and decrypt routines > to other drivers. > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Gilad Ben-Yossef <gilad@benyossef.com> > --- > crypto/sm4_generic.c | 10 ++++++---- > include/crypto/sm4.h | 3 +++ > 2 files changed, 9 insertions(+), 4 deletions(-) > > diff --git a/crypto/sm4_generic.c b/crypto/sm4_generic.c > index f537a2766c55..c18eebfd5edd 100644 > --- a/crypto/sm4_generic.c > +++ b/crypto/sm4_generic.c > @@ -190,21 +190,23 @@ static void sm4_do_crypt(const u32 *rk, u32 *out, const u32 *in) > > /* encrypt a block of text */ > > -static void sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > +void crypto_sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > { > const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); > > sm4_do_crypt(ctx->rkey_enc, (u32 *)out, (u32 *)in); > } > +EXPORT_SYMBOL_GPL(crypto_sm4_encrypt); > > /* decrypt a block of text */ > > -static void sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > +void crypto_sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) > { > const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); > > sm4_do_crypt(ctx->rkey_dec, (u32 *)out, (u32 *)in); > } > +EXPORT_SYMBOL_GPL(crypto_sm4_decrypt); > > static struct crypto_alg sm4_alg = { > .cra_name = "sm4", > @@ -219,8 +221,8 @@ static struct crypto_alg sm4_alg = { > .cia_min_keysize = SM4_KEY_SIZE, > .cia_max_keysize = SM4_KEY_SIZE, > .cia_setkey = crypto_sm4_set_key, > - .cia_encrypt = sm4_encrypt, > - .cia_decrypt = sm4_decrypt > + .cia_encrypt = crypto_sm4_encrypt, > + .cia_decrypt = crypto_sm4_decrypt > } > } > }; > diff --git a/include/crypto/sm4.h b/include/crypto/sm4.h > index b64e64d20b28..7afd730d16ff 100644 > --- a/include/crypto/sm4.h > +++ b/include/crypto/sm4.h > @@ -25,4 +25,7 @@ int crypto_sm4_set_key(struct crypto_tfm *tfm, const u8 *in_key, > int crypto_sm4_expand_key(struct crypto_sm4_ctx *ctx, const u8 *in_key, > unsigned int key_len); > > +void crypto_sm4_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in); > +void crypto_sm4_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in); > + > #endif > -- > 2.17.0 > -- Gilad Ben-Yossef Chief Coffee Drinker "If you take a class in large-scale robotics, can you end up in a situation where the homework eats your dog?" -- Jean-Baptiste Queru ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] crypto: arm64 - add support for SM4 encryption using special instructions 2018-04-25 12:20 [PATCH 0/2] crypto: implement SM4 for arm64 using special instructions Ard Biesheuvel 2018-04-25 12:20 ` [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers Ard Biesheuvel @ 2018-04-25 12:20 ` Ard Biesheuvel 2018-05-05 7:17 ` [PATCH 0/2] crypto: implement SM4 for arm64 " Herbert Xu 2 siblings, 0 replies; 6+ messages in thread From: Ard Biesheuvel @ 2018-04-25 12:20 UTC (permalink / raw) To: linux-arm-kernel Add support for the SM4 symmetric cipher implemented using the special SM4 instructions introduced in ARM architecture revision 8.2. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/arm64/crypto/Kconfig | 6 ++ arch/arm64/crypto/Makefile | 3 + arch/arm64/crypto/sm4-ce-core.S | 36 ++++++++++ arch/arm64/crypto/sm4-ce-glue.c | 73 ++++++++++++++++++++ 4 files changed, 118 insertions(+) diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index cb5a243110c4..e3fdb0fd6f70 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -47,6 +47,12 @@ config CRYPTO_SM3_ARM64_CE select CRYPTO_HASH select CRYPTO_SM3 +config CRYPTO_SM4_ARM64_CE + tristate "SM4 symmetric cipher (ARMv8.2 Crypto Extensions)" + depends on KERNEL_MODE_NEON + select CRYPTO_ALGAPI + select CRYPTO_SM4 + config CRYPTO_GHASH_ARM64_CE tristate "GHASH/AES-GCM using ARMv8 Crypto Extensions" depends on KERNEL_MODE_NEON diff --git a/arch/arm64/crypto/Makefile b/arch/arm64/crypto/Makefile index f35ac684b1c0..bcafd016618e 100644 --- a/arch/arm64/crypto/Makefile +++ b/arch/arm64/crypto/Makefile @@ -23,6 +23,9 @@ sha3-ce-y := sha3-ce-glue.o sha3-ce-core.o obj-$(CONFIG_CRYPTO_SM3_ARM64_CE) += sm3-ce.o sm3-ce-y := sm3-ce-glue.o sm3-ce-core.o +obj-$(CONFIG_CRYPTO_SM4_ARM64_CE) += sm4-ce.o +sm4-ce-y := sm4-ce-glue.o sm4-ce-core.o + obj-$(CONFIG_CRYPTO_GHASH_ARM64_CE) += ghash-ce.o ghash-ce-y := ghash-ce-glue.o ghash-ce-core.o diff --git a/arch/arm64/crypto/sm4-ce-core.S b/arch/arm64/crypto/sm4-ce-core.S new file mode 100644 index 000000000000..af3bfbc3f4d4 --- /dev/null +++ b/arch/arm64/crypto/sm4-ce-core.S @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/linkage.h> +#include <asm/assembler.h> + + .irp b, 0, 1, 2, 3, 4, 5, 6, 7, 8 + .set .Lv\b\().4s, \b + .endr + + .macro sm4e, rd, rn + .inst 0xcec08400 | .L\rd | (.L\rn << 5) + .endm + + /* + * void sm4_ce_do_crypt(const u32 *rk, u32 *out, const u32 *in); + */ + .text +ENTRY(sm4_ce_do_crypt) + ld1 {v8.4s}, [x2] + ld1 {v0.4s-v3.4s}, [x0], #64 +CPU_LE( rev32 v8.16b, v8.16b ) + ld1 {v4.4s-v7.4s}, [x0] + sm4e v8.4s, v0.4s + sm4e v8.4s, v1.4s + sm4e v8.4s, v2.4s + sm4e v8.4s, v3.4s + sm4e v8.4s, v4.4s + sm4e v8.4s, v5.4s + sm4e v8.4s, v6.4s + sm4e v8.4s, v7.4s + rev64 v8.4s, v8.4s + ext v8.16b, v8.16b, v8.16b, #8 +CPU_LE( rev32 v8.16b, v8.16b ) + st1 {v8.4s}, [x1] + ret +ENDPROC(sm4_ce_do_crypt) diff --git a/arch/arm64/crypto/sm4-ce-glue.c b/arch/arm64/crypto/sm4-ce-glue.c new file mode 100644 index 000000000000..b7fb5274b250 --- /dev/null +++ b/arch/arm64/crypto/sm4-ce-glue.c @@ -0,0 +1,73 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <asm/neon.h> +#include <asm/simd.h> +#include <crypto/sm4.h> +#include <linux/module.h> +#include <linux/cpufeature.h> +#include <linux/crypto.h> +#include <linux/types.h> + +MODULE_ALIAS_CRYPTO("sm4"); +MODULE_ALIAS_CRYPTO("sm4-ce"); +MODULE_DESCRIPTION("SM4 symmetric cipher using ARMv8 Crypto Extensions"); +MODULE_AUTHOR("Ard Biesheuvel <ard.biesheuvel@linaro.org>"); +MODULE_LICENSE("GPL v2"); + +asmlinkage void sm4_ce_do_crypt(const u32 *rk, void *out, const void *in); + +static void sm4_ce_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +{ + const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); + + if (!may_use_simd()) { + crypto_sm4_encrypt(tfm, out, in); + } else { + kernel_neon_begin(); + sm4_ce_do_crypt(ctx->rkey_enc, out, in); + kernel_neon_end(); + } +} + +static void sm4_ce_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +{ + const struct crypto_sm4_ctx *ctx = crypto_tfm_ctx(tfm); + + if (!may_use_simd()) { + crypto_sm4_decrypt(tfm, out, in); + } else { + kernel_neon_begin(); + sm4_ce_do_crypt(ctx->rkey_dec, out, in); + kernel_neon_end(); + } +} + +static struct crypto_alg sm4_ce_alg = { + .cra_name = "sm4", + .cra_driver_name = "sm4-ce", + .cra_priority = 200, + .cra_flags = CRYPTO_ALG_TYPE_CIPHER, + .cra_blocksize = SM4_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct crypto_sm4_ctx), + .cra_module = THIS_MODULE, + .cra_u.cipher = { + .cia_min_keysize = SM4_KEY_SIZE, + .cia_max_keysize = SM4_KEY_SIZE, + .cia_setkey = crypto_sm4_set_key, + .cia_encrypt = sm4_ce_encrypt, + .cia_decrypt = sm4_ce_decrypt + } +}; + +static int __init sm4_ce_mod_init(void) +{ + return crypto_register_alg(&sm4_ce_alg); +} + +static void __exit sm4_ce_mod_fini(void) +{ + crypto_unregister_alg(&sm4_ce_alg); +} + +module_cpu_feature_match(SM3, sm4_ce_mod_init); +module_exit(sm4_ce_mod_fini); -- 2.17.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 0/2] crypto: implement SM4 for arm64 using special instructions 2018-04-25 12:20 [PATCH 0/2] crypto: implement SM4 for arm64 using special instructions Ard Biesheuvel 2018-04-25 12:20 ` [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers Ard Biesheuvel 2018-04-25 12:20 ` [PATCH 2/2] crypto: arm64 - add support for SM4 encryption using special instructions Ard Biesheuvel @ 2018-05-05 7:17 ` Herbert Xu 2 siblings, 0 replies; 6+ messages in thread From: Herbert Xu @ 2018-05-05 7:17 UTC (permalink / raw) To: linux-arm-kernel On Wed, Apr 25, 2018 at 02:20:44PM +0200, Ard Biesheuvel wrote: > Patch #1 makes some preparatory changes so the C routines can be used as > a fallback by other drivers. > > Patch #2 implements the SM4 core cipher using the special instructions > introduced as an optional extension by revision 8.2 of the ARM architecture. > > Note that this does not implement cipher+chaining mode combinations as we > do for AES. This can be added later if desiresd. > > Ard Biesheuvel (2): > crypto: sm4 - export encrypt/decrypt routines to other drivers > crypto: arm64 - add support for SM4 encryption using special > instructions > > arch/arm64/crypto/Kconfig | 6 ++ > arch/arm64/crypto/Makefile | 3 + > arch/arm64/crypto/sm4-ce-core.S | 36 ++++++++++ > arch/arm64/crypto/sm4-ce-glue.c | 73 ++++++++++++++++++++ > crypto/sm4_generic.c | 10 +-- > include/crypto/sm4.h | 3 + > 6 files changed, 127 insertions(+), 4 deletions(-) > create mode 100644 arch/arm64/crypto/sm4-ce-core.S > create mode 100644 arch/arm64/crypto/sm4-ce-glue.c All applied. Thanks. -- Email: Herbert Xu <herbert@gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-05-05 7:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-04-25 12:20 [PATCH 0/2] crypto: implement SM4 for arm64 using special instructions Ard Biesheuvel 2018-04-25 12:20 ` [PATCH 1/2] crypto: sm4 - export encrypt/decrypt routines to other drivers Ard Biesheuvel 2018-04-25 12:23 ` Ard Biesheuvel 2018-04-25 12:47 ` Gilad Ben-Yossef 2018-04-25 12:20 ` [PATCH 2/2] crypto: arm64 - add support for SM4 encryption using special instructions Ard Biesheuvel 2018-05-05 7:17 ` [PATCH 0/2] crypto: implement SM4 for arm64 " Herbert Xu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox