From: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
To: Herbert Xu <herbert@gondor.apana.org.au>,
"David S. Miller" <davem@davemloft.net>,
Jussi Kivilinna <jussi.kivilinna@iki.fi>,
Ard Biesheuvel <ardb@kernel.org>, Mark Brown <broonie@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Maxime Coquelin <mcoquelin.stm32@gmail.com>,
Alexandre Torgue <alexandre.torgue@foss.st.com>,
Eric Biggers <ebiggers@kernel.org>,
linux-crypto@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org,
linux-stm32@st-md-mailman.stormreply.com
Cc: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
Subject: [PATCH v2 07/15] crypto: arm64/sm4 - simplify sm4_ce_expand_key() of CE implementation
Date: Tue, 18 Oct 2022 15:09:58 +0800 [thread overview]
Message-ID: <20221018071006.5717-8-tianjia.zhang@linux.alibaba.com> (raw)
In-Reply-To: <20221018071006.5717-1-tianjia.zhang@linux.alibaba.com>
Use a 128-bit swap mask and tbl instruction to simplify the implementation
for generating SM4 rkey_dec.
Also fixed the issue of not being wrapped by kernel_neon_begin/end() when
using the sm4_ce_expand_key() function.
Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
---
arch/arm64/crypto/sm4-ce-core.S | 46 ++++++++++++++++-----------------
arch/arm64/crypto/sm4-ce-glue.c | 2 ++
2 files changed, 24 insertions(+), 24 deletions(-)
diff --git a/arch/arm64/crypto/sm4-ce-core.S b/arch/arm64/crypto/sm4-ce-core.S
index 41fc745a8528..9e4b4f01cdf3 100644
--- a/arch/arm64/crypto/sm4-ce-core.S
+++ b/arch/arm64/crypto/sm4-ce-core.S
@@ -65,32 +65,23 @@ SYM_FUNC_START(sm4_ce_expand_key)
sm4ekey v6.4s, v5.4s, v30.4s;
sm4ekey v7.4s, v6.4s, v31.4s;
+ adr_l x5, .Lbswap128_mask
+ ld1 {v24.16b}, [x5]
+
st1 {v0.16b-v3.16b}, [x1], #64;
st1 {v4.16b-v7.16b}, [x1];
- rev64 v7.4s, v7.4s;
- rev64 v6.4s, v6.4s;
- rev64 v5.4s, v5.4s;
- rev64 v4.4s, v4.4s;
- rev64 v3.4s, v3.4s;
- rev64 v2.4s, v2.4s;
- rev64 v1.4s, v1.4s;
- rev64 v0.4s, v0.4s;
- ext v7.16b, v7.16b, v7.16b, #8;
- ext v6.16b, v6.16b, v6.16b, #8;
- ext v5.16b, v5.16b, v5.16b, #8;
- ext v4.16b, v4.16b, v4.16b, #8;
- ext v3.16b, v3.16b, v3.16b, #8;
- ext v2.16b, v2.16b, v2.16b, #8;
- ext v1.16b, v1.16b, v1.16b, #8;
- ext v0.16b, v0.16b, v0.16b, #8;
- st1 {v7.16b}, [x2], #16;
- st1 {v6.16b}, [x2], #16;
- st1 {v5.16b}, [x2], #16;
- st1 {v4.16b}, [x2], #16;
- st1 {v3.16b}, [x2], #16;
- st1 {v2.16b}, [x2], #16;
- st1 {v1.16b}, [x2], #16;
- st1 {v0.16b}, [x2];
+
+ tbl v16.16b, {v7.16b}, v24.16b
+ tbl v17.16b, {v6.16b}, v24.16b
+ tbl v18.16b, {v5.16b}, v24.16b
+ tbl v19.16b, {v4.16b}, v24.16b
+ tbl v20.16b, {v3.16b}, v24.16b
+ tbl v21.16b, {v2.16b}, v24.16b
+ tbl v22.16b, {v1.16b}, v24.16b
+ tbl v23.16b, {v0.16b}, v24.16b
+
+ st1 {v16.16b-v19.16b}, [x2], #64
+ st1 {v20.16b-v23.16b}, [x2]
ret;
SYM_FUNC_END(sm4_ce_expand_key)
@@ -578,3 +569,10 @@ SYM_FUNC_START(sm4_ce_ctr_enc)
ret
SYM_FUNC_END(sm4_ce_ctr_enc)
+
+
+ .section ".rodata", "a"
+ .align 4
+.Lbswap128_mask:
+ .byte 0x0c, 0x0d, 0x0e, 0x0f, 0x08, 0x09, 0x0a, 0x0b
+ .byte 0x04, 0x05, 0x06, 0x07, 0x00, 0x01, 0x02, 0x03
diff --git a/arch/arm64/crypto/sm4-ce-glue.c b/arch/arm64/crypto/sm4-ce-glue.c
index e56e81b1f35f..ff2d8442d473 100644
--- a/arch/arm64/crypto/sm4-ce-glue.c
+++ b/arch/arm64/crypto/sm4-ce-glue.c
@@ -44,8 +44,10 @@ static int sm4_setkey(struct crypto_skcipher *tfm, const u8 *key,
if (key_len != SM4_KEY_SIZE)
return -EINVAL;
+ kernel_neon_begin();
sm4_ce_expand_key(key, ctx->rkey_enc, ctx->rkey_dec,
crypto_sm4_fk, crypto_sm4_ck);
+ kernel_neon_end();
return 0;
}
--
2.24.3 (Apple Git-128)
next prev parent reply other threads:[~2022-10-18 7:11 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-18 7:09 [PATCH v2 00/15] Optimizing SM3 and SM4 algorithms using arm64 NEON/CE instructions Tianjia Zhang
2022-10-18 7:09 ` [PATCH v2 01/15] crypto: arm64/sm3 - raise the priority of the CE implementation Tianjia Zhang
2022-10-18 7:09 ` [PATCH v2 02/15] crypto: arm64/sm3 - add NEON assembly implementation Tianjia Zhang
2022-10-18 7:09 ` [PATCH v2 03/15] crypto: arm64/sm4 - refactor and simplify NEON implementation Tianjia Zhang
2022-10-18 7:09 ` [PATCH v2 04/15] crypto: testmgr - add SM4 cts-cbc/essiv/xts/xcbc test vectors Tianjia Zhang
2022-10-18 7:09 ` [PATCH v2 05/15] crypto: tcrypt - add SM4 cts-cbc/essiv/xts/xcbc test Tianjia Zhang
2022-10-18 7:09 ` [PATCH v2 06/15] crypto: arm64/sm4 - refactor and simplify CE implementation Tianjia Zhang
2022-10-18 7:09 ` Tianjia Zhang [this message]
2022-10-18 7:09 ` [PATCH v2 08/15] crypto: arm64/sm4 - export reusable CE acceleration functions Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 09/15] crypto: arm64/sm4 - add CE implementation for CTS-CBC mode Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 10/15] crypto: arm64/sm4 - add CE implementation for XTS mode Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 11/15] crypto: essiv - allow digestsize to be greater than keysize Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 12/15] crypto: arm64/sm4 - add CE implementation for ESSIV mode Tianjia Zhang
2022-10-20 3:58 ` Eric Biggers
2022-10-21 2:47 ` Tianjia Zhang
2022-10-25 5:20 ` Eric Biggers
2022-10-25 7:40 ` Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 13/15] crypto: arm64/sm4 - add CE implementation for cmac/xcbc/cbcmac Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 14/15] crypto: arm64/sm4 - add CE implementation for CCM mode Tianjia Zhang
2022-10-18 7:10 ` [PATCH v2 15/15] crypto: arm64/sm4 - add CE implementation for GCM mode Tianjia Zhang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221018071006.5717-8-tianjia.zhang@linux.alibaba.com \
--to=tianjia.zhang@linux.alibaba.com \
--cc=alexandre.torgue@foss.st.com \
--cc=ardb@kernel.org \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=davem@davemloft.net \
--cc=ebiggers@kernel.org \
--cc=herbert@gondor.apana.org.au \
--cc=jussi.kivilinna@iki.fi \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-stm32@st-md-mailman.stormreply.com \
--cc=mcoquelin.stm32@gmail.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox