* [PATCH v2 5/7] crypto: talitos - sg_to_link_tbl() not used anymore, remove it
From: Christophe Leroy @ 2016-06-06 11:20 UTC (permalink / raw)
To: Kim Phillips, Herbert Xu, David S Miller
Cc: linux-kernel, linuxppc-dev, linux-crypto
In-Reply-To: <cover.1464390094.git.christophe.leroy@c-s.fr>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
drivers/crypto/talitos.c | 8 --------
1 file changed, 8 deletions(-)
diff --git a/drivers/crypto/talitos.c b/drivers/crypto/talitos.c
index 4ff03c3..ff8cf39 100644
--- a/drivers/crypto/talitos.c
+++ b/drivers/crypto/talitos.c
@@ -1111,14 +1111,6 @@ next:
return count;
}
-static inline int sg_to_link_tbl(struct scatterlist *sg, int sg_count,
- int cryptlen,
- struct talitos_ptr *link_tbl_ptr)
-{
- return sg_to_link_tbl_offset(sg, sg_count, 0, cryptlen,
- link_tbl_ptr);
-}
-
int talitos_sg_map(struct device *dev, struct scatterlist *src,
unsigned int len, struct talitos_edesc *edesc,
struct talitos_ptr *ptr,
--
2.1.0
^ permalink raw reply related
* [PATCH v2 0/7] crypto: talitos - implementation of AEAD for SEC1
From: Christophe Leroy @ 2016-06-06 11:20 UTC (permalink / raw)
To: Kim Phillips, Herbert Xu, David S Miller
Cc: linux-kernel, linuxppc-dev, linux-crypto
This set of patches provides the implementation of AEAD for
talitos SEC1.
Changes in v2:
* Fixed compilation issue introduced by later cosmetic changes
requested by checkpatch script
* split "making mapping helpers more generic" in two parts
to better see the changes done after the move.
* reworked the DMA sync associated with mapping in order to not
spend time syncing several times
Christophe Leroy (7):
crypto: talitos - using helpers for all talitos_ptr operations
crypto: talitos - move mapping helpers before IPSEC functions
crypto: talitos - making mapping helpers more generic
crypto: talitos - Implement AEAD for SEC1 using HMAC_SNOOP_NO_AFEU
crypto: talitos - sg_to_link_tbl() not used anymore, remove it
crypto: talitos - implement cra_priority
crypto: talitos - templates for AEAD using HMAC_SNOOP_NO_AFEU
drivers/crypto/talitos.c | 672 ++++++++++++++++++++++++++++++-----------------
1 file changed, 426 insertions(+), 246 deletions(-)
--
2.1.0
^ permalink raw reply
* [PATCH v2 7/7] crypto: talitos - templates for AEAD using HMAC_SNOOP_NO_AFEU
From: Christophe Leroy @ 2016-06-06 11:20 UTC (permalink / raw)
To: Kim Phillips, Herbert Xu, David S Miller
Cc: linux-kernel, linuxppc-dev, linux-crypto
In-Reply-To: <cover.1464390094.git.christophe.leroy@c-s.fr>
This will allow IPSEC on SEC1
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
drivers/crypto/talitos.c | 180 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 180 insertions(+)
diff --git a/drivers/crypto/talitos.c b/drivers/crypto/talitos.c
index dfd3a93..0418a2f 100644
--- a/drivers/crypto/talitos.c
+++ b/drivers/crypto/talitos.c
@@ -811,6 +811,11 @@ static void talitos_unregister_rng(struct device *dev)
* crypto alg
*/
#define TALITOS_CRA_PRIORITY 3000
+/*
+ * Defines a priority for doing AEAD with descriptors type
+ * HMAC_SNOOP_NO_AFEA (HSNA) instead of type IPSEC_ESP
+ */
+#define TALITOS_CRA_PRIORITY_AEAD_HSNA (TALITOS_CRA_PRIORITY - 1)
#define TALITOS_MAX_KEY_SIZE 96
#define TALITOS_MAX_IV_LENGTH 16 /* max of AES_BLOCK_SIZE, DES3_EDE_BLOCK_SIZE */
@@ -2152,6 +2157,27 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_SHA1_HMAC,
},
{ .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(sha1),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-sha1-"
+ "cbc-aes-talitos",
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = SHA1_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_AESU |
+ DESC_HDR_MODE0_AESU_CBC |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_SHA1_HMAC,
+ },
+ { .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
.cra_name = "authenc(hmac(sha1),"
@@ -2173,6 +2199,29 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_PAD |
DESC_HDR_MODE1_MDEU_SHA1_HMAC,
},
+ { .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(sha1),"
+ "cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-sha1-"
+ "cbc-3des-talitos",
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = SHA1_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_DEU |
+ DESC_HDR_MODE0_DEU_CBC |
+ DESC_HDR_MODE0_DEU_3DES |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_SHA1_HMAC,
+ },
{ .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
@@ -2193,6 +2242,27 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_PAD |
DESC_HDR_MODE1_MDEU_SHA224_HMAC,
},
+ { .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(sha224),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-sha224-"
+ "cbc-aes-talitos",
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = SHA224_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_AESU |
+ DESC_HDR_MODE0_AESU_CBC |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_SHA224_HMAC,
+ },
{ .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
@@ -2216,6 +2286,29 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_SHA224_HMAC,
},
{ .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(sha224),"
+ "cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-sha224-"
+ "cbc-3des-talitos",
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = SHA224_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_DEU |
+ DESC_HDR_MODE0_DEU_CBC |
+ DESC_HDR_MODE0_DEU_3DES |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_SHA224_HMAC,
+ },
+ { .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
.cra_name = "authenc(hmac(sha256),cbc(aes))",
@@ -2236,6 +2329,27 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_SHA256_HMAC,
},
{ .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(sha256),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-sha256-"
+ "cbc-aes-talitos",
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = SHA256_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_AESU |
+ DESC_HDR_MODE0_AESU_CBC |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_SHA256_HMAC,
+ },
+ { .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
.cra_name = "authenc(hmac(sha256),"
@@ -2258,6 +2372,29 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_SHA256_HMAC,
},
{ .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(sha256),"
+ "cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-sha256-"
+ "cbc-3des-talitos",
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = SHA256_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_DEU |
+ DESC_HDR_MODE0_DEU_CBC |
+ DESC_HDR_MODE0_DEU_3DES |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_SHA256_HMAC,
+ },
+ { .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
.cra_name = "authenc(hmac(sha384),cbc(aes))",
@@ -2362,6 +2499,27 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_MD5_HMAC,
},
{ .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(md5),cbc(aes))",
+ .cra_driver_name = "authenc-hmac-md5-"
+ "cbc-aes-talitos",
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = AES_BLOCK_SIZE,
+ .maxauthsize = MD5_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_AESU |
+ DESC_HDR_MODE0_AESU_CBC |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_MD5_HMAC,
+ },
+ { .type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
.cra_name = "authenc(hmac(md5),cbc(des3_ede))",
@@ -2382,6 +2540,28 @@ static struct talitos_alg_template driver_algs[] = {
DESC_HDR_MODE1_MDEU_PAD |
DESC_HDR_MODE1_MDEU_MD5_HMAC,
},
+ { .type = CRYPTO_ALG_TYPE_AEAD,
+ .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
+ .alg.aead = {
+ .base = {
+ .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
+ .cra_driver_name = "authenc-hmac-md5-"
+ "cbc-3des-talitos",
+ .cra_blocksize = DES3_EDE_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ },
+ .ivsize = DES3_EDE_BLOCK_SIZE,
+ .maxauthsize = MD5_DIGEST_SIZE,
+ },
+ .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
+ DESC_HDR_SEL0_DEU |
+ DESC_HDR_MODE0_DEU_CBC |
+ DESC_HDR_MODE0_DEU_3DES |
+ DESC_HDR_SEL1_MDEUA |
+ DESC_HDR_MODE1_MDEU_INIT |
+ DESC_HDR_MODE1_MDEU_PAD |
+ DESC_HDR_MODE1_MDEU_MD5_HMAC,
+ },
/* ABLKCIPHER algorithms. */
{ .type = CRYPTO_ALG_TYPE_ABLKCIPHER,
.alg.crypto = {
--
2.1.0
^ permalink raw reply related
* [PATCH v2 1/7] crypto: talitos - using helpers for all talitos_ptr operations
From: Christophe Leroy @ 2016-06-06 11:20 UTC (permalink / raw)
To: Kim Phillips, Herbert Xu, David S Miller
Cc: linux-kernel, linuxppc-dev, linux-crypto
In-Reply-To: <cover.1464390094.git.christophe.leroy@c-s.fr>
Use helper for all modifications to talitos_ptr in preparation to
the implementation of AEAD for SEC1
to_talitos_ptr_extent_clear() has been removed in favor of
to_talitos_ptr_ext_set() to set any value and
to_talitos_ptr_ext_or() to or the extent field with a value
name has been shorten to help keeping single lines of 80 chars
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
drivers/crypto/talitos.c | 59 ++++++++++++++++++++++++++++--------------------
1 file changed, 34 insertions(+), 25 deletions(-)
diff --git a/drivers/crypto/talitos.c b/drivers/crypto/talitos.c
index b7ee8d3..a92aa37 100644
--- a/drivers/crypto/talitos.c
+++ b/drivers/crypto/talitos.c
@@ -91,10 +91,17 @@ static unsigned short from_talitos_ptr_len(struct talitos_ptr *ptr,
return be16_to_cpu(ptr->len);
}
-static void to_talitos_ptr_extent_clear(struct talitos_ptr *ptr, bool is_sec1)
+static void to_talitos_ptr_ext_set(struct talitos_ptr *ptr, u8 val,
+ bool is_sec1)
{
if (!is_sec1)
- ptr->j_extent = 0;
+ ptr->j_extent = val;
+}
+
+static void to_talitos_ptr_ext_or(struct talitos_ptr *ptr, u8 val, bool is_sec1)
+{
+ if (!is_sec1)
+ ptr->j_extent |= val;
}
/*
@@ -111,7 +118,7 @@ static void map_single_talitos_ptr(struct device *dev,
to_talitos_ptr_len(ptr, len, is_sec1);
to_talitos_ptr(ptr, dma_addr, is_sec1);
- to_talitos_ptr_extent_clear(ptr, is_sec1);
+ to_talitos_ptr_ext_set(ptr, 0, is_sec1);
}
/*
@@ -1050,8 +1057,8 @@ static int sg_to_link_tbl_offset(struct scatterlist *sg, int sg_count,
to_talitos_ptr(link_tbl_ptr + count,
sg_dma_address(sg) + offset, 0);
- link_tbl_ptr[count].len = cpu_to_be16(len);
- link_tbl_ptr[count].j_extent = 0;
+ to_talitos_ptr_len(link_tbl_ptr + count, len, 0);
+ to_talitos_ptr_ext_set(link_tbl_ptr + count, 0, 0);
count++;
cryptlen -= len;
offset = 0;
@@ -1062,7 +1069,8 @@ next:
/* tag end of link table */
if (count > 0)
- link_tbl_ptr[count - 1].j_extent = DESC_PTR_LNKTBL_RETURN;
+ to_talitos_ptr_ext_set(link_tbl_ptr + count - 1,
+ DESC_PTR_LNKTBL_RETURN, 0);
return count;
}
@@ -1102,14 +1110,14 @@ static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
(areq->src == areq->dst) ? DMA_BIDIRECTIONAL
: DMA_TO_DEVICE);
/* hmac data */
- desc->ptr[1].len = cpu_to_be16(areq->assoclen);
+ to_talitos_ptr_len(&desc->ptr[1], areq->assoclen, 0);
if (sg_count > 1 &&
(ret = sg_to_link_tbl_offset(areq->src, sg_count, 0,
areq->assoclen,
&edesc->link_tbl[tbl_off])) > 1) {
to_talitos_ptr(&desc->ptr[1], edesc->dma_link_tbl + tbl_off *
sizeof(struct talitos_ptr), 0);
- desc->ptr[1].j_extent = DESC_PTR_LNKTBL_JUMP;
+ to_talitos_ptr_ext_set(&desc->ptr[1], DESC_PTR_LNKTBL_JUMP, 0);
dma_sync_single_for_device(dev, edesc->dma_link_tbl,
edesc->dma_len, DMA_BIDIRECTIONAL);
@@ -1117,13 +1125,13 @@ static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
tbl_off += ret;
} else {
to_talitos_ptr(&desc->ptr[1], sg_dma_address(areq->src), 0);
- desc->ptr[1].j_extent = 0;
+ to_talitos_ptr_ext_set(&desc->ptr[1], 0, 0);
}
/* cipher iv */
to_talitos_ptr(&desc->ptr[2], edesc->iv_dma, 0);
- desc->ptr[2].len = cpu_to_be16(ivsize);
- desc->ptr[2].j_extent = 0;
+ to_talitos_ptr_len(&desc->ptr[2], ivsize, 0);
+ to_talitos_ptr_ext_set(&desc->ptr[2], 0, 0);
/* cipher key */
map_single_talitos_ptr(dev, &desc->ptr[3], ctx->enckeylen,
@@ -1136,8 +1144,8 @@ static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
* extent is bytes of HMAC postpended to ciphertext,
* typically 12 for ipsec
*/
- desc->ptr[4].len = cpu_to_be16(cryptlen);
- desc->ptr[4].j_extent = authsize;
+ to_talitos_ptr_len(&desc->ptr[4], cryptlen, 0);
+ to_talitos_ptr_ext_set(&desc->ptr[4], authsize, 0);
sg_link_tbl_len = cryptlen;
if (edesc->desc.hdr & DESC_HDR_MODE1_MDEU_CICV)
@@ -1150,7 +1158,7 @@ static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
areq->assoclen, sg_link_tbl_len,
&edesc->link_tbl[tbl_off])) >
1) {
- desc->ptr[4].j_extent |= DESC_PTR_LNKTBL_JUMP;
+ to_talitos_ptr_ext_or(&desc->ptr[4], DESC_PTR_LNKTBL_JUMP, 0);
to_talitos_ptr(&desc->ptr[4], edesc->dma_link_tbl +
tbl_off *
sizeof(struct talitos_ptr), 0);
@@ -1163,8 +1171,8 @@ static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
}
/* cipher out */
- desc->ptr[5].len = cpu_to_be16(cryptlen);
- desc->ptr[5].j_extent = authsize;
+ to_talitos_ptr_len(&desc->ptr[5], cryptlen, 0);
+ to_talitos_ptr_ext_set(&desc->ptr[5], authsize, 0);
if (areq->src != areq->dst)
sg_count = dma_map_sg(dev, areq->dst, edesc->dst_nents ? : 1,
@@ -1186,17 +1194,17 @@ static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
/* Add an entry to the link table for ICV data */
tbl_ptr += sg_count - 1;
- tbl_ptr->j_extent = 0;
+ to_talitos_ptr_ext_set(tbl_ptr, 0, 0);
tbl_ptr++;
- tbl_ptr->j_extent = DESC_PTR_LNKTBL_RETURN;
- tbl_ptr->len = cpu_to_be16(authsize);
+ to_talitos_ptr_ext_set(tbl_ptr, DESC_PTR_LNKTBL_RETURN, 0);
+ to_talitos_ptr_len(tbl_ptr, authsize, 0);
/* icv data follows link tables */
to_talitos_ptr(tbl_ptr, edesc->dma_link_tbl +
(edesc->src_nents + edesc->dst_nents +
2) * sizeof(struct talitos_ptr) +
authsize, 0);
- desc->ptr[5].j_extent |= DESC_PTR_LNKTBL_JUMP;
+ to_talitos_ptr_ext_or(&desc->ptr[5], DESC_PTR_LNKTBL_JUMP, 0);
dma_sync_single_for_device(ctx->dev, edesc->dma_link_tbl,
edesc->dma_len, DMA_BIDIRECTIONAL);
@@ -1493,7 +1501,7 @@ int map_sg_in_talitos_ptr(struct device *dev, struct scatterlist *src,
len, DMA_TO_DEVICE);
}
} else {
- to_talitos_ptr_extent_clear(ptr, is_sec1);
+ to_talitos_ptr_ext_set(ptr, 0, is_sec1);
sg_count = dma_map_sg(dev, src, edesc->src_nents ? : 1, dir);
@@ -1504,7 +1512,8 @@ int map_sg_in_talitos_ptr(struct device *dev, struct scatterlist *src,
&edesc->link_tbl[0]);
if (sg_count > 1) {
to_talitos_ptr(ptr, edesc->dma_link_tbl, 0);
- ptr->j_extent |= DESC_PTR_LNKTBL_JUMP;
+ to_talitos_ptr_ext_or(ptr, DESC_PTR_LNKTBL_JUMP,
+ 0);
dma_sync_single_for_device(dev,
edesc->dma_link_tbl,
edesc->dma_len,
@@ -1544,7 +1553,7 @@ void map_sg_out_talitos_ptr(struct device *dev, struct scatterlist *dst,
len, DMA_FROM_DEVICE);
}
} else {
- to_talitos_ptr_extent_clear(ptr, is_sec1);
+ to_talitos_ptr_ext_set(ptr, 0, is_sec1);
if (sg_count == 1) {
to_talitos_ptr(ptr, sg_dma_address(dst), is_sec1);
@@ -1555,7 +1564,7 @@ void map_sg_out_talitos_ptr(struct device *dev, struct scatterlist *dst,
to_talitos_ptr(ptr, edesc->dma_link_tbl +
(edesc->src_nents + 1) *
sizeof(struct talitos_ptr), 0);
- ptr->j_extent |= DESC_PTR_LNKTBL_JUMP;
+ to_talitos_ptr_ext_or(ptr, DESC_PTR_LNKTBL_JUMP, 0);
sg_to_link_tbl(dst, sg_count, len, link_tbl_ptr);
dma_sync_single_for_device(dev, edesc->dma_link_tbl,
edesc->dma_len,
@@ -1586,7 +1595,7 @@ static int common_nonsnoop(struct talitos_edesc *edesc,
/* cipher iv */
to_talitos_ptr(&desc->ptr[1], edesc->iv_dma, is_sec1);
to_talitos_ptr_len(&desc->ptr[1], ivsize, is_sec1);
- to_talitos_ptr_extent_clear(&desc->ptr[1], is_sec1);
+ to_talitos_ptr_ext_set(&desc->ptr[1], 0, is_sec1);
/* cipher key */
map_single_talitos_ptr(dev, &desc->ptr[2], ctx->keylen,
--
2.1.0
^ permalink raw reply related
* Re: [PATCH 2/3] dt-bindings: hwrng: Add Amlogic Meson Hardware Random Generator bindings
From: Rob Herring @ 2016-06-06 14:10 UTC (permalink / raw)
To: Neil Armstrong
Cc: Matt Mackall, Herbert Xu, linux-arm-kernel, linux-amlogic,
linux-kernel, linux-crypto, devicetree
In-Reply-To: <1464943621-18278-3-git-send-email-narmstrong@baylibre.com>
On Fri, Jun 03, 2016 at 10:47:00AM +0200, Neil Armstrong wrote:
> Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
> ---
> .../devicetree/bindings/rng/amlogic,meson-rng.txt | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/rng/amlogic,meson-rng.txt
Acked-by: Rob Herring <robh@kernel.org>
^ permalink raw reply
* Re: [PATCH 02/28] crypto: omap-sham: Don't idle/start SHA device between Encrypt operations
From: Herbert Xu @ 2016-06-07 10:08 UTC (permalink / raw)
To: Dave Gerlach
Cc: Grygorii Strashko, tony, Tero Kristo, lokeshvutla, linux-crypto,
linux-omap, davem, linux-arm-kernel
In-Reply-To: <574F69D8.1010809@ti.com>
On Wed, Jun 01, 2016 at 06:03:52PM -0500, Dave Gerlach wrote:
> On 06/01/2016 04:53 AM, Grygorii Strashko wrote:
> >On 06/01/2016 11:56 AM, Tero Kristo wrote:
> >>From: Lokesh Vutla <lokeshvutla@ti.com>
> >>
> >>Calling runtime PM API for every block causes serious perf hit to
> >>crypto operations that are done on a long buffer.
> >>As crypto is performed on a page boundary, encrypting large buffers can
> >>cause a series of crypto operations divided by page. The runtime PM API
> >>is also called those many times.
> >>
> >>We call runtime_pm_get_sync only at beginning on the session (cra_init)
> >>and runtime_pm_put at the end. This result in upto a 50% speedup.
> >>This doesn't make the driver to keep the system awake as runtime get/put
> >>is only called during a crypto session which completes usually quickly.
> >>
> >>Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
> >>Signed-off-by: Tero Kristo <t-kristo@ti.com>
> >>---
> >> drivers/crypto/omap-sham.c | 27 +++++++++++++++++----------
> >> 1 file changed, 17 insertions(+), 10 deletions(-)
> >>
> >>diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
> >>index 6eefaa2..bd0258f 100644
> >>--- a/drivers/crypto/omap-sham.c
> >>+++ b/drivers/crypto/omap-sham.c
> >>@@ -360,14 +360,6 @@ static void omap_sham_copy_ready_hash(struct
> >>ahash_request *req)
> >>
> >> static int omap_sham_hw_init(struct omap_sham_dev *dd)
> >> {
> >>- int err;
> >>-
> >>- err = pm_runtime_get_sync(dd->dev);
> >>- if (err < 0) {
> >>- dev_err(dd->dev, "failed to get sync: %d\n", err);
> >>- return err;
> >>- }
> >>-
>
> Would it be worth it to investigate a pm_runtime autosuspend
> approach rather than knocking runtime PM out here completely? I am
> not clear if the overhead is coming from the pm_runtime calls
> themselves or the actual idling of the IP, but if it's the idling of
> the IP causing the slowdown, with a large enough autosuspend_delay
> we don't actually sleep between each block but after a long enough
> period of idle time we would actually suspend.
Indeed, I think this patch is bogus. cra_init is associated
with the tfm object which is usually long-lived. So doing power
management there makes no sense.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v5 3/9] crypto: add driver-side scomp interface
From: Herbert Xu @ 2016-06-07 10:22 UTC (permalink / raw)
To: Giovanni Cabiddu; +Cc: linux-crypto
In-Reply-To: <1464870543-5988-4-git-send-email-giovanni.cabiddu@intel.com>
On Thu, Jun 02, 2016 at 01:28:57PM +0100, Giovanni Cabiddu wrote:
>
> @@ -96,6 +116,31 @@ struct crypto_acomp *crypto_alloc_acomp(const char *alg_name, u32 type,
> }
> EXPORT_SYMBOL_GPL(crypto_alloc_acomp);
>
> +struct acomp_req *acomp_request_alloc(struct crypto_acomp *acomp, gfp_t gfp)
> +{
> + struct crypto_tfm *tfm = crypto_acomp_tfm(acomp);
> + struct acomp_req *req;
> +
> + req = __acomp_request_alloc(acomp, gfp);
> + if (req && (tfm->__crt_alg->cra_type != &crypto_acomp_type))
> + return crypto_acomp_scomp_alloc_ctx(req);
The gfp argument is pointless since scomp requires GFP_KERNEL.
So please just make it a requirement that acomp_request_alloc
always uses GFP_KERNEL.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v3 1/1] crypto: engine: permit to enqueue ashash_request
From: Herbert Xu @ 2016-06-07 10:31 UTC (permalink / raw)
To: LABBE Corentin; +Cc: davem, baolin.wang, linux-crypto, linux-kernel
In-Reply-To: <1464873212-15426-2-git-send-email-clabbe.montjoie@gmail.com>
On Thu, Jun 02, 2016 at 03:13:32PM +0200, LABBE Corentin wrote:
>
> static int omap_aes_prepare_req(struct crypto_engine *engine,
> - struct ablkcipher_request *req)
> + struct crypto_async_request *areq)
> {
> + struct ablkcipher_request *req = ablkcipher_request_cast(areq);
You're still doing casting in the driver.
I want this to be moved into the crypto engine API. There should
be separate function pointers for each request type.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH V2 2/2] crypto : async implementation for sha1-mb
From: Herbert Xu @ 2016-06-07 10:35 UTC (permalink / raw)
To: Megha Dey
Cc: tim.c.chen, davem, linux-crypto, linux-kernel, fenghua.yu,
Megha Dey
In-Reply-To: <1464922430-26976-3-git-send-email-megha.dey@intel.com>
On Thu, Jun 02, 2016 at 07:53:50PM -0700, Megha Dey wrote:
>
> + struct ahash_alg *shash = crypto_ahash_alg(tfm);
>
> /* alignment is to be done by multi-buffer crypto algorithm if needed */
>
> - return shash->finup(desc, NULL, 0, req->result);
> + return shash->finup(desc);
You're still poking in the guts of the API. Now that it's a real
ahash you don't need to do that.
Just do crypto_ahash_finup. That way you don't need to export
crypto_ahsh_alg either.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH V2 1/2] crypto: sha1-mb - stylistic cleanup
From: Herbert Xu @ 2016-06-07 10:44 UTC (permalink / raw)
To: Megha Dey
Cc: tim.c.chen, davem, linux-crypto, linux-kernel, fenghua.yu,
Megha Dey
In-Reply-To: <1464922430-26976-2-git-send-email-megha.dey@intel.com>
On Thu, Jun 02, 2016 at 07:53:49PM -0700, Megha Dey wrote:
> From: Megha Dey <megha.dey@linux.intel.com>
>
> Currently there are several checkpatch warnings in the sha1_mb.c file:
> 'WARNING: line over 80 characters' in the sha1_mb.c file. Also, the
> syntax of some multi-line comments are not correct. This patch fixes
> these issues.
>
> Signed-off-by: Megha Dey <megha.dey@linux.intel.com>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This has already been applied. Please don't send it agin.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 01/28] crypto: omap-aes: Fix registration of algorithms
From: Herbert Xu @ 2016-06-07 10:48 UTC (permalink / raw)
To: Tero Kristo
Cc: linux-omap, linux-crypto, davem, tony, linux-arm-kernel,
lokeshvutla
In-Reply-To: <1464771389-10640-2-git-send-email-t-kristo@ti.com>
On Wed, Jun 01, 2016 at 11:56:02AM +0300, Tero Kristo wrote:
> From: Lokesh Vutla <lokeshvutla@ti.com>
>
> Algorithms can be registered only once. So skip registration of
> algorithms if already registered (i.e. in case we have two AES cores
> in the system.)
>
> Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
> Signed-off-by: Tero Kristo <t-kristo@ti.com>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v3 8/8] arm64: dts: ls1043a: add crypto node
From: Herbert Xu @ 2016-06-07 10:48 UTC (permalink / raw)
To: Horia Geantă
Cc: Shawn Guo, Rob Herring, Olof Johansson, linux-crypto,
David S. Miller, linux-kernel, Mingkai Hu
In-Reply-To: <1463670709-3193-1-git-send-email-horia.geanta@nxp.com>
On Thu, May 19, 2016 at 06:11:49PM +0300, Horia Geantă wrote:
> LS1043A has a SEC v5.4 security engine.
> For now don't add rtic or sec_mon subnodes, since these features
> haven't been tested yet.
>
> Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] crypto: skcipher - remove unused header cpumask.h
From: Herbert Xu @ 2016-06-07 10:51 UTC (permalink / raw)
To: Geliang Tang; +Cc: David S. Miller, linux-crypto, linux-kernel
In-Reply-To: <40677731b6e82e34c6f1f5b6d9456c6ed4c93234.1464844992.git.geliangtang@gmail.com>
On Thu, Jun 02, 2016 at 01:48:38PM +0800, Geliang Tang wrote:
> Remove unused header cpumask.h from crypto/ablkcipher.c.
>
> Signed-off-by: Geliang Tang <geliangtang@gmail.com>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v5 1/9] crypto: shrink hash down to two types
From: Herbert Xu @ 2016-06-07 10:52 UTC (permalink / raw)
To: Giovanni Cabiddu; +Cc: linux-crypto
In-Reply-To: <1464870543-5988-2-git-send-email-giovanni.cabiddu@intel.com>
On Thu, Jun 02, 2016 at 01:28:55PM +0100, Giovanni Cabiddu wrote:
> Move hash to 0xe to free up the space for acomp/scomp
>
> Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 0/2] hwrng: chaoskey - Add support for Araneus Alea I USB RNG
From: Herbert Xu @ 2016-06-07 10:53 UTC (permalink / raw)
To: Bob Ham
Cc: linux-crypto, linux-usb, Keith Packard, Matt Mackall,
Clemens Ladisch, Greg KH
In-Reply-To: <1464952388-2405-1-git-send-email-bob.ham@collabora.com>
On Fri, Jun 03, 2016 at 12:13:06PM +0100, Bob Ham wrote:
> Two patches to add support for the Araneus Alea I USB RNG to the
> chaoskey driver. The first simply includes the Alea I as a device,
> the second fixes an issue with the timeout on the first read.
>
> Bob Ham (2):
> hwrng: chaoskey - Add support for Araneus Alea I USB RNG
> hwrng: chaoskey - Fix URB warning due to timeout on Alea
>
> drivers/usb/misc/Kconfig | 11 ++++++-----
> drivers/usb/misc/chaoskey.c | 21 +++++++++++++++++++--
> 2 files changed, 25 insertions(+), 7 deletions(-)
Both applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 02/28] crypto: omap-sham: Don't idle/start SHA device between Encrypt operations
From: Tero Kristo @ 2016-06-07 11:52 UTC (permalink / raw)
To: Herbert Xu, Dave Gerlach
Cc: Grygorii Strashko, linux-omap, linux-crypto, davem, tony,
linux-arm-kernel, lokeshvutla
In-Reply-To: <20160607100837.GA30875@gondor.apana.org.au>
On 07/06/16 13:08, Herbert Xu wrote:
> On Wed, Jun 01, 2016 at 06:03:52PM -0500, Dave Gerlach wrote:
>> On 06/01/2016 04:53 AM, Grygorii Strashko wrote:
>>> On 06/01/2016 11:56 AM, Tero Kristo wrote:
>>>> From: Lokesh Vutla <lokeshvutla@ti.com>
>>>>
>>>> Calling runtime PM API for every block causes serious perf hit to
>>>> crypto operations that are done on a long buffer.
>>>> As crypto is performed on a page boundary, encrypting large buffers can
>>>> cause a series of crypto operations divided by page. The runtime PM API
>>>> is also called those many times.
>>>>
>>>> We call runtime_pm_get_sync only at beginning on the session (cra_init)
>>>> and runtime_pm_put at the end. This result in upto a 50% speedup.
>>>> This doesn't make the driver to keep the system awake as runtime get/put
>>>> is only called during a crypto session which completes usually quickly.
>>>>
>>>> Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
>>>> Signed-off-by: Tero Kristo <t-kristo@ti.com>
>>>> ---
>>>> drivers/crypto/omap-sham.c | 27 +++++++++++++++++----------
>>>> 1 file changed, 17 insertions(+), 10 deletions(-)
>>>>
>>>> diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
>>>> index 6eefaa2..bd0258f 100644
>>>> --- a/drivers/crypto/omap-sham.c
>>>> +++ b/drivers/crypto/omap-sham.c
>>>> @@ -360,14 +360,6 @@ static void omap_sham_copy_ready_hash(struct
>>>> ahash_request *req)
>>>>
>>>> static int omap_sham_hw_init(struct omap_sham_dev *dd)
>>>> {
>>>> - int err;
>>>> -
>>>> - err = pm_runtime_get_sync(dd->dev);
>>>> - if (err < 0) {
>>>> - dev_err(dd->dev, "failed to get sync: %d\n", err);
>>>> - return err;
>>>> - }
>>>> -
>>
>> Would it be worth it to investigate a pm_runtime autosuspend
>> approach rather than knocking runtime PM out here completely? I am
>> not clear if the overhead is coming from the pm_runtime calls
>> themselves or the actual idling of the IP, but if it's the idling of
>> the IP causing the slowdown, with a large enough autosuspend_delay
>> we don't actually sleep between each block but after a long enough
>> period of idle time we would actually suspend.
>
> Indeed, I think this patch is bogus. cra_init is associated
> with the tfm object which is usually long-lived. So doing power
> management there makes no sense.
>
> Cheers,
>
I can investigate this further, but I believe this patch itself gave a
noticeable performance boost.
This is an optimization anyway, and not critical for functionality.
-Tero
^ permalink raw reply
* [RFC v4 1/4] block: Introduce blk_bio_map_sg() to map one bio
From: Baolin Wang @ 2016-06-07 12:17 UTC (permalink / raw)
To: axboe, agk, snitzer, dm-devel, herbert, davem
Cc: ebiggers3, js1304, tadeusz.struk, smueller, standby24x7, shli,
dan.j.williams, martin.petersen, sagig, kent.overstreet,
keith.busch, tj, ming.lei, broonie, arnd, linux-crypto,
linux-block, linux-raid, linux-kernel, baolin.wang
In-Reply-To: <cover.1465301616.git.baolin.wang@linaro.org>
In dm-crypt, it need to map one bio to scatterlist for improving the
hardware engine encryption efficiency. Thus this patch introduces the
blk_bio_map_sg() function to map one bio with scatterlists.
Signed-off-by: Baolin Wang <baolin.wang@linaro.org>
---
block/blk-merge.c | 19 +++++++++++++++++++
include/linux/blkdev.h | 2 ++
2 files changed, 21 insertions(+)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 2613531..bca0609 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -417,6 +417,25 @@ single_segment:
}
/*
+ * Map a bio to scatterlist, return number of sg entries setup. Caller must
+ * make sure sg can hold bio segments entries and detach the bio from list.
+ */
+int blk_bio_map_sg(struct request_queue *q, struct bio *bio,
+ struct scatterlist *sglist)
+{
+ struct scatterlist *sg = NULL;
+ int nsegs;
+
+ nsegs = __blk_bios_map_sg(q, bio, sglist, &sg);
+
+ if (sg)
+ sg_mark_end(sg);
+
+ return nsegs;
+}
+EXPORT_SYMBOL(blk_bio_map_sg);
+
+/*
* map a request to scatterlist, return number of sg entries setup. Caller
* must make sure sg can hold rq->nr_phys_segments entries
*/
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 3d9cf32..cbe31f9 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1024,6 +1024,8 @@ extern void blk_queue_write_cache(struct request_queue *q, bool enabled, bool fu
extern struct backing_dev_info *blk_get_backing_dev_info(struct block_device *bdev);
extern int blk_rq_map_sg(struct request_queue *, struct request *, struct scatterlist *);
+extern int blk_bio_map_sg(struct request_queue *q, struct bio *bio,
+ struct scatterlist *sglist);
extern void blk_dump_rq_flags(struct request *, char *);
extern long nr_blockdev_pages(void);
--
1.7.9.5
^ permalink raw reply related
* [RFC v4 2/4] crypto: Introduce CRYPTO_ALG_BULK flag
From: Baolin Wang @ 2016-06-07 12:17 UTC (permalink / raw)
To: axboe, agk, snitzer, dm-devel, herbert, davem
Cc: ebiggers3, js1304, tadeusz.struk, smueller, standby24x7, shli,
dan.j.williams, martin.petersen, sagig, kent.overstreet,
keith.busch, tj, ming.lei, broonie, arnd, linux-crypto,
linux-block, linux-raid, linux-kernel, baolin.wang
In-Reply-To: <cover.1465301616.git.baolin.wang@linaro.org>
Now some cipher hardware engines prefer to handle bulk block rather than one
sector (512 bytes) created by dm-crypt, cause these cipher engines can handle
the intermediate values (IV) by themselves in one bulk block. This means we
can increase the size of the request by merging request rather than always 512
bytes and thus increase the hardware engine processing speed.
So introduce 'CRYPTO_ALG_BULK' flag to indicate this cipher can support bulk
mode.
Signed-off-by: Baolin Wang <baolin.wang@linaro.org>
---
include/crypto/skcipher.h | 7 +++++++
include/linux/crypto.h | 6 ++++++
2 files changed, 13 insertions(+)
diff --git a/include/crypto/skcipher.h b/include/crypto/skcipher.h
index 0f987f5..d89d29a 100644
--- a/include/crypto/skcipher.h
+++ b/include/crypto/skcipher.h
@@ -519,5 +519,12 @@ static inline void skcipher_request_set_crypt(
req->iv = iv;
}
+static inline unsigned int skcipher_is_bulk_mode(struct crypto_skcipher *sk_tfm)
+{
+ struct crypto_tfm *tfm = crypto_skcipher_tfm(sk_tfm);
+
+ return crypto_tfm_alg_bulk(tfm);
+}
+
#endif /* _CRYPTO_SKCIPHER_H */
diff --git a/include/linux/crypto.h b/include/linux/crypto.h
index 6e28c89..a315487 100644
--- a/include/linux/crypto.h
+++ b/include/linux/crypto.h
@@ -63,6 +63,7 @@
#define CRYPTO_ALG_DEAD 0x00000020
#define CRYPTO_ALG_DYING 0x00000040
#define CRYPTO_ALG_ASYNC 0x00000080
+#define CRYPTO_ALG_BULK 0x00000100
/*
* Set this bit if and only if the algorithm requires another algorithm of
@@ -623,6 +624,11 @@ static inline u32 crypto_tfm_alg_type(struct crypto_tfm *tfm)
return tfm->__crt_alg->cra_flags & CRYPTO_ALG_TYPE_MASK;
}
+static inline unsigned int crypto_tfm_alg_bulk(struct crypto_tfm *tfm)
+{
+ return tfm->__crt_alg->cra_flags & CRYPTO_ALG_BULK;
+}
+
static inline unsigned int crypto_tfm_alg_blocksize(struct crypto_tfm *tfm)
{
return tfm->__crt_alg->cra_blocksize;
--
1.7.9.5
^ permalink raw reply related
* [RFC v4 3/4] md: dm-crypt: Introduce the bulk mode method when sending request
From: Baolin Wang @ 2016-06-07 12:17 UTC (permalink / raw)
To: axboe, agk, snitzer, dm-devel, herbert, davem
Cc: ebiggers3, js1304, tadeusz.struk, smueller, standby24x7, shli,
dan.j.williams, martin.petersen, sagig, kent.overstreet,
keith.busch, tj, ming.lei, broonie, arnd, linux-crypto,
linux-block, linux-raid, linux-kernel, baolin.wang
In-Reply-To: <cover.1465301616.git.baolin.wang@linaro.org>
In now dm-crypt code, it is ineffective to map one segment (always one
sector) of one bio with just only one scatterlist at one time for hardware
crypto engine. Especially for some encryption mode (like ecb or xts mode)
cooperating with the crypto engine, they just need one initial IV or null
IV instead of different IV for each sector. In this situation We can consider
to use multiple scatterlists to map the whole bio and send all scatterlists
of one bio to crypto engine to encrypt or decrypt, which can improve the
hardware engine's efficiency.
With this optimization, On my test setup (beaglebone black board with ecb(aes)
cipher and dd testing) using 64KB I/Os on an eMMC storage device I saw about
127% improvement in throughput for encrypted writes, and about 206% improvement
for encrypted reads.
Signed-off-by: Baolin Wang <baolin.wang@linaro.org>
---
drivers/md/dm-crypt.c | 159 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 158 insertions(+), 1 deletion(-)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 4f3cb35..0b1d452 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -33,6 +33,7 @@
#include <linux/device-mapper.h>
#define DM_MSG_PREFIX "crypt"
+#define DM_MAX_SG_LIST 512
/*
* context holding the current state of a multi-part conversion
@@ -142,6 +143,11 @@ struct crypt_config {
char *cipher;
char *cipher_string;
+ struct sg_table sgt_in;
+ struct sg_table sgt_out;
+ atomic_t sgt_init_done;
+ struct completion sgt_restart;
+
struct crypt_iv_operations *iv_gen_ops;
union {
struct iv_essiv_private essiv;
@@ -837,6 +843,141 @@ static u8 *iv_of_dmreq(struct crypt_config *cc,
crypto_skcipher_alignmask(any_tfm(cc)) + 1);
}
+static void crypt_init_sg_table(struct scatterlist *sgl)
+{
+ struct scatterlist *sg;
+ int i;
+
+ for_each_sg(sgl, sg, DM_MAX_SG_LIST, i) {
+ if (i < DM_MAX_SG_LIST - 1 && sg_is_last(sg))
+ sg_unmark_end(sg);
+ else if (i == DM_MAX_SG_LIST - 1)
+ sg_mark_end(sg);
+ }
+
+ for_each_sg(sgl, sg, DM_MAX_SG_LIST, i) {
+ memset(sg, 0, sizeof(struct scatterlist));
+
+ if (i == DM_MAX_SG_LIST - 1)
+ sg_mark_end(sg);
+ }
+}
+
+static void crypt_reinit_sg_table(struct crypt_config *cc)
+{
+ if (!cc->sgt_in.orig_nents || !cc->sgt_out.orig_nents)
+ return;
+
+ crypt_init_sg_table(cc->sgt_in.sgl);
+ crypt_init_sg_table(cc->sgt_out.sgl);
+
+ if (atomic_inc_and_test(&cc->sgt_init_done))
+ complete(&cc->sgt_restart);
+ atomic_set(&cc->sgt_init_done, 1);
+}
+
+static int crypt_alloc_sg_table(struct crypt_config *cc)
+{
+ unsigned int bulk_mode = skcipher_is_bulk_mode(any_tfm(cc));
+ int ret = 0;
+
+ if (!bulk_mode)
+ goto out_skip_alloc;
+
+ ret = sg_alloc_table(&cc->sgt_in, DM_MAX_SG_LIST, GFP_KERNEL);
+ if (ret)
+ goto out_skip_alloc;
+
+ ret = sg_alloc_table(&cc->sgt_out, DM_MAX_SG_LIST, GFP_KERNEL);
+ if (ret)
+ goto out_free_table;
+
+ init_completion(&cc->sgt_restart);
+ atomic_set(&cc->sgt_init_done, 1);
+ return 0;
+
+out_free_table:
+ sg_free_table(&cc->sgt_in);
+out_skip_alloc:
+ cc->sgt_in.orig_nents = 0;
+ cc->sgt_out.orig_nents = 0;
+
+ return ret;
+}
+
+static int crypt_convert_bulk_block(struct crypt_config *cc,
+ struct convert_context *ctx,
+ struct skcipher_request *req)
+{
+ struct bio *bio_in = ctx->bio_in;
+ struct bio *bio_out = ctx->bio_out;
+ unsigned int total_bytes = bio_in->bi_iter.bi_size;
+ unsigned int total_sg_in, total_sg_out;
+ struct scatterlist *sg_in, *sg_out;
+ struct dm_crypt_request *dmreq;
+ u8 *iv;
+ int r;
+
+ if (!cc->sgt_in.orig_nents || !cc->sgt_out.orig_nents)
+ return -EINVAL;
+
+ if (!atomic_dec_and_test(&cc->sgt_init_done)) {
+ wait_for_completion(&cc->sgt_restart);
+ reinit_completion(&cc->sgt_restart);
+ }
+
+ dmreq = dmreq_of_req(cc, req);
+ iv = iv_of_dmreq(cc, dmreq);
+ dmreq->iv_sector = ctx->cc_sector;
+ dmreq->ctx = ctx;
+
+ total_sg_in = blk_bio_map_sg(bdev_get_queue(bio_in->bi_bdev),
+ bio_in, cc->sgt_in.sgl);
+ if ((total_sg_in <= 0) || (total_sg_in > DM_MAX_SG_LIST)) {
+ DMERR("%s in sg map error %d, sg table nents[%d]\n",
+ __func__, total_sg_in, cc->sgt_in.orig_nents);
+ return -EINVAL;
+ }
+
+ ctx->iter_in.bi_size -= total_bytes;
+ sg_in = cc->sgt_in.sgl;
+ sg_out = cc->sgt_in.sgl;
+
+ if (bio_data_dir(bio_in) == READ)
+ goto set_crypt;
+
+ total_sg_out = blk_bio_map_sg(bdev_get_queue(bio_out->bi_bdev),
+ bio_out, cc->sgt_out.sgl);
+ if ((total_sg_out <= 0) || (total_sg_out > DM_MAX_SG_LIST)) {
+ DMERR("%s out sg map error %d, sg table nents[%d]\n",
+ __func__, total_sg_out, cc->sgt_out.orig_nents);
+ return -EINVAL;
+ }
+
+ ctx->iter_out.bi_size -= total_bytes;
+ sg_out = cc->sgt_out.sgl;
+
+set_crypt:
+ if (cc->iv_gen_ops) {
+ r = cc->iv_gen_ops->generator(cc, iv, dmreq);
+ if (r < 0)
+ return r;
+ }
+
+ atomic_set(&cc->sgt_init_done, 0);
+ skcipher_request_set_crypt(req, sg_in, sg_out, total_bytes, iv);
+
+ if (bio_data_dir(ctx->bio_in) == WRITE)
+ r = crypto_skcipher_encrypt(req);
+ else
+ r = crypto_skcipher_decrypt(req);
+
+ if (!r && cc->iv_gen_ops && cc->iv_gen_ops->post)
+ r = cc->iv_gen_ops->post(cc, iv, dmreq);
+
+ return r;
+}
+
static int crypt_convert_block(struct crypt_config *cc,
struct convert_context *ctx,
struct skcipher_request *req)
@@ -920,6 +1061,7 @@ static void crypt_free_req(struct crypt_config *cc,
static int crypt_convert(struct crypt_config *cc,
struct convert_context *ctx)
{
+ unsigned int bulk_mode;
int r;
atomic_set(&ctx->cc_pending, 1);
@@ -930,7 +1072,14 @@ static int crypt_convert(struct crypt_config *cc,
atomic_inc(&ctx->cc_pending);
- r = crypt_convert_block(cc, ctx, ctx->req);
+ bulk_mode = skcipher_is_bulk_mode(any_tfm(cc));
+ if (!bulk_mode) {
+ r = crypt_convert_block(cc, ctx, ctx->req);
+ } else {
+ r = crypt_convert_bulk_block(cc, ctx, ctx->req);
+ if (r == -EINVAL)
+ r = crypt_convert_block(cc, ctx, ctx->req);
+ }
switch (r) {
/*
@@ -1081,6 +1230,7 @@ static void crypt_dec_pending(struct dm_crypt_io *io)
if (io->ctx.req)
crypt_free_req(cc, io->ctx.req, base_bio);
+ crypt_reinit_sg_table(cc);
base_bio->bi_error = error;
bio_endio(base_bio);
}
@@ -1563,6 +1713,9 @@ static void crypt_dtr(struct dm_target *ti)
kzfree(cc->cipher);
kzfree(cc->cipher_string);
+ sg_free_table(&cc->sgt_in);
+ sg_free_table(&cc->sgt_out);
+
/* Must zero key material before freeing */
kzfree(cc);
}
@@ -1718,6 +1871,10 @@ static int crypt_ctr_cipher(struct dm_target *ti,
}
}
+ ret = crypt_alloc_sg_table(cc);
+ if (ret)
+ DMWARN("Allocate sg table for bulk mode failed");
+
ret = 0;
bad:
kfree(cipher_api);
--
1.7.9.5
^ permalink raw reply related
* [RFC v4 0/4] Introduce the bulk mode method when sending request to crypto layer
From: Baolin Wang @ 2016-06-07 12:17 UTC (permalink / raw)
To: axboe, agk, snitzer, dm-devel, herbert, davem
Cc: ebiggers3, js1304, tadeusz.struk, smueller, standby24x7, shli,
dan.j.williams, martin.petersen, sagig, kent.overstreet,
keith.busch, tj, ming.lei, broonie, arnd, linux-crypto,
linux-block, linux-raid, linux-kernel, baolin.wang
This patchset will check if the cipher can support bulk mode, then dm-crypt
will handle different ways to send requests to crypto layer according to
cipher mode. For bulk mode, we can use sg table to map the whole bio and
send all scatterlists of one bio to crypto engine to encrypt or decrypt,
which can improve the hardware engine's efficiency.
Changes since v3:
- Some optimization for blk_bio_map_sg() function.
Changes since v2:
- Add one cipher user with CRYPTO_ALG_BULK flag to support bulk mode.
- Add one atomic variable to avoid the sg table race.
Changes since v1:
- Refactor the blk_bio_map_sg() function to avoid duplicated code.
- Move the sg table allocation to crypt_ctr_cipher() function to avoid memory
allocation in the IO path.
- Remove the crypt_sg_entry() function.
- Other optimization.
Baolin Wang (4):
block: Introduce blk_bio_map_sg() to map one bio
crypto: Introduce CRYPTO_ALG_BULK flag
md: dm-crypt: Introduce the bulk mode method when sending request
crypto: Add the CRYPTO_ALG_BULK flag for ecb(aes) cipher
block/blk-merge.c | 19 ++++++
drivers/crypto/omap-aes.c | 2 +-
drivers/md/dm-crypt.c | 159 ++++++++++++++++++++++++++++++++++++++++++++-
include/crypto/skcipher.h | 7 ++
include/linux/blkdev.h | 2 +
include/linux/crypto.h | 6 ++
6 files changed, 193 insertions(+), 2 deletions(-)
--
1.7.9.5
^ permalink raw reply
* [RFC v4 4/4] crypto: Add the CRYPTO_ALG_BULK flag for ecb(aes) cipher
From: Baolin Wang @ 2016-06-07 12:17 UTC (permalink / raw)
To: axboe, agk, snitzer, dm-devel, herbert, davem
Cc: ebiggers3, js1304, tadeusz.struk, smueller, standby24x7, shli,
dan.j.williams, martin.petersen, sagig, kent.overstreet,
keith.busch, tj, ming.lei, broonie, arnd, linux-crypto,
linux-block, linux-raid, linux-kernel, baolin.wang
In-Reply-To: <cover.1465301616.git.baolin.wang@linaro.org>
Since the ecb(aes) cipher does not need to handle the IV things for encryption
or decryption, that means it can support for bulk block when handling data.
Thus this patch adds the CRYPTO_ALG_BULK flag for ecb(aes) cipher to improve
the hardware aes engine's efficiency.
Signed-off-by: Baolin Wang <baolin.wang@linaro.org>
---
drivers/crypto/omap-aes.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index ce174d3..ab09429 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -804,7 +804,7 @@ static struct crypto_alg algs_ecb_cbc[] = {
.cra_priority = 300,
.cra_flags = CRYPTO_ALG_TYPE_ABLKCIPHER |
CRYPTO_ALG_KERN_DRIVER_ONLY |
- CRYPTO_ALG_ASYNC,
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_BULK,
.cra_blocksize = AES_BLOCK_SIZE,
.cra_ctxsize = sizeof(struct omap_aes_ctx),
.cra_alignmask = 0,
--
1.7.9.5
^ permalink raw reply related
* Re: [PATCH 02/28] crypto: omap-sham: Don't idle/start SHA device between Encrypt operations
From: Grygorii Strashko @ 2016-06-07 12:24 UTC (permalink / raw)
To: Tero Kristo, Herbert Xu, Dave Gerlach
Cc: linux-omap, linux-crypto, davem, tony, linux-arm-kernel,
lokeshvutla
In-Reply-To: <5756B584.2000900@ti.com>
On 06/07/2016 02:52 PM, Tero Kristo wrote:
> On 07/06/16 13:08, Herbert Xu wrote:
>> On Wed, Jun 01, 2016 at 06:03:52PM -0500, Dave Gerlach wrote:
>>> On 06/01/2016 04:53 AM, Grygorii Strashko wrote:
>>>> On 06/01/2016 11:56 AM, Tero Kristo wrote:
>>>>> From: Lokesh Vutla <lokeshvutla@ti.com>
>>>>>
>>>>> Calling runtime PM API for every block causes serious perf hit to
>>>>> crypto operations that are done on a long buffer.
>>>>> As crypto is performed on a page boundary, encrypting large buffers
>>>>> can
>>>>> cause a series of crypto operations divided by page. The runtime PM
>>>>> API
>>>>> is also called those many times.
>>>>>
>>>>> We call runtime_pm_get_sync only at beginning on the session
>>>>> (cra_init)
>>>>> and runtime_pm_put at the end. This result in upto a 50% speedup.
>>>>> This doesn't make the driver to keep the system awake as runtime
>>>>> get/put
>>>>> is only called during a crypto session which completes usually
>>>>> quickly.
>>>>>
>>>>> Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
>>>>> Signed-off-by: Tero Kristo <t-kristo@ti.com>
>>>>> ---
>>>>> drivers/crypto/omap-sham.c | 27 +++++++++++++++++----------
>>>>> 1 file changed, 17 insertions(+), 10 deletions(-)
>>>>>
>>>>> diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
>>>>> index 6eefaa2..bd0258f 100644
>>>>> --- a/drivers/crypto/omap-sham.c
>>>>> +++ b/drivers/crypto/omap-sham.c
>>>>> @@ -360,14 +360,6 @@ static void omap_sham_copy_ready_hash(struct
>>>>> ahash_request *req)
>>>>>
>>>>> static int omap_sham_hw_init(struct omap_sham_dev *dd)
>>>>> {
>>>>> - int err;
>>>>> -
>>>>> - err = pm_runtime_get_sync(dd->dev);
>>>>> - if (err < 0) {
>>>>> - dev_err(dd->dev, "failed to get sync: %d\n", err);
>>>>> - return err;
>>>>> - }
>>>>> -
>>>
>>> Would it be worth it to investigate a pm_runtime autosuspend
>>> approach rather than knocking runtime PM out here completely? I am
>>> not clear if the overhead is coming from the pm_runtime calls
>>> themselves or the actual idling of the IP, but if it's the idling of
>>> the IP causing the slowdown, with a large enough autosuspend_delay
>>> we don't actually sleep between each block but after a long enough
>>> period of idle time we would actually suspend.
>>
>> Indeed, I think this patch is bogus. cra_init is associated
>> with the tfm object which is usually long-lived. So doing power
>> management there makes no sense.
>>
>> Cheers,
>>
>
> I can investigate this further, but I believe this patch itself gave a
> noticeable performance boost.
>
> This is an optimization anyway, and not critical for functionality.
>
It is not critical only if below code would not introduce races
+ spin_lock_bh(&sham.lock);
+ list_for_each_entry(dd, &sham.dev_list, list) {
+ break;
+ }
+ spin_unlock_bh(&sham.lock);
Is it guaranteed that dd will alive always at this moment?
+
+ pm_runtime_get_sync(dd->dev);
--
regards,
-grygorii
^ permalink raw reply
* Re: [RFC v4 2/4] crypto: Introduce CRYPTO_ALG_BULK flag
From: Herbert Xu @ 2016-06-07 14:16 UTC (permalink / raw)
To: Baolin Wang
Cc: axboe, agk, snitzer, dm-devel, davem, ebiggers3, js1304,
tadeusz.struk, smueller, standby24x7, shli, dan.j.williams,
martin.petersen, sagig, kent.overstreet, keith.busch, tj,
ming.lei, broonie, arnd, linux-crypto, linux-block, linux-raid,
linux-kernel
In-Reply-To: <238ce3d506051c863300b90720c3e103175747cc.1465301616.git.baolin.wang@linaro.org>
On Tue, Jun 07, 2016 at 08:17:05PM +0800, Baolin Wang wrote:
> Now some cipher hardware engines prefer to handle bulk block rather than one
> sector (512 bytes) created by dm-crypt, cause these cipher engines can handle
> the intermediate values (IV) by themselves in one bulk block. This means we
> can increase the size of the request by merging request rather than always 512
> bytes and thus increase the hardware engine processing speed.
>
> So introduce 'CRYPTO_ALG_BULK' flag to indicate this cipher can support bulk
> mode.
>
> Signed-off-by: Baolin Wang <baolin.wang@linaro.org>
Nack. As I said before, please do it using explicit IV generators
like we do for IPsec.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* [PATCH v3] crypto: rsa - return raw integers for the ASN.1 parser
From: Tudor Ambarus @ 2016-06-07 14:21 UTC (permalink / raw)
To: herbert; +Cc: linux-crypto, Tudor Ambarus
Return the raw key with no other processing so that the caller
can copy it or MPI parse it, etc.
The scope is to have only one ANS.1 parser for all RSA
implementations.
Update the RSA software implementation so that it does
the MPI conversion on top.
Signed-off-by: Tudor Ambarus <tudor-dan.ambarus@nxp.com>
---
crypto/rsa.c | 119 ++++++++++++++++++++++++++-------------
crypto/rsa_helper.c | 128 +++++++++++++++++++++---------------------
include/crypto/internal/rsa.h | 22 ++++++--
3 files changed, 161 insertions(+), 108 deletions(-)
diff --git a/crypto/rsa.c b/crypto/rsa.c
index 77d737f..ea78d61 100644
--- a/crypto/rsa.c
+++ b/crypto/rsa.c
@@ -14,12 +14,24 @@
#include <crypto/internal/akcipher.h>
#include <crypto/akcipher.h>
#include <crypto/algapi.h>
+#include <linux/mpi.h>
+
+struct rsa_mpi_key {
+ MPI n;
+ MPI e;
+ MPI d;
+};
+
+struct rsa_ctx {
+ struct rsa_key key;
+ struct rsa_mpi_key mpi_key;
+};
/*
* RSAEP function [RFC3447 sec 5.1.1]
* c = m^e mod n;
*/
-static int _rsa_enc(const struct rsa_key *key, MPI c, MPI m)
+static int _rsa_enc(const struct rsa_mpi_key *key, MPI c, MPI m)
{
/* (1) Validate 0 <= m < n */
if (mpi_cmp_ui(m, 0) < 0 || mpi_cmp(m, key->n) >= 0)
@@ -33,7 +45,7 @@ static int _rsa_enc(const struct rsa_key *key, MPI c, MPI m)
* RSADP function [RFC3447 sec 5.1.2]
* m = c^d mod n;
*/
-static int _rsa_dec(const struct rsa_key *key, MPI m, MPI c)
+static int _rsa_dec(const struct rsa_mpi_key *key, MPI m, MPI c)
{
/* (1) Validate 0 <= c < n */
if (mpi_cmp_ui(c, 0) < 0 || mpi_cmp(c, key->n) >= 0)
@@ -47,7 +59,7 @@ static int _rsa_dec(const struct rsa_key *key, MPI m, MPI c)
* RSASP1 function [RFC3447 sec 5.2.1]
* s = m^d mod n
*/
-static int _rsa_sign(const struct rsa_key *key, MPI s, MPI m)
+static int _rsa_sign(const struct rsa_mpi_key *key, MPI s, MPI m)
{
/* (1) Validate 0 <= m < n */
if (mpi_cmp_ui(m, 0) < 0 || mpi_cmp(m, key->n) >= 0)
@@ -61,7 +73,7 @@ static int _rsa_sign(const struct rsa_key *key, MPI s, MPI m)
* RSAVP1 function [RFC3447 sec 5.2.2]
* m = s^e mod n;
*/
-static int _rsa_verify(const struct rsa_key *key, MPI m, MPI s)
+static int _rsa_verify(const struct rsa_mpi_key *key, MPI m, MPI s)
{
/* (1) Validate 0 <= s < n */
if (mpi_cmp_ui(s, 0) < 0 || mpi_cmp(s, key->n) >= 0)
@@ -71,15 +83,17 @@ static int _rsa_verify(const struct rsa_key *key, MPI m, MPI s)
return mpi_powm(m, s, key->e, key->n);
}
-static inline struct rsa_key *rsa_get_key(struct crypto_akcipher *tfm)
+static inline struct rsa_mpi_key *rsa_get_key(struct crypto_akcipher *tfm)
{
- return akcipher_tfm_ctx(tfm);
+ struct rsa_ctx *ctx = akcipher_tfm_ctx(tfm);
+
+ return &ctx->mpi_key;
}
static int rsa_enc(struct akcipher_request *req)
{
struct crypto_akcipher *tfm = crypto_akcipher_reqtfm(req);
- const struct rsa_key *pkey = rsa_get_key(tfm);
+ const struct rsa_mpi_key *pkey = rsa_get_key(tfm);
MPI m, c = mpi_alloc(0);
int ret = 0;
int sign;
@@ -118,7 +132,7 @@ err_free_c:
static int rsa_dec(struct akcipher_request *req)
{
struct crypto_akcipher *tfm = crypto_akcipher_reqtfm(req);
- const struct rsa_key *pkey = rsa_get_key(tfm);
+ const struct rsa_mpi_key *pkey = rsa_get_key(tfm);
MPI c, m = mpi_alloc(0);
int ret = 0;
int sign;
@@ -156,7 +170,7 @@ err_free_m:
static int rsa_sign(struct akcipher_request *req)
{
struct crypto_akcipher *tfm = crypto_akcipher_reqtfm(req);
- const struct rsa_key *pkey = rsa_get_key(tfm);
+ const struct rsa_mpi_key *pkey = rsa_get_key(tfm);
MPI m, s = mpi_alloc(0);
int ret = 0;
int sign;
@@ -195,7 +209,7 @@ err_free_s:
static int rsa_verify(struct akcipher_request *req)
{
struct crypto_akcipher *tfm = crypto_akcipher_reqtfm(req);
- const struct rsa_key *pkey = rsa_get_key(tfm);
+ const struct rsa_mpi_key *pkey = rsa_get_key(tfm);
MPI s, m = mpi_alloc(0);
int ret = 0;
int sign;
@@ -233,67 +247,94 @@ err_free_m:
return ret;
}
-static int rsa_check_key_length(unsigned int len)
+static void rsa_free_mpi_key(struct rsa_mpi_key *key)
{
- switch (len) {
- case 512:
- case 1024:
- case 1536:
- case 2048:
- case 3072:
- case 4096:
- return 0;
- }
-
- return -EINVAL;
+ mpi_free(key->d);
+ mpi_free(key->e);
+ mpi_free(key->n);
+ key->d = NULL;
+ key->e = NULL;
+ key->n = NULL;
}
static int rsa_set_pub_key(struct crypto_akcipher *tfm, const void *key,
unsigned int keylen)
{
- struct rsa_key *pkey = akcipher_tfm_ctx(tfm);
+ struct rsa_ctx *ctx = akcipher_tfm_ctx(tfm);
+ struct rsa_key *pkey = &ctx->key;
+ struct rsa_mpi_key *mpi_key = &ctx->mpi_key;
int ret;
+ /* Free the old MPI key if any */
+ rsa_free_mpi_key(mpi_key);
+
ret = rsa_parse_pub_key(pkey, key, keylen);
if (ret)
return ret;
- if (rsa_check_key_length(mpi_get_size(pkey->n) << 3)) {
- rsa_free_key(pkey);
- ret = -EINVAL;
- }
- return ret;
+ mpi_key->e = mpi_read_raw_data(pkey->e, pkey->e_sz);
+ if (!mpi_key->e)
+ goto err;
+
+ mpi_key->n = mpi_read_raw_data(pkey->n, pkey->n_sz);
+ if (!mpi_key->n)
+ goto err;
+
+ return 0;
+
+err:
+ rsa_free_mpi_key(mpi_key);
+ return -ENOMEM;
}
static int rsa_set_priv_key(struct crypto_akcipher *tfm, const void *key,
unsigned int keylen)
{
- struct rsa_key *pkey = akcipher_tfm_ctx(tfm);
+ struct rsa_ctx *ctx = akcipher_tfm_ctx(tfm);
+ struct rsa_key *pkey = &ctx->key;
+ struct rsa_mpi_key *mpi_key = &ctx->mpi_key;
int ret;
+ /* Free the old MPI key if any */
+ rsa_free_mpi_key(mpi_key);
+
ret = rsa_parse_priv_key(pkey, key, keylen);
if (ret)
return ret;
- if (rsa_check_key_length(mpi_get_size(pkey->n) << 3)) {
- rsa_free_key(pkey);
- ret = -EINVAL;
- }
- return ret;
+ mpi_key->d = mpi_read_raw_data(pkey->d, pkey->n_sz);
+ if (!mpi_key->d)
+ goto err;
+
+ mpi_key->e = mpi_read_raw_data(pkey->e, pkey->e_sz);
+ if (!mpi_key->e)
+ goto err;
+
+ mpi_key->n = mpi_read_raw_data(pkey->n, pkey->n_sz);
+ if (!mpi_key->n)
+ goto err;
+
+ return 0;
+
+err:
+ rsa_free_mpi_key(mpi_key);
+ return -ENOMEM;
}
static int rsa_max_size(struct crypto_akcipher *tfm)
{
- struct rsa_key *pkey = akcipher_tfm_ctx(tfm);
+ struct rsa_ctx *ctx = akcipher_tfm_ctx(tfm);
+ struct rsa_key *pkey = &ctx->key;
- return pkey->n ? mpi_get_size(pkey->n) : -EINVAL;
+ return pkey->n ? pkey->n_sz : -EINVAL;
}
static void rsa_exit_tfm(struct crypto_akcipher *tfm)
{
- struct rsa_key *pkey = akcipher_tfm_ctx(tfm);
+ struct rsa_ctx *ctx = akcipher_tfm_ctx(tfm);
+ struct rsa_mpi_key *mpi_key = &ctx->mpi_key;
- rsa_free_key(pkey);
+ rsa_free_mpi_key(mpi_key);
}
static struct akcipher_alg rsa = {
@@ -310,7 +351,7 @@ static struct akcipher_alg rsa = {
.cra_driver_name = "rsa-generic",
.cra_priority = 100,
.cra_module = THIS_MODULE,
- .cra_ctxsize = sizeof(struct rsa_key),
+ .cra_ctxsize = sizeof(struct rsa_ctx),
},
};
diff --git a/crypto/rsa_helper.c b/crypto/rsa_helper.c
index d226f48..f0677b7 100644
--- a/crypto/rsa_helper.c
+++ b/crypto/rsa_helper.c
@@ -18,24 +18,50 @@
#include "rsapubkey-asn1.h"
#include "rsaprivkey-asn1.h"
+static int rsa_check_key_length(unsigned int len)
+{
+ switch (len) {
+ case 512:
+ case 1024:
+ case 1536:
+ case 2048:
+ case 3072:
+ case 4096:
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
int rsa_get_n(void *context, size_t hdrlen, unsigned char tag,
const void *value, size_t vlen)
{
struct rsa_key *key = context;
+ const u8 *ptr = value;
+ int ret;
- key->n = mpi_read_raw_data(value, vlen);
+ while (!*ptr && vlen) {
+ ptr++;
+ vlen--;
+ }
- if (!key->n)
- return -ENOMEM;
+ /* invalid key provided */
+ if (!ptr)
+ return -EINVAL;
/* In FIPS mode only allow key size 2K & 3K */
- if (fips_enabled && (mpi_get_size(key->n) != 256 &&
- mpi_get_size(key->n) != 384)) {
+ if (fips_enabled && (vlen != 256 && vlen != 384)) {
pr_err("RSA: key size not allowed in FIPS mode\n");
- mpi_free(key->n);
- key->n = NULL;
return -EINVAL;
}
+ /* invalid key size provided */
+ ret = rsa_check_key_length(vlen << 3);
+ if (ret)
+ return ret;
+
+ key->n = ptr;
+ key->n_sz = vlen;
+
return 0;
}
@@ -43,11 +69,19 @@ int rsa_get_e(void *context, size_t hdrlen, unsigned char tag,
const void *value, size_t vlen)
{
struct rsa_key *key = context;
+ const u8 *ptr = value;
+
+ while (!*ptr && vlen) {
+ ptr++;
+ vlen--;
+ }
- key->e = mpi_read_raw_data(value, vlen);
+ /* invalid key provided */
+ if (!ptr || !key->n_sz || !vlen || vlen > key->n_sz)
+ return -EINVAL;
- if (!key->e)
- return -ENOMEM;
+ key->e = ptr;
+ key->e_sz = vlen;
return 0;
}
@@ -56,47 +90,33 @@ int rsa_get_d(void *context, size_t hdrlen, unsigned char tag,
const void *value, size_t vlen)
{
struct rsa_key *key = context;
+ const u8 *ptr = value;
- key->d = mpi_read_raw_data(value, vlen);
+ while (!*ptr && vlen) {
+ ptr++;
+ vlen--;
+ }
- if (!key->d)
- return -ENOMEM;
+ /* invalid key provided */
+ if (!ptr || !key->n_sz || !vlen || vlen > key->n_sz)
+ return -EINVAL;
/* In FIPS mode only allow key size 2K & 3K */
- if (fips_enabled && (mpi_get_size(key->d) != 256 &&
- mpi_get_size(key->d) != 384)) {
+ if (fips_enabled && (vlen != 256 && vlen != 384)) {
pr_err("RSA: key size not allowed in FIPS mode\n");
- mpi_free(key->d);
- key->d = NULL;
return -EINVAL;
}
- return 0;
-}
-static void free_mpis(struct rsa_key *key)
-{
- mpi_free(key->n);
- mpi_free(key->e);
- mpi_free(key->d);
- key->n = NULL;
- key->e = NULL;
- key->d = NULL;
-}
+ key->d = ptr;
+ key->d_sz = vlen;
-/**
- * rsa_free_key() - frees rsa key allocated by rsa_parse_key()
- *
- * @rsa_key: struct rsa_key key representation
- */
-void rsa_free_key(struct rsa_key *key)
-{
- free_mpis(key);
+ return 0;
}
-EXPORT_SYMBOL_GPL(rsa_free_key);
/**
- * rsa_parse_pub_key() - extracts an rsa public key from BER encoded buffer
- * and stores it in the provided struct rsa_key
+ * rsa_parse_pub_key() - decodes the BER encoded buffer and stores in the
+ * provided struct rsa_key, pointers to the raw key as is,
+ * so that the caller can copy it or MPI parse it, etc.
*
* @rsa_key: struct rsa_key key representation
* @key: key in BER format
@@ -107,23 +127,15 @@ EXPORT_SYMBOL_GPL(rsa_free_key);
int rsa_parse_pub_key(struct rsa_key *rsa_key, const void *key,
unsigned int key_len)
{
- int ret;
-
- free_mpis(rsa_key);
- ret = asn1_ber_decoder(&rsapubkey_decoder, rsa_key, key, key_len);
- if (ret < 0)
- goto error;
-
- return 0;
-error:
- free_mpis(rsa_key);
- return ret;
+ return asn1_ber_decoder(&rsapubkey_decoder, rsa_key, key, key_len);
}
EXPORT_SYMBOL_GPL(rsa_parse_pub_key);
/**
- * rsa_parse_pub_key() - extracts an rsa private key from BER encoded buffer
- * and stores it in the provided struct rsa_key
+ * rsa_parse_priv_key() - decodes the BER encoded buffer and stores in the
+ * provided struct rsa_key, pointers to the raw key
+ * as is, so that the caller can copy it or MPI parse it,
+ * etc.
*
* @rsa_key: struct rsa_key key representation
* @key: key in BER format
@@ -134,16 +146,6 @@ EXPORT_SYMBOL_GPL(rsa_parse_pub_key);
int rsa_parse_priv_key(struct rsa_key *rsa_key, const void *key,
unsigned int key_len)
{
- int ret;
-
- free_mpis(rsa_key);
- ret = asn1_ber_decoder(&rsaprivkey_decoder, rsa_key, key, key_len);
- if (ret < 0)
- goto error;
-
- return 0;
-error:
- free_mpis(rsa_key);
- return ret;
+ return asn1_ber_decoder(&rsaprivkey_decoder, rsa_key, key, key_len);
}
EXPORT_SYMBOL_GPL(rsa_parse_priv_key);
diff --git a/include/crypto/internal/rsa.h b/include/crypto/internal/rsa.h
index c7585bd..d6c042a 100644
--- a/include/crypto/internal/rsa.h
+++ b/include/crypto/internal/rsa.h
@@ -12,12 +12,24 @@
*/
#ifndef _RSA_HELPER_
#define _RSA_HELPER_
-#include <linux/mpi.h>
+#include <linux/types.h>
+/**
+ * rsa_key - RSA key structure
+ * @n : RSA modulus raw byte stream
+ * @e : RSA public exponent raw byte stream
+ * @d : RSA private exponent raw byte stream
+ * @n_sz : length in bytes of RSA modulus n
+ * @e_sz : length in bytes of RSA public exponent
+ * @d_sz : length in bytes of RSA private exponent
+ */
struct rsa_key {
- MPI n;
- MPI e;
- MPI d;
+ const u8 *n;
+ const u8 *e;
+ const u8 *d;
+ size_t n_sz;
+ size_t e_sz;
+ size_t d_sz;
};
int rsa_parse_pub_key(struct rsa_key *rsa_key, const void *key,
@@ -26,7 +38,5 @@ int rsa_parse_pub_key(struct rsa_key *rsa_key, const void *key,
int rsa_parse_priv_key(struct rsa_key *rsa_key, const void *key,
unsigned int key_len);
-void rsa_free_key(struct rsa_key *rsa_key);
-
extern struct crypto_template rsa_pkcs1pad_tmpl;
#endif
--
1.8.3.1
^ permalink raw reply related
* [PATCH v7 0/3] crypto: caam - add support for RSA algorithm
From: Tudor Ambarus @ 2016-06-07 14:24 UTC (permalink / raw)
To: herbert; +Cc: linux-crypto, horia.geanta, Tudor Ambarus
Depends on:
[PATCH v3] crypto: rsa - return raw integers for the ASN.1 parser
Changes in v7:
- sync with ASN.1 parser
Changes in v6:
- write descriptor PDB fields with inline append
- move Protocol Data Block (pdb) structures to pdb.h
- move setting of PDB fields in new functions
- unmap sec4_sg_dma on done callback
- remove redundant clean code on error path
- fix doc typos
Changes in v5:
- sync with ASN.1 parser
Changes in v4:
- sync with ASN.1 parser
Changes in v3:
- sync with ASN.1 parser
Changes in v2:
- fix memory leaks on error path
- rename struct akcipher_alg rsa to caam_rsa
Tudor Ambarus (3):
crypto: scatterwak - Add scatterwalk_sg_copychunks
crypto: scatterwalk - export scatterwalk_pagedone
crypto: caam - add support for RSA algorithm
crypto/scatterwalk.c | 31 +-
drivers/crypto/caam/Kconfig | 12 +
drivers/crypto/caam/Makefile | 4 +
drivers/crypto/caam/caampkc.c | 637 ++++++++++++++++++++++++++++++++++++++
drivers/crypto/caam/caampkc.h | 72 +++++
drivers/crypto/caam/compat.h | 3 +
drivers/crypto/caam/desc.h | 2 +
drivers/crypto/caam/desc_constr.h | 7 +
drivers/crypto/caam/pdb.h | 51 ++-
drivers/crypto/caam/pkc_desc.c | 36 +++
include/crypto/scatterwalk.h | 4 +
11 files changed, 856 insertions(+), 3 deletions(-)
create mode 100644 drivers/crypto/caam/caampkc.c
create mode 100644 drivers/crypto/caam/caampkc.h
create mode 100644 drivers/crypto/caam/pkc_desc.c
--
1.8.3.1
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox