* [PATCH 1/10] crypto: aead - Add type-safe init/exit functions
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
@ 2015-05-28 14:07 ` Herbert Xu
2015-05-28 14:07 ` [PATCH 2/10] crypto: aead - Add aead_alg_instance Herbert Xu
` (9 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:07 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
As it stands the only non-type safe functions left in the new
AEAD interface are the cra_init/cra_exit functions. It means
exposing the ugly __crypto_aead_cast to every AEAD implementor.
This patch adds type-safe init/exit functions to AEAD. Existing
algorithms are unaffected while new implementations can simply
fill in these two instead of cra_init/cra_exit.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/aead.c | 14 ++++++++++++++
include/crypto/aead.h | 13 +++++++++++++
2 files changed, 27 insertions(+)
diff --git a/crypto/aead.c b/crypto/aead.c
index 8cdea89..4bab3cf 100644
--- a/crypto/aead.c
+++ b/crypto/aead.c
@@ -174,6 +174,14 @@ static int crypto_old_aead_init_tfm(struct crypto_tfm *tfm)
return 0;
}
+static void crypto_aead_exit_tfm(struct crypto_tfm *tfm)
+{
+ struct crypto_aead *aead = __crypto_aead_cast(tfm);
+ struct aead_alg *alg = crypto_aead_alg(aead);
+
+ alg->exit(aead);
+}
+
static int crypto_aead_init_tfm(struct crypto_tfm *tfm)
{
struct crypto_aead *aead = __crypto_aead_cast(tfm);
@@ -189,6 +197,12 @@ static int crypto_aead_init_tfm(struct crypto_tfm *tfm)
aead->child = __crypto_aead_cast(tfm);
aead->authsize = alg->maxauthsize;
+ if (alg->exit)
+ aead->base.exit = crypto_aead_exit_tfm;
+
+ if (alg->init)
+ return alg->init(aead);
+
return 0;
}
diff --git a/include/crypto/aead.h b/include/crypto/aead.h
index 61306ed..680d4b0 100644
--- a/include/crypto/aead.h
+++ b/include/crypto/aead.h
@@ -108,6 +108,17 @@ struct aead_givcrypt_request {
* @decrypt: see struct ablkcipher_alg
* @geniv: see struct ablkcipher_alg
* @ivsize: see struct ablkcipher_alg
+ * @init: Initialize the cryptographic transformation object. This function
+ * is used to initialize the cryptographic transformation object.
+ * This function is called only once at the instantiation time, right
+ * after the transformation context was allocated. In case the
+ * cryptographic hardware has some special requirements which need to
+ * be handled by software, this function shall check for the precise
+ * requirement of the transformation and put any software fallbacks
+ * in place.
+ * @exit: Deinitialize the cryptographic transformation object. This is a
+ * counterpart to @init, used to remove various changes set in
+ * @init.
*
* All fields except @ivsize is mandatory and must be filled.
*/
@@ -117,6 +128,8 @@ struct aead_alg {
int (*setauthsize)(struct crypto_aead *tfm, unsigned int authsize);
int (*encrypt)(struct aead_request *req);
int (*decrypt)(struct aead_request *req);
+ int (*init)(struct crypto_aead *tfm);
+ void (*exit)(struct crypto_aead *tfm);
const char *geniv;
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 2/10] crypto: aead - Add aead_alg_instance
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
2015-05-28 14:07 ` [PATCH 1/10] crypto: aead - Add type-safe init/exit functions Herbert Xu
@ 2015-05-28 14:07 ` Herbert Xu
2015-05-28 14:07 ` [PATCH 3/10] crypto: api - Include alignment in crypto_alg_extsize Herbert Xu
` (8 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:07 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
Now that type-safe init/exit functions exist, they often need
to access the underlying aead_instance. So this patch adds the
helper aead_alg_instance to access aead_instance from a crypto_aead
object.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
include/crypto/internal/aead.h | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/include/crypto/internal/aead.h b/include/crypto/internal/aead.h
index 4137330..f068d74 100644
--- a/include/crypto/internal/aead.h
+++ b/include/crypto/internal/aead.h
@@ -52,6 +52,11 @@ static inline struct aead_instance *aead_instance(struct crypto_instance *inst)
return container_of(&inst->alg, struct aead_instance, alg.base);
}
+static inline struct aead_instance *aead_alg_instance(struct crypto_aead *aead)
+{
+ return aead_instance(crypto_aead_alg_instance(aead));
+}
+
static inline void *aead_instance_ctx(struct aead_instance *inst)
{
return crypto_instance_ctx(aead_crypto_instance(inst));
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 3/10] crypto: api - Include alignment in crypto_alg_extsize
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
2015-05-28 14:07 ` [PATCH 1/10] crypto: aead - Add type-safe init/exit functions Herbert Xu
2015-05-28 14:07 ` [PATCH 2/10] crypto: aead - Add aead_alg_instance Herbert Xu
@ 2015-05-28 14:07 ` Herbert Xu
2015-05-28 14:07 ` [PATCH 4/10] crypto: aead - Ignore return value from crypto_unregister_alg Herbert Xu
` (7 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:07 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch ensures that the tfm context always has enough extra
memory to ensure that it is aligned according to cra_alignment.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/algapi.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/crypto/algapi.c b/crypto/algapi.c
index abf100c..3c079b7 100644
--- a/crypto/algapi.c
+++ b/crypto/algapi.c
@@ -980,7 +980,8 @@ EXPORT_SYMBOL_GPL(crypto_xor);
unsigned int crypto_alg_extsize(struct crypto_alg *alg)
{
- return alg->cra_ctxsize;
+ return alg->cra_ctxsize +
+ (alg->cra_alignmask & ~(crypto_tfm_ctx_alignment() - 1));
}
EXPORT_SYMBOL_GPL(crypto_alg_extsize);
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 4/10] crypto: aead - Ignore return value from crypto_unregister_alg
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (2 preceding siblings ...)
2015-05-28 14:07 ` [PATCH 3/10] crypto: api - Include alignment in crypto_alg_extsize Herbert Xu
@ 2015-05-28 14:07 ` Herbert Xu
2015-05-28 14:07 ` [PATCH 5/10] crypto: aead - Add multiple algorithm registration interface Herbert Xu
` (6 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:07 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
No new code should be using the return value of crypto_unregister_alg
as it will become void soon.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/aead.c | 4 ++--
include/crypto/internal/aead.h | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/crypto/aead.c b/crypto/aead.c
index 4bab3cf..ac44792 100644
--- a/crypto/aead.c
+++ b/crypto/aead.c
@@ -890,9 +890,9 @@ int crypto_register_aead(struct aead_alg *alg)
}
EXPORT_SYMBOL_GPL(crypto_register_aead);
-int crypto_unregister_aead(struct aead_alg *alg)
+void crypto_unregister_aead(struct aead_alg *alg)
{
- return crypto_unregister_alg(&alg->base);
+ crypto_unregister_alg(&alg->base);
}
EXPORT_SYMBOL_GPL(crypto_unregister_aead);
diff --git a/include/crypto/internal/aead.h b/include/crypto/internal/aead.h
index f068d74..3cb35d8 100644
--- a/include/crypto/internal/aead.h
+++ b/include/crypto/internal/aead.h
@@ -151,7 +151,7 @@ static inline unsigned int crypto_aead_maxauthsize(struct crypto_aead *aead)
}
int crypto_register_aead(struct aead_alg *alg);
-int crypto_unregister_aead(struct aead_alg *alg);
+void crypto_unregister_aead(struct aead_alg *alg);
int aead_register_instance(struct crypto_template *tmpl,
struct aead_instance *inst);
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 5/10] crypto: aead - Add multiple algorithm registration interface
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (3 preceding siblings ...)
2015-05-28 14:07 ` [PATCH 4/10] crypto: aead - Ignore return value from crypto_unregister_alg Herbert Xu
@ 2015-05-28 14:07 ` Herbert Xu
2015-05-28 14:08 ` [PATCH 6/10] crypto: pcrypt - Add support for new AEAD interface Herbert Xu
` (5 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:07 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch adds the helpers that allow the registration and removal
of multiple algorithms.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/aead.c | 29 +++++++++++++++++++++++++++++
include/crypto/internal/aead.h | 2 ++
2 files changed, 31 insertions(+)
diff --git a/crypto/aead.c b/crypto/aead.c
index ac44792..07bf997 100644
--- a/crypto/aead.c
+++ b/crypto/aead.c
@@ -896,6 +896,35 @@ void crypto_unregister_aead(struct aead_alg *alg)
}
EXPORT_SYMBOL_GPL(crypto_unregister_aead);
+int crypto_register_aeads(struct aead_alg *algs, int count)
+{
+ int i, ret;
+
+ for (i = 0; i < count; i++) {
+ ret = crypto_register_aead(&algs[i]);
+ if (ret)
+ goto err;
+ }
+
+ return 0;
+
+err:
+ for (--i; i >= 0; --i)
+ crypto_unregister_aead(&algs[i]);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(crypto_register_aeads);
+
+void crypto_unregister_aeads(struct aead_alg *algs, int count)
+{
+ int i;
+
+ for (i = count - 1; i >= 0; --i)
+ crypto_unregister_aead(&algs[i]);
+}
+EXPORT_SYMBOL_GPL(crypto_unregister_aeads);
+
int aead_register_instance(struct crypto_template *tmpl,
struct aead_instance *inst)
{
diff --git a/include/crypto/internal/aead.h b/include/crypto/internal/aead.h
index 3cb35d8..ba52c37b8 100644
--- a/include/crypto/internal/aead.h
+++ b/include/crypto/internal/aead.h
@@ -152,6 +152,8 @@ static inline unsigned int crypto_aead_maxauthsize(struct crypto_aead *aead)
int crypto_register_aead(struct aead_alg *alg);
void crypto_unregister_aead(struct aead_alg *alg);
+int crypto_register_aeads(struct aead_alg *algs, int count);
+void crypto_unregister_aeads(struct aead_alg *algs, int count);
int aead_register_instance(struct crypto_template *tmpl,
struct aead_instance *inst);
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 6/10] crypto: pcrypt - Add support for new AEAD interface
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (4 preceding siblings ...)
2015-05-28 14:07 ` [PATCH 5/10] crypto: aead - Add multiple algorithm registration interface Herbert Xu
@ 2015-05-28 14:08 ` Herbert Xu
2015-05-28 14:08 ` [PATCH 7/10] crypto: cryptd - Add setkey/setauthsize functions for AEAD Herbert Xu
` (4 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:08 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch converts pcrypt over to the new AEAD interface.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/pcrypt.c | 135 ++++++++++++++++----------------------------------------
1 file changed, 40 insertions(+), 95 deletions(-)
diff --git a/crypto/pcrypt.c b/crypto/pcrypt.c
index ff174b6..45e7d51 100644
--- a/crypto/pcrypt.c
+++ b/crypto/pcrypt.c
@@ -123,14 +123,6 @@ static void pcrypt_aead_serial(struct padata_priv *padata)
aead_request_complete(req->base.data, padata->info);
}
-static void pcrypt_aead_giv_serial(struct padata_priv *padata)
-{
- struct pcrypt_request *preq = pcrypt_padata_request(padata);
- struct aead_givcrypt_request *req = pcrypt_request_ctx(preq);
-
- aead_request_complete(req->areq.base.data, padata->info);
-}
-
static void pcrypt_aead_done(struct crypto_async_request *areq, int err)
{
struct aead_request *req = areq->data;
@@ -176,7 +168,7 @@ static int pcrypt_aead_encrypt(struct aead_request *req)
pcrypt_aead_done, req);
aead_request_set_crypt(creq, req->src, req->dst,
req->cryptlen, req->iv);
- aead_request_set_assoc(creq, req->assoc, req->assoclen);
+ aead_request_set_ad(creq, req->assoclen);
err = pcrypt_do_parallel(padata, &ctx->cb_cpu, &pencrypt);
if (!err)
@@ -218,7 +210,7 @@ static int pcrypt_aead_decrypt(struct aead_request *req)
pcrypt_aead_done, req);
aead_request_set_crypt(creq, req->src, req->dst,
req->cryptlen, req->iv);
- aead_request_set_assoc(creq, req->assoc, req->assoclen);
+ aead_request_set_ad(creq, req->assoclen);
err = pcrypt_do_parallel(padata, &ctx->cb_cpu, &pdecrypt);
if (!err)
@@ -227,56 +219,12 @@ static int pcrypt_aead_decrypt(struct aead_request *req)
return err;
}
-static void pcrypt_aead_givenc(struct padata_priv *padata)
-{
- struct pcrypt_request *preq = pcrypt_padata_request(padata);
- struct aead_givcrypt_request *req = pcrypt_request_ctx(preq);
-
- padata->info = crypto_aead_givencrypt(req);
-
- if (padata->info == -EINPROGRESS)
- return;
-
- padata_do_serial(padata);
-}
-
-static int pcrypt_aead_givencrypt(struct aead_givcrypt_request *req)
-{
- int err;
- struct aead_request *areq = &req->areq;
- struct pcrypt_request *preq = aead_request_ctx(areq);
- struct aead_givcrypt_request *creq = pcrypt_request_ctx(preq);
- struct padata_priv *padata = pcrypt_request_padata(preq);
- struct crypto_aead *aead = aead_givcrypt_reqtfm(req);
- struct pcrypt_aead_ctx *ctx = crypto_aead_ctx(aead);
- u32 flags = aead_request_flags(areq);
-
- memset(padata, 0, sizeof(struct padata_priv));
-
- padata->parallel = pcrypt_aead_givenc;
- padata->serial = pcrypt_aead_giv_serial;
-
- aead_givcrypt_set_tfm(creq, ctx->child);
- aead_givcrypt_set_callback(creq, flags & ~CRYPTO_TFM_REQ_MAY_SLEEP,
- pcrypt_aead_done, areq);
- aead_givcrypt_set_crypt(creq, areq->src, areq->dst,
- areq->cryptlen, areq->iv);
- aead_givcrypt_set_assoc(creq, areq->assoc, areq->assoclen);
- aead_givcrypt_set_giv(creq, req->giv, req->seq);
-
- err = pcrypt_do_parallel(padata, &ctx->cb_cpu, &pencrypt);
- if (!err)
- return -EINPROGRESS;
-
- return err;
-}
-
-static int pcrypt_aead_init_tfm(struct crypto_tfm *tfm)
+static int pcrypt_aead_init_tfm(struct crypto_aead *tfm)
{
int cpu, cpu_index;
- struct crypto_instance *inst = crypto_tfm_alg_instance(tfm);
- struct pcrypt_instance_ctx *ictx = crypto_instance_ctx(inst);
- struct pcrypt_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct aead_instance *inst = aead_alg_instance(tfm);
+ struct pcrypt_instance_ctx *ictx = aead_instance_ctx(inst);
+ struct pcrypt_aead_ctx *ctx = crypto_aead_ctx(tfm);
struct crypto_aead *cipher;
cpu_index = (unsigned int)atomic_inc_return(&ictx->tfm_count) %
@@ -286,23 +234,22 @@ static int pcrypt_aead_init_tfm(struct crypto_tfm *tfm)
for (cpu = 0; cpu < cpu_index; cpu++)
ctx->cb_cpu = cpumask_next(ctx->cb_cpu, cpu_online_mask);
- cipher = crypto_spawn_aead(crypto_instance_ctx(inst));
+ cipher = crypto_spawn_aead(&ictx->spawn);
if (IS_ERR(cipher))
return PTR_ERR(cipher);
ctx->child = cipher;
- crypto_aead_set_reqsize(__crypto_aead_cast(tfm),
- sizeof(struct pcrypt_request) +
- sizeof(struct aead_givcrypt_request) +
- crypto_aead_reqsize(cipher));
+ crypto_aead_set_reqsize(tfm, sizeof(struct pcrypt_request) +
+ sizeof(struct aead_request) +
+ crypto_aead_reqsize(cipher));
return 0;
}
-static void pcrypt_aead_exit_tfm(struct crypto_tfm *tfm)
+static void pcrypt_aead_exit_tfm(struct crypto_aead *tfm)
{
- struct pcrypt_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct pcrypt_aead_ctx *ctx = crypto_aead_ctx(tfm);
crypto_free_aead(ctx->child);
}
@@ -323,78 +270,76 @@ static int pcrypt_init_instance(struct crypto_instance *inst,
return 0;
}
-static struct crypto_instance *pcrypt_alloc_aead(struct rtattr **tb,
- u32 type, u32 mask)
+static int pcrypt_create_aead(struct crypto_template *tmpl, struct rtattr **tb,
+ u32 type, u32 mask)
{
struct pcrypt_instance_ctx *ctx;
- struct crypto_instance *inst;
- struct crypto_alg *alg;
+ struct aead_instance *inst;
+ struct aead_alg *alg;
const char *name;
int err;
name = crypto_attr_alg_name(tb[1]);
if (IS_ERR(name))
- return ERR_CAST(name);
+ return PTR_ERR(name);
inst = kzalloc(sizeof(*inst) + sizeof(*ctx), GFP_KERNEL);
if (!inst)
- return ERR_PTR(-ENOMEM);
+ return -ENOMEM;
- ctx = crypto_instance_ctx(inst);
- crypto_set_aead_spawn(&ctx->spawn, inst);
+ ctx = aead_instance_ctx(inst);
+ crypto_set_aead_spawn(&ctx->spawn, aead_crypto_instance(inst));
err = crypto_grab_aead(&ctx->spawn, name, 0, 0);
if (err)
goto out_free_inst;
- alg = crypto_aead_spawn_alg(&ctx->spawn);
- err = pcrypt_init_instance(inst, alg);
+ alg = crypto_spawn_aead_alg(&ctx->spawn);
+ err = pcrypt_init_instance(aead_crypto_instance(inst), &alg->base);
if (err)
goto out_drop_aead;
- inst->alg.cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC;
- inst->alg.cra_type = &crypto_aead_type;
+ inst->alg.ivsize = crypto_aead_alg_ivsize(alg);
+ inst->alg.maxauthsize = crypto_aead_alg_maxauthsize(alg);
- inst->alg.cra_aead.ivsize = alg->cra_aead.ivsize;
- inst->alg.cra_aead.geniv = alg->cra_aead.geniv;
- inst->alg.cra_aead.maxauthsize = alg->cra_aead.maxauthsize;
+ inst->alg.base.cra_ctxsize = sizeof(struct pcrypt_aead_ctx);
- inst->alg.cra_ctxsize = sizeof(struct pcrypt_aead_ctx);
+ inst->alg.init = pcrypt_aead_init_tfm;
+ inst->alg.exit = pcrypt_aead_exit_tfm;
- inst->alg.cra_init = pcrypt_aead_init_tfm;
- inst->alg.cra_exit = pcrypt_aead_exit_tfm;
+ inst->alg.setkey = pcrypt_aead_setkey;
+ inst->alg.setauthsize = pcrypt_aead_setauthsize;
+ inst->alg.encrypt = pcrypt_aead_encrypt;
+ inst->alg.decrypt = pcrypt_aead_decrypt;
- inst->alg.cra_aead.setkey = pcrypt_aead_setkey;
- inst->alg.cra_aead.setauthsize = pcrypt_aead_setauthsize;
- inst->alg.cra_aead.encrypt = pcrypt_aead_encrypt;
- inst->alg.cra_aead.decrypt = pcrypt_aead_decrypt;
- inst->alg.cra_aead.givencrypt = pcrypt_aead_givencrypt;
+ err = aead_register_instance(tmpl, inst);
+ if (err)
+ goto out_drop_aead;
out:
- return inst;
+ return err;
out_drop_aead:
crypto_drop_aead(&ctx->spawn);
out_free_inst:
kfree(inst);
- inst = ERR_PTR(err);
goto out;
}
-static struct crypto_instance *pcrypt_alloc(struct rtattr **tb)
+static int pcrypt_create(struct crypto_template *tmpl, struct rtattr **tb)
{
struct crypto_attr_type *algt;
algt = crypto_get_attr_type(tb);
if (IS_ERR(algt))
- return ERR_CAST(algt);
+ return PTR_ERR(algt);
switch (algt->type & algt->mask & CRYPTO_ALG_TYPE_MASK) {
case CRYPTO_ALG_TYPE_AEAD:
- return pcrypt_alloc_aead(tb, algt->type, algt->mask);
+ return pcrypt_create_aead(tmpl, tb, algt->type, algt->mask);
}
- return ERR_PTR(-EINVAL);
+ return -EINVAL;
}
static void pcrypt_free(struct crypto_instance *inst)
@@ -516,7 +461,7 @@ static void pcrypt_fini_padata(struct padata_pcrypt *pcrypt)
static struct crypto_template pcrypt_tmpl = {
.name = "pcrypt",
- .alloc = pcrypt_alloc,
+ .create = pcrypt_create,
.free = pcrypt_free,
.module = THIS_MODULE,
};
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 7/10] crypto: cryptd - Add setkey/setauthsize functions for AEAD
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (5 preceding siblings ...)
2015-05-28 14:08 ` [PATCH 6/10] crypto: pcrypt - Add support for new AEAD interface Herbert Xu
@ 2015-05-28 14:08 ` Herbert Xu
2015-05-28 14:08 ` [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface Herbert Xu
` (3 subsequent siblings)
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:08 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch adds setkey and setauthsize for cryptd AEAD.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/cryptd.c | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/crypto/cryptd.c b/crypto/cryptd.c
index 4264c8d..74c4dcb 100644
--- a/crypto/cryptd.c
+++ b/crypto/cryptd.c
@@ -664,6 +664,24 @@ out_put_alg:
return err;
}
+static int cryptd_aead_setkey(struct crypto_aead *parent,
+ const u8 *key, unsigned int keylen)
+{
+ struct cryptd_aead_ctx *ctx = crypto_aead_ctx(parent);
+ struct crypto_aead *child = ctx->child;
+
+ return crypto_aead_setkey(child, key, keylen);
+}
+
+static int cryptd_aead_setauthsize(struct crypto_aead *parent,
+ unsigned int authsize)
+{
+ struct cryptd_aead_ctx *ctx = crypto_aead_ctx(parent);
+ struct crypto_aead *child = ctx->child;
+
+ return crypto_aead_setauthsize(child, authsize);
+}
+
static void cryptd_aead_crypt(struct aead_request *req,
struct crypto_aead *child,
int err,
@@ -793,8 +811,8 @@ static int cryptd_create_aead(struct crypto_template *tmpl,
inst->alg.cra_ctxsize = sizeof(struct cryptd_aead_ctx);
inst->alg.cra_init = cryptd_aead_init_tfm;
inst->alg.cra_exit = cryptd_aead_exit_tfm;
- inst->alg.cra_aead.setkey = alg->cra_aead.setkey;
- inst->alg.cra_aead.setauthsize = alg->cra_aead.setauthsize;
+ inst->alg.cra_aead.setkey = cryptd_aead_setkey;
+ inst->alg.cra_aead.setauthsize = cryptd_aead_setauthsize;
inst->alg.cra_aead.geniv = alg->cra_aead.geniv;
inst->alg.cra_aead.ivsize = alg->cra_aead.ivsize;
inst->alg.cra_aead.maxauthsize = alg->cra_aead.maxauthsize;
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (6 preceding siblings ...)
2015-05-28 14:08 ` [PATCH 7/10] crypto: cryptd - Add setkey/setauthsize functions for AEAD Herbert Xu
@ 2015-05-28 14:08 ` Herbert Xu
2015-05-29 21:39 ` Stephan Mueller
2015-05-28 14:08 ` [PATCH 9/10] crypto: cryptd - Convert to new AEAD interface Herbert Xu
` (2 subsequent siblings)
10 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:08 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch converts rfc4106-gcm-aesni to the new AEAD interface.
The low-level interface remains as is for now because we can't
touch it until cryptd itself is upgraded.
In the conversion I've also removed the duplicate copy of the
context in the top-level algorithm. Now all processing is carried
out in the low-level __driver-gcm-aes-aesni algorithm.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
arch/x86/crypto/aesni-intel_glue.c | 172 +++++++++++++++++--------------------
1 file changed, 83 insertions(+), 89 deletions(-)
diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index 2ade2400..5660a18 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -52,7 +52,6 @@ struct aesni_rfc4106_gcm_ctx {
u8 hash_subkey[16];
struct crypto_aes_ctx aes_key_expanded;
u8 nonce[4];
- struct cryptd_aead *cryptd_tfm;
};
struct aesni_gcm_set_hash_subkey_result {
@@ -790,37 +789,30 @@ static int xts_decrypt(struct blkcipher_desc *desc, struct scatterlist *dst,
#endif
#ifdef CONFIG_X86_64
-static int rfc4106_init(struct crypto_tfm *tfm)
+static int rfc4106_init(struct crypto_aead *aead)
{
struct cryptd_aead *cryptd_tfm;
- struct aesni_rfc4106_gcm_ctx *ctx = (struct aesni_rfc4106_gcm_ctx *)
- PTR_ALIGN((u8 *)crypto_tfm_ctx(tfm), AESNI_ALIGN);
- struct crypto_aead *cryptd_child;
- struct aesni_rfc4106_gcm_ctx *child_ctx;
+ struct cryptd_aead **ctx = crypto_aead_ctx(aead);
+
cryptd_tfm = cryptd_alloc_aead("__driver-gcm-aes-aesni",
CRYPTO_ALG_INTERNAL,
CRYPTO_ALG_INTERNAL);
if (IS_ERR(cryptd_tfm))
return PTR_ERR(cryptd_tfm);
- cryptd_child = cryptd_aead_child(cryptd_tfm);
- child_ctx = aesni_rfc4106_gcm_ctx_get(cryptd_child);
- memcpy(child_ctx, ctx, sizeof(*ctx));
- ctx->cryptd_tfm = cryptd_tfm;
- crypto_aead_set_reqsize(__crypto_aead_cast(tfm),
+ *ctx = cryptd_tfm;
+ crypto_aead_set_reqsize(
+ aead,
sizeof(struct aead_request) +
crypto_aead_reqsize(&cryptd_tfm->base));
return 0;
}
-static void rfc4106_exit(struct crypto_tfm *tfm)
+static void rfc4106_exit(struct crypto_aead *aead)
{
- struct aesni_rfc4106_gcm_ctx *ctx =
- (struct aesni_rfc4106_gcm_ctx *)
- PTR_ALIGN((u8 *)crypto_tfm_ctx(tfm), AESNI_ALIGN);
- if (!IS_ERR(ctx->cryptd_tfm))
- cryptd_free_aead(ctx->cryptd_tfm);
- return;
+ struct cryptd_aead **ctx = crypto_aead_ctx(aead);
+
+ cryptd_free_aead(*ctx);
}
static void
@@ -951,18 +943,10 @@ exit:
static int rfc4106_set_key(struct crypto_aead *parent, const u8 *key,
unsigned int key_len)
{
- struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(parent);
- struct crypto_aead *child = cryptd_aead_child(ctx->cryptd_tfm);
- struct aesni_rfc4106_gcm_ctx *c_ctx = aesni_rfc4106_gcm_ctx_get(child);
- struct cryptd_aead *cryptd_tfm = ctx->cryptd_tfm;
- int ret;
+ struct cryptd_aead **ctx = crypto_aead_ctx(parent);
+ struct cryptd_aead *cryptd_tfm = *ctx;
- ret = crypto_aead_setkey(child, key, key_len);
- if (!ret) {
- memcpy(ctx, c_ctx, sizeof(*ctx));
- ctx->cryptd_tfm = cryptd_tfm;
- }
- return ret;
+ return crypto_aead_setkey(&cryptd_tfm->base, key, key_len);
}
static int common_rfc4106_set_authsize(struct crypto_aead *aead,
@@ -985,14 +969,10 @@ static int common_rfc4106_set_authsize(struct crypto_aead *aead,
static int rfc4106_set_authsize(struct crypto_aead *parent,
unsigned int authsize)
{
- struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(parent);
- struct crypto_aead *child = cryptd_aead_child(ctx->cryptd_tfm);
- int ret;
+ struct cryptd_aead **ctx = crypto_aead_ctx(parent);
+ struct cryptd_aead *cryptd_tfm = *ctx;
- ret = crypto_aead_setauthsize(child, authsize);
- if (!ret)
- crypto_aead_crt(parent)->authsize = authsize;
- return ret;
+ return crypto_aead_setauthsize(&cryptd_tfm->base, authsize);
}
static int __driver_rfc4106_encrypt(struct aead_request *req)
@@ -1171,44 +1151,42 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
static int rfc4106_encrypt(struct aead_request *req)
{
- int ret;
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
- struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
+ struct cryptd_aead **ctx = crypto_aead_ctx(tfm);
+ struct cryptd_aead *cryptd_tfm = *ctx;
+ struct aead_request *subreq = aead_request_ctx(req);
- if (!irq_fpu_usable()) {
- struct aead_request *cryptd_req =
- (struct aead_request *) aead_request_ctx(req);
+ aead_request_set_tfm(subreq, irq_fpu_usable() ?
+ cryptd_aead_child(cryptd_tfm) :
+ &cryptd_tfm->base);
- memcpy(cryptd_req, req, sizeof(*req));
- aead_request_set_tfm(cryptd_req, &ctx->cryptd_tfm->base);
- ret = crypto_aead_encrypt(cryptd_req);
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_encrypt(req);
- kernel_fpu_end();
- }
- return ret;
+ aead_request_set_callback(subreq, req->base.flags,
+ req->base.complete, req->base.data);
+ aead_request_set_crypt(subreq, req->src, req->dst,
+ req->cryptlen, req->iv);
+ aead_request_set_ad(subreq, req->assoclen);
+
+ return crypto_aead_encrypt(subreq);
}
static int rfc4106_decrypt(struct aead_request *req)
{
- int ret;
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
- struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
+ struct cryptd_aead **ctx = crypto_aead_ctx(tfm);
+ struct cryptd_aead *cryptd_tfm = *ctx;
+ struct aead_request *subreq = aead_request_ctx(req);
- if (!irq_fpu_usable()) {
- struct aead_request *cryptd_req =
- (struct aead_request *) aead_request_ctx(req);
+ aead_request_set_tfm(subreq, irq_fpu_usable() ?
+ cryptd_aead_child(cryptd_tfm) :
+ &cryptd_tfm->base);
- memcpy(cryptd_req, req, sizeof(*req));
- aead_request_set_tfm(cryptd_req, &ctx->cryptd_tfm->base);
- ret = crypto_aead_decrypt(cryptd_req);
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_decrypt(req);
- kernel_fpu_end();
- }
- return ret;
+ aead_request_set_callback(subreq, req->base.flags,
+ req->base.complete, req->base.data);
+ aead_request_set_crypt(subreq, req->src, req->dst,
+ req->cryptlen, req->iv);
+ aead_request_set_ad(subreq, req->assoclen);
+
+ return crypto_aead_decrypt(subreq);
}
static int helper_rfc4106_encrypt(struct aead_request *req)
@@ -1432,30 +1410,6 @@ static struct crypto_alg aesni_algs[] = { {
.maxauthsize = 16,
},
},
-}, {
- .cra_name = "rfc4106(gcm(aes))",
- .cra_driver_name = "rfc4106-gcm-aesni",
- .cra_priority = 400,
- .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC,
- .cra_blocksize = 1,
- .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx) +
- AESNI_ALIGN,
- .cra_alignmask = 0,
- .cra_type = &crypto_nivaead_type,
- .cra_module = THIS_MODULE,
- .cra_init = rfc4106_init,
- .cra_exit = rfc4106_exit,
- .cra_u = {
- .aead = {
- .setkey = rfc4106_set_key,
- .setauthsize = rfc4106_set_authsize,
- .encrypt = rfc4106_encrypt,
- .decrypt = rfc4106_decrypt,
- .geniv = "seqiv",
- .ivsize = 8,
- .maxauthsize = 16,
- },
- },
#endif
#if IS_ENABLED(CONFIG_CRYPTO_PCBC)
}, {
@@ -1570,6 +1524,30 @@ static struct crypto_alg aesni_algs[] = { {
},
} };
+#ifdef CONFIG_X86_64
+static struct aead_alg aesni_aead_algs[] = { {
+ .init = rfc4106_init,
+ .exit = rfc4106_exit,
+ .setkey = rfc4106_set_key,
+ .setauthsize = rfc4106_set_authsize,
+ .encrypt = rfc4106_encrypt,
+ .decrypt = rfc4106_decrypt,
+ .ivsize = 8,
+ .maxauthsize = 16,
+ .base = {
+ .cra_name = "rfc4106(gcm(aes))",
+ .cra_driver_name = "rfc4106-gcm-aesni",
+ .cra_priority = 400,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct cryptd_aead *),
+ .cra_module = THIS_MODULE,
+ },
+} };
+#else
+static struct aead_alg aesni_aead_algs[0];
+#endif
+
static const struct x86_cpu_id aesni_cpu_id[] = {
X86_FEATURE_MATCH(X86_FEATURE_AES),
@@ -1617,11 +1595,27 @@ static int __init aesni_init(void)
if (err)
return err;
- return crypto_register_algs(aesni_algs, ARRAY_SIZE(aesni_algs));
+ err = crypto_register_algs(aesni_algs, ARRAY_SIZE(aesni_algs));
+ if (err)
+ goto fpu_exit;
+
+ err = crypto_register_aeads(aesni_aead_algs,
+ ARRAY_SIZE(aesni_aead_algs));
+ if (err)
+ goto unregister_algs;
+
+ return err;
+
+unregister_algs:
+ crypto_unregister_algs(aesni_algs, ARRAY_SIZE(aesni_algs));
+fpu_exit:
+ crypto_fpu_exit();
+ return err;
}
static void __exit aesni_exit(void)
{
+ crypto_unregister_aeads(aesni_aead_algs, ARRAY_SIZE(aesni_aead_algs));
crypto_unregister_algs(aesni_algs, ARRAY_SIZE(aesni_algs));
crypto_fpu_exit();
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface
2015-05-28 14:08 ` [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface Herbert Xu
@ 2015-05-29 21:39 ` Stephan Mueller
2015-05-29 21:50 ` Stephan Mueller
2015-06-01 7:41 ` Herbert Xu
0 siblings, 2 replies; 25+ messages in thread
From: Stephan Mueller @ 2015-05-29 21:39 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
Am Donnerstag, 28. Mai 2015, 22:08:03 schrieb Herbert Xu:
Hi Herbert,
> This patch converts rfc4106-gcm-aesni to the new AEAD interface.
> The low-level interface remains as is for now because we can't
> touch it until cryptd itself is upgraded.
Not sure what goes wrong, but when using algif_aead with the new code, I get:
[ 18.895375] BUG: unable to handle kernel paging request at ffffea000402d1c0
[ 18.895380] IP: [<ffffffff812b7044>] scatterwalk_done+0x54/0x70
[ 18.895384] PGD 7f7da067 PUD 7f7d9067 PMD 0
[ 18.895387] Oops: 0000 [#1] SMP
[ 18.895388] Modules linked in: crypto_user algif_aead af_alg nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack cfg80211 ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_mangle iptable_security iptable_raw crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 glue_helper ablk_helper microcode joydev virtio_balloon pcspkr serio_raw i2c_piix4 acpi_cpufreq virtio_net qxl virtio_blk drm_kms_helper ttm drm virtio_pci virtio_ring virtio
[ 18.895411] CPU: 1 PID: 1617 Comm: kcapi Not tainted 4.0.0+ #228
[ 18.895413] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140709_153950- 04/01/2014
[ 18.895414] task: ffff88007c6cdd80 ti: ffff88007c378000 task.ti: ffff88007c378000
[ 18.895416] RIP: 0010:[<ffffffff812b7044>] [<ffffffff812b7044>] scatterwalk_done+0x54/0x70
[ 18.895418] RSP: 0018:ffff88007c37bc10 EFLAGS: 00010286
[ 18.895420] RAX: ffffea000402d1c0 RBX: ffff880000b4800c RCX: ffffea000002d200
[ 18.895421] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88007c37bc78
[ 18.895422] RBP: ffff88007c37bcb8 R08: 0000000000000000 R09: ffff88007b758208
[ 18.895423] R10: ffff880000b4801c R11: 000000000000000c R12: ffff88007b74b000
[ 18.895424] R13: ffff88007b75ad30 R14: ffff88007a4f9890 R15: 000000000000000c
[ 18.895425] FS: 00007fa99d637700(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
[ 18.895427] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 18.895428] CR2: ffffea000402d1c0 CR3: 000000007b9c3000 CR4: 00000000000407e0
[ 18.895433] Stack:
[ 18.895434] ffffffffa0111681 ffff88007b74b000 000000000000000c ffff880000b4801c
[ 18.895436] 000000000000000c 0000000000000000 ffff88007b74b00c 01ff88000000000c
[ 18.895438] ffff88007a4f98a0 72b5edc49473eb8a 0100000029402a3d ffff88007b758008
[ 18.895440] Call Trace:
[ 18.895446] [<ffffffffa0111681>] ? helper_rfc4106_encrypt+0x2a1/0x2b0 [aesni_intel]
[ 18.895450] [<ffffffffa0110145>] rfc4106_encrypt+0x95/0xb0 [aesni_intel]
[ 18.895452] [<ffffffffa029252b>] aead_recvmsg+0x41b/0x4a0 [algif_aead]
[ 18.895455] [<ffffffff81560bd8>] sock_recvmsg+0x38/0x50
[ 18.895458] [<ffffffff81560c78>] sock_read_iter+0x88/0xd0
[ 18.895460] [<ffffffff811a9990>] __vfs_read+0x90/0xc0
[ 18.895462] [<ffffffff811aa12a>] vfs_read+0x8a/0x140
[ 18.895464] [<ffffffff811aab56>] SyS_read+0x46/0xb0
[ 18.895467] [<ffffffff816877ee>] system_call_fastpath+0x12/0x71
[ 18.895468] Code: a9 ff 0f 00 00 74 08 85 d2 74 04 c3 0f 1f 00 85 f6 74 1b 49 8b 01 48 83 e0 fc 48 89 c1 41 8d 40 ff c1 e8 0c 48 c1 e0 06 48 01 c8 <48> 8b 00 85 d2 74 d9 55 48 89 e5 e8 ac fc ff ff 5d c3 66 2e 0f
[ 18.895489] RIP [<ffffffff812b7044>] scatterwalk_done+0x54/0x70
[ 18.895492] RSP <ffff88007c37bc10>
[ 18.895493] CR2: ffffea000402d1c0
[ 18.895494] ---[ end trace 78452ec206bb3972 ]-----
Ciao
Stephan
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface
2015-05-29 21:39 ` Stephan Mueller
@ 2015-05-29 21:50 ` Stephan Mueller
2015-06-01 7:41 ` Herbert Xu
1 sibling, 0 replies; 25+ messages in thread
From: Stephan Mueller @ 2015-05-29 21:50 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
Am Freitag, 29. Mai 2015, 23:39:03 schrieb Stephan Mueller:
Hi Herbert,
> Am Donnerstag, 28. Mai 2015, 22:08:03 schrieb Herbert Xu:
>
> Hi Herbert,
>
> > This patch converts rfc4106-gcm-aesni to the new AEAD interface.
> > The low-level interface remains as is for now because we can't
> > touch it until cryptd itself is upgraded.
>
> Not sure what goes wrong, but when using algif_aead with the new code, I
> get:
Maybe the following observation can help pinpoint the issue: using the src/dst
buffer allocated as follows, the kernel code works flawless:
outbuf = calloc(1, outbuflen);
Using aligned src/dst buffers created with the following code causes the bug:
if (posix_memalign((void *)&outbuf, PAGE_SIZE, outbuflen))
--
Ciao
Stephan
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface
2015-05-29 21:39 ` Stephan Mueller
2015-05-29 21:50 ` Stephan Mueller
@ 2015-06-01 7:41 ` Herbert Xu
1 sibling, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-06-01 7:41 UTC (permalink / raw)
To: Stephan Mueller
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
On Fri, May 29, 2015 at 11:39:03PM +0200, Stephan Mueller wrote:
>
> Not sure what goes wrong, but when using algif_aead with the new code, I get:
This is a bug in the way it's using scatterwalk. It needs to
advance the walk object by the number of bytes it has processed.
Previously it didn't crash because it didn't set out to 1 which
you should do if you're writing to the buffer.
I'll resend the patch with the bug fixed.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 9/10] crypto: cryptd - Convert to new AEAD interface
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (7 preceding siblings ...)
2015-05-28 14:08 ` [PATCH 8/10] crypto: aesni - Convert top-level rfc4106 algorithm to new interface Herbert Xu
@ 2015-05-28 14:08 ` Herbert Xu
2015-05-28 14:08 ` [PATCH 10/10] crypto: aesni - Convert rfc4106 " Herbert Xu
[not found] ` <E1YxyQL-0002Ha-OL@gondolin.me.apana.org.au>
10 siblings, 0 replies; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:08 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch converts cryptd to the new AEAD interface.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
crypto/cryptd.c | 64 +++++++++++++++++++++++++-------------------------------
1 file changed, 29 insertions(+), 35 deletions(-)
diff --git a/crypto/cryptd.c b/crypto/cryptd.c
index 74c4dcb..22ba81f 100644
--- a/crypto/cryptd.c
+++ b/crypto/cryptd.c
@@ -743,28 +743,26 @@ static int cryptd_aead_decrypt_enqueue(struct aead_request *req)
return cryptd_aead_enqueue(req, cryptd_aead_decrypt );
}
-static int cryptd_aead_init_tfm(struct crypto_tfm *tfm)
+static int cryptd_aead_init_tfm(struct crypto_aead *tfm)
{
- struct crypto_instance *inst = crypto_tfm_alg_instance(tfm);
- struct aead_instance_ctx *ictx = crypto_instance_ctx(inst);
+ struct aead_instance *inst = aead_alg_instance(tfm);
+ struct aead_instance_ctx *ictx = aead_instance_ctx(inst);
struct crypto_aead_spawn *spawn = &ictx->aead_spawn;
- struct cryptd_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct cryptd_aead_ctx *ctx = crypto_aead_ctx(tfm);
struct crypto_aead *cipher;
cipher = crypto_spawn_aead(spawn);
if (IS_ERR(cipher))
return PTR_ERR(cipher);
- crypto_aead_set_flags(cipher, CRYPTO_TFM_REQ_MAY_SLEEP);
ctx->child = cipher;
- crypto_aead_set_reqsize(__crypto_aead_cast(tfm),
- sizeof(struct cryptd_aead_request_ctx));
+ crypto_aead_set_reqsize(tfm, sizeof(struct cryptd_aead_request_ctx));
return 0;
}
-static void cryptd_aead_exit_tfm(struct crypto_tfm *tfm)
+static void cryptd_aead_exit_tfm(struct crypto_aead *tfm)
{
- struct cryptd_aead_ctx *ctx = crypto_tfm_ctx(tfm);
+ struct cryptd_aead_ctx *ctx = crypto_aead_ctx(tfm);
crypto_free_aead(ctx->child);
}
@@ -773,8 +771,8 @@ static int cryptd_create_aead(struct crypto_template *tmpl,
struct cryptd_queue *queue)
{
struct aead_instance_ctx *ctx;
- struct crypto_instance *inst;
- struct crypto_alg *alg;
+ struct aead_instance *inst;
+ struct aead_alg *alg;
const char *name;
u32 type = 0;
u32 mask = 0;
@@ -790,38 +788,34 @@ static int cryptd_create_aead(struct crypto_template *tmpl,
if (!inst)
return -ENOMEM;
- ctx = crypto_instance_ctx(inst);
+ ctx = aead_instance_ctx(inst);
ctx->queue = queue;
- crypto_set_aead_spawn(&ctx->aead_spawn, inst);
+ crypto_set_aead_spawn(&ctx->aead_spawn, aead_crypto_instance(inst));
err = crypto_grab_aead(&ctx->aead_spawn, name, type, mask);
if (err)
goto out_free_inst;
- alg = crypto_aead_spawn_alg(&ctx->aead_spawn);
- err = cryptd_init_instance(inst, alg);
+ alg = crypto_spawn_aead_alg(&ctx->aead_spawn);
+ err = cryptd_init_instance(aead_crypto_instance(inst), &alg->base);
if (err)
goto out_drop_aead;
- type = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_ASYNC;
- if (alg->cra_flags & CRYPTO_ALG_INTERNAL)
- type |= CRYPTO_ALG_INTERNAL;
- inst->alg.cra_flags = type;
- inst->alg.cra_type = alg->cra_type;
- inst->alg.cra_ctxsize = sizeof(struct cryptd_aead_ctx);
- inst->alg.cra_init = cryptd_aead_init_tfm;
- inst->alg.cra_exit = cryptd_aead_exit_tfm;
- inst->alg.cra_aead.setkey = cryptd_aead_setkey;
- inst->alg.cra_aead.setauthsize = cryptd_aead_setauthsize;
- inst->alg.cra_aead.geniv = alg->cra_aead.geniv;
- inst->alg.cra_aead.ivsize = alg->cra_aead.ivsize;
- inst->alg.cra_aead.maxauthsize = alg->cra_aead.maxauthsize;
- inst->alg.cra_aead.encrypt = cryptd_aead_encrypt_enqueue;
- inst->alg.cra_aead.decrypt = cryptd_aead_decrypt_enqueue;
- inst->alg.cra_aead.givencrypt = alg->cra_aead.givencrypt;
- inst->alg.cra_aead.givdecrypt = alg->cra_aead.givdecrypt;
+ inst->alg.base.cra_flags = CRYPTO_ALG_ASYNC |
+ (alg->base.cra_flags & CRYPTO_ALG_INTERNAL);
+ inst->alg.base.cra_ctxsize = sizeof(struct cryptd_aead_ctx);
- err = crypto_register_instance(tmpl, inst);
+ inst->alg.ivsize = crypto_aead_alg_ivsize(alg);
+ inst->alg.maxauthsize = crypto_aead_alg_maxauthsize(alg);
+
+ inst->alg.init = cryptd_aead_init_tfm;
+ inst->alg.exit = cryptd_aead_exit_tfm;
+ inst->alg.setkey = cryptd_aead_setkey;
+ inst->alg.setauthsize = cryptd_aead_setauthsize;
+ inst->alg.encrypt = cryptd_aead_encrypt_enqueue;
+ inst->alg.decrypt = cryptd_aead_decrypt_enqueue;
+
+ err = aead_register_instance(tmpl, inst);
if (err) {
out_drop_aead:
crypto_drop_aead(&ctx->aead_spawn);
@@ -865,8 +859,8 @@ static void cryptd_free(struct crypto_instance *inst)
kfree(ahash_instance(inst));
return;
case CRYPTO_ALG_TYPE_AEAD:
- crypto_drop_spawn(&aead_ctx->aead_spawn.base);
- kfree(inst);
+ crypto_drop_aead(&aead_ctx->aead_spawn);
+ kfree(aead_instance(inst));
return;
default:
crypto_drop_spawn(&ctx->spawn);
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH 10/10] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-05-28 14:03 ` [PATCH 0/10] crypto: Convert pcrypt/cryptd/aesni to new AEAD interface Herbert Xu
` (8 preceding siblings ...)
2015-05-28 14:08 ` [PATCH 9/10] crypto: cryptd - Convert to new AEAD interface Herbert Xu
@ 2015-05-28 14:08 ` Herbert Xu
2015-05-30 11:17 ` Stephan Mueller
[not found] ` <E1YxyQL-0002Ha-OL@gondolin.me.apana.org.au>
10 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-05-28 14:08 UTC (permalink / raw)
To: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey, Tim Chen
This patch converts the low-level __gcm-aes-aesni algorithm to
the new AEAD interface.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
arch/x86/crypto/aesni-intel_glue.c | 246 +++++++++++--------------------------
1 file changed, 79 insertions(+), 167 deletions(-)
diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index 5660a18..c5fa1e6 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -44,13 +44,18 @@
#endif
+#define AESNI_ALIGN 16
+#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE - 1))
+#define RFC4106_HASH_SUBKEY_SIZE 16
+
/* This data is stored at the end of the crypto_tfm struct.
* It's a type of per "session" data storage location.
* This needs to be 16 byte aligned.
*/
struct aesni_rfc4106_gcm_ctx {
- u8 hash_subkey[16];
- struct crypto_aes_ctx aes_key_expanded;
+ u8 hash_subkey[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
+ struct crypto_aes_ctx aes_key_expanded
+ __attribute__ ((__aligned__(AESNI_ALIGN)));
u8 nonce[4];
};
@@ -65,10 +70,6 @@ struct aesni_hash_subkey_req_data {
struct scatterlist sg;
};
-#define AESNI_ALIGN (16)
-#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE-1))
-#define RFC4106_HASH_SUBKEY_SIZE 16
-
struct aesni_lrw_ctx {
struct lrw_table_ctx lrw_table;
u8 raw_aes_ctx[sizeof(struct crypto_aes_ctx) + AESNI_ALIGN - 1];
@@ -282,10 +283,11 @@ static void (*aesni_gcm_dec_tfm)(void *ctx, u8 *out,
static inline struct
aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get(struct crypto_aead *tfm)
{
- return
- (struct aesni_rfc4106_gcm_ctx *)
- PTR_ALIGN((u8 *)
- crypto_tfm_ctx(crypto_aead_tfm(tfm)), AESNI_ALIGN);
+ unsigned long align = AESNI_ALIGN;
+
+ if (align <= crypto_tfm_ctx_alignment())
+ align = 1;
+ return PTR_ALIGN(crypto_aead_ctx(tfm), align);
}
#endif
@@ -838,8 +840,6 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
if (IS_ERR(ctr_tfm))
return PTR_ERR(ctr_tfm);
- crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
-
ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
if (ret)
goto out_free_ablkcipher;
@@ -888,56 +888,20 @@ out_free_ablkcipher:
static int common_rfc4106_set_key(struct crypto_aead *aead, const u8 *key,
unsigned int key_len)
{
- int ret = 0;
- struct crypto_tfm *tfm = crypto_aead_tfm(aead);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(aead);
- u8 *new_key_align, *new_key_mem = NULL;
if (key_len < 4) {
- crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ crypto_aead_set_flags(aead, CRYPTO_TFM_RES_BAD_KEY_LEN);
return -EINVAL;
}
/*Account for 4 byte nonce at the end.*/
key_len -= 4;
- if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256) {
- crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
- return -EINVAL;
- }
memcpy(ctx->nonce, key + key_len, sizeof(ctx->nonce));
- /*This must be on a 16 byte boundary!*/
- if ((unsigned long)(&(ctx->aes_key_expanded.key_enc[0])) % AESNI_ALIGN)
- return -EINVAL;
-
- if ((unsigned long)key % AESNI_ALIGN) {
- /*key is not aligned: use an auxuliar aligned pointer*/
- new_key_mem = kmalloc(key_len+AESNI_ALIGN, GFP_KERNEL);
- if (!new_key_mem)
- return -ENOMEM;
-
- new_key_align = PTR_ALIGN(new_key_mem, AESNI_ALIGN);
- memcpy(new_key_align, key, key_len);
- key = new_key_align;
- }
- if (!irq_fpu_usable())
- ret = crypto_aes_expand_key(&(ctx->aes_key_expanded),
- key, key_len);
- else {
- kernel_fpu_begin();
- ret = aesni_set_key(&(ctx->aes_key_expanded), key, key_len);
- kernel_fpu_end();
- }
- /*This must be on a 16 byte boundary!*/
- if ((unsigned long)(&(ctx->hash_subkey[0])) % AESNI_ALIGN) {
- ret = -EINVAL;
- goto exit;
- }
- ret = rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
-exit:
- kfree(new_key_mem);
- return ret;
+ return aes_set_key_common(crypto_aead_tfm(aead),
+ &ctx->aes_key_expanded, key, key_len) ?:
+ rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
}
static int rfc4106_set_key(struct crypto_aead *parent, const u8 *key,
@@ -960,7 +924,7 @@ static int common_rfc4106_set_authsize(struct crypto_aead *aead,
default:
return -EINVAL;
}
- crypto_aead_crt(aead)->authsize = authsize;
+
return 0;
}
@@ -975,20 +939,17 @@ static int rfc4106_set_authsize(struct crypto_aead *parent,
return crypto_aead_setauthsize(&cryptd_tfm->base, authsize);
}
-static int __driver_rfc4106_encrypt(struct aead_request *req)
+static int helper_rfc4106_encrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
__be32 counter = cpu_to_be32(1);
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
- u32 key_len = ctx->aes_key_expanded.key_length;
void *aes_ctx = &(ctx->aes_key_expanded);
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
- u8 iv_tab[16+AESNI_ALIGN];
- u8* iv = (u8 *) PTR_ALIGN((u8 *)iv_tab, AESNI_ALIGN);
+ u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
struct scatter_walk src_sg_walk;
- struct scatter_walk assoc_sg_walk;
struct scatter_walk dst_sg_walk;
unsigned int i;
@@ -997,12 +958,6 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
/* to 8 or 12 bytes */
if (unlikely(req->assoclen != 8 && req->assoclen != 12))
return -EINVAL;
- if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len != 16))
- return -EINVAL;
- if (unlikely(key_len != AES_KEYSIZE_128 &&
- key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256))
- return -EINVAL;
/* IV below built */
for (i = 0; i < 4; i++)
@@ -1011,55 +966,55 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
*(iv+4+i) = req->iv[i];
*((__be32 *)(iv+12)) = counter;
- if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
+ if (sg_is_last(req->src) &&
+ req->src->offset + req->src->length <= PAGE_SIZE &&
+ sg_is_last(req->dst) &&
+ req->dst->offset + req->dst->length <= PAGE_SIZE) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
- scatterwalk_start(&assoc_sg_walk, req->assoc);
- src = scatterwalk_map(&src_sg_walk);
- assoc = scatterwalk_map(&assoc_sg_walk);
+ assoc = scatterwalk_map(&src_sg_walk);
+ src = assoc + req->assoclen;
dst = src;
if (unlikely(req->src != req->dst)) {
scatterwalk_start(&dst_sg_walk, req->dst);
- dst = scatterwalk_map(&dst_sg_walk);
+ dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
}
-
} else {
/* Allocate memory for src, dst, assoc */
- src = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
+ assoc = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
GFP_ATOMIC);
- if (unlikely(!src))
+ if (unlikely(!assoc))
return -ENOMEM;
- assoc = (src + req->cryptlen + auth_tag_len);
- scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
- scatterwalk_map_and_copy(assoc, req->assoc, 0,
- req->assoclen, 0);
+ scatterwalk_map_and_copy(assoc, req->src, 0,
+ req->assoclen + req->cryptlen, 0);
+ src = assoc + req->assoclen;
dst = src;
}
+ kernel_fpu_begin();
aesni_gcm_enc_tfm(aes_ctx, dst, src, (unsigned long)req->cryptlen, iv,
ctx->hash_subkey, assoc, (unsigned long)req->assoclen, dst
+ ((unsigned long)req->cryptlen), auth_tag_len);
+ kernel_fpu_end();
/* The authTag (aka the Integrity Check Value) needs to be written
* back to the packet. */
if (one_entry_in_sg) {
if (unlikely(req->src != req->dst)) {
- scatterwalk_unmap(dst);
- scatterwalk_done(&dst_sg_walk, 0, 0);
+ scatterwalk_unmap(dst - req->assoclen);
+ scatterwalk_done(&dst_sg_walk, 1, 0);
}
- scatterwalk_unmap(src);
scatterwalk_unmap(assoc);
- scatterwalk_done(&src_sg_walk, 0, 0);
- scatterwalk_done(&assoc_sg_walk, 0, 0);
+ scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0,
- req->cryptlen + auth_tag_len, 1);
- kfree(src);
+ scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
+ req->cryptlen + auth_tag_len, 1);
+ kfree(assoc);
}
return 0;
}
-static int __driver_rfc4106_decrypt(struct aead_request *req)
+static int helper_rfc4106_decrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
@@ -1068,26 +1023,16 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
int retval = 0;
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
- u32 key_len = ctx->aes_key_expanded.key_length;
void *aes_ctx = &(ctx->aes_key_expanded);
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
- u8 iv_and_authTag[32+AESNI_ALIGN];
- u8 *iv = (u8 *) PTR_ALIGN((u8 *)iv_and_authTag, AESNI_ALIGN);
- u8 *authTag = iv + 16;
+ u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
+ u8 authTag[16];
struct scatter_walk src_sg_walk;
- struct scatter_walk assoc_sg_walk;
struct scatter_walk dst_sg_walk;
unsigned int i;
- if (unlikely((req->cryptlen < auth_tag_len) ||
- (req->assoclen != 8 && req->assoclen != 12)))
+ if (unlikely(req->assoclen != 8 && req->assoclen != 12))
return -EINVAL;
- if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len != 16))
- return -EINVAL;
- if (unlikely(key_len != AES_KEYSIZE_128 &&
- key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256))
- return -EINVAL;
/* Assuming we are supporting rfc4106 64-bit extended */
/* sequence numbers We need to have the AAD length */
@@ -1101,33 +1046,36 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
*(iv+4+i) = req->iv[i];
*((__be32 *)(iv+12)) = counter;
- if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
+ if (sg_is_last(req->src) &&
+ req->src->offset + req->src->length <= PAGE_SIZE &&
+ sg_is_last(req->dst) &&
+ req->dst->offset + req->dst->length <= PAGE_SIZE) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
- scatterwalk_start(&assoc_sg_walk, req->assoc);
- src = scatterwalk_map(&src_sg_walk);
- assoc = scatterwalk_map(&assoc_sg_walk);
+ assoc = scatterwalk_map(&src_sg_walk);
+ src = assoc + req->assoclen;
dst = src;
if (unlikely(req->src != req->dst)) {
scatterwalk_start(&dst_sg_walk, req->dst);
- dst = scatterwalk_map(&dst_sg_walk);
+ dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
}
} else {
/* Allocate memory for src, dst, assoc */
- src = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
- if (!src)
+ assoc = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
+ if (!assoc)
return -ENOMEM;
- assoc = (src + req->cryptlen);
- scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
- scatterwalk_map_and_copy(assoc, req->assoc, 0,
- req->assoclen, 0);
+ scatterwalk_map_and_copy(assoc, req->src, 0,
+ req->assoclen + req->cryptlen, 0);
+ src = assoc + req->assoclen;
dst = src;
}
+ kernel_fpu_begin();
aesni_gcm_dec_tfm(aes_ctx, dst, src, tempCipherLen, iv,
ctx->hash_subkey, assoc, (unsigned long)req->assoclen,
authTag, auth_tag_len);
+ kernel_fpu_end();
/* Compare generated tag with passed in tag. */
retval = crypto_memneq(src + tempCipherLen, authTag, auth_tag_len) ?
@@ -1135,16 +1083,15 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
if (one_entry_in_sg) {
if (unlikely(req->src != req->dst)) {
- scatterwalk_unmap(dst);
- scatterwalk_done(&dst_sg_walk, 0, 0);
+ scatterwalk_unmap(dst - req->assoclen);
+ scatterwalk_done(&dst_sg_walk, 1, 0);
}
- scatterwalk_unmap(src);
scatterwalk_unmap(assoc);
- scatterwalk_done(&src_sg_walk, 0, 0);
- scatterwalk_done(&assoc_sg_walk, 0, 0);
+ scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0, tempCipherLen, 1);
- kfree(src);
+ scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
+ tempCipherLen, 1);
+ kfree(assoc);
}
return retval;
}
@@ -1188,36 +1135,6 @@ static int rfc4106_decrypt(struct aead_request *req)
return crypto_aead_decrypt(subreq);
}
-
-static int helper_rfc4106_encrypt(struct aead_request *req)
-{
- int ret;
-
- if (unlikely(!irq_fpu_usable())) {
- WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
- ret = -EINVAL;
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_encrypt(req);
- kernel_fpu_end();
- }
- return ret;
-}
-
-static int helper_rfc4106_decrypt(struct aead_request *req)
-{
- int ret;
-
- if (unlikely(!irq_fpu_usable())) {
- WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
- ret = -EINVAL;
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_decrypt(req);
- kernel_fpu_end();
- }
- return ret;
-}
#endif
static struct crypto_alg aesni_algs[] = { {
@@ -1389,27 +1306,6 @@ static struct crypto_alg aesni_algs[] = { {
.geniv = "chainiv",
},
},
-}, {
- .cra_name = "__gcm-aes-aesni",
- .cra_driver_name = "__driver-gcm-aes-aesni",
- .cra_priority = 0,
- .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_INTERNAL,
- .cra_blocksize = 1,
- .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx) +
- AESNI_ALIGN,
- .cra_alignmask = 0,
- .cra_type = &crypto_aead_type,
- .cra_module = THIS_MODULE,
- .cra_u = {
- .aead = {
- .setkey = common_rfc4106_set_key,
- .setauthsize = common_rfc4106_set_authsize,
- .encrypt = helper_rfc4106_encrypt,
- .decrypt = helper_rfc4106_decrypt,
- .ivsize = 8,
- .maxauthsize = 16,
- },
- },
#endif
#if IS_ENABLED(CONFIG_CRYPTO_PCBC)
}, {
@@ -1526,6 +1422,22 @@ static struct crypto_alg aesni_algs[] = { {
#ifdef CONFIG_X86_64
static struct aead_alg aesni_aead_algs[] = { {
+ .setkey = common_rfc4106_set_key,
+ .setauthsize = common_rfc4106_set_authsize,
+ .encrypt = helper_rfc4106_encrypt,
+ .decrypt = helper_rfc4106_decrypt,
+ .ivsize = 8,
+ .maxauthsize = 16,
+ .base = {
+ .cra_name = "__gcm-aes-aesni",
+ .cra_driver_name = "__driver-gcm-aes-aesni",
+ .cra_flags = CRYPTO_ALG_INTERNAL,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx),
+ .cra_alignmask = AESNI_ALIGN - 1,
+ .cra_module = THIS_MODULE,
+ },
+}, {
.init = rfc4106_init,
.exit = rfc4106_exit,
.setkey = rfc4106_set_key,
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH 10/10] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-05-28 14:08 ` [PATCH 10/10] crypto: aesni - Convert rfc4106 " Herbert Xu
@ 2015-05-30 11:17 ` Stephan Mueller
2015-06-01 7:50 ` [v2 PATCH] " Herbert Xu
0 siblings, 1 reply; 25+ messages in thread
From: Stephan Mueller @ 2015-05-30 11:17 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
Am Donnerstag, 28. Mai 2015, 22:08:06 schrieb Herbert Xu:
Hi Herbert,
> This patch converts the low-level __gcm-aes-aesni algorithm to
> the new AEAD interface.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
> ---
>
> arch/x86/crypto/aesni-intel_glue.c | 246
> +++++++++++-------------------------- 1 file changed, 79 insertions(+), 167
> deletions(-)
>
> diff --git a/arch/x86/crypto/aesni-intel_glue.c
> b/arch/x86/crypto/aesni-intel_glue.c index 5660a18..c5fa1e6 100644
> --- a/arch/x86/crypto/aesni-intel_glue.c
> +++ b/arch/x86/crypto/aesni-intel_glue.c
> @@ -44,13 +44,18 @@
> #endif
>
>
> +#define AESNI_ALIGN 16
> +#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE - 1))
> +#define RFC4106_HASH_SUBKEY_SIZE 16
> +
> /* This data is stored at the end of the crypto_tfm struct.
> * It's a type of per "session" data storage location.
> * This needs to be 16 byte aligned.
> */
> struct aesni_rfc4106_gcm_ctx {
> - u8 hash_subkey[16];
> - struct crypto_aes_ctx aes_key_expanded;
> + u8 hash_subkey[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
> + struct crypto_aes_ctx aes_key_expanded
> + __attribute__ ((__aligned__(AESNI_ALIGN)));
> u8 nonce[4];
> };
>
> @@ -65,10 +70,6 @@ struct aesni_hash_subkey_req_data {
> struct scatterlist sg;
> };
>
> -#define AESNI_ALIGN (16)
> -#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE-1))
> -#define RFC4106_HASH_SUBKEY_SIZE 16
> -
> struct aesni_lrw_ctx {
> struct lrw_table_ctx lrw_table;
> u8 raw_aes_ctx[sizeof(struct crypto_aes_ctx) + AESNI_ALIGN - 1];
> @@ -282,10 +283,11 @@ static void (*aesni_gcm_dec_tfm)(void *ctx, u8 *out,
> static inline struct
> aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get(struct crypto_aead *tfm)
> {
> - return
> - (struct aesni_rfc4106_gcm_ctx *)
> - PTR_ALIGN((u8 *)
> - crypto_tfm_ctx(crypto_aead_tfm(tfm)), AESNI_ALIGN);
> + unsigned long align = AESNI_ALIGN;
> +
> + if (align <= crypto_tfm_ctx_alignment())
> + align = 1;
> + return PTR_ALIGN(crypto_aead_ctx(tfm), align);
> }
> #endif
>
> @@ -838,8 +840,6 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key,
> unsigned int key_len) if (IS_ERR(ctr_tfm))
> return PTR_ERR(ctr_tfm);
>
> - crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
> -
> ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
> if (ret)
> goto out_free_ablkcipher;
> @@ -888,56 +888,20 @@ out_free_ablkcipher:
> static int common_rfc4106_set_key(struct crypto_aead *aead, const u8 *key,
> unsigned int key_len)
> {
> - int ret = 0;
> - struct crypto_tfm *tfm = crypto_aead_tfm(aead);
> struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(aead);
> - u8 *new_key_align, *new_key_mem = NULL;
>
> if (key_len < 4) {
> - crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
> + crypto_aead_set_flags(aead, CRYPTO_TFM_RES_BAD_KEY_LEN);
> return -EINVAL;
> }
> /*Account for 4 byte nonce at the end.*/
> key_len -= 4;
> - if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 &&
> - key_len != AES_KEYSIZE_256) {
> - crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
> - return -EINVAL;
> - }
>
> memcpy(ctx->nonce, key + key_len, sizeof(ctx->nonce));
> - /*This must be on a 16 byte boundary!*/
> - if ((unsigned long)(&(ctx->aes_key_expanded.key_enc[0])) %
AESNI_ALIGN)
> - return -EINVAL;
> -
> - if ((unsigned long)key % AESNI_ALIGN) {
> - /*key is not aligned: use an auxuliar aligned pointer*/
> - new_key_mem = kmalloc(key_len+AESNI_ALIGN, GFP_KERNEL);
> - if (!new_key_mem)
> - return -ENOMEM;
> -
> - new_key_align = PTR_ALIGN(new_key_mem, AESNI_ALIGN);
> - memcpy(new_key_align, key, key_len);
> - key = new_key_align;
> - }
>
> - if (!irq_fpu_usable())
> - ret = crypto_aes_expand_key(&(ctx->aes_key_expanded),
> - key, key_len);
> - else {
> - kernel_fpu_begin();
> - ret = aesni_set_key(&(ctx->aes_key_expanded), key, key_len);
> - kernel_fpu_end();
> - }
> - /*This must be on a 16 byte boundary!*/
> - if ((unsigned long)(&(ctx->hash_subkey[0])) % AESNI_ALIGN) {
> - ret = -EINVAL;
> - goto exit;
> - }
> - ret = rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
> -exit:
> - kfree(new_key_mem);
> - return ret;
> + return aes_set_key_common(crypto_aead_tfm(aead),
> + &ctx->aes_key_expanded, key, key_len) ?:
> + rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
> }
>
> static int rfc4106_set_key(struct crypto_aead *parent, const u8 *key,
> @@ -960,7 +924,7 @@ static int common_rfc4106_set_authsize(struct
> crypto_aead *aead, default:
> return -EINVAL;
> }
> - crypto_aead_crt(aead)->authsize = authsize;
> +
> return 0;
> }
>
> @@ -975,20 +939,17 @@ static int rfc4106_set_authsize(struct crypto_aead
> *parent, return crypto_aead_setauthsize(&cryptd_tfm->base, authsize);
> }
>
> -static int __driver_rfc4106_encrypt(struct aead_request *req)
> +static int helper_rfc4106_encrypt(struct aead_request *req)
> {
> u8 one_entry_in_sg = 0;
> u8 *src, *dst, *assoc;
> __be32 counter = cpu_to_be32(1);
> struct crypto_aead *tfm = crypto_aead_reqtfm(req);
> struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
> - u32 key_len = ctx->aes_key_expanded.key_length;
> void *aes_ctx = &(ctx->aes_key_expanded);
> unsigned long auth_tag_len = crypto_aead_authsize(tfm);
> - u8 iv_tab[16+AESNI_ALIGN];
> - u8* iv = (u8 *) PTR_ALIGN((u8 *)iv_tab, AESNI_ALIGN);
> + u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
> struct scatter_walk src_sg_walk;
> - struct scatter_walk assoc_sg_walk;
> struct scatter_walk dst_sg_walk;
> unsigned int i;
>
> @@ -997,12 +958,6 @@ static int __driver_rfc4106_encrypt(struct aead_request
> *req) /* to 8 or 12 bytes */
> if (unlikely(req->assoclen != 8 && req->assoclen != 12))
> return -EINVAL;
> - if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len
!=
> 16)) - return -EINVAL;
> - if (unlikely(key_len != AES_KEYSIZE_128 &&
> - key_len != AES_KEYSIZE_192 &&
> - key_len != AES_KEYSIZE_256))
> - return -EINVAL;
>
> /* IV below built */
> for (i = 0; i < 4; i++)
> @@ -1011,55 +966,55 @@ static int __driver_rfc4106_encrypt(struct
> aead_request *req) *(iv+4+i) = req->iv[i];
> *((__be32 *)(iv+12)) = counter;
>
> - if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
> + if (sg_is_last(req->src) &&
> + req->src->offset + req->src->length <= PAGE_SIZE &&
> + sg_is_last(req->dst) &&
> + req->dst->offset + req->dst->length <= PAGE_SIZE) {
> one_entry_in_sg = 1;
> scatterwalk_start(&src_sg_walk, req->src);
> - scatterwalk_start(&assoc_sg_walk, req->assoc);
> - src = scatterwalk_map(&src_sg_walk);
> - assoc = scatterwalk_map(&assoc_sg_walk);
> + assoc = scatterwalk_map(&src_sg_walk);
> + src = assoc + req->assoclen;
> dst = src;
> if (unlikely(req->src != req->dst)) {
> scatterwalk_start(&dst_sg_walk, req->dst);
> - dst = scatterwalk_map(&dst_sg_walk);
> + dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
> }
> -
> } else {
> /* Allocate memory for src, dst, assoc */
> - src = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
> + assoc = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
> GFP_ATOMIC);
> - if (unlikely(!src))
> + if (unlikely(!assoc))
> return -ENOMEM;
> - assoc = (src + req->cryptlen + auth_tag_len);
> - scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
> - scatterwalk_map_and_copy(assoc, req->assoc, 0,
> - req->assoclen, 0);
> + scatterwalk_map_and_copy(assoc, req->src, 0,
> + req->assoclen + req->cryptlen, 0);
> + src = assoc + req->assoclen;
> dst = src;
> }
>
> + kernel_fpu_begin();
> aesni_gcm_enc_tfm(aes_ctx, dst, src, (unsigned long)req->cryptlen, iv,
> ctx->hash_subkey, assoc, (unsigned long)req->assoclen, dst
> + ((unsigned long)req->cryptlen), auth_tag_len);
> + kernel_fpu_end();
>
> /* The authTag (aka the Integrity Check Value) needs to be written
> * back to the packet. */
> if (one_entry_in_sg) {
> if (unlikely(req->src != req->dst)) {
> - scatterwalk_unmap(dst);
> - scatterwalk_done(&dst_sg_walk, 0, 0);
> + scatterwalk_unmap(dst - req->assoclen);
> + scatterwalk_done(&dst_sg_walk, 1, 0);
> }
> - scatterwalk_unmap(src);
> scatterwalk_unmap(assoc);
> - scatterwalk_done(&src_sg_walk, 0, 0);
> - scatterwalk_done(&assoc_sg_walk, 0, 0);
> + scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
> } else {
> - scatterwalk_map_and_copy(dst, req->dst, 0,
> - req->cryptlen + auth_tag_len, 1);
> - kfree(src);
> + scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
> + req->cryptlen + auth_tag_len, 1);
> + kfree(assoc);
> }
> return 0;
> }
>
> -static int __driver_rfc4106_decrypt(struct aead_request *req)
> +static int helper_rfc4106_decrypt(struct aead_request *req)
> {
> u8 one_entry_in_sg = 0;
> u8 *src, *dst, *assoc;
> @@ -1068,26 +1023,16 @@ static int __driver_rfc4106_decrypt(struct
> aead_request *req) int retval = 0;
> struct crypto_aead *tfm = crypto_aead_reqtfm(req);
> struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
> - u32 key_len = ctx->aes_key_expanded.key_length;
> void *aes_ctx = &(ctx->aes_key_expanded);
> unsigned long auth_tag_len = crypto_aead_authsize(tfm);
> - u8 iv_and_authTag[32+AESNI_ALIGN];
> - u8 *iv = (u8 *) PTR_ALIGN((u8 *)iv_and_authTag, AESNI_ALIGN);
> - u8 *authTag = iv + 16;
> + u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
> + u8 authTag[16];
> struct scatter_walk src_sg_walk;
> - struct scatter_walk assoc_sg_walk;
> struct scatter_walk dst_sg_walk;
> unsigned int i;
>
> - if (unlikely((req->cryptlen < auth_tag_len) ||
> - (req->assoclen != 8 && req->assoclen != 12)))
> + if (unlikely(req->assoclen != 8 && req->assoclen != 12))
> return -EINVAL;
> - if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len
!=
> 16)) - return -EINVAL;
> - if (unlikely(key_len != AES_KEYSIZE_128 &&
> - key_len != AES_KEYSIZE_192 &&
> - key_len != AES_KEYSIZE_256))
> - return -EINVAL;
>
> /* Assuming we are supporting rfc4106 64-bit extended */
> /* sequence numbers We need to have the AAD length */
> @@ -1101,33 +1046,36 @@ static int __driver_rfc4106_decrypt(struct
> aead_request *req) *(iv+4+i) = req->iv[i];
> *((__be32 *)(iv+12)) = counter;
>
> - if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
> + if (sg_is_last(req->src) &&
> + req->src->offset + req->src->length <= PAGE_SIZE &&
> + sg_is_last(req->dst) &&
> + req->dst->offset + req->dst->length <= PAGE_SIZE) {
> one_entry_in_sg = 1;
> scatterwalk_start(&src_sg_walk, req->src);
> - scatterwalk_start(&assoc_sg_walk, req->assoc);
> - src = scatterwalk_map(&src_sg_walk);
> - assoc = scatterwalk_map(&assoc_sg_walk);
> + assoc = scatterwalk_map(&src_sg_walk);
> + src = assoc + req->assoclen;
> dst = src;
> if (unlikely(req->src != req->dst)) {
> scatterwalk_start(&dst_sg_walk, req->dst);
> - dst = scatterwalk_map(&dst_sg_walk);
> + dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
> }
>
> } else {
> /* Allocate memory for src, dst, assoc */
> - src = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
> - if (!src)
> + assoc = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
> + if (!assoc)
> return -ENOMEM;
> - assoc = (src + req->cryptlen);
> - scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
> - scatterwalk_map_and_copy(assoc, req->assoc, 0,
> - req->assoclen, 0);
> + scatterwalk_map_and_copy(assoc, req->src, 0,
> + req->assoclen + req->cryptlen, 0);
> + src = assoc + req->assoclen;
> dst = src;
> }
>
> + kernel_fpu_begin();
> aesni_gcm_dec_tfm(aes_ctx, dst, src, tempCipherLen, iv,
> ctx->hash_subkey, assoc, (unsigned long)req->assoclen,
> authTag, auth_tag_len);
> + kernel_fpu_end();
>
> /* Compare generated tag with passed in tag. */
> retval = crypto_memneq(src + tempCipherLen, authTag, auth_tag_len) ?
> @@ -1135,16 +1083,15 @@ static int __driver_rfc4106_decrypt(struct
> aead_request *req)
>
> if (one_entry_in_sg) {
> if (unlikely(req->src != req->dst)) {
> - scatterwalk_unmap(dst);
> - scatterwalk_done(&dst_sg_walk, 0, 0);
> + scatterwalk_unmap(dst - req->assoclen);
> + scatterwalk_done(&dst_sg_walk, 1, 0);
The bug mentioned for patch 8 happens exactly at this spot.
> }
> - scatterwalk_unmap(src);
> scatterwalk_unmap(assoc);
> - scatterwalk_done(&src_sg_walk, 0, 0);
> - scatterwalk_done(&assoc_sg_walk, 0, 0);
> + scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
> } else {
> - scatterwalk_map_and_copy(dst, req->dst, 0, tempCipherLen, 1);
> - kfree(src);
> + scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
> + tempCipherLen, 1);
> + kfree(assoc);
> }
> return retval;
> }
> @@ -1188,36 +1135,6 @@ static int rfc4106_decrypt(struct aead_request *req)
>
> return crypto_aead_decrypt(subreq);
> }
> -
> -static int helper_rfc4106_encrypt(struct aead_request *req)
> -{
> - int ret;
> -
> - if (unlikely(!irq_fpu_usable())) {
> - WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
> - ret = -EINVAL;
> - } else {
> - kernel_fpu_begin();
> - ret = __driver_rfc4106_encrypt(req);
> - kernel_fpu_end();
> - }
> - return ret;
> -}
> -
> -static int helper_rfc4106_decrypt(struct aead_request *req)
> -{
> - int ret;
> -
> - if (unlikely(!irq_fpu_usable())) {
> - WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
> - ret = -EINVAL;
> - } else {
> - kernel_fpu_begin();
> - ret = __driver_rfc4106_decrypt(req);
> - kernel_fpu_end();
> - }
> - return ret;
> -}
> #endif
>
> static struct crypto_alg aesni_algs[] = { {
> @@ -1389,27 +1306,6 @@ static struct crypto_alg aesni_algs[] = { {
> .geniv = "chainiv",
> },
> },
> -}, {
> - .cra_name = "__gcm-aes-aesni",
> - .cra_driver_name = "__driver-gcm-aes-aesni",
> - .cra_priority = 0,
> - .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_INTERNAL,
> - .cra_blocksize = 1,
> - .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx) +
> - AESNI_ALIGN,
> - .cra_alignmask = 0,
> - .cra_type = &crypto_aead_type,
> - .cra_module = THIS_MODULE,
> - .cra_u = {
> - .aead = {
> - .setkey = common_rfc4106_set_key,
> - .setauthsize = common_rfc4106_set_authsize,
> - .encrypt = helper_rfc4106_encrypt,
> - .decrypt = helper_rfc4106_decrypt,
> - .ivsize = 8,
> - .maxauthsize = 16,
> - },
> - },
> #endif
> #if IS_ENABLED(CONFIG_CRYPTO_PCBC)
> }, {
> @@ -1526,6 +1422,22 @@ static struct crypto_alg aesni_algs[] = { {
>
> #ifdef CONFIG_X86_64
> static struct aead_alg aesni_aead_algs[] = { {
> + .setkey = common_rfc4106_set_key,
> + .setauthsize = common_rfc4106_set_authsize,
> + .encrypt = helper_rfc4106_encrypt,
> + .decrypt = helper_rfc4106_decrypt,
> + .ivsize = 8,
> + .maxauthsize = 16,
> + .base = {
> + .cra_name = "__gcm-aes-aesni",
> + .cra_driver_name = "__driver-gcm-aes-aesni",
> + .cra_flags = CRYPTO_ALG_INTERNAL,
> + .cra_blocksize = 1,
> + .cra_ctxsize = sizeof(struct
aesni_rfc4106_gcm_ctx),
> + .cra_alignmask = AESNI_ALIGN - 1,
> + .cra_module = THIS_MODULE,
> + },
> +}, {
> .init = rfc4106_init,
> .exit = rfc4106_exit,
> .setkey = rfc4106_set_key,
> --
> To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Ciao
Stephan
^ permalink raw reply [flat|nested] 25+ messages in thread* [v2 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-05-30 11:17 ` Stephan Mueller
@ 2015-06-01 7:50 ` Herbert Xu
2015-06-01 7:53 ` [v3 " Herbert Xu
0 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-06-01 7:50 UTC (permalink / raw)
To: Stephan Mueller
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
This patch converts the low-level __gcm-aes-aesni algorithm to
the new AEAD interface.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index 5660a18..8ab9630 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -44,13 +44,18 @@
#endif
+#define AESNI_ALIGN 16
+#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE - 1))
+#define RFC4106_HASH_SUBKEY_SIZE 16
+
/* This data is stored at the end of the crypto_tfm struct.
* It's a type of per "session" data storage location.
* This needs to be 16 byte aligned.
*/
struct aesni_rfc4106_gcm_ctx {
- u8 hash_subkey[16];
- struct crypto_aes_ctx aes_key_expanded;
+ u8 hash_subkey[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
+ struct crypto_aes_ctx aes_key_expanded
+ __attribute__ ((__aligned__(AESNI_ALIGN)));
u8 nonce[4];
};
@@ -65,10 +70,6 @@ struct aesni_hash_subkey_req_data {
struct scatterlist sg;
};
-#define AESNI_ALIGN (16)
-#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE-1))
-#define RFC4106_HASH_SUBKEY_SIZE 16
-
struct aesni_lrw_ctx {
struct lrw_table_ctx lrw_table;
u8 raw_aes_ctx[sizeof(struct crypto_aes_ctx) + AESNI_ALIGN - 1];
@@ -282,10 +283,11 @@ static void (*aesni_gcm_dec_tfm)(void *ctx, u8 *out,
static inline struct
aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get(struct crypto_aead *tfm)
{
- return
- (struct aesni_rfc4106_gcm_ctx *)
- PTR_ALIGN((u8 *)
- crypto_tfm_ctx(crypto_aead_tfm(tfm)), AESNI_ALIGN);
+ unsigned long align = AESNI_ALIGN;
+
+ if (align <= crypto_tfm_ctx_alignment())
+ align = 1;
+ return PTR_ALIGN(crypto_aead_ctx(tfm), align);
}
#endif
@@ -838,8 +840,6 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
if (IS_ERR(ctr_tfm))
return PTR_ERR(ctr_tfm);
- crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
-
ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
if (ret)
goto out_free_ablkcipher;
@@ -888,56 +888,20 @@ out_free_ablkcipher:
static int common_rfc4106_set_key(struct crypto_aead *aead, const u8 *key,
unsigned int key_len)
{
- int ret = 0;
- struct crypto_tfm *tfm = crypto_aead_tfm(aead);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(aead);
- u8 *new_key_align, *new_key_mem = NULL;
if (key_len < 4) {
- crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ crypto_aead_set_flags(aead, CRYPTO_TFM_RES_BAD_KEY_LEN);
return -EINVAL;
}
/*Account for 4 byte nonce at the end.*/
key_len -= 4;
- if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256) {
- crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
- return -EINVAL;
- }
memcpy(ctx->nonce, key + key_len, sizeof(ctx->nonce));
- /*This must be on a 16 byte boundary!*/
- if ((unsigned long)(&(ctx->aes_key_expanded.key_enc[0])) % AESNI_ALIGN)
- return -EINVAL;
-
- if ((unsigned long)key % AESNI_ALIGN) {
- /*key is not aligned: use an auxuliar aligned pointer*/
- new_key_mem = kmalloc(key_len+AESNI_ALIGN, GFP_KERNEL);
- if (!new_key_mem)
- return -ENOMEM;
-
- new_key_align = PTR_ALIGN(new_key_mem, AESNI_ALIGN);
- memcpy(new_key_align, key, key_len);
- key = new_key_align;
- }
- if (!irq_fpu_usable())
- ret = crypto_aes_expand_key(&(ctx->aes_key_expanded),
- key, key_len);
- else {
- kernel_fpu_begin();
- ret = aesni_set_key(&(ctx->aes_key_expanded), key, key_len);
- kernel_fpu_end();
- }
- /*This must be on a 16 byte boundary!*/
- if ((unsigned long)(&(ctx->hash_subkey[0])) % AESNI_ALIGN) {
- ret = -EINVAL;
- goto exit;
- }
- ret = rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
-exit:
- kfree(new_key_mem);
- return ret;
+ return aes_set_key_common(crypto_aead_tfm(aead),
+ &ctx->aes_key_expanded, key, key_len) ?:
+ rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
}
static int rfc4106_set_key(struct crypto_aead *parent, const u8 *key,
@@ -960,7 +924,7 @@ static int common_rfc4106_set_authsize(struct crypto_aead *aead,
default:
return -EINVAL;
}
- crypto_aead_crt(aead)->authsize = authsize;
+
return 0;
}
@@ -975,20 +939,17 @@ static int rfc4106_set_authsize(struct crypto_aead *parent,
return crypto_aead_setauthsize(&cryptd_tfm->base, authsize);
}
-static int __driver_rfc4106_encrypt(struct aead_request *req)
+static int helper_rfc4106_encrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
__be32 counter = cpu_to_be32(1);
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
- u32 key_len = ctx->aes_key_expanded.key_length;
void *aes_ctx = &(ctx->aes_key_expanded);
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
- u8 iv_tab[16+AESNI_ALIGN];
- u8* iv = (u8 *) PTR_ALIGN((u8 *)iv_tab, AESNI_ALIGN);
+ u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
struct scatter_walk src_sg_walk;
- struct scatter_walk assoc_sg_walk;
struct scatter_walk dst_sg_walk;
unsigned int i;
@@ -997,12 +958,6 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
/* to 8 or 12 bytes */
if (unlikely(req->assoclen != 8 && req->assoclen != 12))
return -EINVAL;
- if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len != 16))
- return -EINVAL;
- if (unlikely(key_len != AES_KEYSIZE_128 &&
- key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256))
- return -EINVAL;
/* IV below built */
for (i = 0; i < 4; i++)
@@ -1011,55 +966,57 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
*(iv+4+i) = req->iv[i];
*((__be32 *)(iv+12)) = counter;
- if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
+ if (sg_is_last(req->src) &&
+ req->src->offset + req->src->length <= PAGE_SIZE &&
+ sg_is_last(req->dst) &&
+ req->dst->offset + req->dst->length <= PAGE_SIZE) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
- scatterwalk_start(&assoc_sg_walk, req->assoc);
- src = scatterwalk_map(&src_sg_walk);
- assoc = scatterwalk_map(&assoc_sg_walk);
+ assoc = scatterwalk_map(&src_sg_walk);
+ src = assoc + req->assoclen;
dst = src;
if (unlikely(req->src != req->dst)) {
scatterwalk_start(&dst_sg_walk, req->dst);
- dst = scatterwalk_map(&dst_sg_walk);
+ dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
}
-
} else {
/* Allocate memory for src, dst, assoc */
- src = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
+ assoc = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
GFP_ATOMIC);
- if (unlikely(!src))
+ if (unlikely(!assoc))
return -ENOMEM;
- assoc = (src + req->cryptlen + auth_tag_len);
- scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
- scatterwalk_map_and_copy(assoc, req->assoc, 0,
- req->assoclen, 0);
+ scatterwalk_map_and_copy(assoc, req->src, 0,
+ req->assoclen + req->cryptlen, 0);
+ src = assoc + req->assoclen;
dst = src;
}
+ kernel_fpu_begin();
aesni_gcm_enc_tfm(aes_ctx, dst, src, (unsigned long)req->cryptlen, iv,
ctx->hash_subkey, assoc, (unsigned long)req->assoclen, dst
+ ((unsigned long)req->cryptlen), auth_tag_len);
+ kernel_fpu_end();
/* The authTag (aka the Integrity Check Value) needs to be written
* back to the packet. */
if (one_entry_in_sg) {
if (unlikely(req->src != req->dst)) {
- scatterwalk_unmap(dst);
- scatterwalk_done(&dst_sg_walk, 0, 0);
+ scatterwalk_unmap(dst - req->assoclen);
+ scatterwalk_advance(&dst_sg_walk, req->dst->length);
+ scatterwalk_done(&dst_sg_walk, 1, 0);
}
- scatterwalk_unmap(src);
scatterwalk_unmap(assoc);
- scatterwalk_done(&src_sg_walk, 0, 0);
- scatterwalk_done(&assoc_sg_walk, 0, 0);
+ scatterwalk_advance(&src_sg_walk, req->src->length);
+ scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0,
- req->cryptlen + auth_tag_len, 1);
- kfree(src);
+ scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
+ req->cryptlen + auth_tag_len, 1);
+ kfree(assoc);
}
return 0;
}
-static int __driver_rfc4106_decrypt(struct aead_request *req)
+static int helper_rfc4106_decrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
@@ -1068,26 +1025,16 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
int retval = 0;
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
- u32 key_len = ctx->aes_key_expanded.key_length;
void *aes_ctx = &(ctx->aes_key_expanded);
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
- u8 iv_and_authTag[32+AESNI_ALIGN];
- u8 *iv = (u8 *) PTR_ALIGN((u8 *)iv_and_authTag, AESNI_ALIGN);
- u8 *authTag = iv + 16;
+ u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
+ u8 authTag[16];
struct scatter_walk src_sg_walk;
- struct scatter_walk assoc_sg_walk;
struct scatter_walk dst_sg_walk;
unsigned int i;
- if (unlikely((req->cryptlen < auth_tag_len) ||
- (req->assoclen != 8 && req->assoclen != 12)))
+ if (unlikely(req->assoclen != 8 && req->assoclen != 12))
return -EINVAL;
- if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len != 16))
- return -EINVAL;
- if (unlikely(key_len != AES_KEYSIZE_128 &&
- key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256))
- return -EINVAL;
/* Assuming we are supporting rfc4106 64-bit extended */
/* sequence numbers We need to have the AAD length */
@@ -1101,33 +1048,36 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
*(iv+4+i) = req->iv[i];
*((__be32 *)(iv+12)) = counter;
- if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
+ if (sg_is_last(req->src) &&
+ req->src->offset + req->src->length <= PAGE_SIZE &&
+ sg_is_last(req->dst) &&
+ req->dst->offset + req->dst->length <= PAGE_SIZE) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
- scatterwalk_start(&assoc_sg_walk, req->assoc);
- src = scatterwalk_map(&src_sg_walk);
- assoc = scatterwalk_map(&assoc_sg_walk);
+ assoc = scatterwalk_map(&src_sg_walk);
+ src = assoc + req->assoclen;
dst = src;
if (unlikely(req->src != req->dst)) {
scatterwalk_start(&dst_sg_walk, req->dst);
- dst = scatterwalk_map(&dst_sg_walk);
+ dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
}
} else {
/* Allocate memory for src, dst, assoc */
- src = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
- if (!src)
+ assoc = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
+ if (!assoc)
return -ENOMEM;
- assoc = (src + req->cryptlen);
- scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
- scatterwalk_map_and_copy(assoc, req->assoc, 0,
- req->assoclen, 0);
+ scatterwalk_map_and_copy(assoc, req->src, 0,
+ req->assoclen + req->cryptlen, 0);
+ src = assoc + req->assoclen;
dst = src;
}
+ kernel_fpu_begin();
aesni_gcm_dec_tfm(aes_ctx, dst, src, tempCipherLen, iv,
ctx->hash_subkey, assoc, (unsigned long)req->assoclen,
authTag, auth_tag_len);
+ kernel_fpu_end();
/* Compare generated tag with passed in tag. */
retval = crypto_memneq(src + tempCipherLen, authTag, auth_tag_len) ?
@@ -1135,16 +1085,15 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
if (one_entry_in_sg) {
if (unlikely(req->src != req->dst)) {
- scatterwalk_unmap(dst);
- scatterwalk_done(&dst_sg_walk, 0, 0);
+ scatterwalk_unmap(dst - req->assoclen);
+ scatterwalk_done(&dst_sg_walk, 1, 0);
}
- scatterwalk_unmap(src);
scatterwalk_unmap(assoc);
- scatterwalk_done(&src_sg_walk, 0, 0);
- scatterwalk_done(&assoc_sg_walk, 0, 0);
+ scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0, tempCipherLen, 1);
- kfree(src);
+ scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
+ tempCipherLen, 1);
+ kfree(assoc);
}
return retval;
}
@@ -1188,36 +1137,6 @@ static int rfc4106_decrypt(struct aead_request *req)
return crypto_aead_decrypt(subreq);
}
-
-static int helper_rfc4106_encrypt(struct aead_request *req)
-{
- int ret;
-
- if (unlikely(!irq_fpu_usable())) {
- WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
- ret = -EINVAL;
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_encrypt(req);
- kernel_fpu_end();
- }
- return ret;
-}
-
-static int helper_rfc4106_decrypt(struct aead_request *req)
-{
- int ret;
-
- if (unlikely(!irq_fpu_usable())) {
- WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
- ret = -EINVAL;
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_decrypt(req);
- kernel_fpu_end();
- }
- return ret;
-}
#endif
static struct crypto_alg aesni_algs[] = { {
@@ -1389,27 +1308,6 @@ static struct crypto_alg aesni_algs[] = { {
.geniv = "chainiv",
},
},
-}, {
- .cra_name = "__gcm-aes-aesni",
- .cra_driver_name = "__driver-gcm-aes-aesni",
- .cra_priority = 0,
- .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_INTERNAL,
- .cra_blocksize = 1,
- .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx) +
- AESNI_ALIGN,
- .cra_alignmask = 0,
- .cra_type = &crypto_aead_type,
- .cra_module = THIS_MODULE,
- .cra_u = {
- .aead = {
- .setkey = common_rfc4106_set_key,
- .setauthsize = common_rfc4106_set_authsize,
- .encrypt = helper_rfc4106_encrypt,
- .decrypt = helper_rfc4106_decrypt,
- .ivsize = 8,
- .maxauthsize = 16,
- },
- },
#endif
#if IS_ENABLED(CONFIG_CRYPTO_PCBC)
}, {
@@ -1526,6 +1424,22 @@ static struct crypto_alg aesni_algs[] = { {
#ifdef CONFIG_X86_64
static struct aead_alg aesni_aead_algs[] = { {
+ .setkey = common_rfc4106_set_key,
+ .setauthsize = common_rfc4106_set_authsize,
+ .encrypt = helper_rfc4106_encrypt,
+ .decrypt = helper_rfc4106_decrypt,
+ .ivsize = 8,
+ .maxauthsize = 16,
+ .base = {
+ .cra_name = "__gcm-aes-aesni",
+ .cra_driver_name = "__driver-gcm-aes-aesni",
+ .cra_flags = CRYPTO_ALG_INTERNAL,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx),
+ .cra_alignmask = AESNI_ALIGN - 1,
+ .cra_module = THIS_MODULE,
+ },
+}, {
.init = rfc4106_init,
.exit = rfc4106_exit,
.setkey = rfc4106_set_key,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply related [flat|nested] 25+ messages in thread* [v3 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-06-01 7:50 ` [v2 PATCH] " Herbert Xu
@ 2015-06-01 7:53 ` Herbert Xu
2015-06-01 9:04 ` Stephan Mueller
0 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-06-01 7:53 UTC (permalink / raw)
To: Stephan Mueller
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
On Mon, Jun 01, 2015 at 03:50:22PM +0800, Herbert Xu wrote:
> This patch converts the low-level __gcm-aes-aesni algorithm to
> the new AEAD interface.
Oops, I missed two more spots.
---8<---
This patch converts the low-level __gcm-aes-aesni algorithm to
the new AEAD interface.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index 5660a18..ebcb981d 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -44,13 +44,18 @@
#endif
+#define AESNI_ALIGN 16
+#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE - 1))
+#define RFC4106_HASH_SUBKEY_SIZE 16
+
/* This data is stored at the end of the crypto_tfm struct.
* It's a type of per "session" data storage location.
* This needs to be 16 byte aligned.
*/
struct aesni_rfc4106_gcm_ctx {
- u8 hash_subkey[16];
- struct crypto_aes_ctx aes_key_expanded;
+ u8 hash_subkey[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
+ struct crypto_aes_ctx aes_key_expanded
+ __attribute__ ((__aligned__(AESNI_ALIGN)));
u8 nonce[4];
};
@@ -65,10 +70,6 @@ struct aesni_hash_subkey_req_data {
struct scatterlist sg;
};
-#define AESNI_ALIGN (16)
-#define AES_BLOCK_MASK (~(AES_BLOCK_SIZE-1))
-#define RFC4106_HASH_SUBKEY_SIZE 16
-
struct aesni_lrw_ctx {
struct lrw_table_ctx lrw_table;
u8 raw_aes_ctx[sizeof(struct crypto_aes_ctx) + AESNI_ALIGN - 1];
@@ -282,10 +283,11 @@ static void (*aesni_gcm_dec_tfm)(void *ctx, u8 *out,
static inline struct
aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get(struct crypto_aead *tfm)
{
- return
- (struct aesni_rfc4106_gcm_ctx *)
- PTR_ALIGN((u8 *)
- crypto_tfm_ctx(crypto_aead_tfm(tfm)), AESNI_ALIGN);
+ unsigned long align = AESNI_ALIGN;
+
+ if (align <= crypto_tfm_ctx_alignment())
+ align = 1;
+ return PTR_ALIGN(crypto_aead_ctx(tfm), align);
}
#endif
@@ -838,8 +840,6 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
if (IS_ERR(ctr_tfm))
return PTR_ERR(ctr_tfm);
- crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
-
ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
if (ret)
goto out_free_ablkcipher;
@@ -888,56 +888,20 @@ out_free_ablkcipher:
static int common_rfc4106_set_key(struct crypto_aead *aead, const u8 *key,
unsigned int key_len)
{
- int ret = 0;
- struct crypto_tfm *tfm = crypto_aead_tfm(aead);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(aead);
- u8 *new_key_align, *new_key_mem = NULL;
if (key_len < 4) {
- crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
+ crypto_aead_set_flags(aead, CRYPTO_TFM_RES_BAD_KEY_LEN);
return -EINVAL;
}
/*Account for 4 byte nonce at the end.*/
key_len -= 4;
- if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256) {
- crypto_tfm_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN);
- return -EINVAL;
- }
memcpy(ctx->nonce, key + key_len, sizeof(ctx->nonce));
- /*This must be on a 16 byte boundary!*/
- if ((unsigned long)(&(ctx->aes_key_expanded.key_enc[0])) % AESNI_ALIGN)
- return -EINVAL;
-
- if ((unsigned long)key % AESNI_ALIGN) {
- /*key is not aligned: use an auxuliar aligned pointer*/
- new_key_mem = kmalloc(key_len+AESNI_ALIGN, GFP_KERNEL);
- if (!new_key_mem)
- return -ENOMEM;
-
- new_key_align = PTR_ALIGN(new_key_mem, AESNI_ALIGN);
- memcpy(new_key_align, key, key_len);
- key = new_key_align;
- }
- if (!irq_fpu_usable())
- ret = crypto_aes_expand_key(&(ctx->aes_key_expanded),
- key, key_len);
- else {
- kernel_fpu_begin();
- ret = aesni_set_key(&(ctx->aes_key_expanded), key, key_len);
- kernel_fpu_end();
- }
- /*This must be on a 16 byte boundary!*/
- if ((unsigned long)(&(ctx->hash_subkey[0])) % AESNI_ALIGN) {
- ret = -EINVAL;
- goto exit;
- }
- ret = rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
-exit:
- kfree(new_key_mem);
- return ret;
+ return aes_set_key_common(crypto_aead_tfm(aead),
+ &ctx->aes_key_expanded, key, key_len) ?:
+ rfc4106_set_hash_subkey(ctx->hash_subkey, key, key_len);
}
static int rfc4106_set_key(struct crypto_aead *parent, const u8 *key,
@@ -960,7 +924,7 @@ static int common_rfc4106_set_authsize(struct crypto_aead *aead,
default:
return -EINVAL;
}
- crypto_aead_crt(aead)->authsize = authsize;
+
return 0;
}
@@ -975,20 +939,17 @@ static int rfc4106_set_authsize(struct crypto_aead *parent,
return crypto_aead_setauthsize(&cryptd_tfm->base, authsize);
}
-static int __driver_rfc4106_encrypt(struct aead_request *req)
+static int helper_rfc4106_encrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
__be32 counter = cpu_to_be32(1);
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
- u32 key_len = ctx->aes_key_expanded.key_length;
void *aes_ctx = &(ctx->aes_key_expanded);
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
- u8 iv_tab[16+AESNI_ALIGN];
- u8* iv = (u8 *) PTR_ALIGN((u8 *)iv_tab, AESNI_ALIGN);
+ u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
struct scatter_walk src_sg_walk;
- struct scatter_walk assoc_sg_walk;
struct scatter_walk dst_sg_walk;
unsigned int i;
@@ -997,12 +958,6 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
/* to 8 or 12 bytes */
if (unlikely(req->assoclen != 8 && req->assoclen != 12))
return -EINVAL;
- if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len != 16))
- return -EINVAL;
- if (unlikely(key_len != AES_KEYSIZE_128 &&
- key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256))
- return -EINVAL;
/* IV below built */
for (i = 0; i < 4; i++)
@@ -1011,55 +966,57 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
*(iv+4+i) = req->iv[i];
*((__be32 *)(iv+12)) = counter;
- if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
+ if (sg_is_last(req->src) &&
+ req->src->offset + req->src->length <= PAGE_SIZE &&
+ sg_is_last(req->dst) &&
+ req->dst->offset + req->dst->length <= PAGE_SIZE) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
- scatterwalk_start(&assoc_sg_walk, req->assoc);
- src = scatterwalk_map(&src_sg_walk);
- assoc = scatterwalk_map(&assoc_sg_walk);
+ assoc = scatterwalk_map(&src_sg_walk);
+ src = assoc + req->assoclen;
dst = src;
if (unlikely(req->src != req->dst)) {
scatterwalk_start(&dst_sg_walk, req->dst);
- dst = scatterwalk_map(&dst_sg_walk);
+ dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
}
-
} else {
/* Allocate memory for src, dst, assoc */
- src = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
+ assoc = kmalloc(req->cryptlen + auth_tag_len + req->assoclen,
GFP_ATOMIC);
- if (unlikely(!src))
+ if (unlikely(!assoc))
return -ENOMEM;
- assoc = (src + req->cryptlen + auth_tag_len);
- scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
- scatterwalk_map_and_copy(assoc, req->assoc, 0,
- req->assoclen, 0);
+ scatterwalk_map_and_copy(assoc, req->src, 0,
+ req->assoclen + req->cryptlen, 0);
+ src = assoc + req->assoclen;
dst = src;
}
+ kernel_fpu_begin();
aesni_gcm_enc_tfm(aes_ctx, dst, src, (unsigned long)req->cryptlen, iv,
ctx->hash_subkey, assoc, (unsigned long)req->assoclen, dst
+ ((unsigned long)req->cryptlen), auth_tag_len);
+ kernel_fpu_end();
/* The authTag (aka the Integrity Check Value) needs to be written
* back to the packet. */
if (one_entry_in_sg) {
if (unlikely(req->src != req->dst)) {
- scatterwalk_unmap(dst);
- scatterwalk_done(&dst_sg_walk, 0, 0);
+ scatterwalk_unmap(dst - req->assoclen);
+ scatterwalk_advance(&dst_sg_walk, req->dst->length);
+ scatterwalk_done(&dst_sg_walk, 1, 0);
}
- scatterwalk_unmap(src);
scatterwalk_unmap(assoc);
- scatterwalk_done(&src_sg_walk, 0, 0);
- scatterwalk_done(&assoc_sg_walk, 0, 0);
+ scatterwalk_advance(&src_sg_walk, req->src->length);
+ scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0,
- req->cryptlen + auth_tag_len, 1);
- kfree(src);
+ scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
+ req->cryptlen + auth_tag_len, 1);
+ kfree(assoc);
}
return 0;
}
-static int __driver_rfc4106_decrypt(struct aead_request *req)
+static int helper_rfc4106_decrypt(struct aead_request *req)
{
u8 one_entry_in_sg = 0;
u8 *src, *dst, *assoc;
@@ -1068,26 +1025,16 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
int retval = 0;
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct aesni_rfc4106_gcm_ctx *ctx = aesni_rfc4106_gcm_ctx_get(tfm);
- u32 key_len = ctx->aes_key_expanded.key_length;
void *aes_ctx = &(ctx->aes_key_expanded);
unsigned long auth_tag_len = crypto_aead_authsize(tfm);
- u8 iv_and_authTag[32+AESNI_ALIGN];
- u8 *iv = (u8 *) PTR_ALIGN((u8 *)iv_and_authTag, AESNI_ALIGN);
- u8 *authTag = iv + 16;
+ u8 iv[16] __attribute__ ((__aligned__(AESNI_ALIGN)));
+ u8 authTag[16];
struct scatter_walk src_sg_walk;
- struct scatter_walk assoc_sg_walk;
struct scatter_walk dst_sg_walk;
unsigned int i;
- if (unlikely((req->cryptlen < auth_tag_len) ||
- (req->assoclen != 8 && req->assoclen != 12)))
+ if (unlikely(req->assoclen != 8 && req->assoclen != 12))
return -EINVAL;
- if (unlikely(auth_tag_len != 8 && auth_tag_len != 12 && auth_tag_len != 16))
- return -EINVAL;
- if (unlikely(key_len != AES_KEYSIZE_128 &&
- key_len != AES_KEYSIZE_192 &&
- key_len != AES_KEYSIZE_256))
- return -EINVAL;
/* Assuming we are supporting rfc4106 64-bit extended */
/* sequence numbers We need to have the AAD length */
@@ -1101,33 +1048,36 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
*(iv+4+i) = req->iv[i];
*((__be32 *)(iv+12)) = counter;
- if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
+ if (sg_is_last(req->src) &&
+ req->src->offset + req->src->length <= PAGE_SIZE &&
+ sg_is_last(req->dst) &&
+ req->dst->offset + req->dst->length <= PAGE_SIZE) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
- scatterwalk_start(&assoc_sg_walk, req->assoc);
- src = scatterwalk_map(&src_sg_walk);
- assoc = scatterwalk_map(&assoc_sg_walk);
+ assoc = scatterwalk_map(&src_sg_walk);
+ src = assoc + req->assoclen;
dst = src;
if (unlikely(req->src != req->dst)) {
scatterwalk_start(&dst_sg_walk, req->dst);
- dst = scatterwalk_map(&dst_sg_walk);
+ dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
}
} else {
/* Allocate memory for src, dst, assoc */
- src = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
- if (!src)
+ assoc = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
+ if (!assoc)
return -ENOMEM;
- assoc = (src + req->cryptlen);
- scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
- scatterwalk_map_and_copy(assoc, req->assoc, 0,
- req->assoclen, 0);
+ scatterwalk_map_and_copy(assoc, req->src, 0,
+ req->assoclen + req->cryptlen, 0);
+ src = assoc + req->assoclen;
dst = src;
}
+ kernel_fpu_begin();
aesni_gcm_dec_tfm(aes_ctx, dst, src, tempCipherLen, iv,
ctx->hash_subkey, assoc, (unsigned long)req->assoclen,
authTag, auth_tag_len);
+ kernel_fpu_end();
/* Compare generated tag with passed in tag. */
retval = crypto_memneq(src + tempCipherLen, authTag, auth_tag_len) ?
@@ -1135,16 +1085,17 @@ static int __driver_rfc4106_decrypt(struct aead_request *req)
if (one_entry_in_sg) {
if (unlikely(req->src != req->dst)) {
- scatterwalk_unmap(dst);
- scatterwalk_done(&dst_sg_walk, 0, 0);
+ scatterwalk_unmap(dst - req->assoclen);
+ scatterwalk_advance(&dst_sg_walk, req->dst->length);
+ scatterwalk_done(&dst_sg_walk, 1, 0);
}
- scatterwalk_unmap(src);
scatterwalk_unmap(assoc);
- scatterwalk_done(&src_sg_walk, 0, 0);
- scatterwalk_done(&assoc_sg_walk, 0, 0);
+ scatterwalk_advance(&src_sg_walk, req->src->length);
+ scatterwalk_done(&src_sg_walk, req->src == req->dst, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0, tempCipherLen, 1);
- kfree(src);
+ scatterwalk_map_and_copy(dst, req->dst, req->assoclen,
+ tempCipherLen, 1);
+ kfree(assoc);
}
return retval;
}
@@ -1188,36 +1139,6 @@ static int rfc4106_decrypt(struct aead_request *req)
return crypto_aead_decrypt(subreq);
}
-
-static int helper_rfc4106_encrypt(struct aead_request *req)
-{
- int ret;
-
- if (unlikely(!irq_fpu_usable())) {
- WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
- ret = -EINVAL;
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_encrypt(req);
- kernel_fpu_end();
- }
- return ret;
-}
-
-static int helper_rfc4106_decrypt(struct aead_request *req)
-{
- int ret;
-
- if (unlikely(!irq_fpu_usable())) {
- WARN_ONCE(1, "__gcm-aes-aesni alg used in invalid context");
- ret = -EINVAL;
- } else {
- kernel_fpu_begin();
- ret = __driver_rfc4106_decrypt(req);
- kernel_fpu_end();
- }
- return ret;
-}
#endif
static struct crypto_alg aesni_algs[] = { {
@@ -1389,27 +1310,6 @@ static struct crypto_alg aesni_algs[] = { {
.geniv = "chainiv",
},
},
-}, {
- .cra_name = "__gcm-aes-aesni",
- .cra_driver_name = "__driver-gcm-aes-aesni",
- .cra_priority = 0,
- .cra_flags = CRYPTO_ALG_TYPE_AEAD | CRYPTO_ALG_INTERNAL,
- .cra_blocksize = 1,
- .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx) +
- AESNI_ALIGN,
- .cra_alignmask = 0,
- .cra_type = &crypto_aead_type,
- .cra_module = THIS_MODULE,
- .cra_u = {
- .aead = {
- .setkey = common_rfc4106_set_key,
- .setauthsize = common_rfc4106_set_authsize,
- .encrypt = helper_rfc4106_encrypt,
- .decrypt = helper_rfc4106_decrypt,
- .ivsize = 8,
- .maxauthsize = 16,
- },
- },
#endif
#if IS_ENABLED(CONFIG_CRYPTO_PCBC)
}, {
@@ -1526,6 +1426,22 @@ static struct crypto_alg aesni_algs[] = { {
#ifdef CONFIG_X86_64
static struct aead_alg aesni_aead_algs[] = { {
+ .setkey = common_rfc4106_set_key,
+ .setauthsize = common_rfc4106_set_authsize,
+ .encrypt = helper_rfc4106_encrypt,
+ .decrypt = helper_rfc4106_decrypt,
+ .ivsize = 8,
+ .maxauthsize = 16,
+ .base = {
+ .cra_name = "__gcm-aes-aesni",
+ .cra_driver_name = "__driver-gcm-aes-aesni",
+ .cra_flags = CRYPTO_ALG_INTERNAL,
+ .cra_blocksize = 1,
+ .cra_ctxsize = sizeof(struct aesni_rfc4106_gcm_ctx),
+ .cra_alignmask = AESNI_ALIGN - 1,
+ .cra_module = THIS_MODULE,
+ },
+}, {
.init = rfc4106_init,
.exit = rfc4106_exit,
.setkey = rfc4106_set_key,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [v3 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-06-01 7:53 ` [v3 " Herbert Xu
@ 2015-06-01 9:04 ` Stephan Mueller
2015-06-01 9:09 ` Herbert Xu
0 siblings, 1 reply; 25+ messages in thread
From: Stephan Mueller @ 2015-06-01 9:04 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
Am Montag, 1. Juni 2015, 15:53:06 schrieb Herbert Xu:
Hi Herbert,
>On Mon, Jun 01, 2015 at 03:50:22PM +0800, Herbert Xu wrote:
>> This patch converts the low-level __gcm-aes-aesni algorithm to
>> the new AEAD interface.
>
>Oops, I missed two more spots.
That patch fixes the crash. Thanks
Just FYI: when testing rfc4106(gcm(aes-aesni)) with the old givcipher API, I
got a crash in my code invoking the cipher which used to work in older kernels
and which works with the C implementations. So, there must be some change in
how givcipher is treated by the AESNI implementation.
As this API is sunset now, shall I dig deeper or shall we simply deactivate
the API entirely?
--
Ciao
Stephan
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [v3 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-06-01 9:04 ` Stephan Mueller
@ 2015-06-01 9:09 ` Herbert Xu
2015-06-01 9:44 ` Stephan Mueller
0 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-06-01 9:09 UTC (permalink / raw)
To: Stephan Mueller
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
On Mon, Jun 01, 2015 at 11:04:39AM +0200, Stephan Mueller wrote:
>
> Just FYI: when testing rfc4106(gcm(aes-aesni)) with the old givcipher API, I
> got a crash in my code invoking the cipher which used to work in older kernels
> and which works with the C implementations. So, there must be some change in
> how givcipher is treated by the AESNI implementation.
>
> As this API is sunset now, shall I dig deeper or shall we simply deactivate
> the API entirely?
What do you mean by the old givcipher API? Are you referring to the
old algif_aead that was disabled?
Can you show me the crash that you got and as much information as
you can about the caller that triggered the crash?
The assumption at this point is that all users outside of crypto/
itself are gone. But I'd still like to look at the crash just in
case it's relevant for users within crypto.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [v3 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-06-01 9:09 ` Herbert Xu
@ 2015-06-01 9:44 ` Stephan Mueller
2015-06-01 9:52 ` Herbert Xu
0 siblings, 1 reply; 25+ messages in thread
From: Stephan Mueller @ 2015-06-01 9:44 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
Am Montag, 1. Juni 2015, 17:09:51 schrieb Herbert Xu:
Hi Herbert,
> On Mon, Jun 01, 2015 at 11:04:39AM +0200, Stephan Mueller wrote:
> > Just FYI: when testing rfc4106(gcm(aes-aesni)) with the old givcipher API,
> > I got a crash in my code invoking the cipher which used to work in older
> > kernels and which works with the C implementations. So, there must be
> > some change in how givcipher is treated by the AESNI implementation.
> >
> > As this API is sunset now, shall I dig deeper or shall we simply
> > deactivate
> > the API entirely?
>
> What do you mean by the old givcipher API? Are you referring to the
> old algif_aead that was disabled?
I am referring to the crypto_aead_givencrypt in-kernel API.
Before the big change, the RFC4106 ciphers could be used with the standard
crypto_aead_encrypt API calls where the caller must provide the IV. When using
the crypto_aead_givencrypt, the IV was generated using the seqiv.
When testing the current code base with the old crypto_aead_givencrypt API
call, I get the following results: using rfc4106(gcm(aes-asm)) with the old
API works just fine. Using rfc4106(gcm(aes-aesni)) with the very same code
crashes somewhere in my code.
>
> Can you show me the crash that you got and as much information as
> you can about the caller that triggered the crash?
Again, the caller is my out-of tree crypto API test harness.
That code crashes with the following stacktrace:
[ 2000.433502] BUG: unable to handle kernel NULL pointer dereference at
(null)
[ 2000.433505] IP: [< (null)>] (null)
[ 2000.433508] PGD 44aa5067 PUD 7c2d3067 PMD 0
[ 2000.433511] Oops: 0010 [#3] SMP
[ 2000.433514] Modules linked in: kcapi_cavs(OE) ctr ghash_generic gcm ecb cbc
sha512_ssse3 sha512_generic sha256_ssse3 sha1_ssse3 sha1_generic
nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT
nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4
nf_defrag_ipv4 xt_conntrack nf_conntrack cfg80211 ebtable_nat ebtable_broute
bridge stp llc ebtable_filter ebtables ip6table_mangle ip6table_security
ip6table_raw ip6table_filter ip6_tables iptable_mangle iptable_security
iptable_raw crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel
aesni_intel aes_x86_64 glue_helper ablk_helper joydev microcode virtio_balloon
serio_raw pcspkr acpi_cpufreq i2c_piix4 virtio_blk qxl virtio_net
drm_kms_helper ttm drm virtio_pci virtio_ring virtio [last unloaded:
kcapi_cavs]
[ 2000.433550] CPU: 0 PID: 12614 Comm: perl Tainted: G D OE 4.0.0+
#228
[ 2000.433552] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.8.1-20150318_183358- 04/01/2014
[ 2000.433554] task: ffff88007beaee80 ti: ffff880001bcc000 task.ti:
ffff880001bcc000
[ 2000.433556] RIP: 0010:[<0000000000000000>] [< (null)>]
(null)
[ 2000.433558] RSP: 0018:ffff880001bcfd20 EFLAGS: 00010206
[ 2000.433560] RAX: ffff8800363a5040 RBX: ffff88007c59e800 RCX:
0200000000000000
[ 2000.433561] RDX: ffffffffa02e2280 RSI: ffffffffa02e051c RDI:
ffff880064291c00
[ 2000.433563] RBP: ffff880001bcfde8 R08: 00000000000191e0 R09:
ffff880001bcfd40
[ 2000.433564] R10: ffffffff811636d1 R11: 0000000000000006 R12:
ffff880064291c00
[ 2000.433566] R13: ffff8800363a5000 R14: 0000000000000000 R15:
0000000000000010
[ 2000.433568] FS: 00007f7b0e85f700(0000) GS:ffff88007fc00000(0000)
knlGS:0000000000000000
[ 2000.433570] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2000.433571] CR2: 0000000000000000 CR3: 000000007bc7d000 CR4:
00000000000407f0
[ 2000.433577] Stack:
[ 2000.433579] ffffffffa02de986 ffff880001bcfd40 0200000000000000
0000000000eaee80
[ 2000.433582] ffffea000005b002 0000002000000000 0000000000000000
0000000000000000
[ 2000.433585] ffffea000005c802 0000000800000000 0000000000000000
0000000000000000
[ 2000.433587] Call Trace:
[ 2000.433593] [<ffffffffa02de986>] ? kccavs_test_aead_givenc+0x226/0x3b0
[kcapi_cavs]
[ 2000.433597] [<ffffffffa02ddf17>] kccavs_data_read+0xf7/0x130 [kcapi_cavs]
[ 2000.433602] [<ffffffff811a9928>] __vfs_read+0x28/0xc0
[ 2000.433605] [<ffffffff81294324>] ? security_file_permission+0x84/0xa0
[ 2000.433607] [<ffffffff811a9ff3>] ? rw_verify_area+0x53/0x100
[ 2000.433609] [<ffffffff811aa12a>] vfs_read+0x8a/0x140
[ 2000.433612] [<ffffffff811aab56>] SyS_read+0x46/0xb0
[ 2000.433616] [<ffffffff8104a0f7>] ? trace_do_page_fault+0x37/0xb0
[ 2000.433620] [<ffffffff816877ee>] system_call_fastpath+0x12/0x71
[ 2000.433622] Code: Bad RIP value.
[ 2000.433625] RIP [< (null)>] (null)
[ 2000.433627] RSP <ffff880001bcfd20>
[ 2000.433628] CR2: 0000000000000000
[ 2000.433630] ---[ end trace d4df29cff525b7a9 ]---
The entire code that triggers the crash is the following:
#ifndef NEWAEAD
/* tie all data structures together */
struct kccavs_givaead_def {
struct crypto_aead *tfm;
struct aead_givcrypt_request *req;
struct kccavs_tcrypt_res result;
};
/* Perform encryption */
static unsigned int kccavs_givaead_enc(struct kccavs_givaead_def *aead)
{
int rc = crypto_aead_givencrypt(aead->req);
switch (rc) {
case 0:
break;
case -EINPROGRESS:
case -EBUSY:
rc = wait_for_completion_interruptible(&aead-
>result.completion);
if (!rc && !aead->result.err) {
#ifdef OLDASYNC
INIT_COMPLETION(aead->result.completion);
#else
reinit_completion(&aead->result.completion);
#endif
break;
}
default:
dbg(DRIVER_NAME": aead cipher operation returned with %d
result"
" %d\n",rc, aead->result.err);
break;
}
init_completion(&aead->result.completion);
return rc;
}
#endif
/*
* GIV AEAD encryption
* input: type
* input: name
* input: plaintext / ciphertext in kccavs_test->data
* input: AuthTag is appended to ciphertext
* input: Authsize
* input: key in kccavs_test->key
* input: associated data in kccavs_test->aead_assoc
* output: ciphertext / plaintext in kccavs_test->data
* output: IV in kccavs_test->iv
*
* Note: for decryption, the data->data will contain deadbeef if the
* authentication failed.
*/
static int kccavs_test_aead_givenc(size_t nbytes)
{
int ret = -EFAULT;
struct crypto_aead *tfm = NULL;
#ifdef NEWAEAD
struct kccavs_aead_def aead;
struct aead_request *req = NULL;
struct scatterlist sg[3];
#else
struct kccavs_givaead_def aead;
struct aead_givcrypt_request *req = NULL;
struct scatterlist sg;
struct scatterlist assocsg;
#endif
struct kccavs_data *data = &kccavs_test->data;
struct kccavs_data *key = &kccavs_test->key;
struct kccavs_data *iv = &kccavs_test->iv;
struct kccavs_data *aead_assoc = &kccavs_test->aead_assoc;
u32 authsize = kccavs_test->aead_authsize;
/* data will hold plaintext and tag */
if (kccavs_test->type & TYPE_ENC &&
data->len + authsize > MAXDATALEN)
return -ENOSPC;
tfm = crypto_alloc_aead(kccavs_test->name, 0, 0);
if (IS_ERR(tfm)) {
pr_info("could not allocate aead handle for %s %ld\n",
kccavs_test->name, PTR_ERR(tfm));
return PTR_ERR(tfm);
}
#ifdef NEWAEAD
req = aead_request_alloc(tfm, GFP_KERNEL);
#else
req = aead_givcrypt_alloc(tfm, GFP_KERNEL);
#endif
if (IS_ERR(req)) {
pr_info("could not allocate request queue\n");
ret = PTR_ERR(req);
goto out;
}
ret = crypto_aead_setkey(tfm, key->data, key->len);
if (ret) {
pr_info("key could not be set %d\n", ret);
goto out;
}
ret = crypto_aead_setauthsize(tfm, authsize);
if (ret) {
pr_info("authsize %u could not be set %d\n", authsize, ret);
ret = -EAGAIN;
goto out;
}
aead.tfm = tfm;
aead.req = req;
#ifdef NEWAEAD
aead_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG,
kccavs_aead_cb, &aead.result);
#else
aead_givcrypt_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG,
kccavs_aead_cb, &aead.result);
#endif
#ifdef NEWAEAD
iv->len = crypto_aead_ivsize(aead.tfm);
sg_init_table(sg, 3);
sg_set_buf(&sg[0], aead_assoc->data, aead_assoc->len);
sg_set_buf(&sg[1], iv->data, iv->len);
sg_set_buf(&sg[2], data->data, data->len +
(kccavs_test->type & TYPE_ENC ? authsize : 0));
aead_request_set_ad(req, aead_assoc->len);
aead_request_set_crypt(req, sg, sg, data->len + iv->len, iv->data);
#else
sg_init_one(&sg, data->data, data->len +
((kccavs_test->type & TYPE_ENC) ? authsize : 0));
sg_init_one(&assocsg, aead_assoc->data, aead_assoc->len);
aead_givcrypt_set_assoc(req, &assocsg, aead_assoc->len);
iv->len = crypto_aead_ivsize(aead_givcrypt_reqtfm(req));
/*
* The IV pointer for AEAD is moved behind the IV value to be
generated
* by seqiv - regardless of what is found there, it will be
overwritten
* by seqiv. Moreover, we do not care what is found there as the
* seqiv generated IV is stored in iv->data anyhow.
*/
aead_givcrypt_set_crypt(req, &sg, &sg, data->len, iv->data);
sg_init_one(&sg, data->data, data->len +
((kccavs_test->type & TYPE_ENC) ? authsize : 0));
/*
* The IV pointer for the seqiv generated IV is iv->data to allow
* it to be extracted with the IV debugfs read function
*
* We use a sequence number starting at 0 as defined by RFC4303
*/
aead_givcrypt_set_giv(req, iv->data, 0);
#endif
init_completion(&aead.result.completion);
if (kccavs_test->type & TYPE_ENC) {
#ifdef NEWAEAD
ret = kccavs_aead_encdec(&aead, 1);
#else
ret = kccavs_givaead_enc(&aead);
#endif
/* data now contains ciphertext and concatenated tag */
data->len += authsize;
if (0 > ret) {
pr_info("AEAD encryption failed: %d\n", ret);
}
} else {
pr_err("AEAD: givcrypt is only intended for encrypt\n");
}
out:
if (tfm)
crypto_free_aead(tfm);
if (req)
#ifdef NEWAEAD
aead_request_free(req);
#else
aead_givcrypt_free(req);
#endif
return ret;
}
--
Ciao
Stephan
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [v3 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-06-01 9:44 ` Stephan Mueller
@ 2015-06-01 9:52 ` Herbert Xu
2015-06-01 10:08 ` Stephan Mueller
0 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-06-01 9:52 UTC (permalink / raw)
To: Stephan Mueller
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
On Mon, Jun 01, 2015 at 11:44:22AM +0200, Stephan Mueller wrote:
>
> That code crashes with the following stacktrace:
>
> [ 2000.433502] BUG: unable to handle kernel NULL pointer dereference at
> (null)
This crash is totally different from the previous crash you sent.
This one is expected because you're calling givencrypt on an
algorithm that has already been converted.
Once converted they no longer have a givencrypt function and
calling crypto_aead_givencrypt on them will crash.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [v3 PATCH] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-06-01 9:52 ` Herbert Xu
@ 2015-06-01 10:08 ` Stephan Mueller
0 siblings, 0 replies; 25+ messages in thread
From: Stephan Mueller @ 2015-06-01 10:08 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Timothy McCaffrey, Tim Chen
Am Montag, 1. Juni 2015, 17:52:45 schrieb Herbert Xu:
Hi Herbert,
> On Mon, Jun 01, 2015 at 11:44:22AM +0200, Stephan Mueller wrote:
> > That code crashes with the following stacktrace:
> >
> > [ 2000.433502] BUG: unable to handle kernel NULL pointer dereference at
> > (null)
>
> This crash is totally different from the previous crash you sent.
> This one is expected because you're calling givencrypt on an
> algorithm that has already been converted.
>
> Once converted they no longer have a givencrypt function and
> calling crypto_aead_givencrypt on them will crash.
Thank you, that was the clarification I am looking for. Please disregard all
communication on the givcrypt API then.
>
> Cheers,
--
Ciao
Stephan
^ permalink raw reply [flat|nested] 25+ messages in thread
[parent not found: <E1YxyQL-0002Ha-OL@gondolin.me.apana.org.au>]
* Re: [PATCH 10/10] crypto: aesni - Convert rfc4106 to new AEAD interface
[not found] ` <E1YxyQL-0002Ha-OL@gondolin.me.apana.org.au>
@ 2015-05-28 21:23 ` Tim Chen
2015-05-29 0:29 ` Herbert Xu
0 siblings, 1 reply; 25+ messages in thread
From: Tim Chen @ 2015-05-28 21:23 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey
On Thu, 2015-05-28 at 22:04 +0800, Herbert Xu wrote:
> @@ -1011,55 +966,55 @@ static int __driver_rfc4106_encrypt(struct aead_request *req)
> *(iv+4+i) = req->iv[i];
> *((__be32 *)(iv+12)) = counter;
>
> - if ((sg_is_last(req->src)) && (sg_is_last(req->assoc))) {
> + if (sg_is_last(req->src) &&
> + req->src->offset + req->src->length <= PAGE_SIZE &&
> + sg_is_last(req->dst) &&
> + req->dst->offset + req->dst->length <= PAGE_SIZE) {
> one_entry_in_sg = 1;
> scatterwalk_start(&src_sg_walk, req->src);
> - scatterwalk_start(&assoc_sg_walk, req->assoc);
> - src = scatterwalk_map(&src_sg_walk);
> - assoc = scatterwalk_map(&assoc_sg_walk);
> + assoc = scatterwalk_map(&src_sg_walk);
> + src = assoc + req->assoclen;
> dst = src;
> if (unlikely(req->src != req->dst)) {
> scatterwalk_start(&dst_sg_walk, req->dst);
> - dst = scatterwalk_map(&dst_sg_walk);
> + dst = scatterwalk_map(&dst_sg_walk) + req->assoclen;
Herbert,
Quick question. When src and dst are different, the above code
leave space for the assoc data in dst. However, I don't think we copy
over the assoc data from src to dst.
I assume we don't have to do that, right? If so, we have not
initialized assoc in dst. This won't be a problem, will it?
Tim
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH 10/10] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-05-28 21:23 ` [PATCH 10/10] " Tim Chen
@ 2015-05-29 0:29 ` Herbert Xu
2015-05-29 17:32 ` Tim Chen
0 siblings, 1 reply; 25+ messages in thread
From: Herbert Xu @ 2015-05-29 0:29 UTC (permalink / raw)
To: Tim Chen
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey
On Thu, May 28, 2015 at 02:23:33PM -0700, Tim Chen wrote:
>
> Quick question. When src and dst are different, the above code
> leave space for the assoc data in dst. However, I don't think we copy
> over the assoc data from src to dst.
> I assume we don't have to do that, right? If so, we have not
> initialized assoc in dst. This won't be a problem, will it?
It shouldn't be an issue. If you look at the documentation for
aead_rquest_set_crypt, it says:
* The content of the AD in the destination buffer after processing
* will either be untouched, or it will contain a copy of the AD
* from the source buffer. In order to ensure that it always has
* a copy of the AD, the user must copy the AD over either before
* or after processing. Of course this is not relevant if the user
* is doing in-place processing where src == dst.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 10/10] crypto: aesni - Convert rfc4106 to new AEAD interface
2015-05-29 0:29 ` Herbert Xu
@ 2015-05-29 17:32 ` Tim Chen
0 siblings, 0 replies; 25+ messages in thread
From: Tim Chen @ 2015-05-29 17:32 UTC (permalink / raw)
To: Herbert Xu
Cc: Linux Crypto Mailing List, Steffen Klassert, Adrian Hoban,
Tadeusz Struk, Stephan Mueller, Timothy McCaffrey
On Fri, 2015-05-29 at 08:29 +0800, Herbert Xu wrote:
> On Thu, May 28, 2015 at 02:23:33PM -0700, Tim Chen wrote:
> >
> > Quick question. When src and dst are different, the above code
> > leave space for the assoc data in dst. However, I don't think we copy
> > over the assoc data from src to dst.
> > I assume we don't have to do that, right? If so, we have not
> > initialized assoc in dst. This won't be a problem, will it?
>
> It shouldn't be an issue. If you look at the documentation for
> aead_rquest_set_crypt, it says:
>
> * The content of the AD in the destination buffer after processing
> * will either be untouched, or it will contain a copy of the AD
> * from the source buffer. In order to ensure that it always has
> * a copy of the AD, the user must copy the AD over either before
> * or after processing. Of course this is not relevant if the user
> * is doing in-place processing where src == dst.
>
Great, then it is clearly a user's responsibility to make sure
that AD in dst is initialized.
Tim
^ permalink raw reply [flat|nested] 25+ messages in thread