* [PATCH v3 4/4] dt-bindings: Add DT bindings document for Broadcom SBA RAID driver
From: Anup Patel @ 2017-02-10 9:07 UTC (permalink / raw)
To: Vinod Koul, Rob Herring, Mark Rutland, Herbert Xu,
David S . Miller, Jassi Brar
Cc: Dan Williams, Ray Jui, Scott Branden, Jon Mason, Rob Rice,
bcm-kernel-feedback-list, dmaengine, devicetree, linux-arm-kernel,
linux-kernel, linux-crypto, linux-raid, Anup Patel
In-Reply-To: <1486717628-17580-1-git-send-email-anup.patel@broadcom.com>
This patch adds the DT bindings document for newly added Broadcom
SBA RAID driver.
Signed-off-by: Anup Patel <anup.patel@broadcom.com>
Reviewed-by: Ray Jui <ray.jui@broadcom.com>
Reviewed-by: Scott Branden <scott.branden@broadcom.com>
---
.../devicetree/bindings/dma/brcm,iproc-sba.txt | 29 ++++++++++++++++++++++
1 file changed, 29 insertions(+)
create mode 100644 Documentation/devicetree/bindings/dma/brcm,iproc-sba.txt
diff --git a/Documentation/devicetree/bindings/dma/brcm,iproc-sba.txt b/Documentation/devicetree/bindings/dma/brcm,iproc-sba.txt
new file mode 100644
index 0000000..092913a
--- /dev/null
+++ b/Documentation/devicetree/bindings/dma/brcm,iproc-sba.txt
@@ -0,0 +1,29 @@
+* Broadcom SBA RAID engine
+
+Required properties:
+- compatible: Should be one of the following
+ "brcm,iproc-sba"
+ "brcm,iproc-sba-v2"
+ The "brcm,iproc-sba" has support for only 6 PQ coefficients
+ The "brcm,iproc-sba-v2" has support for only 30 PQ coefficients
+- mboxes: List of phandle and mailbox channel specifiers
+
+Example:
+
+raid_mbox: mbox@67400000 {
+ ...
+ #mbox-cells = <3>;
+ ...
+};
+
+raid0 {
+ compatible = "brcm,iproc-sba-v2";
+ mboxes = <&raid_mbox 0 0x1 0xffff>,
+ <&raid_mbox 1 0x1 0xffff>,
+ <&raid_mbox 2 0x1 0xffff>,
+ <&raid_mbox 3 0x1 0xffff>,
+ <&raid_mbox 4 0x1 0xffff>,
+ <&raid_mbox 5 0x1 0xffff>,
+ <&raid_mbox 6 0x1 0xffff>,
+ <&raid_mbox 7 0x1 0xffff>;
+};
--
2.7.4
^ permalink raw reply related
* [PATCH 00/12] crypto: caam - fixes
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
This batch consists mostly of DMA API related fixes and simplifications.
Since no no arch calls:
dma_debug_add_bus(&platform_bus_type);
DMA API debugging does not have the chance to report leaks when modules
are removed.
I am not sure why dma_debug_add_bus() is not used for the platform bus,
however when I did that for testing purposes, I could notice quite a few
problems in caam driver.
Thanks,
Horia
Horia Geantă (11):
crypto: caam - don't include unneeded headers
crypto: caam - check return code of dma_set_mask_and_coherent()
crypto: caam - fix HW S/G in ablkcipher_giv_edesc_alloc()
crypto: caam - check sg_count() return value
crypto: caam - replace sg_count() with sg_nents_for_len()
crypto: caam - use dma_map_sg() return code
crypto: caam - don't dma_map key for hash algorithms
crypto: caam - fix DMA API leaks for multiple setkey() calls
crypto: caam - fix error path for ctx_dma mapping failure
crypto: caam - abstract ahash request double buffering
crypto: caam - fix state buffer DMA (un)mapping
Tudor Ambarus (1):
crypto: caam - fix JR IO mapping if one fails
drivers/crypto/caam/caamalg.c | 589 ++++++++++++++++-----------------------
drivers/crypto/caam/caamhash.c | 268 +++++++++---------
drivers/crypto/caam/ctrl.c | 33 ++-
drivers/crypto/caam/error.c | 2 -
drivers/crypto/caam/jr.c | 19 +-
drivers/crypto/caam/sg_sw_sec4.h | 11 -
6 files changed, 407 insertions(+), 515 deletions(-)
--
2.4.4
^ permalink raw reply
* [PATCH 01/12] crypto: caam - don't include unneeded headers
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
intern.h, jr.h are not needed in error.c
error.h is not needed in ctrl.c
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/ctrl.c | 1 -
drivers/crypto/caam/error.c | 2 --
2 files changed, 3 deletions(-)
diff --git a/drivers/crypto/caam/ctrl.c b/drivers/crypto/caam/ctrl.c
index 755109841cfd..8957ec952212 100644
--- a/drivers/crypto/caam/ctrl.c
+++ b/drivers/crypto/caam/ctrl.c
@@ -13,7 +13,6 @@
#include "intern.h"
#include "jr.h"
#include "desc_constr.h"
-#include "error.h"
#include "ctrl.h"
bool caam_little_end;
diff --git a/drivers/crypto/caam/error.c b/drivers/crypto/caam/error.c
index 79a0cc70717f..6f44ccb55c63 100644
--- a/drivers/crypto/caam/error.c
+++ b/drivers/crypto/caam/error.c
@@ -6,9 +6,7 @@
#include "compat.h"
#include "regs.h"
-#include "intern.h"
#include "desc.h"
-#include "jr.h"
#include "error.h"
static const struct {
--
2.4.4
^ permalink raw reply related
* [PATCH 02/12] crypto: caam - check return code of dma_set_mask_and_coherent()
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
Setting the dma mask could fail, thus make sure it succeeds
before going further.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/ctrl.c | 15 ++++++++++-----
drivers/crypto/caam/jr.c | 19 ++++++++++++++-----
2 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/drivers/crypto/caam/ctrl.c b/drivers/crypto/caam/ctrl.c
index 8957ec952212..f825e3765a4b 100644
--- a/drivers/crypto/caam/ctrl.c
+++ b/drivers/crypto/caam/ctrl.c
@@ -586,13 +586,18 @@ static int caam_probe(struct platform_device *pdev)
JRSTART_JR1_START | JRSTART_JR2_START |
JRSTART_JR3_START);
- if (sizeof(dma_addr_t) == sizeof(u64))
+ if (sizeof(dma_addr_t) == sizeof(u64)) {
if (of_device_is_compatible(nprop, "fsl,sec-v5.0"))
- dma_set_mask_and_coherent(dev, DMA_BIT_MASK(40));
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(40));
else
- dma_set_mask_and_coherent(dev, DMA_BIT_MASK(36));
- else
- dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(36));
+ } else {
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ }
+ if (ret) {
+ dev_err(dev, "dma_set_mask_and_coherent failed (%d)\n", ret);
+ goto iounmap_ctrl;
+ }
/*
* Detect and enable JobRs
diff --git a/drivers/crypto/caam/jr.c b/drivers/crypto/caam/jr.c
index c8604dfadbf5..27631000b9f8 100644
--- a/drivers/crypto/caam/jr.c
+++ b/drivers/crypto/caam/jr.c
@@ -498,13 +498,22 @@ static int caam_jr_probe(struct platform_device *pdev)
jrpriv->rregs = (struct caam_job_ring __iomem __force *)ctrl;
- if (sizeof(dma_addr_t) == sizeof(u64))
+ if (sizeof(dma_addr_t) == sizeof(u64)) {
if (of_device_is_compatible(nprop, "fsl,sec-v5.0-job-ring"))
- dma_set_mask_and_coherent(jrdev, DMA_BIT_MASK(40));
+ error = dma_set_mask_and_coherent(jrdev,
+ DMA_BIT_MASK(40));
else
- dma_set_mask_and_coherent(jrdev, DMA_BIT_MASK(36));
- else
- dma_set_mask_and_coherent(jrdev, DMA_BIT_MASK(32));
+ error = dma_set_mask_and_coherent(jrdev,
+ DMA_BIT_MASK(36));
+ } else {
+ error = dma_set_mask_and_coherent(jrdev, DMA_BIT_MASK(32));
+ }
+ if (error) {
+ dev_err(jrdev, "dma_set_mask_and_coherent failed (%d)\n",
+ error);
+ iounmap(ctrl);
+ return error;
+ }
/* Identify the interrupt */
jrpriv->irq = irq_of_parse_and_map(nprop, 0);
--
2.4.4
^ permalink raw reply related
* [PATCH 03/12] crypto: caam - fix JR IO mapping if one fails
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
From: Tudor Ambarus <tudor-dan.ambarus@nxp.com>
If one of the JRs failed at init, the next JR used
the failed JR's IO space. The patch fixes this bug.
Signed-off-by: Tudor Ambarus <tudor-dan.ambarus@nxp.com>
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/ctrl.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/drivers/crypto/caam/ctrl.c b/drivers/crypto/caam/ctrl.c
index f825e3765a4b..579f8263c479 100644
--- a/drivers/crypto/caam/ctrl.c
+++ b/drivers/crypto/caam/ctrl.c
@@ -308,10 +308,8 @@ static int caam_remove(struct platform_device *pdev)
ctrl = (struct caam_ctrl __iomem *)ctrlpriv->ctrl;
/* Remove platform devices for JobRs */
- for (ring = 0; ring < ctrlpriv->total_jobrs; ring++) {
- if (ctrlpriv->jrpdev[ring])
- of_device_unregister(ctrlpriv->jrpdev[ring]);
- }
+ for (ring = 0; ring < ctrlpriv->total_jobrs; ring++)
+ of_device_unregister(ctrlpriv->jrpdev[ring]);
/* De-initialize RNG state handles initialized by this driver. */
if (ctrlpriv->rng4_sh_init)
@@ -423,7 +421,7 @@ DEFINE_SIMPLE_ATTRIBUTE(caam_fops_u64_ro, caam_debugfs_u64_get, NULL, "%llu\n");
/* Probe routine for CAAM top (controller) level */
static int caam_probe(struct platform_device *pdev)
{
- int ret, ring, rspec, gen_sk, ent_delay = RTSDCTL_ENT_DLY_MIN;
+ int ret, ring, ridx, rspec, gen_sk, ent_delay = RTSDCTL_ENT_DLY_MIN;
u64 caam_id;
struct device *dev;
struct device_node *nprop, *np;
@@ -618,6 +616,7 @@ static int caam_probe(struct platform_device *pdev)
}
ring = 0;
+ ridx = 0;
ctrlpriv->total_jobrs = 0;
for_each_available_child_of_node(nprop, np)
if (of_device_is_compatible(np, "fsl,sec-v4.0-job-ring") ||
@@ -625,17 +624,19 @@ static int caam_probe(struct platform_device *pdev)
ctrlpriv->jrpdev[ring] =
of_platform_device_create(np, NULL, dev);
if (!ctrlpriv->jrpdev[ring]) {
- pr_warn("JR%d Platform device creation error\n",
- ring);
+ pr_warn("JR physical index %d: Platform device creation error\n",
+ ridx);
+ ridx++;
continue;
}
ctrlpriv->jr[ring] = (struct caam_job_ring __iomem __force *)
((__force uint8_t *)ctrl +
- (ring + JR_BLOCK_NUMBER) *
+ (ridx + JR_BLOCK_NUMBER) *
BLOCK_OFFSET
);
ctrlpriv->total_jobrs++;
ring++;
+ ridx++;
}
/* Check to see if QI present. If so, enable */
--
2.4.4
^ permalink raw reply related
* [PATCH 07/12] crypto: caam - use dma_map_sg() return code
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
dma_map_sg() might coalesce S/G entries, so use the number of S/G
entries returned by it instead of what sg_nents_for_len() initially
returns.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamalg.c | 133 ++++++++++++++++++++++--------------------
1 file changed, 71 insertions(+), 62 deletions(-)
diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c
index 14b7dc8d5dcb..71d09e896d48 100644
--- a/drivers/crypto/caam/caamalg.c
+++ b/drivers/crypto/caam/caamalg.c
@@ -1328,9 +1328,8 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- int src_nents, dst_nents = 0;
+ int src_nents, mapped_src_nents, dst_nents = 0, mapped_dst_nents = 0;
struct aead_edesc *edesc;
- int sgc;
int sec4_sg_index, sec4_sg_len, sec4_sg_bytes;
unsigned int authsize = ctx->authsize;
@@ -1365,60 +1364,62 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
}
}
- sec4_sg_len = src_nents > 1 ? src_nents : 0;
- sec4_sg_len += dst_nents > 1 ? dst_nents : 0;
- sec4_sg_bytes = sec4_sg_len * sizeof(struct sec4_sg_entry);
-
- /* allocate space for base edesc and hw desc commands, link tables */
- edesc = kzalloc(sizeof(*edesc) + desc_bytes + sec4_sg_bytes,
- GFP_DMA | flags);
- if (!edesc) {
- dev_err(jrdev, "could not allocate extended descriptor\n");
- return ERR_PTR(-ENOMEM);
- }
-
if (likely(req->src == req->dst)) {
- sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_BIDIRECTIONAL);
- if (unlikely(!sgc)) {
+ mapped_src_nents = dma_map_sg(jrdev, req->src, src_nents,
+ DMA_BIDIRECTIONAL);
+ if (unlikely(!mapped_src_nents)) {
dev_err(jrdev, "unable to map source\n");
- kfree(edesc);
return ERR_PTR(-ENOMEM);
}
} else {
/* Cover also the case of null (zero length) input data */
if (src_nents) {
- sgc = dma_map_sg(jrdev, req->src, src_nents,
- DMA_TO_DEVICE);
- if (unlikely(!sgc)) {
+ mapped_src_nents = dma_map_sg(jrdev, req->src,
+ src_nents, DMA_TO_DEVICE);
+ if (unlikely(!mapped_src_nents)) {
dev_err(jrdev, "unable to map source\n");
- kfree(edesc);
return ERR_PTR(-ENOMEM);
}
+ } else {
+ mapped_src_nents = 0;
}
- sgc = dma_map_sg(jrdev, req->dst, dst_nents, DMA_FROM_DEVICE);
- if (unlikely(!sgc)) {
+ mapped_dst_nents = dma_map_sg(jrdev, req->dst, dst_nents,
+ DMA_FROM_DEVICE);
+ if (unlikely(!mapped_dst_nents)) {
dev_err(jrdev, "unable to map destination\n");
dma_unmap_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
- kfree(edesc);
return ERR_PTR(-ENOMEM);
}
}
+ sec4_sg_len = mapped_src_nents > 1 ? mapped_src_nents : 0;
+ sec4_sg_len += mapped_dst_nents > 1 ? mapped_dst_nents : 0;
+ sec4_sg_bytes = sec4_sg_len * sizeof(struct sec4_sg_entry);
+
+ /* allocate space for base edesc and hw desc commands, link tables */
+ edesc = kzalloc(sizeof(*edesc) + desc_bytes + sec4_sg_bytes,
+ GFP_DMA | flags);
+ if (!edesc) {
+ caam_unmap(jrdev, req->src, req->dst, src_nents, dst_nents, 0,
+ 0, 0, 0);
+ return ERR_PTR(-ENOMEM);
+ }
+
edesc->src_nents = src_nents;
edesc->dst_nents = dst_nents;
edesc->sec4_sg = (void *)edesc + sizeof(struct aead_edesc) +
desc_bytes;
- *all_contig_ptr = !(src_nents > 1);
+ *all_contig_ptr = !(mapped_src_nents > 1);
sec4_sg_index = 0;
- if (src_nents > 1) {
- sg_to_sec4_sg_last(req->src, src_nents,
- edesc->sec4_sg + sec4_sg_index, 0);
- sec4_sg_index += src_nents;
+ if (mapped_src_nents > 1) {
+ sg_to_sec4_sg_last(req->src, mapped_src_nents,
+ edesc->sec4_sg + sec4_sg_index, 0);
+ sec4_sg_index += mapped_src_nents;
}
- if (dst_nents > 1) {
- sg_to_sec4_sg_last(req->dst, dst_nents,
+ if (mapped_dst_nents > 1) {
+ sg_to_sec4_sg_last(req->dst, mapped_dst_nents,
edesc->sec4_sg + sec4_sg_index, 0);
}
@@ -1616,13 +1617,12 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ?
GFP_KERNEL : GFP_ATOMIC;
- int src_nents, dst_nents = 0, sec4_sg_bytes;
+ int src_nents, mapped_src_nents, dst_nents = 0, mapped_dst_nents = 0;
struct ablkcipher_edesc *edesc;
dma_addr_t iv_dma = 0;
bool in_contig;
- int sgc;
int ivsize = crypto_ablkcipher_ivsize(ablkcipher);
- int dst_sg_idx, sec4_sg_ents;
+ int dst_sg_idx, sec4_sg_ents, sec4_sg_bytes;
src_nents = sg_nents_for_len(req->src, req->nbytes);
if (unlikely(src_nents < 0)) {
@@ -1641,20 +1641,23 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
}
if (likely(req->src == req->dst)) {
- sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_BIDIRECTIONAL);
- if (unlikely(!sgc)) {
+ mapped_src_nents = dma_map_sg(jrdev, req->src, src_nents,
+ DMA_BIDIRECTIONAL);
+ if (unlikely(!mapped_src_nents)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
} else {
- sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
- if (unlikely(!sgc)) {
+ mapped_src_nents = dma_map_sg(jrdev, req->src, src_nents,
+ DMA_TO_DEVICE);
+ if (unlikely(!mapped_src_nents)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
- sgc = dma_map_sg(jrdev, req->dst, dst_nents, DMA_FROM_DEVICE);
- if (unlikely(!sgc)) {
+ mapped_dst_nents = dma_map_sg(jrdev, req->dst, dst_nents,
+ DMA_FROM_DEVICE);
+ if (unlikely(!mapped_dst_nents)) {
dev_err(jrdev, "unable to map destination\n");
dma_unmap_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
return ERR_PTR(-ENOMEM);
@@ -1669,15 +1672,16 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
return ERR_PTR(-ENOMEM);
}
- if (src_nents == 1 && iv_dma + ivsize == sg_dma_address(req->src)) {
+ if (mapped_src_nents == 1 &&
+ iv_dma + ivsize == sg_dma_address(req->src)) {
in_contig = true;
sec4_sg_ents = 0;
} else {
in_contig = false;
- sec4_sg_ents = 1 + src_nents;
+ sec4_sg_ents = 1 + mapped_src_nents;
}
dst_sg_idx = sec4_sg_ents;
- sec4_sg_ents += dst_nents > 1 ? dst_nents : 0;
+ sec4_sg_ents += mapped_dst_nents > 1 ? mapped_dst_nents : 0;
sec4_sg_bytes = sec4_sg_ents * sizeof(struct sec4_sg_entry);
/* allocate space for base edesc and hw desc commands, link tables */
@@ -1698,13 +1702,13 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
if (!in_contig) {
dma_to_sec4_sg_one(edesc->sec4_sg, iv_dma, ivsize, 0);
- sg_to_sec4_sg_last(req->src, src_nents,
+ sg_to_sec4_sg_last(req->src, mapped_src_nents,
edesc->sec4_sg + 1, 0);
}
- if (dst_nents > 1) {
- sg_to_sec4_sg_last(req->dst, dst_nents,
- edesc->sec4_sg + dst_sg_idx, 0);
+ if (mapped_dst_nents > 1) {
+ sg_to_sec4_sg_last(req->dst, mapped_dst_nents,
+ edesc->sec4_sg + dst_sg_idx, 0);
}
edesc->sec4_sg_dma = dma_map_single(jrdev, edesc->sec4_sg,
@@ -1819,13 +1823,12 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ?
GFP_KERNEL : GFP_ATOMIC;
- int src_nents, dst_nents, sec4_sg_bytes;
+ int src_nents, mapped_src_nents, dst_nents, mapped_dst_nents;
struct ablkcipher_edesc *edesc;
dma_addr_t iv_dma = 0;
bool out_contig;
- int sgc;
int ivsize = crypto_ablkcipher_ivsize(ablkcipher);
- int dst_sg_idx, sec4_sg_ents;
+ int dst_sg_idx, sec4_sg_ents, sec4_sg_bytes;
src_nents = sg_nents_for_len(req->src, req->nbytes);
if (unlikely(src_nents < 0)) {
@@ -1835,16 +1838,19 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
}
if (likely(req->src == req->dst)) {
- sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_BIDIRECTIONAL);
- if (unlikely(!sgc)) {
+ mapped_src_nents = dma_map_sg(jrdev, req->src, src_nents,
+ DMA_BIDIRECTIONAL);
+ if (unlikely(!mapped_src_nents)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
dst_nents = src_nents;
+ mapped_dst_nents = src_nents;
} else {
- sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
- if (unlikely(!sgc)) {
+ mapped_src_nents = dma_map_sg(jrdev, req->src, src_nents,
+ DMA_TO_DEVICE);
+ if (unlikely(!mapped_src_nents)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
@@ -1856,8 +1862,9 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
return ERR_PTR(dst_nents);
}
- sgc = dma_map_sg(jrdev, req->dst, dst_nents, DMA_FROM_DEVICE);
- if (unlikely(!sgc)) {
+ mapped_dst_nents = dma_map_sg(jrdev, req->dst, dst_nents,
+ DMA_FROM_DEVICE);
+ if (unlikely(!mapped_dst_nents)) {
dev_err(jrdev, "unable to map destination\n");
dma_unmap_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
return ERR_PTR(-ENOMEM);
@@ -1876,13 +1883,14 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
return ERR_PTR(-ENOMEM);
}
- sec4_sg_ents = src_nents > 1 ? src_nents : 0;
+ sec4_sg_ents = mapped_src_nents > 1 ? mapped_src_nents : 0;
dst_sg_idx = sec4_sg_ents;
- if (dst_nents == 1 && iv_dma + ivsize == sg_dma_address(req->dst)) {
+ if (mapped_dst_nents == 1 &&
+ iv_dma + ivsize == sg_dma_address(req->dst)) {
out_contig = true;
} else {
out_contig = false;
- sec4_sg_ents += 1 + dst_nents;
+ sec4_sg_ents += 1 + mapped_dst_nents;
}
/* allocate space for base edesc and hw desc commands, link tables */
@@ -1902,13 +1910,14 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
edesc->sec4_sg = (void *)edesc + sizeof(struct ablkcipher_edesc) +
desc_bytes;
- if (src_nents > 1)
- sg_to_sec4_sg_last(req->src, src_nents, edesc->sec4_sg, 0);
+ if (mapped_src_nents > 1)
+ sg_to_sec4_sg_last(req->src, mapped_src_nents, edesc->sec4_sg,
+ 0);
if (!out_contig) {
dma_to_sec4_sg_one(edesc->sec4_sg + dst_sg_idx,
iv_dma, ivsize, 0);
- sg_to_sec4_sg_last(req->dst, dst_nents,
+ sg_to_sec4_sg_last(req->dst, mapped_dst_nents,
edesc->sec4_sg + dst_sg_idx + 1, 0);
}
--
2.4.4
^ permalink raw reply related
* [PATCH 05/12] crypto: caam - check sg_count() return value
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
sg_count() internally calls sg_nents_for_len(), which could fail
in case the required number of bytes is larger than the total
bytes in the S/G.
Thus, add checks to validate the input.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamalg.c | 44 +++++++++++++++++++++++++++++++++++++++++--
1 file changed, 42 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c
index 05d4690351b9..ed8a04412767 100644
--- a/drivers/crypto/caam/caamalg.c
+++ b/drivers/crypto/caam/caamalg.c
@@ -1335,13 +1335,31 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
if (unlikely(req->dst != req->src)) {
src_nents = sg_count(req->src, req->assoclen + req->cryptlen);
+ if (unlikely(src_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
+ req->assoclen + req->cryptlen);
+ return ERR_PTR(src_nents);
+ }
+
dst_nents = sg_count(req->dst,
req->assoclen + req->cryptlen +
(encrypt ? authsize : (-authsize)));
+ if (unlikely(dst_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in dst S/G\n",
+ req->assoclen + req->cryptlen +
+ (encrypt ? authsize : (-authsize)));
+ return ERR_PTR(dst_nents);
+ }
} else {
src_nents = sg_count(req->src,
req->assoclen + req->cryptlen +
(encrypt ? authsize : 0));
+ if (unlikely(src_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
+ req->assoclen + req->cryptlen +
+ (encrypt ? authsize : 0));
+ return ERR_PTR(src_nents);
+ }
}
/* Check if data are contiguous. */
@@ -1609,9 +1627,20 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
int sec4_sg_index;
src_nents = sg_count(req->src, req->nbytes);
+ if (unlikely(src_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
+ req->nbytes);
+ return ERR_PTR(src_nents);
+ }
- if (req->dst != req->src)
+ if (req->dst != req->src) {
dst_nents = sg_count(req->dst, req->nbytes);
+ if (unlikely(dst_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in dst S/G\n",
+ req->nbytes);
+ return ERR_PTR(dst_nents);
+ }
+ }
if (likely(req->src == req->dst)) {
sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
@@ -1807,6 +1836,11 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
int sec4_sg_index;
src_nents = sg_count(req->src, req->nbytes);
+ if (unlikely(src_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
+ req->nbytes);
+ return ERR_PTR(src_nents);
+ }
if (likely(req->src == req->dst)) {
sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
@@ -1826,6 +1860,12 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
}
dst_nents = sg_count(req->dst, req->nbytes);
+ if (unlikely(dst_nents < 0)) {
+ dev_err(jrdev, "Insufficient bytes (%d) in dst S/G\n",
+ req->nbytes);
+ return ERR_PTR(dst_nents);
+ }
+
sgc = dma_map_sg(jrdev, req->dst, dst_nents ? : 1,
DMA_FROM_DEVICE);
if (unlikely(!sgc)) {
@@ -1914,7 +1954,7 @@ static int ablkcipher_givencrypt(struct skcipher_givcrypt_request *creq)
struct crypto_ablkcipher *ablkcipher = crypto_ablkcipher_reqtfm(req);
struct caam_ctx *ctx = crypto_ablkcipher_ctx(ablkcipher);
struct device *jrdev = ctx->jrdev;
- bool iv_contig;
+ bool iv_contig = false;
u32 *desc;
int ret = 0;
--
2.4.4
^ permalink raw reply related
* [PATCH 08/12] crypto: caam - don't dma_map key for hash algorithms
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
Shared descriptors for hash algorithms are small enough
for (split) keys to be inlined in all cases.
Since driver already does this, all what's left is to remove
unused ctx->key_dma.
Fixes: 045e36780f115 ("crypto: caam - ahash hmac support")
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamhash.c | 18 +-----------------
1 file changed, 1 insertion(+), 17 deletions(-)
diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c
index e58639ea53b1..117bbd8c08d4 100644
--- a/drivers/crypto/caam/caamhash.c
+++ b/drivers/crypto/caam/caamhash.c
@@ -109,7 +109,6 @@ struct caam_hash_ctx {
dma_addr_t sh_desc_digest_dma;
struct device *jrdev;
u8 key[CAAM_MAX_HASH_KEY_SIZE];
- dma_addr_t key_dma;
int ctx_len;
struct alginfo adata;
};
@@ -420,7 +419,6 @@ static int ahash_setkey(struct crypto_ahash *ahash,
const u8 *key, unsigned int keylen)
{
struct caam_hash_ctx *ctx = crypto_ahash_ctx(ahash);
- struct device *jrdev = ctx->jrdev;
int blocksize = crypto_tfm_alg_blocksize(&ahash->base);
int digestsize = crypto_ahash_digestsize(ahash);
int ret;
@@ -448,28 +446,14 @@ static int ahash_setkey(struct crypto_ahash *ahash,
if (ret)
goto bad_free_key;
- ctx->key_dma = dma_map_single(jrdev, ctx->key, ctx->adata.keylen_pad,
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- ret = -ENOMEM;
- goto error_free_key;
- }
#ifdef DEBUG
print_hex_dump(KERN_ERR, "ctx.key@"__stringify(__LINE__)": ",
DUMP_PREFIX_ADDRESS, 16, 4, ctx->key,
ctx->adata.keylen_pad, 1);
#endif
- ret = ahash_set_sh_desc(ahash);
- if (ret) {
- dma_unmap_single(jrdev, ctx->key_dma, ctx->adata.keylen_pad,
- DMA_TO_DEVICE);
- }
-
- error_free_key:
kfree(hashed_key);
- return ret;
+ return ahash_set_sh_desc(ahash);
bad_free_key:
kfree(hashed_key);
crypto_ahash_set_flags(ahash, CRYPTO_TFM_RES_BAD_KEY_LEN);
--
2.4.4
^ permalink raw reply related
* [PATCH 12/12] crypto: caam - fix state buffer DMA (un)mapping
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
If we register the DMA API debug notification chain to
receive platform bus events:
dma_debug_add_bus(&platform_bus_type);
we start receiving warnings after a simple test like "modprobe caam_jr &&
modprobe caamhash && modprobe -r caamhash && modprobe -r caam_jr":
platform ffe301000.jr: DMA-API: device driver has pending DMA allocations while released from device [count=1938]
One of leaked entries details: [device address=0x0000000173fda090] [size=63 bytes] [mapped with DMA_TO_DEVICE] [mapped as single]
It turns out there are several issues with handling buf_dma (mapping of buffer
holding the previous chunk smaller than hash block size):
-detection of buf_dma mapping failure occurs too late, after a job descriptor
using that value has been submitted for execution
-dma mapping leak - unmapping is not performed in all places: for e.g.
in ahash_export or in most ahash_fin* callbacks (due to current back-to-back
implementation of buf_dma unmapping/mapping)
Fix these by:
-calling dma_mapping_error() on buf_dma right after the mapping and providing
an error code if needed
-unmapping buf_dma during the "job done" (ahash_done_*) callbacks
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamhash.c | 107 ++++++++++++++++++++---------------------
1 file changed, 52 insertions(+), 55 deletions(-)
diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c
index b37d555a80d0..da4f94eab3da 100644
--- a/drivers/crypto/caam/caamhash.c
+++ b/drivers/crypto/caam/caamhash.c
@@ -194,36 +194,27 @@ static inline dma_addr_t map_seq_out_ptr_result(u32 *desc, struct device *jrdev,
return dst_dma;
}
-/* Map current buffer in state and put it in link table */
-static inline dma_addr_t buf_map_to_sec4_sg(struct device *jrdev,
- struct sec4_sg_entry *sec4_sg,
- u8 *buf, int buflen)
+/* Map current buffer in state (if length > 0) and put it in link table */
+static inline int buf_map_to_sec4_sg(struct device *jrdev,
+ struct sec4_sg_entry *sec4_sg,
+ struct caam_hash_state *state)
{
- dma_addr_t buf_dma;
+ int buflen = *current_buflen(state);
- buf_dma = dma_map_single(jrdev, buf, buflen, DMA_TO_DEVICE);
- dma_to_sec4_sg_one(sec4_sg, buf_dma, buflen, 0);
+ if (!buflen)
+ return 0;
- return buf_dma;
-}
+ state->buf_dma = dma_map_single(jrdev, current_buf(state), buflen,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(jrdev, state->buf_dma)) {
+ dev_err(jrdev, "unable to map buf\n");
+ state->buf_dma = 0;
+ return -ENOMEM;
+ }
-/*
- * Only put buffer in link table if it contains data, which is possible,
- * since a buffer has previously been used, and needs to be unmapped,
- */
-static inline dma_addr_t
-try_buf_map_to_sec4_sg(struct device *jrdev, struct sec4_sg_entry *sec4_sg,
- u8 *buf, dma_addr_t buf_dma, int buflen,
- int last_buflen)
-{
- if (buf_dma && !dma_mapping_error(jrdev, buf_dma))
- dma_unmap_single(jrdev, buf_dma, last_buflen, DMA_TO_DEVICE);
- if (buflen)
- buf_dma = buf_map_to_sec4_sg(jrdev, sec4_sg, buf, buflen);
- else
- buf_dma = 0;
-
- return buf_dma;
+ dma_to_sec4_sg_one(sec4_sg, state->buf_dma, buflen, 0);
+
+ return 0;
}
/* Map state->caam_ctx, and add it to link table */
@@ -491,6 +482,8 @@ static inline void ahash_unmap(struct device *dev,
struct ahash_edesc *edesc,
struct ahash_request *req, int dst_len)
{
+ struct caam_hash_state *state = ahash_request_ctx(req);
+
if (edesc->src_nents)
dma_unmap_sg(dev, req->src, edesc->src_nents, DMA_TO_DEVICE);
if (edesc->dst_dma)
@@ -499,6 +492,12 @@ static inline void ahash_unmap(struct device *dev,
if (edesc->sec4_sg_bytes)
dma_unmap_single(dev, edesc->sec4_sg_dma,
edesc->sec4_sg_bytes, DMA_TO_DEVICE);
+
+ if (state->buf_dma) {
+ dma_unmap_single(dev, state->buf_dma, *current_buflen(state),
+ DMA_TO_DEVICE);
+ state->buf_dma = 0;
+ }
}
static inline void ahash_unmap_ctx(struct device *dev,
@@ -557,8 +556,8 @@ static void ahash_done_bi(struct device *jrdev, u32 *desc, u32 err,
struct ahash_edesc *edesc;
struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
struct caam_hash_ctx *ctx = crypto_ahash_ctx(ahash);
-#ifdef DEBUG
struct caam_hash_state *state = ahash_request_ctx(req);
+#ifdef DEBUG
int digestsize = crypto_ahash_digestsize(ahash);
dev_err(jrdev, "%s %d: err 0x%x\n", __func__, __LINE__, err);
@@ -569,6 +568,7 @@ static void ahash_done_bi(struct device *jrdev, u32 *desc, u32 err,
caam_jr_strstatus(jrdev, err);
ahash_unmap_ctx(jrdev, edesc, req, ctx->ctx_len, DMA_BIDIRECTIONAL);
+ switch_buf(state);
kfree(edesc);
#ifdef DEBUG
@@ -625,8 +625,8 @@ static void ahash_done_ctx_dst(struct device *jrdev, u32 *desc, u32 err,
struct ahash_edesc *edesc;
struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
struct caam_hash_ctx *ctx = crypto_ahash_ctx(ahash);
-#ifdef DEBUG
struct caam_hash_state *state = ahash_request_ctx(req);
+#ifdef DEBUG
int digestsize = crypto_ahash_digestsize(ahash);
dev_err(jrdev, "%s %d: err 0x%x\n", __func__, __LINE__, err);
@@ -637,6 +637,7 @@ static void ahash_done_ctx_dst(struct device *jrdev, u32 *desc, u32 err,
caam_jr_strstatus(jrdev, err);
ahash_unmap_ctx(jrdev, edesc, req, ctx->ctx_len, DMA_FROM_DEVICE);
+ switch_buf(state);
kfree(edesc);
#ifdef DEBUG
@@ -777,10 +778,9 @@ static int ahash_update_ctx(struct ahash_request *req)
if (ret)
goto unmap_ctx;
- state->buf_dma = try_buf_map_to_sec4_sg(jrdev,
- edesc->sec4_sg + 1,
- buf, state->buf_dma,
- *buflen, last_buflen);
+ ret = buf_map_to_sec4_sg(jrdev, edesc->sec4_sg + 1, state);
+ if (ret)
+ goto unmap_ctx;
if (mapped_nents) {
sg_to_sec4_sg_last(req->src, mapped_nents,
@@ -795,8 +795,6 @@ static int ahash_update_ctx(struct ahash_request *req)
cpu_to_caam32(SEC4_SG_LEN_FIN);
}
- switch_buf(state);
-
desc = edesc->hw_desc;
edesc->sec4_sg_dma = dma_map_single(jrdev, edesc->sec4_sg,
@@ -853,9 +851,7 @@ static int ahash_final_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = current_buf(state);
int buflen = *current_buflen(state);
- int last_buflen = *alt_buflen(state);
u32 *desc;
int sec4_sg_bytes, sec4_sg_src_index;
int digestsize = crypto_ahash_digestsize(ahash);
@@ -882,9 +878,10 @@ static int ahash_final_ctx(struct ahash_request *req)
if (ret)
goto unmap_ctx;
- state->buf_dma = try_buf_map_to_sec4_sg(jrdev, edesc->sec4_sg + 1,
- buf, state->buf_dma, buflen,
- last_buflen);
+ ret = buf_map_to_sec4_sg(jrdev, edesc->sec4_sg + 1, state);
+ if (ret)
+ goto unmap_ctx;
+
(edesc->sec4_sg + sec4_sg_src_index - 1)->len |=
cpu_to_caam32(SEC4_SG_LEN_FIN);
@@ -931,9 +928,7 @@ static int ahash_finup_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = current_buf(state);
int buflen = *current_buflen(state);
- int last_buflen = *alt_buflen(state);
u32 *desc;
int sec4_sg_src_index;
int src_nents, mapped_nents;
@@ -978,9 +973,9 @@ static int ahash_finup_ctx(struct ahash_request *req)
if (ret)
goto unmap_ctx;
- state->buf_dma = try_buf_map_to_sec4_sg(jrdev, edesc->sec4_sg + 1,
- buf, state->buf_dma, buflen,
- last_buflen);
+ ret = buf_map_to_sec4_sg(jrdev, edesc->sec4_sg + 1, state);
+ if (ret)
+ goto unmap_ctx;
ret = ahash_edesc_add_src(ctx, edesc, req, mapped_nents,
sec4_sg_src_index, ctx->ctx_len + buflen,
@@ -1016,6 +1011,7 @@ static int ahash_digest(struct ahash_request *req)
{
struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
struct caam_hash_ctx *ctx = crypto_ahash_ctx(ahash);
+ struct caam_hash_state *state = ahash_request_ctx(req);
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
@@ -1025,6 +1021,8 @@ static int ahash_digest(struct ahash_request *req)
struct ahash_edesc *edesc;
int ret;
+ state->buf_dma = 0;
+
src_nents = sg_nents_for_len(req->src, req->nbytes);
if (src_nents < 0) {
dev_err(jrdev, "Invalid number of src SG.\n");
@@ -1210,8 +1208,10 @@ static int ahash_update_no_ctx(struct ahash_request *req)
edesc->sec4_sg_bytes = sec4_sg_bytes;
edesc->dst_dma = 0;
- state->buf_dma = buf_map_to_sec4_sg(jrdev, edesc->sec4_sg,
- buf, *buflen);
+ ret = buf_map_to_sec4_sg(jrdev, edesc->sec4_sg, state);
+ if (ret)
+ goto unmap_ctx;
+
sg_to_sec4_sg_last(req->src, mapped_nents,
edesc->sec4_sg + 1, 0);
@@ -1221,8 +1221,6 @@ static int ahash_update_no_ctx(struct ahash_request *req)
*next_buflen, 0);
}
- switch_buf(state);
-
desc = edesc->hw_desc;
edesc->sec4_sg_dma = dma_map_single(jrdev, edesc->sec4_sg,
@@ -1284,9 +1282,7 @@ static int ahash_finup_no_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = current_buf(state);
int buflen = *current_buflen(state);
- int last_buflen = *alt_buflen(state);
u32 *desc;
int sec4_sg_bytes, sec4_sg_src_index, src_nents, mapped_nents;
int digestsize = crypto_ahash_digestsize(ahash);
@@ -1328,9 +1324,9 @@ static int ahash_finup_no_ctx(struct ahash_request *req)
edesc->src_nents = src_nents;
edesc->sec4_sg_bytes = sec4_sg_bytes;
- state->buf_dma = try_buf_map_to_sec4_sg(jrdev, edesc->sec4_sg, buf,
- state->buf_dma, buflen,
- last_buflen);
+ ret = buf_map_to_sec4_sg(jrdev, edesc->sec4_sg, state);
+ if (ret)
+ goto unmap;
ret = ahash_edesc_add_src(ctx, edesc, req, mapped_nents, 1, buflen,
req->nbytes);
@@ -1376,8 +1372,8 @@ static int ahash_update_first(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *next_buf = current_buf(state);
- int *next_buflen = current_buflen(state);
+ u8 *next_buf = alt_buf(state);
+ int *next_buflen = alt_buflen(state);
int to_hash;
u32 *desc;
int src_nents, mapped_nents;
@@ -1459,6 +1455,7 @@ static int ahash_update_first(struct ahash_request *req)
state->final = ahash_final_no_ctx;
scatterwalk_map_and_copy(next_buf, req->src, 0,
req->nbytes, 0);
+ switch_buf(state);
}
#ifdef DEBUG
print_hex_dump(KERN_ERR, "next buf@"__stringify(__LINE__)": ",
--
2.4.4
^ permalink raw reply related
* [PATCH 10/12] crypto: caam - fix error path for ctx_dma mapping failure
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
In case ctx_dma dma mapping fails, ahash_unmap_ctx() tries to
dma unmap an invalid address:
map_seq_out_ptr_ctx() / ctx_map_to_sec4_sg() -> goto unmap_ctx ->
-> ahash_unmap_ctx() -> dma unmap ctx_dma
There is also possible to reach ahash_unmap_ctx() with ctx_dma
uninitialzed or to try to unmap the same address twice.
Fix these by setting ctx_dma = 0 where needed:
-initialize ctx_dma in ahash_init()
-clear ctx_dma in case of mapping error (instead of holding
the error code returned by the dma map function)
-clear ctx_dma after each unmapping
Fixes: 32686d34f8fb6 ("crypto: caam - ensure that we clean up after an error")
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamhash.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c
index 2ad83a8dc0fe..6c6c005f417b 100644
--- a/drivers/crypto/caam/caamhash.c
+++ b/drivers/crypto/caam/caamhash.c
@@ -148,6 +148,7 @@ static inline int map_seq_out_ptr_ctx(u32 *desc, struct device *jrdev,
ctx_len, DMA_FROM_DEVICE);
if (dma_mapping_error(jrdev, state->ctx_dma)) {
dev_err(jrdev, "unable to map ctx\n");
+ state->ctx_dma = 0;
return -ENOMEM;
}
@@ -208,6 +209,7 @@ static inline int ctx_map_to_sec4_sg(u32 *desc, struct device *jrdev,
state->ctx_dma = dma_map_single(jrdev, state->caam_ctx, ctx_len, flag);
if (dma_mapping_error(jrdev, state->ctx_dma)) {
dev_err(jrdev, "unable to map ctx\n");
+ state->ctx_dma = 0;
return -ENOMEM;
}
@@ -482,8 +484,10 @@ static inline void ahash_unmap_ctx(struct device *dev,
struct caam_hash_ctx *ctx = crypto_ahash_ctx(ahash);
struct caam_hash_state *state = ahash_request_ctx(req);
- if (state->ctx_dma)
+ if (state->ctx_dma) {
dma_unmap_single(dev, state->ctx_dma, ctx->ctx_len, flag);
+ state->ctx_dma = 0;
+ }
ahash_unmap(dev, edesc, req, dst_len);
}
@@ -1463,6 +1467,7 @@ static int ahash_init(struct ahash_request *req)
state->finup = ahash_finup_first;
state->final = ahash_final_no_ctx;
+ state->ctx_dma = 0;
state->current_buf = 0;
state->buf_dma = 0;
state->buflen_0 = 0;
--
2.4.4
^ permalink raw reply related
* [PATCH 09/12] crypto: caam - fix DMA API leaks for multiple setkey() calls
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
setkey() callback may be invoked multiple times for the same tfm.
In this case, DMA API leaks are caused by shared descriptors
(and key for caamalg) being mapped several times and unmapped only once.
Fix this by performing mapping / unmapping only in crypto algorithm's
cra_init() / cra_exit() callbacks and sync_for_device in the setkey()
tfm callback.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamalg.c | 275 +++++++++++------------------------------
drivers/crypto/caam/caamhash.c | 79 +++++-------
2 files changed, 102 insertions(+), 252 deletions(-)
diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c
index 71d09e896d48..9bc80eb06934 100644
--- a/drivers/crypto/caam/caamalg.c
+++ b/drivers/crypto/caam/caamalg.c
@@ -134,15 +134,15 @@ struct caam_aead_alg {
* per-session context
*/
struct caam_ctx {
- struct device *jrdev;
u32 sh_desc_enc[DESC_MAX_USED_LEN];
u32 sh_desc_dec[DESC_MAX_USED_LEN];
u32 sh_desc_givenc[DESC_MAX_USED_LEN];
+ u8 key[CAAM_MAX_KEY_SIZE];
dma_addr_t sh_desc_enc_dma;
dma_addr_t sh_desc_dec_dma;
dma_addr_t sh_desc_givenc_dma;
- u8 key[CAAM_MAX_KEY_SIZE];
dma_addr_t key_dma;
+ struct device *jrdev;
struct alginfo adata;
struct alginfo cdata;
unsigned int authsize;
@@ -171,13 +171,8 @@ static int aead_null_set_sh_desc(struct crypto_aead *aead)
/* aead_encrypt shared descriptor */
desc = ctx->sh_desc_enc;
cnstr_shdsc_aead_null_encap(desc, &ctx->adata, ctx->authsize);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/*
* Job Descriptor and Shared Descriptors
@@ -194,13 +189,8 @@ static int aead_null_set_sh_desc(struct crypto_aead *aead)
/* aead_decrypt shared descriptor */
desc = ctx->sh_desc_dec;
cnstr_shdsc_aead_null_decap(desc, &ctx->adata, ctx->authsize);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
return 0;
}
@@ -278,13 +268,8 @@ static int aead_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_enc;
cnstr_shdsc_aead_encap(desc, &ctx->cdata, &ctx->adata, ctx->authsize,
is_rfc3686, nonce, ctx1_iv_off);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
skip_enc:
/*
@@ -315,13 +300,8 @@ static int aead_set_sh_desc(struct crypto_aead *aead)
cnstr_shdsc_aead_decap(desc, &ctx->cdata, &ctx->adata, ivsize,
ctx->authsize, alg->caam.geniv, is_rfc3686,
nonce, ctx1_iv_off);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
if (!alg->caam.geniv)
goto skip_givenc;
@@ -354,13 +334,8 @@ static int aead_set_sh_desc(struct crypto_aead *aead)
cnstr_shdsc_aead_givencap(desc, &ctx->cdata, &ctx->adata, ivsize,
ctx->authsize, is_rfc3686, nonce,
ctx1_iv_off);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
skip_givenc:
return 0;
@@ -403,13 +378,8 @@ static int gcm_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_enc;
cnstr_shdsc_gcm_encap(desc, &ctx->cdata, ctx->authsize);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/*
* Job Descriptor and Shared Descriptors
@@ -425,13 +395,8 @@ static int gcm_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_dec;
cnstr_shdsc_gcm_decap(desc, &ctx->cdata, ctx->authsize);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
return 0;
}
@@ -472,13 +437,8 @@ static int rfc4106_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_enc;
cnstr_shdsc_rfc4106_encap(desc, &ctx->cdata, ctx->authsize);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/*
* Job Descriptor and Shared Descriptors
@@ -494,13 +454,8 @@ static int rfc4106_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_dec;
cnstr_shdsc_rfc4106_decap(desc, &ctx->cdata, ctx->authsize);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
return 0;
}
@@ -542,13 +497,8 @@ static int rfc4543_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_enc;
cnstr_shdsc_rfc4543_encap(desc, &ctx->cdata, ctx->authsize);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/*
* Job Descriptor and Shared Descriptors
@@ -564,13 +514,8 @@ static int rfc4543_set_sh_desc(struct crypto_aead *aead)
desc = ctx->sh_desc_dec;
cnstr_shdsc_rfc4543_decap(desc, &ctx->cdata, ctx->authsize);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
return 0;
}
@@ -614,28 +559,15 @@ static int aead_setkey(struct crypto_aead *aead,
/* postpend encryption key to auth split key */
memcpy(ctx->key + ctx->adata.keylen_pad, keys.enckey, keys.enckeylen);
-
- ctx->key_dma = dma_map_single(jrdev, ctx->key, ctx->adata.keylen_pad +
- keys.enckeylen, DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->key_dma, ctx->adata.keylen_pad +
+ keys.enckeylen, DMA_TO_DEVICE);
#ifdef DEBUG
print_hex_dump(KERN_ERR, "ctx.key@"__stringify(__LINE__)": ",
DUMP_PREFIX_ADDRESS, 16, 4, ctx->key,
ctx->adata.keylen_pad + keys.enckeylen, 1);
#endif
-
ctx->cdata.keylen = keys.enckeylen;
-
- ret = aead_set_sh_desc(aead);
- if (ret) {
- dma_unmap_single(jrdev, ctx->key_dma, ctx->adata.keylen_pad +
- keys.enckeylen, DMA_TO_DEVICE);
- }
-
- return ret;
+ return aead_set_sh_desc(aead);
badkey:
crypto_aead_set_flags(aead, CRYPTO_TFM_RES_BAD_KEY_LEN);
return -EINVAL;
@@ -646,7 +578,6 @@ static int gcm_setkey(struct crypto_aead *aead,
{
struct caam_ctx *ctx = crypto_aead_ctx(aead);
struct device *jrdev = ctx->jrdev;
- int ret = 0;
#ifdef DEBUG
print_hex_dump(KERN_ERR, "key in @"__stringify(__LINE__)": ",
@@ -654,21 +585,10 @@ static int gcm_setkey(struct crypto_aead *aead,
#endif
memcpy(ctx->key, key, keylen);
- ctx->key_dma = dma_map_single(jrdev, ctx->key, keylen,
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->key_dma, keylen, DMA_TO_DEVICE);
ctx->cdata.keylen = keylen;
- ret = gcm_set_sh_desc(aead);
- if (ret) {
- dma_unmap_single(jrdev, ctx->key_dma, ctx->cdata.keylen,
- DMA_TO_DEVICE);
- }
-
- return ret;
+ return gcm_set_sh_desc(aead);
}
static int rfc4106_setkey(struct crypto_aead *aead,
@@ -676,7 +596,6 @@ static int rfc4106_setkey(struct crypto_aead *aead,
{
struct caam_ctx *ctx = crypto_aead_ctx(aead);
struct device *jrdev = ctx->jrdev;
- int ret = 0;
if (keylen < 4)
return -EINVAL;
@@ -693,21 +612,9 @@ static int rfc4106_setkey(struct crypto_aead *aead,
* in the nonce. Update the AES key length.
*/
ctx->cdata.keylen = keylen - 4;
-
- ctx->key_dma = dma_map_single(jrdev, ctx->key, ctx->cdata.keylen,
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
- }
-
- ret = rfc4106_set_sh_desc(aead);
- if (ret) {
- dma_unmap_single(jrdev, ctx->key_dma, ctx->cdata.keylen,
- DMA_TO_DEVICE);
- }
-
- return ret;
+ dma_sync_single_for_device(jrdev, ctx->key_dma, ctx->cdata.keylen,
+ DMA_TO_DEVICE);
+ return rfc4106_set_sh_desc(aead);
}
static int rfc4543_setkey(struct crypto_aead *aead,
@@ -715,7 +622,6 @@ static int rfc4543_setkey(struct crypto_aead *aead,
{
struct caam_ctx *ctx = crypto_aead_ctx(aead);
struct device *jrdev = ctx->jrdev;
- int ret = 0;
if (keylen < 4)
return -EINVAL;
@@ -732,21 +638,9 @@ static int rfc4543_setkey(struct crypto_aead *aead,
* in the nonce. Update the AES key length.
*/
ctx->cdata.keylen = keylen - 4;
-
- ctx->key_dma = dma_map_single(jrdev, ctx->key, ctx->cdata.keylen,
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
- }
-
- ret = rfc4543_set_sh_desc(aead);
- if (ret) {
- dma_unmap_single(jrdev, ctx->key_dma, ctx->cdata.keylen,
- DMA_TO_DEVICE);
- }
-
- return ret;
+ dma_sync_single_for_device(jrdev, ctx->key_dma, ctx->cdata.keylen,
+ DMA_TO_DEVICE);
+ return rfc4543_set_sh_desc(aead);
}
static int ablkcipher_setkey(struct crypto_ablkcipher *ablkcipher,
@@ -787,12 +681,7 @@ static int ablkcipher_setkey(struct crypto_ablkcipher *ablkcipher,
keylen -= CTR_RFC3686_NONCE_SIZE;
}
- ctx->key_dma = dma_map_single(jrdev, ctx->key, keylen,
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->key_dma, keylen, DMA_TO_DEVICE);
ctx->cdata.keylen = keylen;
ctx->cdata.key_virt = ctx->key;
ctx->cdata.key_inline = true;
@@ -801,37 +690,22 @@ static int ablkcipher_setkey(struct crypto_ablkcipher *ablkcipher,
desc = ctx->sh_desc_enc;
cnstr_shdsc_ablkcipher_encap(desc, &ctx->cdata, ivsize, is_rfc3686,
ctx1_iv_off);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/* ablkcipher_decrypt shared descriptor */
desc = ctx->sh_desc_dec;
cnstr_shdsc_ablkcipher_decap(desc, &ctx->cdata, ivsize, is_rfc3686,
ctx1_iv_off);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/* ablkcipher_givencrypt shared descriptor */
desc = ctx->sh_desc_givenc;
cnstr_shdsc_ablkcipher_givencap(desc, &ctx->cdata, ivsize, is_rfc3686,
ctx1_iv_off);
- ctx->sh_desc_givenc_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_givenc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_givenc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
return 0;
}
@@ -851,11 +725,7 @@ static int xts_ablkcipher_setkey(struct crypto_ablkcipher *ablkcipher,
}
memcpy(ctx->key, key, keylen);
- ctx->key_dma = dma_map_single(jrdev, ctx->key, keylen, DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->key_dma)) {
- dev_err(jrdev, "unable to map key i/o memory\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->key_dma, keylen, DMA_TO_DEVICE);
ctx->cdata.keylen = keylen;
ctx->cdata.key_virt = ctx->key;
ctx->cdata.key_inline = true;
@@ -863,24 +733,14 @@ static int xts_ablkcipher_setkey(struct crypto_ablkcipher *ablkcipher,
/* xts_ablkcipher_encrypt shared descriptor */
desc = ctx->sh_desc_enc;
cnstr_shdsc_xts_ablkcipher_encap(desc, &ctx->cdata);
- ctx->sh_desc_enc_dma = dma_map_single(jrdev, desc, desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_enc_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_enc_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
/* xts_ablkcipher_decrypt shared descriptor */
desc = ctx->sh_desc_dec;
cnstr_shdsc_xts_ablkcipher_decap(desc, &ctx->cdata);
- ctx->sh_desc_dec_dma = dma_map_single(jrdev, desc, desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_dec_dma)) {
- dma_unmap_single(jrdev, ctx->sh_desc_enc_dma,
- desc_bytes(ctx->sh_desc_enc), DMA_TO_DEVICE);
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_dec_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
return 0;
}
@@ -3391,12 +3251,31 @@ struct caam_crypto_alg {
static int caam_init_common(struct caam_ctx *ctx, struct caam_alg_entry *caam)
{
+ dma_addr_t dma_addr;
+
ctx->jrdev = caam_jr_alloc();
if (IS_ERR(ctx->jrdev)) {
pr_err("Job Ring Device allocation for transform failed\n");
return PTR_ERR(ctx->jrdev);
}
+ dma_addr = dma_map_single_attrs(ctx->jrdev, ctx->sh_desc_enc,
+ offsetof(struct caam_ctx,
+ sh_desc_enc_dma),
+ DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(ctx->jrdev, dma_addr)) {
+ dev_err(ctx->jrdev, "unable to map key, shared descriptors\n");
+ caam_jr_free(ctx->jrdev);
+ return -ENOMEM;
+ }
+
+ ctx->sh_desc_enc_dma = dma_addr;
+ ctx->sh_desc_dec_dma = dma_addr + offsetof(struct caam_ctx,
+ sh_desc_dec);
+ ctx->sh_desc_givenc_dma = dma_addr + offsetof(struct caam_ctx,
+ sh_desc_givenc);
+ ctx->key_dma = dma_addr + offsetof(struct caam_ctx, key);
+
/* copy descriptor header template value */
ctx->cdata.algtype = OP_TYPE_CLASS1_ALG | caam->class1_alg_type;
ctx->adata.algtype = OP_TYPE_CLASS2_ALG | caam->class2_alg_type;
@@ -3426,25 +3305,9 @@ static int caam_aead_init(struct crypto_aead *tfm)
static void caam_exit_common(struct caam_ctx *ctx)
{
- if (ctx->sh_desc_enc_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_enc_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_enc_dma,
- desc_bytes(ctx->sh_desc_enc), DMA_TO_DEVICE);
- if (ctx->sh_desc_dec_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_dec_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_dec_dma,
- desc_bytes(ctx->sh_desc_dec), DMA_TO_DEVICE);
- if (ctx->sh_desc_givenc_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_givenc_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_givenc_dma,
- desc_bytes(ctx->sh_desc_givenc),
- DMA_TO_DEVICE);
- if (ctx->key_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->key_dma))
- dma_unmap_single(ctx->jrdev, ctx->key_dma,
- ctx->cdata.keylen + ctx->adata.keylen_pad,
- DMA_TO_DEVICE);
-
+ dma_unmap_single_attrs(ctx->jrdev, ctx->sh_desc_enc_dma,
+ offsetof(struct caam_ctx, sh_desc_enc_dma),
+ DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
caam_jr_free(ctx->jrdev);
}
diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c
index 117bbd8c08d4..2ad83a8dc0fe 100644
--- a/drivers/crypto/caam/caamhash.c
+++ b/drivers/crypto/caam/caamhash.c
@@ -276,12 +276,8 @@ static int ahash_set_sh_desc(struct crypto_ahash *ahash)
/* ahash_update shared descriptor */
desc = ctx->sh_desc_update;
ahash_gen_sh_desc(desc, OP_ALG_AS_UPDATE, ctx->ctx_len, ctx, true);
- ctx->sh_desc_update_dma = dma_map_single(jrdev, desc, desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_update_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_update_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
#ifdef DEBUG
print_hex_dump(KERN_ERR,
"ahash update shdesc@"__stringify(__LINE__)": ",
@@ -291,13 +287,8 @@ static int ahash_set_sh_desc(struct crypto_ahash *ahash)
/* ahash_update_first shared descriptor */
desc = ctx->sh_desc_update_first;
ahash_gen_sh_desc(desc, OP_ALG_AS_INIT, ctx->ctx_len, ctx, false);
- ctx->sh_desc_update_first_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_update_first_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_update_first_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
#ifdef DEBUG
print_hex_dump(KERN_ERR,
"ahash update first shdesc@"__stringify(__LINE__)": ",
@@ -307,12 +298,8 @@ static int ahash_set_sh_desc(struct crypto_ahash *ahash)
/* ahash_final shared descriptor */
desc = ctx->sh_desc_fin;
ahash_gen_sh_desc(desc, OP_ALG_AS_FINALIZE, digestsize, ctx, true);
- ctx->sh_desc_fin_dma = dma_map_single(jrdev, desc, desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_fin_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_fin_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
#ifdef DEBUG
print_hex_dump(KERN_ERR, "ahash final shdesc@"__stringify(__LINE__)": ",
DUMP_PREFIX_ADDRESS, 16, 4, desc,
@@ -322,13 +309,8 @@ static int ahash_set_sh_desc(struct crypto_ahash *ahash)
/* ahash_digest shared descriptor */
desc = ctx->sh_desc_digest;
ahash_gen_sh_desc(desc, OP_ALG_AS_INITFINAL, digestsize, ctx, false);
- ctx->sh_desc_digest_dma = dma_map_single(jrdev, desc,
- desc_bytes(desc),
- DMA_TO_DEVICE);
- if (dma_mapping_error(jrdev, ctx->sh_desc_digest_dma)) {
- dev_err(jrdev, "unable to map shared descriptor\n");
- return -ENOMEM;
- }
+ dma_sync_single_for_device(jrdev, ctx->sh_desc_digest_dma,
+ desc_bytes(desc), DMA_TO_DEVICE);
#ifdef DEBUG
print_hex_dump(KERN_ERR,
"ahash digest shdesc@"__stringify(__LINE__)": ",
@@ -1716,6 +1698,7 @@ static int caam_hash_cra_init(struct crypto_tfm *tfm)
HASH_MSG_LEN + SHA256_DIGEST_SIZE,
HASH_MSG_LEN + 64,
HASH_MSG_LEN + SHA512_DIGEST_SIZE };
+ dma_addr_t dma_addr;
/*
* Get a Job ring from Job Ring driver to ensure in-order
@@ -1726,6 +1709,26 @@ static int caam_hash_cra_init(struct crypto_tfm *tfm)
pr_err("Job Ring Device allocation for transform failed\n");
return PTR_ERR(ctx->jrdev);
}
+
+ dma_addr = dma_map_single_attrs(ctx->jrdev, ctx->sh_desc_update,
+ offsetof(struct caam_hash_ctx,
+ sh_desc_update_dma),
+ DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(ctx->jrdev, dma_addr)) {
+ dev_err(ctx->jrdev, "unable to map shared descriptors\n");
+ caam_jr_free(ctx->jrdev);
+ return -ENOMEM;
+ }
+
+ ctx->sh_desc_update_dma = dma_addr;
+ ctx->sh_desc_update_first_dma = dma_addr +
+ offsetof(struct caam_hash_ctx,
+ sh_desc_update_first);
+ ctx->sh_desc_fin_dma = dma_addr + offsetof(struct caam_hash_ctx,
+ sh_desc_fin);
+ ctx->sh_desc_digest_dma = dma_addr + offsetof(struct caam_hash_ctx,
+ sh_desc_digest);
+
/* copy descriptor header template value */
ctx->adata.algtype = OP_TYPE_CLASS2_ALG | caam_hash->alg_type;
@@ -1742,26 +1745,10 @@ static void caam_hash_cra_exit(struct crypto_tfm *tfm)
{
struct caam_hash_ctx *ctx = crypto_tfm_ctx(tfm);
- if (ctx->sh_desc_update_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_update_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_update_dma,
- desc_bytes(ctx->sh_desc_update),
- DMA_TO_DEVICE);
- if (ctx->sh_desc_update_first_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_update_first_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_update_first_dma,
- desc_bytes(ctx->sh_desc_update_first),
- DMA_TO_DEVICE);
- if (ctx->sh_desc_fin_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_fin_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_fin_dma,
- desc_bytes(ctx->sh_desc_fin), DMA_TO_DEVICE);
- if (ctx->sh_desc_digest_dma &&
- !dma_mapping_error(ctx->jrdev, ctx->sh_desc_digest_dma))
- dma_unmap_single(ctx->jrdev, ctx->sh_desc_digest_dma,
- desc_bytes(ctx->sh_desc_digest),
- DMA_TO_DEVICE);
-
+ dma_unmap_single_attrs(ctx->jrdev, ctx->sh_desc_update_dma,
+ offsetof(struct caam_hash_ctx,
+ sh_desc_update_dma),
+ DMA_TO_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
caam_jr_free(ctx->jrdev);
}
--
2.4.4
^ permalink raw reply related
* [PATCH 04/12] crypto: caam - fix HW S/G in ablkcipher_giv_edesc_alloc()
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
HW S/G generation does not work properly when the following conditions
are met:
-src == dst
-src/dst is S/G
-IV is right before (contiguous with) the first src/dst S/G entry
since "iv_contig" is set to true (iv_contig is a misnomer here and
it actually refers to the whole output being contiguous)
Fix this by setting dst S/G nents equal to src S/G nents, instead of
leaving it set to init value (0).
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamalg.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c
index 662fe94cb2f8..05d4690351b9 100644
--- a/drivers/crypto/caam/caamalg.c
+++ b/drivers/crypto/caam/caamalg.c
@@ -1798,7 +1798,7 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ?
GFP_KERNEL : GFP_ATOMIC;
- int src_nents, dst_nents = 0, sec4_sg_bytes;
+ int src_nents, dst_nents, sec4_sg_bytes;
struct ablkcipher_edesc *edesc;
dma_addr_t iv_dma = 0;
bool iv_contig = false;
@@ -1808,9 +1808,6 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
src_nents = sg_count(req->src, req->nbytes);
- if (unlikely(req->dst != req->src))
- dst_nents = sg_count(req->dst, req->nbytes);
-
if (likely(req->src == req->dst)) {
sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
DMA_BIDIRECTIONAL);
@@ -1818,6 +1815,8 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
+
+ dst_nents = src_nents;
} else {
sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
DMA_TO_DEVICE);
@@ -1826,6 +1825,7 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
return ERR_PTR(-ENOMEM);
}
+ dst_nents = sg_count(req->dst, req->nbytes);
sgc = dma_map_sg(jrdev, req->dst, dst_nents ? : 1,
DMA_FROM_DEVICE);
if (unlikely(!sgc)) {
--
2.4.4
^ permalink raw reply related
* [PATCH 06/12] crypto: caam - replace sg_count() with sg_nents_for_len()
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
Replace internal sg_count() function and the convoluted logic
around it with the standard sg_nents_for_len() function.
src_nents, dst_nents now hold the number of SW S/G entries,
instead of the HW S/G table entries.
With this change, null (zero length) input data for AEAD case
needs to be handled in a visible way. req->src is no longer
(un)mapped, pointer address is set to 0 in SEQ IN PTR command.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamalg.c | 189 ++++++++++++++++++---------------------
drivers/crypto/caam/sg_sw_sec4.h | 11 ---
2 files changed, 88 insertions(+), 112 deletions(-)
diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c
index ed8a04412767..14b7dc8d5dcb 100644
--- a/drivers/crypto/caam/caamalg.c
+++ b/drivers/crypto/caam/caamalg.c
@@ -887,8 +887,8 @@ static int xts_ablkcipher_setkey(struct crypto_ablkcipher *ablkcipher,
/*
* aead_edesc - s/w-extended aead descriptor
- * @src_nents: number of segments in input scatterlist
- * @dst_nents: number of segments in output scatterlist
+ * @src_nents: number of segments in input s/w scatterlist
+ * @dst_nents: number of segments in output s/w scatterlist
* @sec4_sg_bytes: length of dma mapped sec4_sg space
* @sec4_sg_dma: bus physical mapped address of h/w link table
* @sec4_sg: pointer to h/w link table
@@ -905,8 +905,8 @@ struct aead_edesc {
/*
* ablkcipher_edesc - s/w-extended ablkcipher descriptor
- * @src_nents: number of segments in input scatterlist
- * @dst_nents: number of segments in output scatterlist
+ * @src_nents: number of segments in input s/w scatterlist
+ * @dst_nents: number of segments in output s/w scatterlist
* @iv_dma: dma address of iv for checking continuity and link table
* @sec4_sg_bytes: length of dma mapped sec4_sg space
* @sec4_sg_dma: bus physical mapped address of h/w link table
@@ -930,10 +930,11 @@ static void caam_unmap(struct device *dev, struct scatterlist *src,
int sec4_sg_bytes)
{
if (dst != src) {
- dma_unmap_sg(dev, src, src_nents ? : 1, DMA_TO_DEVICE);
- dma_unmap_sg(dev, dst, dst_nents ? : 1, DMA_FROM_DEVICE);
+ if (src_nents)
+ dma_unmap_sg(dev, src, src_nents, DMA_TO_DEVICE);
+ dma_unmap_sg(dev, dst, dst_nents, DMA_FROM_DEVICE);
} else {
- dma_unmap_sg(dev, src, src_nents ? : 1, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(dev, src, src_nents, DMA_BIDIRECTIONAL);
}
if (iv_dma)
@@ -1102,7 +1103,7 @@ static void init_aead_job(struct aead_request *req,
init_job_desc_shared(desc, ptr, len, HDR_SHARE_DEFER | HDR_REVERSE);
if (all_contig) {
- src_dma = sg_dma_address(req->src);
+ src_dma = edesc->src_nents ? sg_dma_address(req->src) : 0;
in_options = 0;
} else {
src_dma = edesc->sec4_sg_dma;
@@ -1117,7 +1118,7 @@ static void init_aead_job(struct aead_request *req,
out_options = in_options;
if (unlikely(req->src != req->dst)) {
- if (!edesc->dst_nents) {
+ if (edesc->dst_nents == 1) {
dst_dma = sg_dma_address(req->dst);
} else {
dst_dma = edesc->sec4_sg_dma +
@@ -1227,10 +1228,11 @@ static void init_ablkcipher_job(u32 *sh_desc, dma_addr_t ptr,
print_hex_dump(KERN_ERR, "presciv@"__stringify(__LINE__)": ",
DUMP_PREFIX_ADDRESS, 16, 4, req->info,
ivsize, 1);
- printk(KERN_ERR "asked=%d, nbytes%d\n", (int)edesc->src_nents ? 100 : req->nbytes, req->nbytes);
+ pr_err("asked=%d, nbytes%d\n",
+ (int)edesc->src_nents > 1 ? 100 : req->nbytes, req->nbytes);
dbg_dump_sg(KERN_ERR, "src @"__stringify(__LINE__)": ",
DUMP_PREFIX_ADDRESS, 16, 4, req->src,
- edesc->src_nents ? 100 : req->nbytes, 1);
+ edesc->src_nents > 1 ? 100 : req->nbytes, 1);
#endif
len = desc_len(sh_desc);
@@ -1247,7 +1249,7 @@ static void init_ablkcipher_job(u32 *sh_desc, dma_addr_t ptr,
append_seq_in_ptr(desc, src_dma, req->nbytes + ivsize, in_options);
if (likely(req->src == req->dst)) {
- if (!edesc->src_nents && iv_contig) {
+ if (edesc->src_nents == 1 && iv_contig) {
dst_dma = sg_dma_address(req->src);
} else {
dst_dma = edesc->sec4_sg_dma +
@@ -1255,7 +1257,7 @@ static void init_ablkcipher_job(u32 *sh_desc, dma_addr_t ptr,
out_options = LDST_SGF;
}
} else {
- if (!edesc->dst_nents) {
+ if (edesc->dst_nents == 1) {
dst_dma = sg_dma_address(req->dst);
} else {
dst_dma = edesc->sec4_sg_dma +
@@ -1287,13 +1289,13 @@ static void init_ablkcipher_giv_job(u32 *sh_desc, dma_addr_t ptr,
ivsize, 1);
dbg_dump_sg(KERN_ERR, "src @" __stringify(__LINE__) ": ",
DUMP_PREFIX_ADDRESS, 16, 4, req->src,
- edesc->src_nents ? 100 : req->nbytes, 1);
+ edesc->src_nents > 1 ? 100 : req->nbytes, 1);
#endif
len = desc_len(sh_desc);
init_job_desc_shared(desc, ptr, len, HDR_SHARE_DEFER | HDR_REVERSE);
- if (!edesc->src_nents) {
+ if (edesc->src_nents == 1) {
src_dma = sg_dma_address(req->src);
in_options = 0;
} else {
@@ -1329,21 +1331,22 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
int src_nents, dst_nents = 0;
struct aead_edesc *edesc;
int sgc;
- bool all_contig = true;
- int sec4_sg_index, sec4_sg_len = 0, sec4_sg_bytes;
+ int sec4_sg_index, sec4_sg_len, sec4_sg_bytes;
unsigned int authsize = ctx->authsize;
if (unlikely(req->dst != req->src)) {
- src_nents = sg_count(req->src, req->assoclen + req->cryptlen);
+ src_nents = sg_nents_for_len(req->src, req->assoclen +
+ req->cryptlen);
if (unlikely(src_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
req->assoclen + req->cryptlen);
return ERR_PTR(src_nents);
}
- dst_nents = sg_count(req->dst,
- req->assoclen + req->cryptlen +
- (encrypt ? authsize : (-authsize)));
+ dst_nents = sg_nents_for_len(req->dst, req->assoclen +
+ req->cryptlen +
+ (encrypt ? authsize :
+ (-authsize)));
if (unlikely(dst_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in dst S/G\n",
req->assoclen + req->cryptlen +
@@ -1351,9 +1354,9 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
return ERR_PTR(dst_nents);
}
} else {
- src_nents = sg_count(req->src,
- req->assoclen + req->cryptlen +
- (encrypt ? authsize : 0));
+ src_nents = sg_nents_for_len(req->src, req->assoclen +
+ req->cryptlen +
+ (encrypt ? authsize : 0));
if (unlikely(src_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
req->assoclen + req->cryptlen +
@@ -1362,13 +1365,8 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
}
}
- /* Check if data are contiguous. */
- all_contig = !src_nents;
- if (!all_contig)
- sec4_sg_len = src_nents;
-
- sec4_sg_len += dst_nents;
-
+ sec4_sg_len = src_nents > 1 ? src_nents : 0;
+ sec4_sg_len += dst_nents > 1 ? dst_nents : 0;
sec4_sg_bytes = sec4_sg_len * sizeof(struct sec4_sg_entry);
/* allocate space for base edesc and hw desc commands, link tables */
@@ -1380,28 +1378,28 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
}
if (likely(req->src == req->dst)) {
- sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
- DMA_BIDIRECTIONAL);
+ sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_BIDIRECTIONAL);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map source\n");
kfree(edesc);
return ERR_PTR(-ENOMEM);
}
} else {
- sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
- DMA_TO_DEVICE);
- if (unlikely(!sgc)) {
- dev_err(jrdev, "unable to map source\n");
- kfree(edesc);
- return ERR_PTR(-ENOMEM);
+ /* Cover also the case of null (zero length) input data */
+ if (src_nents) {
+ sgc = dma_map_sg(jrdev, req->src, src_nents,
+ DMA_TO_DEVICE);
+ if (unlikely(!sgc)) {
+ dev_err(jrdev, "unable to map source\n");
+ kfree(edesc);
+ return ERR_PTR(-ENOMEM);
+ }
}
- sgc = dma_map_sg(jrdev, req->dst, dst_nents ? : 1,
- DMA_FROM_DEVICE);
+ sgc = dma_map_sg(jrdev, req->dst, dst_nents, DMA_FROM_DEVICE);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map destination\n");
- dma_unmap_sg(jrdev, req->src, src_nents ? : 1,
- DMA_TO_DEVICE);
+ dma_unmap_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
kfree(edesc);
return ERR_PTR(-ENOMEM);
}
@@ -1411,15 +1409,15 @@ static struct aead_edesc *aead_edesc_alloc(struct aead_request *req,
edesc->dst_nents = dst_nents;
edesc->sec4_sg = (void *)edesc + sizeof(struct aead_edesc) +
desc_bytes;
- *all_contig_ptr = all_contig;
+ *all_contig_ptr = !(src_nents > 1);
sec4_sg_index = 0;
- if (!all_contig) {
+ if (src_nents > 1) {
sg_to_sec4_sg_last(req->src, src_nents,
edesc->sec4_sg + sec4_sg_index, 0);
sec4_sg_index += src_nents;
}
- if (dst_nents) {
+ if (dst_nents > 1) {
sg_to_sec4_sg_last(req->dst, dst_nents,
edesc->sec4_sg + sec4_sg_index, 0);
}
@@ -1621,12 +1619,12 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
int src_nents, dst_nents = 0, sec4_sg_bytes;
struct ablkcipher_edesc *edesc;
dma_addr_t iv_dma = 0;
- bool iv_contig = false;
+ bool in_contig;
int sgc;
int ivsize = crypto_ablkcipher_ivsize(ablkcipher);
- int sec4_sg_index;
+ int dst_sg_idx, sec4_sg_ents;
- src_nents = sg_count(req->src, req->nbytes);
+ src_nents = sg_nents_for_len(req->src, req->nbytes);
if (unlikely(src_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
req->nbytes);
@@ -1634,7 +1632,7 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
}
if (req->dst != req->src) {
- dst_nents = sg_count(req->dst, req->nbytes);
+ dst_nents = sg_nents_for_len(req->dst, req->nbytes);
if (unlikely(dst_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in dst S/G\n",
req->nbytes);
@@ -1643,26 +1641,22 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
}
if (likely(req->src == req->dst)) {
- sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
- DMA_BIDIRECTIONAL);
+ sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_BIDIRECTIONAL);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
} else {
- sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
- DMA_TO_DEVICE);
+ sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
- sgc = dma_map_sg(jrdev, req->dst, dst_nents ? : 1,
- DMA_FROM_DEVICE);
+ sgc = dma_map_sg(jrdev, req->dst, dst_nents, DMA_FROM_DEVICE);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map destination\n");
- dma_unmap_sg(jrdev, req->src, src_nents ? : 1,
- DMA_TO_DEVICE);
+ dma_unmap_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
return ERR_PTR(-ENOMEM);
}
}
@@ -1675,16 +1669,16 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
return ERR_PTR(-ENOMEM);
}
- /*
- * Check if iv can be contiguous with source and destination.
- * If so, include it. If not, create scatterlist.
- */
- if (!src_nents && iv_dma + ivsize == sg_dma_address(req->src))
- iv_contig = true;
- else
- src_nents = src_nents ? : 1;
- sec4_sg_bytes = ((iv_contig ? 0 : 1) + src_nents + dst_nents) *
- sizeof(struct sec4_sg_entry);
+ if (src_nents == 1 && iv_dma + ivsize == sg_dma_address(req->src)) {
+ in_contig = true;
+ sec4_sg_ents = 0;
+ } else {
+ in_contig = false;
+ sec4_sg_ents = 1 + src_nents;
+ }
+ dst_sg_idx = sec4_sg_ents;
+ sec4_sg_ents += dst_nents > 1 ? dst_nents : 0;
+ sec4_sg_bytes = sec4_sg_ents * sizeof(struct sec4_sg_entry);
/* allocate space for base edesc and hw desc commands, link tables */
edesc = kzalloc(sizeof(*edesc) + desc_bytes + sec4_sg_bytes,
@@ -1702,17 +1696,15 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
edesc->sec4_sg = (void *)edesc + sizeof(struct ablkcipher_edesc) +
desc_bytes;
- sec4_sg_index = 0;
- if (!iv_contig) {
+ if (!in_contig) {
dma_to_sec4_sg_one(edesc->sec4_sg, iv_dma, ivsize, 0);
sg_to_sec4_sg_last(req->src, src_nents,
edesc->sec4_sg + 1, 0);
- sec4_sg_index += 1 + src_nents;
}
- if (dst_nents) {
+ if (dst_nents > 1) {
sg_to_sec4_sg_last(req->dst, dst_nents,
- edesc->sec4_sg + sec4_sg_index, 0);
+ edesc->sec4_sg + dst_sg_idx, 0);
}
edesc->sec4_sg_dma = dma_map_single(jrdev, edesc->sec4_sg,
@@ -1733,7 +1725,7 @@ static struct ablkcipher_edesc *ablkcipher_edesc_alloc(struct ablkcipher_request
sec4_sg_bytes, 1);
#endif
- *iv_contig_out = iv_contig;
+ *iv_contig_out = in_contig;
return edesc;
}
@@ -1830,12 +1822,12 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
int src_nents, dst_nents, sec4_sg_bytes;
struct ablkcipher_edesc *edesc;
dma_addr_t iv_dma = 0;
- bool iv_contig = false;
+ bool out_contig;
int sgc;
int ivsize = crypto_ablkcipher_ivsize(ablkcipher);
- int sec4_sg_index;
+ int dst_sg_idx, sec4_sg_ents;
- src_nents = sg_count(req->src, req->nbytes);
+ src_nents = sg_nents_for_len(req->src, req->nbytes);
if (unlikely(src_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in src S/G\n",
req->nbytes);
@@ -1843,8 +1835,7 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
}
if (likely(req->src == req->dst)) {
- sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
- DMA_BIDIRECTIONAL);
+ sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_BIDIRECTIONAL);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
@@ -1852,26 +1843,23 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
dst_nents = src_nents;
} else {
- sgc = dma_map_sg(jrdev, req->src, src_nents ? : 1,
- DMA_TO_DEVICE);
+ sgc = dma_map_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map source\n");
return ERR_PTR(-ENOMEM);
}
- dst_nents = sg_count(req->dst, req->nbytes);
+ dst_nents = sg_nents_for_len(req->dst, req->nbytes);
if (unlikely(dst_nents < 0)) {
dev_err(jrdev, "Insufficient bytes (%d) in dst S/G\n",
req->nbytes);
return ERR_PTR(dst_nents);
}
- sgc = dma_map_sg(jrdev, req->dst, dst_nents ? : 1,
- DMA_FROM_DEVICE);
+ sgc = dma_map_sg(jrdev, req->dst, dst_nents, DMA_FROM_DEVICE);
if (unlikely(!sgc)) {
dev_err(jrdev, "unable to map destination\n");
- dma_unmap_sg(jrdev, req->src, src_nents ? : 1,
- DMA_TO_DEVICE);
+ dma_unmap_sg(jrdev, req->src, src_nents, DMA_TO_DEVICE);
return ERR_PTR(-ENOMEM);
}
}
@@ -1888,14 +1876,17 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
return ERR_PTR(-ENOMEM);
}
- if (!dst_nents && iv_dma + ivsize == sg_dma_address(req->dst))
- iv_contig = true;
- else
- dst_nents = dst_nents ? : 1;
- sec4_sg_bytes = ((iv_contig ? 0 : 1) + src_nents + dst_nents) *
- sizeof(struct sec4_sg_entry);
+ sec4_sg_ents = src_nents > 1 ? src_nents : 0;
+ dst_sg_idx = sec4_sg_ents;
+ if (dst_nents == 1 && iv_dma + ivsize == sg_dma_address(req->dst)) {
+ out_contig = true;
+ } else {
+ out_contig = false;
+ sec4_sg_ents += 1 + dst_nents;
+ }
/* allocate space for base edesc and hw desc commands, link tables */
+ sec4_sg_bytes = sec4_sg_ents * sizeof(struct sec4_sg_entry);
edesc = kzalloc(sizeof(*edesc) + desc_bytes + sec4_sg_bytes,
GFP_DMA | flags);
if (!edesc) {
@@ -1911,18 +1902,14 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
edesc->sec4_sg = (void *)edesc + sizeof(struct ablkcipher_edesc) +
desc_bytes;
- sec4_sg_index = 0;
- if (src_nents) {
+ if (src_nents > 1)
sg_to_sec4_sg_last(req->src, src_nents, edesc->sec4_sg, 0);
- sec4_sg_index += src_nents;
- }
- if (!iv_contig) {
- dma_to_sec4_sg_one(edesc->sec4_sg + sec4_sg_index,
+ if (!out_contig) {
+ dma_to_sec4_sg_one(edesc->sec4_sg + dst_sg_idx,
iv_dma, ivsize, 0);
- sec4_sg_index += 1;
sg_to_sec4_sg_last(req->dst, dst_nents,
- edesc->sec4_sg + sec4_sg_index, 0);
+ edesc->sec4_sg + dst_sg_idx + 1, 0);
}
edesc->sec4_sg_dma = dma_map_single(jrdev, edesc->sec4_sg,
@@ -1943,7 +1930,7 @@ static struct ablkcipher_edesc *ablkcipher_giv_edesc_alloc(
sec4_sg_bytes, 1);
#endif
- *iv_contig_out = iv_contig;
+ *iv_contig_out = out_contig;
return edesc;
}
diff --git a/drivers/crypto/caam/sg_sw_sec4.h b/drivers/crypto/caam/sg_sw_sec4.h
index 6afa20c4a013..c6adad09c972 100644
--- a/drivers/crypto/caam/sg_sw_sec4.h
+++ b/drivers/crypto/caam/sg_sw_sec4.h
@@ -73,14 +73,3 @@ static inline struct sec4_sg_entry *sg_to_sec4_sg_len(
} while (total);
return sec4_sg_ptr - 1;
}
-
-/* derive number of elements in scatterlist, but return 0 for 1 */
-static inline int sg_count(struct scatterlist *sg_list, int nbytes)
-{
- int sg_nents = sg_nents_for_len(sg_list, nbytes);
-
- if (likely(sg_nents == 1))
- return 0;
-
- return sg_nents;
-}
--
2.4.4
^ permalink raw reply related
* [PATCH 11/12] crypto: caam - abstract ahash request double buffering
From: Horia Geantă @ 2017-02-10 12:07 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Dan Douglass, Tudor Ambarus, Cristian Stoica,
linux-crypto
In-Reply-To: <1486728445-13047-1-git-send-email-horia.geanta@nxp.com>
caamhash uses double buffering for holding previous/current
and next chunks (data smaller than block size) to be hashed.
Add (inline) functions to abstract this mechanism.
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
drivers/crypto/caam/caamhash.c | 77 ++++++++++++++++++++++++++----------------
1 file changed, 48 insertions(+), 29 deletions(-)
diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c
index 6c6c005f417b..b37d555a80d0 100644
--- a/drivers/crypto/caam/caamhash.c
+++ b/drivers/crypto/caam/caamhash.c
@@ -137,6 +137,31 @@ struct caam_export_state {
int (*finup)(struct ahash_request *req);
};
+static inline void switch_buf(struct caam_hash_state *state)
+{
+ state->current_buf ^= 1;
+}
+
+static inline u8 *current_buf(struct caam_hash_state *state)
+{
+ return state->current_buf ? state->buf_1 : state->buf_0;
+}
+
+static inline u8 *alt_buf(struct caam_hash_state *state)
+{
+ return state->current_buf ? state->buf_0 : state->buf_1;
+}
+
+static inline int *current_buflen(struct caam_hash_state *state)
+{
+ return state->current_buf ? &state->buflen_1 : &state->buflen_0;
+}
+
+static inline int *alt_buflen(struct caam_hash_state *state)
+{
+ return state->current_buf ? &state->buflen_0 : &state->buflen_1;
+}
+
/* Common job descriptor seq in/out ptr routines */
/* Map state->caam_ctx, and append seq_out_ptr command that points to it */
@@ -695,11 +720,10 @@ static int ahash_update_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = state->current_buf ? state->buf_1 : state->buf_0;
- int *buflen = state->current_buf ? &state->buflen_1 : &state->buflen_0;
- u8 *next_buf = state->current_buf ? state->buf_0 : state->buf_1;
- int *next_buflen = state->current_buf ? &state->buflen_0 :
- &state->buflen_1, last_buflen;
+ u8 *buf = current_buf(state);
+ int *buflen = current_buflen(state);
+ u8 *next_buf = alt_buf(state);
+ int *next_buflen = alt_buflen(state), last_buflen;
int in_len = *buflen + req->nbytes, to_hash;
u32 *desc;
int src_nents, mapped_nents, sec4_sg_bytes, sec4_sg_src_index;
@@ -771,7 +795,7 @@ static int ahash_update_ctx(struct ahash_request *req)
cpu_to_caam32(SEC4_SG_LEN_FIN);
}
- state->current_buf = !state->current_buf;
+ switch_buf(state);
desc = edesc->hw_desc;
@@ -829,10 +853,9 @@ static int ahash_final_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = state->current_buf ? state->buf_1 : state->buf_0;
- int buflen = state->current_buf ? state->buflen_1 : state->buflen_0;
- int last_buflen = state->current_buf ? state->buflen_0 :
- state->buflen_1;
+ u8 *buf = current_buf(state);
+ int buflen = *current_buflen(state);
+ int last_buflen = *alt_buflen(state);
u32 *desc;
int sec4_sg_bytes, sec4_sg_src_index;
int digestsize = crypto_ahash_digestsize(ahash);
@@ -908,10 +931,9 @@ static int ahash_finup_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = state->current_buf ? state->buf_1 : state->buf_0;
- int buflen = state->current_buf ? state->buflen_1 : state->buflen_0;
- int last_buflen = state->current_buf ? state->buflen_0 :
- state->buflen_1;
+ u8 *buf = current_buf(state);
+ int buflen = *current_buflen(state);
+ int last_buflen = *alt_buflen(state);
u32 *desc;
int sec4_sg_src_index;
int src_nents, mapped_nents;
@@ -1075,8 +1097,8 @@ static int ahash_final_no_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = state->current_buf ? state->buf_1 : state->buf_0;
- int buflen = state->current_buf ? state->buflen_1 : state->buflen_0;
+ u8 *buf = current_buf(state);
+ int buflen = *current_buflen(state);
u32 *desc;
int digestsize = crypto_ahash_digestsize(ahash);
struct ahash_edesc *edesc;
@@ -1136,11 +1158,10 @@ static int ahash_update_no_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = state->current_buf ? state->buf_1 : state->buf_0;
- int *buflen = state->current_buf ? &state->buflen_1 : &state->buflen_0;
- u8 *next_buf = state->current_buf ? state->buf_0 : state->buf_1;
- int *next_buflen = state->current_buf ? &state->buflen_0 :
- &state->buflen_1;
+ u8 *buf = current_buf(state);
+ int *buflen = current_buflen(state);
+ u8 *next_buf = alt_buf(state);
+ int *next_buflen = alt_buflen(state);
int in_len = *buflen + req->nbytes, to_hash;
int sec4_sg_bytes, src_nents, mapped_nents;
struct ahash_edesc *edesc;
@@ -1200,7 +1221,7 @@ static int ahash_update_no_ctx(struct ahash_request *req)
*next_buflen, 0);
}
- state->current_buf = !state->current_buf;
+ switch_buf(state);
desc = edesc->hw_desc;
@@ -1263,10 +1284,9 @@ static int ahash_finup_no_ctx(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *buf = state->current_buf ? state->buf_1 : state->buf_0;
- int buflen = state->current_buf ? state->buflen_1 : state->buflen_0;
- int last_buflen = state->current_buf ? state->buflen_0 :
- state->buflen_1;
+ u8 *buf = current_buf(state);
+ int buflen = *current_buflen(state);
+ int last_buflen = *alt_buflen(state);
u32 *desc;
int sec4_sg_bytes, sec4_sg_src_index, src_nents, mapped_nents;
int digestsize = crypto_ahash_digestsize(ahash);
@@ -1356,9 +1376,8 @@ static int ahash_update_first(struct ahash_request *req)
struct device *jrdev = ctx->jrdev;
gfp_t flags = (req->base.flags & (CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP)) ? GFP_KERNEL : GFP_ATOMIC;
- u8 *next_buf = state->current_buf ? state->buf_1 : state->buf_0;
- int *next_buflen = state->current_buf ?
- &state->buflen_1 : &state->buflen_0;
+ u8 *next_buf = current_buf(state);
+ int *next_buflen = current_buflen(state);
int to_hash;
u32 *desc;
int src_nents, mapped_nents;
--
2.4.4
^ permalink raw reply related
* Re: [PATCH v5] KEYS: add SP800-56A KDF support for DH
From: Stephan Müller @ 2017-02-10 13:18 UTC (permalink / raw)
To: dhowells; +Cc: herbert, mathew.j.martineau, linux-crypto, keyrings
In-Reply-To: <1571629.ErTDR5PMQO@positron.chronox.de>
Am Freitag, 19. August 2016, 20:39:09 CET schrieb Stephan Mueller:
Hi David,
> Hi,
>
> This patch now folds the KDF into the keys support as requested by
> Herbert. The caller can only supply the hash name used for the KDF.
>
> Note, the KDF implementation is identical to the kdf_ctr() support in
> the now unneeded KDF patches to the kernel crypto API.
>
> The new patch also changes the variable name from kdfname to hashname.
>
> Also, the patch adds a missing semicolon.
>
> Finally, the patch adds a guard against compiling the compat code
> if the general Linux kernel configuration does not have the compat
> code enabled. Without that guard, compilation warnings are seen.
May I ask which plans do you have with the KDF support for DH?
Ciao
Stephan
^ permalink raw reply
* Re: [PATCH v3 3/4] dmaengine: Add Broadcom SBA RAID driver
From: Dan Williams @ 2017-02-10 17:50 UTC (permalink / raw)
To: Anup Patel
Cc: Mark Rutland, Device Tree, Herbert Xu, Scott Branden, Vinod Koul,
Ray Jui, Jassi Brar, linux-kernel@vger.kernel.org, linux-raid,
Jon Mason, Rob Herring, BCM Kernel Feedback, linux-crypto,
Rob Rice, dmaengine@vger.kernel.org, David S . Miller,
linux-arm-kernel@lists.infradead.org
In-Reply-To: <1486717628-17580-4-git-send-email-anup.patel@broadcom.com>
On Fri, Feb 10, 2017 at 1:07 AM, Anup Patel <anup.patel@broadcom.com> wrote:
> The Broadcom stream buffer accelerator (SBA) provides offloading
> capabilities for RAID operations. This SBA offload engine is
> accessible via Broadcom SoC specific ring manager.
>
> This patch adds Broadcom SBA RAID driver which provides one
> DMA device with RAID capabilities using one or more Broadcom
> SoC specific ring manager channels. The SBA RAID driver in its
> current shape implements memcpy, xor, and pq operations.
>
> Signed-off-by: Anup Patel <anup.patel@broadcom.com>
> Reviewed-by: Ray Jui <ray.jui@broadcom.com>
> ---
> drivers/dma/Kconfig | 13 +
> drivers/dma/Makefile | 1 +
> drivers/dma/bcm-sba-raid.c | 1711 ++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 1725 insertions(+)
> create mode 100644 drivers/dma/bcm-sba-raid.c
>
> diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
> index 263495d..bf8fb84 100644
> --- a/drivers/dma/Kconfig
> +++ b/drivers/dma/Kconfig
> @@ -99,6 +99,19 @@ config AXI_DMAC
> controller is often used in Analog Device's reference designs for FPGA
> platforms.
>
> +config BCM_SBA_RAID
> + tristate "Broadcom SBA RAID engine support"
> + depends on (ARM64 && MAILBOX && RAID6_PQ) || COMPILE_TEST
> + select DMA_ENGINE
> + select DMA_ENGINE_RAID
> + select ASYNC_TX_ENABLE_CHANNEL_SWITCH
ASYNC_TX_ENABLE_CHANNEL_SWITCH violates the DMA mapping API and
Russell has warned it's especially problematic on ARM [1]. If you
need channel switching for this offload engine to be useful then you
need to move DMA mapping and channel switching responsibilities to MD
itself.
[1]: http://lists.infradead.org/pipermail/linux-arm-kernel/2011-January/036753.html
[..]
> diff --git a/drivers/dma/bcm-sba-raid.c b/drivers/dma/bcm-sba-raid.c
> new file mode 100644
> index 0000000..bab9918
> --- /dev/null
> +++ b/drivers/dma/bcm-sba-raid.c
> @@ -0,0 +1,1711 @@
> +/*
> + * Copyright (C) 2017 Broadcom
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +/*
> + * Broadcom SBA RAID Driver
> + *
> + * The Broadcom stream buffer accelerator (SBA) provides offloading
> + * capabilities for RAID operations. The SBA offload engine is accessible
> + * via Broadcom SoC specific ring manager. Two or more offload engines
> + * can share same Broadcom SoC specific ring manager due to this Broadcom
> + * SoC specific ring manager driver is implemented as a mailbox controller
> + * driver and offload engine drivers are implemented as mallbox clients.
> + *
> + * Typically, Broadcom SoC specific ring manager will implement larger
> + * number of hardware rings over one or more SBA hardware devices. By
> + * design, the internal buffer size of SBA hardware device is limited
> + * but all offload operations supported by SBA can be broken down into
> + * multiple small size requests and executed parallely on multiple SBA
> + * hardware devices for achieving high through-put.
> + *
> + * The Broadcom SBA RAID driver does not require any register programming
> + * except submitting request to SBA hardware device via mailbox channels.
> + * This driver implements a DMA device with one DMA channel using a set
> + * of mailbox channels provided by Broadcom SoC specific ring manager
> + * driver. To exploit parallelism (as described above), all DMA request
> + * coming to SBA RAID DMA channel are broken down to smaller requests
> + * and submitted to multiple mailbox channels in round-robin fashion.
> + * For having more SBA DMA channels, we can create more SBA device nodes
> + * in Broadcom SoC specific DTS based on number of hardware rings supported
> + * by Broadcom SoC ring manager.
> + */
> +
> +#include <linux/bitops.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/dmaengine.h>
> +#include <linux/list.h>
> +#include <linux/mailbox_client.h>
> +#include <linux/mailbox/brcm-message.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/slab.h>
> +#include <linux/raid/pq.h>
> +
> +#include "dmaengine.h"
> +
> +/* SBA command helper macros */
> +#define SBA_DEC(_d, _s, _m) (((_d) >> (_s)) & (_m))
> +#define SBA_ENC(_d, _v, _s, _m) \
> + do { \
> + (_d) &= ~((u64)(_m) << (_s)); \
> + (_d) |= (((u64)(_v) & (_m)) << (_s)); \
> + } while (0)
Reusing a macro argument multiple times is problematic, consider
SBA_ENC(..., arg++, ...), and hiding assignments in a macro make this
hard to read. The compiler should inline it properly if you just make
this a function that returns a value. You could also mark it __pure.
[..]
> +
> +static struct sba_request *sba_alloc_request(struct sba_device *sba)
> +{
> + unsigned long flags;
> + struct sba_request *req = NULL;
> +
> + spin_lock_irqsave(&sba->reqs_lock, flags);
> +
> + if (!list_empty(&sba->reqs_free_list)) {
> + req = list_first_entry(&sba->reqs_free_list,
> + struct sba_request,
> + node);
You could use list_first_entry_or_null() here.
[..]
> +
> +/* Note: Must be called with sba->reqs_lock held */
> +static void _sba_pending_request(struct sba_device *sba,
> + struct sba_request *req)
> +{
You can validate the locking assumptions here with
lockdep_assert_head(sba->reqs_lock).
[..]
> +
> +static void sba_cleanup_nonpending_requests(struct sba_device *sba)
> +{
> + unsigned long flags;
> + struct sba_request *req, *req1;
> +
> + spin_lock_irqsave(&sba->reqs_lock, flags);
> +
> + /* Freeup all alloced request */
> + list_for_each_entry_safe(req, req1, &sba->reqs_alloc_list, node) {
> + _sba_free_request(sba, req);
> + }
> +
> + /* Freeup all received request */
> + list_for_each_entry_safe(req, req1, &sba->reqs_received_list, node) {
> + _sba_free_request(sba, req);
> + }
> +
> + /* Freeup all completed request */
> + list_for_each_entry_safe(req, req1, &sba->reqs_completed_list, node) {
> + _sba_free_request(sba, req);
> + }
> +
> + /* Set all active requests as aborted */
> + list_for_each_entry_safe(req, req1, &sba->reqs_active_list, node) {
> + _sba_abort_request(sba, req);
> + }
In some parts of the driver you leave off unneeded braces like the for
loop in sba_prep_dma_pq(), and in some case you include them. I'd say
remove them if they're not necessary, but either way make it
consistent across the driver.
[..]
> +
> +static struct dma_async_tx_descriptor *
> +sba_prep_dma_pq(struct dma_chan *dchan, dma_addr_t *dst, dma_addr_t *src,
> + u32 src_cnt, const u8 *scf, size_t len, unsigned long flags)
> +{
> + u32 i, dst_q_index;
> + size_t req_len;
> + bool slow = false;
> + dma_addr_t off = 0;
> + dma_addr_t *dst_p = NULL, *dst_q = NULL;
> + struct sba_device *sba = to_sba_device(dchan);
> + struct sba_request *first = NULL, *req;
> +
> + /* Sanity checks */
> + if (unlikely(src_cnt > sba->max_pq_srcs))
> + return NULL;
> + for (i = 0; i < src_cnt; i++)
> + if (sba->max_pq_coefs <= raid6_gflog[scf[i]])
> + slow = true;
Thanks, yes, I do think this is cleaner here than in async_tx itself.
[..]
> +static void sba_receive_message(struct mbox_client *cl, void *msg)
> +{
> + unsigned long flags;
> + struct brcm_message *m = msg;
> + struct sba_request *req = m->ctx, *req1;
> + struct sba_device *sba = req->sba;
> +
> + /* Error count if message has error */
> + if (m->error < 0) {
> + dev_err(sba->dev, "%s got message with error %d",
> + dma_chan_name(&sba->dma_chan), m->error);
> + }
> +
> + /* Mark request as received */
> + sba_received_request(req);
> +
> + /* Wait for all chained requests to be completed */
> + if (atomic_dec_return(&req->first->next_pending_count))
> + goto done;
> +
> + /* Point to first request */
> + req = req->first;
> +
> + /* Update request */
> + if (req->state == SBA_REQUEST_STATE_RECEIVED)
> + sba_dma_tx_actions(req);
> + else
> + sba_free_chained_requests(req);
> +
> + spin_lock_irqsave(&sba->reqs_lock, flags);
> +
> + /* Re-check all completed request waiting for 'ack' */
> + list_for_each_entry_safe(req, req1, &sba->reqs_completed_list, node) {
> + spin_unlock_irqrestore(&sba->reqs_lock, flags);
> + sba_dma_tx_actions(req);
You've now required all callback paths to be hardirq safe whereas
previously the callbacks only assumed softirq exclusion. Have you run
this with CONFIG_PROVE_LOCKING enabled?
^ permalink raw reply
* Re: [PATCH] crypto: arm/aes-ce: assign err return conditionally
From: Herbert Xu @ 2017-02-11 9:39 UTC (permalink / raw)
To: Nicholas Mc Guire
Cc: David S. Miller, Russell King, linux-crypto, linux-arm-kernel,
linux-kernel
In-Reply-To: <1486560979-25240-1-git-send-email-der.herr@hofr.at>
On Wed, Feb 08, 2017 at 02:36:19PM +0100, Nicholas Mc Guire wrote:
> As the err value is not used unless there was an error it can be assigned
> conditionally here.
>
> Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at>
This patch just adds noise to the commit history. So I'm not going
to apply it.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] crypto: arm/aes-ce: assign err return conditionally
From: Nicholas Mc Guire @ 2017-02-11 10:15 UTC (permalink / raw)
To: Herbert Xu
Cc: David S. Miller, Russell King, linux-crypto, linux-arm-kernel,
linux-kernel
In-Reply-To: <20170211093946.GA749@gondor.apana.org.au>
On Sat, Feb 11, 2017 at 05:39:46PM +0800, Herbert Xu wrote:
> On Wed, Feb 08, 2017 at 02:36:19PM +0100, Nicholas Mc Guire wrote:
> > As the err value is not used unless there was an error it can be assigned
> > conditionally here.
> >
> > Signed-off-by: Nicholas Mc Guire <der.herr@hofr.at>
>
> This patch just adds noise to the commit history. So I'm not going
> to apply it.
ok - I was not sure about how sensible this sort of changes are
either - will drop those code-to-object refactoring experiments
then again.
thx!
hofrat
^ permalink raw reply
* Re: [PATCH v4 0/3] Add Broadcom SPU Crypto Driver
From: Herbert Xu @ 2017-02-11 10:54 UTC (permalink / raw)
To: Rob Rice
Cc: David S. Miller, Rob Herring, Mark Rutland, linux-crypto,
devicetree, linux-kernel, bcm-kernel-feedback-list,
Catalin Marinas, Will Deacon, linux-arm-kernel
In-Reply-To: <1486144534-44887-1-git-send-email-rob.rice@broadcom.com>
On Fri, Feb 03, 2017 at 12:55:31PM -0500, Rob Rice wrote:
> Changes in v4:
> - Added Rob Herring's Acked-by to patch 1/3 for bindings doc
> - In response to Herbert's comment, in ahash_export() and
> ahash_import(), only copy the hash state, not state params
> related to cipher or aead algos.
> - Noticed that hmac_offset in iproc_reqctx_s and spu_hash_params
> wasn't really used. So removed.
Patches 1-2 applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/2] crypto: atmel - refine Kconfig dependencies
From: Herbert Xu @ 2017-02-11 10:55 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Cyrille Pitchen, David S. Miller, linux-crypto, linux-kernel
In-Reply-To: <20170206123240.1698839-1-arnd@arndb.de>
On Mon, Feb 06, 2017 at 01:32:15PM +0100, Arnd Bergmann wrote:
> With the new authenc support, we get a harmless Kconfig warning:
>
> warning: (CRYPTO_DEV_ATMEL_AUTHENC) selects CRYPTO_DEV_ATMEL_SHA which has unmet direct dependencies (CRYPTO && CRYPTO_HW && ARCH_AT91)
>
> The problem is that each of the options has slightly different dependencies,
> although they all seem to want the same thing: allow building for real AT91
> targets that actually have the hardware, and possibly for compile testing.
>
> This makes all four options consistent: instead of depending on a particular
> dmaengine implementation, we depend on the ARM platform, CONFIG_COMPILE_TEST
> as an alternative when that is turned off. This makes the 'select' statements
> work correctly.
>
> Fixes: 89a82ef87e01 ("crypto: atmel-authenc - add support to authenc(hmac(shaX), Y(aes)) modes")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Both patches applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: linux-next: build warnings after merge of the crypto tree
From: Herbert Xu @ 2017-02-11 10:56 UTC (permalink / raw)
To: Stephen Rothwell
Cc: linux-next, linux-kernel, Cyrille Pitchen,
Linux Crypto Mailing List
In-Reply-To: <20170210141251.03b39b40@canb.auug.org.au>
On Fri, Feb 10, 2017 at 02:12:51PM +1100, Stephen Rothwell wrote:
>
> I am still getting these warnings ... I have seen no updates to the
> crypot tree since Feb 2.
Sorry Stephen. I have now applied Arnd's fixes for this problem
and it should be pushed out.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH] hwrng: cavium: Use per device name to allow for multiple devices.
From: Herbert Xu @ 2017-02-11 10:56 UTC (permalink / raw)
To: David Daney
Cc: linux-crypto, Matt Mackall, linux-kernel, Omer Khaliq,
Ananth Jasty
In-Reply-To: <20170206222846.22493-1-david.daney@cavium.com>
On Mon, Feb 06, 2017 at 02:28:46PM -0800, David Daney wrote:
> Systems containing the Cavium HW RNG may have one device per NUMA
> node. A typical configuration is a 2-node NUMA system, which results
> in 2 RNG devices. The hwrng subsystem refuses (and rightly so) to
> register more than one device with he same name, so we get failure
> messages on these systems.
>
> Make the hwrng name unique by including the underlying device name.
> Also remove spaces from the name to make it possible to switch devices
> via the sysfs knobs.
>
> Signed-off-by: David Daney <david.daney@cavium.com>
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v3 0/3] crypto: time invariant AES for CCM (and CMAC/XCBC)
From: Ard Biesheuvel @ 2017-02-11 18:05 UTC (permalink / raw)
To: Herbert Xu
Cc: linux-crypto@vger.kernel.org, Eric Biggers,
linux-arm-kernel@lists.infradead.org
In-Reply-To: <20170211105339.GE855@gondor.apana.org.au>
On 11 February 2017 at 10:53, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Fri, Feb 03, 2017 at 02:49:34PM +0000, Ard Biesheuvel wrote:
>> This series is primarily directed at improving the performance and security
>> of CCM on the Rasperry Pi 3. This involves splitting the MAC handling of
>> CCM into a separate driver so that we can efficiently replace it by something
>> else using the ordinary algo resolution machinery.
>>
>> Patch #1 adds some testcases for cbcmac(aes), which will be introduced later.
>>
>> Patch #2 replaces the open coded CBC MAC hashing routines in the CCM driver
>> with calls to a cbcmac() hash, and implements a template for producing such
>> cbcmac transforms. This eliminates all the fuzzy scatterwalk code as well.
>>
>> Patch #3 implements cbcmac(aes) using NEON on arm64, and CMAC/XCBC at the
>> same time, since it is trivially implemented reusing the same core transform
>
> All applied. Please send any fixups on top of these patches.
Thanks Herbert. I do have a fixup for #2, which currently does not
correctly take the alignmask of the MAC's subordinate cipher into
account. I will send a fix for that shortly.
^ permalink raw reply
* Re: [PATCH] crypto: improve gcc optimization flags for serpent and wp512
From: Herbert Xu @ 2017-02-11 10:54 UTC (permalink / raw)
To: Arnd Bergmann
Cc: David S. Miller, kernel-build-reports, Ralf Baechle, linux-crypto,
linux-kernel
In-Reply-To: <20170203223334.3888507-1-arnd@arndb.de>
On Fri, Feb 03, 2017 at 11:33:23PM +0100, Arnd Bergmann wrote:
> An ancient gcc bug (first reported in 2003) has apparently resurfaced
> on MIPS, where kernelci.org reports an overly large stack frame in the
> whirlpool hash algorithm:
>
> crypto/wp512.c:987:1: warning: the frame size of 1112 bytes is larger than 1024 bytes [-Wframe-larger-than=]
Patch applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v6 0/3] Add Support for Cavium Cryptographic Acceleration Unit
From: Herbert Xu @ 2017-02-11 10:57 UTC (permalink / raw)
To: George Cherian
Cc: davem, david.daney, clabbe.montjoie, smueller, levinsasha928,
linux-kernel, linux-crypto
In-Reply-To: <1486479075-1813-1-git-send-email-george.cherian@cavium.com>
On Tue, Feb 07, 2017 at 02:51:12PM +0000, George Cherian wrote:
> This series adds the support for Cavium Cryptographic Accelerarion Unit (CPT)
> CPT is available in Cavium's Octeon-Tx SoC series.
>
> The series was tested with ecryptfs and dm-crypt for in kernel cryptographic
> offload operations. This driver needs a firmware to work, I will be sending the
> firmware to linux-firmware once the driver is accepted.
>
> Cahnges v5 -> v6
> -- Addressed Sasha Levin's comments.
> - Remove open coding of hlist_for_each_entry_safe, instead use
> the same.
> - Remove the direct access of hlist members
> - Remove unwanted argument checks.
All applied. Thanks.
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox