[PATCHv3 00/11] crypto: omap HW crypto fixes

[PATCHv3 00/11] crypto: omap HW crypto fixes

[Tero Kristo]

Hi,

This revision took quite a bit time to craft due to the rework needed
for sham buffer handling and export/import. I ended up implementing
a flush functionality for draining out the sham buffer when doing
export/import; just shrinking the buffer to sufficiently small size
impacted the performance with small data chunks too much so I dropped
this approach.

The series also fixes a couple of existing issues with omap2/omap3
hardware acceleration, I ran a full boot test / crypto manager
test suite on all boards accessible to me now.

Based on top of latest mainline, which is somewhere before 4.8-rc1
as of writing this, I am unable to rebase the series during the next
three weeks so wanted to get this out now. Targeted for 4.9 merge
window, some fixes could be picked up earlier though if needed.

-Tero

[PATCHv3 01/11] crypto: omap-sham: avoid executing tasklet where not needed

[Tero Kristo]

Some of the call paths of OMAP SHA driver can avoid executing the next
step of the crypto queue under tasklet; instead, execute the next step
directly via function call. This avoids a costly round-trip via the
scheduler giving a slight performance boost.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-sham.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index 7fe4eef..fd50005 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -1005,9 +1005,6 @@ static void omap_sham_finish_req(struct ahash_request *req, int err)
 
 	if (req->base.complete)
 		req->base.complete(&req->base, err);
-
-	/* handle new request */
-	tasklet_schedule(&dd->done_task);
 }
 
 static int omap_sham_handle_queue(struct omap_sham_dev *dd,
@@ -1018,6 +1015,7 @@ static int omap_sham_handle_queue(struct omap_sham_dev *dd,
 	unsigned long flags;
 	int err = 0, ret = 0;
 
+retry:
 	spin_lock_irqsave(&dd->lock, flags);
 	if (req)
 		ret = ahash_enqueue_request(&dd->queue, req);
@@ -1061,11 +1059,19 @@ static int omap_sham_handle_queue(struct omap_sham_dev *dd,
 		err = omap_sham_final_req(dd);
 	}
 err1:
-	if (err != -EINPROGRESS)
+	dev_dbg(dd->dev, "exit, err: %d\n", err);
+
+	if (err != -EINPROGRESS) {
 		/* done_task will not finish it, so do it here */
 		omap_sham_finish_req(req, err);
+		req = NULL;
 
-	dev_dbg(dd->dev, "exit, err: %d\n", err);
+		/*
+		 * Execute next request immediately if there is anything
+		 * in queue.
+		 */
+		goto retry;
+	}
 
 	return ret;
 }
@@ -1653,6 +1659,10 @@ finish:
 	dev_dbg(dd->dev, "update done: err: %d\n", err);
 	/* finish curent request */
 	omap_sham_finish_req(dd->req, err);
+
+	/* If we are not busy, process next req */
+	if (!test_bit(FLAGS_BUSY, &dd->flags))
+		omap_sham_handle_queue(dd, NULL);
 }
 
 static irqreturn_t omap_sham_irq_common(struct omap_sham_dev *dd)
-- 
1.9.1

[PATCHv3 02/11] crypto: omap-sham: add support for flushing the buffer

[Tero Kristo]

This flushes any full blocks of data from the data buffer. Required for
implementing the export/import APIs for the driver, as the flush allows
saving a much smaller context; basically only one block of buffer is
required.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-sham.c | 60 ++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 55 insertions(+), 5 deletions(-)

diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index fd50005..6e53944 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -112,6 +112,7 @@
 #define FLAGS_DMA_READY		6
 #define FLAGS_AUTO_XOR		7
 #define FLAGS_BE32_SHA1		8
+#define FLAGS_FLUSH		9
 /* context flags */
 #define FLAGS_FINUP		16
 #define FLAGS_SG		17
@@ -996,15 +997,16 @@ static void omap_sham_finish_req(struct ahash_request *req, int err)
 		ctx->flags |= BIT(FLAGS_ERROR);
 	}
 
-	/* atomic operation is not needed here */
-	dd->flags &= ~(BIT(FLAGS_BUSY) | BIT(FLAGS_FINAL) | BIT(FLAGS_CPU) |
-			BIT(FLAGS_DMA_READY) | BIT(FLAGS_OUTPUT_READY));
-
 	pm_runtime_mark_last_busy(dd->dev);
 	pm_runtime_put_autosuspend(dd->dev);
 
-	if (req->base.complete)
+	if (!test_bit(FLAGS_FLUSH, &dd->flags) && req->base.complete)
 		req->base.complete(&req->base, err);
+
+	/* atomic operation is not needed here */
+	dd->flags &= ~(BIT(FLAGS_BUSY) | BIT(FLAGS_FINAL) | BIT(FLAGS_CPU) |
+		       BIT(FLAGS_DMA_READY) | BIT(FLAGS_OUTPUT_READY) |
+		       BIT(FLAGS_FLUSH));
 }
 
 static int omap_sham_handle_queue(struct omap_sham_dev *dd,
@@ -1329,6 +1331,54 @@ static void omap_sham_cra_exit(struct crypto_tfm *tfm)
 	}
 }
 
+static int omap_sham_flush(struct ahash_request *req)
+{
+	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
+	struct omap_sham_dev *dd = rctx->dd;
+	int ret, len, bs;
+	unsigned long flags;
+
+	bs = get_block_size(rctx);
+
+	len = rctx->bufcnt / bs * bs;
+
+	spin_lock_irqsave(&dd->lock, flags);
+
+	if (test_bit(FLAGS_BUSY, &dd->flags)) {
+		ret = -EINPROGRESS;
+		goto exit_unlock;
+	}
+
+	if (!len) {
+		ret = 0;
+		goto exit_unlock;
+	}
+
+	set_bit(FLAGS_BUSY, &dd->flags);
+
+	spin_unlock_irqrestore(&dd->lock, flags);
+
+	rctx->total = 0;
+
+	ret = omap_sham_hw_init(dd);
+	if (ret)
+		goto exit_unlock;
+
+	set_bit(FLAGS_FLUSH, &dd->flags);
+
+	ret = omap_sham_xmit_cpu(dd, rctx->buffer, len, 0);
+	if (ret == -EINPROGRESS) {
+		memcpy(rctx->buffer, rctx->buffer + len, rctx->bufcnt - len);
+		rctx->bufcnt -= len;
+	}
+
+	return ret;
+
+exit_unlock:
+	spin_unlock_irqrestore(&dd->lock, flags);
+	return ret;
+}
+
 static struct ahash_alg algs_sha1_md5[] = {
 {
 	.init		= omap_sham_init,
-- 
1.9.1

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

Context export/import are now required for ahash algorithms due to
required support in algif_hash. Implement these for OMAP SHA driver,
saving and restoring the internal state of the driver.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-sham.c | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index 6e53944..aa71e61 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -1379,6 +1379,27 @@ exit_unlock:
 	return ret;
 }
 
+static int omap_sham_export(struct ahash_request *req, void *out)
+{
+	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
+
+	while (omap_sham_flush(req) == -EINPROGRESS)
+		msleep(10);
+
+	memcpy(out, rctx, sizeof(*rctx) + SHA512_BLOCK_SIZE);
+
+	return 0;
+}
+
+static int omap_sham_import(struct ahash_request *req, const void *in)
+{
+	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
+
+	memcpy(rctx, in, sizeof(*rctx) + SHA512_BLOCK_SIZE);
+
+	return 0;
+}
+
 static struct ahash_alg algs_sha1_md5[] = {
 {
 	.init		= omap_sham_init,
@@ -2037,8 +2058,14 @@ static int omap_sham_probe(struct platform_device *pdev)
 
 	for (i = 0; i < dd->pdata->algs_info_size; i++) {
 		for (j = 0; j < dd->pdata->algs_info[i].size; j++) {
-			err = crypto_register_ahash(
-					&dd->pdata->algs_info[i].algs_list[j]);
+			struct ahash_alg *alg;
+
+			alg = &dd->pdata->algs_info[i].algs_list[j];
+			alg->export = omap_sham_export;
+			alg->import = omap_sham_import;
+			alg->halg.statesize = sizeof(struct omap_sham_reqctx) +
+				SHA512_BLOCK_SIZE;
+			err = crypto_register_ahash(alg);
 			if (err)
 				goto err_algs;
 
-- 
1.9.1

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Herbert Xu]

On Thu, Aug 04, 2016 at 01:28:38PM +0300, Tero Kristo wrote:
> Context export/import are now required for ahash algorithms due to
> required support in algif_hash. Implement these for OMAP SHA driver,
> saving and restoring the internal state of the driver.
> 
> Signed-off-by: Tero Kristo <t-kristo@ti.com>
> ---
>  drivers/crypto/omap-sham.c | 31 +++++++++++++++++++++++++++++--
>  1 file changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
> index 6e53944..aa71e61 100644
> --- a/drivers/crypto/omap-sham.c
> +++ b/drivers/crypto/omap-sham.c
> @@ -1379,6 +1379,27 @@ exit_unlock:
>  	return ret;
>  }
>  
> +static int omap_sham_export(struct ahash_request *req, void *out)
> +{
> +	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
> +
> +	while (omap_sham_flush(req) == -EINPROGRESS)
> +		msleep(10);

Do we really need this? You must not call export until the previous
operation has completed.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

On 09/08/16 13:06, Herbert Xu wrote:
> On Thu, Aug 04, 2016 at 01:28:38PM +0300, Tero Kristo wrote:
>> Context export/import are now required for ahash algorithms due to
>> required support in algif_hash. Implement these for OMAP SHA driver,
>> saving and restoring the internal state of the driver.
>>
>> Signed-off-by: Tero Kristo <t-kristo@ti.com>
>> ---
>>  drivers/crypto/omap-sham.c | 31 +++++++++++++++++++++++++++++--
>>  1 file changed, 29 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
>> index 6e53944..aa71e61 100644
>> --- a/drivers/crypto/omap-sham.c
>> +++ b/drivers/crypto/omap-sham.c
>> @@ -1379,6 +1379,27 @@ exit_unlock:
>>  	return ret;
>>  }
>>
>> +static int omap_sham_export(struct ahash_request *req, void *out)
>> +{
>> +	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
>> +
>> +	while (omap_sham_flush(req) == -EINPROGRESS)
>> +		msleep(10);
>
> Do we really need this? You must not call export until the previous
> operation has completed.
>
> Cheers,
>

Sorry about a late reply, I was out on vacation.

For OMAP SHAM, this is actually needed, because the driver still has a 
very large internal buffer for performance reasons, and the whole buffer 
can't be exported. The flush functionality pushes out sufficient amount 
of data to the hardware, so that the rest of the buffer can be exported 
to the available space.

This is pretty much related to the discussion we had previously here:

https://patchwork.kernel.org/patch/9192881/

Basically I decided to keep the driver buffer the same size as 
previously, but flush out any extra data.

-Tero

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Herbert Xu]

On Mon, Aug 29, 2016 at 05:11:35PM +0300, Tero Kristo wrote:
>
> >>+static int omap_sham_export(struct ahash_request *req, void *out)
> >>+{
> >>+	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
> >>+
> >>+	while (omap_sham_flush(req) == -EINPROGRESS)
> >>+		msleep(10);
> >
> >Do we really need this? You must not call export until the previous
> >operation has completed.
> >
> >Cheers,
> >
> 
> Sorry about a late reply, I was out on vacation.
> 
> For OMAP SHAM, this is actually needed, because the driver still has
> a very large internal buffer for performance reasons, and the whole
> buffer can't be exported. The flush functionality pushes out
> sufficient amount of data to the hardware, so that the rest of the
> buffer can be exported to the available space.

It doesn't matter whether you have a buffer or not.  The point
is that the completion function should not be called until the
operation is actually complete.  This is the whole point of the
async interface.

As the user must not call export until the completion function
has been called, there should be no need to wait in the export
function.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

On 01/09/16 06:33, Herbert Xu wrote:
> On Mon, Aug 29, 2016 at 05:11:35PM +0300, Tero Kristo wrote:
>>
>>>> +static int omap_sham_export(struct ahash_request *req, void *out)
>>>> +{
>>>> +	struct omap_sham_reqctx *rctx = ahash_request_ctx(req);
>>>> +
>>>> +	while (omap_sham_flush(req) == -EINPROGRESS)
>>>> +		msleep(10);
>>>
>>> Do we really need this? You must not call export until the previous
>>> operation has completed.
>>>
>>> Cheers,
>>>
>>
>> Sorry about a late reply, I was out on vacation.
>>
>> For OMAP SHAM, this is actually needed, because the driver still has
>> a very large internal buffer for performance reasons, and the whole
>> buffer can't be exported. The flush functionality pushes out
>> sufficient amount of data to the hardware, so that the rest of the
>> buffer can be exported to the available space.
>
> It doesn't matter whether you have a buffer or not.  The point
> is that the completion function should not be called until the
> operation is actually complete.  This is the whole point of the
> async interface.
>
> As the user must not call export until the completion function
> has been called, there should be no need to wait in the export
> function.

Well, but the driver doesn't flush its buffers automatically, it caches 
data until it has sufficient amount available. So, assuming you want to 
do this:

sham_init
   sham_update 256 bytes
   sham_update 256 bytes
   wait until two above updates are complete
   sham_export

... the execution hangs at the wait phase as the driver is still waiting 
for more data to cache, and will never complete the two update requests. 
Currently, the driver is written in such way that it waits until it has 
enough data cached before starting to push it out to hardware, or waits 
until sham_final to be called. Pushing out small pieces of data causes 
severe performance degradation on the driver, as setting up the DMA 
operation itself is rather costly.

Either way, flush for the buffers is needed, I wonder if automatic flush 
should be added also based on some timer.

-Tero

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Herbert Xu]

On Thu, Sep 01, 2016 at 09:12:59AM +0300, Tero Kristo wrote:
> 
> Well, but the driver doesn't flush its buffers automatically, it
> caches data until it has sufficient amount available. So, assuming
> you want to do this:
> 
> sham_init
>   sham_update 256 bytes
>   sham_update 256 bytes
>   wait until two above updates are complete
>   sham_export
> 
> ... the execution hangs at the wait phase as the driver is still

Well that's a bug in the driver.  While it's not illegal to wait
for more data, it's usually unnecessary.  Because we instead try
to get our users to generate as big a request as possible, e.g.,
one packet for IPsec.

If you really have to do the hold thing, then you must install a
timer like sha1-mb does on x86 to do the flush.

In any case, the completion function must not be called until
you're actually complete.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

On 01/09/16 09:16, Herbert Xu wrote:
> On Thu, Sep 01, 2016 at 09:12:59AM +0300, Tero Kristo wrote:
>>
>> Well, but the driver doesn't flush its buffers automatically, it
>> caches data until it has sufficient amount available. So, assuming
>> you want to do this:
>>
>> sham_init
>>   sham_update 256 bytes
>>   sham_update 256 bytes
>>   wait until two above updates are complete
>>   sham_export
>>
>> ... the execution hangs at the wait phase as the driver is still
>
> Well that's a bug in the driver.  While it's not illegal to wait
> for more data, it's usually unnecessary.  Because we instead try
> to get our users to generate as big a request as possible, e.g.,
> one packet for IPsec.
>
> If you really have to do the hold thing, then you must install a
> timer like sha1-mb does on x86 to do the flush.
>
> In any case, the completion function must not be called until
> you're actually complete.

Hmm, looking at the driver, sham_update returns 0 immediately if it just 
caches data. In a sense, the update is not completed at this point. Are 
you saying this is illegal and can't be done?

 From my understanding, valid results are expected from the driver only 
after ->final is called.

-Tero

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Herbert Xu]

On Thu, Sep 01, 2016 at 09:56:06AM +0300, Tero Kristo wrote:
>
> Hmm, looking at the driver, sham_update returns 0 immediately if it
> just caches data. In a sense, the update is not completed at this
> point. Are you saying this is illegal and can't be done?

Once you call the completion function (and returning zero from
the update itself is equivalent to calling the completion function)
the hardware must not touch the request anymore.

> From my understanding, valid results are expected from the driver
> only after ->final is called.

That's because you never implemented export/import :)

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

On 01/09/16 10:19, Herbert Xu wrote:
> On Thu, Sep 01, 2016 at 09:56:06AM +0300, Tero Kristo wrote:
>>
>> Hmm, looking at the driver, sham_update returns 0 immediately if it
>> just caches data. In a sense, the update is not completed at this
>> point. Are you saying this is illegal and can't be done?
>
> Once you call the completion function (and returning zero from
> the update itself is equivalent to calling the completion function)
> the hardware must not touch the request anymore.

Yeah, it is not touching it anymore. All the data has been copied to the 
local buffer at this point, and the driver isn't retaining any kind of a 
handle to the request itself anymore. It is just that it has not been 
processed by the hardware yet.

>
>> From my understanding, valid results are expected from the driver
>> only after ->final is called.
>
> That's because you never implemented export/import :)

Yeah, the flush should do the trick now. Kind of a chicken-egg problem 
here. :P How do you see the situation with the above explanation?


-Tero

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Herbert Xu]

On Thu, Sep 01, 2016 at 10:28:47AM +0300, Tero Kristo wrote:
>
> Yeah, the flush should do the trick now. Kind of a chicken-egg
> problem here. :P How do you see the situation with the above
> explanation?

The export function is not allowed to sleep so you must not wait
for the hardware to complete in it.

If you need to wait, then you must use the completion mechanism.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

On 01/09/16 10:31, Herbert Xu wrote:
> On Thu, Sep 01, 2016 at 10:28:47AM +0300, Tero Kristo wrote:
>>
>> Yeah, the flush should do the trick now. Kind of a chicken-egg
>> problem here. :P How do you see the situation with the above
>> explanation?
>
> The export function is not allowed to sleep so you must not wait
> for the hardware to complete in it.
>
> If you need to wait, then you must use the completion mechanism.

I don't think export is allowed to return -EINPROGRESS either? At least 
currently the kernel pieces using this functionality won't work if I do 
that.

If thats the case, I need to think of something else to handle this...

-Tero

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Tero Kristo]

On 01/09/16 10:46, Tero Kristo wrote:
> On 01/09/16 10:31, Herbert Xu wrote:
>> On Thu, Sep 01, 2016 at 10:28:47AM +0300, Tero Kristo wrote:
>>>
>>> Yeah, the flush should do the trick now. Kind of a chicken-egg
>>> problem here. :P How do you see the situation with the above
>>> explanation?
>>
>> The export function is not allowed to sleep so you must not wait
>> for the hardware to complete in it.
>>
>> If you need to wait, then you must use the completion mechanism.
>
> I don't think export is allowed to return -EINPROGRESS either? At least
> currently the kernel pieces using this functionality won't work if I do
> that.
>
> If thats the case, I need to think of something else to handle this...
>
> -Tero
>

Hi Herbert,

Additional request, would it be possible for you to check the rest of 
the series and just ignore patches #2 and #3 for now, the rest don't 
have any dependencies against these and can be applied cleanly without.

I would like to see these move forward while I figure out how to handle 
the buffer / export+import...

-Tero

[PATCHv3 03/11] crypto: omap-sham: implement context export/import APIs

[Herbert Xu]

On Mon, Sep 05, 2016 at 03:06:05PM +0300, Tero Kristo wrote:
>
> Additional request, would it be possible for you to check the rest
> of the series and just ignore patches #2 and #3 for now, the rest
> don't have any dependencies against these and can be applied cleanly
> without.
> 
> I would like to see these move forward while I figure out how to
> handle the buffer / export+import...

Sure I'll review them again.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 04/11] crypto: omap-sham: fix software fallback handling

[Tero Kristo]

If we have processed any data with the hardware accelerator (digcnt > 0),
we must complete the entire hash by using it. This is because the current
hash value can't be imported to the software fallback algorithm. Otherwise
we end up with wrong hash results.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-sham.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index aa71e61..b4f5131 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -1165,7 +1165,7 @@ static int omap_sham_final(struct ahash_request *req)
 	 * If buffersize is less than 240, we use fallback SW encoding,
 	 * as using DMA + HW in this case doesn't provide any benefit.
 	 */
-	if ((ctx->digcnt + ctx->bufcnt) < 240)
+	if (!ctx->digcnt && ctx->bufcnt < 240)
 		return omap_sham_final_shash(req);
 	else if (ctx->bufcnt)
 		return omap_sham_enqueue(req, OP_FINAL);
-- 
1.9.1

[PATCHv3 05/11] crypto: omap-sham: fix SW fallback HMAC handling for omap2/omap3

[Tero Kristo]

If software fallback is used on older hardware accelerator setup (OMAP2/
OMAP3), the first block of data must be purged from the buffer. The
first block contains the pre-generated ipad value required by the HW,
but the software fallback algorithm generates its own, causing wrong
results.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-sham.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index b4f5131..f9c5fe5 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -1145,9 +1145,20 @@ static int omap_sham_final_shash(struct ahash_request *req)
 {
 	struct omap_sham_ctx *tctx = crypto_tfm_ctx(req->base.tfm);
 	struct omap_sham_reqctx *ctx = ahash_request_ctx(req);
+	int offset = 0;
+
+	/*
+	 * If we are running HMAC on limited hardware support, skip
+	 * the ipad in the beginning of the buffer if we are going for
+	 * software fallback algorithm.
+	 */
+	if (test_bit(FLAGS_HMAC, &ctx->flags) &&
+	    !test_bit(FLAGS_AUTO_XOR, &ctx->dd->flags))
+		offset = get_block_size(ctx);
 
 	return omap_sham_shash_digest(tctx->fallback, req->base.flags,
-				      ctx->buffer, ctx->bufcnt, req->result);
+				      ctx->buffer + offset,
+				      ctx->bufcnt - offset, req->result);
 }
 
 static int omap_sham_final(struct ahash_request *req)
-- 
1.9.1

[PATCHv3 06/11] crypto: omap-des: Fix support for unequal lengths

[Tero Kristo]

From: Lokesh Vutla <a0131933@ti.com>

For cases where total length of an input SGs is not same as
length of the input data for encryption, omap-des driver
crashes. This happens in the case when IPsec is trying to use
omap-des driver.

To avoid this, we copy all the pages from the input SG list
into a contiguous buffer and prepare a single element SG list
for this buffer with length as the total bytes to crypt, which is
similar thing that is done in case of unaligned lengths.

Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
Tested-by: Aparna Balasubramanian <aparnab@ti.com>
Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-des.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/drivers/crypto/omap-des.c b/drivers/crypto/omap-des.c
index 5691434..d0b59f6 100644
--- a/drivers/crypto/omap-des.c
+++ b/drivers/crypto/omap-des.c
@@ -521,29 +521,36 @@ static int omap_des_crypt_dma_stop(struct omap_des_dev *dd)
 	return 0;
 }
 
-static int omap_des_copy_needed(struct scatterlist *sg)
+static int omap_des_copy_needed(struct scatterlist *sg, int total)
 {
+	int len = 0;
+
+	if (!IS_ALIGNED(total, DES_BLOCK_SIZE))
+		return -1;
+
 	while (sg) {
 		if (!IS_ALIGNED(sg->offset, 4))
 			return -1;
 		if (!IS_ALIGNED(sg->length, DES_BLOCK_SIZE))
 			return -1;
+
+		len += sg->length;
 		sg = sg_next(sg);
 	}
+
+	if (len != total)
+		return -1;
+
 	return 0;
 }
 
 static int omap_des_copy_sgs(struct omap_des_dev *dd)
 {
 	void *buf_in, *buf_out;
-	int pages;
-
-	pages = dd->total >> PAGE_SHIFT;
-
-	if (dd->total & (PAGE_SIZE-1))
-		pages++;
+	int pages, total;
 
-	BUG_ON(!pages);
+	total = ALIGN(dd->total, DES_BLOCK_SIZE);
+	pages = get_order(total);
 
 	buf_in = (void *)__get_free_pages(GFP_ATOMIC, pages);
 	buf_out = (void *)__get_free_pages(GFP_ATOMIC, pages);
@@ -605,8 +612,8 @@ static int omap_des_prepare_req(struct crypto_engine *engine,
 	if (dd->out_sg_len < 0)
 		return dd->out_sg_len;
 
-	if (omap_des_copy_needed(dd->in_sg) ||
-	    omap_des_copy_needed(dd->out_sg)) {
+	if (omap_des_copy_needed(dd->in_sg, dd->total) ||
+	    omap_des_copy_needed(dd->out_sg, dd->total)) {
 		if (omap_des_copy_sgs(dd))
 			pr_err("Failed to copy SGs for unaligned cases\n");
 		dd->sgs_copied = 1;
-- 
1.9.1

[PATCHv3 06/11] crypto: omap-des: Fix support for unequal lengths

[Herbert Xu]

On Thu, Aug 04, 2016 at 01:28:41PM +0300, Tero Kristo wrote:
> From: Lokesh Vutla <a0131933@ti.com>
> 
> For cases where total length of an input SGs is not same as
> length of the input data for encryption, omap-des driver
> crashes. This happens in the case when IPsec is trying to use
> omap-des driver.
> 
> To avoid this, we copy all the pages from the input SG list
> into a contiguous buffer and prepare a single element SG list
> for this buffer with length as the total bytes to crypt, which is
> similar thing that is done in case of unaligned lengths.

Ugh, that means copying every single packet, right?

So if it's just the SG list that's the problem, why don't you
copy that instead? That is, allocate a new SG list and set it
up so that there is no excess data.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 06/11] crypto: omap-des: Fix support for unequal lengths

[Tero Kristo]

On 13/09/16 12:35, Herbert Xu wrote:
> On Thu, Aug 04, 2016 at 01:28:41PM +0300, Tero Kristo wrote:
>> From: Lokesh Vutla <a0131933@ti.com>
>>
>> For cases where total length of an input SGs is not same as
>> length of the input data for encryption, omap-des driver
>> crashes. This happens in the case when IPsec is trying to use
>> omap-des driver.
>>
>> To avoid this, we copy all the pages from the input SG list
>> into a contiguous buffer and prepare a single element SG list
>> for this buffer with length as the total bytes to crypt, which is
>> similar thing that is done in case of unaligned lengths.
>
> Ugh, that means copying every single packet, right?
>
> So if it's just the SG list that's the problem, why don't you
> copy that instead? That is, allocate a new SG list and set it
> up so that there is no excess data.
>
> Cheers,
>

I'll take a look at this. I have this kind of solution in place for the 
re-worked SHA driver, so can probably re-use it here.

-Tero

[PATCHv3 07/11] crypto: omap-aes: use runtime_pm autosuspend for clock handling

[Tero Kristo]

Calling runtime PM API at the cra_init/exit is bad for power management
purposes, as the lifetime for a CRA can be very long. Instead, use
pm_runtime autosuspend approach for handling the device clocks. Clocks
are enabled when they are actually required, and autosuspend disables
these if they have not been used for a sufficiently long time period.
By default, the timeout value is 1 second.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-aes.c | 43 ++++++++++++++++---------------------------
 1 file changed, 16 insertions(+), 27 deletions(-)

diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index 4ab53a6..f443042 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -85,6 +85,8 @@
 #define AES_REG_IRQ_DATA_OUT           BIT(2)
 #define DEFAULT_TIMEOUT		(5*HZ)
 
+#define DEFAULT_AUTOSUSPEND_DELAY	1000
+
 #define FLAGS_MODE_MASK		0x000f
 #define FLAGS_ENCRYPT		BIT(0)
 #define FLAGS_CBC		BIT(1)
@@ -238,11 +240,19 @@ static void omap_aes_write_n(struct omap_aes_dev *dd, u32 offset,
 
 static int omap_aes_hw_init(struct omap_aes_dev *dd)
 {
+	int err;
+
 	if (!(dd->flags & FLAGS_INIT)) {
 		dd->flags |= FLAGS_INIT;
 		dd->err = 0;
 	}
 
+	err = pm_runtime_get_sync(dd->dev);
+	if (err < 0) {
+		dev_err(dd->dev, "failed to get sync: %d\n", err);
+		return err;
+	}
+
 	return 0;
 }
 
@@ -520,6 +530,9 @@ static void omap_aes_finish_req(struct omap_aes_dev *dd, int err)
 	pr_debug("err: %d\n", err);
 
 	crypto_finalize_request(dd->engine, req, err);
+
+	pm_runtime_mark_last_busy(dd->dev);
+	pm_runtime_put_autosuspend(dd->dev);
 }
 
 static int omap_aes_crypt_dma_stop(struct omap_aes_dev *dd)
@@ -761,23 +774,6 @@ static int omap_aes_ctr_decrypt(struct ablkcipher_request *req)
 
 static int omap_aes_cra_init(struct crypto_tfm *tfm)
 {
-	struct omap_aes_dev *dd = NULL;
-	int err;
-
-	/* Find AES device, currently picks the first device */
-	spin_lock_bh(&list_lock);
-	list_for_each_entry(dd, &dev_list, list) {
-		break;
-	}
-	spin_unlock_bh(&list_lock);
-
-	err = pm_runtime_get_sync(dd->dev);
-	if (err < 0) {
-		dev_err(dd->dev, "%s: failed to get_sync(%d)\n",
-			__func__, err);
-		return err;
-	}
-
 	tfm->crt_ablkcipher.reqsize = sizeof(struct omap_aes_reqctx);
 
 	return 0;
@@ -785,16 +781,6 @@ static int omap_aes_cra_init(struct crypto_tfm *tfm)
 
 static void omap_aes_cra_exit(struct crypto_tfm *tfm)
 {
-	struct omap_aes_dev *dd = NULL;
-
-	/* Find AES device, currently picks the first device */
-	spin_lock_bh(&list_lock);
-	list_for_each_entry(dd, &dev_list, list) {
-		break;
-	}
-	spin_unlock_bh(&list_lock);
-
-	pm_runtime_put_sync(dd->dev);
 }
 
 /* ********************** ALGS ************************************ */
@@ -1140,6 +1126,9 @@ static int omap_aes_probe(struct platform_device *pdev)
 	}
 	dd->phys_base = res.start;
 
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_set_autosuspend_delay(dev, DEFAULT_AUTOSUSPEND_DELAY);
+
 	pm_runtime_enable(dev);
 	err = pm_runtime_get_sync(dev);
 	if (err < 0) {
-- 
1.9.1

[PATCHv3 08/11] crypto: omap-aes: Add support for multiple cores

[Tero Kristo]

From: Lokesh Vutla <lokeshvutla@ti.com>

Some SoCs like omap4/omap5/dra7 contain multiple AES crypto accelerator
cores. Adapt the driver to support this. The driver picks the last used
device from a list of AES devices.

Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
[t-kristo at ti.com: forward ported to 4.7 kernel]
Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-aes.c | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index f443042..3cf7b8f 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -329,20 +329,12 @@ static void omap_aes_dma_stop(struct omap_aes_dev *dd)
 
 static struct omap_aes_dev *omap_aes_find_dev(struct omap_aes_ctx *ctx)
 {
-	struct omap_aes_dev *dd = NULL, *tmp;
+	struct omap_aes_dev *dd;
 
 	spin_lock_bh(&list_lock);
-	if (!ctx->dd) {
-		list_for_each_entry(tmp, &dev_list, list) {
-			/* FIXME: take fist available aes core */
-			dd = tmp;
-			break;
-		}
-		ctx->dd = dd;
-	} else {
-		/* already found before */
-		dd = ctx->dd;
-	}
+	dd = list_first_entry(&dev_list, struct omap_aes_dev, list);
+	list_move_tail(&dd->list, &dev_list);
+	ctx->dd = dd;
 	spin_unlock_bh(&list_lock);
 
 	return dd;
@@ -615,7 +607,7 @@ static int omap_aes_prepare_req(struct crypto_engine *engine,
 {
 	struct omap_aes_ctx *ctx = crypto_ablkcipher_ctx(
 			crypto_ablkcipher_reqtfm(req));
-	struct omap_aes_dev *dd = omap_aes_find_dev(ctx);
+	struct omap_aes_dev *dd = ctx->dd;
 	struct omap_aes_reqctx *rctx;
 
 	if (!dd)
@@ -661,7 +653,7 @@ static int omap_aes_crypt_req(struct crypto_engine *engine,
 {
 	struct omap_aes_ctx *ctx = crypto_ablkcipher_ctx(
 			crypto_ablkcipher_reqtfm(req));
-	struct omap_aes_dev *dd = omap_aes_find_dev(ctx);
+	struct omap_aes_dev *dd = ctx->dd;
 
 	if (!dd)
 		return -ENODEV;
-- 
1.9.1

[PATCHv3 09/11] crypto: omap-aes: Add fallback support

[Tero Kristo]

From: Lokesh Vutla <lokeshvutla@ti.com>

As setting up the DMA operations is quite costly, add software fallback
support for requests smaller than 200 bytes. This change gives some 10%
extra performance in ipsec use case.

Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
[t-kristo at ti.com: udpated against latest upstream, to use skcipher mainly]
Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/Kconfig    |  3 +++
 drivers/crypto/omap-aes.c | 53 +++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 52 insertions(+), 4 deletions(-)

diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 1af94e2..19ee6ee 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -318,6 +318,9 @@ config CRYPTO_DEV_OMAP_AES
 	select CRYPTO_AES
 	select CRYPTO_BLKCIPHER
 	select CRYPTO_ENGINE
+	select CRYPTO_CBC
+	select CRYPTO_ECB
+	select CRYPTO_CTR
 	help
 	  OMAP processors have AES module accelerator. Select this if you
 	  want to use the OMAP module for AES algorithms.
diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index 3cf7b8f..6ba4f70 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -35,7 +35,7 @@
 #include <linux/interrupt.h>
 #include <crypto/scatterwalk.h>
 #include <crypto/aes.h>
-#include <crypto/algapi.h>
+#include <crypto/internal/skcipher.h>
 
 #define DST_MAXBURST			4
 #define DMA_MIN				(DST_MAXBURST * sizeof(u32))
@@ -105,6 +105,7 @@ struct omap_aes_ctx {
 	int		keylen;
 	u32		key[AES_KEYSIZE_256 / sizeof(u32)];
 	unsigned long	flags;
+	struct crypto_skcipher	*fallback;
 };
 
 struct omap_aes_reqctx {
@@ -701,11 +702,29 @@ static int omap_aes_crypt(struct ablkcipher_request *req, unsigned long mode)
 			crypto_ablkcipher_reqtfm(req));
 	struct omap_aes_reqctx *rctx = ablkcipher_request_ctx(req);
 	struct omap_aes_dev *dd;
+	int ret;
 
 	pr_debug("nbytes: %d, enc: %d, cbc: %d\n", req->nbytes,
 		  !!(mode & FLAGS_ENCRYPT),
 		  !!(mode & FLAGS_CBC));
 
+	if (req->nbytes < 200) {
+		SKCIPHER_REQUEST_ON_STACK(subreq, ctx->fallback);
+
+		skcipher_request_set_tfm(subreq, ctx->fallback);
+		skcipher_request_set_callback(subreq, req->base.flags, NULL,
+					      NULL);
+		skcipher_request_set_crypt(subreq, req->src, req->dst,
+					   req->nbytes, req->info);
+
+		if (mode & FLAGS_ENCRYPT)
+			ret = crypto_skcipher_encrypt(subreq);
+		else
+			ret = crypto_skcipher_decrypt(subreq);
+
+		skcipher_request_zero(subreq);
+		return ret;
+	}
 	dd = omap_aes_find_dev(ctx);
 	if (!dd)
 		return -ENODEV;
@@ -721,6 +740,7 @@ static int omap_aes_setkey(struct crypto_ablkcipher *tfm, const u8 *key,
 			   unsigned int keylen)
 {
 	struct omap_aes_ctx *ctx = crypto_ablkcipher_ctx(tfm);
+	int ret;
 
 	if (keylen != AES_KEYSIZE_128 && keylen != AES_KEYSIZE_192 &&
 		   keylen != AES_KEYSIZE_256)
@@ -731,6 +751,14 @@ static int omap_aes_setkey(struct crypto_ablkcipher *tfm, const u8 *key,
 	memcpy(ctx->key, key, keylen);
 	ctx->keylen = keylen;
 
+	crypto_skcipher_clear_flags(ctx->fallback, CRYPTO_TFM_REQ_MASK);
+	crypto_skcipher_set_flags(ctx->fallback, tfm->base.crt_flags &
+						 CRYPTO_TFM_REQ_MASK);
+
+	ret = crypto_skcipher_setkey(ctx->fallback, key, keylen);
+	if (!ret)
+		return 0;
+
 	return 0;
 }
 
@@ -766,6 +794,17 @@ static int omap_aes_ctr_decrypt(struct ablkcipher_request *req)
 
 static int omap_aes_cra_init(struct crypto_tfm *tfm)
 {
+	const char *name = crypto_tfm_alg_name(tfm);
+	const u32 flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK;
+	struct omap_aes_ctx *ctx = crypto_tfm_ctx(tfm);
+	struct crypto_skcipher *blk;
+
+	blk = crypto_alloc_skcipher(name, 0, flags);
+	if (IS_ERR(blk))
+		return PTR_ERR(blk);
+
+	ctx->fallback = blk;
+
 	tfm->crt_ablkcipher.reqsize = sizeof(struct omap_aes_reqctx);
 
 	return 0;
@@ -773,6 +812,12 @@ static int omap_aes_cra_init(struct crypto_tfm *tfm)
 
 static void omap_aes_cra_exit(struct crypto_tfm *tfm)
 {
+	struct omap_aes_ctx *ctx = crypto_tfm_ctx(tfm);
+
+	if (ctx->fallback)
+		crypto_free_skcipher(ctx->fallback);
+
+	ctx->fallback = NULL;
 }
 
 /* ********************** ALGS ************************************ */
@@ -784,7 +829,7 @@ static struct crypto_alg algs_ecb_cbc[] = {
 	.cra_priority		= 300,
 	.cra_flags		= CRYPTO_ALG_TYPE_ABLKCIPHER |
 				  CRYPTO_ALG_KERN_DRIVER_ONLY |
-				  CRYPTO_ALG_ASYNC,
+				  CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
 	.cra_blocksize		= AES_BLOCK_SIZE,
 	.cra_ctxsize		= sizeof(struct omap_aes_ctx),
 	.cra_alignmask		= 0,
@@ -806,7 +851,7 @@ static struct crypto_alg algs_ecb_cbc[] = {
 	.cra_priority		= 300,
 	.cra_flags		= CRYPTO_ALG_TYPE_ABLKCIPHER |
 				  CRYPTO_ALG_KERN_DRIVER_ONLY |
-				  CRYPTO_ALG_ASYNC,
+				  CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
 	.cra_blocksize		= AES_BLOCK_SIZE,
 	.cra_ctxsize		= sizeof(struct omap_aes_ctx),
 	.cra_alignmask		= 0,
@@ -832,7 +877,7 @@ static struct crypto_alg algs_ctr[] = {
 	.cra_priority		= 300,
 	.cra_flags		= CRYPTO_ALG_TYPE_ABLKCIPHER |
 				  CRYPTO_ALG_KERN_DRIVER_ONLY |
-				  CRYPTO_ALG_ASYNC,
+				  CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
 	.cra_blocksize		= AES_BLOCK_SIZE,
 	.cra_ctxsize		= sizeof(struct omap_aes_ctx),
 	.cra_alignmask		= 0,
-- 
1.9.1

[PATCHv3 10/11] crypto: omap-aes: fix crypto engine initialization order

[Tero Kristo]

The crypto engine must be initialized before registering algorithms,
otherwise the test manager will crash as it attempts to execute
tests for the algos while they are being registered.

Fixes: 0529900a01cb ("crypto: omap-aes - Support crypto engine framework")
Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-aes.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index 6ba4f70..8159523 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -1212,6 +1212,17 @@ static int omap_aes_probe(struct platform_device *pdev)
 	list_add_tail(&dd->list, &dev_list);
 	spin_unlock(&list_lock);
 
+	/* Initialize crypto engine */
+	dd->engine = crypto_engine_alloc_init(dev, 1);
+	if (!dd->engine)
+		goto err_engine;
+
+	dd->engine->prepare_request = omap_aes_prepare_req;
+	dd->engine->crypt_one_request = omap_aes_crypt_req;
+	err = crypto_engine_start(dd->engine);
+	if (err)
+		goto err_engine;
+
 	for (i = 0; i < dd->pdata->algs_info_size; i++) {
 		if (!dd->pdata->algs_info[i].registered) {
 			for (j = 0; j < dd->pdata->algs_info[i].size; j++) {
@@ -1229,26 +1240,17 @@ static int omap_aes_probe(struct platform_device *pdev)
 		}
 	}
 
-	/* Initialize crypto engine */
-	dd->engine = crypto_engine_alloc_init(dev, 1);
-	if (!dd->engine)
-		goto err_algs;
-
-	dd->engine->prepare_request = omap_aes_prepare_req;
-	dd->engine->crypt_one_request = omap_aes_crypt_req;
-	err = crypto_engine_start(dd->engine);
-	if (err)
-		goto err_engine;
-
 	return 0;
-err_engine:
-	crypto_engine_exit(dd->engine);
 err_algs:
 	for (i = dd->pdata->algs_info_size - 1; i >= 0; i--)
 		for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--)
 			crypto_unregister_alg(
 					&dd->pdata->algs_info[i].algs_list[j]);
 
+err_engine:
+	if (dd->engine)
+		crypto_engine_exit(dd->engine);
+
 	omap_aes_dma_cleanup(dd);
 err_irq:
 	tasklet_kill(&dd->done_task);
-- 
1.9.1

[PATCHv3 11/11] crypto: omap-des: fix crypto engine initialization order

[Tero Kristo]

The crypto engine must be initialized before registering algorithms,
otherwise the test manager will crash as it attempts to execute
tests for the algos while they are being registered.

Fixes: f1b77aaca85a ("crypto: omap-des - Integrate with the crypto engine framework")
Signed-off-by: Tero Kristo <t-kristo@ti.com>
---
 drivers/crypto/omap-des.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/crypto/omap-des.c b/drivers/crypto/omap-des.c
index d0b59f6..2b07e60 100644
--- a/drivers/crypto/omap-des.c
+++ b/drivers/crypto/omap-des.c
@@ -1085,6 +1085,17 @@ static int omap_des_probe(struct platform_device *pdev)
 	list_add_tail(&dd->list, &dev_list);
 	spin_unlock(&list_lock);
 
+	/* Initialize des crypto engine */
+	dd->engine = crypto_engine_alloc_init(dev, 1);
+	if (!dd->engine)
+		goto err_engine;
+
+	dd->engine->prepare_request = omap_des_prepare_req;
+	dd->engine->crypt_one_request = omap_des_crypt_req;
+	err = crypto_engine_start(dd->engine);
+	if (err)
+		goto err_engine;
+
 	for (i = 0; i < dd->pdata->algs_info_size; i++) {
 		for (j = 0; j < dd->pdata->algs_info[i].size; j++) {
 			algp = &dd->pdata->algs_info[i].algs_list[j];
@@ -1100,27 +1111,18 @@ static int omap_des_probe(struct platform_device *pdev)
 		}
 	}
 
-	/* Initialize des crypto engine */
-	dd->engine = crypto_engine_alloc_init(dev, 1);
-	if (!dd->engine)
-		goto err_algs;
-
-	dd->engine->prepare_request = omap_des_prepare_req;
-	dd->engine->crypt_one_request = omap_des_crypt_req;
-	err = crypto_engine_start(dd->engine);
-	if (err)
-		goto err_engine;
-
 	return 0;
 
-err_engine:
-	crypto_engine_exit(dd->engine);
 err_algs:
 	for (i = dd->pdata->algs_info_size - 1; i >= 0; i--)
 		for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--)
 			crypto_unregister_alg(
 					&dd->pdata->algs_info[i].algs_list[j]);
 
+err_engine:
+	if (dd->engine)
+		crypto_engine_exit(dd->engine);
+
 	omap_des_dma_cleanup(dd);
 err_irq:
 	tasklet_kill(&dd->done_task);
-- 
1.9.1

[PATCHv3 00/11] crypto: omap HW crypto fixes

[Herbert Xu]

On Thu, Aug 04, 2016 at 01:28:35PM +0300, Tero Kristo wrote:
> Hi,
> 
> This revision took quite a bit time to craft due to the rework needed
> for sham buffer handling and export/import. I ended up implementing
> a flush functionality for draining out the sham buffer when doing
> export/import; just shrinking the buffer to sufficiently small size
> impacted the performance with small data chunks too much so I dropped
> this approach.
> 
> The series also fixes a couple of existing issues with omap2/omap3
> hardware acceleration, I ran a full boot test / crypto manager
> test suite on all boards accessible to me now.
> 
> Based on top of latest mainline, which is somewhere before 4.8-rc1
> as of writing this, I am unable to rebase the series during the next
> three weeks so wanted to get this out now. Targeted for 4.9 merge
> window, some fixes could be picked up earlier though if needed.

I have applied patches 1,4-5,7-11.  Some of them didn't apply
cleanly so please check the result in my tree.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCHv3 00/11] crypto: omap HW crypto fixes

[Tero Kristo]

On 13/09/16 15:38, Herbert Xu wrote:
> On Thu, Aug 04, 2016 at 01:28:35PM +0300, Tero Kristo wrote:
>> Hi,
>>
>> This revision took quite a bit time to craft due to the rework needed
>> for sham buffer handling and export/import. I ended up implementing
>> a flush functionality for draining out the sham buffer when doing
>> export/import; just shrinking the buffer to sufficiently small size
>> impacted the performance with small data chunks too much so I dropped
>> this approach.
>>
>> The series also fixes a couple of existing issues with omap2/omap3
>> hardware acceleration, I ran a full boot test / crypto manager
>> test suite on all boards accessible to me now.
>>
>> Based on top of latest mainline, which is somewhere before 4.8-rc1
>> as of writing this, I am unable to rebase the series during the next
>> three weeks so wanted to get this out now. Targeted for 4.9 merge
>> window, some fixes could be picked up earlier though if needed.
>
> I have applied patches 1,4-5,7-11.  Some of them didn't apply
> cleanly so please check the result in my tree.
>
> Thanks,
>

Thanks Herbert,

I just gave a trial for your branch, and seems to be working for me. 
Also checked the patches you applied and they seem fine also.

I have also a new version of the sha buffer handling and export/import 
available now, but need to cleanup it quite a bit, and figure out how to 
split the patch properly.

  drivers/crypto/omap-sham.c | 532 
++++++++++++++++++++++++++-------------------

... It now uses sg for xmitting data where possible, and avoids the need 
for a large internal buffer. The buffer size is also now properly 
configurable, which can be used to overcome the performance issues if 
needed (this however, requires the max statesize hack within the 
crypto/ahash.c file, but thats fine.) I'll hopefully post this out maybe 
tomorrow, but its going to be targeted for 4.10 I believe due to the 
(ahem, rather) intrusive changes.

-Tero