A question on crypto_engine and a possible bug

A question on crypto_engine and a possible bug

[T Pratham]

Hi,

Since the in-kernel APIs are not well documented, I got into a road bump recently with crypto engine APIs while implementing my new driver. So I thought of asking my doubts directly, and reporting a behaviour I saw which I think is not expected and possibly a bug.

So, the do_one_op function registered by the user in *_engine_alg, what is it supposed to return? Seeing the int return type, I assumed it should be 0 for success and error codes if any failure occurs (-EINVAL, -ENOMEM, etc. for appropriate failure). Before returning from this function, we also call crypto_finalize_*_request, and pass the return error code to this as well. So do we return the same error code at both places?

The above doesn't seem to be correct while working with AEADs. I was returning same error code from do_one_op function which was being passed to crypto_finalize_aead_request. This was causing selftests to fail in a peculiar way: a random test for encryption was failing with:
"alg: aead: <driver_name> encryption failed on test vector <test vector>; expected error=0, actual error=-74, cfg=<cfg>"

Now, you may recognize that -74 is the code for EBADMSG, which is only returned in decryption when the authentication tag does not match. My driver cannot return this error in encryption in any case. So the error was not coming from my code for sure.

I did some inspection, added some extra prints in testmgr as well as my code, and a pattern emerged. The failing test was always the encryption test after the first decryption test that tests for failing tag authentication (i.e. the decryption test expects -EBADMSG being returned). This was being propagated by the crypto engine queue to the next request somehow (how? a bug? an edge case?). Crypto engine also prints the log "Failed to do one request from queue: -74" for all decryptions returning -EBADMSG. When I changed the return value of do_one_op function from the error code to 0, suddenly all selftests passed (including all random tests)!

So for AEADs -EBADMSG case, are we supposed to return 0, or return 0 always, or what is the return value crypto engine expects?

Regards
T Pratham <t-pratham@ti.com>

Re: A question on crypto_engine and a possible bug

[Herbert Xu]

On Sat, Sep 13, 2025 at 02:48:11PM +0530, T Pratham wrote:
>
> So, the do_one_op function registered by the user in *_engine_alg, what is
> it supposed to return? Seeing the int return type, I assumed it should be
> 0 for success and error codes if any failure occurs (-EINVAL, -ENOMEM,
> etc. for appropriate failure). Before returning from this function,
> we also call crypto_finalize_*_request, and pass the return error code
> to this as well. So do we return the same error code at both places?

The do_one_op return value is used to represent errors that occur
before or during the submission of the request to hardware, e.g.,
the hardware queue was full.

If you return an error via do_one_op, then the crypto_engine will
carry out the completion for you.

If you returned zero from do_one_op, then the request is yours and
yours only and you must finalize the request when it is complete,
with either 0 or an error value.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: A question on crypto_engine and a possible bug

[T Pratham]

On 17/09/25 09:10, Herbert Xu wrote:
> On Sat, Sep 13, 2025 at 02:48:11PM +0530, T Pratham wrote:
>>
>> So, the do_one_op function registered by the user in *_engine_alg, what is
>> it supposed to return? Seeing the int return type, I assumed it should be
>> 0 for success and error codes if any failure occurs (-EINVAL, -ENOMEM,
>> etc. for appropriate failure). Before returning from this function,
>> we also call crypto_finalize_*_request, and pass the return error code
>> to this as well. So do we return the same error code at both places?
> 
> The do_one_op return value is used to represent errors that occur
> before or during the submission of the request to hardware, e.g.,
> the hardware queue was full.
> 
> If you return an error via do_one_op, then the crypto_engine will
> carry out the completion for you.
> 
> If you returned zero from do_one_op, then the request is yours and
> yours only and you must finalize the request when it is complete,
> with either 0 or an error value.
> 
> Cheers,

Thanks for the response. I'll take care of this in future.

-- 
Regards
T Pratham <t-pratham@ti.com>