* Re: [PATCH 17/29] crypto: talitos/aead - Use macro for algorithm definitions
From: Christophe Leroy (CS GROUP) @ 2026-06-01 12:12 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-17-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Replace the repetitive struct initializer entries in aead_driver_algs[]
> with preprocessor macros (TALITOS_AEAD_ALG, TALITOS_AEAD_ALG_HSNA).
>
> Move the function pointer assignments (init, exit, encrypt, decrypt)
> from the registration loop into the static initializer, since they are
> identical for all algorithms.
>
> The fallback setkey assignment (aead_alg->setkey ?: aead_setkey) is
> replaced by specifying the correct setkey handler directly in each macro
> invocation.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Wondering if we could go even more far with the COMMON flags, as for
instance all TALITOS_AEAD_ALG_HSNA have DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU
while TALITOS_AEAD_ALG have DESC_HDR_TYPE_IPSEC_ESP
> ---
> drivers/crypto/talitos/talitos-aead.c | 751 ++++++++++------------------------
> 1 file changed, 218 insertions(+), 533 deletions(-)
>
> diff --git a/drivers/crypto/talitos/talitos-aead.c b/drivers/crypto/talitos/talitos-aead.c
> index 38df616c9b22..cd1b8e6d371b 100644
> --- a/drivers/crypto/talitos/talitos-aead.c
> +++ b/drivers/crypto/talitos/talitos-aead.c
> @@ -405,535 +405,225 @@ static void talitos_cra_exit_aead(struct crypto_aead *tfm)
> talitos_cra_exit(crypto_aead_tfm(tfm));
> }
>
> +#define TALITOS_AEAD_ALG_COMMON(name, name_prefix, set_key, block_size, \
> + max_auth_size, template, priority) \
> + { \
> + .type = CRYPTO_ALG_TYPE_AEAD, \
> + .alg.aead = { \
> + .base = { \
> + .cra_name = name, \
> + .cra_driver_name = name"-talitos"name_prefix, \
> + .cra_blocksize = block_size, \
> + .cra_flags = CRYPTO_ALG_ASYNC | \
> + CRYPTO_ALG_ALLOCATES_MEMORY | \
> + CRYPTO_ALG_KERN_DRIVER_ONLY, \
> + .cra_priority = (priority), \
> + .cra_ctxsize = sizeof(struct talitos_ctx), \
> + .cra_module = THIS_MODULE, \
> + }, \
> + .ivsize = block_size, \
> + .maxauthsize = max_auth_size, \
> + .setkey = set_key, \
> + .init = talitos_cra_init_aead, \
> + .exit = talitos_cra_exit_aead, \
> + .encrypt = aead_encrypt, \
> + .decrypt = aead_decrypt, \
> + }, \
> + .desc_hdr_template = template, \
> + }
> +
> +#define TALITOS_AEAD_ALG(name, set_key, block_size, max_auth_size, template) \
> + TALITOS_AEAD_ALG_COMMON(name, "", set_key, block_size, max_auth_size, \
> + template, TALITOS_CRA_PRIORITY)
> +
> +#define TALITOS_AEAD_ALG_HSNA(name, set_key, block_size, max_auth_size, \
> + template) \
> + TALITOS_AEAD_ALG_COMMON(name, "-hsna", set_key, block_size, \
> + max_auth_size, template, \
> + TALITOS_CRA_PRIORITY_AEAD_HSNA)
> +
> static struct talitos_alg_template aead_driver_algs[] = {
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha384),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha384-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA384_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA384_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha384),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha384-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA384_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA384_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha512),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha512-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA512_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA512_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha512),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha512-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA512_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA512_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> + /* AEAD algorithms. These use a single-pass ipsec_esp descriptor */
> +
> + /* sha1 auth */
> +
> + TALITOS_AEAD_ALG("authenc(hmac(sha1),cbc(aes))", aead_setkey,
> + AES_BLOCK_SIZE, SHA1_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(sha1),cbc(aes))", aead_setkey, AES_BLOCK_SIZE,
> + SHA1_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT | DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC),
> +
> + TALITOS_AEAD_ALG("authenc(hmac(sha1),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA1_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(sha1),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA1_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC),
> +
> + /* sha224 auth */
> +
> + TALITOS_AEAD_ALG("authenc(hmac(sha224),cbc(aes))", aead_setkey,
> + AES_BLOCK_SIZE, SHA224_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(sha224),cbc(aes))", aead_setkey, AES_BLOCK_SIZE,
> + SHA224_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT | DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC),
> +
> + TALITOS_AEAD_ALG(
> + "authenc(hmac(sha224),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA224_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(sha224),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA224_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC),
> +
> + /* sha256 auth */
> +
> + TALITOS_AEAD_ALG("authenc(hmac(sha256),cbc(aes))", aead_setkey,
> + AES_BLOCK_SIZE, SHA256_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(sha256),cbc(aes))", aead_setkey, AES_BLOCK_SIZE,
> + SHA256_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT | DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC),
> +
> + TALITOS_AEAD_ALG(
> + "authenc(hmac(sha256),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA256_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(sha256),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA256_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC),
> +
> + /* sha384 auth */
> +
> + TALITOS_AEAD_ALG("authenc(hmac(sha384),cbc(aes))", aead_setkey,
> + AES_BLOCK_SIZE, SHA384_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUB |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA384_HMAC),
> +
> + TALITOS_AEAD_ALG(
> + "authenc(hmac(sha384),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA384_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUB | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA384_HMAC),
> +
> + /* sha512 auth */
> +
> + TALITOS_AEAD_ALG("authenc(hmac(sha512),cbc(aes))", aead_setkey,
> + AES_BLOCK_SIZE, SHA512_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUB |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA512_HMAC),
> +
> + TALITOS_AEAD_ALG(
> + "authenc(hmac(sha512),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, SHA512_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUB | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA512_HMAC),
> +
> + /* md5 auth */
> +
> + TALITOS_AEAD_ALG("authenc(hmac(md5),cbc(aes))", aead_setkey,
> + AES_BLOCK_SIZE, MD5_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_MD5_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(md5),cbc(aes))", aead_setkey, AES_BLOCK_SIZE,
> + MD5_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC | DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT | DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_MD5_HMAC),
> +
> + TALITOS_AEAD_ALG(
> + "authenc(hmac(md5),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, MD5_DIGEST_SIZE,
> + DESC_HDR_TYPE_IPSEC_ESP | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD | DESC_HDR_MODE1_MDEU_MD5_HMAC),
> +
> + TALITOS_AEAD_ALG_HSNA(
> + "authenc(hmac(md5),cbc(des3_ede))", aead_des3_setkey,
> + DES3_EDE_BLOCK_SIZE, MD5_DIGEST_SIZE,
> + DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA | DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD | DESC_HDR_MODE1_MDEU_MD5_HMAC),
> };
>
> int talitos_register_aead(struct device *dev)
> @@ -955,11 +645,6 @@ int talitos_register_aead(struct device *dev)
> if (has_ftr_sec1(priv))
> alg->cra_alignmask = 3;
>
> - aead_alg->init = talitos_cra_init_aead;
> - aead_alg->exit = talitos_cra_exit_aead;
> - aead_alg->setkey = aead_alg->setkey ?: aead_setkey;
> - aead_alg->encrypt = aead_encrypt;
> - aead_alg->decrypt = aead_decrypt;
> if (!(priv->features & TALITOS_FTR_SHA224_HWINIT) &&
> !strncmp(alg->cra_name, "authenc(hmac(sha224)", 20)) {
> continue;
>
^ permalink raw reply
* Re: [PATCH 16/29] crypto: talitos/skcipher - Use macro for algorithm definitions
From: Christophe Leroy (CS GROUP) @ 2026-06-01 12:02 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-16-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Replace the repetitive struct initializer entries in
> skcipher_driver_algs[] with preprocessor macros
> (TALITOS_SKCIPHER_ALG_AES, TALITOS_SKCIPHER_ALG_DES,
> TALITOS_SKCIPHER_ALG_DES3).
>
> Move the function pointer assignments (init, exit, encrypt, decrypt)
> from the registration loop into the static initializer, since they are
> identical for all algorithms.
>
> The fallback setkey assignment (skcipher_alg->setkey ?: skcipher_setkey)
> is no longer needed because each macro specifies the correct setkey
> handler directly.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/talitos-skcipher.c | 244 ++++++++++--------------------
> 1 file changed, 82 insertions(+), 162 deletions(-)
>
> diff --git a/drivers/crypto/talitos/talitos-skcipher.c b/drivers/crypto/talitos/talitos-skcipher.c
> index f86a0a9a0ffe..b12191243aae 100644
> --- a/drivers/crypto/talitos/talitos-skcipher.c
> +++ b/drivers/crypto/talitos/talitos-skcipher.c
> @@ -237,163 +237,89 @@ static void talitos_cra_exit_skcipher(struct crypto_skcipher *tfm)
> talitos_cra_exit(crypto_skcipher_tfm(tfm));
> }
>
> +#define TALITOS_SKCIPHER_ALG_COMMON(name, blk_sz, iv_sz, min_ksz, max_ksz, \
> + set_key, desc_template) \
> + { \
> + .type = CRYPTO_ALG_TYPE_SKCIPHER, \
> + .alg.skcipher = { \
> + .base.cra_name = name, \
> + .base.cra_driver_name = name"-talitos", \
> + .base.cra_blocksize = blk_sz, \
> + .base.cra_flags = CRYPTO_ALG_ASYNC | \
> + CRYPTO_ALG_ALLOCATES_MEMORY | \
> + CRYPTO_ALG_KERN_DRIVER_ONLY, \
> + .base.cra_priority = TALITOS_CRA_PRIORITY, \
> + .base.cra_ctxsize = sizeof(struct talitos_ctx), \
> + .base.cra_module = THIS_MODULE, \
> + .min_keysize = min_ksz, \
> + .max_keysize = max_ksz, \
> + .ivsize = iv_sz, \
> + .setkey = set_key, \
> + .init = talitos_cra_init_skcipher, \
> + .exit = talitos_cra_exit_skcipher, \
> + .encrypt = skcipher_encrypt, \
> + .decrypt = skcipher_decrypt, \
> + }, \
> + .desc_hdr_template = desc_template, \
> + }
> +
> +#define TALITOS_SKCIPHER_ALG_AES(name, blk_sz, iv_sz, desc_template) \
> + TALITOS_SKCIPHER_ALG_COMMON(name, blk_sz, iv_sz, AES_MIN_KEY_SIZE, \
> + AES_MAX_KEY_SIZE, skcipher_aes_setkey, \
> + desc_template)
> +
> +#define TALITOS_SKCIPHER_ALG_DES(name, blk_sz, iv_sz, desc_template) \
> + TALITOS_SKCIPHER_ALG_COMMON(name, blk_sz, iv_sz, DES_KEY_SIZE, \
> + DES_KEY_SIZE, skcipher_des_setkey, \
> + desc_template)
> +
> +#define TALITOS_SKCIPHER_ALG_DES3(name, blk_sz, iv_sz, desc_template) \
> + TALITOS_SKCIPHER_ALG_COMMON(name, blk_sz, iv_sz, DES3_EDE_KEY_SIZE, \
> + DES3_EDE_KEY_SIZE, skcipher_des3_setkey, \
> + desc_template)
> +
> static struct talitos_alg_template skcipher_driver_algs[] = {
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ecb(aes)",
> - .base.cra_driver_name = "ecb-aes-talitos",
> - .base.cra_blocksize = AES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "cbc(aes)",
> - .base.cra_driver_name = "cbc-aes-talitos",
> - .base.cra_blocksize = AES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .ivsize = AES_BLOCK_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ctr(aes)",
> - .base.cra_driver_name = "ctr-aes-talitos",
> - .base.cra_blocksize = 1,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .ivsize = AES_BLOCK_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_AESU_CTR_NONSNOOP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CTR,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ctr(aes)",
> - .base.cra_driver_name = "ctr-aes-talitos",
> - .base.cra_blocksize = 1,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .ivsize = AES_BLOCK_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CTR,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ecb(des)",
> - .base.cra_driver_name = "ecb-des-talitos",
> - .base.cra_blocksize = DES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = DES_KEY_SIZE,
> - .max_keysize = DES_KEY_SIZE,
> - .setkey = skcipher_des_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "cbc(des)",
> - .base.cra_driver_name = "cbc-des-talitos",
> - .base.cra_blocksize = DES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = DES_KEY_SIZE,
> - .max_keysize = DES_KEY_SIZE,
> - .ivsize = DES_BLOCK_SIZE,
> - .setkey = skcipher_des_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ecb(des3_ede)",
> - .base.cra_driver_name = "ecb-3des-talitos",
> - .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = DES3_EDE_KEY_SIZE,
> - .max_keysize = DES3_EDE_KEY_SIZE,
> - .setkey = skcipher_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_3DES,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "cbc(des3_ede)",
> - .base.cra_driver_name = "cbc-3des-talitos",
> - .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY,
> - .base.cra_priority = TALITOS_CRA_PRIORITY,
> - .base.cra_ctxsize = sizeof(struct talitos_ctx),
> - .base.cra_module = THIS_MODULE,
> - .min_keysize = DES3_EDE_KEY_SIZE,
> - .max_keysize = DES3_EDE_KEY_SIZE,
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .setkey = skcipher_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES,
> - },
> + /* AES */
> +
> + TALITOS_SKCIPHER_ALG_AES("ecb(aes)", AES_BLOCK_SIZE, 0,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU),
> +
> + TALITOS_SKCIPHER_ALG_AES("cbc(aes)", AES_BLOCK_SIZE, AES_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC),
> +
> + TALITOS_SKCIPHER_ALG_AES("ctr(aes)", 1, AES_BLOCK_SIZE,
> + DESC_HDR_TYPE_AESU_CTR_NONSNOOP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CTR),
> +
> + TALITOS_SKCIPHER_ALG_AES("ctr(aes)", 1, AES_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CTR),
> + /* DES */
> +
> + TALITOS_SKCIPHER_ALG_DES("ecb(des)", DES_BLOCK_SIZE, 0,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU),
> +
> + TALITOS_SKCIPHER_ALG_DES("cbc(des)", DES_BLOCK_SIZE, DES_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC),
> + /* DES3 */
> +
> + TALITOS_SKCIPHER_ALG_DES3("ecb(des3_ede)", DES3_EDE_BLOCK_SIZE, 0,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_3DES),
> +
> + TALITOS_SKCIPHER_ALG_DES3(
> + "cbc(des3_ede)", DES3_EDE_BLOCK_SIZE, DES3_EDE_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU | DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC | DESC_HDR_MODE0_DEU_3DES),
> };
>
> int talitos_register_skcipher(struct device *dev)
> @@ -415,12 +341,6 @@ int talitos_register_skcipher(struct device *dev)
> if (has_ftr_sec1(priv))
> alg->cra_alignmask = 3;
>
> - skcipher_alg->init = talitos_cra_init_skcipher;
> - skcipher_alg->exit = talitos_cra_exit_skcipher;
> - skcipher_alg->setkey = skcipher_alg->setkey ?: skcipher_setkey;
> - skcipher_alg->encrypt = skcipher_encrypt;
> - skcipher_alg->decrypt = skcipher_decrypt;
> -
> if (!strcmp(alg->cra_name, "ctr(aes)") && !has_ftr_sec1(priv) &&
> DESC_TYPE(skcipher_driver_algs[i].desc_hdr_template) !=
> DESC_TYPE(DESC_HDR_TYPE_AESU_CTR_NONSNOOP)) {
>
^ permalink raw reply
* Re: [PATCH 15/29] crypto: talitos/hash - Use macro for algorithm definitions
From: Christophe Leroy (CS GROUP) @ 2026-06-01 12:02 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-15-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Replace the repetitive struct initializer entries in hash_driver_algs[]
> with preprocessor macros (TALITOS_HASH_ALG, TALITOS_HMAC_HASH_ALG).
>
> Remove the function pointer assignments (init_tfm, exit_tfm, init, update,
> final, finup, digest, export, import).
>
> The HMAC setkey assignment, previously done by comparing the algorithm
> name at runtime, is now handled by passing ahash_setkey directly through
> the TALITOS_HMAC_HASH_ALG macro variant.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/talitos-hash.c | 392 +++++++++-------------------------
> 1 file changed, 104 insertions(+), 288 deletions(-)
>
> diff --git a/drivers/crypto/talitos/talitos-hash.c b/drivers/crypto/talitos/talitos-hash.c
> index f7f6f01cfddf..9e6d849c3123 100644
> --- a/drivers/crypto/talitos/talitos-hash.c
> +++ b/drivers/crypto/talitos/talitos-hash.c
> @@ -551,283 +551,111 @@ static void talitos_cra_exit_ahash(struct crypto_ahash *tfm)
> talitos_cra_exit(crypto_ahash_tfm(tfm));
> }
>
> -static struct talitos_alg_template hash_driver_algs[] = {
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = MD5_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "md5",
> - .cra_driver_name = "md5-talitos",
> - .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_MD5,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA1_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha1",
> - .cra_driver_name = "sha1-talitos",
> - .cra_blocksize = SHA1_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA1,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA224_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha224",
> - .cra_driver_name = "sha224-talitos",
> - .cra_blocksize = SHA224_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA224,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA256_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha256",
> - .cra_driver_name = "sha256-talitos",
> - .cra_blocksize = SHA256_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA256,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA384_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha384",
> - .cra_driver_name = "sha384-talitos",
> - .cra_blocksize = SHA384_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA384,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA512_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha512",
> - .cra_driver_name = "sha512-talitos",
> - .cra_blocksize = SHA512_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA512,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = MD5_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(md5)",
> - .cra_driver_name = "hmac-md5-talitos",
> - .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_MD5,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA1_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha1)",
> - .cra_driver_name = "hmac-sha1-talitos",
> - .cra_blocksize = SHA1_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA1,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA224_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha224)",
> - .cra_driver_name = "hmac-sha224-talitos",
> - .cra_blocksize = SHA224_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA224,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA256_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha256)",
> - .cra_driver_name = "hmac-sha256-talitos",
> - .cra_blocksize = SHA256_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA256,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA384_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha384)",
> - .cra_driver_name = "hmac-sha384-talitos",
> - .cra_blocksize = SHA384_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA384,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA512_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha512)",
> - .cra_driver_name = "hmac-sha512-talitos",
> - .cra_blocksize = SHA512_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_ALG_KERN_DRIVER_ONLY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - .cra_priority = TALITOS_CRA_PRIORITY,
> - .cra_ctxsize = sizeof(struct talitos_ctx),
> - .cra_module = THIS_MODULE,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA512,
> +#define TALITOS_HASH_ALG_COMMON(name, digest_size, block_size, template, \
> + set_key) \
> + { \
> + .type = CRYPTO_ALG_TYPE_AHASH, \
> + .alg.hash = { \
> + .init_tfm = talitos_cra_init_ahash, \
> + .exit_tfm = talitos_cra_exit_ahash, \
> + .init = ahash_init, \
> + .update = ahash_update, \
> + .final = ahash_final, \
> + .finup = ahash_finup, \
> + .digest = ahash_digest, \
> + .setkey = set_key, \
> + .import = ahash_import, \
> + .export = ahash_export, \
> + .halg.digestsize = digest_size, \
> + .halg.statesize = sizeof(struct talitos_export_state), \
> + .halg.base = { \
> + .cra_name = name, \
> + .cra_driver_name = name"-talitos", \
> + .cra_blocksize = block_size, \
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx), \
> + .cra_flags = CRYPTO_ALG_ASYNC | \
> + CRYPTO_ALG_ALLOCATES_MEMORY | \
> + CRYPTO_ALG_KERN_DRIVER_ONLY | \
> + CRYPTO_AHASH_ALG_BLOCK_ONLY | \
> + CRYPTO_AHASH_ALG_FINAL_NONZERO, \
> + .cra_priority = TALITOS_CRA_PRIORITY, \
> + .cra_ctxsize = sizeof(struct talitos_ctx), \
> + .cra_module = THIS_MODULE, \
> + }, \
> + }, \
> + .desc_hdr_template = template, \
> }
> +
> +#define TALITOS_HASH_ALG(name, digest_size, block_size, desc_hdr_template) \
> + TALITOS_HASH_ALG_COMMON(name, digest_size, block_size, \
> + desc_hdr_template, NULL)
> +
> +#define TALITOS_HMAC_HASH_ALG(name, digest_size, block_size, \
> + desc_hdr_template) \
> + TALITOS_HASH_ALG_COMMON("hmac(" name ")", digest_size, block_size, \
> + desc_hdr_template, ahash_setkey)
> +
> +static struct talitos_alg_template hash_driver_algs[] = {
> + TALITOS_HASH_ALG("md5", MD5_DIGEST_SIZE, MD5_HMAC_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA | DESC_HDR_MODE0_MDEU_MD5),
> +
> + TALITOS_HASH_ALG("sha1", SHA1_DIGEST_SIZE, SHA1_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA1),
> +
> + TALITOS_HASH_ALG("sha224", SHA224_DIGEST_SIZE, SHA224_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA224),
> +
> + TALITOS_HASH_ALG("sha256", SHA256_DIGEST_SIZE, SHA256_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA256),
> +
> + TALITOS_HASH_ALG("sha384", SHA384_DIGEST_SIZE, SHA384_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA384),
> +
> + TALITOS_HASH_ALG("sha512", SHA512_DIGEST_SIZE, SHA512_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA512),
> +
> + /* HMAC */
> +
> + TALITOS_HMAC_HASH_ALG("md5", MD5_DIGEST_SIZE, MD5_HMAC_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_MD5),
> +
> + TALITOS_HMAC_HASH_ALG("sha1", SHA1_DIGEST_SIZE, SHA1_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA1),
> +
> + TALITOS_HMAC_HASH_ALG("sha224", SHA224_DIGEST_SIZE, SHA224_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA224),
> +
> + TALITOS_HMAC_HASH_ALG("sha256", SHA256_DIGEST_SIZE, SHA256_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA256),
> +
> + TALITOS_HMAC_HASH_ALG("sha384", SHA384_DIGEST_SIZE, SHA384_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA384),
> +
> + TALITOS_HMAC_HASH_ALG("sha512", SHA512_DIGEST_SIZE, SHA512_BLOCK_SIZE,
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA512),
> };
>
> int talitos_register_hash(struct device *dev)
> @@ -846,18 +674,6 @@ int talitos_register_hash(struct device *dev)
> ahash_alg = &hash_driver_algs[i].alg.hash;
> alg = &ahash_alg->halg.base;
>
> - ahash_alg->init_tfm = talitos_cra_init_ahash;
> - ahash_alg->exit_tfm = talitos_cra_exit_ahash;
> - ahash_alg->init = ahash_init;
> - ahash_alg->update = ahash_update;
> - ahash_alg->final = ahash_final;
> - ahash_alg->finup = ahash_finup;
> - ahash_alg->digest = ahash_digest;
> - if (!strncmp(alg->cra_name, "hmac", 4))
> - ahash_alg->setkey = ahash_setkey;
> - ahash_alg->import = ahash_import;
> - ahash_alg->export = ahash_export;
> -
> if (!(priv->features & TALITOS_FTR_HMAC_OK) &&
> !strncmp(alg->cra_name, "hmac", 4)) {
> /* not supported */
>
^ permalink raw reply
* Re: [PATCH 14/29] crypto: talitos/aead - Convert to init/exit type-specific API
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:59 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-14-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Since commit 6eed1e3552fc0 ("crypto: api - Mark cra_init/cra_exit as
> deprecated"), both cra_{init,exit} are deprecated.
>
> Restore the type-specific talitos_cra_exit_aead() wrapper and use
> aead_alg->exit instead of the generic cra_exit field, matching the
> pattern used by init.
When you say "restore", do you mean it was removed at some point in the
past ?
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/talitos-aead.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/crypto/talitos/talitos-aead.c b/drivers/crypto/talitos/talitos-aead.c
> index c09ed08be2ef..38df616c9b22 100644
> --- a/drivers/crypto/talitos/talitos-aead.c
> +++ b/drivers/crypto/talitos/talitos-aead.c
> @@ -400,6 +400,11 @@ static int talitos_cra_init_aead(struct crypto_aead *tfm)
> return talitos_init_common(ctx, talitos_alg);
> }
>
> +static void talitos_cra_exit_aead(struct crypto_aead *tfm)
> +{
> + talitos_cra_exit(crypto_aead_tfm(tfm));
> +}
> +
> static struct talitos_alg_template aead_driver_algs[] = {
> { .type = CRYPTO_ALG_TYPE_AEAD,
> .alg.aead = {
> @@ -950,8 +955,8 @@ int talitos_register_aead(struct device *dev)
> if (has_ftr_sec1(priv))
> alg->cra_alignmask = 3;
>
> - alg->cra_exit = talitos_cra_exit;
> aead_alg->init = talitos_cra_init_aead;
> + aead_alg->exit = talitos_cra_exit_aead;
> aead_alg->setkey = aead_alg->setkey ?: aead_setkey;
> aead_alg->encrypt = aead_encrypt;
> aead_alg->decrypt = aead_decrypt;
>
^ permalink raw reply
* Re: [PATCH 13/29] crypto: talitos/skcipher - Convert to init/exit type-specific API
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:58 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-13-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Since commit 6eed1e3552fc0 ("crypto: api - Mark cra_init/cra_exit as
> deprecated"), both cra_{init,exit} are deprecated.
>
> Restore the type-specific talitos_cra_exit_skcipher() wrapper and use
> skcipher_alg->exit instead of the generic cra_exit field, matching the
> pattern used by init.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/talitos-skcipher.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/crypto/talitos/talitos-skcipher.c b/drivers/crypto/talitos/talitos-skcipher.c
> index ff7b8f9344c4..f86a0a9a0ffe 100644
> --- a/drivers/crypto/talitos/talitos-skcipher.c
> +++ b/drivers/crypto/talitos/talitos-skcipher.c
> @@ -232,6 +232,11 @@ static int talitos_cra_init_skcipher(struct crypto_skcipher *tfm)
> return talitos_init_common(ctx, talitos_alg);
> }
>
> +static void talitos_cra_exit_skcipher(struct crypto_skcipher *tfm)
> +{
> + talitos_cra_exit(crypto_skcipher_tfm(tfm));
> +}
> +
> static struct talitos_alg_template skcipher_driver_algs[] = {
> { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> .alg.skcipher = {
> @@ -410,8 +415,8 @@ int talitos_register_skcipher(struct device *dev)
> if (has_ftr_sec1(priv))
> alg->cra_alignmask = 3;
>
> - alg->cra_exit = talitos_cra_exit;
> skcipher_alg->init = talitos_cra_init_skcipher;
> + skcipher_alg->exit = talitos_cra_exit_skcipher;
> skcipher_alg->setkey = skcipher_alg->setkey ?: skcipher_setkey;
> skcipher_alg->encrypt = skcipher_encrypt;
> skcipher_alg->decrypt = skcipher_decrypt;
>
^ permalink raw reply
* Re: [PATCH 12/29] crypto: talitos/hash - Convert to init_tfm/exit_tfm type-specific API
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:57 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-12-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Since commit 6eed1e3552fc0 ("crypto: api - Mark cra_init/cra_exit as
> deprecated"), both cra_{init,exit} are deprecated.
>
> Switch hash from the deprecated cra_init/cra_exit fields on crypto_alg
> to the preferred init_tfm/exit_tfm fields on ahash_alg.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/talitos-hash.c | 18 +++++++++++-------
> 1 file changed, 11 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/crypto/talitos/talitos-hash.c b/drivers/crypto/talitos/talitos-hash.c
> index 3793b6fd5b75..f7f6f01cfddf 100644
> --- a/drivers/crypto/talitos/talitos-hash.c
> +++ b/drivers/crypto/talitos/talitos-hash.c
> @@ -531,22 +531,26 @@ static int ahash_setkey(struct crypto_ahash *tfm, const u8 *key,
> return 0;
> }
>
> -static int talitos_cra_init_ahash(struct crypto_tfm *tfm)
> +static int talitos_cra_init_ahash(struct crypto_ahash *tfm)
> {
> - struct crypto_alg *alg = tfm->__crt_alg;
> + struct ahash_alg *alg = crypto_ahash_alg(tfm);
> struct talitos_crypto_alg *talitos_alg;
> - struct talitos_ctx *ctx = crypto_tfm_ctx(tfm);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
>
> - talitos_alg = container_of(__crypto_ahash_alg(alg),
> + talitos_alg = container_of(alg,
> struct talitos_crypto_alg,
> algt.alg.hash);
>
> ctx->keylen = 0;
> - sizeof(struct talitos_ahash_req_ctx));
Should be in patch 1 ?
>
> return talitos_init_common(ctx, talitos_alg);
> }
>
> +static void talitos_cra_exit_ahash(struct crypto_ahash *tfm)
> +{
> + talitos_cra_exit(crypto_ahash_tfm(tfm));
> +}
> +
> static struct talitos_alg_template hash_driver_algs[] = {
> { .type = CRYPTO_ALG_TYPE_AHASH,
> .alg.hash = {
> @@ -842,8 +846,8 @@ int talitos_register_hash(struct device *dev)
> ahash_alg = &hash_driver_algs[i].alg.hash;
> alg = &ahash_alg->halg.base;
>
> - alg->cra_init = talitos_cra_init_ahash;
> - alg->cra_exit = talitos_cra_exit;
> + ahash_alg->init_tfm = talitos_cra_init_ahash;
> + ahash_alg->exit_tfm = talitos_cra_exit_ahash;
> ahash_alg->init = ahash_init;
> ahash_alg->update = ahash_update;
> ahash_alg->final = ahash_final;
>
^ permalink raw reply
* Re: [PATCH 11/29] crypto: talitos - Remove unused priority field in struct talitos_alg_template
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:54 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-11-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> After algorithm properties are now set at definition time, the priority
> field in struct talitos_alg_template is no longer used. Remove it.
Should probably be sqashed (with the above explanation) in the previous
commit.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
> ---
> drivers/crypto/talitos/talitos.h | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/drivers/crypto/talitos/talitos.h b/drivers/crypto/talitos/talitos.h
> index 438be8c8f08d..6cf3628c52c2 100644
> --- a/drivers/crypto/talitos/talitos.h
> +++ b/drivers/crypto/talitos/talitos.h
> @@ -203,7 +203,6 @@ struct talitos_ctx {
>
> struct talitos_alg_template {
> u32 type;
> - u32 priority;
> union {
> struct skcipher_alg skcipher;
> struct ahash_alg hash;
>
^ permalink raw reply
* Re: [PATCH 10/29] crypto: talitos - Remove alg settings in talitos_register_common()
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:53 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-10-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Algorithm properties should be set at definition time.
Can you provide more details on why it _should_ be set at definition time ?
Also, couldn't this change be done after the "Use macro for algorithm
definitions" patches in order to minimise churn ?
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
> ---
> drivers/crypto/talitos/talitos-aead.c | 131 +++++++++++++++++++++++-------
> drivers/crypto/talitos/talitos-hash.c | 72 +++++++++++++---
> drivers/crypto/talitos/talitos-skcipher.c | 51 ++++++++++--
> drivers/crypto/talitos/talitos.c | 23 ------
> 4 files changed, 206 insertions(+), 71 deletions(-)
>
> diff --git a/drivers/crypto/talitos/talitos-aead.c b/drivers/crypto/talitos/talitos-aead.c
> index ce6bd6133fd0..c09ed08be2ef 100644
> --- a/drivers/crypto/talitos/talitos-aead.c
> +++ b/drivers/crypto/talitos/talitos-aead.c
> @@ -409,7 +409,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA1_DIGEST_SIZE,
> @@ -423,7 +427,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(sha1),cbc(aes))",
> @@ -431,7 +434,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos-hsna",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA1_DIGEST_SIZE,
> @@ -453,7 +460,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA1_DIGEST_SIZE,
> @@ -469,7 +480,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(sha1),"
> @@ -478,7 +488,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos-hsna",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA1_DIGEST_SIZE,
> @@ -501,7 +515,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA224_DIGEST_SIZE,
> @@ -515,7 +533,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(sha224),cbc(aes))",
> @@ -523,7 +540,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos-hsna",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA224_DIGEST_SIZE,
> @@ -545,7 +566,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA224_DIGEST_SIZE,
> @@ -561,7 +586,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(sha224),"
> @@ -570,7 +594,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos-hsna",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA224_DIGEST_SIZE,
> @@ -593,7 +621,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA256_DIGEST_SIZE,
> @@ -607,7 +639,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(sha256),cbc(aes))",
> @@ -615,7 +646,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos-hsna",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA256_DIGEST_SIZE,
> @@ -637,7 +672,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA256_DIGEST_SIZE,
> @@ -653,7 +692,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(sha256),"
> @@ -662,7 +700,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos-hsna",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA256_DIGEST_SIZE,
> @@ -685,7 +727,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA384_DIGEST_SIZE,
> @@ -707,7 +753,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA384_DIGEST_SIZE,
> @@ -730,7 +780,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = SHA512_DIGEST_SIZE,
> @@ -752,7 +806,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = SHA512_DIGEST_SIZE,
> @@ -775,7 +833,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = MD5_DIGEST_SIZE,
> @@ -789,7 +851,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_MD5_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(md5),cbc(aes))",
> @@ -797,7 +858,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-aes-talitos-hsna",
> .cra_blocksize = AES_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = AES_BLOCK_SIZE,
> .maxauthsize = MD5_DIGEST_SIZE,
> @@ -818,7 +883,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = MD5_DIGEST_SIZE,
> @@ -834,7 +903,6 @@ static struct talitos_alg_template aead_driver_algs[] = {
> DESC_HDR_MODE1_MDEU_MD5_HMAC,
> },
> { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> .alg.aead = {
> .base = {
> .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> @@ -842,7 +910,11 @@ static struct talitos_alg_template aead_driver_algs[] = {
> "cbc-3des-talitos-hsna",
> .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .cra_priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> },
> .ivsize = DES3_EDE_BLOCK_SIZE,
> .maxauthsize = MD5_DIGEST_SIZE,
> @@ -875,6 +947,9 @@ int talitos_register_aead(struct device *dev)
> aead_alg = &aead_driver_algs[i].alg.aead;
> alg = &aead_alg->base;
>
> + if (has_ftr_sec1(priv))
> + alg->cra_alignmask = 3;
> +
> alg->cra_exit = talitos_cra_exit;
> aead_alg->init = talitos_cra_init_aead;
> aead_alg->setkey = aead_alg->setkey ?: aead_setkey;
> diff --git a/drivers/crypto/talitos/talitos-hash.c b/drivers/crypto/talitos/talitos-hash.c
> index 5792e7093392..3793b6fd5b75 100644
> --- a/drivers/crypto/talitos/talitos-hash.c
> +++ b/drivers/crypto/talitos/talitos-hash.c
> @@ -559,8 +559,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -578,8 +582,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -597,8 +605,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -616,8 +628,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -635,8 +651,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -654,8 +674,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -673,8 +697,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -692,8 +720,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -711,8 +743,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -730,8 +766,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -749,8 +789,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> @@ -768,8 +812,12 @@ static struct talitos_alg_template hash_driver_algs[] = {
> .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> .cra_flags = CRYPTO_ALG_ASYNC |
> CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + .cra_priority = TALITOS_CRA_PRIORITY,
> + .cra_ctxsize = sizeof(struct talitos_ctx),
> + .cra_module = THIS_MODULE,
> }
> },
> .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> diff --git a/drivers/crypto/talitos/talitos-skcipher.c b/drivers/crypto/talitos/talitos-skcipher.c
> index 4f742930ec47..ff7b8f9344c4 100644
> --- a/drivers/crypto/talitos/talitos-skcipher.c
> +++ b/drivers/crypto/talitos/talitos-skcipher.c
> @@ -239,7 +239,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "ecb-aes-talitos",
> .base.cra_blocksize = AES_BLOCK_SIZE,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = AES_MIN_KEY_SIZE,
> .max_keysize = AES_MAX_KEY_SIZE,
> .setkey = skcipher_aes_setkey,
> @@ -253,7 +257,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "cbc-aes-talitos",
> .base.cra_blocksize = AES_BLOCK_SIZE,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = AES_MIN_KEY_SIZE,
> .max_keysize = AES_MAX_KEY_SIZE,
> .ivsize = AES_BLOCK_SIZE,
> @@ -269,7 +277,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "ctr-aes-talitos",
> .base.cra_blocksize = 1,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = AES_MIN_KEY_SIZE,
> .max_keysize = AES_MAX_KEY_SIZE,
> .ivsize = AES_BLOCK_SIZE,
> @@ -285,7 +297,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "ctr-aes-talitos",
> .base.cra_blocksize = 1,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = AES_MIN_KEY_SIZE,
> .max_keysize = AES_MAX_KEY_SIZE,
> .ivsize = AES_BLOCK_SIZE,
> @@ -301,7 +317,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "ecb-des-talitos",
> .base.cra_blocksize = DES_BLOCK_SIZE,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = DES_KEY_SIZE,
> .max_keysize = DES_KEY_SIZE,
> .setkey = skcipher_des_setkey,
> @@ -315,7 +335,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "cbc-des-talitos",
> .base.cra_blocksize = DES_BLOCK_SIZE,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = DES_KEY_SIZE,
> .max_keysize = DES_KEY_SIZE,
> .ivsize = DES_BLOCK_SIZE,
> @@ -331,7 +355,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "ecb-3des-talitos",
> .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = DES3_EDE_KEY_SIZE,
> .max_keysize = DES3_EDE_KEY_SIZE,
> .setkey = skcipher_des3_setkey,
> @@ -346,7 +374,11 @@ static struct talitos_alg_template skcipher_driver_algs[] = {
> .base.cra_driver_name = "cbc-3des-talitos",
> .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_ALG_KERN_DRIVER_ONLY,
> + .base.cra_priority = TALITOS_CRA_PRIORITY,
> + .base.cra_ctxsize = sizeof(struct talitos_ctx),
> + .base.cra_module = THIS_MODULE,
> .min_keysize = DES3_EDE_KEY_SIZE,
> .max_keysize = DES3_EDE_KEY_SIZE,
> .ivsize = DES3_EDE_BLOCK_SIZE,
> @@ -375,6 +407,9 @@ int talitos_register_skcipher(struct device *dev)
> skcipher_alg = &skcipher_driver_algs[i].alg.skcipher;
> alg = &skcipher_alg->base;
>
> + if (has_ftr_sec1(priv))
> + alg->cra_alignmask = 3;
> +
> alg->cra_exit = talitos_cra_exit;
> skcipher_alg->init = talitos_cra_init_skcipher;
> skcipher_alg->setkey = skcipher_alg->setkey ?: skcipher_setkey;
> diff --git a/drivers/crypto/talitos/talitos.c b/drivers/crypto/talitos/talitos.c
> index 41d7d0e570e3..f38a156a0459 100644
> --- a/drivers/crypto/talitos/talitos.c
> +++ b/drivers/crypto/talitos/talitos.c
> @@ -1133,23 +1133,6 @@ static void talitos_remove(struct platform_device *ofdev)
> tasklet_kill(&priv->done_task[1]);
> }
>
> -static void talitos_alg_set_common(struct talitos_private *priv,
> - struct crypto_alg *alg, u32 custom_priority,
> - u32 type)
> -{
> - alg->cra_module = THIS_MODULE;
> - if (custom_priority)
> - alg->cra_priority = custom_priority;
> - else
> - alg->cra_priority = TALITOS_CRA_PRIORITY;
> - if (has_ftr_sec1(priv) && type != CRYPTO_ALG_TYPE_AHASH)
> - alg->cra_alignmask = 3;
> - else
> - alg->cra_alignmask = 0;
> - alg->cra_ctxsize = sizeof(struct talitos_ctx);
> - alg->cra_flags |= CRYPTO_ALG_KERN_DRIVER_ONLY;
> -}
> -
> int talitos_register_common(struct device *dev,
> struct talitos_alg_template *template)
> {
> @@ -1168,20 +1151,14 @@ int talitos_register_common(struct device *dev,
> switch (t_alg->algt.type) {
> case CRYPTO_ALG_TYPE_AHASH:
> alg = &t_alg->algt.alg.hash.halg.base;
> - talitos_alg_set_common(priv, alg, t_alg->algt.priority,
> - t_alg->algt.type);
> ret = crypto_register_ahash(&t_alg->algt.alg.hash);
> break;
> case CRYPTO_ALG_TYPE_SKCIPHER:
> alg = &t_alg->algt.alg.skcipher.base;
> - talitos_alg_set_common(priv, alg, t_alg->algt.priority,
> - t_alg->algt.type);
> ret = crypto_register_skcipher(&t_alg->algt.alg.skcipher);
> break;
> case CRYPTO_ALG_TYPE_AEAD:
> alg = &t_alg->algt.alg.aead.base;
> - talitos_alg_set_common(priv, alg, t_alg->algt.priority,
> - t_alg->algt.type);
> ret = crypto_register_aead(&t_alg->algt.alg.aead);
> break;
> default:
>
^ permalink raw reply
* Re: [PATCH 09/29] crypto: talitos/aead - Move into separate file
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:48 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-9-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Move the AEAD algorithm implementations from talitos.c into
> a dedicated talitos-aead.c file.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/Makefile | 2 +-
> drivers/crypto/talitos/talitos-aead.c | 894 ++++++++++++++++++++++++++++++++
> drivers/crypto/talitos/talitos.c | 947 +---------------------------------
> drivers/crypto/talitos/talitos.h | 3 +
> 4 files changed, 907 insertions(+), 939 deletions(-)
>
> diff --git a/drivers/crypto/talitos/Makefile b/drivers/crypto/talitos/Makefile
> index d4f19f2f6375..9e80bb094507 100644
> --- a/drivers/crypto/talitos/Makefile
> +++ b/drivers/crypto/talitos/Makefile
> @@ -1,3 +1,3 @@
> obj-$(CONFIG_CRYPTO_DEV_TALITOS) += talitos.o
>
> -talitos-y := talitos.o talitos-rng.o talitos-hash.o talitos-skcipher.o
> +talitos-y := talitos.o talitos-rng.o talitos-hash.o talitos-skcipher.o talitos-aead.o
> diff --git a/drivers/crypto/talitos/talitos-aead.c b/drivers/crypto/talitos/talitos-aead.c
> new file mode 100644
> index 000000000000..ce6bd6133fd0
> --- /dev/null
> +++ b/drivers/crypto/talitos/talitos-aead.c
> @@ -0,0 +1,894 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +
> +#include <crypto/authenc.h>
> +#include <crypto/internal/des.h>
> +#include <crypto/internal/aead.h>
> +#include <crypto/md5.h>
> +#include <crypto/sha1.h>
> +#include <crypto/sha2.h>
> +
> +#include "talitos.h"
> +
> +/*
> + * Defines a priority for doing AEAD with descriptors type
> + * HMAC_SNOOP_NO_AFEA (HSNA) instead of type IPSEC_ESP
> + */
> +#define TALITOS_CRA_PRIORITY_AEAD_HSNA (TALITOS_CRA_PRIORITY - 1)
> +
> +static int aead_setkey(struct crypto_aead *authenc,
> + const u8 *key, unsigned int keylen)
> +{
> + struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> + struct device *dev = ctx->dev;
> + struct crypto_authenc_keys keys;
> +
> + if (crypto_authenc_extractkeys(&keys, key, keylen) != 0)
> + goto badkey;
> +
> + if (keys.authkeylen + keys.enckeylen > TALITOS_MAX_KEY_SIZE)
> + goto badkey;
> +
> + if (ctx->keylen)
> + dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> +
> + memcpy(ctx->key, keys.authkey, keys.authkeylen);
> + memcpy(&ctx->key[keys.authkeylen], keys.enckey, keys.enckeylen);
> +
> + ctx->keylen = keys.authkeylen + keys.enckeylen;
> + ctx->enckeylen = keys.enckeylen;
> + ctx->authkeylen = keys.authkeylen;
> + ctx->dma_key = dma_map_single(dev, ctx->key, ctx->keylen,
> + DMA_TO_DEVICE);
> +
> + memzero_explicit(&keys, sizeof(keys));
> + return 0;
> +
> +badkey:
> + memzero_explicit(&keys, sizeof(keys));
> + return -EINVAL;
> +}
> +
> +static int aead_des3_setkey(struct crypto_aead *authenc,
> + const u8 *key, unsigned int keylen)
> +{
> + struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> + struct device *dev = ctx->dev;
> + struct crypto_authenc_keys keys;
> + int err;
> +
> + err = crypto_authenc_extractkeys(&keys, key, keylen);
> + if (unlikely(err))
> + goto out;
> +
> + err = -EINVAL;
> + if (keys.authkeylen + keys.enckeylen > TALITOS_MAX_KEY_SIZE)
> + goto out;
> +
> + err = verify_aead_des3_key(authenc, keys.enckey, keys.enckeylen);
> + if (err)
> + goto out;
> +
> + if (ctx->keylen)
> + dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> +
> + memcpy(ctx->key, keys.authkey, keys.authkeylen);
> + memcpy(&ctx->key[keys.authkeylen], keys.enckey, keys.enckeylen);
> +
> + ctx->keylen = keys.authkeylen + keys.enckeylen;
> + ctx->enckeylen = keys.enckeylen;
> + ctx->authkeylen = keys.authkeylen;
> + ctx->dma_key = dma_map_single(dev, ctx->key, ctx->keylen,
> + DMA_TO_DEVICE);
> +
> +out:
> + memzero_explicit(&keys, sizeof(keys));
> + return err;
> +}
> +
> +static void ipsec_esp_unmap(struct device *dev,
> + struct talitos_edesc *edesc,
> + struct aead_request *areq, bool encrypt)
> +{
> + struct crypto_aead *aead = crypto_aead_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_aead_ctx(aead);
> + unsigned int ivsize = crypto_aead_ivsize(aead);
> + unsigned int authsize = crypto_aead_authsize(aead);
> + unsigned int cryptlen = areq->cryptlen - (encrypt ? 0 : authsize);
> + bool is_ipsec_esp = edesc->desc.hdr & DESC_HDR_TYPE_IPSEC_ESP;
> + struct talitos_ptr *civ_ptr = &edesc->desc.ptr[is_ipsec_esp ? 2 : 3];
> +
> + if (is_ipsec_esp)
> + unmap_single_talitos_ptr(dev, &edesc->desc.ptr[6],
> + DMA_FROM_DEVICE);
> + unmap_single_talitos_ptr(dev, civ_ptr, DMA_TO_DEVICE);
> +
> + talitos_sg_unmap(dev, edesc, areq->src, areq->dst,
> + cryptlen + authsize, areq->assoclen);
> +
> + if (edesc->dma_len)
> + dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len,
> + DMA_BIDIRECTIONAL);
> +
> + if (!is_ipsec_esp) {
> + unsigned int dst_nents = edesc->dst_nents ? : 1;
> +
> + sg_pcopy_to_buffer(areq->dst, dst_nents, ctx->iv, ivsize,
> + areq->assoclen + cryptlen - ivsize);
> + }
> +}
> +
> +/*
> + * ipsec_esp descriptor callbacks
> + */
> +static void ipsec_esp_encrypt_done(struct device *dev,
> + struct talitos_desc *desc, void *context,
> + int err)
> +{
> + struct aead_request *areq = context;
> + struct crypto_aead *authenc = crypto_aead_reqtfm(areq);
> + unsigned int ivsize = crypto_aead_ivsize(authenc);
> + struct talitos_edesc *edesc;
> +
> + edesc = container_of(desc, struct talitos_edesc, desc);
> +
> + ipsec_esp_unmap(dev, edesc, areq, true);
> +
> + dma_unmap_single(dev, edesc->iv_dma, ivsize, DMA_TO_DEVICE);
> +
> + kfree(edesc);
> +
> + aead_request_complete(areq, err);
> +}
> +
> +static void ipsec_esp_decrypt_swauth_done(struct device *dev,
> + struct talitos_desc *desc,
> + void *context, int err)
> +{
> + struct aead_request *req = context;
> + struct crypto_aead *authenc = crypto_aead_reqtfm(req);
> + unsigned int authsize = crypto_aead_authsize(authenc);
> + struct talitos_edesc *edesc;
> + char *oicv, *icv;
> +
> + edesc = container_of(desc, struct talitos_edesc, desc);
> +
> + ipsec_esp_unmap(dev, edesc, req, false);
> +
> + if (!err) {
> + /* auth check */
> + oicv = edesc->buf + edesc->dma_len;
> + icv = oicv - authsize;
> +
> + err = crypto_memneq(oicv, icv, authsize) ? -EBADMSG : 0;
> + }
> +
> + kfree(edesc);
> +
> + aead_request_complete(req, err);
> +}
> +
> +static void ipsec_esp_decrypt_hwauth_done(struct device *dev,
> + struct talitos_desc *desc,
> + void *context, int err)
> +{
> + struct aead_request *req = context;
> + struct talitos_edesc *edesc;
> +
> + edesc = container_of(desc, struct talitos_edesc, desc);
> +
> + ipsec_esp_unmap(dev, edesc, req, false);
> +
> + /* check ICV auth status */
> + if (!err && ((desc->hdr_lo & DESC_HDR_LO_ICCR1_MASK) !=
> + DESC_HDR_LO_ICCR1_PASS))
> + err = -EBADMSG;
> +
> + kfree(edesc);
> +
> + aead_request_complete(req, err);
> +}
> +
> +/*
> + * fill in and submit ipsec_esp descriptor
> + */
> +static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
> + bool encrypt,
> + void (*callback)(struct device *dev,
> + struct talitos_desc *desc,
> + void *context, int error))
> +{
> + struct crypto_aead *aead = crypto_aead_reqtfm(areq);
> + unsigned int authsize = crypto_aead_authsize(aead);
> + struct talitos_ctx *ctx = crypto_aead_ctx(aead);
> + struct device *dev = ctx->dev;
> + struct talitos_desc *desc = &edesc->desc;
> + unsigned int cryptlen = areq->cryptlen - (encrypt ? 0 : authsize);
> + unsigned int ivsize = crypto_aead_ivsize(aead);
> + int tbl_off = 0;
> + int sg_count, ret;
> + int elen = 0;
> + bool sync_needed = false;
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + bool is_sec1 = has_ftr_sec1(priv);
> + bool is_ipsec_esp = desc->hdr & DESC_HDR_TYPE_IPSEC_ESP;
> + struct talitos_ptr *civ_ptr = &desc->ptr[is_ipsec_esp ? 2 : 3];
> + struct talitos_ptr *ckey_ptr = &desc->ptr[is_ipsec_esp ? 3 : 2];
> + dma_addr_t dma_icv = edesc->dma_link_tbl + edesc->dma_len - authsize;
> +
> + /* hmac key */
> + to_talitos_ptr(&desc->ptr[0], ctx->dma_key, ctx->authkeylen, is_sec1);
> +
> + sg_count = edesc->src_nents ?: 1;
> + if (is_sec1 && sg_count > 1)
> + sg_copy_to_buffer(areq->src, sg_count, edesc->buf,
> + areq->assoclen + cryptlen);
> + else
> + sg_count = dma_map_sg(dev, areq->src, sg_count,
> + (areq->src == areq->dst) ?
> + DMA_BIDIRECTIONAL : DMA_TO_DEVICE);
> +
> + /* hmac data */
> + ret = talitos_sg_map(dev, areq->src, areq->assoclen, edesc,
> + &desc->ptr[1], sg_count, 0, tbl_off);
> +
> + if (ret > 1) {
> + tbl_off += ret;
> + sync_needed = true;
> + }
> +
> + /* cipher iv */
> + to_talitos_ptr(civ_ptr, edesc->iv_dma, ivsize, is_sec1);
> +
> + /* cipher key */
> + to_talitos_ptr(ckey_ptr, ctx->dma_key + ctx->authkeylen,
> + ctx->enckeylen, is_sec1);
> +
> + /*
> + * cipher in
> + * map and adjust cipher len to aead request cryptlen.
> + * extent is bytes of HMAC postpended to ciphertext,
> + * typically 12 for ipsec
> + */
> + if (is_ipsec_esp && (desc->hdr & DESC_HDR_MODE1_MDEU_CICV))
> + elen = authsize;
> +
> + ret = talitos_sg_map_ext(dev, areq->src, cryptlen, edesc, &desc->ptr[4],
> + sg_count, areq->assoclen, tbl_off, elen,
> + false, 1);
> +
> + if (ret > 1) {
> + tbl_off += ret;
> + sync_needed = true;
> + }
> +
> + /* cipher out */
> + if (areq->src != areq->dst) {
> + sg_count = edesc->dst_nents ? : 1;
> + if (!is_sec1 || sg_count == 1)
> + dma_map_sg(dev, areq->dst, sg_count, DMA_FROM_DEVICE);
> + }
> +
> + if (is_ipsec_esp && encrypt)
> + elen = authsize;
> + else
> + elen = 0;
> + ret = talitos_sg_map_ext(dev, areq->dst, cryptlen, edesc, &desc->ptr[5],
> + sg_count, areq->assoclen, tbl_off, elen,
> + is_ipsec_esp && !encrypt, 1);
> + tbl_off += ret;
> +
> + if (!encrypt && is_ipsec_esp) {
> + struct talitos_ptr *tbl_ptr = &edesc->link_tbl[tbl_off];
> +
> + /* Add an entry to the link table for ICV data */
> + to_talitos_ptr_ext_set(tbl_ptr - 1, 0, is_sec1);
> + to_talitos_ptr_ext_set(tbl_ptr, DESC_PTR_LNKTBL_RET, is_sec1);
> +
> + /* icv data follows link tables */
> + to_talitos_ptr(tbl_ptr, dma_icv, authsize, is_sec1);
> + to_talitos_ptr_ext_or(&desc->ptr[5], authsize, is_sec1);
> + sync_needed = true;
> + } else if (!encrypt) {
> + to_talitos_ptr(&desc->ptr[6], dma_icv, authsize, is_sec1);
> + sync_needed = true;
> + } else if (!is_ipsec_esp) {
> + talitos_sg_map(dev, areq->dst, authsize, edesc, &desc->ptr[6],
> + sg_count, areq->assoclen + cryptlen, tbl_off);
> + }
> +
> + /* iv out */
> + if (is_ipsec_esp)
> + map_single_talitos_ptr(dev, &desc->ptr[6], ivsize, ctx->iv,
> + DMA_FROM_DEVICE);
> +
> + if (sync_needed)
> + dma_sync_single_for_device(dev, edesc->dma_link_tbl,
> + edesc->dma_len,
> + DMA_BIDIRECTIONAL);
> +
> + ret = talitos_submit(dev, ctx->ch, desc, callback, areq);
> + if (ret != -EINPROGRESS) {
> + ipsec_esp_unmap(dev, edesc, areq, encrypt);
> + kfree(edesc);
> + }
> + return ret;
> +}
> +
> +static struct talitos_edesc *aead_edesc_alloc(struct aead_request *areq, u8 *iv,
> + int icv_stashing, bool encrypt)
> +{
> + struct crypto_aead *authenc = crypto_aead_reqtfm(areq);
> + unsigned int authsize = crypto_aead_authsize(authenc);
> + struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> + unsigned int ivsize = crypto_aead_ivsize(authenc);
> + unsigned int cryptlen = areq->cryptlen - (encrypt ? 0 : authsize);
> +
> + return talitos_edesc_alloc(ctx->dev, areq->src, areq->dst,
> + iv, areq->assoclen, cryptlen,
> + authsize, ivsize, icv_stashing,
> + areq->base.flags, encrypt);
> +}
> +
> +static int aead_encrypt(struct aead_request *req)
> +{
> + struct crypto_aead *authenc = crypto_aead_reqtfm(req);
> + struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> + struct talitos_edesc *edesc;
> +
> + /* allocate extended descriptor */
> + edesc = aead_edesc_alloc(req, req->iv, 0, true);
> + if (IS_ERR(edesc))
> + return PTR_ERR(edesc);
> +
> + /* set encrypt */
> + edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_MODE0_ENCRYPT;
> +
> + return ipsec_esp(edesc, req, true, ipsec_esp_encrypt_done);
> +}
> +
> +static int aead_decrypt(struct aead_request *req)
> +{
> + struct crypto_aead *authenc = crypto_aead_reqtfm(req);
> + unsigned int authsize = crypto_aead_authsize(authenc);
> + struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> + struct talitos_private *priv = dev_get_drvdata(ctx->dev);
> + struct talitos_edesc *edesc;
> + void *icvdata;
> +
> + /* allocate extended descriptor */
> + edesc = aead_edesc_alloc(req, req->iv, 1, false);
> + if (IS_ERR(edesc))
> + return PTR_ERR(edesc);
> +
> + if ((edesc->desc.hdr & DESC_HDR_TYPE_IPSEC_ESP) &&
> + (priv->features & TALITOS_FTR_HW_AUTH_CHECK) &&
> + ((!edesc->src_nents && !edesc->dst_nents) ||
> + priv->features & TALITOS_FTR_SRC_LINK_TBL_LEN_INCLUDES_EXTENT)) {
> +
> + /* decrypt and check the ICV */
> + edesc->desc.hdr = ctx->desc_hdr_template |
> + DESC_HDR_DIR_INBOUND |
> + DESC_HDR_MODE1_MDEU_CICV;
> +
> + /* reset integrity check result bits */
> +
> + return ipsec_esp(edesc, req, false,
> + ipsec_esp_decrypt_hwauth_done);
> + }
> +
> + /* Have to check the ICV with software */
> + edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_DIR_INBOUND;
> +
> + /* stash incoming ICV for later cmp with ICV generated by the h/w */
> + icvdata = edesc->buf + edesc->dma_len;
> +
> + sg_pcopy_to_buffer(req->src, edesc->src_nents ? : 1, icvdata, authsize,
> + req->assoclen + req->cryptlen - authsize);
> +
> + return ipsec_esp(edesc, req, false, ipsec_esp_decrypt_swauth_done);
> +}
> +
> +static int talitos_cra_init_aead(struct crypto_aead *tfm)
> +{
> + struct aead_alg *alg = crypto_aead_alg(tfm);
> + struct talitos_crypto_alg *talitos_alg;
> + struct talitos_ctx *ctx = crypto_aead_ctx(tfm);
> +
> + talitos_alg = container_of(alg, struct talitos_crypto_alg,
> + algt.alg.aead);
> +
> + return talitos_init_common(ctx, talitos_alg);
> +}
> +
> +static struct talitos_alg_template aead_driver_algs[] = {
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha1),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha1-"
> + "cbc-aes-talitos",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA1_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha1),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha1-"
> + "cbc-aes-talitos-hsna",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA1_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha1),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha1-"
> + "cbc-3des-talitos",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA1_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha1),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha1-"
> + "cbc-3des-talitos-hsna",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA1_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha224),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha224-"
> + "cbc-aes-talitos",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA224_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha224),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha224-"
> + "cbc-aes-talitos-hsna",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA224_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha224),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha224-"
> + "cbc-3des-talitos",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA224_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha224),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha224-"
> + "cbc-3des-talitos-hsna",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA224_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha256),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha256-"
> + "cbc-aes-talitos",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA256_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha256),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha256-"
> + "cbc-aes-talitos-hsna",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA256_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha256),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha256-"
> + "cbc-3des-talitos",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA256_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha256),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha256-"
> + "cbc-3des-talitos-hsna",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA256_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha384),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha384-"
> + "cbc-aes-talitos",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA384_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUB |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA384_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha384),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha384-"
> + "cbc-3des-talitos",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA384_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUB |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA384_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha512),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-sha512-"
> + "cbc-aes-talitos",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = SHA512_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUB |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA512_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(sha512),"
> + "cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-sha512-"
> + "cbc-3des-talitos",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = SHA512_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUB |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEUB_SHA512_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(md5),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-md5-"
> + "cbc-aes-talitos",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = MD5_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_MD5_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(md5),cbc(aes))",
> + .cra_driver_name = "authenc-hmac-md5-"
> + "cbc-aes-talitos-hsna",
> + .cra_blocksize = AES_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = AES_BLOCK_SIZE,
> + .maxauthsize = MD5_DIGEST_SIZE,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_MD5_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-md5-"
> + "cbc-3des-talitos",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = MD5_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_MD5_HMAC,
> + },
> + { .type = CRYPTO_ALG_TYPE_AEAD,
> + .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> + .alg.aead = {
> + .base = {
> + .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> + .cra_driver_name = "authenc-hmac-md5-"
> + "cbc-3des-talitos-hsna",
> + .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + },
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .maxauthsize = MD5_DIGEST_SIZE,
> + .setkey = aead_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES |
> + DESC_HDR_SEL1_MDEUA |
> + DESC_HDR_MODE1_MDEU_INIT |
> + DESC_HDR_MODE1_MDEU_PAD |
> + DESC_HDR_MODE1_MDEU_MD5_HMAC,
> + },
> +};
> +
> +int talitos_register_aead(struct device *dev)
> +{
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + struct aead_alg *aead_alg;
> + struct crypto_alg *alg;
> + size_t i;
> + int ret;
> +
> + for (i = 0; i < ARRAY_SIZE(aead_driver_algs); i++) {
> + if (!talitos_hw_supports(dev,
> + aead_driver_algs[i].desc_hdr_template))
> + continue;
> +
> + aead_alg = &aead_driver_algs[i].alg.aead;
> + alg = &aead_alg->base;
> +
> + alg->cra_exit = talitos_cra_exit;
> + aead_alg->init = talitos_cra_init_aead;
> + aead_alg->setkey = aead_alg->setkey ?: aead_setkey;
> + aead_alg->encrypt = aead_encrypt;
> + aead_alg->decrypt = aead_decrypt;
> + if (!(priv->features & TALITOS_FTR_SHA224_HWINIT) &&
> + !strncmp(alg->cra_name, "authenc(hmac(sha224)", 20)) {
> + continue;
> + }
> +
> + ret = talitos_register_common(dev, &aead_driver_algs[i]);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/crypto/talitos/talitos.c b/drivers/crypto/talitos/talitos.c
> index cd37bc379f86..41d7d0e570e3 100644
> --- a/drivers/crypto/talitos/talitos.c
> +++ b/drivers/crypto/talitos/talitos.c
> @@ -820,87 +820,6 @@ DEF_TALITOS2_INTERRUPT(ch0_2, TALITOS2_ISR_CH_0_2_DONE, TALITOS2_ISR_CH_0_2_ERR,
> DEF_TALITOS2_INTERRUPT(ch1_3, TALITOS2_ISR_CH_1_3_DONE, TALITOS2_ISR_CH_1_3_ERR,
> 1)
>
> -
> -/*
> - * crypto alg
> - */
> -#define TALITOS_CRA_PRIORITY 3000
> -/*
> - * Defines a priority for doing AEAD with descriptors type
> - * HMAC_SNOOP_NO_AFEA (HSNA) instead of type IPSEC_ESP
> - */
> -#define TALITOS_CRA_PRIORITY_AEAD_HSNA (TALITOS_CRA_PRIORITY - 1)
> -
> -static int aead_setkey(struct crypto_aead *authenc,
> - const u8 *key, unsigned int keylen)
> -{
> - struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> - struct device *dev = ctx->dev;
> - struct crypto_authenc_keys keys;
> -
> - if (crypto_authenc_extractkeys(&keys, key, keylen) != 0)
> - goto badkey;
> -
> - if (keys.authkeylen + keys.enckeylen > TALITOS_MAX_KEY_SIZE)
> - goto badkey;
> -
> - if (ctx->keylen)
> - dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> -
> - memcpy(ctx->key, keys.authkey, keys.authkeylen);
> - memcpy(&ctx->key[keys.authkeylen], keys.enckey, keys.enckeylen);
> -
> - ctx->keylen = keys.authkeylen + keys.enckeylen;
> - ctx->enckeylen = keys.enckeylen;
> - ctx->authkeylen = keys.authkeylen;
> - ctx->dma_key = dma_map_single(dev, ctx->key, ctx->keylen,
> - DMA_TO_DEVICE);
> -
> - memzero_explicit(&keys, sizeof(keys));
> - return 0;
> -
> -badkey:
> - memzero_explicit(&keys, sizeof(keys));
> - return -EINVAL;
> -}
> -
> -static int aead_des3_setkey(struct crypto_aead *authenc,
> - const u8 *key, unsigned int keylen)
> -{
> - struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> - struct device *dev = ctx->dev;
> - struct crypto_authenc_keys keys;
> - int err;
> -
> - err = crypto_authenc_extractkeys(&keys, key, keylen);
> - if (unlikely(err))
> - goto out;
> -
> - err = -EINVAL;
> - if (keys.authkeylen + keys.enckeylen > TALITOS_MAX_KEY_SIZE)
> - goto out;
> -
> - err = verify_aead_des3_key(authenc, keys.enckey, keys.enckeylen);
> - if (err)
> - goto out;
> -
> - if (ctx->keylen)
> - dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> -
> - memcpy(ctx->key, keys.authkey, keys.authkeylen);
> - memcpy(&ctx->key[keys.authkeylen], keys.enckey, keys.enckeylen);
> -
> - ctx->keylen = keys.authkeylen + keys.enckeylen;
> - ctx->enckeylen = keys.enckeylen;
> - ctx->authkeylen = keys.authkeylen;
> - ctx->dma_key = dma_map_single(dev, ctx->key, ctx->keylen,
> - DMA_TO_DEVICE);
> -
> -out:
> - memzero_explicit(&keys, sizeof(keys));
> - return err;
> -}
> -
> void talitos_sg_unmap(struct device *dev,
> struct talitos_edesc *edesc,
> struct scatterlist *src,
> @@ -929,109 +848,6 @@ void talitos_sg_unmap(struct device *dev,
> }
> }
>
> -static void ipsec_esp_unmap(struct device *dev,
> - struct talitos_edesc *edesc,
> - struct aead_request *areq, bool encrypt)
> -{
> - struct crypto_aead *aead = crypto_aead_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_aead_ctx(aead);
> - unsigned int ivsize = crypto_aead_ivsize(aead);
> - unsigned int authsize = crypto_aead_authsize(aead);
> - unsigned int cryptlen = areq->cryptlen - (encrypt ? 0 : authsize);
> - bool is_ipsec_esp = edesc->desc.hdr & DESC_HDR_TYPE_IPSEC_ESP;
> - struct talitos_ptr *civ_ptr = &edesc->desc.ptr[is_ipsec_esp ? 2 : 3];
> -
> - if (is_ipsec_esp)
> - unmap_single_talitos_ptr(dev, &edesc->desc.ptr[6],
> - DMA_FROM_DEVICE);
> - unmap_single_talitos_ptr(dev, civ_ptr, DMA_TO_DEVICE);
> -
> - talitos_sg_unmap(dev, edesc, areq->src, areq->dst,
> - cryptlen + authsize, areq->assoclen);
> -
> - if (edesc->dma_len)
> - dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len,
> - DMA_BIDIRECTIONAL);
> -
> - if (!is_ipsec_esp) {
> - unsigned int dst_nents = edesc->dst_nents ? : 1;
> -
> - sg_pcopy_to_buffer(areq->dst, dst_nents, ctx->iv, ivsize,
> - areq->assoclen + cryptlen - ivsize);
> - }
> -}
> -
> -/*
> - * ipsec_esp descriptor callbacks
> - */
> -static void ipsec_esp_encrypt_done(struct device *dev,
> - struct talitos_desc *desc, void *context,
> - int err)
> -{
> - struct aead_request *areq = context;
> - struct crypto_aead *authenc = crypto_aead_reqtfm(areq);
> - unsigned int ivsize = crypto_aead_ivsize(authenc);
> - struct talitos_edesc *edesc;
> -
> - edesc = container_of(desc, struct talitos_edesc, desc);
> -
> - ipsec_esp_unmap(dev, edesc, areq, true);
> -
> - dma_unmap_single(dev, edesc->iv_dma, ivsize, DMA_TO_DEVICE);
> -
> - kfree(edesc);
> -
> - aead_request_complete(areq, err);
> -}
> -
> -static void ipsec_esp_decrypt_swauth_done(struct device *dev,
> - struct talitos_desc *desc,
> - void *context, int err)
> -{
> - struct aead_request *req = context;
> - struct crypto_aead *authenc = crypto_aead_reqtfm(req);
> - unsigned int authsize = crypto_aead_authsize(authenc);
> - struct talitos_edesc *edesc;
> - char *oicv, *icv;
> -
> - edesc = container_of(desc, struct talitos_edesc, desc);
> -
> - ipsec_esp_unmap(dev, edesc, req, false);
> -
> - if (!err) {
> - /* auth check */
> - oicv = edesc->buf + edesc->dma_len;
> - icv = oicv - authsize;
> -
> - err = crypto_memneq(oicv, icv, authsize) ? -EBADMSG : 0;
> - }
> -
> - kfree(edesc);
> -
> - aead_request_complete(req, err);
> -}
> -
> -static void ipsec_esp_decrypt_hwauth_done(struct device *dev,
> - struct talitos_desc *desc,
> - void *context, int err)
> -{
> - struct aead_request *req = context;
> - struct talitos_edesc *edesc;
> -
> - edesc = container_of(desc, struct talitos_edesc, desc);
> -
> - ipsec_esp_unmap(dev, edesc, req, false);
> -
> - /* check ICV auth status */
> - if (!err && ((desc->hdr_lo & DESC_HDR_LO_ICCR1_MASK) !=
> - DESC_HDR_LO_ICCR1_PASS))
> - err = -EBADMSG;
> -
> - kfree(edesc);
> -
> - aead_request_complete(req, err);
> -}
> -
> /*
> * convert scatterlist to SEC h/w link table format
> * stop at cryptlen bytes
> @@ -1132,132 +948,6 @@ int talitos_sg_map(struct device *dev, struct scatterlist *src,
> tbl_off, 0, false, 1);
> }
>
> -/*
> - * fill in and submit ipsec_esp descriptor
> - */
> -static int ipsec_esp(struct talitos_edesc *edesc, struct aead_request *areq,
> - bool encrypt,
> - void (*callback)(struct device *dev,
> - struct talitos_desc *desc,
> - void *context, int error))
> -{
> - struct crypto_aead *aead = crypto_aead_reqtfm(areq);
> - unsigned int authsize = crypto_aead_authsize(aead);
> - struct talitos_ctx *ctx = crypto_aead_ctx(aead);
> - struct device *dev = ctx->dev;
> - struct talitos_desc *desc = &edesc->desc;
> - unsigned int cryptlen = areq->cryptlen - (encrypt ? 0 : authsize);
> - unsigned int ivsize = crypto_aead_ivsize(aead);
> - int tbl_off = 0;
> - int sg_count, ret;
> - int elen = 0;
> - bool sync_needed = false;
> - struct talitos_private *priv = dev_get_drvdata(dev);
> - bool is_sec1 = has_ftr_sec1(priv);
> - bool is_ipsec_esp = desc->hdr & DESC_HDR_TYPE_IPSEC_ESP;
> - struct talitos_ptr *civ_ptr = &desc->ptr[is_ipsec_esp ? 2 : 3];
> - struct talitos_ptr *ckey_ptr = &desc->ptr[is_ipsec_esp ? 3 : 2];
> - dma_addr_t dma_icv = edesc->dma_link_tbl + edesc->dma_len - authsize;
> -
> - /* hmac key */
> - to_talitos_ptr(&desc->ptr[0], ctx->dma_key, ctx->authkeylen, is_sec1);
> -
> - sg_count = edesc->src_nents ?: 1;
> - if (is_sec1 && sg_count > 1)
> - sg_copy_to_buffer(areq->src, sg_count, edesc->buf,
> - areq->assoclen + cryptlen);
> - else
> - sg_count = dma_map_sg(dev, areq->src, sg_count,
> - (areq->src == areq->dst) ?
> - DMA_BIDIRECTIONAL : DMA_TO_DEVICE);
> -
> - /* hmac data */
> - ret = talitos_sg_map(dev, areq->src, areq->assoclen, edesc,
> - &desc->ptr[1], sg_count, 0, tbl_off);
> -
> - if (ret > 1) {
> - tbl_off += ret;
> - sync_needed = true;
> - }
> -
> - /* cipher iv */
> - to_talitos_ptr(civ_ptr, edesc->iv_dma, ivsize, is_sec1);
> -
> - /* cipher key */
> - to_talitos_ptr(ckey_ptr, ctx->dma_key + ctx->authkeylen,
> - ctx->enckeylen, is_sec1);
> -
> - /*
> - * cipher in
> - * map and adjust cipher len to aead request cryptlen.
> - * extent is bytes of HMAC postpended to ciphertext,
> - * typically 12 for ipsec
> - */
> - if (is_ipsec_esp && (desc->hdr & DESC_HDR_MODE1_MDEU_CICV))
> - elen = authsize;
> -
> - ret = talitos_sg_map_ext(dev, areq->src, cryptlen, edesc, &desc->ptr[4],
> - sg_count, areq->assoclen, tbl_off, elen,
> - false, 1);
> -
> - if (ret > 1) {
> - tbl_off += ret;
> - sync_needed = true;
> - }
> -
> - /* cipher out */
> - if (areq->src != areq->dst) {
> - sg_count = edesc->dst_nents ? : 1;
> - if (!is_sec1 || sg_count == 1)
> - dma_map_sg(dev, areq->dst, sg_count, DMA_FROM_DEVICE);
> - }
> -
> - if (is_ipsec_esp && encrypt)
> - elen = authsize;
> - else
> - elen = 0;
> - ret = talitos_sg_map_ext(dev, areq->dst, cryptlen, edesc, &desc->ptr[5],
> - sg_count, areq->assoclen, tbl_off, elen,
> - is_ipsec_esp && !encrypt, 1);
> - tbl_off += ret;
> -
> - if (!encrypt && is_ipsec_esp) {
> - struct talitos_ptr *tbl_ptr = &edesc->link_tbl[tbl_off];
> -
> - /* Add an entry to the link table for ICV data */
> - to_talitos_ptr_ext_set(tbl_ptr - 1, 0, is_sec1);
> - to_talitos_ptr_ext_set(tbl_ptr, DESC_PTR_LNKTBL_RET, is_sec1);
> -
> - /* icv data follows link tables */
> - to_talitos_ptr(tbl_ptr, dma_icv, authsize, is_sec1);
> - to_talitos_ptr_ext_or(&desc->ptr[5], authsize, is_sec1);
> - sync_needed = true;
> - } else if (!encrypt) {
> - to_talitos_ptr(&desc->ptr[6], dma_icv, authsize, is_sec1);
> - sync_needed = true;
> - } else if (!is_ipsec_esp) {
> - talitos_sg_map(dev, areq->dst, authsize, edesc, &desc->ptr[6],
> - sg_count, areq->assoclen + cryptlen, tbl_off);
> - }
> -
> - /* iv out */
> - if (is_ipsec_esp)
> - map_single_talitos_ptr(dev, &desc->ptr[6], ivsize, ctx->iv,
> - DMA_FROM_DEVICE);
> -
> - if (sync_needed)
> - dma_sync_single_for_device(dev, edesc->dma_link_tbl,
> - edesc->dma_len,
> - DMA_BIDIRECTIONAL);
> -
> - ret = talitos_submit(dev, ctx->ch, desc, callback, areq);
> - if (ret != -EINPROGRESS) {
> - ipsec_esp_unmap(dev, edesc, areq, encrypt);
> - kfree(edesc);
> - }
> - return ret;
> -}
> -
> /*
> * allocate and map the extended descriptor
> */
> @@ -1356,540 +1046,6 @@ struct talitos_edesc *talitos_edesc_alloc(struct device *dev,
> return edesc;
> }
>
> -static struct talitos_edesc *aead_edesc_alloc(struct aead_request *areq, u8 *iv,
> - int icv_stashing, bool encrypt)
> -{
> - struct crypto_aead *authenc = crypto_aead_reqtfm(areq);
> - unsigned int authsize = crypto_aead_authsize(authenc);
> - struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> - unsigned int ivsize = crypto_aead_ivsize(authenc);
> - unsigned int cryptlen = areq->cryptlen - (encrypt ? 0 : authsize);
> -
> - return talitos_edesc_alloc(ctx->dev, areq->src, areq->dst,
> - iv, areq->assoclen, cryptlen,
> - authsize, ivsize, icv_stashing,
> - areq->base.flags, encrypt);
> -}
> -
> -static int aead_encrypt(struct aead_request *req)
> -{
> - struct crypto_aead *authenc = crypto_aead_reqtfm(req);
> - struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> - struct talitos_edesc *edesc;
> -
> - /* allocate extended descriptor */
> - edesc = aead_edesc_alloc(req, req->iv, 0, true);
> - if (IS_ERR(edesc))
> - return PTR_ERR(edesc);
> -
> - /* set encrypt */
> - edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_MODE0_ENCRYPT;
> -
> - return ipsec_esp(edesc, req, true, ipsec_esp_encrypt_done);
> -}
> -
> -static int aead_decrypt(struct aead_request *req)
> -{
> - struct crypto_aead *authenc = crypto_aead_reqtfm(req);
> - unsigned int authsize = crypto_aead_authsize(authenc);
> - struct talitos_ctx *ctx = crypto_aead_ctx(authenc);
> - struct talitos_private *priv = dev_get_drvdata(ctx->dev);
> - struct talitos_edesc *edesc;
> - void *icvdata;
> -
> - /* allocate extended descriptor */
> - edesc = aead_edesc_alloc(req, req->iv, 1, false);
> - if (IS_ERR(edesc))
> - return PTR_ERR(edesc);
> -
> - if ((edesc->desc.hdr & DESC_HDR_TYPE_IPSEC_ESP) &&
> - (priv->features & TALITOS_FTR_HW_AUTH_CHECK) &&
> - ((!edesc->src_nents && !edesc->dst_nents) ||
> - priv->features & TALITOS_FTR_SRC_LINK_TBL_LEN_INCLUDES_EXTENT)) {
> -
> - /* decrypt and check the ICV */
> - edesc->desc.hdr = ctx->desc_hdr_template |
> - DESC_HDR_DIR_INBOUND |
> - DESC_HDR_MODE1_MDEU_CICV;
> -
> - /* reset integrity check result bits */
> -
> - return ipsec_esp(edesc, req, false,
> - ipsec_esp_decrypt_hwauth_done);
> - }
> -
> - /* Have to check the ICV with software */
> - edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_DIR_INBOUND;
> -
> - /* stash incoming ICV for later cmp with ICV generated by the h/w */
> - icvdata = edesc->buf + edesc->dma_len;
> -
> - sg_pcopy_to_buffer(req->src, edesc->src_nents ? : 1, icvdata, authsize,
> - req->assoclen + req->cryptlen - authsize);
> -
> - return ipsec_esp(edesc, req, false, ipsec_esp_decrypt_swauth_done);
> -}
> -
> -static struct talitos_alg_template driver_algs[] = {
> - /* AEAD algorithms. These use a single-pass ipsec_esp descriptor */
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha1),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha1-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA1_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA1_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha224),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha224-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA224_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA224_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha256),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha256-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA256_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_SHA256_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha384),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha384-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA384_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA384_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha384),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha384-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA384_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA384_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha512),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-sha512-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = SHA512_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA512_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(sha512),"
> - "cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-sha512-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = SHA512_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUB |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEUB_SHA512_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-aes-talitos",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(aes))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-aes-talitos-hsna",
> - .cra_blocksize = AES_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = AES_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-3des-talitos",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_IPSEC_ESP |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> - { .type = CRYPTO_ALG_TYPE_AEAD,
> - .priority = TALITOS_CRA_PRIORITY_AEAD_HSNA,
> - .alg.aead = {
> - .base = {
> - .cra_name = "authenc(hmac(md5),cbc(des3_ede))",
> - .cra_driver_name = "authenc-hmac-md5-"
> - "cbc-3des-talitos-hsna",
> - .cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - },
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .maxauthsize = MD5_DIGEST_SIZE,
> - .setkey = aead_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES |
> - DESC_HDR_SEL1_MDEUA |
> - DESC_HDR_MODE1_MDEU_INIT |
> - DESC_HDR_MODE1_MDEU_PAD |
> - DESC_HDR_MODE1_MDEU_MD5_HMAC,
> - },
> -};
> -
> int talitos_init_common(struct talitos_ctx *ctx,
> struct talitos_crypto_alg *talitos_alg)
> {
> @@ -1912,18 +1068,6 @@ int talitos_init_common(struct talitos_ctx *ctx,
> return 0;
> }
>
> -static int talitos_cra_init_aead(struct crypto_aead *tfm)
> -{
> - struct aead_alg *alg = crypto_aead_alg(tfm);
> - struct talitos_crypto_alg *talitos_alg;
> - struct talitos_ctx *ctx = crypto_aead_ctx(tfm);
> -
> - talitos_alg = container_of(alg, struct talitos_crypto_alg,
> - algt.alg.aead);
> -
> - return talitos_init_common(ctx, talitos_alg);
> -}
> -
> void talitos_cra_exit(struct crypto_tfm *tfm)
> {
> struct talitos_ctx *ctx = crypto_tfm_ctx(tfm);
> @@ -2034,6 +1178,12 @@ int talitos_register_common(struct device *dev,
> t_alg->algt.type);
> ret = crypto_register_skcipher(&t_alg->algt.alg.skcipher);
> break;
> + case CRYPTO_ALG_TYPE_AEAD:
> + alg = &t_alg->algt.alg.aead.base;
> + talitos_alg_set_common(priv, alg, t_alg->algt.priority,
> + t_alg->algt.type);
> + ret = crypto_register_aead(&t_alg->algt.alg.aead);
> + break;
> default:
> dev_err(dev, "unknown algorithm type %d\n", t_alg->algt.type);
> devm_kfree(dev, t_alg);
> @@ -2054,59 +1204,6 @@ int talitos_register_common(struct device *dev,
> return 0;
> }
>
> -static struct talitos_crypto_alg *talitos_alg_alloc(struct device *dev,
> - struct talitos_alg_template
> - *template)
> -{
> - struct talitos_private *priv = dev_get_drvdata(dev);
> - struct talitos_crypto_alg *t_alg;
> - struct crypto_alg *alg;
> -
> - t_alg = devm_kzalloc(dev, sizeof(struct talitos_crypto_alg),
> - GFP_KERNEL);
> - if (!t_alg)
> - return ERR_PTR(-ENOMEM);
> -
> - t_alg->algt = *template;
> -
> - switch (t_alg->algt.type) {
> - case CRYPTO_ALG_TYPE_AEAD:
> - alg = &t_alg->algt.alg.aead.base;
> - alg->cra_exit = talitos_cra_exit;
> - t_alg->algt.alg.aead.init = talitos_cra_init_aead;
> - t_alg->algt.alg.aead.setkey = t_alg->algt.alg.aead.setkey ?:
> - aead_setkey;
> - t_alg->algt.alg.aead.encrypt = aead_encrypt;
> - t_alg->algt.alg.aead.decrypt = aead_decrypt;
> - if (!(priv->features & TALITOS_FTR_SHA224_HWINIT) &&
> - !strncmp(alg->cra_name, "authenc(hmac(sha224)", 20)) {
> - devm_kfree(dev, t_alg);
> - return ERR_PTR(-ENOTSUPP);
> - }
> - break;
> - default:
> - dev_err(dev, "unknown algorithm type %d\n", t_alg->algt.type);
> - devm_kfree(dev, t_alg);
> - return ERR_PTR(-EINVAL);
> - }
> -
> - alg->cra_module = THIS_MODULE;
> - if (t_alg->algt.priority)
> - alg->cra_priority = t_alg->algt.priority;
> - else
> - alg->cra_priority = TALITOS_CRA_PRIORITY;
> - if (has_ftr_sec1(priv) && t_alg->algt.type != CRYPTO_ALG_TYPE_AHASH)
> - alg->cra_alignmask = 3;
> - else
> - alg->cra_alignmask = 0;
> - alg->cra_ctxsize = sizeof(struct talitos_ctx);
> - alg->cra_flags |= CRYPTO_ALG_KERN_DRIVER_ONLY;
> -
> - t_alg->dev = dev;
> -
> - return t_alg;
> -}
> -
> static int talitos_probe_irq(struct platform_device *ofdev)
> {
> struct device *dev = &ofdev->dev;
> @@ -2320,36 +1417,10 @@ static int talitos_probe(struct platform_device *ofdev)
> if (err)
> goto err_out;
>
> - /* register crypto algorithms the device supports */
> - for (i = 0; i < ARRAY_SIZE(driver_algs); i++) {
> - if (talitos_hw_supports(dev,
> - driver_algs[i].desc_hdr_template)) {
> - struct talitos_crypto_alg *t_alg;
> - struct crypto_alg *alg = NULL;
> -
> - t_alg = talitos_alg_alloc(dev, &driver_algs[i]);
> - if (IS_ERR(t_alg)) {
> - err = PTR_ERR(t_alg);
> - if (err == -ENOTSUPP)
> - continue;
> - goto err_out;
> - }
> + err = talitos_register_aead(dev);
> + if (err)
> + goto err_out;
>
> - switch (t_alg->algt.type) {
> - case CRYPTO_ALG_TYPE_AEAD:
> - err = crypto_register_aead(
> - &t_alg->algt.alg.aead);
> - alg = &t_alg->algt.alg.aead.base;
> - break;
> - }
> - if (err) {
> - dev_err(dev, "%s alg registration failed\n",
> - alg->cra_driver_name);
> - devm_kfree(dev, t_alg);
> - } else
> - list_add_tail(&t_alg->entry, &priv->alg_list);
> - }
> - }
> if (!list_empty(&priv->alg_list))
> dev_info(dev, "%s algorithms registered in /proc/crypto\n",
> (char *)of_get_property(np, "compatible", NULL));
> diff --git a/drivers/crypto/talitos/talitos.h b/drivers/crypto/talitos/talitos.h
> index 7e7d41673fa5..438be8c8f08d 100644
> --- a/drivers/crypto/talitos/talitos.h
> +++ b/drivers/crypto/talitos/talitos.h
> @@ -21,6 +21,8 @@
> #define TALITOS1_MAX_DATA_LEN 32768
> #define TALITOS2_MAX_DATA_LEN 65535
>
> +#define TALITOS_CRA_PRIORITY 3000
> +
> #define DESC_TYPE(desc_hdr) ((be32_to_cpu(desc_hdr) >> 3) & 0x1f)
> #define PRIMARY_EU(desc_hdr) ((be32_to_cpu(desc_hdr) >> 28) & 0xf)
> #define SECONDARY_EU(desc_hdr) ((be32_to_cpu(desc_hdr) >> 16) & 0xf)
> @@ -535,3 +537,4 @@ void talitos_unregister_rng(struct device *dev);
>
> int talitos_register_hash(struct device *dev);
> int talitos_register_skcipher(struct device *dev);
> +int talitos_register_aead(struct device *dev);
>
^ permalink raw reply
* Re: [PATCH 08/29] crypto: talitos/skcipher - Move into separate file
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:47 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-8-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Move the skcipher algorithm implementations from talitos.c into
> a dedicated talitos-skcipher.c file.
>
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> ---
> drivers/crypto/talitos/Makefile | 2 +-
> drivers/crypto/talitos/talitos-skcipher.c | 396 ++++++++++++++++++++++++++++++
> drivers/crypto/talitos/talitos.c | 377 +---------------------------
> drivers/crypto/talitos/talitos.h | 1 +
> 4 files changed, 408 insertions(+), 368 deletions(-)
>
> diff --git a/drivers/crypto/talitos/Makefile b/drivers/crypto/talitos/Makefile
> index 40d37f9364ef..d4f19f2f6375 100644
> --- a/drivers/crypto/talitos/Makefile
> +++ b/drivers/crypto/talitos/Makefile
> @@ -1,3 +1,3 @@
> obj-$(CONFIG_CRYPTO_DEV_TALITOS) += talitos.o
>
> -talitos-y := talitos.o talitos-rng.o talitos-hash.o
> +talitos-y := talitos.o talitos-rng.o talitos-hash.o talitos-skcipher.o
> diff --git a/drivers/crypto/talitos/talitos-skcipher.c b/drivers/crypto/talitos/talitos-skcipher.c
> new file mode 100644
> index 000000000000..4f742930ec47
> --- /dev/null
> +++ b/drivers/crypto/talitos/talitos-skcipher.c
> @@ -0,0 +1,396 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +
> +/*
> + * Freescale SEC (talitos) skcipher implementation
> + *
> + * Copyright (c) 2006-2011 Freescale Semiconductor, Inc.
> + */
> +
> +#include <crypto/internal/des.h>
> +#include <crypto/internal/skcipher.h>
> +
> +#include "talitos.h"
> +
> +static void common_nonsnoop_unmap(struct device *dev,
> + struct talitos_edesc *edesc,
> + struct skcipher_request *areq)
> +{
> + unmap_single_talitos_ptr(dev, &edesc->desc.ptr[5], DMA_FROM_DEVICE);
> +
> + talitos_sg_unmap(dev, edesc, areq->src, areq->dst, areq->cryptlen, 0);
> + unmap_single_talitos_ptr(dev, &edesc->desc.ptr[1], DMA_TO_DEVICE);
> +
> + if (edesc->dma_len)
> + dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len,
> + DMA_BIDIRECTIONAL);
> +}
> +
> +static void skcipher_done(struct device *dev,
> + struct talitos_desc *desc, void *context,
> + int err)
> +{
> + struct skcipher_request *areq = context;
> + struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> + unsigned int ivsize = crypto_skcipher_ivsize(cipher);
> + struct talitos_edesc *edesc;
> +
> + edesc = container_of(desc, struct talitos_edesc, desc);
> +
> + common_nonsnoop_unmap(dev, edesc, areq);
> + memcpy(areq->iv, ctx->iv, ivsize);
> +
> + kfree(edesc);
> +
> + skcipher_request_complete(areq, err);
> +}
> +
> +static int common_nonsnoop(struct talitos_edesc *edesc,
> + struct skcipher_request *areq,
> + void (*callback) (struct device *dev,
> + struct talitos_desc *desc,
> + void *context, int error))
> +{
> + struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> + struct device *dev = ctx->dev;
> + struct talitos_desc *desc = &edesc->desc;
> + unsigned int cryptlen = areq->cryptlen;
> + unsigned int ivsize = crypto_skcipher_ivsize(cipher);
> + int sg_count, ret;
> + bool sync_needed = false;
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + bool is_sec1 = has_ftr_sec1(priv);
> + bool is_ctr = (desc->hdr & DESC_HDR_SEL0_MASK) == DESC_HDR_SEL0_AESU &&
> + (desc->hdr & DESC_HDR_MODE0_AESU_MASK) == DESC_HDR_MODE0_AESU_CTR;
> +
> + /* first DWORD empty */
> +
> + /* cipher iv */
> + to_talitos_ptr(&desc->ptr[1], edesc->iv_dma, ivsize, is_sec1);
> +
> + /* cipher key */
> + to_talitos_ptr(&desc->ptr[2], ctx->dma_key, ctx->keylen, is_sec1);
> +
> + sg_count = edesc->src_nents ?: 1;
> + if (is_sec1 && sg_count > 1)
> + sg_copy_to_buffer(areq->src, sg_count, edesc->buf,
> + cryptlen);
> + else
> + sg_count = dma_map_sg(dev, areq->src, sg_count,
> + (areq->src == areq->dst) ?
> + DMA_BIDIRECTIONAL : DMA_TO_DEVICE);
> + /*
> + * cipher in
> + */
> + sg_count = talitos_sg_map_ext(dev, areq->src, cryptlen, edesc, &desc->ptr[3],
> + sg_count, 0, 0, 0, false, is_ctr ? 16 : 1);
> + if (sg_count > 1)
> + sync_needed = true;
> +
> + /* cipher out */
> + if (areq->src != areq->dst) {
> + sg_count = edesc->dst_nents ? : 1;
> + if (!is_sec1 || sg_count == 1)
> + dma_map_sg(dev, areq->dst, sg_count, DMA_FROM_DEVICE);
> + }
> +
> + ret = talitos_sg_map(dev, areq->dst, cryptlen, edesc, &desc->ptr[4],
> + sg_count, 0, (edesc->src_nents + 1));
> + if (ret > 1)
> + sync_needed = true;
> +
> + /* iv out */
> + map_single_talitos_ptr(dev, &desc->ptr[5], ivsize, ctx->iv,
> + DMA_FROM_DEVICE);
> +
> + /* last DWORD empty */
> +
> + if (sync_needed)
> + dma_sync_single_for_device(dev, edesc->dma_link_tbl,
> + edesc->dma_len, DMA_BIDIRECTIONAL);
> +
> + ret = talitos_submit(dev, ctx->ch, desc, callback, areq);
> + if (ret != -EINPROGRESS) {
> + common_nonsnoop_unmap(dev, edesc, areq);
> + kfree(edesc);
> + }
> + return ret;
> +}
> +
> +static int skcipher_setkey(struct crypto_skcipher *cipher,
> + const u8 *key, unsigned int keylen)
> +{
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> + struct device *dev = ctx->dev;
> +
> + if (ctx->keylen)
> + dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> +
> + memcpy(&ctx->key, key, keylen);
> + ctx->keylen = keylen;
> +
> + ctx->dma_key = dma_map_single(dev, ctx->key, keylen, DMA_TO_DEVICE);
> +
> + return 0;
> +}
> +
> +static int skcipher_des_setkey(struct crypto_skcipher *cipher,
> + const u8 *key, unsigned int keylen)
> +{
> + return verify_skcipher_des_key(cipher, key) ?:
> + skcipher_setkey(cipher, key, keylen);
> +}
> +
> +static int skcipher_des3_setkey(struct crypto_skcipher *cipher,
> + const u8 *key, unsigned int keylen)
> +{
> + return verify_skcipher_des3_key(cipher, key) ?:
> + skcipher_setkey(cipher, key, keylen);
> +}
> +
> +static int skcipher_aes_setkey(struct crypto_skcipher *cipher,
> + const u8 *key, unsigned int keylen)
> +{
> + if (keylen == AES_KEYSIZE_128 || keylen == AES_KEYSIZE_192 ||
> + keylen == AES_KEYSIZE_256)
> + return skcipher_setkey(cipher, key, keylen);
> +
> + return -EINVAL;
> +}
> +
> +static struct talitos_edesc *skcipher_edesc_alloc(struct skcipher_request *
> + areq, bool encrypt)
> +{
> + struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> + unsigned int ivsize = crypto_skcipher_ivsize(cipher);
> +
> + return talitos_edesc_alloc(ctx->dev, areq->src, areq->dst,
> + areq->iv, 0, areq->cryptlen, 0, ivsize, 0,
> + areq->base.flags, encrypt);
> +}
> +
> +static int skcipher_encrypt(struct skcipher_request *areq)
> +{
> + struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> + struct talitos_edesc *edesc;
> + unsigned int blocksize =
> + crypto_tfm_alg_blocksize(crypto_skcipher_tfm(cipher));
> +
> + if (!areq->cryptlen)
> + return 0;
> +
> + if (areq->cryptlen % blocksize)
> + return -EINVAL;
> +
> + /* allocate extended descriptor */
> + edesc = skcipher_edesc_alloc(areq, true);
> + if (IS_ERR(edesc))
> + return PTR_ERR(edesc);
> +
> + /* set encrypt */
> + edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_MODE0_ENCRYPT;
> +
> + return common_nonsnoop(edesc, areq, skcipher_done);
> +}
> +
> +static int skcipher_decrypt(struct skcipher_request *areq)
> +{
> + struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> + struct talitos_edesc *edesc;
> + unsigned int blocksize =
> + crypto_tfm_alg_blocksize(crypto_skcipher_tfm(cipher));
> +
> + if (!areq->cryptlen)
> + return 0;
> +
> + if (areq->cryptlen % blocksize)
> + return -EINVAL;
> +
> + /* allocate extended descriptor */
> + edesc = skcipher_edesc_alloc(areq, false);
> + if (IS_ERR(edesc))
> + return PTR_ERR(edesc);
> +
> + edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_DIR_INBOUND;
> +
> + return common_nonsnoop(edesc, areq, skcipher_done);
> +}
> +
> +static int talitos_cra_init_skcipher(struct crypto_skcipher *tfm)
> +{
> + struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
> + struct talitos_crypto_alg *talitos_alg;
> + struct talitos_ctx *ctx = crypto_skcipher_ctx(tfm);
> +
> + talitos_alg = container_of(alg, struct talitos_crypto_alg,
> + algt.alg.skcipher);
> +
> + return talitos_init_common(ctx, talitos_alg);
> +}
> +
> +static struct talitos_alg_template skcipher_driver_algs[] = {
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "ecb(aes)",
> + .base.cra_driver_name = "ecb-aes-talitos",
> + .base.cra_blocksize = AES_BLOCK_SIZE,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = AES_MIN_KEY_SIZE,
> + .max_keysize = AES_MAX_KEY_SIZE,
> + .setkey = skcipher_aes_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "cbc(aes)",
> + .base.cra_driver_name = "cbc-aes-talitos",
> + .base.cra_blocksize = AES_BLOCK_SIZE,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = AES_MIN_KEY_SIZE,
> + .max_keysize = AES_MAX_KEY_SIZE,
> + .ivsize = AES_BLOCK_SIZE,
> + .setkey = skcipher_aes_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CBC,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "ctr(aes)",
> + .base.cra_driver_name = "ctr-aes-talitos",
> + .base.cra_blocksize = 1,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = AES_MIN_KEY_SIZE,
> + .max_keysize = AES_MAX_KEY_SIZE,
> + .ivsize = AES_BLOCK_SIZE,
> + .setkey = skcipher_aes_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_AESU_CTR_NONSNOOP |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CTR,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "ctr(aes)",
> + .base.cra_driver_name = "ctr-aes-talitos",
> + .base.cra_blocksize = 1,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = AES_MIN_KEY_SIZE,
> + .max_keysize = AES_MAX_KEY_SIZE,
> + .ivsize = AES_BLOCK_SIZE,
> + .setkey = skcipher_aes_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_AESU |
> + DESC_HDR_MODE0_AESU_CTR,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "ecb(des)",
> + .base.cra_driver_name = "ecb-des-talitos",
> + .base.cra_blocksize = DES_BLOCK_SIZE,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = DES_KEY_SIZE,
> + .max_keysize = DES_KEY_SIZE,
> + .setkey = skcipher_des_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "cbc(des)",
> + .base.cra_driver_name = "cbc-des-talitos",
> + .base.cra_blocksize = DES_BLOCK_SIZE,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = DES_KEY_SIZE,
> + .max_keysize = DES_KEY_SIZE,
> + .ivsize = DES_BLOCK_SIZE,
> + .setkey = skcipher_des_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "ecb(des3_ede)",
> + .base.cra_driver_name = "ecb-3des-talitos",
> + .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = DES3_EDE_KEY_SIZE,
> + .max_keysize = DES3_EDE_KEY_SIZE,
> + .setkey = skcipher_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_3DES,
> + },
> + { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> + .alg.skcipher = {
> + .base.cra_name = "cbc(des3_ede)",
> + .base.cra_driver_name = "cbc-3des-talitos",
> + .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> + .base.cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY,
> + .min_keysize = DES3_EDE_KEY_SIZE,
> + .max_keysize = DES3_EDE_KEY_SIZE,
> + .ivsize = DES3_EDE_BLOCK_SIZE,
> + .setkey = skcipher_des3_setkey,
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_DEU |
> + DESC_HDR_MODE0_DEU_CBC |
> + DESC_HDR_MODE0_DEU_3DES,
> + },
> +};
> +
> +int talitos_register_skcipher(struct device *dev)
> +{
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + struct skcipher_alg *skcipher_alg;
> + struct crypto_alg *alg;
> + size_t i;
> + int ret;
> +
> + for (i = 0; i < ARRAY_SIZE(skcipher_driver_algs); i++) {
> + if (!talitos_hw_supports(
> + dev, skcipher_driver_algs[i].desc_hdr_template))
> + continue;
> +
> + skcipher_alg = &skcipher_driver_algs[i].alg.skcipher;
> + alg = &skcipher_alg->base;
> +
> + alg->cra_exit = talitos_cra_exit;
> + skcipher_alg->init = talitos_cra_init_skcipher;
> + skcipher_alg->setkey = skcipher_alg->setkey ?: skcipher_setkey;
> + skcipher_alg->encrypt = skcipher_encrypt;
> + skcipher_alg->decrypt = skcipher_decrypt;
> +
> + if (!strcmp(alg->cra_name, "ctr(aes)") && !has_ftr_sec1(priv) &&
> + DESC_TYPE(skcipher_driver_algs[i].desc_hdr_template) !=
> + DESC_TYPE(DESC_HDR_TYPE_AESU_CTR_NONSNOOP)) {
> + continue;
> + }
> +
> + ret = talitos_register_common(dev, &skcipher_driver_algs[i]);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/crypto/talitos/talitos.c b/drivers/crypto/talitos/talitos.c
> index b8bcb970d7d5..cd37bc379f86 100644
> --- a/drivers/crypto/talitos/talitos.c
> +++ b/drivers/crypto/talitos/talitos.c
> @@ -1430,215 +1430,6 @@ static int aead_decrypt(struct aead_request *req)
> return ipsec_esp(edesc, req, false, ipsec_esp_decrypt_swauth_done);
> }
>
> -static int skcipher_setkey(struct crypto_skcipher *cipher,
> - const u8 *key, unsigned int keylen)
> -{
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> - struct device *dev = ctx->dev;
> -
> - if (ctx->keylen)
> - dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> -
> - memcpy(&ctx->key, key, keylen);
> - ctx->keylen = keylen;
> -
> - ctx->dma_key = dma_map_single(dev, ctx->key, keylen, DMA_TO_DEVICE);
> -
> - return 0;
> -}
> -
> -static int skcipher_des_setkey(struct crypto_skcipher *cipher,
> - const u8 *key, unsigned int keylen)
> -{
> - return verify_skcipher_des_key(cipher, key) ?:
> - skcipher_setkey(cipher, key, keylen);
> -}
> -
> -static int skcipher_des3_setkey(struct crypto_skcipher *cipher,
> - const u8 *key, unsigned int keylen)
> -{
> - return verify_skcipher_des3_key(cipher, key) ?:
> - skcipher_setkey(cipher, key, keylen);
> -}
> -
> -static int skcipher_aes_setkey(struct crypto_skcipher *cipher,
> - const u8 *key, unsigned int keylen)
> -{
> - if (keylen == AES_KEYSIZE_128 || keylen == AES_KEYSIZE_192 ||
> - keylen == AES_KEYSIZE_256)
> - return skcipher_setkey(cipher, key, keylen);
> -
> - return -EINVAL;
> -}
> -
> -static void common_nonsnoop_unmap(struct device *dev,
> - struct talitos_edesc *edesc,
> - struct skcipher_request *areq)
> -{
> - unmap_single_talitos_ptr(dev, &edesc->desc.ptr[5], DMA_FROM_DEVICE);
> -
> - talitos_sg_unmap(dev, edesc, areq->src, areq->dst, areq->cryptlen, 0);
> - unmap_single_talitos_ptr(dev, &edesc->desc.ptr[1], DMA_TO_DEVICE);
> -
> - if (edesc->dma_len)
> - dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len,
> - DMA_BIDIRECTIONAL);
> -}
> -
> -static void skcipher_done(struct device *dev,
> - struct talitos_desc *desc, void *context,
> - int err)
> -{
> - struct skcipher_request *areq = context;
> - struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> - unsigned int ivsize = crypto_skcipher_ivsize(cipher);
> - struct talitos_edesc *edesc;
> -
> - edesc = container_of(desc, struct talitos_edesc, desc);
> -
> - common_nonsnoop_unmap(dev, edesc, areq);
> - memcpy(areq->iv, ctx->iv, ivsize);
> -
> - kfree(edesc);
> -
> - skcipher_request_complete(areq, err);
> -}
> -
> -static int common_nonsnoop(struct talitos_edesc *edesc,
> - struct skcipher_request *areq,
> - void (*callback) (struct device *dev,
> - struct talitos_desc *desc,
> - void *context, int error))
> -{
> - struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> - struct device *dev = ctx->dev;
> - struct talitos_desc *desc = &edesc->desc;
> - unsigned int cryptlen = areq->cryptlen;
> - unsigned int ivsize = crypto_skcipher_ivsize(cipher);
> - int sg_count, ret;
> - bool sync_needed = false;
> - struct talitos_private *priv = dev_get_drvdata(dev);
> - bool is_sec1 = has_ftr_sec1(priv);
> - bool is_ctr = (desc->hdr & DESC_HDR_SEL0_MASK) == DESC_HDR_SEL0_AESU &&
> - (desc->hdr & DESC_HDR_MODE0_AESU_MASK) == DESC_HDR_MODE0_AESU_CTR;
> -
> - /* first DWORD empty */
> -
> - /* cipher iv */
> - to_talitos_ptr(&desc->ptr[1], edesc->iv_dma, ivsize, is_sec1);
> -
> - /* cipher key */
> - to_talitos_ptr(&desc->ptr[2], ctx->dma_key, ctx->keylen, is_sec1);
> -
> - sg_count = edesc->src_nents ?: 1;
> - if (is_sec1 && sg_count > 1)
> - sg_copy_to_buffer(areq->src, sg_count, edesc->buf,
> - cryptlen);
> - else
> - sg_count = dma_map_sg(dev, areq->src, sg_count,
> - (areq->src == areq->dst) ?
> - DMA_BIDIRECTIONAL : DMA_TO_DEVICE);
> - /*
> - * cipher in
> - */
> - sg_count = talitos_sg_map_ext(dev, areq->src, cryptlen, edesc, &desc->ptr[3],
> - sg_count, 0, 0, 0, false, is_ctr ? 16 : 1);
> - if (sg_count > 1)
> - sync_needed = true;
> -
> - /* cipher out */
> - if (areq->src != areq->dst) {
> - sg_count = edesc->dst_nents ? : 1;
> - if (!is_sec1 || sg_count == 1)
> - dma_map_sg(dev, areq->dst, sg_count, DMA_FROM_DEVICE);
> - }
> -
> - ret = talitos_sg_map(dev, areq->dst, cryptlen, edesc, &desc->ptr[4],
> - sg_count, 0, (edesc->src_nents + 1));
> - if (ret > 1)
> - sync_needed = true;
> -
> - /* iv out */
> - map_single_talitos_ptr(dev, &desc->ptr[5], ivsize, ctx->iv,
> - DMA_FROM_DEVICE);
> -
> - /* last DWORD empty */
> -
> - if (sync_needed)
> - dma_sync_single_for_device(dev, edesc->dma_link_tbl,
> - edesc->dma_len, DMA_BIDIRECTIONAL);
> -
> - ret = talitos_submit(dev, ctx->ch, desc, callback, areq);
> - if (ret != -EINPROGRESS) {
> - common_nonsnoop_unmap(dev, edesc, areq);
> - kfree(edesc);
> - }
> - return ret;
> -}
> -
> -static struct talitos_edesc *skcipher_edesc_alloc(struct skcipher_request *
> - areq, bool encrypt)
> -{
> - struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> - unsigned int ivsize = crypto_skcipher_ivsize(cipher);
> -
> - return talitos_edesc_alloc(ctx->dev, areq->src, areq->dst,
> - areq->iv, 0, areq->cryptlen, 0, ivsize, 0,
> - areq->base.flags, encrypt);
> -}
> -
> -static int skcipher_encrypt(struct skcipher_request *areq)
> -{
> - struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> - struct talitos_edesc *edesc;
> - unsigned int blocksize =
> - crypto_tfm_alg_blocksize(crypto_skcipher_tfm(cipher));
> -
> - if (!areq->cryptlen)
> - return 0;
> -
> - if (areq->cryptlen % blocksize)
> - return -EINVAL;
> -
> - /* allocate extended descriptor */
> - edesc = skcipher_edesc_alloc(areq, true);
> - if (IS_ERR(edesc))
> - return PTR_ERR(edesc);
> -
> - /* set encrypt */
> - edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_MODE0_ENCRYPT;
> -
> - return common_nonsnoop(edesc, areq, skcipher_done);
> -}
> -
> -static int skcipher_decrypt(struct skcipher_request *areq)
> -{
> - struct crypto_skcipher *cipher = crypto_skcipher_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(cipher);
> - struct talitos_edesc *edesc;
> - unsigned int blocksize =
> - crypto_tfm_alg_blocksize(crypto_skcipher_tfm(cipher));
> -
> - if (!areq->cryptlen)
> - return 0;
> -
> - if (areq->cryptlen % blocksize)
> - return -EINVAL;
> -
> - /* allocate extended descriptor */
> - edesc = skcipher_edesc_alloc(areq, false);
> - if (IS_ERR(edesc))
> - return PTR_ERR(edesc);
> -
> - edesc->desc.hdr = ctx->desc_hdr_template | DESC_HDR_DIR_INBOUND;
> -
> - return common_nonsnoop(edesc, areq, skcipher_done);
> -}
> -
> static struct talitos_alg_template driver_algs[] = {
> /* AEAD algorithms. These use a single-pass ipsec_esp descriptor */
> { .type = CRYPTO_ALG_TYPE_AEAD,
> @@ -2097,131 +1888,6 @@ static struct talitos_alg_template driver_algs[] = {
> DESC_HDR_MODE1_MDEU_PAD |
> DESC_HDR_MODE1_MDEU_MD5_HMAC,
> },
> - /* SKCIPHER algorithms. */
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ecb(aes)",
> - .base.cra_driver_name = "ecb-aes-talitos",
> - .base.cra_blocksize = AES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "cbc(aes)",
> - .base.cra_driver_name = "cbc-aes-talitos",
> - .base.cra_blocksize = AES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .ivsize = AES_BLOCK_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CBC,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ctr(aes)",
> - .base.cra_driver_name = "ctr-aes-talitos",
> - .base.cra_blocksize = 1,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .ivsize = AES_BLOCK_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_AESU_CTR_NONSNOOP |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CTR,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ctr(aes)",
> - .base.cra_driver_name = "ctr-aes-talitos",
> - .base.cra_blocksize = 1,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = AES_MIN_KEY_SIZE,
> - .max_keysize = AES_MAX_KEY_SIZE,
> - .ivsize = AES_BLOCK_SIZE,
> - .setkey = skcipher_aes_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_AESU |
> - DESC_HDR_MODE0_AESU_CTR,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ecb(des)",
> - .base.cra_driver_name = "ecb-des-talitos",
> - .base.cra_blocksize = DES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = DES_KEY_SIZE,
> - .max_keysize = DES_KEY_SIZE,
> - .setkey = skcipher_des_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "cbc(des)",
> - .base.cra_driver_name = "cbc-des-talitos",
> - .base.cra_blocksize = DES_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = DES_KEY_SIZE,
> - .max_keysize = DES_KEY_SIZE,
> - .ivsize = DES_BLOCK_SIZE,
> - .setkey = skcipher_des_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "ecb(des3_ede)",
> - .base.cra_driver_name = "ecb-3des-talitos",
> - .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = DES3_EDE_KEY_SIZE,
> - .max_keysize = DES3_EDE_KEY_SIZE,
> - .setkey = skcipher_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_3DES,
> - },
> - { .type = CRYPTO_ALG_TYPE_SKCIPHER,
> - .alg.skcipher = {
> - .base.cra_name = "cbc(des3_ede)",
> - .base.cra_driver_name = "cbc-3des-talitos",
> - .base.cra_blocksize = DES3_EDE_BLOCK_SIZE,
> - .base.cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY,
> - .min_keysize = DES3_EDE_KEY_SIZE,
> - .max_keysize = DES3_EDE_KEY_SIZE,
> - .ivsize = DES3_EDE_BLOCK_SIZE,
> - .setkey = skcipher_des3_setkey,
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_DEU |
> - DESC_HDR_MODE0_DEU_CBC |
> - DESC_HDR_MODE0_DEU_3DES,
> - },
> };
>
> int talitos_init_common(struct talitos_ctx *ctx,
> @@ -2258,18 +1924,6 @@ static int talitos_cra_init_aead(struct crypto_aead *tfm)
> return talitos_init_common(ctx, talitos_alg);
> }
>
> -static int talitos_cra_init_skcipher(struct crypto_skcipher *tfm)
> -{
> - struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
> - struct talitos_crypto_alg *talitos_alg;
> - struct talitos_ctx *ctx = crypto_skcipher_ctx(tfm);
> -
> - talitos_alg = container_of(alg, struct talitos_crypto_alg,
> - algt.alg.skcipher);
> -
> - return talitos_init_common(ctx, talitos_alg);
> -}
> -
> void talitos_cra_exit(struct crypto_tfm *tfm)
> {
> struct talitos_ctx *ctx = crypto_tfm_ctx(tfm);
> @@ -2374,6 +2028,12 @@ int talitos_register_common(struct device *dev,
> t_alg->algt.type);
> ret = crypto_register_ahash(&t_alg->algt.alg.hash);
> break;
> + case CRYPTO_ALG_TYPE_SKCIPHER:
> + alg = &t_alg->algt.alg.skcipher.base;
> + talitos_alg_set_common(priv, alg, t_alg->algt.priority,
> + t_alg->algt.type);
> + ret = crypto_register_skcipher(&t_alg->algt.alg.skcipher);
> + break;
> default:
> dev_err(dev, "unknown algorithm type %d\n", t_alg->algt.type);
> devm_kfree(dev, t_alg);
> @@ -2410,21 +2070,6 @@ static struct talitos_crypto_alg *talitos_alg_alloc(struct device *dev,
> t_alg->algt = *template;
>
> switch (t_alg->algt.type) {
> - case CRYPTO_ALG_TYPE_SKCIPHER:
> - alg = &t_alg->algt.alg.skcipher.base;
> - alg->cra_exit = talitos_cra_exit;
> - t_alg->algt.alg.skcipher.init = talitos_cra_init_skcipher;
> - t_alg->algt.alg.skcipher.setkey =
> - t_alg->algt.alg.skcipher.setkey ?: skcipher_setkey;
> - t_alg->algt.alg.skcipher.encrypt = skcipher_encrypt;
> - t_alg->algt.alg.skcipher.decrypt = skcipher_decrypt;
> - if (!strcmp(alg->cra_name, "ctr(aes)") && !has_ftr_sec1(priv) &&
> - DESC_TYPE(t_alg->algt.desc_hdr_template) !=
> - DESC_TYPE(DESC_HDR_TYPE_AESU_CTR_NONSNOOP)) {
> - devm_kfree(dev, t_alg);
> - return ERR_PTR(-ENOTSUPP);
> - }
> - break;
> case CRYPTO_ALG_TYPE_AEAD:
> alg = &t_alg->algt.alg.aead.base;
> alg->cra_exit = talitos_cra_exit;
> @@ -2671,6 +2316,10 @@ static int talitos_probe(struct platform_device *ofdev)
> if (err)
> goto err_out;
>
> + err = talitos_register_skcipher(dev);
> + if (err)
> + goto err_out;
> +
> /* register crypto algorithms the device supports */
> for (i = 0; i < ARRAY_SIZE(driver_algs); i++) {
> if (talitos_hw_supports(dev,
> @@ -2687,12 +2336,6 @@ static int talitos_probe(struct platform_device *ofdev)
> }
>
> switch (t_alg->algt.type) {
> - case CRYPTO_ALG_TYPE_SKCIPHER:
> - err = crypto_register_skcipher(
> - &t_alg->algt.alg.skcipher);
> - alg = &t_alg->algt.alg.skcipher.base;
> - break;
> -
> case CRYPTO_ALG_TYPE_AEAD:
> err = crypto_register_aead(
> &t_alg->algt.alg.aead);
> diff --git a/drivers/crypto/talitos/talitos.h b/drivers/crypto/talitos/talitos.h
> index e703c18cb81f..7e7d41673fa5 100644
> --- a/drivers/crypto/talitos/talitos.h
> +++ b/drivers/crypto/talitos/talitos.h
> @@ -534,3 +534,4 @@ void talitos_unregister_rng(struct device *dev);
> /* Hash */
>
> int talitos_register_hash(struct device *dev);
> +int talitos_register_skcipher(struct device *dev);
>
^ permalink raw reply
* Re: [PATCH 07/29] crypto: talitos/hash - Move into separate file
From: Christophe Leroy (CS GROUP) @ 2026-06-01 11:47 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <20260528-7-1-rc1_talitos_cleanup-v1-7-cb1ad6cdea49@bootlin.com>
Le 28/05/2026 à 11:08, Paul Louvel a écrit :
> Move the ahash algorithm implementations from talitos.c into a dedicated
> talitos-hash.c file.
>
> Signed-off-by: Paul Louvel <paul.louvel@bootlin.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
While at it, (maybe another patch) the problem with size 0 hash on SEC1
is not a driver bug but a hardware bug, it is therefore probably not
worth a pr_err() because there is nothing the user can do about it and
it is transparent as there is a workaround in the driver.
> ---
> drivers/crypto/talitos/Makefile | 2 +-
> drivers/crypto/talitos/talitos-hash.c | 832 ++++++++++++++++++++++++++++++++++
> drivers/crypto/talitos/talitos.c | 807 +--------------------------------
> drivers/crypto/talitos/talitos.h | 4 +
> 4 files changed, 847 insertions(+), 798 deletions(-)
>
> diff --git a/drivers/crypto/talitos/Makefile b/drivers/crypto/talitos/Makefile
> index 901ec681f010..40d37f9364ef 100644
> --- a/drivers/crypto/talitos/Makefile
> +++ b/drivers/crypto/talitos/Makefile
> @@ -1,3 +1,3 @@
> obj-$(CONFIG_CRYPTO_DEV_TALITOS) += talitos.o
>
> -talitos-y := talitos.o talitos-rng.o
> +talitos-y := talitos.o talitos-rng.o talitos-hash.o
> diff --git a/drivers/crypto/talitos/talitos-hash.c b/drivers/crypto/talitos/talitos-hash.c
> new file mode 100644
> index 000000000000..5792e7093392
> --- /dev/null
> +++ b/drivers/crypto/talitos/talitos-hash.c
> @@ -0,0 +1,832 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +
> +/*
> + * Freescale SEC (talitos) hash implementation
> + *
> + * Copyright (c) 2006-2011 Freescale Semiconductor, Inc.
> + */
> +
> +#include <linux/scatterlist.h>
> +
> +#include <crypto/hash.h>
> +#include <crypto/internal/hash.h>
> +#include <crypto/md5.h>
> +#include <crypto/scatterwalk.h>
> +#include <crypto/sha1.h>
> +
> +#include "talitos.h"
> +
> +#define HASH_MAX_BLOCK_SIZE SHA512_BLOCK_SIZE
> +#define TALITOS_MDEU_MAX_CONTEXT_SIZE TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512
> +
> +struct talitos_ahash_req_ctx {
> + u32 hw_context[TALITOS_MDEU_MAX_CONTEXT_SIZE / sizeof(u32)];
> + unsigned int hw_context_size;
> + unsigned int swinit;
> + unsigned int first_request;
> + unsigned int last_request;
> + unsigned int to_hash_later;
> +};
> +
> +struct talitos_export_state {
> + u32 hw_context[TALITOS_MDEU_MAX_CONTEXT_SIZE / sizeof(u32)];
> + unsigned int swinit;
> + unsigned int first_request;
> + unsigned int last_request;
> + unsigned int to_hash_later;
> +};
> +
> +static void common_nonsnoop_hash_unmap(struct device *dev,
> + struct talitos_edesc *edesc,
> + struct ahash_request *areq)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + bool is_sec1 = has_ftr_sec1(priv);
> + struct talitos_desc *desc = &edesc->desc;
> +
> + unmap_single_talitos_ptr(dev, &desc->ptr[5], DMA_FROM_DEVICE);
> +
> + if (edesc->last && req_ctx->last_request)
> + memcpy(areq->result, req_ctx->hw_context,
> + crypto_ahash_digestsize(tfm));
> +
> + if (edesc->src)
> + talitos_sg_unmap(dev, edesc, edesc->src, NULL, 0, 0);
> +
> + /* When using hashctx-in, must unmap it. */
> + if (from_talitos_ptr_len(&desc->ptr[1], is_sec1))
> + unmap_single_talitos_ptr(dev, &desc->ptr[1],
> + DMA_TO_DEVICE);
> +
> + if (edesc->dma_len)
> + dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len,
> + DMA_BIDIRECTIONAL);
> +}
> +
> +static void free_edesc_list_from(struct ahash_request *areq, struct talitos_edesc *edesc)
> +{
> + struct talitos_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(areq));
> + struct talitos_edesc *next;
> +
> + while (edesc) {
> + next = edesc->next_desc;
> + common_nonsnoop_hash_unmap(ctx->dev, edesc, areq);
> + kfree(edesc);
> + edesc = next;
> + }
> +}
> +
> +static void ahash_done(struct device *dev,
> + struct talitos_desc *desc, void *context,
> + int err)
> +{
> + struct ahash_request *areq = context;
> + struct talitos_edesc *edesc =
> + container_of(desc, struct talitos_edesc, desc);
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + bool is_sec1 = has_ftr_sec1(dev_get_drvdata(dev));
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> + struct talitos_edesc *next;
> +
> + if (is_sec1) {
> + free_edesc_list_from(areq, edesc);
> + ahash_request_complete(areq, err ?: req_ctx->to_hash_later);
> + } else {
> + next = edesc->next_desc;
> +
> + common_nonsnoop_hash_unmap(dev, edesc, areq);
> + kfree(edesc);
> +
> + if (err)
> + goto out;
> +
> + if (next) {
> + err = talitos_submit(dev, ctx->ch, &next->desc,
> + ahash_done, areq);
> + if (err != -EINPROGRESS)
> + goto out;
> + return;
> + }
> +out:
> + if (err && next)
> + free_edesc_list_from(areq, next);
> + ahash_request_complete(areq, err ?: req_ctx->to_hash_later);
> + }
> +}
> +
> +/*
> + * SEC1 doesn't like hashing of 0 sized message, so we do the padding
> + * ourself and submit a padded block
> + */
> +static void talitos_handle_buggy_hash(struct talitos_ctx *ctx,
> + struct talitos_edesc *edesc,
> + struct talitos_ptr *ptr)
> +{
> + static u8 padded_hash[64] = {
> + 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> + };
> +
> + pr_err_once("Bug in SEC1, padding ourself\n");
> + edesc->desc.hdr &= ~DESC_HDR_MODE0_MDEU_PAD;
> + map_single_talitos_ptr(ctx->dev, ptr, sizeof(padded_hash),
> + (char *)padded_hash, DMA_TO_DEVICE);
> +}
> +
> +static void common_nonsnoop_hash(struct talitos_edesc *edesc,
> + struct ahash_request *areq,
> + unsigned int length)
> +{
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct device *dev = ctx->dev;
> + struct talitos_desc *desc = &edesc->desc;
> + bool sync_needed = false;
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + bool is_sec1 = has_ftr_sec1(priv);
> + int sg_count;
> +
> + /* first DWORD empty */
> +
> + /* hash context in */
> + if (!edesc->first || !req_ctx->first_request || req_ctx->swinit) {
> + map_single_talitos_ptr_nosync(dev, &desc->ptr[1],
> + req_ctx->hw_context_size,
> + req_ctx->hw_context,
> + DMA_TO_DEVICE);
> + req_ctx->swinit = 0;
> + }
> + /* Indicate next op is not the first. */
> + req_ctx->first_request = 0;
> +
> + /* HMAC key */
> + if (ctx->keylen)
> + to_talitos_ptr(&desc->ptr[2], ctx->dma_key, ctx->keylen,
> + is_sec1);
> +
> + sg_count = edesc->src_nents ?: 1;
> + if (is_sec1 && sg_count > 1)
> + sg_copy_to_buffer(edesc->src, sg_count, edesc->buf, length);
> + else if (length)
> + sg_count = dma_map_sg(dev, edesc->src, sg_count, DMA_TO_DEVICE);
> +
> + /*
> + * data in
> + */
> + sg_count = talitos_sg_map(dev, edesc->src, length, edesc, &desc->ptr[3],
> + sg_count, 0, 0);
> + if (sg_count > 1)
> + sync_needed = true;
> +
> + /* fifth DWORD empty */
> +
> + /* hash/HMAC out -or- hash context out */
> + if (edesc->last && req_ctx->last_request)
> + map_single_talitos_ptr(dev, &desc->ptr[5],
> + crypto_ahash_digestsize(tfm),
> + req_ctx->hw_context, DMA_FROM_DEVICE);
> + else
> + map_single_talitos_ptr_nosync(dev, &desc->ptr[5],
> + req_ctx->hw_context_size,
> + req_ctx->hw_context,
> + DMA_FROM_DEVICE);
> +
> + /* last DWORD empty */
> +
> + if (is_sec1 && from_talitos_ptr_len(&desc->ptr[3], true) == 0)
> + talitos_handle_buggy_hash(ctx, edesc, &desc->ptr[3]);
> +
> + if (sync_needed)
> + dma_sync_single_for_device(dev, edesc->dma_link_tbl,
> + edesc->dma_len, DMA_BIDIRECTIONAL);
> +}
> +
> +static struct talitos_edesc *ahash_edesc_alloc(struct ahash_request *areq,
> + struct scatterlist *src,
> + unsigned int nbytes)
> +{
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> +
> + return talitos_edesc_alloc(ctx->dev, src, NULL, NULL, 0,
> + nbytes, 0, 0, 0, areq->base.flags, false);
> +}
> +
> +static struct talitos_edesc *
> +ahash_process_req_prepare(struct ahash_request *areq, unsigned int nbytes,
> + unsigned int blocksize, bool is_sec1)
> +{
> + struct talitos_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(areq));
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct talitos_edesc *first = NULL, *prev_edesc = NULL, *edesc;
> + size_t desc_max = is_sec1 ? TALITOS1_MAX_DATA_LEN :
> + TALITOS2_MAX_DATA_LEN;
> + struct scatterlist tmp[2];
> + size_t to_hash_this_desc;
> + struct scatterlist *src;
> + size_t offset = 0;
> +
> + do {
> + src = scatterwalk_ffwd(tmp, areq->src, offset);
> +
> + to_hash_this_desc =
> + min(nbytes, ALIGN_DOWN(desc_max, blocksize));
> +
> + /* Allocate extended descriptor */
> + edesc = ahash_edesc_alloc(areq, src, to_hash_this_desc);
> + if (IS_ERR(edesc)) {
> + if (first)
> + free_edesc_list_from(areq, first);
> + return edesc;
> + }
> +
> + edesc->src = scatterwalk_ffwd(edesc->bufsl, areq->src, offset);
> + edesc->desc.hdr = ctx->desc_hdr_template;
> + edesc->first = offset == 0;
> + edesc->last = nbytes - to_hash_this_desc == 0;
> +
> + /* On last one, request SEC to pad; otherwise continue */
> + if (req_ctx->last_request && edesc->last)
> + edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_PAD;
> + else
> + edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_CONT;
> +
> + /* request SEC to INIT hash. */
> + if (req_ctx->first_request && edesc->first && !req_ctx->swinit)
> + edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_INIT;
> +
> + /*
> + * When the tfm context has a keylen, it's an HMAC.
> + * A first or last (ie. not middle) descriptor must request HMAC.
> + */
> + if (ctx->keylen && ((req_ctx->first_request && edesc->first) ||
> + (req_ctx->last_request && edesc->last)))
> + edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_HMAC;
> +
> + /* clear the DN bit */
> + if (is_sec1 && !edesc->last)
> + edesc->desc.hdr &= ~DESC_HDR_DONE_NOTIFY;
> +
> + common_nonsnoop_hash(edesc, areq, to_hash_this_desc);
> +
> + offset += to_hash_this_desc;
> + nbytes -= to_hash_this_desc;
> +
> + if (!prev_edesc)
> + first = edesc;
> + else
> + prev_edesc->next_desc = edesc;
> + prev_edesc = edesc;
> + } while (nbytes);
> +
> + return first;
> +}
> +
> +static int ahash_process_req(struct ahash_request *areq, unsigned int nbytes)
> +{
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct talitos_edesc *edesc;
> + unsigned int blocksize =
> + crypto_tfm_alg_blocksize(crypto_ahash_tfm(tfm));
> + bool is_sec1 = has_ftr_sec1(dev_get_drvdata(ctx->dev));
> + unsigned int nbytes_to_hash;
> + unsigned int to_hash_later;
> + struct device *dev = ctx->dev;
> + int ret;
> +
> + nbytes_to_hash = ALIGN_DOWN(nbytes, blocksize);
> + to_hash_later = nbytes - nbytes_to_hash;
> +
> + if (req_ctx->last_request) {
> + nbytes_to_hash = nbytes;
> + to_hash_later = 0;
> + }
> +
> + req_ctx->to_hash_later = to_hash_later;
> +
> + edesc = ahash_process_req_prepare(areq, nbytes_to_hash, blocksize,
> + is_sec1);
> + if (IS_ERR(edesc))
> + return PTR_ERR(edesc);
> +
> + ret = talitos_submit(dev, ctx->ch, &edesc->desc, ahash_done, areq);
> + if (ret != -EINPROGRESS)
> + free_edesc_list_from(areq, edesc);
> +
> + return ret;
> +}
> +
> +static int ahash_init(struct ahash_request *areq)
> +{
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> + struct device *dev = ctx->dev;
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + unsigned int size;
> + dma_addr_t dma;
> +
> + /* Initialize the context */
> + req_ctx->first_request = 1;
> + req_ctx->swinit = 0; /* assume h/w init of context */
> + size = (crypto_ahash_digestsize(tfm) <= SHA256_DIGEST_SIZE)
> + ? TALITOS_MDEU_CONTEXT_SIZE_MD5_SHA1_SHA256
> + : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512;
> + req_ctx->hw_context_size = size;
> + req_ctx->last_request = 0;
> +
> + dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size,
> + DMA_TO_DEVICE);
> + dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_TO_DEVICE);
> +
> + return 0;
> +}
> +
> +/*
> + * on h/w without explicit sha224 support, we initialize h/w context
> + * manually with sha224 constants, and tell it to run sha256.
> + */
> +static int ahash_init_sha224_swinit(struct ahash_request *areq)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> +
> + req_ctx->hw_context[0] = SHA224_H0;
> + req_ctx->hw_context[1] = SHA224_H1;
> + req_ctx->hw_context[2] = SHA224_H2;
> + req_ctx->hw_context[3] = SHA224_H3;
> + req_ctx->hw_context[4] = SHA224_H4;
> + req_ctx->hw_context[5] = SHA224_H5;
> + req_ctx->hw_context[6] = SHA224_H6;
> + req_ctx->hw_context[7] = SHA224_H7;
> +
> + /* init 64-bit count */
> + req_ctx->hw_context[8] = 0;
> + req_ctx->hw_context[9] = 0;
> +
> + ahash_init(areq);
> + req_ctx->swinit = 1;/* prevent h/w initting context with sha256 values*/
> +
> + return 0;
> +}
> +
> +static int ahash_update(struct ahash_request *areq)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> +
> + req_ctx->last_request = 0;
> +
> + return ahash_process_req(areq, areq->nbytes);
> +}
> +
> +static int ahash_final(struct ahash_request *areq)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> +
> + req_ctx->last_request = 1;
> +
> + return ahash_process_req(areq, 0);
> +}
> +
> +static int ahash_finup(struct ahash_request *areq)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> +
> + req_ctx->last_request = 1;
> +
> + return ahash_process_req(areq, areq->nbytes);
> +}
> +
> +static int ahash_digest(struct ahash_request *areq)
> +{
> + ahash_init(areq);
> + return ahash_finup(areq);
> +}
> +
> +static int ahash_digest_sha224_swinit(struct ahash_request *areq)
> +{
> + ahash_init_sha224_swinit(areq);
> + return ahash_finup(areq);
> +}
> +
> +static int ahash_export(struct ahash_request *areq, void *out)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct talitos_export_state *export = out;
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> + struct device *dev = ctx->dev;
> + dma_addr_t dma;
> +
> + dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size,
> + DMA_FROM_DEVICE);
> + dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_FROM_DEVICE);
> +
> + memcpy(export->hw_context, req_ctx->hw_context,
> + req_ctx->hw_context_size);
> + export->swinit = req_ctx->swinit;
> + export->first_request = req_ctx->first_request;
> + export->last_request = req_ctx->last_request;
> + export->to_hash_later = req_ctx->to_hash_later;
> +
> + return 0;
> +}
> +
> +static int ahash_import(struct ahash_request *areq, const void *in)
> +{
> + struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> + struct device *dev = ctx->dev;
> + const struct talitos_export_state *export = in;
> + unsigned int size;
> + dma_addr_t dma;
> +
> + memset(req_ctx, 0, sizeof(*req_ctx));
> + size = (crypto_ahash_digestsize(tfm) <= SHA256_DIGEST_SIZE)
> + ? TALITOS_MDEU_CONTEXT_SIZE_MD5_SHA1_SHA256
> + : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512;
> + req_ctx->hw_context_size = size;
> + memcpy(req_ctx->hw_context, export->hw_context, size);
> + req_ctx->swinit = export->swinit;
> + req_ctx->first_request = export->first_request;
> + req_ctx->last_request = export->last_request;
> + req_ctx->to_hash_later = export->to_hash_later;
> +
> + dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size,
> + DMA_TO_DEVICE);
> + dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_TO_DEVICE);
> +
> + return 0;
> +}
> +
> +static int keyhash(struct crypto_ahash *tfm, const u8 *key, unsigned int keylen,
> + u8 *hash)
> +{
> + struct talitos_ctx *ctx = crypto_tfm_ctx(crypto_ahash_tfm(tfm));
> +
> + struct scatterlist sg[1];
> + struct ahash_request *req;
> + struct crypto_wait wait;
> + int ret;
> +
> + crypto_init_wait(&wait);
> +
> + req = ahash_request_alloc(tfm, GFP_KERNEL);
> + if (!req)
> + return -ENOMEM;
> +
> + /* Keep tfm keylen == 0 during hash of the long key */
> + ctx->keylen = 0;
> + ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG,
> + crypto_req_done, &wait);
> +
> + sg_init_one(&sg[0], key, keylen);
> +
> + ahash_request_set_crypt(req, sg, hash, keylen);
> + ret = crypto_wait_req(crypto_ahash_digest(req), &wait);
> +
> + ahash_request_free(req);
> +
> + return ret;
> +}
> +
> +static int ahash_setkey(struct crypto_ahash *tfm, const u8 *key,
> + unsigned int keylen)
> +{
> + struct talitos_ctx *ctx = crypto_tfm_ctx(crypto_ahash_tfm(tfm));
> + struct device *dev = ctx->dev;
> + unsigned int blocksize =
> + crypto_tfm_alg_blocksize(crypto_ahash_tfm(tfm));
> + unsigned int digestsize = crypto_ahash_digestsize(tfm);
> + unsigned int keysize = keylen;
> + u8 hash[SHA512_DIGEST_SIZE];
> + int ret;
> +
> + if (keylen <= blocksize)
> + memcpy(ctx->key, key, keysize);
> + else {
> + /* Must get the hash of the long key */
> + ret = keyhash(tfm, key, keylen, hash);
> +
> + if (ret)
> + return -EINVAL;
> +
> + keysize = digestsize;
> + memcpy(ctx->key, hash, digestsize);
> + }
> +
> + if (ctx->keylen)
> + dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> +
> + ctx->keylen = keysize;
> + ctx->dma_key = dma_map_single(dev, ctx->key, keysize, DMA_TO_DEVICE);
> +
> + return 0;
> +}
> +
> +static int talitos_cra_init_ahash(struct crypto_tfm *tfm)
> +{
> + struct crypto_alg *alg = tfm->__crt_alg;
> + struct talitos_crypto_alg *talitos_alg;
> + struct talitos_ctx *ctx = crypto_tfm_ctx(tfm);
> +
> + talitos_alg = container_of(__crypto_ahash_alg(alg),
> + struct talitos_crypto_alg,
> + algt.alg.hash);
> +
> + ctx->keylen = 0;
> + sizeof(struct talitos_ahash_req_ctx));
> +
> + return talitos_init_common(ctx, talitos_alg);
> +}
> +
> +static struct talitos_alg_template hash_driver_algs[] = {
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = MD5_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "md5",
> + .cra_driver_name = "md5-talitos",
> + .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_MD5,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA1_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "sha1",
> + .cra_driver_name = "sha1-talitos",
> + .cra_blocksize = SHA1_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA1,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA224_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "sha224",
> + .cra_driver_name = "sha224-talitos",
> + .cra_blocksize = SHA224_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA224,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA256_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "sha256",
> + .cra_driver_name = "sha256-talitos",
> + .cra_blocksize = SHA256_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA256,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA384_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "sha384",
> + .cra_driver_name = "sha384-talitos",
> + .cra_blocksize = SHA384_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA384,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA512_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "sha512",
> + .cra_driver_name = "sha512-talitos",
> + .cra_blocksize = SHA512_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA512,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = MD5_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "hmac(md5)",
> + .cra_driver_name = "hmac-md5-talitos",
> + .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_MD5,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA1_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "hmac(sha1)",
> + .cra_driver_name = "hmac-sha1-talitos",
> + .cra_blocksize = SHA1_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA1,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA224_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "hmac(sha224)",
> + .cra_driver_name = "hmac-sha224-talitos",
> + .cra_blocksize = SHA224_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA224,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA256_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "hmac(sha256)",
> + .cra_driver_name = "hmac-sha256-talitos",
> + .cra_blocksize = SHA256_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA256,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA384_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "hmac(sha384)",
> + .cra_driver_name = "hmac-sha384-talitos",
> + .cra_blocksize = SHA384_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA384,
> + },
> + { .type = CRYPTO_ALG_TYPE_AHASH,
> + .alg.hash = {
> + .halg.digestsize = SHA512_DIGEST_SIZE,
> + .halg.statesize = sizeof(struct talitos_export_state),
> + .halg.base = {
> + .cra_name = "hmac(sha512)",
> + .cra_driver_name = "hmac-sha512-talitos",
> + .cra_blocksize = SHA512_BLOCK_SIZE,
> + .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> + .cra_flags = CRYPTO_ALG_ASYNC |
> + CRYPTO_ALG_ALLOCATES_MEMORY |
> + CRYPTO_AHASH_ALG_BLOCK_ONLY |
> + CRYPTO_AHASH_ALG_FINAL_NONZERO,
> + }
> + },
> + .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUB |
> + DESC_HDR_MODE0_MDEUB_SHA512,
> + }
> +};
> +
> +int talitos_register_hash(struct device *dev)
> +{
> + struct talitos_private *priv = dev_get_drvdata(dev);
> + struct ahash_alg *ahash_alg;
> + struct crypto_alg *alg;
> + size_t i;
> + int ret;
> +
> + for (i = 0; i < ARRAY_SIZE(hash_driver_algs); i++) {
> + if (!talitos_hw_supports(dev,
> + hash_driver_algs[i].desc_hdr_template))
> + continue;
> +
> + ahash_alg = &hash_driver_algs[i].alg.hash;
> + alg = &ahash_alg->halg.base;
> +
> + alg->cra_init = talitos_cra_init_ahash;
> + alg->cra_exit = talitos_cra_exit;
> + ahash_alg->init = ahash_init;
> + ahash_alg->update = ahash_update;
> + ahash_alg->final = ahash_final;
> + ahash_alg->finup = ahash_finup;
> + ahash_alg->digest = ahash_digest;
> + if (!strncmp(alg->cra_name, "hmac", 4))
> + ahash_alg->setkey = ahash_setkey;
> + ahash_alg->import = ahash_import;
> + ahash_alg->export = ahash_export;
> +
> + if (!(priv->features & TALITOS_FTR_HMAC_OK) &&
> + !strncmp(alg->cra_name, "hmac", 4)) {
> + /* not supported */
> + continue;
> + }
> +
> + if (!(priv->features & TALITOS_FTR_SHA224_HWINIT) &&
> + (!strcmp(alg->cra_name, "sha224") ||
> + !strcmp(alg->cra_name, "hmac(sha224)"))) {
> + ahash_alg->init = ahash_init_sha224_swinit;
> + ahash_alg->digest = ahash_digest_sha224_swinit;
> + hash_driver_algs[i].desc_hdr_template =
> + DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> + DESC_HDR_SEL0_MDEUA |
> + DESC_HDR_MODE0_MDEU_SHA256;
> + }
> +
> + ret = talitos_register_common(dev, &hash_driver_algs[i]);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/crypto/talitos/talitos.c b/drivers/crypto/talitos/talitos.c
> index 869739dcc4d7..b8bcb970d7d5 100644
> --- a/drivers/crypto/talitos/talitos.c
> +++ b/drivers/crypto/talitos/talitos.c
> @@ -831,26 +831,6 @@ DEF_TALITOS2_INTERRUPT(ch1_3, TALITOS2_ISR_CH_1_3_DONE, TALITOS2_ISR_CH_1_3_ERR,
> */
> #define TALITOS_CRA_PRIORITY_AEAD_HSNA (TALITOS_CRA_PRIORITY - 1)
>
> -#define HASH_MAX_BLOCK_SIZE SHA512_BLOCK_SIZE
> -#define TALITOS_MDEU_MAX_CONTEXT_SIZE TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512
> -
> -struct talitos_ahash_req_ctx {
> - u32 hw_context[TALITOS_MDEU_MAX_CONTEXT_SIZE / sizeof(u32)];
> - unsigned int hw_context_size;
> - unsigned int swinit;
> - unsigned int first_request;
> - unsigned int last_request;
> - unsigned int to_hash_later;
> -};
> -
> -struct talitos_export_state {
> - u32 hw_context[TALITOS_MDEU_MAX_CONTEXT_SIZE / sizeof(u32)];
> - unsigned int swinit;
> - unsigned int first_request;
> - unsigned int last_request;
> - unsigned int to_hash_later;
> -};
> -
> static int aead_setkey(struct crypto_aead *authenc,
> const u8 *key, unsigned int keylen)
> {
> @@ -1659,501 +1639,6 @@ static int skcipher_decrypt(struct skcipher_request *areq)
> return common_nonsnoop(edesc, areq, skcipher_done);
> }
>
> -static void common_nonsnoop_hash_unmap(struct device *dev,
> - struct talitos_edesc *edesc,
> - struct ahash_request *areq)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_private *priv = dev_get_drvdata(dev);
> - bool is_sec1 = has_ftr_sec1(priv);
> - struct talitos_desc *desc = &edesc->desc;
> -
> - unmap_single_talitos_ptr(dev, &desc->ptr[5], DMA_FROM_DEVICE);
> -
> - if (edesc->last && req_ctx->last_request)
> - memcpy(areq->result, req_ctx->hw_context,
> - crypto_ahash_digestsize(tfm));
> -
> - if (edesc->src)
> - talitos_sg_unmap(dev, edesc, edesc->src, NULL, 0, 0);
> -
> - /* When using hashctx-in, must unmap it. */
> - if (from_talitos_ptr_len(&desc->ptr[1], is_sec1))
> - unmap_single_talitos_ptr(dev, &desc->ptr[1],
> - DMA_TO_DEVICE);
> -
> - if (edesc->dma_len)
> - dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len,
> - DMA_BIDIRECTIONAL);
> -}
> -
> -static void free_edesc_list_from(struct ahash_request *areq, struct talitos_edesc *edesc)
> -{
> - struct talitos_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(areq));
> - struct talitos_edesc *next;
> -
> - while (edesc) {
> - next = edesc->next_desc;
> - common_nonsnoop_hash_unmap(ctx->dev, edesc, areq);
> - kfree(edesc);
> - edesc = next;
> - }
> -}
> -
> -static void ahash_done(struct device *dev,
> - struct talitos_desc *desc, void *context,
> - int err)
> -{
> - struct ahash_request *areq = context;
> - struct talitos_edesc *edesc =
> - container_of(desc, struct talitos_edesc, desc);
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - bool is_sec1 = has_ftr_sec1(dev_get_drvdata(dev));
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> - struct talitos_edesc *next;
> -
> - if (is_sec1) {
> - free_edesc_list_from(areq, edesc);
> - ahash_request_complete(areq, err ?: req_ctx->to_hash_later);
> - } else {
> - next = edesc->next_desc;
> -
> - common_nonsnoop_hash_unmap(dev, edesc, areq);
> - kfree(edesc);
> -
> - if (err)
> - goto out;
> -
> - if (next) {
> - err = talitos_submit(dev, ctx->ch, &next->desc,
> - ahash_done, areq);
> - if (err != -EINPROGRESS)
> - goto out;
> - return;
> - }
> -out:
> - if (err && next)
> - free_edesc_list_from(areq, next);
> - ahash_request_complete(areq, err ?: req_ctx->to_hash_later);
> - }
> -}
> -
> -/*
> - * SEC1 doesn't like hashing of 0 sized message, so we do the padding
> - * ourself and submit a padded block
> - */
> -static void talitos_handle_buggy_hash(struct talitos_ctx *ctx,
> - struct talitos_edesc *edesc,
> - struct talitos_ptr *ptr)
> -{
> - static u8 padded_hash[64] = {
> - 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
> - };
> -
> - pr_err_once("Bug in SEC1, padding ourself\n");
> - edesc->desc.hdr &= ~DESC_HDR_MODE0_MDEU_PAD;
> - map_single_talitos_ptr(ctx->dev, ptr, sizeof(padded_hash),
> - (char *)padded_hash, DMA_TO_DEVICE);
> -}
> -
> -static void common_nonsnoop_hash(struct talitos_edesc *edesc,
> - struct ahash_request *areq,
> - unsigned int length)
> -{
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct device *dev = ctx->dev;
> - struct talitos_desc *desc = &edesc->desc;
> - bool sync_needed = false;
> - struct talitos_private *priv = dev_get_drvdata(dev);
> - bool is_sec1 = has_ftr_sec1(priv);
> - int sg_count;
> -
> - /* first DWORD empty */
> -
> - /* hash context in */
> - if (!edesc->first || !req_ctx->first_request || req_ctx->swinit) {
> - map_single_talitos_ptr_nosync(dev, &desc->ptr[1],
> - req_ctx->hw_context_size,
> - req_ctx->hw_context,
> - DMA_TO_DEVICE);
> - req_ctx->swinit = 0;
> - }
> - /* Indicate next op is not the first. */
> - req_ctx->first_request = 0;
> -
> - /* HMAC key */
> - if (ctx->keylen)
> - to_talitos_ptr(&desc->ptr[2], ctx->dma_key, ctx->keylen,
> - is_sec1);
> -
> - sg_count = edesc->src_nents ?: 1;
> - if (is_sec1 && sg_count > 1)
> - sg_copy_to_buffer(edesc->src, sg_count, edesc->buf, length);
> - else if (length)
> - sg_count = dma_map_sg(dev, edesc->src, sg_count, DMA_TO_DEVICE);
> -
> - /*
> - * data in
> - */
> - sg_count = talitos_sg_map(dev, edesc->src, length, edesc, &desc->ptr[3],
> - sg_count, 0, 0);
> - if (sg_count > 1)
> - sync_needed = true;
> -
> - /* fifth DWORD empty */
> -
> - /* hash/HMAC out -or- hash context out */
> - if (edesc->last && req_ctx->last_request)
> - map_single_talitos_ptr(dev, &desc->ptr[5],
> - crypto_ahash_digestsize(tfm),
> - req_ctx->hw_context, DMA_FROM_DEVICE);
> - else
> - map_single_talitos_ptr_nosync(dev, &desc->ptr[5],
> - req_ctx->hw_context_size,
> - req_ctx->hw_context,
> - DMA_FROM_DEVICE);
> -
> - /* last DWORD empty */
> -
> - if (is_sec1 && from_talitos_ptr_len(&desc->ptr[3], true) == 0)
> - talitos_handle_buggy_hash(ctx, edesc, &desc->ptr[3]);
> -
> - if (sync_needed)
> - dma_sync_single_for_device(dev, edesc->dma_link_tbl,
> - edesc->dma_len, DMA_BIDIRECTIONAL);
> -}
> -
> -static struct talitos_edesc *ahash_edesc_alloc(struct ahash_request *areq,
> - struct scatterlist *src,
> - unsigned int nbytes)
> -{
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> -
> - return talitos_edesc_alloc(ctx->dev, src, NULL, NULL, 0,
> - nbytes, 0, 0, 0, areq->base.flags, false);
> -}
> -
> -static struct talitos_edesc *
> -ahash_process_req_prepare(struct ahash_request *areq, unsigned int nbytes,
> - unsigned int blocksize, bool is_sec1)
> -{
> - struct talitos_ctx *ctx = crypto_ahash_ctx(crypto_ahash_reqtfm(areq));
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct talitos_edesc *first = NULL, *prev_edesc = NULL, *edesc;
> - size_t desc_max = is_sec1 ? TALITOS1_MAX_DATA_LEN :
> - TALITOS2_MAX_DATA_LEN;
> - struct scatterlist tmp[2];
> - size_t to_hash_this_desc;
> - struct scatterlist *src;
> - size_t offset = 0;
> -
> - do {
> - src = scatterwalk_ffwd(tmp, areq->src, offset);
> -
> - to_hash_this_desc =
> - min(nbytes, ALIGN_DOWN(desc_max, blocksize));
> -
> - /* Allocate extended descriptor */
> - edesc = ahash_edesc_alloc(areq, src, to_hash_this_desc);
> - if (IS_ERR(edesc)) {
> - if (first)
> - free_edesc_list_from(areq, first);
> - return edesc;
> - }
> -
> - edesc->src = scatterwalk_ffwd(edesc->bufsl, areq->src, offset);
> - edesc->desc.hdr = ctx->desc_hdr_template;
> - edesc->first = offset == 0;
> - edesc->last = nbytes - to_hash_this_desc == 0;
> -
> - /* On last one, request SEC to pad; otherwise continue */
> - if (req_ctx->last_request && edesc->last)
> - edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_PAD;
> - else
> - edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_CONT;
> -
> - /* request SEC to INIT hash. */
> - if (req_ctx->first_request && edesc->first && !req_ctx->swinit)
> - edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_INIT;
> -
> - /*
> - * When the tfm context has a keylen, it's an HMAC.
> - * A first or last (ie. not middle) descriptor must request HMAC.
> - */
> - if (ctx->keylen && ((req_ctx->first_request && edesc->first) ||
> - (req_ctx->last_request && edesc->last)))
> - edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_HMAC;
> -
> - /* clear the DN bit */
> - if (is_sec1 && !edesc->last)
> - edesc->desc.hdr &= ~DESC_HDR_DONE_NOTIFY;
> -
> - common_nonsnoop_hash(edesc, areq, to_hash_this_desc);
> -
> - offset += to_hash_this_desc;
> - nbytes -= to_hash_this_desc;
> -
> - if (!prev_edesc)
> - first = edesc;
> - else
> - prev_edesc->next_desc = edesc;
> - prev_edesc = edesc;
> - } while (nbytes);
> -
> - return first;
> -}
> -
> -static int ahash_process_req(struct ahash_request *areq, unsigned int nbytes)
> -{
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct talitos_edesc *edesc;
> - unsigned int blocksize =
> - crypto_tfm_alg_blocksize(crypto_ahash_tfm(tfm));
> - bool is_sec1 = has_ftr_sec1(dev_get_drvdata(ctx->dev));
> - unsigned int nbytes_to_hash;
> - unsigned int to_hash_later;
> - struct device *dev = ctx->dev;
> - int ret;
> -
> - nbytes_to_hash = ALIGN_DOWN(nbytes, blocksize);
> - to_hash_later = nbytes - nbytes_to_hash;
> -
> - if (req_ctx->last_request) {
> - nbytes_to_hash = nbytes;
> - to_hash_later = 0;
> - }
> -
> - req_ctx->to_hash_later = to_hash_later;
> -
> - edesc = ahash_process_req_prepare(areq, nbytes_to_hash, blocksize,
> - is_sec1);
> - if (IS_ERR(edesc))
> - return PTR_ERR(edesc);
> -
> - ret = talitos_submit(dev, ctx->ch, &edesc->desc, ahash_done, areq);
> - if (ret != -EINPROGRESS)
> - free_edesc_list_from(areq, edesc);
> -
> - return ret;
> -}
> -
> -static int ahash_init(struct ahash_request *areq)
> -{
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> - struct device *dev = ctx->dev;
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - unsigned int size;
> - dma_addr_t dma;
> -
> - /* Initialize the context */
> - req_ctx->first_request = 1;
> - req_ctx->swinit = 0; /* assume h/w init of context */
> - size = (crypto_ahash_digestsize(tfm) <= SHA256_DIGEST_SIZE)
> - ? TALITOS_MDEU_CONTEXT_SIZE_MD5_SHA1_SHA256
> - : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512;
> - req_ctx->hw_context_size = size;
> - req_ctx->last_request = 0;
> -
> - dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size,
> - DMA_TO_DEVICE);
> - dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_TO_DEVICE);
> -
> - return 0;
> -}
> -
> -/*
> - * on h/w without explicit sha224 support, we initialize h/w context
> - * manually with sha224 constants, and tell it to run sha256.
> - */
> -static int ahash_init_sha224_swinit(struct ahash_request *areq)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> -
> - req_ctx->hw_context[0] = SHA224_H0;
> - req_ctx->hw_context[1] = SHA224_H1;
> - req_ctx->hw_context[2] = SHA224_H2;
> - req_ctx->hw_context[3] = SHA224_H3;
> - req_ctx->hw_context[4] = SHA224_H4;
> - req_ctx->hw_context[5] = SHA224_H5;
> - req_ctx->hw_context[6] = SHA224_H6;
> - req_ctx->hw_context[7] = SHA224_H7;
> -
> - /* init 64-bit count */
> - req_ctx->hw_context[8] = 0;
> - req_ctx->hw_context[9] = 0;
> -
> - ahash_init(areq);
> - req_ctx->swinit = 1;/* prevent h/w initting context with sha256 values*/
> -
> - return 0;
> -}
> -
> -static int ahash_update(struct ahash_request *areq)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> -
> - req_ctx->last_request = 0;
> -
> - return ahash_process_req(areq, areq->nbytes);
> -}
> -
> -static int ahash_final(struct ahash_request *areq)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> -
> - req_ctx->last_request = 1;
> -
> - return ahash_process_req(areq, 0);
> -}
> -
> -static int ahash_finup(struct ahash_request *areq)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> -
> - req_ctx->last_request = 1;
> -
> - return ahash_process_req(areq, areq->nbytes);
> -}
> -
> -static int ahash_digest(struct ahash_request *areq)
> -{
> - ahash_init(areq);
> - return ahash_finup(areq);
> -}
> -
> -static int ahash_digest_sha224_swinit(struct ahash_request *areq)
> -{
> - ahash_init_sha224_swinit(areq);
> - return ahash_finup(areq);
> -}
> -
> -static int ahash_export(struct ahash_request *areq, void *out)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct talitos_export_state *export = out;
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> - struct device *dev = ctx->dev;
> - dma_addr_t dma;
> -
> - dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size,
> - DMA_FROM_DEVICE);
> - dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_FROM_DEVICE);
> -
> - memcpy(export->hw_context, req_ctx->hw_context,
> - req_ctx->hw_context_size);
> - export->swinit = req_ctx->swinit;
> - export->first_request = req_ctx->first_request;
> - export->last_request = req_ctx->last_request;
> - export->to_hash_later = req_ctx->to_hash_later;
> -
> - return 0;
> -}
> -
> -static int ahash_import(struct ahash_request *areq, const void *in)
> -{
> - struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq);
> - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq);
> - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm);
> - struct device *dev = ctx->dev;
> - const struct talitos_export_state *export = in;
> - unsigned int size;
> - dma_addr_t dma;
> -
> - memset(req_ctx, 0, sizeof(*req_ctx));
> - size = (crypto_ahash_digestsize(tfm) <= SHA256_DIGEST_SIZE)
> - ? TALITOS_MDEU_CONTEXT_SIZE_MD5_SHA1_SHA256
> - : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512;
> - req_ctx->hw_context_size = size;
> - memcpy(req_ctx->hw_context, export->hw_context, size);
> - req_ctx->swinit = export->swinit;
> - req_ctx->first_request = export->first_request;
> - req_ctx->last_request = export->last_request;
> - req_ctx->to_hash_later = export->to_hash_later;
> -
> - dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size,
> - DMA_TO_DEVICE);
> - dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_TO_DEVICE);
> -
> - return 0;
> -}
> -
> -static int keyhash(struct crypto_ahash *tfm, const u8 *key, unsigned int keylen,
> - u8 *hash)
> -{
> - struct talitos_ctx *ctx = crypto_tfm_ctx(crypto_ahash_tfm(tfm));
> -
> - struct scatterlist sg[1];
> - struct ahash_request *req;
> - struct crypto_wait wait;
> - int ret;
> -
> - crypto_init_wait(&wait);
> -
> - req = ahash_request_alloc(tfm, GFP_KERNEL);
> - if (!req)
> - return -ENOMEM;
> -
> - /* Keep tfm keylen == 0 during hash of the long key */
> - ctx->keylen = 0;
> - ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG,
> - crypto_req_done, &wait);
> -
> - sg_init_one(&sg[0], key, keylen);
> -
> - ahash_request_set_crypt(req, sg, hash, keylen);
> - ret = crypto_wait_req(crypto_ahash_digest(req), &wait);
> -
> - ahash_request_free(req);
> -
> - return ret;
> -}
> -
> -static int ahash_setkey(struct crypto_ahash *tfm, const u8 *key,
> - unsigned int keylen)
> -{
> - struct talitos_ctx *ctx = crypto_tfm_ctx(crypto_ahash_tfm(tfm));
> - struct device *dev = ctx->dev;
> - unsigned int blocksize =
> - crypto_tfm_alg_blocksize(crypto_ahash_tfm(tfm));
> - unsigned int digestsize = crypto_ahash_digestsize(tfm);
> - unsigned int keysize = keylen;
> - u8 hash[SHA512_DIGEST_SIZE];
> - int ret;
> -
> - if (keylen <= blocksize)
> - memcpy(ctx->key, key, keysize);
> - else {
> - /* Must get the hash of the long key */
> - ret = keyhash(tfm, key, keylen, hash);
> -
> - if (ret)
> - return -EINVAL;
> -
> - keysize = digestsize;
> - memcpy(ctx->key, hash, digestsize);
> - }
> -
> - if (ctx->keylen)
> - dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE);
> -
> - ctx->keylen = keysize;
> - ctx->dma_key = dma_map_single(dev, ctx->key, keysize, DMA_TO_DEVICE);
> -
> - return 0;
> -}
> -
> static struct talitos_alg_template driver_algs[] = {
> /* AEAD algorithms. These use a single-pass ipsec_esp descriptor */
> { .type = CRYPTO_ALG_TYPE_AEAD,
> @@ -2737,235 +2222,6 @@ static struct talitos_alg_template driver_algs[] = {
> DESC_HDR_MODE0_DEU_CBC |
> DESC_HDR_MODE0_DEU_3DES,
> },
> - /* AHASH algorithms. */
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = MD5_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "md5",
> - .cra_driver_name = "md5-talitos",
> - .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_MD5,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA1_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha1",
> - .cra_driver_name = "sha1-talitos",
> - .cra_blocksize = SHA1_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA1,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA224_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha224",
> - .cra_driver_name = "sha224-talitos",
> - .cra_blocksize = SHA224_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA224,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA256_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha256",
> - .cra_driver_name = "sha256-talitos",
> - .cra_blocksize = SHA256_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA256,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA384_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha384",
> - .cra_driver_name = "sha384-talitos",
> - .cra_blocksize = SHA384_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA384,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA512_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "sha512",
> - .cra_driver_name = "sha512-talitos",
> - .cra_blocksize = SHA512_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA512,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = MD5_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(md5)",
> - .cra_driver_name = "hmac-md5-talitos",
> - .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_MD5,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA1_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha1)",
> - .cra_driver_name = "hmac-sha1-talitos",
> - .cra_blocksize = SHA1_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA1,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA224_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha224)",
> - .cra_driver_name = "hmac-sha224-talitos",
> - .cra_blocksize = SHA224_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA224,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA256_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha256)",
> - .cra_driver_name = "hmac-sha256-talitos",
> - .cra_blocksize = SHA256_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA256,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA384_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha384)",
> - .cra_driver_name = "hmac-sha384-talitos",
> - .cra_blocksize = SHA384_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA384,
> - },
> - { .type = CRYPTO_ALG_TYPE_AHASH,
> - .alg.hash = {
> - .halg.digestsize = SHA512_DIGEST_SIZE,
> - .halg.statesize = sizeof(struct talitos_export_state),
> - .halg.base = {
> - .cra_name = "hmac(sha512)",
> - .cra_driver_name = "hmac-sha512-talitos",
> - .cra_blocksize = SHA512_BLOCK_SIZE,
> - .cra_reqsize = sizeof(struct talitos_ahash_req_ctx),
> - .cra_flags = CRYPTO_ALG_ASYNC |
> - CRYPTO_ALG_ALLOCATES_MEMORY |
> - CRYPTO_AHASH_ALG_BLOCK_ONLY |
> - CRYPTO_AHASH_ALG_FINAL_NONZERO,
> - }
> - },
> - .desc_hdr_template = DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUB |
> - DESC_HDR_MODE0_MDEUB_SHA512,
> - }
> };
>
> int talitos_init_common(struct talitos_ctx *ctx,
> @@ -3014,22 +2270,6 @@ static int talitos_cra_init_skcipher(struct crypto_skcipher *tfm)
> return talitos_init_common(ctx, talitos_alg);
> }
>
> -static int talitos_cra_init_ahash(struct crypto_tfm *tfm)
> -{
> - struct crypto_alg *alg = tfm->__crt_alg;
> - struct talitos_crypto_alg *talitos_alg;
> - struct talitos_ctx *ctx = crypto_tfm_ctx(tfm);
> -
> - talitos_alg = container_of(__crypto_ahash_alg(alg),
> - struct talitos_crypto_alg,
> - algt.alg.hash);
> -
> - ctx->keylen = 0;
> - sizeof(struct talitos_ahash_req_ctx));
> -
> - return talitos_init_common(ctx, talitos_alg);
> -}
> -
> void talitos_cra_exit(struct crypto_tfm *tfm)
> {
> struct talitos_ctx *ctx = crypto_tfm_ctx(tfm);
> @@ -3128,6 +2368,12 @@ int talitos_register_common(struct device *dev,
> t_alg->algt = *template;
>
> switch (t_alg->algt.type) {
> + case CRYPTO_ALG_TYPE_AHASH:
> + alg = &t_alg->algt.alg.hash.halg.base;
> + talitos_alg_set_common(priv, alg, t_alg->algt.priority,
> + t_alg->algt.type);
> + ret = crypto_register_ahash(&t_alg->algt.alg.hash);
> + break;
> default:
> dev_err(dev, "unknown algorithm type %d\n", t_alg->algt.type);
> devm_kfree(dev, t_alg);
> @@ -3193,37 +2439,6 @@ static struct talitos_crypto_alg *talitos_alg_alloc(struct device *dev,
> return ERR_PTR(-ENOTSUPP);
> }
> break;
> - case CRYPTO_ALG_TYPE_AHASH:
> - alg = &t_alg->algt.alg.hash.halg.base;
> - alg->cra_init = talitos_cra_init_ahash;
> - alg->cra_exit = talitos_cra_exit;
> - t_alg->algt.alg.hash.init = ahash_init;
> - t_alg->algt.alg.hash.update = ahash_update;
> - t_alg->algt.alg.hash.final = ahash_final;
> - t_alg->algt.alg.hash.finup = ahash_finup;
> - t_alg->algt.alg.hash.digest = ahash_digest;
> - if (!strncmp(alg->cra_name, "hmac", 4))
> - t_alg->algt.alg.hash.setkey = ahash_setkey;
> - t_alg->algt.alg.hash.import = ahash_import;
> - t_alg->algt.alg.hash.export = ahash_export;
> -
> - if (!(priv->features & TALITOS_FTR_HMAC_OK) &&
> - !strncmp(alg->cra_name, "hmac", 4)) {
> - devm_kfree(dev, t_alg);
> - return ERR_PTR(-ENOTSUPP);
> - }
> - if (!(priv->features & TALITOS_FTR_SHA224_HWINIT) &&
> - (!strcmp(alg->cra_name, "sha224") ||
> - !strcmp(alg->cra_name, "hmac(sha224)"))) {
> - t_alg->algt.alg.hash.init = ahash_init_sha224_swinit;
> - t_alg->algt.alg.hash.digest =
> - ahash_digest_sha224_swinit;
> - t_alg->algt.desc_hdr_template =
> - DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU |
> - DESC_HDR_SEL0_MDEUA |
> - DESC_HDR_MODE0_MDEU_SHA256;
> - }
> - break;
> default:
> dev_err(dev, "unknown algorithm type %d\n", t_alg->algt.type);
> devm_kfree(dev, t_alg);
> @@ -3452,6 +2667,10 @@ static int talitos_probe(struct platform_device *ofdev)
> dev_info(dev, "hwrng\n");
> }
>
> + err = talitos_register_hash(dev);
> + if (err)
> + goto err_out;
> +
> /* register crypto algorithms the device supports */
> for (i = 0; i < ARRAY_SIZE(driver_algs); i++) {
> if (talitos_hw_supports(dev,
> @@ -3479,12 +2698,6 @@ static int talitos_probe(struct platform_device *ofdev)
> &t_alg->algt.alg.aead);
> alg = &t_alg->algt.alg.aead.base;
> break;
> -
> - case CRYPTO_ALG_TYPE_AHASH:
> - err = crypto_register_ahash(
> - &t_alg->algt.alg.hash);
> - alg = &t_alg->algt.alg.hash.halg.base;
> - break;
> }
> if (err) {
> dev_err(dev, "%s alg registration failed\n",
> diff --git a/drivers/crypto/talitos/talitos.h b/drivers/crypto/talitos/talitos.h
> index afed9947f4c0..e703c18cb81f 100644
> --- a/drivers/crypto/talitos/talitos.h
> +++ b/drivers/crypto/talitos/talitos.h
> @@ -530,3 +530,7 @@ int talitos_register_common(struct device *dev,
>
> int talitos_register_rng(struct device *dev);
> void talitos_unregister_rng(struct device *dev);
> +
> +/* Hash */
> +
> +int talitos_register_hash(struct device *dev);
>
^ permalink raw reply
* Re: [PATCH 00/29] crypto: talitos - Driver cleanup
From: Christophe Leroy (CS GROUP) @ 2026-06-01 10:27 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <DIXLMBNKMF1N.2FVTXFA6MP1NF@bootlin.com>
Le 01/06/2026 à 11:17, Paul Louvel a écrit :
>>> The Freescale Integrated Security Engine (SEC) aka "Talitos" driver
>>> implementation is a monolithic ~3800-line file that mixes SEC1 and SEC2
>>> hardware variants with hash, skcipher, aead and hwrng algorithm.
>>>
>>> This series reorganises the driver to improve readability and
>>> maintainability.
>>
>> Did you analyse the cost of this series ? bloat-o-meter gives the
>> following result, allthough I'm a bit surprised there are only added
>> items, no removed items:
>
> When you say 'cost', do you mean cost in terms of code size ? performance cost ?
> or both ?
> Regarding code size, I trusted the differences shown in the cover letter by git:
>
>> 13 files changed, 3810 insertions(+), 3707 deletions(-)
No I mean cost in terms of text size, and cost in terms of execution flow.
>
> There is 103 insertions more than deletions. This is due to the fact that
> splitting up SEC1/SEC2 code requires additional function and structures.
> I find it acceptable given the readability improvement.
>
> As for performance, I ran ftrace with the function graph tracer, hashing a 100kb
> file.
The thing is you are doing a test in an ideal world. But the cost can be
a lot higher on a busy board where you have to consider task switches,
cache loading, etc ....
The best is to look at the object text. The more flat it is the most
efficient it is. No branching, no function calls in fast pathes.
When you take a function like this:
00001b14 <to_talitos_ptr>:
1b14: 2c 06 00 00 cmpwi r6,0
1b18: 90 83 00 04 stw r4,4(r3)
1b1c: 41 82 00 0c beq 1b28 <to_talitos_ptr+0x14>
1b20: b0 a3 00 02 sth r5,2(r3)
1b24: 4e 80 00 20 blr
1b28: b0 a3 00 00 sth r5,0(r3)
1b2c: 98 c3 00 03 stb r6,3(r3)
1b30: 4e 80 00 20 blr
You see that is case is_sec1 is 1 it is just two stores, when is_sec1 is
0 it is three stores. So inlining those two/three stores at build is
definitely more efficient than having such a function.
And it is even worth with that one:
00001b84 <to_talitos_ptr_ext_set>:
1b84: 2c 05 00 00 cmpwi r5,0
1b88: 4c 82 00 20 bnelr
1b8c: 98 83 00 02 stb r4,2(r3)
1b90: 4e 80 00 20 blr
If is_sec1 is not 0 the function does nothing, so all the cost of a
function call to do nothing at the end.
>
> It looks like there is a slight performance penalty with ahash_finup().
> Otherwise, there is a slight performance improvement for the other measurements.
> I do not know if there is a better way to measure the performance impact of this
> series. If you know, do not hesitate to share it to me.
With experience, the best way to evaluate performance impact is ... look
at generated object text.
>
>
> Best regards,
> Paul.
>
^ permalink raw reply
* Re: [PATCH 05/29] crypto: talitos - Prepare crypto implementation file splitting
From: Christophe Leroy (CS GROUP) @ 2026-06-01 10:16 UTC (permalink / raw)
To: Paul Louvel, Herbert Xu, David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <DIXL0OBC6IP9.3UNVLNBMZ7EZ9@bootlin.com>
Le 01/06/2026 à 10:49, Paul Louvel a écrit :
> Hi Christophe,
>
>> Le 28/05/2026 à 11:08, Paul Louvel a écrit :
>>> Remove the static qualifier on multiple function that will be called
>>> inside each crypto implementation file.
>>> Add them to the main driver header file.
>>
>> I didn't have time to look at the generated text yet but I'm a bit
>> sceptic with this change, or more than the change itself, about its
>> purpose. And even more when I see patches 24 and 25.
>
> About patch 24 and 25: one of the main purpose of this series is to get rid of
> the is_sec1 scattered around the core code of the driver.
I think we can find more efficient ways to do it, for instance using
static branches (jump label).
>
>> Most functions here are small helpers. To be shared between several C
>> files they deserve becoming static inlines in talitos.h, not global
>> functions.
>
> I did not look at the generated text either for this change but I bet the
> compiler is inlining these calls. Adding them as static inline is more explicit.
Today the compiler is inlining these calls for sure, and it also gets
rid of the is_sec1 at all unless you unlikely build with both
CRYPTO_DEV_TALITOS1 and CRYPTO_DEV_TALITOS2.
But once you call them from a different C file, then it can't be inlined
anymore.
>
> I understand that there is a performance penalty, since there will be no
> inlining, and a memory dereferencing for a call through function pointer.
Not only a memory dereferencing, but all the preparation for calling a
subfunction: creating a stack frame, saving the link register, saving
volatile registers, loading the address to call into a register, saving
it into CTR register, then doing a branch to CTR. And of course the
impact on the call flow, instruction cache loading, pipeline, etc ...
So this is definitely a huge cost compared to the cost of the one or two
loads done by the helper itself.
>
>> Indeed, most of the time is_sec1 is known at build time because in most
>> cases has_ftr_sec1() will constant fold into true or false during build.
>> This is because it is very unlikely that someone build a kernel to run
>> on both MPC 82xx and MPC 83xx at the same time. Therefore it is really
>> unlikely that this in built with both CRYPTO_DEV_TALITOS1 and
>> CRYPTO_DEV_TALITOS2 at the same time.
>
> As for patch 24, 25 and onwards, the same space optimization apply here. If the
> kernel is built with CRYPTO_DEV_TALITOS1, there will be no SEC2-related function
> in the generated text, and vice versa.
I'm not thinking about space optimisation but processing optimisation.
In the current implementation you have a flat branchless execution flow.
Here you are introducing branching, and worse: indirect branching. As
mentioned by David, it has a serious cost, more than what (young) people
may think. When only two choices are needed an if/else remains the most
efficient.
Christophe
>
>>
>> I can understand for a function like talitos_submit() but not for
>> functions like to_talitos_ptr() or to_talitos_ptr_ext_set() whose
>> purpose is really to get inlined into the caller.
>>
>> Christophe
>>
>
> These changes are needed to split the driver into multiple files.
>
> Best regards,
> Paul.
>
^ permalink raw reply
* [PATCH v3 0/2] hwrng: starfive: updates for jh7110-trng DT binding and driver fixes
From: lianfeng.ouyang @ 2026-06-01 9:37 UTC (permalink / raw)
To: Olivia Mackall, Herbert Xu, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Philipp Zabel
Cc: linux-crypto, devicetree, linux-kernel, Lianfeng Ouyang
From: Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
- the obsolete JH8100 compatible string is dropped, support for the new
JHB100 SoC is added, and the schema is simplified to an enum.
- The driver is updated to handle the JHB100's specific hardware
requirement where clocks must be disabled beforeasserting reset during
teardown, to avoid glitches. This is implemented via a per-compatible
data flag (SEQ_CLK_FIRSTfor JHB100, preserving SEQ_RST_FIRSTfor JH7110).
- Several Runtime Power Management (RPM) fixes are included to ensure
proper autosuspend behavior, balanced runtime usage counting, and
correct cleanup ordering on probe failure or device removal.
Lianfeng Ouyang (2):
dt-bindings: rng: starfive,jh7110-trng: add jhb100, drop jh8100
hwrng: starfive: rework clk/reset teardown order for JHB100 & fix RPM
.../bindings/rng/starfive,jh7110-trng.yaml | 10 +-
MAINTAINERS | 2 +-
drivers/char/hw_random/jh7110-trng.c | 191 ++++++++++++++----
3 files changed, 152 insertions(+), 51 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH v3 1/2] dt-bindings: rng: starfive,jh7110-trng: add jhb100, drop jh8100
From: lianfeng.ouyang @ 2026-06-01 9:37 UTC (permalink / raw)
To: Olivia Mackall, Herbert Xu, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Philipp Zabel
Cc: linux-crypto, devicetree, linux-kernel, Lianfeng Ouyang
In-Reply-To: <20260601093744.84210-1-lianfeng.ouyang@starfivetech.com>
From: Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
- Drop "starfive,jh8100-trng" since JH8100 SoC is no longer
supported
- Add "starfive,jhb100-trng" for the JHB100 SoC TRNG.
- Update maintainer to current owner
Signed-off-by: Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
---
.../devicetree/bindings/rng/starfive,jh7110-trng.yaml | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/Documentation/devicetree/bindings/rng/starfive,jh7110-trng.yaml b/Documentation/devicetree/bindings/rng/starfive,jh7110-trng.yaml
index 4639247e9e51..d21769b7d54e 100644
--- a/Documentation/devicetree/bindings/rng/starfive,jh7110-trng.yaml
+++ b/Documentation/devicetree/bindings/rng/starfive,jh7110-trng.yaml
@@ -7,15 +7,13 @@ $schema: http://devicetree.org/meta-schemas/core.yaml#
title: StarFive SoC TRNG Module
maintainers:
- - Jia Jie Ho <jiajie.ho@starfivetech.com>
+ - Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
properties:
compatible:
- oneOf:
- - items:
- - const: starfive,jh8100-trng
- - const: starfive,jh7110-trng
- - const: starfive,jh7110-trng
+ enum:
+ - starfive,jh7110-trng
+ - starfive,jhb100-trng
reg:
maxItems: 1
--
2.43.0
^ permalink raw reply related
* Re: [PATCH] crypto: exynos-rng - Remove exynos-rng driver
From: Krzysztof Kozlowski @ 2026-06-01 9:39 UTC (permalink / raw)
To: Eric Biggers, linux-crypto, Herbert Xu
Cc: linux-samsung-soc, Alim Akhtar, linux-kernel
In-Reply-To: <20260531175932.32171-1-ebiggers@kernel.org>
On 31/05/2026 19:59, Eric Biggers wrote:
> This driver has no purpose. It doesn't feed into the Linux RNG, nor
> does it implement the hwrng interface. It is accessible only via the
> "rng" algorithm type of AF_ALG, which isn't used in practice. Everyone
> uses either the Linux RNG, or rarely /dev/hwrng.
>
> Moreover, this is a PRNG whose only source of entropy is the 160-bit
> seed the user passes in. So this can be used only by a user who already
> has a source of cryptographically secure random numbers, such as
> /dev/random. Which they can, and do, just use in the first place.
I don't know nowadays about any real world usage of this driver, so fine
by me:
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Best regards,
Krzysztof
^ permalink raw reply
* [PATCH v3 2/2] hwrng: starfive: rework clk/reset teardown order for JHB100 & fix RPM
From: lianfeng.ouyang @ 2026-06-01 9:37 UTC (permalink / raw)
To: Olivia Mackall, Herbert Xu, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Philipp Zabel
Cc: linux-crypto, devicetree, linux-kernel, Lianfeng Ouyang
In-Reply-To: <20260601093744.84210-1-lianfeng.ouyang@starfivetech.com>
From: Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
JHB100: to avoid RDC glitch, assert reset only after clocks are gated.
Introduce a per-compatible seq_rst_clk flag (RST_FIRST/CLK_FIRST)
selected via of_device_get_match_data().
RPM fixes:
- Mark device RPM_ACTIVE in probe after clocks & reset deassert,
before pm_runtime_enable(), so usage count can eventually drop to 0
and autosuspend actually triggers.
- Balance pm_runtime_get/put in init/read/cleanup, and reject
register access when RPM resume fails.
- Move low-level disable/reset into a devm action
(starfive_trng_release()) so error-path unwind order is correct.
Signed-off-by: Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
---
MAINTAINERS | 2 +-
drivers/char/hw_random/jh7110-trng.c | 191 +++++++++++++++++++++------
2 files changed, 148 insertions(+), 45 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index d3a6b3f6b6a0..729b20ecc697 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -25280,7 +25280,7 @@ F: Documentation/devicetree/bindings/perf/starfive,jh8100-starlink-pmu.yaml
F: drivers/perf/starfive_starlink_pmu.c
STARFIVE TRNG DRIVER
-M: Jia Jie Ho <jiajie.ho@starfivetech.com>
+M: Lianfeng Ouyang <lianfeng.ouyang@starfivetech.com>
S: Supported
F: Documentation/devicetree/bindings/rng/starfive*
F: drivers/char/hw_random/jh7110-trng.c
diff --git a/drivers/char/hw_random/jh7110-trng.c b/drivers/char/hw_random/jh7110-trng.c
index 9776f4daa044..9c164a0011ae 100644
--- a/drivers/char/hw_random/jh7110-trng.c
+++ b/drivers/char/hw_random/jh7110-trng.c
@@ -92,20 +92,36 @@ enum mode {
PRNG_256BIT,
};
+/*
+ * For JHB100, assert reset after disabling clocks to avoid
+ * reset-domain crossing (RDC) induced glitches that can affect
+ * downstream IPs.
+ */
+enum seq_rst_clk {
+ SEQ_RST_FIRST,
+ SEQ_CLK_FIRST,
+};
+
+struct starfive_trng_data {
+ enum seq_rst_clk seq_rst_clk;
+};
+
struct starfive_trng {
- struct device *dev;
- void __iomem *base;
- struct clk *hclk;
- struct clk *ahb;
- struct reset_control *rst;
- struct hwrng rng;
- struct completion random_done;
- struct completion reseed_done;
- u32 mode;
- u32 mission;
- u32 reseed;
- /* protects against concurrent write to ctrl register */
- spinlock_t write_lock;
+ struct device *dev;
+ void __iomem *base;
+ int irq;
+ struct clk *hclk;
+ struct clk *ahb;
+ struct reset_control *rst;
+ struct hwrng rng;
+ struct completion random_done;
+ struct completion reseed_done;
+ const struct starfive_trng_data *data;
+ u32 mode;
+ u32 mission;
+ u32 reseed;
+ struct mutex lock; /* protect trng cmd seq */
+ spinlock_t write_lock; /* protects register access seq */
};
static u16 autoreq;
@@ -130,7 +146,7 @@ static inline int starfive_trng_wait_idle(struct starfive_trng *trng)
10, 100000);
}
-static inline void starfive_trng_irq_mask_clear(struct starfive_trng *trng)
+static inline void starfive_trng_irq_clear(struct starfive_trng *trng)
{
/* clear register: ISTAT */
u32 data = readl(trng->base + STARFIVE_ISTAT);
@@ -138,6 +154,31 @@ static inline void starfive_trng_irq_mask_clear(struct starfive_trng *trng)
writel(data, trng->base + STARFIVE_ISTAT);
}
+static void starfive_trng_release(void *data)
+{
+ struct starfive_trng *trng = data;
+
+ if (!pm_runtime_status_suspended(trng->dev)) {
+ writel(0, trng->base + STARFIVE_IE);
+ starfive_trng_irq_clear(trng);
+
+ if (trng->irq >= 0)
+ synchronize_irq(trng->irq);
+
+ if (trng->data->seq_rst_clk == SEQ_RST_FIRST)
+ reset_control_assert(trng->rst);
+
+ clk_disable_unprepare(trng->ahb);
+ clk_disable_unprepare(trng->hclk);
+
+ if (trng->data->seq_rst_clk == SEQ_CLK_FIRST)
+ reset_control_assert(trng->rst);
+ }
+
+ pm_runtime_dont_use_autosuspend(trng->dev);
+ pm_runtime_disable(trng->dev);
+}
+
static int starfive_trng_cmd(struct starfive_trng *trng, u32 cmd, bool wait)
{
int wait_time = 1000;
@@ -174,13 +215,22 @@ static int starfive_trng_init(struct hwrng *rng)
{
struct starfive_trng *trng = to_trng(rng);
u32 mode, intr = 0;
+ int ret;
+
+ ret = pm_runtime_resume_and_get(trng->dev);
+ if (ret < 0) {
+ dev_warn(trng->dev, "Failed to wake device for init: %d\n", ret);
+ return ret;
+ }
+
+ mutex_lock(&trng->lock);
/* setup Auto Request/Age register */
writel(autoage, trng->base + STARFIVE_AUTO_AGE);
writel(autoreq, trng->base + STARFIVE_AUTO_RQSTS);
/* clear register: ISTAT */
- starfive_trng_irq_mask_clear(trng);
+ starfive_trng_irq_clear(trng);
intr |= STARFIVE_IE_ALL;
writel(intr, trng->base + STARFIVE_IE);
@@ -201,7 +251,13 @@ static int starfive_trng_init(struct hwrng *rng)
writel(mode, trng->base + STARFIVE_MODE);
- return starfive_trng_cmd(trng, STARFIVE_CTRL_EXEC_RANDRESEED, 1);
+ ret = starfive_trng_cmd(trng, STARFIVE_CTRL_EXEC_RANDRESEED, 1);
+
+ mutex_unlock(&trng->lock);
+
+ pm_runtime_put_autosuspend(trng->dev);
+
+ return ret;
}
static irqreturn_t starfive_trng_irq(int irq, void *priv)
@@ -209,16 +265,17 @@ static irqreturn_t starfive_trng_irq(int irq, void *priv)
u32 status;
struct starfive_trng *trng = (struct starfive_trng *)priv;
+ if (!pm_runtime_get_if_active(trng->dev)) {
+ dev_err_ratelimited(trng->dev, "pm is inactive in irq\n");
+ return IRQ_NONE;
+ }
+
status = readl(trng->base + STARFIVE_ISTAT);
- if (status & STARFIVE_ISTAT_RAND_RDY) {
+ if (status & STARFIVE_ISTAT_RAND_RDY)
writel(STARFIVE_ISTAT_RAND_RDY, trng->base + STARFIVE_ISTAT);
- complete(&trng->random_done);
- }
- if (status & STARFIVE_ISTAT_SEED_DONE) {
+ if (status & STARFIVE_ISTAT_SEED_DONE)
writel(STARFIVE_ISTAT_SEED_DONE, trng->base + STARFIVE_ISTAT);
- complete(&trng->reseed_done);
- }
if (status & STARFIVE_ISTAT_LFSR_LOCKUP) {
writel(STARFIVE_ISTAT_LFSR_LOCKUP, trng->base + STARFIVE_ISTAT);
@@ -228,18 +285,37 @@ static irqreturn_t starfive_trng_irq(int irq, void *priv)
spin_unlock(&trng->write_lock);
}
+ if (status & STARFIVE_ISTAT_RAND_RDY)
+ complete(&trng->random_done);
+
+ if (status & STARFIVE_ISTAT_SEED_DONE)
+ complete(&trng->reseed_done);
+
+ pm_runtime_put_noidle(trng->dev);
+
return IRQ_HANDLED;
}
static void starfive_trng_cleanup(struct hwrng *rng)
{
struct starfive_trng *trng = to_trng(rng);
+ int ret;
+
+ ret = pm_runtime_resume_and_get(trng->dev);
+ if (ret < 0) {
+ dev_warn(trng->dev, "Failed to wake device for cleanup: %d\n", ret);
+ return;
+ }
+
+ writel(0, trng->base + STARFIVE_IE);
+ starfive_trng_irq_clear(trng);
+
+ if (trng->irq >= 0)
+ synchronize_irq(trng->irq);
writel(0, trng->base + STARFIVE_CTRL);
- reset_control_assert(trng->rst);
- clk_disable_unprepare(trng->hclk);
- clk_disable_unprepare(trng->ahb);
+ pm_runtime_put_sync(trng->dev);
}
static int starfive_trng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
@@ -247,7 +323,13 @@ static int starfive_trng_read(struct hwrng *rng, void *buf, size_t max, bool wai
struct starfive_trng *trng = to_trng(rng);
int ret;
- pm_runtime_get_sync(trng->dev);
+ ret = pm_runtime_resume_and_get(trng->dev);
+ if (ret < 0) {
+ dev_warn(trng->dev, "Failed to wake device for read: %d\n", ret);
+ return ret;
+ }
+
+ mutex_lock(&trng->lock);
if (trng->mode == PRNG_256BIT)
max = min_t(size_t, max, (STARFIVE_RAND_LEN * 8));
@@ -257,24 +339,28 @@ static int starfive_trng_read(struct hwrng *rng, void *buf, size_t max, bool wai
if (wait) {
ret = starfive_trng_wait_idle(trng);
if (ret)
- return -ETIMEDOUT;
+ goto end;
}
ret = starfive_trng_cmd(trng, STARFIVE_CTRL_GENE_RANDNUM, wait);
if (ret)
- return ret;
+ goto end;
memcpy_fromio(buf, trng->base + STARFIVE_RAND0, max);
- pm_runtime_put_sync_autosuspend(trng->dev);
+ ret = max;
+
+end:
+ mutex_unlock(&trng->lock);
- return max;
+ pm_runtime_put_autosuspend(trng->dev);
+
+ return ret;
}
static int starfive_trng_probe(struct platform_device *pdev)
{
int ret;
- int irq;
struct starfive_trng *trng;
trng = devm_kzalloc(&pdev->dev, sizeof(*trng), GFP_KERNEL);
@@ -282,22 +368,32 @@ static int starfive_trng_probe(struct platform_device *pdev)
return -ENOMEM;
platform_set_drvdata(pdev, trng);
+
trng->dev = &pdev->dev;
+ trng->data = of_device_get_match_data(&pdev->dev);
+ if (!trng->data)
+ return -EINVAL;
+
+ if (trng->data->seq_rst_clk != SEQ_RST_FIRST && trng->data->seq_rst_clk != SEQ_CLK_FIRST) {
+ dev_err(&pdev->dev, "Unknown seq_rst_clk value\n");
+ return -EINVAL;
+ }
trng->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(trng->base))
return dev_err_probe(&pdev->dev, PTR_ERR(trng->base),
"Error remapping memory for platform device.\n");
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
+ trng->irq = platform_get_irq(pdev, 0);
+ if (trng->irq < 0)
+ return trng->irq;
init_completion(&trng->random_done);
init_completion(&trng->reseed_done);
+ mutex_init(&trng->lock);
spin_lock_init(&trng->write_lock);
- ret = devm_request_irq(&pdev->dev, irq, starfive_trng_irq, 0, pdev->name,
+ ret = devm_request_irq(&pdev->dev, trng->irq, starfive_trng_irq, 0, pdev->name,
(void *)trng);
if (ret)
return dev_err_probe(&pdev->dev, ret,
@@ -333,18 +429,16 @@ static int starfive_trng_probe(struct platform_device *pdev)
pm_runtime_use_autosuspend(&pdev->dev);
pm_runtime_set_autosuspend_delay(&pdev->dev, 100);
+ pm_runtime_set_active(&pdev->dev);
pm_runtime_enable(&pdev->dev);
- ret = devm_hwrng_register(&pdev->dev, &trng->rng);
- if (ret) {
- pm_runtime_disable(&pdev->dev);
-
- reset_control_assert(trng->rst);
- clk_disable_unprepare(trng->ahb);
- clk_disable_unprepare(trng->hclk);
+ ret = devm_add_action_or_reset(&pdev->dev, starfive_trng_release, trng);
+ if (ret)
+ return ret;
+ ret = devm_hwrng_register(&pdev->dev, &trng->rng);
+ if (ret)
return dev_err_probe(&pdev->dev, ret, "Failed to register hwrng\n");
- }
return 0;
}
@@ -376,8 +470,17 @@ static const struct dev_pm_ops starfive_trng_pm_ops = {
starfive_trng_resume, NULL)
};
+static const struct starfive_trng_data jh7110_data = {
+ .seq_rst_clk = SEQ_RST_FIRST,
+};
+
+static const struct starfive_trng_data jhb100_data = {
+ .seq_rst_clk = SEQ_CLK_FIRST,
+};
+
static const struct of_device_id trng_dt_ids[] __maybe_unused = {
- { .compatible = "starfive,jh7110-trng" },
+ { .compatible = "starfive,jh7110-trng", .data = &jh7110_data },
+ { .compatible = "starfive,jhb100-trng", .data = &jhb100_data },
{ }
};
MODULE_DEVICE_TABLE(of, trng_dt_ids);
--
2.43.0
^ permalink raw reply related
* Re: [PATCH] crypto: sun4i-ss - Remove insecure and unused rng_alg
From: Tianchu Chen @ 2026-06-01 9:19 UTC (permalink / raw)
To: ebiggers
Cc: clabbe.montjoie, herbert, jernej.skrabec, linux-arm-kernel,
linux-crypto, linux-kernel, linux-sunxi, samuel, stable, wens
From: Tianchu Chen <flynnnchen@tencent.com>
In-Reply-To: <20260529193648.18172-1-ebiggers@kernel.org>
References: <20260529193648.18172-1-ebiggers@kernel.org>
On Fri, May 29, 2026 at 12:36:48PM -0700, Eric Biggers wrote:
> Remove sun4i_ss_rng, as it is insecure and unused:
>
> - It has multiple vulnerabilities. sun4i_ss_prng_seed() is missing
> locking and has a buffer overflow.
Thanks for cleaning this up.
For the record, the sun4i_ss_prng_seed() buffer overflow you mention here
is the same issue we reported earlier with a targeted fix:
https://lore.kernel.org/linux-crypto/20260529194152.GA3628@quark/
It is an unauthenticated, unbounded memcpy() into the 24-byte ss->seed[]
buffer, reachable from any user via AF_ALG ALG_SET_KEY with no privileges
on affected Allwinner sun4i hardware.
Please note that this should be treated as a security fix. For the earlier
stable releases, keeping the rng_alg but adding a proper bounds check in
sun4i_ss_prng_seed() might still be a preferable option to consider.
Given the above, would you mind adding the following trailers to the commit
message? Besides crediting the discovery and report, they would also make
this security issue easier to track and reference across the stable trees:
Discovered by Atuin - Automated Vulnerability Discovery Engine
Reported-by: Tianchu Chen <flynnnchen@tencent.com>
Thanks,
Tianchu
^ permalink raw reply
* Re: [PATCH 00/29] crypto: talitos - Driver cleanup
From: Paul Louvel @ 2026-06-01 9:17 UTC (permalink / raw)
To: Christophe Leroy (CS GROUP), Paul Louvel, Herbert Xu,
David S. Miller
Cc: Thomas Petazzoni, Herve Codina, linux-crypto, linux-kernel
In-Reply-To: <1488f7b3-cda0-4267-827c-fae23b17c1e8@kernel.org>
>> The Freescale Integrated Security Engine (SEC) aka "Talitos" driver
>> implementation is a monolithic ~3800-line file that mixes SEC1 and SEC2
>> hardware variants with hash, skcipher, aead and hwrng algorithm.
>>
>> This series reorganises the driver to improve readability and
>> maintainability.
>
> Did you analyse the cost of this series ? bloat-o-meter gives the
> following result, allthough I'm a bit surprised there are only added
> items, no removed items:
When you say 'cost', do you mean cost in terms of code size ? performance cost ?
or both ?
Regarding code size, I trusted the differences shown in the cover letter by git:
> 13 files changed, 3810 insertions(+), 3707 deletions(-)
There is 103 insertions more than deletions. This is due to the fact that
splitting up SEC1/SEC2 code requires additional function and structures.
I find it acceptable given the readability improvement.
As for performance, I ran ftrace with the function graph tracer, hashing a 100kb
file.
Without the series applied:
be935f36ae14489758e28a83cfec418d3ad600b64628166f275c7ae6aac7b9af ./test_100k.bin
# tracer: function_graph
#
# CPU DURATION FUNCTION CALLS
# | | | | | | |
0) + 20.256 us | ahash_init();
0) | ahash_do_req_chain() {
0) | ahash_update() {
0) + 41.088 us | ahash_process_req();
0) + 54.272 us | }
0) + 61.536 us | }
------------------------------------------
0) sha256s-196 => <idle>-0
------------------------------------------
0) + 45.248 us | ahash_done();
------------------------------------------
0) <idle>-0 => sha256s-196
------------------------------------------
0) | ahash_do_req_chain() {
0) | ahash_update() {
0) + 39.552 us | ahash_process_req();
0) + 53.472 us | }
0) + 68.576 us | }
------------------------------------------
0) sha256s-196 => <idle>-0
------------------------------------------
0) + 39.680 us | ahash_done();
------------------------------------------
0) <idle>-0 => sha256s-196
------------------------------------------
0) | ahash_do_req_chain() {
0) | ahash_finup() {
0) | ahash_process_req() {
0) + 16.800 us | ahash_done();
0) + 96.192 us | }
0) ! 103.616 us | }
0) ! 121.344 us | }
With the series applied:
be935f36ae14489758e28a83cfec418d3ad600b64628166f275c7ae6aac7b9af ./test_100k.bin
# tracer: function_graph
#
# CPU DURATION FUNCTION CALLS
# | | | | | | |
0) + 20.576 us | ahash_init();
0) | ahash_do_req_chain() {
0) | ahash_update() {
0) + 32.896 us | ahash_process_req();
0) + 46.688 us | }
0) + 54.016 us | }
------------------------------------------
0) sha256s-196 => <idle>-0
------------------------------------------
0) | ahash_done() {
0) | ahash_update_done() {
0) 9.312 us | ahash_update_finish();
0) + 44.416 us | }
0) + 73.216 us | }
------------------------------------------
0) <idle>-0 => sha256s-196
------------------------------------------
0) | ahash_do_req_chain() {
0) | ahash_update() {
0) + 33.120 us | ahash_process_req();
0) + 46.912 us | }
0) + 61.664 us | }
------------------------------------------
0) sha256s-196 => <idle>-0
------------------------------------------
0) | ahash_done() {
0) | ahash_update_done() {
0) 8.928 us | ahash_update_finish();
0) + 42.720 us | }
0) + 69.440 us | }
------------------------------------------
0) <idle>-0 => sha256s-196
------------------------------------------
0) | ahash_do_req_chain() {
0) | ahash_finup() {
0) | ahash_process_req() {
0) | ahash_done() {
0) 5.696 us | ahash_finup_done();
0) + 29.760 us | }
0) ! 107.168 us | }
0) ! 113.696 us | }
0) ! 131.840 us | }
It looks like there is a slight performance penalty with ahash_finup().
Otherwise, there is a slight performance improvement for the other measurements.
I do not know if there is a better way to measure the performance impact of this
series. If you know, do not hesitate to share it to me.
Best regards,
Paul.
--
Paul Louvel, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply
* Re: [PATCH 10/12] crypto: atmel - update workqueue flags and add flush on exit
From: Marco Crivellari @ 2026-06-01 9:03 UTC (permalink / raw)
To: l.rubusch
Cc: alexandre.belloni, claudiu.beznea, davem, herbert,
linux-arm-kernel, linux-crypto, linux-kernel, nicolas.ferre,
thorsten.blum, Marco Crivellari
In-Reply-To: <20260512224349.64621-11-l.rubusch@gmail.com>
Hi,
> Update workqueue initialization to use WQ_MEM_RECLAIM instead of
> WQ_PERCPU
Not sure if you're working on this series right now, but this must keep
the WQ_PERCPU flag. WQ_PERCPU has been added to mark explicitly mark
workqueue that are per-CPU (it is the complement of WQ_UNBOUND).
Thanks!
^ permalink raw reply
* [PATCH v3 3/4] crypto: testmgr - exercise multi-data-unit path for skcipher
From: Leonid Ravich @ 2026-06-01 8:56 UTC (permalink / raw)
To: Herbert Xu
Cc: Alasdair Kergon, Ard Biesheuvel, Eric Biggers, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260601085644.13026-1-lravich@amazon.com>
Add a self-comparison test that runs whenever an skcipher algorithm
advertises CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT in cra_flags. The test
encrypts the same random plaintext two ways:
1. as one batched request with data_unit_size set, and
2. as N back-to-back single-data-unit requests with IVs derived from
the original IV by adding the data-unit index (treated as a
128-bit little-endian counter, matching the convention documented
in crypto_skcipher_set_data_unit_size()).
Both encrypts must produce byte-identical ciphertext, otherwise the
algorithm's multi-DU implementation is inconsistent with its single-DU
behaviour. Iterates over a fixed set of typical data unit sizes
(512, 1024, 2048, 4096) which cover the dm-crypt sector-size range.
The test is gated on ivsize == 16 (XTS, the only multi-DU consumer in
the kernel today) and on the algorithm advertising the capability,
so it costs nothing for the existing fleet of skcipher drivers.
Signed-off-by: Leonid Ravich <lravich@amazon.com>
---
crypto/testmgr.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 129 insertions(+)
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 4d86efae65b2..8ca92ee6b37c 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -3211,6 +3211,123 @@ static int test_skcipher(int enc, const struct cipher_test_suite *suite,
return 0;
}
+/*
+ * For algorithms that advertise CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT,
+ * verify that one request batching N data units produces the same
+ * ciphertext as N back-to-back single-data-unit requests with IVs
+ * derived from the original IV by adding the data-unit index (treated
+ * as a 128-bit little-endian counter).
+ *
+ * This is a self-comparison: it does not depend on test-vector
+ * authoritativeness, only on the algorithm being internally consistent
+ * between its single-DU and multi-DU paths.
+ */
+#define TEST_MDU_NR_UNITS 4
+static int test_skcipher_multi_du(struct crypto_skcipher *tfm,
+ unsigned int du_size)
+{
+ const char *driver = crypto_skcipher_driver_name(tfm);
+ const unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ const unsigned int total = du_size * TEST_MDU_NR_UNITS;
+ struct skcipher_request *req = NULL;
+ struct scatterlist sg_in, sg_out;
+ DECLARE_CRYPTO_WAIT(wait);
+ u8 iv_orig[16] = {0};
+ u8 iv_work[16];
+ u8 *plain = NULL, *batched = NULL, *unit = NULL;
+ unsigned int i;
+ int err;
+
+ if (ivsize != 16)
+ return 0;
+
+ plain = kmalloc(total, GFP_KERNEL);
+ batched = kmalloc(total, GFP_KERNEL);
+ unit = kmalloc(total, GFP_KERNEL);
+ req = skcipher_request_alloc(tfm, GFP_KERNEL);
+ if (!plain || !batched || !unit || !req) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ get_random_bytes(plain, total);
+ get_random_bytes(iv_orig, ivsize);
+
+ /* Pass 1: one batched encrypt with data_unit_size set. */
+ err = crypto_skcipher_set_data_unit_size(tfm, du_size);
+ if (err) {
+ pr_err("alg: skcipher: %s set_data_unit_size(%u) failed: %d\n",
+ driver, du_size, err);
+ goto out;
+ }
+ memcpy(batched, plain, total);
+ memcpy(iv_work, iv_orig, ivsize);
+ sg_init_one(&sg_in, batched, total);
+ sg_out = sg_in;
+ skcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG |
+ CRYPTO_TFM_REQ_MAY_SLEEP,
+ crypto_req_done, &wait);
+ skcipher_request_set_crypt(req, &sg_in, &sg_out, total, iv_work);
+ err = crypto_wait_req(crypto_skcipher_encrypt(req), &wait);
+ if (err) {
+ pr_err("alg: skcipher: %s multi-DU batched encrypt failed: %d\n",
+ driver, err);
+ goto out_clear_du;
+ }
+
+ /* Pass 2: TEST_MDU_NR_UNITS single-DU encrypts with derived IVs. */
+ err = crypto_skcipher_set_data_unit_size(tfm, 0);
+ if (err)
+ goto out;
+ memcpy(unit, plain, total);
+ memcpy(iv_work, iv_orig, ivsize);
+ for (i = 0; i < TEST_MDU_NR_UNITS; i++) {
+ sg_init_one(&sg_in, unit + i * du_size, du_size);
+ sg_out = sg_in;
+ skcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG |
+ CRYPTO_TFM_REQ_MAY_SLEEP,
+ crypto_req_done, &wait);
+ skcipher_request_set_crypt(req, &sg_in, &sg_out, du_size,
+ iv_work);
+ err = crypto_wait_req(crypto_skcipher_encrypt(req), &wait);
+ if (err) {
+ pr_err("alg: skcipher: %s single-DU[%u] encrypt failed: %d\n",
+ driver, i, err);
+ goto out;
+ }
+ /* Increment iv_work as a 128-bit little-endian counter. */
+ {
+ __le64 lo_le, hi_le;
+ u64 lo;
+
+ memcpy(&lo_le, iv_work, 8);
+ memcpy(&hi_le, iv_work + 8, 8);
+ lo = le64_to_cpu(lo_le) + 1;
+ lo_le = cpu_to_le64(lo);
+ memcpy(iv_work, &lo_le, 8);
+ if (lo == 0) {
+ hi_le = cpu_to_le64(le64_to_cpu(hi_le) + 1);
+ memcpy(iv_work + 8, &hi_le, 8);
+ }
+ }
+ }
+
+ if (memcmp(batched, unit, total) != 0) {
+ pr_err("alg: skcipher: %s multi-DU mismatch (du=%u, n=%u)\n",
+ driver, du_size, TEST_MDU_NR_UNITS);
+ err = -EINVAL;
+ }
+
+out_clear_du:
+ (void)crypto_skcipher_set_data_unit_size(tfm, 0);
+out:
+ skcipher_request_free(req);
+ kfree(unit);
+ kfree(batched);
+ kfree(plain);
+ return err;
+}
+
static int alg_test_skcipher(const struct alg_test_desc *desc,
const char *driver, u32 type, u32 mask)
{
@@ -3259,6 +3376,18 @@ static int alg_test_skcipher(const struct alg_test_desc *desc,
if (err)
goto out;
+ if (crypto_skcipher_supports_multi_data_unit(tfm)) {
+ static const unsigned int du_sizes[] = { 512, 1024, 2048, 4096 };
+ unsigned int j;
+
+ for (j = 0; j < ARRAY_SIZE(du_sizes); j++) {
+ err = test_skcipher_multi_du(tfm, du_sizes[j]);
+ if (err)
+ goto out;
+ cond_resched();
+ }
+ }
+
err = test_skcipher_vs_generic_impl(desc->generic_driver, req, tsgls);
out:
free_cipher_test_sglists(tsgls);
--
2.47.3
^ permalink raw reply related
* [PATCH v3 4/4] dm crypt: batch all sectors of a bio per crypto request
From: Leonid Ravich @ 2026-06-01 8:56 UTC (permalink / raw)
To: Herbert Xu
Cc: Alasdair Kergon, Ard Biesheuvel, Eric Biggers, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260601085644.13026-1-lravich@amazon.com>
When the underlying skcipher driver advertises support for multiple
data units in a single request (CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT),
configure the cipher with cc->sector_size as data_unit_size and
submit one request per bio instead of one request per sector. This
removes per-sector overhead in the crypto API hot path: request
allocation, callback dispatch, completion handling, and SG setup.
The optimisation is enabled automatically at table load when all
of the following hold:
- the cipher is non-aead (i.e. skcipher);
- tfms_count is 1 (interleaved per-sector keys would break batching);
- the IV mode is plain or plain64 (the only modes whose generator
produces a sequential 64-bit little-endian counter that the cipher
can extend by adding the data-unit index, matching the convention
documented in crypto_skcipher_set_data_unit_size());
- the iv_gen_ops->post() hook is unset (lmk and tcw use it; both are
already excluded by the IV-mode test, but the explicit check makes
the assumption durable against future IV modes);
- dm-integrity is not stacked (no integrity tag or integrity IV);
- the cipher driver advertises multi-data-unit support.
A new CRYPT_MULTI_DATA_UNIT cipher_flag, set once at construction
time, gates the multi-data-unit path. The existing per-sector path
in crypt_convert_block_skcipher() is unchanged; the new
crypt_convert_block_skcipher_multi() is reached from a small dispatch
in crypt_convert() and shares the same backlog/-EBUSY/-EINPROGRESS
flow control with the per-sector path.
Heap-allocated scatterlists are stashed in dm_crypt_request and freed
in crypt_free_req_skcipher() to avoid races between the synchronous-
success free path and async-completion reuse from the request pool.
On -ENOMEM during scatterlist allocation, the bio is requeued via
BLK_STS_DEV_RESOURCE rather than failed, matching the behaviour of
the existing -ENOMEM path for crypto request allocation.
Verified end-to-end with a byte-equivalence test: encrypted output of
plain64 dm-crypt with the multi-data-unit path matches output of the
single-data-unit path bit-for-bit over a 256 MB device.
Signed-off-by: Leonid Ravich <lravich@amazon.com>
---
drivers/md/dm-crypt.c | 281 ++++++++++++++++++++++++++++++++++++++++--
1 file changed, 274 insertions(+), 7 deletions(-)
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 608b617fb817..df20ffa6e61e 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -101,6 +101,14 @@ struct dm_crypt_request {
struct scatterlist sg_in[4];
struct scatterlist sg_out[4];
u64 iv_sector;
+ /*
+ * Heap-allocated scatterlists used by the multi-data-unit path
+ * when one bio is processed in a single skcipher request. NULL
+ * when the inline sg_in[]/sg_out[] arrays above are sufficient
+ * (single-data-unit path). Freed in crypt_free_req_skcipher().
+ */
+ struct scatterlist *sg_in_ext;
+ struct scatterlist *sg_out_ext;
};
struct crypt_config;
@@ -151,6 +159,7 @@ enum cipher_flags {
CRYPT_IV_LARGE_SECTORS, /* Calculate IV from sector_size, not 512B sectors */
CRYPT_ENCRYPT_PREPROCESS, /* Must preprocess data for encryption (elephant) */
CRYPT_KEY_MAC_SIZE_SET, /* The integrity_key_size option was used */
+ CRYPT_MULTI_DATA_UNIT, /* Batch all sectors of a bio per crypto request */
};
/*
@@ -1426,12 +1435,162 @@ static int crypt_convert_block_skcipher(struct crypt_config *cc,
return r;
}
+/*
+ * Multi-data-unit variant of crypt_convert_block_skcipher. Submits all
+ * remaining sectors of the current bio in one skcipher request whose
+ * data_unit_size is cc->sector_size. The cipher walks the IV between
+ * data units (see crypto_skcipher_set_data_unit_size()).
+ *
+ * Returns the same set of values as crypt_convert_block_skcipher:
+ * 0 on synchronous success (full chunk processed),
+ * -EINPROGRESS / -EBUSY on asynchronous dispatch,
+ * -EAGAIN if the per-bio scatterlist allocation cannot be made. The
+ * caller MUST disable multi-data-unit batching for the rest
+ * of this bio and re-enter the per-sector path, which uses
+ * only mempool reserves and is therefore safe even on the
+ * swap-out-to-dm-crypt path under total memory exhaustion.
+ * negative errno otherwise.
+ *
+ * On success the bio iterators have been advanced by the chunk size.
+ *
+ * Walks the bio with __bio_for_each_bvec so that multi-page folios
+ * produce one scatterlist entry rather than N (one per PAGE_SIZE).
+ */
+static int crypt_convert_block_skcipher_multi(struct crypt_config *cc,
+ struct convert_context *ctx,
+ struct skcipher_request *req,
+ unsigned int *out_processed)
+{
+ const unsigned int sector_size = cc->sector_size;
+ const gfp_t gfp = GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN;
+ unsigned int total = ctx->iter_in.bi_size;
+ unsigned int n_sg_in = 0, n_sg_out = 0;
+ struct dm_crypt_request *dmreq = dmreq_of_req(cc, req);
+ struct scatterlist *sg_in = NULL, *sg_out = NULL;
+ struct bvec_iter iter_in, iter_out;
+ struct bio_vec bv;
+ u8 *iv, *org_iv;
+ int r;
+
+ /*
+ * crypt_convert_init() sets bio_in == bio_out for reads and aligns
+ * the read/write iterators to the same byte count, so iter_in and
+ * iter_out always describe equally-sized payloads. WARN if that
+ * invariant is ever violated by a future change.
+ */
+ if (WARN_ON_ONCE(ctx->iter_in.bi_size != ctx->iter_out.bi_size))
+ return -EIO;
+
+ /*
+ * crypt_convert()'s outer loop only enters this helper when
+ * iter_in.bi_size > 0, so total is non-zero here; reject any
+ * sub-DU residue.
+ */
+ if (unlikely(total & (sector_size - 1)))
+ return -EIO;
+
+ /*
+ * Walk the bio_vec iterators to count how many SG entries we need
+ * for exactly @total bytes. bi_size of the iterators is at least
+ * @total by construction above.
+ */
+ iter_in = ctx->iter_in;
+ iter_in.bi_size = total;
+ __bio_for_each_bvec(bv, ctx->bio_in, iter_in, iter_in)
+ n_sg_in++;
+
+ iter_out = ctx->iter_out;
+ iter_out.bi_size = total;
+ __bio_for_each_bvec(bv, ctx->bio_out, iter_out, iter_out)
+ n_sg_out++;
+
+ sg_in = kmalloc_array(n_sg_in, sizeof(*sg_in), gfp);
+ sg_out = (ctx->bio_in == ctx->bio_out) ? sg_in :
+ kmalloc_array(n_sg_out, sizeof(*sg_out), gfp);
+ if (!sg_in || !sg_out) {
+ /*
+ * Allocation may legitimately fail under memory pressure on
+ * the swap-out-to-dm-crypt path. Return -EAGAIN so the
+ * caller falls back to the per-sector path for this bio
+ * rather than looping forever in the allocator or requeueing
+ * the bio just to fail again.
+ */
+ kfree(sg_in);
+ if (sg_out != sg_in)
+ kfree(sg_out);
+ return -EAGAIN;
+ }
+
+ sg_init_table(sg_in, n_sg_in);
+ {
+ unsigned int i = 0;
+
+ iter_in = ctx->iter_in;
+ iter_in.bi_size = total;
+ __bio_for_each_bvec(bv, ctx->bio_in, iter_in, iter_in)
+ sg_set_page(&sg_in[i++], bv.bv_page, bv.bv_len,
+ bv.bv_offset);
+ }
+
+ if (sg_out != sg_in) {
+ unsigned int i = 0;
+
+ sg_init_table(sg_out, n_sg_out);
+ iter_out = ctx->iter_out;
+ iter_out.bi_size = total;
+ __bio_for_each_bvec(bv, ctx->bio_out, iter_out, iter_out)
+ sg_set_page(&sg_out[i++], bv.bv_page, bv.bv_len,
+ bv.bv_offset);
+ }
+
+ /*
+ * Compute the IV for the first data unit. The cipher will derive
+ * IVs for subsequent data units by treating this one as a 128-bit
+ * little-endian counter and adding the data-unit index, which
+ * matches the layout produced by plain and plain64.
+ */
+ dmreq->iv_sector = ctx->cc_sector;
+ if (test_bit(CRYPT_IV_LARGE_SECTORS, &cc->cipher_flags))
+ dmreq->iv_sector >>= cc->sector_shift;
+ dmreq->ctx = ctx;
+
+ iv = iv_of_dmreq(cc, dmreq);
+ org_iv = org_iv_of_dmreq(cc, dmreq);
+ r = cc->iv_gen_ops->generator(cc, org_iv, dmreq);
+ if (r < 0)
+ goto out_free_sg;
+ memcpy(iv, org_iv, cc->iv_size);
+
+ /* Stash the SG arrays for cleanup on completion / free. */
+ dmreq->sg_in_ext = sg_in;
+ dmreq->sg_out_ext = (sg_out == sg_in) ? NULL : sg_out;
+
+ skcipher_request_set_crypt(req, sg_in, sg_out, total, iv);
+
+ if (bio_data_dir(ctx->bio_in) == WRITE)
+ r = crypto_skcipher_encrypt(req);
+ else
+ r = crypto_skcipher_decrypt(req);
+
+ *out_processed = total;
+ return r;
+
+out_free_sg:
+ kfree(sg_in);
+ if (sg_out != sg_in)
+ kfree(sg_out);
+ dmreq->sg_in_ext = NULL;
+ dmreq->sg_out_ext = NULL;
+ return r;
+}
+
static void kcryptd_async_done(void *async_req, int error);
static int crypt_alloc_req_skcipher(struct crypt_config *cc,
struct convert_context *ctx)
{
unsigned int key_index = ctx->cc_sector & (cc->tfms_count - 1);
+ struct dm_crypt_request *dmreq;
if (!ctx->r.req) {
ctx->r.req = mempool_alloc(&cc->req_pool, in_interrupt() ? GFP_ATOMIC : GFP_NOIO);
@@ -1441,6 +1600,18 @@ static int crypt_alloc_req_skcipher(struct crypt_config *cc,
skcipher_request_set_tfm(ctx->r.req, cc->cipher_tfm.tfms[key_index]);
+ /*
+ * Initialise the heap-allocated scatterlist pointers so that
+ * crypt_free_req_skcipher() does not read uninitialised memory
+ * for paths that don't take the multi-data-unit branch. The
+ * dmreq trailer lives in the per-bio data area which is not
+ * zeroed by the dm core, and the request is reused from the
+ * mempool across many bios.
+ */
+ dmreq = dmreq_of_req(cc, ctx->r.req);
+ dmreq->sg_in_ext = NULL;
+ dmreq->sg_out_ext = NULL;
+
/*
* Use REQ_MAY_BACKLOG so a cipher driver internally backlogs
* requests if driver request queue is full.
@@ -1487,6 +1658,12 @@ static void crypt_free_req_skcipher(struct crypt_config *cc,
struct skcipher_request *req, struct bio *base_bio)
{
struct dm_crypt_io *io = dm_per_bio_data(base_bio, cc->per_bio_data_size);
+ struct dm_crypt_request *dmreq = dmreq_of_req(cc, req);
+
+ kfree(dmreq->sg_in_ext);
+ dmreq->sg_in_ext = NULL;
+ kfree(dmreq->sg_out_ext);
+ dmreq->sg_out_ext = NULL;
if ((struct skcipher_request *)(io + 1) != req)
mempool_free(req, &cc->req_pool);
@@ -1515,7 +1692,9 @@ static void crypt_free_req(struct crypt_config *cc, void *req, struct bio *base_
static blk_status_t crypt_convert(struct crypt_config *cc,
struct convert_context *ctx, bool atomic, bool reset_pending)
{
- unsigned int sector_step = cc->sector_size >> SECTOR_SHIFT;
+ const unsigned int sector_step = cc->sector_size >> SECTOR_SHIFT;
+ bool multi_du = test_bit(CRYPT_MULTI_DATA_UNIT, &cc->cipher_flags);
+ unsigned int processed;
int r;
/*
@@ -1536,8 +1715,13 @@ static blk_status_t crypt_convert(struct crypt_config *cc,
atomic_inc(&ctx->cc_pending);
+ processed = cc->sector_size;
if (crypt_integrity_aead(cc))
r = crypt_convert_block_aead(cc, ctx, ctx->r.req_aead, ctx->tag_offset);
+ else if (multi_du)
+ r = crypt_convert_block_skcipher_multi(cc, ctx,
+ ctx->r.req,
+ &processed);
else
r = crypt_convert_block_skcipher(cc, ctx, ctx->r.req, ctx->tag_offset);
@@ -1559,8 +1743,19 @@ static blk_status_t crypt_convert(struct crypt_config *cc,
* exit and continue processing in a workqueue
*/
ctx->r.req = NULL;
- ctx->tag_offset++;
- ctx->cc_sector += sector_step;
+ if (!multi_du) {
+ ctx->tag_offset++;
+ ctx->cc_sector += sector_step;
+ } else {
+ bio_advance_iter(ctx->bio_in,
+ &ctx->iter_in,
+ processed);
+ bio_advance_iter(ctx->bio_out,
+ &ctx->iter_out,
+ processed);
+ ctx->cc_sector +=
+ processed >> SECTOR_SHIFT;
+ }
return BLK_STS_DEV_RESOURCE;
}
} else {
@@ -1574,19 +1769,52 @@ static blk_status_t crypt_convert(struct crypt_config *cc,
*/
case -EINPROGRESS:
ctx->r.req = NULL;
- ctx->tag_offset++;
- ctx->cc_sector += sector_step;
+ if (!multi_du) {
+ ctx->tag_offset++;
+ ctx->cc_sector += sector_step;
+ } else {
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in,
+ processed);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out,
+ processed);
+ ctx->cc_sector += processed >> SECTOR_SHIFT;
+ }
continue;
/*
* The request was already processed (synchronously).
*/
case 0:
atomic_dec(&ctx->cc_pending);
- ctx->cc_sector += sector_step;
- ctx->tag_offset++;
+ if (!multi_du) {
+ ctx->cc_sector += sector_step;
+ ctx->tag_offset++;
+ } else {
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in,
+ processed);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out,
+ processed);
+ ctx->cc_sector += processed >> SECTOR_SHIFT;
+ }
if (!atomic)
cond_resched();
continue;
+ /*
+ * Multi-data-unit scatterlist allocation failed. This can
+ * happen on the swap-out-to-dm-crypt path under memory
+ * pressure, where retrying with the same allocation policy
+ * could loop forever. Disable multi-data-unit batching for
+ * the rest of this crypt_convert() invocation and re-enter
+ * the per-sector path, which uses only mempool reserves and
+ * is guaranteed to make forward progress even under total
+ * memory exhaustion. The per-tfm data_unit_size is left
+ * unchanged, so subsequent bios (which start a fresh
+ * crypt_convert() and re-read cipher_flags) will retry the
+ * multi-data-unit path once memory pressure eases.
+ */
+ case -EAGAIN:
+ atomic_dec(&ctx->cc_pending);
+ multi_du = false;
+ continue;
/*
* There was a data integrity error.
*/
@@ -3063,6 +3291,45 @@ static int crypt_ctr_cipher(struct dm_target *ti, char *cipher_in, char *key)
}
}
+ /*
+ * Enable multi-data-unit batching when the cipher supports it and
+ * the IV layout is one we can derive per-DU from a single starting
+ * IV: plain or plain64 produce a sequential 64-bit little-endian
+ * counter, which matches the convention of
+ * crypto_skcipher_set_data_unit_size(). Restrict to the simple
+ * case (single tfm, no integrity, no per-sector post() callback)
+ * to keep the consumer path small; modes like essiv, lmk, tcw,
+ * eboiv, plain64be, random, null, benbi, and elephant are
+ * deliberately excluded because their generators or post-IV hooks
+ * cannot be re-derived by the cipher between data units.
+ */
+ if (!crypt_integrity_aead(cc) && cc->tfms_count == 1 &&
+ cc->iv_gen_ops &&
+ (cc->iv_gen_ops == &crypt_iv_plain_ops ||
+ cc->iv_gen_ops == &crypt_iv_plain64_ops) &&
+ !cc->iv_gen_ops->post &&
+ !cc->integrity_tag_size && !cc->integrity_iv_size &&
+ crypto_skcipher_supports_multi_data_unit(cc->cipher_tfm.tfms[0])) {
+ ret = crypto_skcipher_set_data_unit_size(cc->cipher_tfm.tfms[0],
+ cc->sector_size);
+ if (!ret) {
+ set_bit(CRYPT_MULTI_DATA_UNIT, &cc->cipher_flags);
+ DMINFO("Using multi-data-unit crypto offload (du=%u)",
+ cc->sector_size);
+ } else {
+ /*
+ * The driver advertised the capability via cra_flags
+ * but rejected the requested data unit size. This is
+ * a driver bug worth seeing in dmesg; fall back to
+ * the per-sector path so the device still activates.
+ */
+ DMWARN_LIMIT("multi-DU offload disabled: %s rejected du=%u (%d)",
+ crypto_skcipher_driver_name(cc->cipher_tfm.tfms[0]),
+ cc->sector_size, ret);
+ ret = 0;
+ }
+ }
+
/* wipe the kernel key payload copy */
if (cc->key_string)
memset(cc->key, 0, cc->key_size * sizeof(u8));
--
2.47.3
^ permalink raw reply related
* [PATCH v3 2/4] crypto: xts - support multiple data units per request in template
From: Leonid Ravich @ 2026-06-01 8:56 UTC (permalink / raw)
To: Herbert Xu
Cc: Alasdair Kergon, Ard Biesheuvel, Eric Biggers, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260601085644.13026-1-lravich@amazon.com>
Teach the generic xts() template to consume cryptlen larger than one
data unit when the caller has configured a non-zero data_unit_size on
the tfm. Each data unit is processed with its own IV, derived from
the caller-supplied IV by treating it as a 128-bit little-endian
counter and adding the data-unit index. This matches the
sector-indexed XTS used by dm-crypt's plain64 IV mode and by typical
inline-encryption hardware.
The single-data-unit body is unchanged and is now reached via a thin
xts_crypt_multi() dispatcher that skips straight to the body when
data_unit_size is zero (the legacy default), so existing users see
no extra cost.
Advertise CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT in cra_flags only when
the inner cipher is synchronous. An async inner cipher would require
a per-DU completion chain which is out of scope for the slow software
template; consumers that need multi-DU on async hardware will use one
of the arch-specific drivers added later in this series.
Signed-off-by: Leonid Ravich <lravich@amazon.com>
---
crypto/xts.c | 25 +++++++++++++++++++++++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/crypto/xts.c b/crypto/xts.c
index ad97c8091582..f0585ea9d6d5 100644
--- a/crypto/xts.c
+++ b/crypto/xts.c
@@ -258,7 +258,7 @@ static int xts_init_crypt(struct skcipher_request *req,
return 0;
}
-static int xts_encrypt(struct skcipher_request *req)
+static int xts_encrypt_one(struct skcipher_request *req)
{
struct xts_request_ctx *rctx = skcipher_request_ctx(req);
struct skcipher_request *subreq = &rctx->subreq;
@@ -275,7 +275,7 @@ static int xts_encrypt(struct skcipher_request *req)
return xts_cts_final(req, crypto_skcipher_encrypt);
}
-static int xts_decrypt(struct skcipher_request *req)
+static int xts_decrypt_one(struct skcipher_request *req)
{
struct xts_request_ctx *rctx = skcipher_request_ctx(req);
struct skcipher_request *subreq = &rctx->subreq;
@@ -292,6 +292,16 @@ static int xts_decrypt(struct skcipher_request *req)
return xts_cts_final(req, crypto_skcipher_decrypt);
}
+static int xts_encrypt(struct skcipher_request *req)
+{
+ return skcipher_walk_data_units(req, xts_encrypt_one);
+}
+
+static int xts_decrypt(struct skcipher_request *req)
+{
+ return skcipher_walk_data_units(req, xts_decrypt_one);
+}
+
static int xts_init_tfm(struct crypto_skcipher *tfm)
{
struct skcipher_instance *inst = skcipher_alg_instance(tfm);
@@ -427,6 +437,17 @@ static int xts_create(struct crypto_template *tmpl, struct rtattr **tb)
inst->alg.base.cra_alignmask = alg->base.cra_alignmask |
(__alignof__(u64) - 1);
+ /*
+ * Advertise multi-data-unit support only when the inner cipher is
+ * synchronous. The dispatcher in skcipher_walk_data_units() calls
+ * the single-DU body in a loop and assumes synchronous completion;
+ * supporting async would require a per-DU callback chain, which
+ * the slow software template does not need.
+ */
+ if (!(alg->base.cra_flags & CRYPTO_ALG_ASYNC))
+ inst->alg.base.cra_flags |=
+ CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT;
+
inst->alg.ivsize = XTS_BLOCK_SIZE;
inst->alg.min_keysize = alg->min_keysize * 2;
inst->alg.max_keysize = alg->max_keysize * 2;
--
2.47.3
^ permalink raw reply related
* [PATCH v3 1/4] crypto: skcipher - add per-tfm data_unit_size for batched requests
From: Leonid Ravich @ 2026-06-01 8:56 UTC (permalink / raw)
To: Herbert Xu
Cc: Alasdair Kergon, Ard Biesheuvel, Eric Biggers, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260601085644.13026-1-lravich@amazon.com>
Add a per-tfm data_unit_size and an algorithm capability flag that
together allow a caller to submit several data units in a single
skcipher request. The IV passed in the request applies to the first
data unit; the algorithm advances the tweak between data units
according to the mode specification (e.g., LE128 multiply for XTS per
IEEE 1619).
This mirrors the data_unit_size concept already exposed by
struct blk_crypto_config for inline encryption hardware, but at the
software skcipher layer. The first user is dm-crypt, which today
issues one request per sector and so pays a per-sector cost in
request allocation, IV generation, callback dispatch, and completion
handling. Allowing the cipher to consume a whole bio per request
removes that overhead for drivers that can chain across data units
internally.
The data_unit_size lives on struct crypto_skcipher rather than on
struct skcipher_request because it does not change between requests
for any plausible consumer: dm-crypt picks one sector size per
mapped target at table load time; fscrypt would pick one per master
key. Anchoring it to the tfm also lets the driver validate it once
at setkey() time and avoids per-request initialisation hazards on
mempool-recycled requests.
Capability is advertised with CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT
in cra_flags (type-specific high-byte range, mirroring the
CRYPTO_AHASH_ALG_* convention). This makes the capability visible
in /proc/crypto and lets templates OR it into their derived
algorithms.
crypto_skcipher_set_data_unit_size() returns -EOPNOTSUPP if the
algorithm does not advertise the flag, and accepts 0 (the default)
unconditionally so callers can re-disable batching cheaply.
crypto_skcipher_encrypt()/decrypt() reject requests whose cryptlen
is not a multiple of the configured data_unit_size with -EINVAL.
The check is gated on data_unit_size != 0 so it costs nothing for
the common single-data-unit case.
No in-tree algorithm advertises the flag yet; subsequent patches
add the generic xts() template, arm64, and x86 producers as well
as the dm-crypt consumer.
Signed-off-by: Leonid Ravich <lravich@amazon.com>
---
crypto/skcipher.c | 120 +++++++++++++++++++++++++++++
include/crypto/internal/skcipher.h | 34 ++++++++
include/crypto/skcipher.h | 85 ++++++++++++++++++++
3 files changed, 239 insertions(+)
diff --git a/crypto/skcipher.c b/crypto/skcipher.c
index 2b31d1d5d268..bc37bd554aec 100644
--- a/crypto/skcipher.c
+++ b/crypto/skcipher.c
@@ -432,13 +432,119 @@ int crypto_skcipher_setkey(struct crypto_skcipher *tfm, const u8 *key,
}
EXPORT_SYMBOL_GPL(crypto_skcipher_setkey);
+int crypto_skcipher_set_data_unit_size(struct crypto_skcipher *tfm,
+ unsigned int data_unit_size)
+{
+ unsigned int blocksize;
+
+ if (!data_unit_size) {
+ tfm->data_unit_size = 0;
+ return 0;
+ }
+
+ if (!crypto_skcipher_supports_multi_data_unit(tfm))
+ return -EOPNOTSUPP;
+
+ blocksize = crypto_skcipher_blocksize(tfm);
+ if (data_unit_size < blocksize || data_unit_size % blocksize)
+ return -EINVAL;
+
+ tfm->data_unit_size = data_unit_size;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(crypto_skcipher_set_data_unit_size);
+
+static int crypto_skcipher_check_data_unit_size(struct crypto_skcipher *tfm,
+ struct skcipher_request *req)
+{
+ unsigned int du = tfm->data_unit_size;
+
+ if (likely(!du))
+ return 0;
+ if (req->cryptlen % du)
+ return -EINVAL;
+ return 0;
+}
+
+/*
+ * Increment a 16-byte little-endian counter held in @iv. See
+ * crypto_skcipher_set_data_unit_size() for the convention.
+ */
+static inline void skcipher_iv_inc_le128(u8 *iv)
+{
+ __le64 lo_le, hi_le;
+ u64 lo;
+
+ memcpy(&lo_le, iv, 8);
+ memcpy(&hi_le, iv + 8, 8);
+ lo = le64_to_cpu(lo_le) + 1;
+ lo_le = cpu_to_le64(lo);
+ memcpy(iv, &lo_le, 8);
+ if (unlikely(lo == 0)) {
+ hi_le = cpu_to_le64(le64_to_cpu(hi_le) + 1);
+ memcpy(iv + 8, &hi_le, 8);
+ }
+}
+
+int skcipher_walk_data_units(struct skcipher_request *req,
+ int (*body)(struct skcipher_request *))
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ const unsigned int du = tfm->data_unit_size;
+ const unsigned int total = req->cryptlen;
+ struct scatterlist *orig_src = req->src;
+ struct scatterlist *orig_dst = req->dst;
+ struct scatterlist src_sg[2], dst_sg[2];
+ u8 iv_save[16];
+ unsigned int off;
+ int err = 0;
+
+ if (likely(!du))
+ return body(req);
+
+ /*
+ * Registration of an algorithm advertising
+ * CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT enforces ivsize == 16
+ * (see skcipher_prepare_alg_common()), so this is purely
+ * defensive against algorithm-registration bugs.
+ */
+ if (WARN_ON_ONCE(crypto_skcipher_ivsize(tfm) != 16))
+ return -EINVAL;
+
+ memcpy(iv_save, req->iv, 16);
+
+ for (off = 0; off < total; off += du) {
+ req->cryptlen = du;
+ req->src = scatterwalk_ffwd(src_sg, orig_src, off);
+ req->dst = (orig_src == orig_dst) ? req->src :
+ scatterwalk_ffwd(dst_sg, orig_dst, off);
+
+ err = body(req);
+ if (err)
+ break;
+
+ skcipher_iv_inc_le128(iv_save);
+ memcpy(req->iv, iv_save, 16);
+ }
+
+ req->src = orig_src;
+ req->dst = orig_dst;
+ req->cryptlen = total;
+ return err;
+}
+EXPORT_SYMBOL_GPL(skcipher_walk_data_units);
+
int crypto_skcipher_encrypt(struct skcipher_request *req)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ int err;
if (crypto_skcipher_get_flags(tfm) & CRYPTO_TFM_NEED_KEY)
return -ENOKEY;
+ err = crypto_skcipher_check_data_unit_size(tfm, req);
+ if (err)
+ return err;
if (alg->co.base.cra_type != &crypto_skcipher_type)
return crypto_lskcipher_encrypt_sg(req);
return alg->encrypt(req);
@@ -449,9 +555,13 @@ int crypto_skcipher_decrypt(struct skcipher_request *req)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ int err;
if (crypto_skcipher_get_flags(tfm) & CRYPTO_TFM_NEED_KEY)
return -ENOKEY;
+ err = crypto_skcipher_check_data_unit_size(tfm, req);
+ if (err)
+ return err;
if (alg->co.base.cra_type != &crypto_skcipher_type)
return crypto_lskcipher_decrypt_sg(req);
return alg->decrypt(req);
@@ -680,6 +790,16 @@ int skcipher_prepare_alg_common(struct skcipher_alg_common *alg)
(alg->ivsize + alg->statesize) > PAGE_SIZE / 2)
return -EINVAL;
+ /*
+ * Algorithms advertising multi-data-unit support must use the
+ * 16-byte little-endian counter convention documented in
+ * crypto_skcipher_set_data_unit_size(); see also
+ * skcipher_walk_data_units().
+ */
+ if ((base->cra_flags & CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT) &&
+ alg->ivsize != 16)
+ return -EINVAL;
+
if (!alg->chunksize)
alg->chunksize = base->cra_blocksize;
diff --git a/include/crypto/internal/skcipher.h b/include/crypto/internal/skcipher.h
index a965b6aabf61..bed1b1f1bbdc 100644
--- a/include/crypto/internal/skcipher.h
+++ b/include/crypto/internal/skcipher.h
@@ -21,6 +21,40 @@
*/
#define CRYPTO_ALG_SKCIPHER_REQSIZE_LARGE CRYPTO_ALG_OPTIONAL_KEY
+/**
+ * skcipher_walk_data_units - dispatch a request as one body call per data unit
+ * @req: the caller's skcipher request
+ * @body: the algorithm's single-data-unit encrypt or decrypt function
+ *
+ * When tfm->data_unit_size is zero this is a tail call into @body with
+ * @req unchanged. Otherwise the request is split into
+ * cryptlen / data_unit_size sub-ranges and @body is called once per
+ * sub-range with req->cryptlen, req->src, req->dst, and req->iv adjusted
+ * for that sub-range. The IV passed to data unit n is the caller-
+ * supplied IV plus n, where + is a 128-bit little-endian add — this
+ * matches the convention documented in
+ * crypto_skcipher_set_data_unit_size().
+ *
+ * Many single-data-unit XTS bodies modify the IV buffer in place during
+ * processing (the tweak is walked block by block). This helper saves
+ * the caller's IV before each call and rewrites the next data unit's
+ * IV from the saved value, so the body always sees a fresh per-DU IV
+ * regardless of any in-place mutation it performs.
+ *
+ * The body MUST run to completion synchronously. Drivers that use this
+ * helper therefore advertise CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT only
+ * for synchronous configurations.
+ *
+ * After the call returns, the contents of req->iv are unspecified per
+ * the documented contract. src/dst/cryptlen are restored to the
+ * caller's values to keep skcipher request post-conditions intact.
+ *
+ * Return: 0 on success, or the body's negative errno on the first
+ * data unit that returned non-zero.
+ */
+int skcipher_walk_data_units(struct skcipher_request *req,
+ int (*body)(struct skcipher_request *));
+
struct aead_request;
struct rtattr;
diff --git a/include/crypto/skcipher.h b/include/crypto/skcipher.h
index 4efe2ca8c4d1..5941b6b24b98 100644
--- a/include/crypto/skcipher.h
+++ b/include/crypto/skcipher.h
@@ -26,6 +26,15 @@
/* Set this bit if the skcipher operation is not final. */
#define CRYPTO_SKCIPHER_REQ_NOTFINAL 0x00000002
+/*
+ * Set in cra_flags by an skcipher algorithm that supports processing
+ * multiple data units in a single request. See
+ * crypto_skcipher_set_data_unit_size().
+ *
+ * Type-specific flag in the 0xff000000 reserved range.
+ */
+#define CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT 0x01000000
+
struct scatterlist;
/**
@@ -53,6 +62,22 @@ struct skcipher_request {
struct crypto_skcipher {
unsigned int reqsize;
+ /*
+ * Number of bytes in one data unit when batching multiple data units
+ * per request. 0 means "single data unit per request" (legacy
+ * behaviour). Set via crypto_skcipher_set_data_unit_size().
+ *
+ * When non-zero, cryptlen must be a multiple of data_unit_size. The
+ * IV passed in skcipher_request::iv applies to the first data unit;
+ * the algorithm advances the tweak between data units according to
+ * the mode specification (e.g., LE128 multiply for XTS per
+ * IEEE 1619).
+ *
+ * Only algorithms that advertise CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT
+ * in cra_flags accept a non-zero value.
+ */
+ unsigned int data_unit_size;
+
struct crypto_tfm base;
};
@@ -492,6 +517,66 @@ static inline unsigned int crypto_lskcipher_chunksize(
return crypto_lskcipher_alg(tfm)->co.chunksize;
}
+/**
+ * crypto_skcipher_supports_multi_data_unit() - test multi-data-unit support
+ * @tfm: cipher handle
+ *
+ * Return: true if the algorithm advertises that it can process multiple
+ * data units in a single skcipher_request.
+ */
+static inline bool
+crypto_skcipher_supports_multi_data_unit(struct crypto_skcipher *tfm)
+{
+ return crypto_skcipher_alg_common(tfm)->base.cra_flags &
+ CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT;
+}
+
+/**
+ * crypto_skcipher_set_data_unit_size() - set data unit size for the tfm
+ * @tfm: cipher handle
+ * @data_unit_size: data unit size in bytes; 0 disables multi-data-unit mode
+ *
+ * Configure the tfm to process multiple data units per request. When set
+ * to a non-zero value, every subsequent encrypt/decrypt request must have
+ * cryptlen that is a multiple of @data_unit_size. Each data unit is
+ * processed as if it were a separate request whose IV is derived from the
+ * preceding data unit's IV by the algorithm-specific tweak update rule:
+ * the implementation treats the caller-supplied IV as a 128-bit
+ * little-endian counter and adds the data-unit index for each subsequent
+ * data unit.
+ *
+ * The contents of req->iv after a multi-data-unit request returns are
+ * unspecified — callers MUST NOT rely on it being either the original
+ * value or the final-data-unit value. Set a fresh IV before every
+ * request.
+ *
+ * The algorithm must advertise CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT in its
+ * cra_flags. @data_unit_size must be a positive multiple of the
+ * algorithm's cra_blocksize, otherwise -EINVAL is returned.
+ *
+ * Setting @data_unit_size to 0 reverts the tfm to single-data-unit
+ * behaviour and is always permitted.
+ *
+ * Return: 0 on success; -EOPNOTSUPP if the algorithm does not advertise
+ * multi-data-unit support; -EINVAL if @data_unit_size is not a
+ * positive multiple of the cipher block size.
+ */
+int crypto_skcipher_set_data_unit_size(struct crypto_skcipher *tfm,
+ unsigned int data_unit_size);
+
+/**
+ * crypto_skcipher_data_unit_size() - obtain data unit size
+ * @tfm: cipher handle
+ *
+ * Return: configured data unit size in bytes; 0 if multi-data-unit mode
+ * is disabled.
+ */
+static inline unsigned int
+crypto_skcipher_data_unit_size(struct crypto_skcipher *tfm)
+{
+ return tfm->data_unit_size;
+}
+
/**
* crypto_skcipher_statesize() - obtain state size
* @tfm: cipher handle
--
2.47.3
^ permalink raw reply related
* [PATCH v3 0/4] crypto: skcipher - per-tfm multi-data-unit batching
From: Leonid Ravich @ 2026-06-01 8:56 UTC (permalink / raw)
To: Herbert Xu
Cc: Alasdair Kergon, Ard Biesheuvel, Eric Biggers, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
This is v3 of the multi-data-unit skcipher request series, addressing
review feedback from Mikulas Patocka on v2.
v2: https://lore.kernel.org/linux-crypto/20260527065021.19525-1-lravich@amazon.com/
v1: https://lore.kernel.org/linux-crypto/20260519115955.27267-1-lravich@amazon.com/
The series adds a per-tfm "data unit size" to the skcipher API so a
caller can submit several data units in one crypto request, mirroring
the data_unit_size concept already exposed by struct blk_crypto_config
for inline encryption hardware. The first user is dm-crypt, which
today issues one skcipher request per sector and so pays a per-sector
cost in request allocation, callback dispatch, completion handling,
and scatterlist setup.
Proof-of-concept performance numbers from the RFC reply [1]: +19%
throughput / -40% CPU on a single-core arm64 system with a hardware
XTS-AES-256 accelerator running fio 4 KiB sequential writes through
dm-crypt, when an out-of-tree arm64 xts driver advertises the new
flag. This series itself does not include arch enablement.
[1] https://lore.kernel.org/linux-crypto/20260428101225.24316-1-lravich@amazon.com/
Changes since v2
----------------
Patch 4 (dm-crypt) only. Patches 1-3 are unchanged from v2.
- Replace integer division with the equivalent shift, and tighten
the size sanity check from "is total < sector_size?" to "is
total a multiple of sector_size?". Reject unaligned residues
explicitly instead of silently truncating them. The local
n_sectors variable used only for a now-redundant !=0 check was
dropped — crypt_convert()'s outer while-loop already guarantees
iter_in.bi_size > 0 on entry. (Mikulas)
- Drop `min(iter_in.bi_size, iter_out.bi_size)` in favour of using
iter_in.bi_size directly, with a WARN_ON_ONCE() to flag any
future violation of the "iter_in and iter_out describe equally-
sized payloads" invariant maintained by crypt_convert_init().
Replaces a silent mask of a real bug with an explicit warning.
(Mikulas)
Changes since v1
----------------
Patch 4 only. Addressed Mikulas's review of v1:
- Multi-DU scatterlist allocation uses
GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN.
- On scatterlist allocation failure, return -EAGAIN.
crypt_convert() handles -EAGAIN by clearing its local multi_du
flag and re-entering the per-sector path for the rest of this
crypt_convert() invocation. The per-tfm data_unit_size on the
cipher remains set, so subsequent bios (which start a fresh
crypt_convert() and re-read cipher_flags) get to try multi-DU
again once memory pressure eases.
This gives forward progress under total memory exhaustion: the
per-sector path uses only cc->req_pool (a mempool with reservoir
set up at table-load time) and the inline
dmreq->sg_in[]/sg_out[] arrays, never doing any allocation that
could fail.
- Walk the bio with __bio_for_each_bvec instead of
__bio_for_each_segment for folio-friendly SG construction.
Design overview (unchanged from v1)
-----------------------------------
* Patch 1 adds an `unsigned int data_unit_size` field to
`struct crypto_skcipher` (per-tfm: invariant for the consumer's
lifetime, set once via `crypto_skcipher_set_data_unit_size()`),
plus a capability flag CRYPTO_ALG_SKCIPHER_MULTI_DATA_UNIT in
`cra_flags` (type-specific high-byte range, mirroring the
CRYPTO_AHASH_ALG_BLOCK_ONLY precedent). `crypto_skcipher_encrypt()`
and `crypto_skcipher_decrypt()` validate that `cryptlen` is a
positive multiple of `data_unit_size`. The setter rejects
sub-blocksize values; algorithm registration rejects the flag for
algorithms with `ivsize != 16`.
Also exposes `skcipher_walk_data_units()` in
<crypto/internal/skcipher.h> as a default per-DU dispatcher for
drivers that don't want to roll their own.
* Patch 2 lets the generic `xts(...)` template advertise the flag
when the inner cipher is synchronous.
* Patch 3 extends `testmgr` with a self-comparison test that fires
automatically for every alg advertising the flag.
* Patch 4 turns dm-crypt on automatically when all of the
following hold at table load: skcipher (not aead), tfms_count
== 1, IV mode is plain or plain64, no per-sector
iv_gen_ops->post() hook, no dm-integrity stacking, and the
underlying cipher advertises the capability.
This series intentionally does NOT add the capability flag to any
arch crypto driver. Arch maintainers can opt in independently in
follow-up patches.
Verification
------------
A formal regression protocol is included in the project tree
(.claude/regression-protocol.md, .claude/run-regression.sh). The
v3 reference run reports 12/12 cases PASS:
- x86 + arm64 build clean (with and without out-of-tree arch
enablement).
- checkpatch.pl --strict: clean on all 4 patches.
- testmgr self-comparison: PASS for any algorithm advertising the
flag (verified end-to-end against an out-of-tree arm64/x86 xts
driver during regression).
- dm-crypt activation gating: plain/plain64 enabled,
essiv:sha256 / plain64be fall back.
- dm-crypt round-trip plain64: PASS with multi-DU active.
- dm-crypt round-trip essiv:sha256 (per-sector path on multi-DU
kernel): PASS.
- dm-crypt low-memory (mem=128M): PASS, no OOM kill.
- Byte-equivalence: 256 MB of ciphertext written through the
multi-DU path is bit-identical to ciphertext written through
the per-sector path on an unpatched axboe/for-next baseline
(sha256
4913910b1aa6f8859fcb8f4adec20230274993a3ade8f4dd0140a323dc43efc0).
The on-disk format is unchanged.
- arm64 functional (activation + round-trip) under qemu-aarch64:
PASS.
The OOM-fallback path (multi-DU helper returns -EAGAIN, caller
reverts to per-sector) is verified by inspection: the fallback is
two lines in crypt_convert(), the per-sector path uses only the
existing mempool reserve and the inline dmreq SG arrays (no
allocation that could fail), and there is no shared state between
the two paths that could deadlock.
Leonid Ravich (4):
crypto: skcipher - add per-tfm data_unit_size for batched requests
crypto: xts - support multiple data units per request in template
crypto: testmgr - exercise multi-data-unit path for skcipher
dm crypt: batch all sectors of a bio per crypto request
crypto/skcipher.c | 120 ++++++++++++
crypto/testmgr.c | 129 +++++++++++++
crypto/xts.c | 25 ++-
drivers/md/dm-crypt.c | 281 ++++++++++++++++++++++++++++-
include/crypto/internal/skcipher.h | 34 ++++
include/crypto/skcipher.h | 85 +++++++++
6 files changed, 665 insertions(+), 9 deletions(-)
base-commit: a8cafdf8c949f17c92eca0045532e88ac0dac30d
--
2.47.3
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox