* [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0
@ 2026-10-02 9:56 Radu Nicolau
2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw)
To: dev; +Cc: Radu Nicolau
Intel(R) Multi-Buffer Crypto for IPsec Library released version 3.0.0
codenamed Túnel Pablo
https://github.com/intel/intel-ipsec-mb/releases/tag/v3.0.0
This patchset adds support for the new library version and support
for the new algorithms included in this release.
Radu Nicolau (4):
crypto/ipsec_mb: add support for IPsecMB 3.0
crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms
crypto/ipsec_mb: add support for ML-KEM and ML-DSA
test/crypto: add asym test suite for AESNI_MB
app/test/test_cryptodev_asym.c | 7 +
doc/guides/cryptodevs/aesni_gcm.rst | 4 +-
doc/guides/cryptodevs/aesni_mb.rst | 26 +-
doc/guides/cryptodevs/features/aesni_mb.ini | 12 +
doc/guides/rel_notes/release_26_11.rst | 12 +
drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 58 ++
drivers/crypto/ipsec_mb/ipsec_mb_private.h | 20 +
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++--
drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +-
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 695 +++++++++++++++++--
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 360 +++++++++-
11 files changed, 1354 insertions(+), 138 deletions(-)
--
2.52.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/4] crypto/ipsec_mb: add support for IPsecMB 3.0 2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau @ 2026-10-02 9:56 ` Radu Nicolau 2026-10-06 12:06 ` Ji, Kai 2026-10-02 9:56 ` [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms Radu Nicolau ` (3 subsequent siblings) 4 siblings, 1 reply; 7+ messages in thread From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw) To: dev; +Cc: Radu Nicolau, Kai Ji, Pablo de Lara Add support for IPsecMB release 3.0 - removed all _BITLEN enums, all lengths are in bytes - removed feature flag - use non const IMB_MGR - use IMB API instead of function pointers in the GCM PMD - remove ZUC 256 support Signed-off-by: Radu Nicolau <radu.nicolau@intel.com> --- doc/guides/cryptodevs/aesni_gcm.rst | 4 +- doc/guides/cryptodevs/aesni_mb.rst | 11 +- doc/guides/rel_notes/release_26_11.rst | 4 + drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++++++++++++---- drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 107 +++++--- drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 47 +++- 7 files changed, 349 insertions(+), 122 deletions(-) diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst index aefc1cedcb..e68dc4eb1e 100644 --- a/doc/guides/cryptodevs/aesni_gcm.rst +++ b/doc/guides/cryptodevs/aesni_gcm.rst @@ -40,8 +40,8 @@ Installation To build DPDK with the AESNI_GCM_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v2.0, which -can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_. +The latest version of the library supported by this PMD is v3.0, which +can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_. For Arm system, ARM64 port of the multi-buffer library can be downloaded from https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/. diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index add3c03b74..4851997fd9 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -73,6 +73,13 @@ Protocol offloads: * RTE_SECURITY_PROTOCOL_DOCSIS +Note +---- +Version 3.0 of the multi-buffer library removed support for: + +* the 256 bit versions of ZUC EEA3 and ZUC EIA3 algorithms. +* non byte aligned bit length (``RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA`` removed). + Limitations ----------- @@ -96,8 +103,8 @@ Installation To build DPDK with the AESNI_MB_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v2.0, which -can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_. +The latest version of the library supported by this PMD is v3.0, which +can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_. For Arm system, ARM64 port of the multi-buffer library can be downloaded from https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/. diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index 69ab5af8a3..ea126aa9bb 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -33,6 +33,10 @@ New Features ``pss_salt`` field in ``rte_crypto_rsa_op_param`` to carry it per sign operation. +* **Updated AESNI_MB crypto driver.** + + * Added support for version 3.0 of IPsec MB Library. + .. This section should contain new features added in this release. Sample format: diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c index 3c266cc47e..0c4358a7b1 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c @@ -6,56 +6,205 @@ #include "pmd_aesni_gcm_priv.h" +/* wrapper functions for library versions < 3.0.0 */ +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_pre(const void *key, \ + struct gcm_key_data *key_data, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_pre(key, key_data); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_init( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, const void *iv, \ + const void *aad, uint64_t aad_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_init(key_data, context_data, iv, aad, aad_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, uint64_t len, const void *iv, \ + const void *aad, uint64_t aad_len, void *auth_tag, \ + uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_enc(key_data, context_data, out, in, len, iv, \ + aad, aad_len, auth_tag, auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_update( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, const uint64_t len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_enc_update(key_data, context_data, out, in, len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_finalize( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *auth_tag, \ + const uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_enc_finalize(key_data, context_data, auth_tag, \ + auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, uint64_t len, const void *iv, \ + const void *aad, uint64_t aad_len, void *auth_tag, \ + uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_dec(key_data, context_data, out, in, len, iv, \ + aad, aad_len, auth_tag, auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_update( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, const uint64_t len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_dec_update(key_data, context_data, out, in, len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_finalize( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *auth_tag, \ + const uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_dec_finalize(key_data, context_data, auth_tag, \ + auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gmac_init( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, const void *iv, \ + const uint64_t iv_len, IMB_MGR *state) \ +{ \ + state->gmac##bitlen##_init(key_data, context_data, iv, iv_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gmac_update( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, const void *in, \ + const uint64_t len, IMB_MGR *state) \ +{ \ + state->gmac##bitlen##_update(key_data, context_data, in, len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gmac_finalize( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *auth_tag, \ + const uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gmac##bitlen##_finalize(key_data, context_data, auth_tag, \ + auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#endif + static void aesni_gcm_set_ops(struct aesni_gcm_ops *ops, IMB_MGR *mb_mgr) { + /* Set IMB_MGR pointers. */ + ops[GCM_KEY_128].mb_mgr = mb_mgr; + ops[GCM_KEY_192].mb_mgr = mb_mgr; + ops[GCM_KEY_256].mb_mgr = mb_mgr; + /* Set 128 bit function pointers. */ - ops[GCM_KEY_128].pre = mb_mgr->gcm128_pre; - ops[GCM_KEY_128].init = mb_mgr->gcm128_init; + ops[GCM_KEY_128].pre = imb_aes128_gcm_pre; + ops[GCM_KEY_128].init = imb_aes128_gcm_init; - ops[GCM_KEY_128].enc = mb_mgr->gcm128_enc; - ops[GCM_KEY_128].update_enc = mb_mgr->gcm128_enc_update; - ops[GCM_KEY_128].finalize_enc = mb_mgr->gcm128_enc_finalize; + ops[GCM_KEY_128].enc = imb_aes128_gcm_enc; + ops[GCM_KEY_128].update_enc = imb_aes128_gcm_enc_update; + ops[GCM_KEY_128].finalize_enc = imb_aes128_gcm_enc_finalize; - ops[GCM_KEY_128].dec = mb_mgr->gcm128_dec; - ops[GCM_KEY_128].update_dec = mb_mgr->gcm128_dec_update; - ops[GCM_KEY_128].finalize_dec = mb_mgr->gcm128_dec_finalize; + ops[GCM_KEY_128].dec = imb_aes128_gcm_dec; + ops[GCM_KEY_128].update_dec = imb_aes128_gcm_dec_update; + ops[GCM_KEY_128].finalize_dec = imb_aes128_gcm_dec_finalize; - ops[GCM_KEY_128].gmac_init = mb_mgr->gmac128_init; - ops[GCM_KEY_128].gmac_update = mb_mgr->gmac128_update; - ops[GCM_KEY_128].gmac_finalize = mb_mgr->gmac128_finalize; + ops[GCM_KEY_128].gmac_init = imb_aes128_gmac_init; + ops[GCM_KEY_128].gmac_update = imb_aes128_gmac_update; + ops[GCM_KEY_128].gmac_finalize = imb_aes128_gmac_finalize; /* Set 192 bit function pointers. */ - ops[GCM_KEY_192].pre = mb_mgr->gcm192_pre; - ops[GCM_KEY_192].init = mb_mgr->gcm192_init; + ops[GCM_KEY_192].pre = imb_aes192_gcm_pre; + ops[GCM_KEY_192].init = imb_aes192_gcm_init; - ops[GCM_KEY_192].enc = mb_mgr->gcm192_enc; - ops[GCM_KEY_192].update_enc = mb_mgr->gcm192_enc_update; - ops[GCM_KEY_192].finalize_enc = mb_mgr->gcm192_enc_finalize; + ops[GCM_KEY_192].enc = imb_aes192_gcm_enc; + ops[GCM_KEY_192].update_enc = imb_aes192_gcm_enc_update; + ops[GCM_KEY_192].finalize_enc = imb_aes192_gcm_enc_finalize; - ops[GCM_KEY_192].dec = mb_mgr->gcm192_dec; - ops[GCM_KEY_192].update_dec = mb_mgr->gcm192_dec_update; - ops[GCM_KEY_192].finalize_dec = mb_mgr->gcm192_dec_finalize; + ops[GCM_KEY_192].dec = imb_aes192_gcm_dec; + ops[GCM_KEY_192].update_dec = imb_aes192_gcm_dec_update; + ops[GCM_KEY_192].finalize_dec = imb_aes192_gcm_dec_finalize; - ops[GCM_KEY_192].gmac_init = mb_mgr->gmac192_init; - ops[GCM_KEY_192].gmac_update = mb_mgr->gmac192_update; - ops[GCM_KEY_192].gmac_finalize = mb_mgr->gmac192_finalize; + ops[GCM_KEY_192].gmac_init = imb_aes192_gmac_init; + ops[GCM_KEY_192].gmac_update = imb_aes192_gmac_update; + ops[GCM_KEY_192].gmac_finalize = imb_aes192_gmac_finalize; /* Set 256 bit function pointers. */ - ops[GCM_KEY_256].pre = mb_mgr->gcm256_pre; - ops[GCM_KEY_256].init = mb_mgr->gcm256_init; + ops[GCM_KEY_256].pre = imb_aes256_gcm_pre; + ops[GCM_KEY_256].init = imb_aes256_gcm_init; - ops[GCM_KEY_256].enc = mb_mgr->gcm256_enc; - ops[GCM_KEY_256].update_enc = mb_mgr->gcm256_enc_update; - ops[GCM_KEY_256].finalize_enc = mb_mgr->gcm256_enc_finalize; + ops[GCM_KEY_256].enc = imb_aes256_gcm_enc; + ops[GCM_KEY_256].update_enc = imb_aes256_gcm_enc_update; + ops[GCM_KEY_256].finalize_enc = imb_aes256_gcm_enc_finalize; - ops[GCM_KEY_256].dec = mb_mgr->gcm256_dec; - ops[GCM_KEY_256].update_dec = mb_mgr->gcm256_dec_update; - ops[GCM_KEY_256].finalize_dec = mb_mgr->gcm256_dec_finalize; + ops[GCM_KEY_256].dec = imb_aes256_gcm_dec; + ops[GCM_KEY_256].update_dec = imb_aes256_gcm_dec_update; + ops[GCM_KEY_256].finalize_dec = imb_aes256_gcm_dec_finalize; - ops[GCM_KEY_256].gmac_init = mb_mgr->gmac256_init; - ops[GCM_KEY_256].gmac_update = mb_mgr->gmac256_update; - ops[GCM_KEY_256].gmac_finalize = mb_mgr->gmac256_finalize; + ops[GCM_KEY_256].gmac_init = imb_aes256_gmac_init; + ops[GCM_KEY_256].gmac_update = imb_aes256_gmac_update; + ops[GCM_KEY_256].gmac_finalize = imb_aes256_gmac_finalize; } static int @@ -135,15 +284,15 @@ aesni_gcm_session_configure(IMB_MGR *mb_mgr, void *session, switch (key_length) { case 16: sess->key_length = GCM_KEY_128; - mb_mgr->gcm128_pre(key, &sess->gdata_key); + imb_aes128_gcm_pre(key, &sess->gdata_key, mb_mgr); break; case 24: sess->key_length = GCM_KEY_192; - mb_mgr->gcm192_pre(key, &sess->gdata_key); + imb_aes192_gcm_pre(key, &sess->gdata_key, mb_mgr); break; case 32: sess->key_length = GCM_KEY_256; - mb_mgr->gcm256_pre(key, &sess->gdata_key); + imb_aes256_gcm_pre(key, &sess->gdata_key, mb_mgr); break; default: IPSEC_MB_LOG(ERR, "Invalid key length"); @@ -324,10 +473,10 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT) { ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr, sym_op->aead.aad.data, - (uint64_t)session->aad_length); + (uint64_t)session->aad_length, ops->mb_mgr); ops->update_enc(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -345,7 +494,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->update_enc(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } @@ -355,14 +504,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, tag = sym_op->aead.digest.data; ops->finalize_enc(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } else if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT) { ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr, sym_op->aead.aad.data, - (uint64_t)session->aad_length); + (uint64_t)session->aad_length, ops->mb_mgr); ops->update_dec(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -380,19 +529,19 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->update_dec(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } tag = qp_data->temp_digest; ops->finalize_dec(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } else if (session->op == IPSEC_MB_OP_HASH_GEN_ONLY) { ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data, - iv_ptr, session->iv.length); + iv_ptr, session->iv.length, ops->mb_mgr); ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, - src, (uint64_t)part_len); + src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -406,7 +555,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, src, - (uint64_t)part_len); + (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } @@ -416,13 +565,13 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, tag = sym_op->auth.digest.data; ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } else { /* IPSEC_MB_OP_HASH_VERIFY_ONLY */ ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data, - iv_ptr, session->iv.length); + iv_ptr, session->iv.length, ops->mb_mgr); ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, - src, (uint64_t)part_len); + src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -436,14 +585,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, src, - (uint64_t)part_len); + (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } tag = qp_data->temp_digest; ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } return 0; } @@ -540,11 +689,11 @@ aesni_gcm_sgl_op_finalize_encryption(const struct aesni_gcm_session *s, uint8_t tmpdigest[s->gen_digest_length]; ops.finalize_enc(&s->gdata_key, gdata_ctx, tmpdigest, - s->gen_digest_length); + s->gen_digest_length, ops.mb_mgr); memcpy(digest, tmpdigest, s->req_digest_length); } else { ops.finalize_enc(&s->gdata_key, gdata_ctx, digest, - s->gen_digest_length); + s->gen_digest_length, ops.mb_mgr); } return 0; @@ -558,7 +707,7 @@ aesni_gcm_sgl_op_finalize_decryption(const struct aesni_gcm_session *s, uint8_t tmpdigest[s->gen_digest_length]; ops.finalize_dec(&s->gdata_key, gdata_ctx, tmpdigest, - s->gen_digest_length); + s->gen_digest_length, ops.mb_mgr); return rte_memeq_timingsafe(digest, tmpdigest, s->req_digest_length) ? 0 : EBADMSG; } @@ -573,7 +722,7 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s, /* init crypto operation */ ops.init(&s->gdata_key, gdata_ctx, iv, aad, - (uint64_t)s->aad_length); + (uint64_t)s->aad_length, ops.mb_mgr); /* update with sgl data */ for (i = 0; i < sgl->num; i++) { @@ -582,11 +731,11 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s, switch (s->op) { case IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT: ops.update_enc(&s->gdata_key, gdata_ctx, - vec->base, vec->base, vec->len); + vec->base, vec->base, vec->len, ops.mb_mgr); break; case IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT: ops.update_dec(&s->gdata_key, gdata_ctx, - vec->base, vec->base, vec->len); + vec->base, vec->base, vec->len, ops.mb_mgr); break; default: IPSEC_MB_LOG(ERR, "Invalid session op"); @@ -603,7 +752,7 @@ aesni_gcm_process_gmac_sgl_op(const struct aesni_gcm_session *s, struct aesni_gcm_ops ops) { ops.init(&s->gdata_key, gdata_ctx, iv, sgl->vec[0].base, - sgl->vec[0].len); + sgl->vec[0].len, ops.mb_mgr); } static inline uint32_t diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h index 8a5a41fb19..4074430d6e 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h +++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h @@ -79,41 +79,41 @@ enum aesni_gcm_key_length { typedef void (*aesni_gcm_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *out, const uint8_t *in, - uint64_t plaintext_len, const uint8_t *iv, - const uint8_t *aad, uint64_t aad_len, - uint8_t *auth_tag, uint64_t auth_tag_len); + void *out, const void *in, + uint64_t plaintext_len, const void *iv, + const void *aad, uint64_t aad_len, + void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state); typedef void (*aesni_gcm_pre_t)(const void *key, - struct gcm_key_data *gcm_data); + struct gcm_key_data *gcm_data, IMB_MGR *state); typedef void (*aesni_gcm_init_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - const uint8_t *iv, uint8_t const *aad, - uint64_t aad_len); + const void *iv, const void *aad, + uint64_t aad_len, IMB_MGR *state); typedef void (*aesni_gcm_update_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *out, const uint8_t *in, - uint64_t plaintext_len); + void *out, const void *in, + uint64_t plaintext_len, IMB_MGR *state); typedef void (*aesni_gcm_finalize_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *auth_tag, uint64_t auth_tag_len); + void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state); typedef void (*aesni_gmac_init_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - const uint8_t *iv, const uint64_t iv_len); + const void *iv, const uint64_t iv_len, IMB_MGR *state); typedef void (*aesni_gmac_update_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - const uint8_t *in, - const uint64_t plaintext_len); + const void *in, + const uint64_t plaintext_len, IMB_MGR *state); typedef void (*aesni_gmac_finalize_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *auth_tag, - const uint64_t auth_tag_len); + void *auth_tag, + const uint64_t auth_tag_len, IMB_MGR *state); /** GCM operation handlers */ struct aesni_gcm_ops { @@ -128,6 +128,7 @@ struct aesni_gcm_ops { aesni_gmac_init_t gmac_init; aesni_gmac_update_t gmac_update; aesni_gmac_finalize_t gmac_finalize; + IMB_MGR *mb_mgr; }; RTE_DEFINE_PER_LCORE(struct aesni_gcm_ops, gcm_ops[GCM_NUM_KEY_TYPES]); diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c index 02bacd020b..e8f6e77d21 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c @@ -161,20 +161,22 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, if (xform->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) { if (xform->auth.key.length == 16) { - sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3_BITLEN; + sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3; if (sess->auth.req_digest_len != 4) { IPSEC_MB_LOG(ERR, "Invalid digest size"); return -EINVAL; } +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM } else if (xform->auth.key.length == 32) { - sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3_BITLEN; + sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3; if (sess->auth.req_digest_len != 4 && sess->auth.req_digest_len != 8 && sess->auth.req_digest_len != 16) { IPSEC_MB_LOG(ERR, "Invalid digest size"); return -EINVAL; } +#endif } else { IPSEC_MB_LOG(ERR, "Invalid authentication key length"); return -EINVAL; @@ -187,10 +189,10 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, sess->template_job.u.ZUC_EIA3._key = sess->auth.zuc_auth_key; return 0; } else if (xform->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) { - sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2_BITLEN; + sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2; uint16_t snow3g_uia2_digest_len = get_truncated_digest_byte_length( - IMB_AUTH_SNOW3G_UIA2_BITLEN); + IMB_AUTH_SNOW3G_UIA2); if (sess->auth.req_digest_len != snow3g_uia2_digest_len) { IPSEC_MB_LOG(ERR, "Invalid digest size"); return -EINVAL; @@ -339,7 +341,7 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, /** Set session cipher parameters */ static int -aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr, +aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr, struct aesni_mb_session *sess, const struct rte_crypto_sym_xform *xform) { @@ -407,11 +409,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr, is_zuc = 1; break; case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: - sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2_BITLEN; + sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2; is_snow3g = 1; break; case RTE_CRYPTO_CIPHER_KASUMI_F8: - sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1_BITLEN; + sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1; is_kasumi = 1; break; case RTE_CRYPTO_CIPHER_NULL: @@ -536,8 +538,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr, sess->template_job.dec_keys = sess->cipher.exp_3des_keys.ks_ptr; sess->template_job.key_len_in_bytes = 24; } else if (is_zuc) { - if (xform->cipher.key.length != 16 && - xform->cipher.key.length != 32) { + if (xform->cipher.key.length != 16 +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + && xform->cipher.key.length != 32 +#endif + ) { IPSEC_MB_LOG(ERR, "Invalid cipher key length"); return -EINVAL; } @@ -1177,19 +1182,23 @@ sgl_linear_cipher_auth_len(IMB_JOB *job, uint64_t *auth_len) { uint64_t cipher_len; - if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || - job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1) cipher_len = (job->msg_len_to_cipher_in_bits >> 3) + (job->cipher_start_src_offset_in_bits >> 3); else +#endif cipher_len = job->msg_len_to_cipher_in_bytes + job->cipher_start_src_offset_in_bytes; - if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || - job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN) +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2 || + job->hash_alg == IMB_AUTH_ZUC_EIA3) *auth_len = (job->msg_len_to_hash_in_bits >> 3) + job->hash_start_src_offset_in_bytes; else +#endif *auth_len = job->msg_len_to_hash_in_bytes + job->hash_start_src_offset_in_bytes; @@ -1457,18 +1466,20 @@ aesni_mb_digest_appended_in_src(struct rte_crypto_op *op, IMB_JOB *job, return NULL; if (job->cipher_mode == IMB_CIPHER_ZUC_EEA3 || - job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || - job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) { + job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1) { cipher_size = (op->sym->cipher.data.offset >> 3) + (op->sym->cipher.data.length >> 3); } else { cipher_size = (op->sym->cipher.data.offset) + (op->sym->cipher.data.length); } - if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN || - job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || - job->hash_alg == IMB_AUTH_KASUMI_UIA1 || - job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) { + if (job->hash_alg == IMB_AUTH_ZUC_EIA3 || + job->hash_alg == IMB_AUTH_SNOW3G_UIA2 || +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + job->hash_alg == IMB_AUTH_ZUC256_EIA3 || +#endif + job->hash_alg == IMB_AUTH_KASUMI_UIA1) { auth_size = (op->sym->auth.data.offset >> 3) + (op->sym->auth.data.length >> 3); } else { @@ -1585,12 +1596,14 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->u.GMAC._iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->auth_iv.offset); break; - case IMB_AUTH_ZUC_EIA3_BITLEN: - case IMB_AUTH_ZUC256_EIA3_BITLEN: + case IMB_AUTH_ZUC_EIA3: +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + case IMB_AUTH_ZUC256_EIA3: +#endif job->u.ZUC_EIA3._iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->auth_iv.offset); break; - case IMB_AUTH_SNOW3G_UIA2_BITLEN: + case IMB_AUTH_SNOW3G_UIA2: job->u.SNOW3G_UIA2._iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->auth_iv.offset); @@ -1616,12 +1629,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, else m_offset = op->sym->cipher.data.offset; +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM if (cipher_mode == IMB_CIPHER_ZUC_EEA3) m_offset >>= 3; - else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN) + else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2) m_offset = 0; - else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) + else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1) m_offset = 0; +#else + if (cipher_mode == IMB_CIPHER_ZUC_EEA3 || + cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + cipher_mode == IMB_CIPHER_KASUMI_UEA1) + m_offset >>= 3; +#endif /* Set digest output location */ if (job->hash_alg != IMB_AUTH_NULL && @@ -1701,9 +1721,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, session->iv.offset); break; /* ZUC and SNOW3G require length in bits and offset in bytes */ - case IMB_AUTH_ZUC_EIA3_BITLEN: - case IMB_AUTH_ZUC256_EIA3_BITLEN: - case IMB_AUTH_SNOW3G_UIA2_BITLEN: + case IMB_AUTH_ZUC_EIA3: +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + case IMB_AUTH_ZUC256_EIA3: +#endif + case IMB_AUTH_SNOW3G_UIA2: auth_off_in_bytes = op->sym->auth.data.offset >> 3; ciph_off_in_bytes = op->sym->cipher.data.offset >> 3; auth_len_in_bytes = op->sym->auth.data.length >> 3; @@ -1714,7 +1736,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, session, oop, auth_off_in_bytes, ciph_off_in_bytes, auth_len_in_bytes, ciph_len_in_bytes, lb_sgl); +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM job->msg_len_to_hash_in_bits = op->sym->auth.data.length; +#else + job->msg_len_to_hash_in_bytes = op->sym->auth.data.length >> 3; +#endif job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); @@ -1772,12 +1798,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, op->sym->cipher.data.length >> 3; break; /* ZUC and SNOW3G require length and offset in bits */ - case IMB_CIPHER_SNOW3G_UEA2_BITLEN: - case IMB_CIPHER_KASUMI_UEA1_BITLEN: + case IMB_CIPHER_SNOW3G_UEA2: + case IMB_CIPHER_KASUMI_UEA1: +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM job->cipher_start_src_offset_in_bits = op->sym->cipher.data.offset; job->msg_len_to_cipher_in_bits = op->sym->cipher.data.length; +#else + job->cipher_start_src_offset_in_bytes = + op->sym->cipher.data.offset >> 3; + job->msg_len_to_cipher_in_bytes = + op->sym->cipher.data.length >> 3; +#endif break; case IMB_CIPHER_GCM: job->cipher_start_src_offset_in_bytes = @@ -2047,8 +2080,8 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) aesni_mb_digest_appended_in_src(op, job, oop) != NULL) { unsigned int auth_size, cipher_size; int unencrypted_bytes = 0; - if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || - job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN || + if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1 || job->cipher_mode == IMB_CIPHER_ZUC_EEA3) { cipher_size = (op->sym->cipher.data.offset >> 3) + (op->sym->cipher.data.length >> 3); @@ -2056,10 +2089,12 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) cipher_size = (op->sym->cipher.data.offset) + (op->sym->cipher.data.length); } - if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN || - job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || - job->hash_alg == IMB_AUTH_KASUMI_UIA1 || - job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) { + if (job->hash_alg == IMB_AUTH_ZUC_EIA3 || + job->hash_alg == IMB_AUTH_SNOW3G_UIA2 || +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + job->hash_alg == IMB_AUTH_ZUC256_EIA3 || +#endif + job->hash_alg == IMB_AUTH_KASUMI_UIA1) { auth_size = (op->sym->auth.data.offset >> 3) + (op->sym->auth.data.length >> 3); } else { @@ -2550,7 +2585,9 @@ RTE_INIT(ipsec_mb_register_aesni_mb) RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO | +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | +#endif RTE_CRYPTODEV_FF_SYM_SESSIONLESS | RTE_CRYPTODEV_FF_IN_PLACE_SGL | RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h index ad38a1e28a..6afda45414 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h @@ -19,6 +19,15 @@ #define MAX_NUM_SEGS 16 +/* bit len removed in 3.0 */ +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM +#define IMB_CIPHER_SNOW3G_UEA2 IMB_CIPHER_SNOW3G_UEA2_BITLEN +#define IMB_CIPHER_KASUMI_UEA1 IMB_CIPHER_KASUMI_UEA1_BITLEN +#define IMB_AUTH_ZUC_EIA3 IMB_AUTH_ZUC_EIA3_BITLEN +#define IMB_AUTH_SNOW3G_UIA2 IMB_AUTH_SNOW3G_UIA2_BITLEN +#define IMB_AUTH_ZUC256_EIA3 IMB_AUTH_ZUC256_EIA3_BITLEN +#endif + int aesni_mb_session_configure(IMB_MGR * m __rte_unused, void *priv_sess, const struct rte_crypto_sym_xform *xform); @@ -573,8 +582,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { .block_size = 16, .key_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 32, .increment = 16 +#else + .max = 16, + .increment = 0 +#endif }, .digest_size = { .min = 4, @@ -583,8 +597,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { }, .iv_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 25, .increment = 9 +#else + .max = 16, + .increment = 0 +#endif } }, } }, } @@ -598,13 +617,23 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { .block_size = 16, .key_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 32, .increment = 16 +#else + .max = 16, + .increment = 0 +#endif }, .iv_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 25, .increment = 9 +#else + .max = 16, + .increment = 0 +#endif }, }, } }, } @@ -888,9 +917,9 @@ static const unsigned int auth_blocksize[] = { [IMB_AUTH_SHA_256] = 64, [IMB_AUTH_SHA_384] = 128, [IMB_AUTH_SHA_512] = 128, - [IMB_AUTH_ZUC_EIA3_BITLEN] = 16, - [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 16, - [IMB_AUTH_KASUMI_UIA1] = 16 + [IMB_AUTH_ZUC_EIA3] = 16, + [IMB_AUTH_SNOW3G_UIA2] = 16, + [IMB_AUTH_KASUMI_UIA1] = 16 }; /** @@ -922,9 +951,9 @@ static const unsigned int auth_truncated_digest_byte_lengths[] = { [IMB_AUTH_SHA_256] = 32, [IMB_AUTH_SHA_384] = 48, [IMB_AUTH_SHA_512] = 64, - [IMB_AUTH_ZUC_EIA3_BITLEN] = 4, - [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4, - [IMB_AUTH_KASUMI_UIA1] = 4 + [IMB_AUTH_ZUC_EIA3] = 4, + [IMB_AUTH_SNOW3G_UIA2] = 4, + [IMB_AUTH_KASUMI_UIA1] = 4 }; /** @@ -957,9 +986,9 @@ static const unsigned int auth_digest_byte_lengths[] = { [IMB_AUTH_SHA_256] = 32, [IMB_AUTH_SHA_384] = 48, [IMB_AUTH_SHA_512] = 64, - [IMB_AUTH_ZUC_EIA3_BITLEN] = 4, - [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4, - [IMB_AUTH_KASUMI_UIA1] = 4, + [IMB_AUTH_ZUC_EIA3] = 4, + [IMB_AUTH_SNOW3G_UIA2] = 4, + [IMB_AUTH_KASUMI_UIA1] = 4, [IMB_AUTH_SM3] = 32, [IMB_AUTH_HMAC_SM3] = 32, /**< Vector mode dependent pointer table of the multi-buffer APIs */ -- 2.52.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/4] crypto/ipsec_mb: add support for IPsecMB 3.0 2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau @ 2026-10-06 12:06 ` Ji, Kai 0 siblings, 0 replies; 7+ messages in thread From: Ji, Kai @ 2026-10-06 12:06 UTC (permalink / raw) To: Nicolau, Radu, dev@dpdk.org; +Cc: De Lara Guarch, Pablo [-- Attachment #1: Type: text/plain, Size: 45720 bytes --] Series-acked-by: Kai Ji <kai.ji@intel.com> ________________________________ From: Nicolau, Radu <radu.nicolau@intel.com> Sent: 02 October 2026 10:56 To: dev@dpdk.org <dev@dpdk.org> Cc: Nicolau, Radu <radu.nicolau@intel.com>; Ji, Kai <kai.ji@intel.com>; De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com> Subject: [PATCH 1/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Add support for IPsecMB release 3.0 - removed all _BITLEN enums, all lengths are in bytes - removed feature flag - use non const IMB_MGR - use IMB API instead of function pointers in the GCM PMD - remove ZUC 256 support Signed-off-by: Radu Nicolau <radu.nicolau@intel.com> --- doc/guides/cryptodevs/aesni_gcm.rst | 4 +- doc/guides/cryptodevs/aesni_mb.rst | 11 +- doc/guides/rel_notes/release_26_11.rst | 4 + drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++++++++++++---- drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 107 +++++--- drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 47 +++- 7 files changed, 349 insertions(+), 122 deletions(-) diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst index aefc1cedcb..e68dc4eb1e 100644 --- a/doc/guides/cryptodevs/aesni_gcm.rst +++ b/doc/guides/cryptodevs/aesni_gcm.rst @@ -40,8 +40,8 @@ Installation To build DPDK with the AESNI_GCM_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v2.0, which -can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_. +The latest version of the library supported by this PMD is v3.0, which +can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_. For Arm system, ARM64 port of the multi-buffer library can be downloaded from https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/. diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index add3c03b74..4851997fd9 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -73,6 +73,13 @@ Protocol offloads: * RTE_SECURITY_PROTOCOL_DOCSIS +Note +---- +Version 3.0 of the multi-buffer library removed support for: + +* the 256 bit versions of ZUC EEA3 and ZUC EIA3 algorithms. +* non byte aligned bit length (``RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA`` removed). + Limitations ----------- @@ -96,8 +103,8 @@ Installation To build DPDK with the AESNI_MB_PMD the user is required to download the multi-buffer library from `here <https://github.com/01org/intel-ipsec-mb>`_ and compile it on their user system before building DPDK. -The latest version of the library supported by this PMD is v2.0, which -can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_. +The latest version of the library supported by this PMD is v3.0, which +can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_. For Arm system, ARM64 port of the multi-buffer library can be downloaded from https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/. diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index 69ab5af8a3..ea126aa9bb 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -33,6 +33,10 @@ New Features ``pss_salt`` field in ``rte_crypto_rsa_op_param`` to carry it per sign operation. +* **Updated AESNI_MB crypto driver.** + + * Added support for version 3.0 of IPsec MB Library. + .. This section should contain new features added in this release. Sample format: diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c index 3c266cc47e..0c4358a7b1 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c @@ -6,56 +6,205 @@ #include "pmd_aesni_gcm_priv.h" +/* wrapper functions for library versions < 3.0.0 */ +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_pre(const void *key, \ + struct gcm_key_data *key_data, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_pre(key, key_data); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_init( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, const void *iv, \ + const void *aad, uint64_t aad_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_init(key_data, context_data, iv, aad, aad_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, uint64_t len, const void *iv, \ + const void *aad, uint64_t aad_len, void *auth_tag, \ + uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_enc(key_data, context_data, out, in, len, iv, \ + aad, aad_len, auth_tag, auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_update( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, const uint64_t len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_enc_update(key_data, context_data, out, in, len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_finalize( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *auth_tag, \ + const uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_enc_finalize(key_data, context_data, auth_tag, \ + auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, uint64_t len, const void *iv, \ + const void *aad, uint64_t aad_len, void *auth_tag, \ + uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_dec(key_data, context_data, out, in, len, iv, \ + aad, aad_len, auth_tag, auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_update( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *out, \ + const void *in, const uint64_t len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_dec_update(key_data, context_data, out, in, len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_finalize( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *auth_tag, \ + const uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gcm##bitlen##_dec_finalize(key_data, context_data, auth_tag, \ + auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gmac_init( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, const void *iv, \ + const uint64_t iv_len, IMB_MGR *state) \ +{ \ + state->gmac##bitlen##_init(key_data, context_data, iv, iv_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gmac_update( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, const void *in, \ + const uint64_t len, IMB_MGR *state) \ +{ \ + state->gmac##bitlen##_update(key_data, context_data, in, len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#define _fn(bitlen) static void imb_aes##bitlen##_gmac_finalize( \ + const struct gcm_key_data *key_data, \ + struct gcm_context_data *context_data, void *auth_tag, \ + const uint64_t auth_tag_len, IMB_MGR *state) \ +{ \ + state->gmac##bitlen##_finalize(key_data, context_data, auth_tag, \ + auth_tag_len); \ +} +_fn(128) +_fn(192) +_fn(256) +#undef _fn + +#endif + static void aesni_gcm_set_ops(struct aesni_gcm_ops *ops, IMB_MGR *mb_mgr) { + /* Set IMB_MGR pointers. */ + ops[GCM_KEY_128].mb_mgr = mb_mgr; + ops[GCM_KEY_192].mb_mgr = mb_mgr; + ops[GCM_KEY_256].mb_mgr = mb_mgr; + /* Set 128 bit function pointers. */ - ops[GCM_KEY_128].pre = mb_mgr->gcm128_pre; - ops[GCM_KEY_128].init = mb_mgr->gcm128_init; + ops[GCM_KEY_128].pre = imb_aes128_gcm_pre; + ops[GCM_KEY_128].init = imb_aes128_gcm_init; - ops[GCM_KEY_128].enc = mb_mgr->gcm128_enc; - ops[GCM_KEY_128].update_enc = mb_mgr->gcm128_enc_update; - ops[GCM_KEY_128].finalize_enc = mb_mgr->gcm128_enc_finalize; + ops[GCM_KEY_128].enc = imb_aes128_gcm_enc; + ops[GCM_KEY_128].update_enc = imb_aes128_gcm_enc_update; + ops[GCM_KEY_128].finalize_enc = imb_aes128_gcm_enc_finalize; - ops[GCM_KEY_128].dec = mb_mgr->gcm128_dec; - ops[GCM_KEY_128].update_dec = mb_mgr->gcm128_dec_update; - ops[GCM_KEY_128].finalize_dec = mb_mgr->gcm128_dec_finalize; + ops[GCM_KEY_128].dec = imb_aes128_gcm_dec; + ops[GCM_KEY_128].update_dec = imb_aes128_gcm_dec_update; + ops[GCM_KEY_128].finalize_dec = imb_aes128_gcm_dec_finalize; - ops[GCM_KEY_128].gmac_init = mb_mgr->gmac128_init; - ops[GCM_KEY_128].gmac_update = mb_mgr->gmac128_update; - ops[GCM_KEY_128].gmac_finalize = mb_mgr->gmac128_finalize; + ops[GCM_KEY_128].gmac_init = imb_aes128_gmac_init; + ops[GCM_KEY_128].gmac_update = imb_aes128_gmac_update; + ops[GCM_KEY_128].gmac_finalize = imb_aes128_gmac_finalize; /* Set 192 bit function pointers. */ - ops[GCM_KEY_192].pre = mb_mgr->gcm192_pre; - ops[GCM_KEY_192].init = mb_mgr->gcm192_init; + ops[GCM_KEY_192].pre = imb_aes192_gcm_pre; + ops[GCM_KEY_192].init = imb_aes192_gcm_init; - ops[GCM_KEY_192].enc = mb_mgr->gcm192_enc; - ops[GCM_KEY_192].update_enc = mb_mgr->gcm192_enc_update; - ops[GCM_KEY_192].finalize_enc = mb_mgr->gcm192_enc_finalize; + ops[GCM_KEY_192].enc = imb_aes192_gcm_enc; + ops[GCM_KEY_192].update_enc = imb_aes192_gcm_enc_update; + ops[GCM_KEY_192].finalize_enc = imb_aes192_gcm_enc_finalize; - ops[GCM_KEY_192].dec = mb_mgr->gcm192_dec; - ops[GCM_KEY_192].update_dec = mb_mgr->gcm192_dec_update; - ops[GCM_KEY_192].finalize_dec = mb_mgr->gcm192_dec_finalize; + ops[GCM_KEY_192].dec = imb_aes192_gcm_dec; + ops[GCM_KEY_192].update_dec = imb_aes192_gcm_dec_update; + ops[GCM_KEY_192].finalize_dec = imb_aes192_gcm_dec_finalize; - ops[GCM_KEY_192].gmac_init = mb_mgr->gmac192_init; - ops[GCM_KEY_192].gmac_update = mb_mgr->gmac192_update; - ops[GCM_KEY_192].gmac_finalize = mb_mgr->gmac192_finalize; + ops[GCM_KEY_192].gmac_init = imb_aes192_gmac_init; + ops[GCM_KEY_192].gmac_update = imb_aes192_gmac_update; + ops[GCM_KEY_192].gmac_finalize = imb_aes192_gmac_finalize; /* Set 256 bit function pointers. */ - ops[GCM_KEY_256].pre = mb_mgr->gcm256_pre; - ops[GCM_KEY_256].init = mb_mgr->gcm256_init; + ops[GCM_KEY_256].pre = imb_aes256_gcm_pre; + ops[GCM_KEY_256].init = imb_aes256_gcm_init; - ops[GCM_KEY_256].enc = mb_mgr->gcm256_enc; - ops[GCM_KEY_256].update_enc = mb_mgr->gcm256_enc_update; - ops[GCM_KEY_256].finalize_enc = mb_mgr->gcm256_enc_finalize; + ops[GCM_KEY_256].enc = imb_aes256_gcm_enc; + ops[GCM_KEY_256].update_enc = imb_aes256_gcm_enc_update; + ops[GCM_KEY_256].finalize_enc = imb_aes256_gcm_enc_finalize; - ops[GCM_KEY_256].dec = mb_mgr->gcm256_dec; - ops[GCM_KEY_256].update_dec = mb_mgr->gcm256_dec_update; - ops[GCM_KEY_256].finalize_dec = mb_mgr->gcm256_dec_finalize; + ops[GCM_KEY_256].dec = imb_aes256_gcm_dec; + ops[GCM_KEY_256].update_dec = imb_aes256_gcm_dec_update; + ops[GCM_KEY_256].finalize_dec = imb_aes256_gcm_dec_finalize; - ops[GCM_KEY_256].gmac_init = mb_mgr->gmac256_init; - ops[GCM_KEY_256].gmac_update = mb_mgr->gmac256_update; - ops[GCM_KEY_256].gmac_finalize = mb_mgr->gmac256_finalize; + ops[GCM_KEY_256].gmac_init = imb_aes256_gmac_init; + ops[GCM_KEY_256].gmac_update = imb_aes256_gmac_update; + ops[GCM_KEY_256].gmac_finalize = imb_aes256_gmac_finalize; } static int @@ -135,15 +284,15 @@ aesni_gcm_session_configure(IMB_MGR *mb_mgr, void *session, switch (key_length) { case 16: sess->key_length = GCM_KEY_128; - mb_mgr->gcm128_pre(key, &sess->gdata_key); + imb_aes128_gcm_pre(key, &sess->gdata_key, mb_mgr); break; case 24: sess->key_length = GCM_KEY_192; - mb_mgr->gcm192_pre(key, &sess->gdata_key); + imb_aes192_gcm_pre(key, &sess->gdata_key, mb_mgr); break; case 32: sess->key_length = GCM_KEY_256; - mb_mgr->gcm256_pre(key, &sess->gdata_key); + imb_aes256_gcm_pre(key, &sess->gdata_key, mb_mgr); break; default: IPSEC_MB_LOG(ERR, "Invalid key length"); @@ -324,10 +473,10 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT) { ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr, sym_op->aead.aad.data, - (uint64_t)session->aad_length); + (uint64_t)session->aad_length, ops->mb_mgr); ops->update_enc(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -345,7 +494,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->update_enc(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } @@ -355,14 +504,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, tag = sym_op->aead.digest.data; ops->finalize_enc(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } else if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT) { ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr, sym_op->aead.aad.data, - (uint64_t)session->aad_length); + (uint64_t)session->aad_length, ops->mb_mgr); ops->update_dec(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -380,19 +529,19 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->update_dec(&session->gdata_key, &qp_data->gcm_ctx_data, - dst, src, (uint64_t)part_len); + dst, src, (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } tag = qp_data->temp_digest; ops->finalize_dec(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } else if (session->op == IPSEC_MB_OP_HASH_GEN_ONLY) { ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data, - iv_ptr, session->iv.length); + iv_ptr, session->iv.length, ops->mb_mgr); ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, - src, (uint64_t)part_len); + src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -406,7 +555,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, src, - (uint64_t)part_len); + (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } @@ -416,13 +565,13 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, tag = sym_op->auth.digest.data; ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } else { /* IPSEC_MB_OP_HASH_VERIFY_ONLY */ ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data, - iv_ptr, session->iv.length); + iv_ptr, session->iv.length, ops->mb_mgr); ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, - src, (uint64_t)part_len); + src, (uint64_t)part_len, ops->mb_mgr); total_len = data_length - part_len; while (total_len) { @@ -436,14 +585,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op, ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data, src, - (uint64_t)part_len); + (uint64_t)part_len, ops->mb_mgr); total_len -= part_len; } tag = qp_data->temp_digest; ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data, - tag, session->gen_digest_length); + tag, session->gen_digest_length, ops->mb_mgr); } return 0; } @@ -540,11 +689,11 @@ aesni_gcm_sgl_op_finalize_encryption(const struct aesni_gcm_session *s, uint8_t tmpdigest[s->gen_digest_length]; ops.finalize_enc(&s->gdata_key, gdata_ctx, tmpdigest, - s->gen_digest_length); + s->gen_digest_length, ops.mb_mgr); memcpy(digest, tmpdigest, s->req_digest_length); } else { ops.finalize_enc(&s->gdata_key, gdata_ctx, digest, - s->gen_digest_length); + s->gen_digest_length, ops.mb_mgr); } return 0; @@ -558,7 +707,7 @@ aesni_gcm_sgl_op_finalize_decryption(const struct aesni_gcm_session *s, uint8_t tmpdigest[s->gen_digest_length]; ops.finalize_dec(&s->gdata_key, gdata_ctx, tmpdigest, - s->gen_digest_length); + s->gen_digest_length, ops.mb_mgr); return rte_memeq_timingsafe(digest, tmpdigest, s->req_digest_length) ? 0 : EBADMSG; } @@ -573,7 +722,7 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s, /* init crypto operation */ ops.init(&s->gdata_key, gdata_ctx, iv, aad, - (uint64_t)s->aad_length); + (uint64_t)s->aad_length, ops.mb_mgr); /* update with sgl data */ for (i = 0; i < sgl->num; i++) { @@ -582,11 +731,11 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s, switch (s->op) { case IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT: ops.update_enc(&s->gdata_key, gdata_ctx, - vec->base, vec->base, vec->len); + vec->base, vec->base, vec->len, ops.mb_mgr); break; case IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT: ops.update_dec(&s->gdata_key, gdata_ctx, - vec->base, vec->base, vec->len); + vec->base, vec->base, vec->len, ops.mb_mgr); break; default: IPSEC_MB_LOG(ERR, "Invalid session op"); @@ -603,7 +752,7 @@ aesni_gcm_process_gmac_sgl_op(const struct aesni_gcm_session *s, struct aesni_gcm_ops ops) { ops.init(&s->gdata_key, gdata_ctx, iv, sgl->vec[0].base, - sgl->vec[0].len); + sgl->vec[0].len, ops.mb_mgr); } static inline uint32_t diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h index 8a5a41fb19..4074430d6e 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h +++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h @@ -79,41 +79,41 @@ enum aesni_gcm_key_length { typedef void (*aesni_gcm_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *out, const uint8_t *in, - uint64_t plaintext_len, const uint8_t *iv, - const uint8_t *aad, uint64_t aad_len, - uint8_t *auth_tag, uint64_t auth_tag_len); + void *out, const void *in, + uint64_t plaintext_len, const void *iv, + const void *aad, uint64_t aad_len, + void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state); typedef void (*aesni_gcm_pre_t)(const void *key, - struct gcm_key_data *gcm_data); + struct gcm_key_data *gcm_data, IMB_MGR *state); typedef void (*aesni_gcm_init_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - const uint8_t *iv, uint8_t const *aad, - uint64_t aad_len); + const void *iv, const void *aad, + uint64_t aad_len, IMB_MGR *state); typedef void (*aesni_gcm_update_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *out, const uint8_t *in, - uint64_t plaintext_len); + void *out, const void *in, + uint64_t plaintext_len, IMB_MGR *state); typedef void (*aesni_gcm_finalize_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *auth_tag, uint64_t auth_tag_len); + void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state); typedef void (*aesni_gmac_init_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - const uint8_t *iv, const uint64_t iv_len); + const void *iv, const uint64_t iv_len, IMB_MGR *state); typedef void (*aesni_gmac_update_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - const uint8_t *in, - const uint64_t plaintext_len); + const void *in, + const uint64_t plaintext_len, IMB_MGR *state); typedef void (*aesni_gmac_finalize_t)(const struct gcm_key_data *gcm_key_data, struct gcm_context_data *gcm_ctx_data, - uint8_t *auth_tag, - const uint64_t auth_tag_len); + void *auth_tag, + const uint64_t auth_tag_len, IMB_MGR *state); /** GCM operation handlers */ struct aesni_gcm_ops { @@ -128,6 +128,7 @@ struct aesni_gcm_ops { aesni_gmac_init_t gmac_init; aesni_gmac_update_t gmac_update; aesni_gmac_finalize_t gmac_finalize; + IMB_MGR *mb_mgr; }; RTE_DEFINE_PER_LCORE(struct aesni_gcm_ops, gcm_ops[GCM_NUM_KEY_TYPES]); diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c index 02bacd020b..e8f6e77d21 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c @@ -161,20 +161,22 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, if (xform->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) { if (xform->auth.key.length == 16) { - sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3_BITLEN; + sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3; if (sess->auth.req_digest_len != 4) { IPSEC_MB_LOG(ERR, "Invalid digest size"); return -EINVAL; } +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM } else if (xform->auth.key.length == 32) { - sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3_BITLEN; + sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3; if (sess->auth.req_digest_len != 4 && sess->auth.req_digest_len != 8 && sess->auth.req_digest_len != 16) { IPSEC_MB_LOG(ERR, "Invalid digest size"); return -EINVAL; } +#endif } else { IPSEC_MB_LOG(ERR, "Invalid authentication key length"); return -EINVAL; @@ -187,10 +189,10 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, sess->template_job.u.ZUC_EIA3._key = sess->auth.zuc_auth_key; return 0; } else if (xform->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) { - sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2_BITLEN; + sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2; uint16_t snow3g_uia2_digest_len = get_truncated_digest_byte_length( - IMB_AUTH_SNOW3G_UIA2_BITLEN); + IMB_AUTH_SNOW3G_UIA2); if (sess->auth.req_digest_len != snow3g_uia2_digest_len) { IPSEC_MB_LOG(ERR, "Invalid digest size"); return -EINVAL; @@ -339,7 +341,7 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, /** Set session cipher parameters */ static int -aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr, +aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr, struct aesni_mb_session *sess, const struct rte_crypto_sym_xform *xform) { @@ -407,11 +409,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr, is_zuc = 1; break; case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: - sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2_BITLEN; + sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2; is_snow3g = 1; break; case RTE_CRYPTO_CIPHER_KASUMI_F8: - sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1_BITLEN; + sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1; is_kasumi = 1; break; case RTE_CRYPTO_CIPHER_NULL: @@ -536,8 +538,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr, sess->template_job.dec_keys = sess->cipher.exp_3des_keys.ks_ptr; sess->template_job.key_len_in_bytes = 24; } else if (is_zuc) { - if (xform->cipher.key.length != 16 && - xform->cipher.key.length != 32) { + if (xform->cipher.key.length != 16 +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + && xform->cipher.key.length != 32 +#endif + ) { IPSEC_MB_LOG(ERR, "Invalid cipher key length"); return -EINVAL; } @@ -1177,19 +1182,23 @@ sgl_linear_cipher_auth_len(IMB_JOB *job, uint64_t *auth_len) { uint64_t cipher_len; - if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || - job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1) cipher_len = (job->msg_len_to_cipher_in_bits >> 3) + (job->cipher_start_src_offset_in_bits >> 3); else +#endif cipher_len = job->msg_len_to_cipher_in_bytes + job->cipher_start_src_offset_in_bytes; - if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || - job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN) +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2 || + job->hash_alg == IMB_AUTH_ZUC_EIA3) *auth_len = (job->msg_len_to_hash_in_bits >> 3) + job->hash_start_src_offset_in_bytes; else +#endif *auth_len = job->msg_len_to_hash_in_bytes + job->hash_start_src_offset_in_bytes; @@ -1457,18 +1466,20 @@ aesni_mb_digest_appended_in_src(struct rte_crypto_op *op, IMB_JOB *job, return NULL; if (job->cipher_mode == IMB_CIPHER_ZUC_EEA3 || - job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || - job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) { + job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1) { cipher_size = (op->sym->cipher.data.offset >> 3) + (op->sym->cipher.data.length >> 3); } else { cipher_size = (op->sym->cipher.data.offset) + (op->sym->cipher.data.length); } - if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN || - job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || - job->hash_alg == IMB_AUTH_KASUMI_UIA1 || - job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) { + if (job->hash_alg == IMB_AUTH_ZUC_EIA3 || + job->hash_alg == IMB_AUTH_SNOW3G_UIA2 || +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + job->hash_alg == IMB_AUTH_ZUC256_EIA3 || +#endif + job->hash_alg == IMB_AUTH_KASUMI_UIA1) { auth_size = (op->sym->auth.data.offset >> 3) + (op->sym->auth.data.length >> 3); } else { @@ -1585,12 +1596,14 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->u.GMAC._iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->auth_iv.offset); break; - case IMB_AUTH_ZUC_EIA3_BITLEN: - case IMB_AUTH_ZUC256_EIA3_BITLEN: + case IMB_AUTH_ZUC_EIA3: +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + case IMB_AUTH_ZUC256_EIA3: +#endif job->u.ZUC_EIA3._iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->auth_iv.offset); break; - case IMB_AUTH_SNOW3G_UIA2_BITLEN: + case IMB_AUTH_SNOW3G_UIA2: job->u.SNOW3G_UIA2._iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->auth_iv.offset); @@ -1616,12 +1629,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, else m_offset = op->sym->cipher.data.offset; +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM if (cipher_mode == IMB_CIPHER_ZUC_EEA3) m_offset >>= 3; - else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN) + else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2) m_offset = 0; - else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) + else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1) m_offset = 0; +#else + if (cipher_mode == IMB_CIPHER_ZUC_EEA3 || + cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + cipher_mode == IMB_CIPHER_KASUMI_UEA1) + m_offset >>= 3; +#endif /* Set digest output location */ if (job->hash_alg != IMB_AUTH_NULL && @@ -1701,9 +1721,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, session->iv.offset); break; /* ZUC and SNOW3G require length in bits and offset in bytes */ - case IMB_AUTH_ZUC_EIA3_BITLEN: - case IMB_AUTH_ZUC256_EIA3_BITLEN: - case IMB_AUTH_SNOW3G_UIA2_BITLEN: + case IMB_AUTH_ZUC_EIA3: +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + case IMB_AUTH_ZUC256_EIA3: +#endif + case IMB_AUTH_SNOW3G_UIA2: auth_off_in_bytes = op->sym->auth.data.offset >> 3; ciph_off_in_bytes = op->sym->cipher.data.offset >> 3; auth_len_in_bytes = op->sym->auth.data.length >> 3; @@ -1714,7 +1736,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, session, oop, auth_off_in_bytes, ciph_off_in_bytes, auth_len_in_bytes, ciph_len_in_bytes, lb_sgl); +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM job->msg_len_to_hash_in_bits = op->sym->auth.data.length; +#else + job->msg_len_to_hash_in_bytes = op->sym->auth.data.length >> 3; +#endif job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); @@ -1772,12 +1798,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, op->sym->cipher.data.length >> 3; break; /* ZUC and SNOW3G require length and offset in bits */ - case IMB_CIPHER_SNOW3G_UEA2_BITLEN: - case IMB_CIPHER_KASUMI_UEA1_BITLEN: + case IMB_CIPHER_SNOW3G_UEA2: + case IMB_CIPHER_KASUMI_UEA1: +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM job->cipher_start_src_offset_in_bits = op->sym->cipher.data.offset; job->msg_len_to_cipher_in_bits = op->sym->cipher.data.length; +#else + job->cipher_start_src_offset_in_bytes = + op->sym->cipher.data.offset >> 3; + job->msg_len_to_cipher_in_bytes = + op->sym->cipher.data.length >> 3; +#endif break; case IMB_CIPHER_GCM: job->cipher_start_src_offset_in_bytes = @@ -2047,8 +2080,8 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) aesni_mb_digest_appended_in_src(op, job, oop) != NULL) { unsigned int auth_size, cipher_size; int unencrypted_bytes = 0; - if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN || - job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN || + if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 || + job->cipher_mode == IMB_CIPHER_KASUMI_UEA1 || job->cipher_mode == IMB_CIPHER_ZUC_EEA3) { cipher_size = (op->sym->cipher.data.offset >> 3) + (op->sym->cipher.data.length >> 3); @@ -2056,10 +2089,12 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) cipher_size = (op->sym->cipher.data.offset) + (op->sym->cipher.data.length); } - if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN || - job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN || - job->hash_alg == IMB_AUTH_KASUMI_UIA1 || - job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) { + if (job->hash_alg == IMB_AUTH_ZUC_EIA3 || + job->hash_alg == IMB_AUTH_SNOW3G_UIA2 || +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + job->hash_alg == IMB_AUTH_ZUC256_EIA3 || +#endif + job->hash_alg == IMB_AUTH_KASUMI_UIA1) { auth_size = (op->sym->auth.data.offset >> 3) + (op->sym->auth.data.length >> 3); } else { @@ -2550,7 +2585,9 @@ RTE_INIT(ipsec_mb_register_aesni_mb) RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO | +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | +#endif RTE_CRYPTODEV_FF_SYM_SESSIONLESS | RTE_CRYPTODEV_FF_IN_PLACE_SGL | RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT | diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h index ad38a1e28a..6afda45414 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h @@ -19,6 +19,15 @@ #define MAX_NUM_SEGS 16 +/* bit len removed in 3.0 */ +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM +#define IMB_CIPHER_SNOW3G_UEA2 IMB_CIPHER_SNOW3G_UEA2_BITLEN +#define IMB_CIPHER_KASUMI_UEA1 IMB_CIPHER_KASUMI_UEA1_BITLEN +#define IMB_AUTH_ZUC_EIA3 IMB_AUTH_ZUC_EIA3_BITLEN +#define IMB_AUTH_SNOW3G_UIA2 IMB_AUTH_SNOW3G_UIA2_BITLEN +#define IMB_AUTH_ZUC256_EIA3 IMB_AUTH_ZUC256_EIA3_BITLEN +#endif + int aesni_mb_session_configure(IMB_MGR * m __rte_unused, void *priv_sess, const struct rte_crypto_sym_xform *xform); @@ -573,8 +582,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { .block_size = 16, .key_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 32, .increment = 16 +#else + .max = 16, + .increment = 0 +#endif }, .digest_size = { .min = 4, @@ -583,8 +597,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { }, .iv_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 25, .increment = 9 +#else + .max = 16, + .increment = 0 +#endif } }, } }, } @@ -598,13 +617,23 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { .block_size = 16, .key_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 32, .increment = 16 +#else + .max = 16, + .increment = 0 +#endif }, .iv_size = { .min = 16, +#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM .max = 25, .increment = 9 +#else + .max = 16, + .increment = 0 +#endif }, }, } }, } @@ -888,9 +917,9 @@ static const unsigned int auth_blocksize[] = { [IMB_AUTH_SHA_256] = 64, [IMB_AUTH_SHA_384] = 128, [IMB_AUTH_SHA_512] = 128, - [IMB_AUTH_ZUC_EIA3_BITLEN] = 16, - [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 16, - [IMB_AUTH_KASUMI_UIA1] = 16 + [IMB_AUTH_ZUC_EIA3] = 16, + [IMB_AUTH_SNOW3G_UIA2] = 16, + [IMB_AUTH_KASUMI_UIA1] = 16 }; /** @@ -922,9 +951,9 @@ static const unsigned int auth_truncated_digest_byte_lengths[] = { [IMB_AUTH_SHA_256] = 32, [IMB_AUTH_SHA_384] = 48, [IMB_AUTH_SHA_512] = 64, - [IMB_AUTH_ZUC_EIA3_BITLEN] = 4, - [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4, - [IMB_AUTH_KASUMI_UIA1] = 4 + [IMB_AUTH_ZUC_EIA3] = 4, + [IMB_AUTH_SNOW3G_UIA2] = 4, + [IMB_AUTH_KASUMI_UIA1] = 4 }; /** @@ -957,9 +986,9 @@ static const unsigned int auth_digest_byte_lengths[] = { [IMB_AUTH_SHA_256] = 32, [IMB_AUTH_SHA_384] = 48, [IMB_AUTH_SHA_512] = 64, - [IMB_AUTH_ZUC_EIA3_BITLEN] = 4, - [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4, - [IMB_AUTH_KASUMI_UIA1] = 4, + [IMB_AUTH_ZUC_EIA3] = 4, + [IMB_AUTH_SNOW3G_UIA2] = 4, + [IMB_AUTH_KASUMI_UIA1] = 4, [IMB_AUTH_SM3] = 32, [IMB_AUTH_HMAC_SM3] = 32, /**< Vector mode dependent pointer table of the multi-buffer APIs */ -- 2.52.0 [-- Attachment #2: Type: text/html, Size: 110255 bytes --] ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms 2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau 2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau @ 2026-10-02 9:56 ` Radu Nicolau 2026-10-02 9:56 ` [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA Radu Nicolau ` (2 subsequent siblings) 4 siblings, 0 replies; 7+ messages in thread From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw) To: dev; +Cc: Radu Nicolau, Kai Ji, Pablo de Lara Add support for NEA4, NIA4, NCA4: Snow 5G confidentiality, integrity and AEAD modes NEA5, NIA5, NCA5: AES 256 confidentiality, integrity and AEAD modes NEA6, NIA6, NCA6: ZUC 256 confidentiality, integrity and AEAD modes IPsec MB library version 3.0 is required for the above algorithms. Signed-off-by: Radu Nicolau <radu.nicolau@intel.com> --- doc/guides/cryptodevs/aesni_mb.rst | 9 + doc/guides/cryptodevs/features/aesni_mb.ini | 9 + doc/guides/rel_notes/release_26_11.rst | 5 + drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 216 +++++++++++++++++- drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 229 ++++++++++++++++++++ 5 files changed, 467 insertions(+), 1 deletion(-) diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index 4851997fd9..5fb73e31b4 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -39,6 +39,9 @@ Cipher algorithms: * RTE_CRYPTO_CIPHER_SM4_CBC * RTE_CRYPTO_CIPHER_SM4_ECB * RTE_CRYPTO_CIPHER_SM4_CTR +* RTE_CRYPTO_CIPHER_SNOW5G_NEA4 +* RTE_CRYPTO_CIPHER_AES_NEA5 +* RTE_CRYPTO_CIPHER_ZUC_NEA6 Hash algorithms: @@ -61,6 +64,9 @@ Hash algorithms: * RTE_CRYPTO_AUTH_KASUMI_F9 * RTE_CRYPTO_AUTH_SM3 * RTE_CRYPTO_AUTH_SM3 HMAC +* RTE_CRYPTO_AUTH_SNOW5G_NIA4 +* RTE_CRYPTO_AUTH_AES_NIA5 +* RTE_CRYPTO_AUTH_ZUC_NIA6 AEAD algorithms: @@ -68,6 +74,9 @@ AEAD algorithms: * RTE_CRYPTO_AEAD_AES_GCM * RTE_CRYPTO_AEAD_CHACHA20_POLY1305 * RTE_CRYPTO_AEAD_SM4_GCM +* RTE_CRYPTO_AEAD_SNOW5G_NCA4 +* RTE_CRYPTO_AEAD_AES_NCA5 +* RTE_CRYPTO_AEAD_ZUC_NCA6 Protocol offloads: diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini index ce6b43e48b..e64818bf46 100644 --- a/doc/guides/cryptodevs/features/aesni_mb.ini +++ b/doc/guides/cryptodevs/features/aesni_mb.ini @@ -45,6 +45,9 @@ KASUMI F8 = Y SM4 CBC = Y SM4 ECB = Y SM4 CTR = Y +SNOW5G NEA4 = Y +AES NEA5 = Y +ZUC-256 NEA6 = Y ; ; Supported authentication algorithms of the 'aesni_mb' crypto driver. @@ -69,6 +72,9 @@ SNOW3G UIA2 = Y KASUMI F9 = Y SM3 = Y SM3 HMAC = Y +SNOW5G NIA4 = Y +AES NIA5 = Y +ZUC-256 NIA6 = Y ; ; Supported AEAD algorithms of the 'aesni_mb' crypto driver. @@ -81,6 +87,9 @@ AES GCM (192) = Y AES GCM (256) = Y CHACHA20-POLY1305 = Y SM4 GCM = Y +SNOW5G NCA4 = Y +AES NCA5 = Y +ZUC-256 NCA6 = Y ; ; Supported Asymmetric algorithms of the 'aesni_mb' crypto driver. diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index ea126aa9bb..1ee3cdfba3 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -36,8 +36,13 @@ New Features * **Updated AESNI_MB crypto driver.** * Added support for version 3.0 of IPsec MB Library. + * Added support for the following wireless algorithms: + - NEA4, NIA4, NCA4: Snow 5G confidentiality, integrity and AEAD modes. + - NEA5, NIA5, NCA5: AES 256 confidentiality, integrity and AEAD modes. + - NEA6, NIA6, NCA6: ZUC 256 confidentiality, integrity and AEAD modes. .. This section should contain new features added in this release. + Sample format: * **Add a title in the past tense with a full stop.** diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c index e8f6e77d21..a49a82d561 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c @@ -22,6 +22,11 @@ is_aead_algo(IMB_HASH_ALG hash_alg, IMB_CIPHER_MODE cipher_mode) hash_alg == IMB_AUTH_AES_CCM || cipher_mode == IMB_CIPHER_GCM || cipher_mode == IMB_CIPHER_SM4_GCM +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + || cipher_mode == IMB_CIPHER_SNOW5G_NCA4 + || cipher_mode == IMB_CIPHER_AES_NCA5 + || cipher_mode == IMB_CIPHER_ZUC_NCA6 +#endif ); } @@ -221,6 +226,36 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr, return 0; } +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + if (xform->auth.algo == RTE_CRYPTO_AUTH_SNOW5G_NIA4) { + sess->template_job.hash_alg = IMB_AUTH_SNOW5G_NIA4; + sess->template_job.auth_tag_output_len_in_bytes = + sess->auth.req_digest_len; + memcpy(sess->auth.snow5g_auth_key, xform->auth.key.data, + xform->auth.key.length); + sess->template_job.u.NIA._key = sess->auth.snow5g_auth_key; + return 0; + } else if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_NIA5) { + sess->template_job.hash_alg = IMB_AUTH_AES_NIA5; + sess->template_job.auth_tag_output_len_in_bytes = + sess->auth.req_digest_len; + IMB_AES_KEYEXP_256(mb_mgr, xform->auth.key.data, + sess->cipher.expanded_aes_keys.encode, + sess->cipher.expanded_aes_keys.decode); + sess->template_job.u.NIA._key = + (uint8_t *)sess->cipher.expanded_aes_keys.encode; + return 0; + } else if (xform->auth.algo == RTE_CRYPTO_AUTH_ZUC_NIA6) { + sess->template_job.hash_alg = IMB_AUTH_ZUC_NIA6; + sess->template_job.auth_tag_output_len_in_bytes = + sess->auth.req_digest_len; + memcpy(sess->auth.zuc_auth_key, xform->auth.key.data, + xform->auth.key.length); + sess->template_job.u.ZUC_EIA3._key = sess->auth.zuc_auth_key; + return 0; + } +#endif + switch (xform->auth.algo) { case RTE_CRYPTO_AUTH_MD5_HMAC: sess->template_job.hash_alg = IMB_AUTH_MD5; @@ -352,6 +387,9 @@ aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr, uint8_t is_snow3g = 0; uint8_t is_kasumi = 0; uint8_t is_sm4 = 0; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + uint8_t is_nxan = 0; +#endif if (xform == NULL) { sess->template_job.cipher_mode = IMB_CIPHER_NULL; @@ -434,6 +472,20 @@ aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr, sess->template_job.cipher_mode = IMB_CIPHER_SM4_CNTR; is_sm4 = 1; break; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + case RTE_CRYPTO_CIPHER_SNOW5G_NEA4: + sess->template_job.cipher_mode = IMB_CIPHER_SNOW5G_NEA4; + is_nxan = 1; + break; + case RTE_CRYPTO_CIPHER_AES_NEA5: + sess->template_job.cipher_mode = IMB_CIPHER_AES_NEA5; + is_nxan = 1; + break; + case RTE_CRYPTO_CIPHER_ZUC_NEA6: + sess->template_job.cipher_mode = IMB_CIPHER_ZUC_NEA6; + is_nxan = 1; + break; +#endif default: IPSEC_MB_LOG(ERR, "Unsupported cipher mode parameter"); return -ENOTSUP; @@ -578,6 +630,23 @@ aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr, sess->cipher.expanded_sm4_keys.decode); sess->template_job.enc_keys = sess->cipher.expanded_sm4_keys.encode; sess->template_job.dec_keys = sess->cipher.expanded_sm4_keys.decode; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + } else if (is_nxan) { + if (xform->cipher.key.length != 32) { + IPSEC_MB_LOG(ERR, "Invalid cipher key length"); + return -EINVAL; + } + if (xform->cipher.iv.length != 16) { + IPSEC_MB_LOG(ERR, "Invalid cipher IV length"); + return -EINVAL; + } + sess->template_job.key_len_in_bytes = 32; + IMB_AES_KEYEXP_256(mb_mgr, xform->cipher.key.data, + sess->cipher.expanded_aes_keys.encode, + sess->cipher.expanded_aes_keys.decode); + sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode; + sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode; +#endif } else { if (xform->cipher.key.length != 8) { IPSEC_MB_LOG(ERR, "Invalid cipher key length"); @@ -731,6 +800,71 @@ aesni_mb_set_session_aead_parameters(IMB_MGR *mb_mgr, sess->template_job.enc_keys = &sess->cipher.gcm_key; sess->template_job.dec_keys = &sess->cipher.gcm_key; break; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + case RTE_CRYPTO_AEAD_SNOW5G_NCA4: + sess->template_job.cipher_mode = IMB_CIPHER_SNOW5G_NCA4; + sess->template_job.hash_alg = IMB_AUTH_SNOW5G_NCA4; + sess->template_job.u.GCM.aad_len_in_bytes = xform->aead.aad_length; + if (xform->aead.key.length != 32) { + IPSEC_MB_LOG(ERR, "Invalid key length"); + return -EINVAL; + } + sess->template_job.key_len_in_bytes = 32; + IMB_AES_KEYEXP_256(mb_mgr, xform->aead.key.data, + sess->cipher.expanded_aes_keys.encode, + sess->cipher.expanded_aes_keys.decode); + sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode; + sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode; + /* digest size must be between 4 and 16 */ + if (sess->auth.req_digest_len < 4 || + sess->auth.req_digest_len > 16) { + IPSEC_MB_LOG(ERR, "Invalid digest size"); + return -EINVAL; + } + break; + case RTE_CRYPTO_AEAD_AES_NCA5: + sess->template_job.cipher_mode = IMB_CIPHER_AES_NCA5; + sess->template_job.hash_alg = IMB_AUTH_AES_NCA5; + sess->template_job.u.GCM.aad_len_in_bytes = xform->aead.aad_length; + if (xform->aead.key.length != 32) { + IPSEC_MB_LOG(ERR, "Invalid key length"); + return -EINVAL; + } + sess->template_job.key_len_in_bytes = 32; + IMB_AES_KEYEXP_256(mb_mgr, xform->aead.key.data, + sess->cipher.expanded_aes_keys.encode, + sess->cipher.expanded_aes_keys.decode); + sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode; + sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode; + /* digest size must be between 4 and 16 */ + if (sess->auth.req_digest_len < 4 || + sess->auth.req_digest_len > 16) { + IPSEC_MB_LOG(ERR, "Invalid digest size"); + return -EINVAL; + } + break; + case RTE_CRYPTO_AEAD_ZUC_NCA6: + sess->template_job.cipher_mode = IMB_CIPHER_ZUC_NCA6; + sess->template_job.hash_alg = IMB_AUTH_ZUC_NCA6; + sess->template_job.u.NCA.aad_len_in_bytes = xform->aead.aad_length; + if (xform->aead.key.length != 32) { + IPSEC_MB_LOG(ERR, "Invalid key length"); + return -EINVAL; + } + sess->template_job.key_len_in_bytes = 32; + IMB_AES_KEYEXP_256(mb_mgr, xform->aead.key.data, + sess->cipher.expanded_aes_keys.encode, + sess->cipher.expanded_aes_keys.decode); + sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode; + sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode; + /* digest size must be between 4 and 16 */ + if (sess->auth.req_digest_len < 4 || + sess->auth.req_digest_len > 16) { + IPSEC_MB_LOG(ERR, "Invalid digest size"); + return -EINVAL; + } + break; +#endif default: IPSEC_MB_LOG(ERR, "Unsupported aead mode parameter"); return -ENOTSUP; @@ -1060,7 +1194,24 @@ set_cpu_mb_job_params(IMB_JOB *job, struct aesni_mb_session *session, case IMB_AUTH_SM4_GCM: job->u.GCM.aad = aad->va; break; - +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + case IMB_AUTH_SNOW5G_NCA4: + case IMB_AUTH_AES_NCA5: + job->u.GCM.aad = aad->va; + break; + case IMB_AUTH_ZUC_NCA6: + job->u.NCA.aad = aad->va; + break; + case IMB_AUTH_SNOW5G_NIA4: + job->u.NIA._iv = iv->va; + break; + case IMB_AUTH_AES_NIA5: + job->u.NIA._iv = iv->va; + break; + case IMB_AUTH_ZUC_NIA6: + job->u.ZUC_EIA3._iv = iv->va; + break; +#endif default: break; @@ -1235,8 +1386,17 @@ handle_sgl_linear(IMB_JOB *job, struct rte_crypto_op *op, uint32_t dst_offset, job->dst = linear_buf + dst_offset; job->user_data2 = linear_buf; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + if (job->hash_alg == IMB_AUTH_AES_GMAC || + job->hash_alg == IMB_AUTH_SNOW5G_NCA4 || + job->hash_alg == IMB_AUTH_AES_NCA5) + job->u.GCM.aad = linear_buf; + else if (job->hash_alg == IMB_AUTH_ZUC_NCA6) + job->u.NCA.aad = linear_buf; +#else if (job->hash_alg == IMB_AUTH_AES_GMAC) job->u.GCM.aad = linear_buf; +#endif if (session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) job->auth_tag_output = linear_buf + lb_offset; @@ -1620,6 +1780,28 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, case IMB_AUTH_SM4_GCM: job->u.GCM.aad = op->sym->aead.aad.data; break; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + case IMB_AUTH_SNOW5G_NCA4: + case IMB_AUTH_AES_NCA5: + job->u.GCM.aad = op->sym->aead.aad.data; + break; + case IMB_AUTH_ZUC_NCA6: + job->u.NCA.aad = op->sym->aead.aad.data; + break; + case IMB_AUTH_SNOW5G_NIA4: + job->u.NIA._iv = rte_crypto_op_ctod_offset(op, uint8_t *, + session->auth_iv.offset); + break; + case IMB_AUTH_AES_NIA5: + job->u.NIA._iv = rte_crypto_op_ctod_offset(op, uint8_t *, + session->auth_iv.offset); + break; + case IMB_AUTH_ZUC_NIA6: + job->u.ZUC_EIA3._iv = rte_crypto_op_ctod_offset(op, uint8_t *, + session->auth_iv.offset); + break; +#endif + default: break; } @@ -1775,6 +1957,29 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); break; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + case IMB_AUTH_SNOW5G_NCA4: + case IMB_AUTH_AES_NCA5: + case IMB_AUTH_ZUC_NCA6: + job->hash_start_src_offset_in_bytes = + op->sym->aead.data.offset; + job->msg_len_to_hash_in_bytes = + op->sym->aead.data.length; + job->cipher_start_src_offset_in_bytes = + op->sym->aead.data.offset; + job->msg_len_to_cipher_in_bytes = op->sym->aead.data.length; + job->iv = rte_crypto_op_ctod_offset(op, uint8_t *, + session->iv.offset); + break; + case IMB_AUTH_SNOW5G_NIA4: + case IMB_AUTH_AES_NIA5: + case IMB_AUTH_ZUC_NIA6: + job->hash_start_src_offset_in_bytes = + op->sym->auth.data.offset >> 3; + job->msg_len_to_hash_in_bytes = + op->sym->auth.data.length >> 3; + break; +#endif default: job->hash_start_src_offset_in_bytes = auth_start_offset( @@ -1831,6 +2036,15 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, case IMB_CIPHER_SM4_GCM: job->msg_len_to_cipher_in_bytes = op->sym->aead.data.length; break; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + case IMB_CIPHER_SNOW5G_NCA4: + case IMB_CIPHER_AES_NCA5: + case IMB_CIPHER_ZUC_NCA6: + job->cipher_start_src_offset_in_bytes = + op->sym->aead.data.offset; + job->msg_len_to_cipher_in_bytes = op->sym->aead.data.length; + break; +#endif default: job->cipher_start_src_offset_in_bytes = op->sym->cipher.data.offset; diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h index 6afda45414..9390bad208 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h @@ -882,6 +882,233 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { }, } }, } }, +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + { /* 256-NEA4 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_SNOW5G_NEA4, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* 256-NEA5 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_NEA5, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* 256-NEA6 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_ZUC_NEA6, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* 256-NIA4 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SNOW5G_NIA4, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 16, + .increment = 1 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* 256-NIA5 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_AES_NIA5, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 16, + .increment = 1 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* 256-NIA6 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_ZUC_NIA6, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 16, + .increment = 1 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* 256-NCA4 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_SNOW5G_NCA4, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 16, + .increment = 1 + }, + .aad_size = { + .min = 0, + .max = 65535, + .increment = 1 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + }, } + }, } + }, + { /* 256-NCA5 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_AES_NCA5, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 16, + .increment = 1 + }, + .aad_size = { + .min = 0, + .max = 65535, + .increment = 1 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + }, } + }, } + }, + { /* 256-NCA6 */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD, + {.aead = { + .algo = RTE_CRYPTO_AEAD_ZUC_NCA6, + .block_size = 8, + .key_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 16, + .increment = 1 + }, + .aad_size = { + .min = 0, + .max = 65535, + .increment = 1 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + }, } + }, } + }, +#endif RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() }; @@ -1101,6 +1328,8 @@ struct __rte_cache_aligned aesni_mb_session { /* *< Expanded XCBC authentication keys */ uint8_t zuc_auth_key[32]; /* *< ZUC authentication key */ + uint8_t snow5g_auth_key[32]; + /* *< SNOW5G authentication key */ snow3g_key_schedule_t pKeySched_snow3g_auth; /* *< SNOW3G scheduled authentication key */ kasumi_key_sched_t pKeySched_kasumi_auth; -- 2.52.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA 2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau 2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau 2026-10-02 9:56 ` [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms Radu Nicolau @ 2026-10-02 9:56 ` Radu Nicolau 2026-10-02 9:56 ` [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB Radu Nicolau 2026-10-06 13:08 ` [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Ji, Kai 4 siblings, 0 replies; 7+ messages in thread From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw) To: dev; +Cc: Radu Nicolau, Kai Ji, Pablo de Lara Enable basic support for asymmetric crypto operations. Add support for ML-KEM and ML-DSA algorithms. IPsecMB release 3.0 and newer required. Signed-off-by: Radu Nicolau <radu.nicolau@intel.com> --- doc/guides/cryptodevs/aesni_mb.rst | 6 + doc/guides/cryptodevs/features/aesni_mb.ini | 3 + doc/guides/rel_notes/release_26_11.rst | 3 + drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 58 +++ drivers/crypto/ipsec_mb/ipsec_mb_private.h | 20 ++ drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 372 +++++++++++++++++++- drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 86 ++++- 7 files changed, 532 insertions(+), 16 deletions(-) diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst index 5fb73e31b4..395bdc77da 100644 --- a/doc/guides/cryptodevs/aesni_mb.rst +++ b/doc/guides/cryptodevs/aesni_mb.rst @@ -78,6 +78,12 @@ AEAD algorithms: * RTE_CRYPTO_AEAD_AES_NCA5 * RTE_CRYPTO_AEAD_ZUC_NCA6 +PQC Asymmetric Crypto algorithms + +* RTE_CRYPTO_ASYM_XFORM_ML_KEM +* RTE_CRYPTO_ASYM_XFORM_ML_DSA + + Protocol offloads: * RTE_SECURITY_PROTOCOL_DOCSIS diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini index e64818bf46..c6a28987dd 100644 --- a/doc/guides/cryptodevs/features/aesni_mb.ini +++ b/doc/guides/cryptodevs/features/aesni_mb.ini @@ -21,6 +21,7 @@ CPU crypto = Y Symmetric sessionless = Y Non-Byte aligned data = Y Digest encrypted = Y +Asymmetric crypto = Y ; ; Supported crypto algorithms of the 'aesni_mb' crypto driver. @@ -95,6 +96,8 @@ ZUC-256 NCA6 = Y ; Supported Asymmetric algorithms of the 'aesni_mb' crypto driver. ; [Asymmetric] +ML-KEM = Y +ML-DSA = Y ; ; Supported Operating systems of the 'aesni_mb' crypto driver. diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index 1ee3cdfba3..7dd8231706 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -40,6 +40,9 @@ New Features - NEA4, NIA4, NCA4: Snow 5G confidentiality, integrity and AEAD modes. - NEA5, NIA5, NCA5: AES 256 confidentiality, integrity and AEAD modes. - NEA6, NIA6, NCA6: ZUC 256 confidentiality, integrity and AEAD modes. + * Added support for the following PQC asymmetric algorithms: + - ML-KEM: Module-Lattice-Based Key-Encapsulation Mechanism. + - ML-DSA: Module-Lattice-Based Digital Signature Algorithm. .. This section should contain new features added in this release. diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_ops.c b/drivers/crypto/ipsec_mb/ipsec_mb_ops.c index 50ee140ccd..308775882c 100644 --- a/drivers/crypto/ipsec_mb/ipsec_mb_ops.c +++ b/drivers/crypto/ipsec_mb/ipsec_mb_ops.c @@ -453,3 +453,61 @@ void ipsec_mb_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess __rte_unused) {} + +/** Configure pmd specific multi-buffer asym session from a crypto xform chain */ +int +ipsec_mb_asym_session_configure( + struct rte_cryptodev *dev, struct rte_crypto_asym_xform *xform, + struct rte_cryptodev_asym_session *sess) +{ + struct ipsec_mb_dev_private *internals = dev->data->dev_private; + struct ipsec_mb_internals *pmd_data = + &ipsec_mb_pmds[internals->pmd_type]; + struct ipsec_mb_qp *qp = dev->data->queue_pairs[0]; + IMB_MGR *mb_mgr; + int ret = 0; + + if (qp != NULL) + mb_mgr = qp->mb_mgr; + else + mb_mgr = alloc_init_mb_mgr(); + + if (!mb_mgr) + return -ENOMEM; + + if (unlikely(sess == NULL)) { + IPSEC_MB_LOG(ERR, "invalid session struct"); + if (qp == NULL) + free_mb_mgr(mb_mgr); + return -EINVAL; + } + + ret = (*pmd_data->asym_session_configure)(mb_mgr, + CRYPTODEV_GET_ASYM_SESS_PRIV(sess), xform); + if (ret != 0) { + IPSEC_MB_LOG(ERR, "failed configure asym session parameters"); + + /* Return session to mempool */ + if (qp == NULL) + free_mb_mgr(mb_mgr); + return ret; + } + + if (qp == NULL) + free_mb_mgr(mb_mgr); + + pmd_data->asym_session_created = true; + return 0; +} + +/** Clear the asym session memory */ +void +ipsec_mb_asym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_asym_session *sess) +{ + struct ipsec_mb_dev_private *internals = dev->data->dev_private; + struct ipsec_mb_internals *pmd_data = + &ipsec_mb_pmds[internals->pmd_type]; + + (*pmd_data->asym_session_clear)(CRYPTODEV_GET_ASYM_SESS_PRIV(sess)); +} diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_private.h b/drivers/crypto/ipsec_mb/ipsec_mb_private.h index fc7d6f5f06..5509dc21f3 100644 --- a/drivers/crypto/ipsec_mb/ipsec_mb_private.h +++ b/drivers/crypto/ipsec_mb/ipsec_mb_private.h @@ -356,6 +356,12 @@ typedef int (*ipsec_mb_session_configure_t)(IMB_MGR *mbr_mgr, void *session_private, const struct rte_crypto_sym_xform *xform); +typedef int (*ipsec_mb_asym_session_configure_t)(IMB_MGR *mbr_mgr, + void *session_private, + const struct rte_crypto_asym_xform *xform); + +typedef void (*ipsec_mb_asym_session_clear_t)(void *session_private); + /** Configure internals PMD type specific data */ typedef int (*ipsec_mb_dev_configure_t)(struct rte_cryptodev *dev); @@ -366,6 +372,9 @@ struct ipsec_mb_internals { ipsec_mb_dev_configure_t dev_config; ipsec_mb_queue_pair_configure_t queue_pair_configure; ipsec_mb_session_configure_t session_configure; + ipsec_mb_asym_session_configure_t asym_session_configure; + ipsec_mb_asym_session_clear_t asym_session_clear; + bool asym_session_created; const struct rte_cryptodev_capabilities *caps; struct rte_cryptodev_ops *ops; struct rte_security_ops *security_ops; @@ -423,11 +432,22 @@ int ipsec_mb_sym_session_configure( struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *sess); +/** Configure an aesni multi-buffer asym session from a crypto xform chain */ +int ipsec_mb_asym_session_configure( + struct rte_cryptodev *dev, + struct rte_crypto_asym_xform *xform, + struct rte_cryptodev_asym_session *sess); + /** Clear the memory of session so it does not leave key material behind */ void ipsec_mb_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess); +/** Clear the asym session memory */ +void +ipsec_mb_asym_session_clear(struct rte_cryptodev *dev, + struct rte_cryptodev_asym_session *sess); + /** Get session from op. If sessionless create a session */ static __rte_always_inline void * ipsec_mb_get_session_private(struct ipsec_mb_qp *qp, struct rte_crypto_op *op) diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c index a49a82d561..ab65316978 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c @@ -874,7 +874,7 @@ aesni_mb_set_session_aead_parameters(IMB_MGR *mb_mgr, } /** Configure a aesni multi-buffer session from a crypto xform chain */ -int +static int aesni_mb_session_configure(IMB_MGR *mb_mgr, void *priv_sess, const struct rte_crypto_sym_xform *xform) @@ -965,6 +965,97 @@ aesni_mb_session_configure(IMB_MGR *mb_mgr, return 0; } +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM +/** Configure a aesni multi-buffer asym session from a crypto xform chain */ +static int +aesni_mb_asym_session_configure(IMB_MGR *mb_mgr, + void *priv_sess, + const struct rte_crypto_asym_xform *xform) +{ + (void)mb_mgr; + struct aesni_mb_session *sess = (struct aesni_mb_session *) priv_sess; + int ret = 0; + + sess->asym.xform_type = xform->xform_type; + + switch (sess->asym.xform_type) { + case RTE_CRYPTO_ASYM_XFORM_ML_KEM: + { + switch (xform->mlkem.type) { + case RTE_CRYPTO_ML_KEM_512: + sess->asym.ml_kem.alg = IMB_ML_KEM_512; + break; + case RTE_CRYPTO_ML_KEM_768: + sess->asym.ml_kem.alg = IMB_ML_KEM_768; + break; + case RTE_CRYPTO_ML_KEM_1024: + sess->asym.ml_kem.alg = IMB_ML_KEM_1024; + break; + default: + ret = -ENOTSUP; + IPSEC_MB_LOG(ERR, "Unsupported type"); + return ret; + } + ret = imb_ml_kem_new(mb_mgr, sess->asym.ml_kem.alg, &sess->asym.ml_kem.ctx); + if (ret) { + IPSEC_MB_LOG(ERR, "Failed to create ML KEM session"); + return ret; + } + IMB_ML_KEM_KEYGEN_PARAMS_INIT(&sess->asym.ml_kem.keygen_params); + IMB_ML_KEM_ENCAP_PARAMS_INIT(&sess->asym.ml_kem.encap_params); + } + break; + case RTE_CRYPTO_ASYM_XFORM_ML_DSA: + { + switch (xform->mldsa.type) { + case RTE_CRYPTO_ML_DSA_44: + sess->asym.ml_dsa.alg = IMB_ML_DSA_44; + break; + case RTE_CRYPTO_ML_DSA_65: + sess->asym.ml_dsa.alg = IMB_ML_DSA_65; + break; + case RTE_CRYPTO_ML_DSA_87: + sess->asym.ml_dsa.alg = IMB_ML_DSA_87; + break; + default: + ret = -ENOTSUP; + IPSEC_MB_LOG(ERR, "Unsupported type"); + return ret; + } + ret = imb_ml_dsa_new(mb_mgr, sess->asym.ml_dsa.alg, &sess->asym.ml_dsa.ctx); + if (ret) { + IPSEC_MB_LOG(ERR, "Failed to create ML DSA session"); + return ret; + } + IMB_ML_DSA_KEYGEN_PARAMS_INIT(&sess->asym.ml_dsa.keygen_params); + IMB_ML_DSA_SIGN_PARAMS_INIT(&sess->asym.ml_dsa.sign_params); + IMB_ML_DSA_VERIFY_PARAMS_INIT(&sess->asym.ml_dsa.verify_params); + } + break; + default: + ret = -ENOTSUP; + IPSEC_MB_LOG(ERR, "Unsupported algorithm"); + return ret; + }; + + return ret; +} + +static void +aesni_mb_asym_session_clear(void *priv_sess) +{ + struct aesni_mb_session *sess = (struct aesni_mb_session *) priv_sess; + if (sess->asym.xform_type == RTE_CRYPTO_ASYM_XFORM_ML_KEM) { + imb_ml_kem_free(sess->asym.ml_kem.ctx); + sess->asym.ml_kem.ctx = NULL; + } else if (sess->asym.xform_type == RTE_CRYPTO_ASYM_XFORM_ML_DSA) { + imb_ml_dsa_free(sess->asym.ml_dsa.ctx); + sess->asym.ml_dsa.ctx = NULL; + } +} + +#endif + /** Check DOCSIS security session configuration is valid */ static int check_docsis_sec_session(struct rte_security_session_conf *conf) @@ -2391,7 +2482,237 @@ set_job_null_op(IMB_JOB *job, struct rte_crypto_op *op) return job; } -uint16_t +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM +static inline size_t aesni_mb_ml_kem_ek_sz(const IMB_ML_KEM_ALG alg) +{ + switch (alg) { + case IMB_ML_KEM_512: + return IMB_ML_KEM_512_PUBKEY_BYTES; + case IMB_ML_KEM_768: + return IMB_ML_KEM_768_PUBKEY_BYTES; + case IMB_ML_KEM_1024: + return IMB_ML_KEM_1024_PUBKEY_BYTES; + } + return 0; +} +static inline size_t aesni_mb_ml_kem_dk_sz(const IMB_ML_KEM_ALG alg) +{ + switch (alg) { + case IMB_ML_KEM_512: + return IMB_ML_KEM_512_PRIVKEY_BYTES; + case IMB_ML_KEM_768: + return IMB_ML_KEM_768_PRIVKEY_BYTES; + case IMB_ML_KEM_1024: + return IMB_ML_KEM_1024_PRIVKEY_BYTES; + } + return 0; +} +static inline size_t aesni_mb_ml_kem_ct_sz(const IMB_ML_KEM_ALG alg) +{ + switch (alg) { + case IMB_ML_KEM_512: + return IMB_ML_KEM_512_CIPHERTEXT_BYTES; + case IMB_ML_KEM_768: + return IMB_ML_KEM_768_CIPHERTEXT_BYTES; + case IMB_ML_KEM_1024: + return IMB_ML_KEM_1024_CIPHERTEXT_BYTES; + } + return 0; +} +static inline size_t aesni_mb_ml_dsa_pk_sz(const IMB_ML_DSA_ALG alg) +{ + switch (alg) { + case IMB_ML_DSA_44: + return IMB_ML_DSA_44_PUBKEY_BYTES; + case IMB_ML_DSA_65: + return IMB_ML_DSA_65_PUBKEY_BYTES; + case IMB_ML_DSA_87: + return IMB_ML_DSA_87_PUBKEY_BYTES; + } + return 0; +} +static inline size_t aesni_mb_ml_dsa_sk_sz(const IMB_ML_DSA_ALG alg) +{ + switch (alg) { + case IMB_ML_DSA_44: + return IMB_ML_DSA_44_PRIVKEY_BYTES; + case IMB_ML_DSA_65: + return IMB_ML_DSA_65_PRIVKEY_BYTES; + case IMB_ML_DSA_87: + return IMB_ML_DSA_87_PRIVKEY_BYTES; + } + return 0; +} +static inline size_t aesni_mb_ml_dsa_sig_sz(const IMB_ML_DSA_ALG alg) +{ + switch (alg) { + case IMB_ML_DSA_44: + return IMB_ML_DSA_44_SIG_BYTES; + case IMB_ML_DSA_65: + return IMB_ML_DSA_65_SIG_BYTES; + case IMB_ML_DSA_87: + return IMB_ML_DSA_87_SIG_BYTES; + } + return 0; +} + +static void +aesni_mb_asym_process(struct rte_crypto_op *op) +{ + struct rte_crypto_asym_op *aop = op->asym; + struct aesni_mb_session *sess = + CRYPTODEV_GET_ASYM_SESS_PRIV(aop->session); + op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + int rc = 0; + + switch (sess->asym.xform_type) { + case RTE_CRYPTO_ASYM_XFORM_ML_KEM: + { + struct rte_crypto_ml_kem_op *mlkem = &aop->mlkem; + switch (mlkem->op) { + case RTE_CRYPTO_ML_KEM_OP_KEYGEN: + { + uint8_t seed_d_z[64]; + struct rte_crypto_ml_kem_keygen_op *keygen = &mlkem->keygen; + memcpy(seed_d_z, keygen->d.data, 32); + memcpy(&seed_d_z[32], keygen->z.data, 32); + sess->asym.ml_kem.keygen_params.seed_d_z = seed_d_z; + sess->asym.ml_kem.keygen_params.seed_d_z_len = 64; + keygen->ek.length = aesni_mb_ml_kem_ek_sz(sess->asym.ml_kem.alg); + keygen->dk.length = aesni_mb_ml_kem_dk_sz(sess->asym.ml_kem.alg); + rc = imb_ml_kem_keypair(sess->asym.ml_kem.ctx, + keygen->ek.data, keygen->ek.length, + keygen->dk.data, keygen->dk.length, + &sess->asym.ml_kem.keygen_params); + } + break; + case RTE_CRYPTO_ML_KEM_OP_ENCAP: + { + struct rte_crypto_ml_kem_encap_op *encap = &mlkem->encap; + sess->asym.ml_kem.encap_params.m_32 = encap->message.data; + sess->asym.ml_kem.encap_params.m_len = encap->message.length; + encap->ek.length = aesni_mb_ml_kem_ek_sz(sess->asym.ml_kem.alg); + encap->cipher.length = aesni_mb_ml_kem_ct_sz(sess->asym.ml_kem.alg); + rc = imb_ml_kem_set_pubkey(sess->asym.ml_kem.ctx, + encap->ek.data, encap->ek.length); + if (rc == 0) + rc = imb_ml_kem_encap(sess->asym.ml_kem.ctx, + encap->cipher.data, encap->cipher.length, + encap->sk.data, encap->sk.length, + &sess->asym.ml_kem.encap_params); + } + break; + case RTE_CRYPTO_ML_KEM_OP_DECAP: + { + struct rte_crypto_ml_kem_decap_op *decap = &mlkem->decap; + decap->dk.length = aesni_mb_ml_kem_dk_sz(sess->asym.ml_kem.alg); + decap->cipher.length = aesni_mb_ml_kem_ct_sz(sess->asym.ml_kem.alg); + rc = imb_ml_kem_set_privkey(sess->asym.ml_kem.ctx, + decap->dk.data, decap->dk.length); + if (rc == 0) + rc = imb_ml_kem_decap(sess->asym.ml_kem.ctx, + decap->sk.data, decap->sk.length, + decap->cipher.data, decap->cipher.length, + NULL); + } + break; + default: + op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + break; + }; + } + break; + case RTE_CRYPTO_ASYM_XFORM_ML_DSA: + { + struct rte_crypto_ml_dsa_op *mldsa = &aop->mldsa; + switch (mldsa->op) { + case RTE_CRYPTO_ML_DSA_OP_KEYGEN: + { + struct rte_crypto_ml_dsa_keygen_op *keygen = &mldsa->keygen; + sess->asym.ml_dsa.keygen_params.xi_32 = keygen->seed.data; + sess->asym.ml_dsa.keygen_params.xi_len = keygen->seed.length; + keygen->pubkey.length = + aesni_mb_ml_dsa_pk_sz(sess->asym.ml_dsa.alg); + keygen->privkey.length = + aesni_mb_ml_dsa_sk_sz(sess->asym.ml_dsa.alg); + rc = imb_ml_dsa_keypair(sess->asym.ml_dsa.ctx, + keygen->pubkey.data, keygen->pubkey.length, + keygen->privkey.data, keygen->privkey.length, + &sess->asym.ml_dsa.keygen_params); + } + break; + case RTE_CRYPTO_ML_DSA_OP_SIGN: + { + struct rte_crypto_ml_dsa_siggen_op *siggen = &mldsa->siggen; + void *md; + size_t md_len; + sess->asym.ml_dsa.sign_params.ctx = siggen->ctx.data; + sess->asym.ml_dsa.sign_params.ctx_len = siggen->ctx.length; + sess->asym.ml_dsa.sign_params.rnd_32 = siggen->seed.data; + sess->asym.ml_dsa.sign_params.rnd_len = siggen->seed.length; + if (siggen->mu.length) { + md = siggen->mu.data; + md_len = siggen->mu.length; + sess->asym.ml_dsa.sign_params.msg_is_mu = 1; + } else { + md = siggen->message.data; + md_len = siggen->message.length; + sess->asym.ml_dsa.sign_params.msg_is_mu = 0; + } + rc = imb_ml_dsa_set_privkey(sess->asym.ml_dsa.ctx, + siggen->privkey.data, siggen->privkey.length); + if (rc == 0) + rc = imb_ml_dsa_sign(sess->asym.ml_dsa.ctx, + siggen->sign.data, &siggen->sign.length, + md, md_len, &sess->asym.ml_dsa.sign_params); + } + break; + case RTE_CRYPTO_ML_DSA_OP_VERIFY: + { + struct rte_crypto_ml_dsa_sigver_op *sigver = &mldsa->sigver; + void *md; + size_t md_len; + sess->asym.ml_dsa.verify_params.ctx = sigver->ctx.data; + sess->asym.ml_dsa.verify_params.ctx_len = sigver->ctx.length; + if (sigver->mu.length) { + md = sigver->mu.data; + md_len = sigver->mu.length; + sess->asym.ml_dsa.verify_params.msg_is_mu = 1; + } else { + md = sigver->message.data; + md_len = sigver->message.length; + sess->asym.ml_dsa.verify_params.msg_is_mu = 0; + } + rc = imb_ml_dsa_set_pubkey(sess->asym.ml_dsa.ctx, + sigver->pubkey.data, sigver->pubkey.length); + if (rc == 0) + rc = imb_ml_dsa_verify(sess->asym.ml_dsa.ctx, + md, md_len, sigver->sign.data, sigver->sign.length, + &sess->asym.ml_dsa.verify_params); + } + break; + default: + op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; + break; + }; + + } + break; + default: + op->status = RTE_CRYPTO_OP_STATUS_ERROR; + break; + } + if (op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED) + op->status = (rc == 0) ? RTE_CRYPTO_OP_STATUS_SUCCESS : + RTE_CRYPTO_OP_STATUS_ERROR; +#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG + if (rc) + IPSEC_MB_LOG(ERR, "%s", imb_get_strerror(rc)); +#endif +} +#endif + +static uint16_t aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, uint16_t nb_ops) { @@ -2456,8 +2777,38 @@ aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, * the IMB_MGR can use that pointer again in next * get_next calls. */ +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + { + struct ipsec_mb_internals *pmd_data = + &ipsec_mb_pmds[qp->pmd_type]; + /* If there is at least one asym session created process + * the asym ops here and copy the sym ops to deqd_ops array + */ + if (pmd_data->asym_session_created) { + struct rte_crypto_op *tmp_dq[IMB_MAX_BURST_SIZE]; + uint16_t sym_cnt = 0; + nb_submit_ops = rte_ring_dequeue_burst(qp->ingress_queue, + (void **)tmp_dq, n, NULL); + for (i = 0; i < nb_submit_ops; i++) { + if (tmp_dq[i]->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + aesni_mb_asym_process(tmp_dq[i]); + ops[processed_jobs++] = tmp_dq[i]; + } else { + deqd_ops[sym_cnt++] = tmp_dq[i]; + } + } + /* What remains are only symmetric ops. */ + nb_submit_ops = sym_cnt; + } else { + /* No asym session created, all ops are sym*/ + nb_submit_ops = rte_ring_dequeue_burst(qp->ingress_queue, + (void **)deqd_ops, n, NULL); + } + } +#else nb_submit_ops = rte_ring_dequeue_burst(qp->ingress_queue, (void **)deqd_ops, n, NULL); +#endif for (i = 0; i < nb_submit_ops; i++) { job = jobs[i]; op = deqd_ops[i]; @@ -2587,7 +2938,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec, return k; } -uint32_t +static uint32_t aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs, struct rte_crypto_sym_vec *vec) @@ -2674,7 +3025,12 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = { .sym_session_get_size = ipsec_mb_sym_session_get_size, .sym_session_configure = ipsec_mb_sym_session_configure, - .sym_session_clear = ipsec_mb_sym_session_clear + .sym_session_clear = ipsec_mb_sym_session_clear, +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + .asym_session_get_size = ipsec_mb_sym_session_get_size, + .asym_session_configure = ipsec_mb_asym_session_configure, + .asym_session_clear = ipsec_mb_asym_session_clear, +#endif }; /** @@ -2800,7 +3156,11 @@ RTE_INIT(ipsec_mb_register_aesni_mb) RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT | RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO | #if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM + /* removed in 3.0 */ RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA | +#else + /* added in 3.0 */ + RTE_CRYPTODEV_FF_ASYMMETRIC_CRYPTO | #endif RTE_CRYPTODEV_FF_SYM_SESSIONLESS | RTE_CRYPTODEV_FF_IN_PLACE_SGL | @@ -2817,5 +3177,9 @@ RTE_INIT(ipsec_mb_register_aesni_mb) aesni_mb_data->security_ops = &aesni_mb_pmd_sec_ops; aesni_mb_data->dev_config = aesni_mb_configure_dev; aesni_mb_data->session_configure = aesni_mb_session_configure; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + aesni_mb_data->asym_session_configure = aesni_mb_asym_session_configure; + aesni_mb_data->asym_session_clear = aesni_mb_asym_session_clear; +#endif aesni_mb_data->session_priv_size = sizeof(struct aesni_mb_session); } diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h index 9390bad208..99ca4e49e6 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h @@ -28,18 +28,6 @@ #define IMB_AUTH_ZUC256_EIA3 IMB_AUTH_ZUC256_EIA3_BITLEN #endif -int -aesni_mb_session_configure(IMB_MGR * m __rte_unused, void *priv_sess, - const struct rte_crypto_sym_xform *xform); - -uint16_t -aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, - uint16_t nb_ops); - -uint32_t -aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused, - struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs, - struct rte_crypto_sym_vec *vec); static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { { /* MD5 HMAC */ @@ -1108,6 +1096,60 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = { }, } }, } }, + { + /* ML-KEM */ + .op = RTE_CRYPTO_OP_TYPE_ASYMMETRIC, + .asym = { + .xform_capa = { + .xform_type = RTE_CRYPTO_ASYM_XFORM_ML_KEM, + .op_types = + ((1 << RTE_CRYPTO_ML_KEM_OP_KEYGEN) | + (1 << RTE_CRYPTO_ML_KEM_OP_ENCAP) | + (1 << RTE_CRYPTO_ML_KEM_OP_DECAP)), + .mlkem_capa = { + [RTE_CRYPTO_ML_KEM_OP_KEYGEN] = + (1 << RTE_CRYPTO_ML_KEM_512) | + (1 << RTE_CRYPTO_ML_KEM_768) | + (1 << RTE_CRYPTO_ML_KEM_1024), + [RTE_CRYPTO_ML_KEM_OP_ENCAP] = + (1 << RTE_CRYPTO_ML_KEM_512) | + (1 << RTE_CRYPTO_ML_KEM_768) | + (1 << RTE_CRYPTO_ML_KEM_1024), + [RTE_CRYPTO_ML_KEM_OP_DECAP] = + (1 << RTE_CRYPTO_ML_KEM_512) | + (1 << RTE_CRYPTO_ML_KEM_768) | + (1 << RTE_CRYPTO_ML_KEM_1024) + } + } + } + }, + { + /* ML-DSA */ + .op = RTE_CRYPTO_OP_TYPE_ASYMMETRIC, + .asym = { + .xform_capa = { + .xform_type = RTE_CRYPTO_ASYM_XFORM_ML_DSA, + .op_types = + ((1 << RTE_CRYPTO_ML_DSA_OP_SIGN) | + (1 << RTE_CRYPTO_ML_DSA_OP_KEYGEN) | + (1 << RTE_CRYPTO_ML_DSA_OP_VERIFY)), + .mldsa_capa = { + [RTE_CRYPTO_ML_DSA_OP_KEYGEN] = + (1 << RTE_CRYPTO_ML_DSA_44) | + (1 << RTE_CRYPTO_ML_DSA_65) | + (1 << RTE_CRYPTO_ML_DSA_87), + [RTE_CRYPTO_ML_DSA_OP_SIGN] = + (1 << RTE_CRYPTO_ML_DSA_44) | + (1 << RTE_CRYPTO_ML_DSA_65) | + (1 << RTE_CRYPTO_ML_DSA_87), + [RTE_CRYPTO_ML_DSA_OP_VERIFY] = + (1 << RTE_CRYPTO_ML_DSA_44) | + (1 << RTE_CRYPTO_ML_DSA_65) | + (1 << RTE_CRYPTO_ML_DSA_87) + } + } + } + }, #endif RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() }; @@ -1339,6 +1381,26 @@ struct __rte_cache_aligned aesni_mb_session { uint16_t req_digest_len; } auth; +#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM + struct { + enum rte_crypto_asym_xform_type xform_type; + union { + struct { + IMB_ML_KEM *ctx; + IMB_ML_KEM_ALG alg; + IMB_ML_KEM_KEYGEN_PARAMS keygen_params; + IMB_ML_KEM_ENCAP_PARAMS encap_params; + } ml_kem; + struct { + IMB_ML_DSA *ctx; + IMB_ML_DSA_ALG alg; + IMB_ML_DSA_KEYGEN_PARAMS keygen_params; + IMB_ML_DSA_SIGN_PARAMS sign_params; + IMB_ML_DSA_VERIFY_PARAMS verify_params; + } ml_dsa; + }; + } asym; + #endif }; typedef void (*hash_one_block_t)(const void *data, void *digest); -- 2.52.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB 2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau ` (2 preceding siblings ...) 2026-10-02 9:56 ` [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA Radu Nicolau @ 2026-10-02 9:56 ` Radu Nicolau 2026-10-06 13:08 ` [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Ji, Kai 4 siblings, 0 replies; 7+ messages in thread From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw) To: dev; +Cc: Radu Nicolau, Akhil Goyal, Fan Zhang Enable asymmetric crypto unit test suite for AESNI_MB PMD. Signed-off-by: Radu Nicolau <radu.nicolau@intel.com> --- app/test/test_cryptodev_asym.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c index 4f0408c87a..1870011610 100644 --- a/app/test/test_cryptodev_asym.c +++ b/app/test/test_cryptodev_asym.c @@ -6181,6 +6181,12 @@ test_cryptodev_virtio_user_asym(void) return run_cryptodev_asym_testsuite(RTE_STR(CRYPTODEV_NAME_VIRTIO_USER_PMD)); } +static int +test_cryptodev_aesni_mb_asym(void) +{ + return run_cryptodev_asym_testsuite(RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)); +} + REGISTER_DRIVER_TEST(cryptodev_openssl_asym_autotest, test_cryptodev_openssl_asym); REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym); REGISTER_DRIVER_TEST(cryptodev_octeontx_asym_autotest, test_cryptodev_octeontx_asym); @@ -6189,3 +6195,4 @@ REGISTER_DRIVER_TEST(cryptodev_cn10k_asym_autotest, test_cryptodev_cn10k_asym); REGISTER_DRIVER_TEST(cryptodev_cn20k_asym_autotest, test_cryptodev_cn20k_asym); REGISTER_DRIVER_TEST(cryptodev_virtio_asym_autotest, test_cryptodev_virtio_asym); REGISTER_DRIVER_TEST(cryptodev_virtio_user_asym_autotest, test_cryptodev_virtio_user_asym); +REGISTER_DRIVER_TEST(cryptodev_aesni_mb_asym_autotest, test_cryptodev_aesni_mb_asym); -- 2.52.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau ` (3 preceding siblings ...) 2026-10-02 9:56 ` [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB Radu Nicolau @ 2026-10-06 13:08 ` Ji, Kai 4 siblings, 0 replies; 7+ messages in thread From: Ji, Kai @ 2026-10-06 13:08 UTC (permalink / raw) To: Nicolau, Radu, dev@dpdk.org [-- Attachment #1: Type: text/plain, Size: 1526 bytes --] Series-acked-by: Kai Ji <kai.ji@intel.com> ________________________________ From: Radu Nicolau <radu.nicolau@intel.com> Sent: 02 October 2026 10:56 To: dev@dpdk.org <dev@dpdk.org> Cc: Nicolau, Radu <radu.nicolau@intel.com> Subject: [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Intel(R) Multi-Buffer Crypto for IPsec Library released version 3.0.0 codenamed Túnel Pablo https://github.com/intel/intel-ipsec-mb/releases/tag/v3.0.0 This patchset adds support for the new library version and support for the new algorithms included in this release. Radu Nicolau (4): crypto/ipsec_mb: add support for IPsecMB 3.0 crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms crypto/ipsec_mb: add support for ML-KEM and ML-DSA test/crypto: add asym test suite for AESNI_MB app/test/test_cryptodev_asym.c | 7 + doc/guides/cryptodevs/aesni_gcm.rst | 4 +- doc/guides/cryptodevs/aesni_mb.rst | 26 +- doc/guides/cryptodevs/features/aesni_mb.ini | 12 + doc/guides/rel_notes/release_26_11.rst | 12 + drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 58 ++ drivers/crypto/ipsec_mb/ipsec_mb_private.h | 20 + drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++-- drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 695 +++++++++++++++++-- drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 360 +++++++++- 11 files changed, 1354 insertions(+), 138 deletions(-) -- 2.52.0 [-- Attachment #2: Type: text/html, Size: 3034 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-10-06 13:08 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau 2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau 2026-10-06 12:06 ` Ji, Kai 2026-10-02 9:56 ` [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms Radu Nicolau 2026-10-02 9:56 ` [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA Radu Nicolau 2026-10-02 9:56 ` [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB Radu Nicolau 2026-10-06 13:08 ` [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Ji, Kai
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox