* [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0
@ 2026-10-02 9:56 Radu Nicolau
2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw)
To: dev; +Cc: Radu Nicolau
Intel(R) Multi-Buffer Crypto for IPsec Library released version 3.0.0
codenamed Túnel Pablo
https://github.com/intel/intel-ipsec-mb/releases/tag/v3.0.0
This patchset adds support for the new library version and support
for the new algorithms included in this release.
Radu Nicolau (4):
crypto/ipsec_mb: add support for IPsecMB 3.0
crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms
crypto/ipsec_mb: add support for ML-KEM and ML-DSA
test/crypto: add asym test suite for AESNI_MB
app/test/test_cryptodev_asym.c | 7 +
doc/guides/cryptodevs/aesni_gcm.rst | 4 +-
doc/guides/cryptodevs/aesni_mb.rst | 26 +-
doc/guides/cryptodevs/features/aesni_mb.ini | 12 +
doc/guides/rel_notes/release_26_11.rst | 12 +
drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 58 ++
drivers/crypto/ipsec_mb/ipsec_mb_private.h | 20 +
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++--
drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +-
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 695 +++++++++++++++++--
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 360 +++++++++-
11 files changed, 1354 insertions(+), 138 deletions(-)
--
2.52.0
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/4] crypto/ipsec_mb: add support for IPsecMB 3.0
2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau
@ 2026-10-02 9:56 ` Radu Nicolau
2026-10-06 12:06 ` Ji, Kai
2026-10-02 9:56 ` [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms Radu Nicolau
` (3 subsequent siblings)
4 siblings, 1 reply; 7+ messages in thread
From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw)
To: dev; +Cc: Radu Nicolau, Kai Ji, Pablo de Lara
Add support for IPsecMB release 3.0
- removed all _BITLEN enums, all lengths are in bytes
- removed feature flag
- use non const IMB_MGR
- use IMB API instead of function pointers in the GCM PMD
- remove ZUC 256 support
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
doc/guides/cryptodevs/aesni_gcm.rst | 4 +-
doc/guides/cryptodevs/aesni_mb.rst | 11 +-
doc/guides/rel_notes/release_26_11.rst | 4 +
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++++++++++++----
drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +--
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 107 +++++---
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 47 +++-
7 files changed, 349 insertions(+), 122 deletions(-)
diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst
index aefc1cedcb..e68dc4eb1e 100644
--- a/doc/guides/cryptodevs/aesni_gcm.rst
+++ b/doc/guides/cryptodevs/aesni_gcm.rst
@@ -40,8 +40,8 @@ Installation
To build DPDK with the AESNI_GCM_PMD the user is required to download the multi-buffer
library from `here <https://github.com/01org/intel-ipsec-mb>`_
and compile it on their user system before building DPDK.
-The latest version of the library supported by this PMD is v2.0, which
-can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_.
+The latest version of the library supported by this PMD is v3.0, which
+can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_.
For Arm system, ARM64 port of the multi-buffer library can be downloaded from
https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/.
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index add3c03b74..4851997fd9 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -73,6 +73,13 @@ Protocol offloads:
* RTE_SECURITY_PROTOCOL_DOCSIS
+Note
+----
+Version 3.0 of the multi-buffer library removed support for:
+
+* the 256 bit versions of ZUC EEA3 and ZUC EIA3 algorithms.
+* non byte aligned bit length (``RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA`` removed).
+
Limitations
-----------
@@ -96,8 +103,8 @@ Installation
To build DPDK with the AESNI_MB_PMD the user is required to download the multi-buffer
library from `here <https://github.com/01org/intel-ipsec-mb>`_
and compile it on their user system before building DPDK.
-The latest version of the library supported by this PMD is v2.0, which
-can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_.
+The latest version of the library supported by this PMD is v3.0, which
+can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_.
For Arm system, ARM64 port of the multi-buffer library can be downloaded from
https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/.
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 69ab5af8a3..ea126aa9bb 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -33,6 +33,10 @@ New Features
``pss_salt`` field in ``rte_crypto_rsa_op_param`` to carry it per
sign operation.
+* **Updated AESNI_MB crypto driver.**
+
+ * Added support for version 3.0 of IPsec MB Library.
+
.. This section should contain new features added in this release.
Sample format:
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c
index 3c266cc47e..0c4358a7b1 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c
@@ -6,56 +6,205 @@
#include "pmd_aesni_gcm_priv.h"
+/* wrapper functions for library versions < 3.0.0 */
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_pre(const void *key, \
+ struct gcm_key_data *key_data, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_pre(key, key_data); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_init( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, const void *iv, \
+ const void *aad, uint64_t aad_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_init(key_data, context_data, iv, aad, aad_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, uint64_t len, const void *iv, \
+ const void *aad, uint64_t aad_len, void *auth_tag, \
+ uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_enc(key_data, context_data, out, in, len, iv, \
+ aad, aad_len, auth_tag, auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_update( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, const uint64_t len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_enc_update(key_data, context_data, out, in, len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_finalize( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *auth_tag, \
+ const uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_enc_finalize(key_data, context_data, auth_tag, \
+ auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, uint64_t len, const void *iv, \
+ const void *aad, uint64_t aad_len, void *auth_tag, \
+ uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_dec(key_data, context_data, out, in, len, iv, \
+ aad, aad_len, auth_tag, auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_update( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, const uint64_t len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_dec_update(key_data, context_data, out, in, len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_finalize( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *auth_tag, \
+ const uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_dec_finalize(key_data, context_data, auth_tag, \
+ auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gmac_init( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, const void *iv, \
+ const uint64_t iv_len, IMB_MGR *state) \
+{ \
+ state->gmac##bitlen##_init(key_data, context_data, iv, iv_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gmac_update( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, const void *in, \
+ const uint64_t len, IMB_MGR *state) \
+{ \
+ state->gmac##bitlen##_update(key_data, context_data, in, len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gmac_finalize( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *auth_tag, \
+ const uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gmac##bitlen##_finalize(key_data, context_data, auth_tag, \
+ auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#endif
+
static void
aesni_gcm_set_ops(struct aesni_gcm_ops *ops, IMB_MGR *mb_mgr)
{
+ /* Set IMB_MGR pointers. */
+ ops[GCM_KEY_128].mb_mgr = mb_mgr;
+ ops[GCM_KEY_192].mb_mgr = mb_mgr;
+ ops[GCM_KEY_256].mb_mgr = mb_mgr;
+
/* Set 128 bit function pointers. */
- ops[GCM_KEY_128].pre = mb_mgr->gcm128_pre;
- ops[GCM_KEY_128].init = mb_mgr->gcm128_init;
+ ops[GCM_KEY_128].pre = imb_aes128_gcm_pre;
+ ops[GCM_KEY_128].init = imb_aes128_gcm_init;
- ops[GCM_KEY_128].enc = mb_mgr->gcm128_enc;
- ops[GCM_KEY_128].update_enc = mb_mgr->gcm128_enc_update;
- ops[GCM_KEY_128].finalize_enc = mb_mgr->gcm128_enc_finalize;
+ ops[GCM_KEY_128].enc = imb_aes128_gcm_enc;
+ ops[GCM_KEY_128].update_enc = imb_aes128_gcm_enc_update;
+ ops[GCM_KEY_128].finalize_enc = imb_aes128_gcm_enc_finalize;
- ops[GCM_KEY_128].dec = mb_mgr->gcm128_dec;
- ops[GCM_KEY_128].update_dec = mb_mgr->gcm128_dec_update;
- ops[GCM_KEY_128].finalize_dec = mb_mgr->gcm128_dec_finalize;
+ ops[GCM_KEY_128].dec = imb_aes128_gcm_dec;
+ ops[GCM_KEY_128].update_dec = imb_aes128_gcm_dec_update;
+ ops[GCM_KEY_128].finalize_dec = imb_aes128_gcm_dec_finalize;
- ops[GCM_KEY_128].gmac_init = mb_mgr->gmac128_init;
- ops[GCM_KEY_128].gmac_update = mb_mgr->gmac128_update;
- ops[GCM_KEY_128].gmac_finalize = mb_mgr->gmac128_finalize;
+ ops[GCM_KEY_128].gmac_init = imb_aes128_gmac_init;
+ ops[GCM_KEY_128].gmac_update = imb_aes128_gmac_update;
+ ops[GCM_KEY_128].gmac_finalize = imb_aes128_gmac_finalize;
/* Set 192 bit function pointers. */
- ops[GCM_KEY_192].pre = mb_mgr->gcm192_pre;
- ops[GCM_KEY_192].init = mb_mgr->gcm192_init;
+ ops[GCM_KEY_192].pre = imb_aes192_gcm_pre;
+ ops[GCM_KEY_192].init = imb_aes192_gcm_init;
- ops[GCM_KEY_192].enc = mb_mgr->gcm192_enc;
- ops[GCM_KEY_192].update_enc = mb_mgr->gcm192_enc_update;
- ops[GCM_KEY_192].finalize_enc = mb_mgr->gcm192_enc_finalize;
+ ops[GCM_KEY_192].enc = imb_aes192_gcm_enc;
+ ops[GCM_KEY_192].update_enc = imb_aes192_gcm_enc_update;
+ ops[GCM_KEY_192].finalize_enc = imb_aes192_gcm_enc_finalize;
- ops[GCM_KEY_192].dec = mb_mgr->gcm192_dec;
- ops[GCM_KEY_192].update_dec = mb_mgr->gcm192_dec_update;
- ops[GCM_KEY_192].finalize_dec = mb_mgr->gcm192_dec_finalize;
+ ops[GCM_KEY_192].dec = imb_aes192_gcm_dec;
+ ops[GCM_KEY_192].update_dec = imb_aes192_gcm_dec_update;
+ ops[GCM_KEY_192].finalize_dec = imb_aes192_gcm_dec_finalize;
- ops[GCM_KEY_192].gmac_init = mb_mgr->gmac192_init;
- ops[GCM_KEY_192].gmac_update = mb_mgr->gmac192_update;
- ops[GCM_KEY_192].gmac_finalize = mb_mgr->gmac192_finalize;
+ ops[GCM_KEY_192].gmac_init = imb_aes192_gmac_init;
+ ops[GCM_KEY_192].gmac_update = imb_aes192_gmac_update;
+ ops[GCM_KEY_192].gmac_finalize = imb_aes192_gmac_finalize;
/* Set 256 bit function pointers. */
- ops[GCM_KEY_256].pre = mb_mgr->gcm256_pre;
- ops[GCM_KEY_256].init = mb_mgr->gcm256_init;
+ ops[GCM_KEY_256].pre = imb_aes256_gcm_pre;
+ ops[GCM_KEY_256].init = imb_aes256_gcm_init;
- ops[GCM_KEY_256].enc = mb_mgr->gcm256_enc;
- ops[GCM_KEY_256].update_enc = mb_mgr->gcm256_enc_update;
- ops[GCM_KEY_256].finalize_enc = mb_mgr->gcm256_enc_finalize;
+ ops[GCM_KEY_256].enc = imb_aes256_gcm_enc;
+ ops[GCM_KEY_256].update_enc = imb_aes256_gcm_enc_update;
+ ops[GCM_KEY_256].finalize_enc = imb_aes256_gcm_enc_finalize;
- ops[GCM_KEY_256].dec = mb_mgr->gcm256_dec;
- ops[GCM_KEY_256].update_dec = mb_mgr->gcm256_dec_update;
- ops[GCM_KEY_256].finalize_dec = mb_mgr->gcm256_dec_finalize;
+ ops[GCM_KEY_256].dec = imb_aes256_gcm_dec;
+ ops[GCM_KEY_256].update_dec = imb_aes256_gcm_dec_update;
+ ops[GCM_KEY_256].finalize_dec = imb_aes256_gcm_dec_finalize;
- ops[GCM_KEY_256].gmac_init = mb_mgr->gmac256_init;
- ops[GCM_KEY_256].gmac_update = mb_mgr->gmac256_update;
- ops[GCM_KEY_256].gmac_finalize = mb_mgr->gmac256_finalize;
+ ops[GCM_KEY_256].gmac_init = imb_aes256_gmac_init;
+ ops[GCM_KEY_256].gmac_update = imb_aes256_gmac_update;
+ ops[GCM_KEY_256].gmac_finalize = imb_aes256_gmac_finalize;
}
static int
@@ -135,15 +284,15 @@ aesni_gcm_session_configure(IMB_MGR *mb_mgr, void *session,
switch (key_length) {
case 16:
sess->key_length = GCM_KEY_128;
- mb_mgr->gcm128_pre(key, &sess->gdata_key);
+ imb_aes128_gcm_pre(key, &sess->gdata_key, mb_mgr);
break;
case 24:
sess->key_length = GCM_KEY_192;
- mb_mgr->gcm192_pre(key, &sess->gdata_key);
+ imb_aes192_gcm_pre(key, &sess->gdata_key, mb_mgr);
break;
case 32:
sess->key_length = GCM_KEY_256;
- mb_mgr->gcm256_pre(key, &sess->gdata_key);
+ imb_aes256_gcm_pre(key, &sess->gdata_key, mb_mgr);
break;
default:
IPSEC_MB_LOG(ERR, "Invalid key length");
@@ -324,10 +473,10 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT) {
ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr,
sym_op->aead.aad.data,
- (uint64_t)session->aad_length);
+ (uint64_t)session->aad_length, ops->mb_mgr);
ops->update_enc(&session->gdata_key, &qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -345,7 +494,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->update_enc(&session->gdata_key,
&qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
@@ -355,14 +504,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
tag = sym_op->aead.digest.data;
ops->finalize_enc(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
} else if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT) {
ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr,
sym_op->aead.aad.data,
- (uint64_t)session->aad_length);
+ (uint64_t)session->aad_length, ops->mb_mgr);
ops->update_dec(&session->gdata_key, &qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -380,19 +529,19 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->update_dec(&session->gdata_key,
&qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
tag = qp_data->temp_digest;
ops->finalize_dec(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
} else if (session->op == IPSEC_MB_OP_HASH_GEN_ONLY) {
ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data,
- iv_ptr, session->iv.length);
+ iv_ptr, session->iv.length, ops->mb_mgr);
ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data,
- src, (uint64_t)part_len);
+ src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -406,7 +555,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->gmac_update(&session->gdata_key,
&qp_data->gcm_ctx_data, src,
- (uint64_t)part_len);
+ (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
@@ -416,13 +565,13 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
tag = sym_op->auth.digest.data;
ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
} else { /* IPSEC_MB_OP_HASH_VERIFY_ONLY */
ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data,
- iv_ptr, session->iv.length);
+ iv_ptr, session->iv.length, ops->mb_mgr);
ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data,
- src, (uint64_t)part_len);
+ src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -436,14 +585,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->gmac_update(&session->gdata_key,
&qp_data->gcm_ctx_data, src,
- (uint64_t)part_len);
+ (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
tag = qp_data->temp_digest;
ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
}
return 0;
}
@@ -540,11 +689,11 @@ aesni_gcm_sgl_op_finalize_encryption(const struct aesni_gcm_session *s,
uint8_t tmpdigest[s->gen_digest_length];
ops.finalize_enc(&s->gdata_key, gdata_ctx, tmpdigest,
- s->gen_digest_length);
+ s->gen_digest_length, ops.mb_mgr);
memcpy(digest, tmpdigest, s->req_digest_length);
} else {
ops.finalize_enc(&s->gdata_key, gdata_ctx, digest,
- s->gen_digest_length);
+ s->gen_digest_length, ops.mb_mgr);
}
return 0;
@@ -558,7 +707,7 @@ aesni_gcm_sgl_op_finalize_decryption(const struct aesni_gcm_session *s,
uint8_t tmpdigest[s->gen_digest_length];
ops.finalize_dec(&s->gdata_key, gdata_ctx, tmpdigest,
- s->gen_digest_length);
+ s->gen_digest_length, ops.mb_mgr);
return rte_memeq_timingsafe(digest, tmpdigest, s->req_digest_length) ? 0 : EBADMSG;
}
@@ -573,7 +722,7 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s,
/* init crypto operation */
ops.init(&s->gdata_key, gdata_ctx, iv, aad,
- (uint64_t)s->aad_length);
+ (uint64_t)s->aad_length, ops.mb_mgr);
/* update with sgl data */
for (i = 0; i < sgl->num; i++) {
@@ -582,11 +731,11 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s,
switch (s->op) {
case IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT:
ops.update_enc(&s->gdata_key, gdata_ctx,
- vec->base, vec->base, vec->len);
+ vec->base, vec->base, vec->len, ops.mb_mgr);
break;
case IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT:
ops.update_dec(&s->gdata_key, gdata_ctx,
- vec->base, vec->base, vec->len);
+ vec->base, vec->base, vec->len, ops.mb_mgr);
break;
default:
IPSEC_MB_LOG(ERR, "Invalid session op");
@@ -603,7 +752,7 @@ aesni_gcm_process_gmac_sgl_op(const struct aesni_gcm_session *s,
struct aesni_gcm_ops ops)
{
ops.init(&s->gdata_key, gdata_ctx, iv, sgl->vec[0].base,
- sgl->vec[0].len);
+ sgl->vec[0].len, ops.mb_mgr);
}
static inline uint32_t
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h
index 8a5a41fb19..4074430d6e 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h
@@ -79,41 +79,41 @@ enum aesni_gcm_key_length {
typedef void (*aesni_gcm_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *out, const uint8_t *in,
- uint64_t plaintext_len, const uint8_t *iv,
- const uint8_t *aad, uint64_t aad_len,
- uint8_t *auth_tag, uint64_t auth_tag_len);
+ void *out, const void *in,
+ uint64_t plaintext_len, const void *iv,
+ const void *aad, uint64_t aad_len,
+ void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state);
typedef void (*aesni_gcm_pre_t)(const void *key,
- struct gcm_key_data *gcm_data);
+ struct gcm_key_data *gcm_data, IMB_MGR *state);
typedef void (*aesni_gcm_init_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- const uint8_t *iv, uint8_t const *aad,
- uint64_t aad_len);
+ const void *iv, const void *aad,
+ uint64_t aad_len, IMB_MGR *state);
typedef void (*aesni_gcm_update_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *out, const uint8_t *in,
- uint64_t plaintext_len);
+ void *out, const void *in,
+ uint64_t plaintext_len, IMB_MGR *state);
typedef void (*aesni_gcm_finalize_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *auth_tag, uint64_t auth_tag_len);
+ void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state);
typedef void (*aesni_gmac_init_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- const uint8_t *iv, const uint64_t iv_len);
+ const void *iv, const uint64_t iv_len, IMB_MGR *state);
typedef void (*aesni_gmac_update_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- const uint8_t *in,
- const uint64_t plaintext_len);
+ const void *in,
+ const uint64_t plaintext_len, IMB_MGR *state);
typedef void (*aesni_gmac_finalize_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *auth_tag,
- const uint64_t auth_tag_len);
+ void *auth_tag,
+ const uint64_t auth_tag_len, IMB_MGR *state);
/** GCM operation handlers */
struct aesni_gcm_ops {
@@ -128,6 +128,7 @@ struct aesni_gcm_ops {
aesni_gmac_init_t gmac_init;
aesni_gmac_update_t gmac_update;
aesni_gmac_finalize_t gmac_finalize;
+ IMB_MGR *mb_mgr;
};
RTE_DEFINE_PER_LCORE(struct aesni_gcm_ops, gcm_ops[GCM_NUM_KEY_TYPES]);
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
index 02bacd020b..e8f6e77d21 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
@@ -161,20 +161,22 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
if (xform->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) {
if (xform->auth.key.length == 16) {
- sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3_BITLEN;
+ sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3;
if (sess->auth.req_digest_len != 4) {
IPSEC_MB_LOG(ERR, "Invalid digest size");
return -EINVAL;
}
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
} else if (xform->auth.key.length == 32) {
- sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3_BITLEN;
+ sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3;
if (sess->auth.req_digest_len != 4 &&
sess->auth.req_digest_len != 8 &&
sess->auth.req_digest_len != 16) {
IPSEC_MB_LOG(ERR, "Invalid digest size");
return -EINVAL;
}
+#endif
} else {
IPSEC_MB_LOG(ERR, "Invalid authentication key length");
return -EINVAL;
@@ -187,10 +189,10 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
sess->template_job.u.ZUC_EIA3._key = sess->auth.zuc_auth_key;
return 0;
} else if (xform->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) {
- sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2_BITLEN;
+ sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2;
uint16_t snow3g_uia2_digest_len =
get_truncated_digest_byte_length(
- IMB_AUTH_SNOW3G_UIA2_BITLEN);
+ IMB_AUTH_SNOW3G_UIA2);
if (sess->auth.req_digest_len != snow3g_uia2_digest_len) {
IPSEC_MB_LOG(ERR, "Invalid digest size");
return -EINVAL;
@@ -339,7 +341,7 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
/** Set session cipher parameters */
static int
-aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr,
+aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr,
struct aesni_mb_session *sess,
const struct rte_crypto_sym_xform *xform)
{
@@ -407,11 +409,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr,
is_zuc = 1;
break;
case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2_BITLEN;
+ sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2;
is_snow3g = 1;
break;
case RTE_CRYPTO_CIPHER_KASUMI_F8:
- sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1_BITLEN;
+ sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1;
is_kasumi = 1;
break;
case RTE_CRYPTO_CIPHER_NULL:
@@ -536,8 +538,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr,
sess->template_job.dec_keys = sess->cipher.exp_3des_keys.ks_ptr;
sess->template_job.key_len_in_bytes = 24;
} else if (is_zuc) {
- if (xform->cipher.key.length != 16 &&
- xform->cipher.key.length != 32) {
+ if (xform->cipher.key.length != 16
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ && xform->cipher.key.length != 32
+#endif
+ ) {
IPSEC_MB_LOG(ERR, "Invalid cipher key length");
return -EINVAL;
}
@@ -1177,19 +1182,23 @@ sgl_linear_cipher_auth_len(IMB_JOB *job, uint64_t *auth_len)
{
uint64_t cipher_len;
- if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN ||
- job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN)
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ job->cipher_mode == IMB_CIPHER_KASUMI_UEA1)
cipher_len = (job->msg_len_to_cipher_in_bits >> 3) +
(job->cipher_start_src_offset_in_bits >> 3);
else
+#endif
cipher_len = job->msg_len_to_cipher_in_bytes +
job->cipher_start_src_offset_in_bytes;
- if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN ||
- job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN)
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2 ||
+ job->hash_alg == IMB_AUTH_ZUC_EIA3)
*auth_len = (job->msg_len_to_hash_in_bits >> 3) +
job->hash_start_src_offset_in_bytes;
else
+#endif
*auth_len = job->msg_len_to_hash_in_bytes +
job->hash_start_src_offset_in_bytes;
@@ -1457,18 +1466,20 @@ aesni_mb_digest_appended_in_src(struct rte_crypto_op *op, IMB_JOB *job,
return NULL;
if (job->cipher_mode == IMB_CIPHER_ZUC_EEA3 ||
- job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN ||
- job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) {
+ job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ job->cipher_mode == IMB_CIPHER_KASUMI_UEA1) {
cipher_size = (op->sym->cipher.data.offset >> 3) +
(op->sym->cipher.data.length >> 3);
} else {
cipher_size = (op->sym->cipher.data.offset) +
(op->sym->cipher.data.length);
}
- if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN ||
- job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN ||
- job->hash_alg == IMB_AUTH_KASUMI_UIA1 ||
- job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) {
+ if (job->hash_alg == IMB_AUTH_ZUC_EIA3 ||
+ job->hash_alg == IMB_AUTH_SNOW3G_UIA2 ||
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ job->hash_alg == IMB_AUTH_ZUC256_EIA3 ||
+#endif
+ job->hash_alg == IMB_AUTH_KASUMI_UIA1) {
auth_size = (op->sym->auth.data.offset >> 3) +
(op->sym->auth.data.length >> 3);
} else {
@@ -1585,12 +1596,14 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
job->u.GMAC._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->auth_iv.offset);
break;
- case IMB_AUTH_ZUC_EIA3_BITLEN:
- case IMB_AUTH_ZUC256_EIA3_BITLEN:
+ case IMB_AUTH_ZUC_EIA3:
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ case IMB_AUTH_ZUC256_EIA3:
+#endif
job->u.ZUC_EIA3._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->auth_iv.offset);
break;
- case IMB_AUTH_SNOW3G_UIA2_BITLEN:
+ case IMB_AUTH_SNOW3G_UIA2:
job->u.SNOW3G_UIA2._iv =
rte_crypto_op_ctod_offset(op, uint8_t *,
session->auth_iv.offset);
@@ -1616,12 +1629,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
else
m_offset = op->sym->cipher.data.offset;
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
if (cipher_mode == IMB_CIPHER_ZUC_EEA3)
m_offset >>= 3;
- else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN)
+ else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2)
m_offset = 0;
- else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN)
+ else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1)
m_offset = 0;
+#else
+ if (cipher_mode == IMB_CIPHER_ZUC_EEA3 ||
+ cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ cipher_mode == IMB_CIPHER_KASUMI_UEA1)
+ m_offset >>= 3;
+#endif
/* Set digest output location */
if (job->hash_alg != IMB_AUTH_NULL &&
@@ -1701,9 +1721,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
session->iv.offset);
break;
/* ZUC and SNOW3G require length in bits and offset in bytes */
- case IMB_AUTH_ZUC_EIA3_BITLEN:
- case IMB_AUTH_ZUC256_EIA3_BITLEN:
- case IMB_AUTH_SNOW3G_UIA2_BITLEN:
+ case IMB_AUTH_ZUC_EIA3:
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ case IMB_AUTH_ZUC256_EIA3:
+#endif
+ case IMB_AUTH_SNOW3G_UIA2:
auth_off_in_bytes = op->sym->auth.data.offset >> 3;
ciph_off_in_bytes = op->sym->cipher.data.offset >> 3;
auth_len_in_bytes = op->sym->auth.data.length >> 3;
@@ -1714,7 +1736,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
session, oop, auth_off_in_bytes,
ciph_off_in_bytes, auth_len_in_bytes,
ciph_len_in_bytes, lb_sgl);
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
job->msg_len_to_hash_in_bits = op->sym->auth.data.length;
+#else
+ job->msg_len_to_hash_in_bytes = op->sym->auth.data.length >> 3;
+#endif
job->iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->iv.offset);
@@ -1772,12 +1798,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
op->sym->cipher.data.length >> 3;
break;
/* ZUC and SNOW3G require length and offset in bits */
- case IMB_CIPHER_SNOW3G_UEA2_BITLEN:
- case IMB_CIPHER_KASUMI_UEA1_BITLEN:
+ case IMB_CIPHER_SNOW3G_UEA2:
+ case IMB_CIPHER_KASUMI_UEA1:
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
job->cipher_start_src_offset_in_bits =
op->sym->cipher.data.offset;
job->msg_len_to_cipher_in_bits =
op->sym->cipher.data.length;
+#else
+ job->cipher_start_src_offset_in_bytes =
+ op->sym->cipher.data.offset >> 3;
+ job->msg_len_to_cipher_in_bytes =
+ op->sym->cipher.data.length >> 3;
+#endif
break;
case IMB_CIPHER_GCM:
job->cipher_start_src_offset_in_bytes =
@@ -2047,8 +2080,8 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)
aesni_mb_digest_appended_in_src(op, job, oop) != NULL) {
unsigned int auth_size, cipher_size;
int unencrypted_bytes = 0;
- if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN ||
- job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN ||
+ if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ job->cipher_mode == IMB_CIPHER_KASUMI_UEA1 ||
job->cipher_mode == IMB_CIPHER_ZUC_EEA3) {
cipher_size = (op->sym->cipher.data.offset >> 3) +
(op->sym->cipher.data.length >> 3);
@@ -2056,10 +2089,12 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)
cipher_size = (op->sym->cipher.data.offset) +
(op->sym->cipher.data.length);
}
- if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN ||
- job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN ||
- job->hash_alg == IMB_AUTH_KASUMI_UIA1 ||
- job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) {
+ if (job->hash_alg == IMB_AUTH_ZUC_EIA3 ||
+ job->hash_alg == IMB_AUTH_SNOW3G_UIA2 ||
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ job->hash_alg == IMB_AUTH_ZUC256_EIA3 ||
+#endif
+ job->hash_alg == IMB_AUTH_KASUMI_UIA1) {
auth_size = (op->sym->auth.data.offset >> 3) +
(op->sym->auth.data.length >> 3);
} else {
@@ -2550,7 +2585,9 @@ RTE_INIT(ipsec_mb_register_aesni_mb)
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO |
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA |
+#endif
RTE_CRYPTODEV_FF_SYM_SESSIONLESS |
RTE_CRYPTODEV_FF_IN_PLACE_SGL |
RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT |
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
index ad38a1e28a..6afda45414 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
@@ -19,6 +19,15 @@
#define MAX_NUM_SEGS 16
+/* bit len removed in 3.0 */
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+#define IMB_CIPHER_SNOW3G_UEA2 IMB_CIPHER_SNOW3G_UEA2_BITLEN
+#define IMB_CIPHER_KASUMI_UEA1 IMB_CIPHER_KASUMI_UEA1_BITLEN
+#define IMB_AUTH_ZUC_EIA3 IMB_AUTH_ZUC_EIA3_BITLEN
+#define IMB_AUTH_SNOW3G_UIA2 IMB_AUTH_SNOW3G_UIA2_BITLEN
+#define IMB_AUTH_ZUC256_EIA3 IMB_AUTH_ZUC256_EIA3_BITLEN
+#endif
+
int
aesni_mb_session_configure(IMB_MGR * m __rte_unused, void *priv_sess,
const struct rte_crypto_sym_xform *xform);
@@ -573,8 +582,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
.block_size = 16,
.key_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 32,
.increment = 16
+#else
+ .max = 16,
+ .increment = 0
+#endif
},
.digest_size = {
.min = 4,
@@ -583,8 +597,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
},
.iv_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 25,
.increment = 9
+#else
+ .max = 16,
+ .increment = 0
+#endif
}
}, }
}, }
@@ -598,13 +617,23 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
.block_size = 16,
.key_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 32,
.increment = 16
+#else
+ .max = 16,
+ .increment = 0
+#endif
},
.iv_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 25,
.increment = 9
+#else
+ .max = 16,
+ .increment = 0
+#endif
},
}, }
}, }
@@ -888,9 +917,9 @@ static const unsigned int auth_blocksize[] = {
[IMB_AUTH_SHA_256] = 64,
[IMB_AUTH_SHA_384] = 128,
[IMB_AUTH_SHA_512] = 128,
- [IMB_AUTH_ZUC_EIA3_BITLEN] = 16,
- [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 16,
- [IMB_AUTH_KASUMI_UIA1] = 16
+ [IMB_AUTH_ZUC_EIA3] = 16,
+ [IMB_AUTH_SNOW3G_UIA2] = 16,
+ [IMB_AUTH_KASUMI_UIA1] = 16
};
/**
@@ -922,9 +951,9 @@ static const unsigned int auth_truncated_digest_byte_lengths[] = {
[IMB_AUTH_SHA_256] = 32,
[IMB_AUTH_SHA_384] = 48,
[IMB_AUTH_SHA_512] = 64,
- [IMB_AUTH_ZUC_EIA3_BITLEN] = 4,
- [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4,
- [IMB_AUTH_KASUMI_UIA1] = 4
+ [IMB_AUTH_ZUC_EIA3] = 4,
+ [IMB_AUTH_SNOW3G_UIA2] = 4,
+ [IMB_AUTH_KASUMI_UIA1] = 4
};
/**
@@ -957,9 +986,9 @@ static const unsigned int auth_digest_byte_lengths[] = {
[IMB_AUTH_SHA_256] = 32,
[IMB_AUTH_SHA_384] = 48,
[IMB_AUTH_SHA_512] = 64,
- [IMB_AUTH_ZUC_EIA3_BITLEN] = 4,
- [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4,
- [IMB_AUTH_KASUMI_UIA1] = 4,
+ [IMB_AUTH_ZUC_EIA3] = 4,
+ [IMB_AUTH_SNOW3G_UIA2] = 4,
+ [IMB_AUTH_KASUMI_UIA1] = 4,
[IMB_AUTH_SM3] = 32,
[IMB_AUTH_HMAC_SM3] = 32,
/**< Vector mode dependent pointer table of the multi-buffer APIs */
--
2.52.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms
2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau
2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau
@ 2026-10-02 9:56 ` Radu Nicolau
2026-10-02 9:56 ` [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA Radu Nicolau
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw)
To: dev; +Cc: Radu Nicolau, Kai Ji, Pablo de Lara
Add support for
NEA4, NIA4, NCA4: Snow 5G confidentiality, integrity and AEAD modes
NEA5, NIA5, NCA5: AES 256 confidentiality, integrity and AEAD modes
NEA6, NIA6, NCA6: ZUC 256 confidentiality, integrity and AEAD modes
IPsec MB library version 3.0 is required for the above algorithms.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
doc/guides/cryptodevs/aesni_mb.rst | 9 +
doc/guides/cryptodevs/features/aesni_mb.ini | 9 +
doc/guides/rel_notes/release_26_11.rst | 5 +
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 216 +++++++++++++++++-
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 229 ++++++++++++++++++++
5 files changed, 467 insertions(+), 1 deletion(-)
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index 4851997fd9..5fb73e31b4 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -39,6 +39,9 @@ Cipher algorithms:
* RTE_CRYPTO_CIPHER_SM4_CBC
* RTE_CRYPTO_CIPHER_SM4_ECB
* RTE_CRYPTO_CIPHER_SM4_CTR
+* RTE_CRYPTO_CIPHER_SNOW5G_NEA4
+* RTE_CRYPTO_CIPHER_AES_NEA5
+* RTE_CRYPTO_CIPHER_ZUC_NEA6
Hash algorithms:
@@ -61,6 +64,9 @@ Hash algorithms:
* RTE_CRYPTO_AUTH_KASUMI_F9
* RTE_CRYPTO_AUTH_SM3
* RTE_CRYPTO_AUTH_SM3 HMAC
+* RTE_CRYPTO_AUTH_SNOW5G_NIA4
+* RTE_CRYPTO_AUTH_AES_NIA5
+* RTE_CRYPTO_AUTH_ZUC_NIA6
AEAD algorithms:
@@ -68,6 +74,9 @@ AEAD algorithms:
* RTE_CRYPTO_AEAD_AES_GCM
* RTE_CRYPTO_AEAD_CHACHA20_POLY1305
* RTE_CRYPTO_AEAD_SM4_GCM
+* RTE_CRYPTO_AEAD_SNOW5G_NCA4
+* RTE_CRYPTO_AEAD_AES_NCA5
+* RTE_CRYPTO_AEAD_ZUC_NCA6
Protocol offloads:
diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini
index ce6b43e48b..e64818bf46 100644
--- a/doc/guides/cryptodevs/features/aesni_mb.ini
+++ b/doc/guides/cryptodevs/features/aesni_mb.ini
@@ -45,6 +45,9 @@ KASUMI F8 = Y
SM4 CBC = Y
SM4 ECB = Y
SM4 CTR = Y
+SNOW5G NEA4 = Y
+AES NEA5 = Y
+ZUC-256 NEA6 = Y
;
; Supported authentication algorithms of the 'aesni_mb' crypto driver.
@@ -69,6 +72,9 @@ SNOW3G UIA2 = Y
KASUMI F9 = Y
SM3 = Y
SM3 HMAC = Y
+SNOW5G NIA4 = Y
+AES NIA5 = Y
+ZUC-256 NIA6 = Y
;
; Supported AEAD algorithms of the 'aesni_mb' crypto driver.
@@ -81,6 +87,9 @@ AES GCM (192) = Y
AES GCM (256) = Y
CHACHA20-POLY1305 = Y
SM4 GCM = Y
+SNOW5G NCA4 = Y
+AES NCA5 = Y
+ZUC-256 NCA6 = Y
;
; Supported Asymmetric algorithms of the 'aesni_mb' crypto driver.
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index ea126aa9bb..1ee3cdfba3 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -36,8 +36,13 @@ New Features
* **Updated AESNI_MB crypto driver.**
* Added support for version 3.0 of IPsec MB Library.
+ * Added support for the following wireless algorithms:
+ - NEA4, NIA4, NCA4: Snow 5G confidentiality, integrity and AEAD modes.
+ - NEA5, NIA5, NCA5: AES 256 confidentiality, integrity and AEAD modes.
+ - NEA6, NIA6, NCA6: ZUC 256 confidentiality, integrity and AEAD modes.
.. This section should contain new features added in this release.
+
Sample format:
* **Add a title in the past tense with a full stop.**
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
index e8f6e77d21..a49a82d561 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
@@ -22,6 +22,11 @@ is_aead_algo(IMB_HASH_ALG hash_alg, IMB_CIPHER_MODE cipher_mode)
hash_alg == IMB_AUTH_AES_CCM ||
cipher_mode == IMB_CIPHER_GCM ||
cipher_mode == IMB_CIPHER_SM4_GCM
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ || cipher_mode == IMB_CIPHER_SNOW5G_NCA4
+ || cipher_mode == IMB_CIPHER_AES_NCA5
+ || cipher_mode == IMB_CIPHER_ZUC_NCA6
+#endif
);
}
@@ -221,6 +226,36 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
return 0;
}
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ if (xform->auth.algo == RTE_CRYPTO_AUTH_SNOW5G_NIA4) {
+ sess->template_job.hash_alg = IMB_AUTH_SNOW5G_NIA4;
+ sess->template_job.auth_tag_output_len_in_bytes =
+ sess->auth.req_digest_len;
+ memcpy(sess->auth.snow5g_auth_key, xform->auth.key.data,
+ xform->auth.key.length);
+ sess->template_job.u.NIA._key = sess->auth.snow5g_auth_key;
+ return 0;
+ } else if (xform->auth.algo == RTE_CRYPTO_AUTH_AES_NIA5) {
+ sess->template_job.hash_alg = IMB_AUTH_AES_NIA5;
+ sess->template_job.auth_tag_output_len_in_bytes =
+ sess->auth.req_digest_len;
+ IMB_AES_KEYEXP_256(mb_mgr, xform->auth.key.data,
+ sess->cipher.expanded_aes_keys.encode,
+ sess->cipher.expanded_aes_keys.decode);
+ sess->template_job.u.NIA._key =
+ (uint8_t *)sess->cipher.expanded_aes_keys.encode;
+ return 0;
+ } else if (xform->auth.algo == RTE_CRYPTO_AUTH_ZUC_NIA6) {
+ sess->template_job.hash_alg = IMB_AUTH_ZUC_NIA6;
+ sess->template_job.auth_tag_output_len_in_bytes =
+ sess->auth.req_digest_len;
+ memcpy(sess->auth.zuc_auth_key, xform->auth.key.data,
+ xform->auth.key.length);
+ sess->template_job.u.ZUC_EIA3._key = sess->auth.zuc_auth_key;
+ return 0;
+ }
+#endif
+
switch (xform->auth.algo) {
case RTE_CRYPTO_AUTH_MD5_HMAC:
sess->template_job.hash_alg = IMB_AUTH_MD5;
@@ -352,6 +387,9 @@ aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr,
uint8_t is_snow3g = 0;
uint8_t is_kasumi = 0;
uint8_t is_sm4 = 0;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ uint8_t is_nxan = 0;
+#endif
if (xform == NULL) {
sess->template_job.cipher_mode = IMB_CIPHER_NULL;
@@ -434,6 +472,20 @@ aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr,
sess->template_job.cipher_mode = IMB_CIPHER_SM4_CNTR;
is_sm4 = 1;
break;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ case RTE_CRYPTO_CIPHER_SNOW5G_NEA4:
+ sess->template_job.cipher_mode = IMB_CIPHER_SNOW5G_NEA4;
+ is_nxan = 1;
+ break;
+ case RTE_CRYPTO_CIPHER_AES_NEA5:
+ sess->template_job.cipher_mode = IMB_CIPHER_AES_NEA5;
+ is_nxan = 1;
+ break;
+ case RTE_CRYPTO_CIPHER_ZUC_NEA6:
+ sess->template_job.cipher_mode = IMB_CIPHER_ZUC_NEA6;
+ is_nxan = 1;
+ break;
+#endif
default:
IPSEC_MB_LOG(ERR, "Unsupported cipher mode parameter");
return -ENOTSUP;
@@ -578,6 +630,23 @@ aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr,
sess->cipher.expanded_sm4_keys.decode);
sess->template_job.enc_keys = sess->cipher.expanded_sm4_keys.encode;
sess->template_job.dec_keys = sess->cipher.expanded_sm4_keys.decode;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ } else if (is_nxan) {
+ if (xform->cipher.key.length != 32) {
+ IPSEC_MB_LOG(ERR, "Invalid cipher key length");
+ return -EINVAL;
+ }
+ if (xform->cipher.iv.length != 16) {
+ IPSEC_MB_LOG(ERR, "Invalid cipher IV length");
+ return -EINVAL;
+ }
+ sess->template_job.key_len_in_bytes = 32;
+ IMB_AES_KEYEXP_256(mb_mgr, xform->cipher.key.data,
+ sess->cipher.expanded_aes_keys.encode,
+ sess->cipher.expanded_aes_keys.decode);
+ sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode;
+ sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode;
+#endif
} else {
if (xform->cipher.key.length != 8) {
IPSEC_MB_LOG(ERR, "Invalid cipher key length");
@@ -731,6 +800,71 @@ aesni_mb_set_session_aead_parameters(IMB_MGR *mb_mgr,
sess->template_job.enc_keys = &sess->cipher.gcm_key;
sess->template_job.dec_keys = &sess->cipher.gcm_key;
break;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ case RTE_CRYPTO_AEAD_SNOW5G_NCA4:
+ sess->template_job.cipher_mode = IMB_CIPHER_SNOW5G_NCA4;
+ sess->template_job.hash_alg = IMB_AUTH_SNOW5G_NCA4;
+ sess->template_job.u.GCM.aad_len_in_bytes = xform->aead.aad_length;
+ if (xform->aead.key.length != 32) {
+ IPSEC_MB_LOG(ERR, "Invalid key length");
+ return -EINVAL;
+ }
+ sess->template_job.key_len_in_bytes = 32;
+ IMB_AES_KEYEXP_256(mb_mgr, xform->aead.key.data,
+ sess->cipher.expanded_aes_keys.encode,
+ sess->cipher.expanded_aes_keys.decode);
+ sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode;
+ sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode;
+ /* digest size must be between 4 and 16 */
+ if (sess->auth.req_digest_len < 4 ||
+ sess->auth.req_digest_len > 16) {
+ IPSEC_MB_LOG(ERR, "Invalid digest size");
+ return -EINVAL;
+ }
+ break;
+ case RTE_CRYPTO_AEAD_AES_NCA5:
+ sess->template_job.cipher_mode = IMB_CIPHER_AES_NCA5;
+ sess->template_job.hash_alg = IMB_AUTH_AES_NCA5;
+ sess->template_job.u.GCM.aad_len_in_bytes = xform->aead.aad_length;
+ if (xform->aead.key.length != 32) {
+ IPSEC_MB_LOG(ERR, "Invalid key length");
+ return -EINVAL;
+ }
+ sess->template_job.key_len_in_bytes = 32;
+ IMB_AES_KEYEXP_256(mb_mgr, xform->aead.key.data,
+ sess->cipher.expanded_aes_keys.encode,
+ sess->cipher.expanded_aes_keys.decode);
+ sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode;
+ sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode;
+ /* digest size must be between 4 and 16 */
+ if (sess->auth.req_digest_len < 4 ||
+ sess->auth.req_digest_len > 16) {
+ IPSEC_MB_LOG(ERR, "Invalid digest size");
+ return -EINVAL;
+ }
+ break;
+ case RTE_CRYPTO_AEAD_ZUC_NCA6:
+ sess->template_job.cipher_mode = IMB_CIPHER_ZUC_NCA6;
+ sess->template_job.hash_alg = IMB_AUTH_ZUC_NCA6;
+ sess->template_job.u.NCA.aad_len_in_bytes = xform->aead.aad_length;
+ if (xform->aead.key.length != 32) {
+ IPSEC_MB_LOG(ERR, "Invalid key length");
+ return -EINVAL;
+ }
+ sess->template_job.key_len_in_bytes = 32;
+ IMB_AES_KEYEXP_256(mb_mgr, xform->aead.key.data,
+ sess->cipher.expanded_aes_keys.encode,
+ sess->cipher.expanded_aes_keys.decode);
+ sess->template_job.enc_keys = sess->cipher.expanded_aes_keys.encode;
+ sess->template_job.dec_keys = sess->cipher.expanded_aes_keys.decode;
+ /* digest size must be between 4 and 16 */
+ if (sess->auth.req_digest_len < 4 ||
+ sess->auth.req_digest_len > 16) {
+ IPSEC_MB_LOG(ERR, "Invalid digest size");
+ return -EINVAL;
+ }
+ break;
+#endif
default:
IPSEC_MB_LOG(ERR, "Unsupported aead mode parameter");
return -ENOTSUP;
@@ -1060,7 +1194,24 @@ set_cpu_mb_job_params(IMB_JOB *job, struct aesni_mb_session *session,
case IMB_AUTH_SM4_GCM:
job->u.GCM.aad = aad->va;
break;
-
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ case IMB_AUTH_SNOW5G_NCA4:
+ case IMB_AUTH_AES_NCA5:
+ job->u.GCM.aad = aad->va;
+ break;
+ case IMB_AUTH_ZUC_NCA6:
+ job->u.NCA.aad = aad->va;
+ break;
+ case IMB_AUTH_SNOW5G_NIA4:
+ job->u.NIA._iv = iv->va;
+ break;
+ case IMB_AUTH_AES_NIA5:
+ job->u.NIA._iv = iv->va;
+ break;
+ case IMB_AUTH_ZUC_NIA6:
+ job->u.ZUC_EIA3._iv = iv->va;
+ break;
+#endif
default:
break;
@@ -1235,8 +1386,17 @@ handle_sgl_linear(IMB_JOB *job, struct rte_crypto_op *op, uint32_t dst_offset,
job->dst = linear_buf + dst_offset;
job->user_data2 = linear_buf;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ if (job->hash_alg == IMB_AUTH_AES_GMAC ||
+ job->hash_alg == IMB_AUTH_SNOW5G_NCA4 ||
+ job->hash_alg == IMB_AUTH_AES_NCA5)
+ job->u.GCM.aad = linear_buf;
+ else if (job->hash_alg == IMB_AUTH_ZUC_NCA6)
+ job->u.NCA.aad = linear_buf;
+#else
if (job->hash_alg == IMB_AUTH_AES_GMAC)
job->u.GCM.aad = linear_buf;
+#endif
if (session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY)
job->auth_tag_output = linear_buf + lb_offset;
@@ -1620,6 +1780,28 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
case IMB_AUTH_SM4_GCM:
job->u.GCM.aad = op->sym->aead.aad.data;
break;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ case IMB_AUTH_SNOW5G_NCA4:
+ case IMB_AUTH_AES_NCA5:
+ job->u.GCM.aad = op->sym->aead.aad.data;
+ break;
+ case IMB_AUTH_ZUC_NCA6:
+ job->u.NCA.aad = op->sym->aead.aad.data;
+ break;
+ case IMB_AUTH_SNOW5G_NIA4:
+ job->u.NIA._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ session->auth_iv.offset);
+ break;
+ case IMB_AUTH_AES_NIA5:
+ job->u.NIA._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ session->auth_iv.offset);
+ break;
+ case IMB_AUTH_ZUC_NIA6:
+ job->u.ZUC_EIA3._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ session->auth_iv.offset);
+ break;
+#endif
+
default:
break;
}
@@ -1775,6 +1957,29 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
job->iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->iv.offset);
break;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ case IMB_AUTH_SNOW5G_NCA4:
+ case IMB_AUTH_AES_NCA5:
+ case IMB_AUTH_ZUC_NCA6:
+ job->hash_start_src_offset_in_bytes =
+ op->sym->aead.data.offset;
+ job->msg_len_to_hash_in_bytes =
+ op->sym->aead.data.length;
+ job->cipher_start_src_offset_in_bytes =
+ op->sym->aead.data.offset;
+ job->msg_len_to_cipher_in_bytes = op->sym->aead.data.length;
+ job->iv = rte_crypto_op_ctod_offset(op, uint8_t *,
+ session->iv.offset);
+ break;
+ case IMB_AUTH_SNOW5G_NIA4:
+ case IMB_AUTH_AES_NIA5:
+ case IMB_AUTH_ZUC_NIA6:
+ job->hash_start_src_offset_in_bytes =
+ op->sym->auth.data.offset >> 3;
+ job->msg_len_to_hash_in_bytes =
+ op->sym->auth.data.length >> 3;
+ break;
+#endif
default:
job->hash_start_src_offset_in_bytes = auth_start_offset(
@@ -1831,6 +2036,15 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
case IMB_CIPHER_SM4_GCM:
job->msg_len_to_cipher_in_bytes = op->sym->aead.data.length;
break;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ case IMB_CIPHER_SNOW5G_NCA4:
+ case IMB_CIPHER_AES_NCA5:
+ case IMB_CIPHER_ZUC_NCA6:
+ job->cipher_start_src_offset_in_bytes =
+ op->sym->aead.data.offset;
+ job->msg_len_to_cipher_in_bytes = op->sym->aead.data.length;
+ break;
+#endif
default:
job->cipher_start_src_offset_in_bytes =
op->sym->cipher.data.offset;
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
index 6afda45414..9390bad208 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
@@ -882,6 +882,233 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
}, }
}, }
},
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ { /* 256-NEA4 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+ {.cipher = {
+ .algo = RTE_CRYPTO_CIPHER_SNOW5G_NEA4,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+ { /* 256-NEA5 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+ {.cipher = {
+ .algo = RTE_CRYPTO_CIPHER_AES_NEA5,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+ { /* 256-NEA6 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+ {.cipher = {
+ .algo = RTE_CRYPTO_CIPHER_ZUC_NEA6,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+ { /* 256-NIA4 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+ {.auth = {
+ .algo = RTE_CRYPTO_AUTH_SNOW5G_NIA4,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 4,
+ .max = 16,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+ { /* 256-NIA5 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+ {.auth = {
+ .algo = RTE_CRYPTO_AUTH_AES_NIA5,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 4,
+ .max = 16,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+ { /* 256-NIA6 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+ {.auth = {
+ .algo = RTE_CRYPTO_AUTH_ZUC_NIA6,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 4,
+ .max = 16,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+ { /* 256-NCA4 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+ {.aead = {
+ .algo = RTE_CRYPTO_AEAD_SNOW5G_NCA4,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 4,
+ .max = 16,
+ .increment = 1
+ },
+ .aad_size = {
+ .min = 0,
+ .max = 65535,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ },
+ }, }
+ }, }
+ },
+ { /* 256-NCA5 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+ {.aead = {
+ .algo = RTE_CRYPTO_AEAD_AES_NCA5,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 4,
+ .max = 16,
+ .increment = 1
+ },
+ .aad_size = {
+ .min = 0,
+ .max = 65535,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ },
+ }, }
+ }, }
+ },
+ { /* 256-NCA6 */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AEAD,
+ {.aead = {
+ .algo = RTE_CRYPTO_AEAD_ZUC_NCA6,
+ .block_size = 8,
+ .key_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 4,
+ .max = 16,
+ .increment = 1
+ },
+ .aad_size = {
+ .min = 0,
+ .max = 65535,
+ .increment = 1
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ },
+ }, }
+ }, }
+ },
+#endif
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
};
@@ -1101,6 +1328,8 @@ struct __rte_cache_aligned aesni_mb_session {
/* *< Expanded XCBC authentication keys */
uint8_t zuc_auth_key[32];
/* *< ZUC authentication key */
+ uint8_t snow5g_auth_key[32];
+ /* *< SNOW5G authentication key */
snow3g_key_schedule_t pKeySched_snow3g_auth;
/* *< SNOW3G scheduled authentication key */
kasumi_key_sched_t pKeySched_kasumi_auth;
--
2.52.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA
2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau
2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau
2026-10-02 9:56 ` [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms Radu Nicolau
@ 2026-10-02 9:56 ` Radu Nicolau
2026-10-02 9:56 ` [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB Radu Nicolau
2026-10-06 13:08 ` [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Ji, Kai
4 siblings, 0 replies; 7+ messages in thread
From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw)
To: dev; +Cc: Radu Nicolau, Kai Ji, Pablo de Lara
Enable basic support for asymmetric crypto operations.
Add support for ML-KEM and ML-DSA algorithms.
IPsecMB release 3.0 and newer required.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
doc/guides/cryptodevs/aesni_mb.rst | 6 +
doc/guides/cryptodevs/features/aesni_mb.ini | 3 +
doc/guides/rel_notes/release_26_11.rst | 3 +
drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 58 +++
drivers/crypto/ipsec_mb/ipsec_mb_private.h | 20 ++
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 372 +++++++++++++++++++-
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 86 ++++-
7 files changed, 532 insertions(+), 16 deletions(-)
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index 5fb73e31b4..395bdc77da 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -78,6 +78,12 @@ AEAD algorithms:
* RTE_CRYPTO_AEAD_AES_NCA5
* RTE_CRYPTO_AEAD_ZUC_NCA6
+PQC Asymmetric Crypto algorithms
+
+* RTE_CRYPTO_ASYM_XFORM_ML_KEM
+* RTE_CRYPTO_ASYM_XFORM_ML_DSA
+
+
Protocol offloads:
* RTE_SECURITY_PROTOCOL_DOCSIS
diff --git a/doc/guides/cryptodevs/features/aesni_mb.ini b/doc/guides/cryptodevs/features/aesni_mb.ini
index e64818bf46..c6a28987dd 100644
--- a/doc/guides/cryptodevs/features/aesni_mb.ini
+++ b/doc/guides/cryptodevs/features/aesni_mb.ini
@@ -21,6 +21,7 @@ CPU crypto = Y
Symmetric sessionless = Y
Non-Byte aligned data = Y
Digest encrypted = Y
+Asymmetric crypto = Y
;
; Supported crypto algorithms of the 'aesni_mb' crypto driver.
@@ -95,6 +96,8 @@ ZUC-256 NCA6 = Y
; Supported Asymmetric algorithms of the 'aesni_mb' crypto driver.
;
[Asymmetric]
+ML-KEM = Y
+ML-DSA = Y
;
; Supported Operating systems of the 'aesni_mb' crypto driver.
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 1ee3cdfba3..7dd8231706 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -40,6 +40,9 @@ New Features
- NEA4, NIA4, NCA4: Snow 5G confidentiality, integrity and AEAD modes.
- NEA5, NIA5, NCA5: AES 256 confidentiality, integrity and AEAD modes.
- NEA6, NIA6, NCA6: ZUC 256 confidentiality, integrity and AEAD modes.
+ * Added support for the following PQC asymmetric algorithms:
+ - ML-KEM: Module-Lattice-Based Key-Encapsulation Mechanism.
+ - ML-DSA: Module-Lattice-Based Digital Signature Algorithm.
.. This section should contain new features added in this release.
diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_ops.c b/drivers/crypto/ipsec_mb/ipsec_mb_ops.c
index 50ee140ccd..308775882c 100644
--- a/drivers/crypto/ipsec_mb/ipsec_mb_ops.c
+++ b/drivers/crypto/ipsec_mb/ipsec_mb_ops.c
@@ -453,3 +453,61 @@ void
ipsec_mb_sym_session_clear(struct rte_cryptodev *dev __rte_unused,
struct rte_cryptodev_sym_session *sess __rte_unused)
{}
+
+/** Configure pmd specific multi-buffer asym session from a crypto xform chain */
+int
+ipsec_mb_asym_session_configure(
+ struct rte_cryptodev *dev, struct rte_crypto_asym_xform *xform,
+ struct rte_cryptodev_asym_session *sess)
+{
+ struct ipsec_mb_dev_private *internals = dev->data->dev_private;
+ struct ipsec_mb_internals *pmd_data =
+ &ipsec_mb_pmds[internals->pmd_type];
+ struct ipsec_mb_qp *qp = dev->data->queue_pairs[0];
+ IMB_MGR *mb_mgr;
+ int ret = 0;
+
+ if (qp != NULL)
+ mb_mgr = qp->mb_mgr;
+ else
+ mb_mgr = alloc_init_mb_mgr();
+
+ if (!mb_mgr)
+ return -ENOMEM;
+
+ if (unlikely(sess == NULL)) {
+ IPSEC_MB_LOG(ERR, "invalid session struct");
+ if (qp == NULL)
+ free_mb_mgr(mb_mgr);
+ return -EINVAL;
+ }
+
+ ret = (*pmd_data->asym_session_configure)(mb_mgr,
+ CRYPTODEV_GET_ASYM_SESS_PRIV(sess), xform);
+ if (ret != 0) {
+ IPSEC_MB_LOG(ERR, "failed configure asym session parameters");
+
+ /* Return session to mempool */
+ if (qp == NULL)
+ free_mb_mgr(mb_mgr);
+ return ret;
+ }
+
+ if (qp == NULL)
+ free_mb_mgr(mb_mgr);
+
+ pmd_data->asym_session_created = true;
+ return 0;
+}
+
+/** Clear the asym session memory */
+void
+ipsec_mb_asym_session_clear(struct rte_cryptodev *dev __rte_unused,
+ struct rte_cryptodev_asym_session *sess)
+{
+ struct ipsec_mb_dev_private *internals = dev->data->dev_private;
+ struct ipsec_mb_internals *pmd_data =
+ &ipsec_mb_pmds[internals->pmd_type];
+
+ (*pmd_data->asym_session_clear)(CRYPTODEV_GET_ASYM_SESS_PRIV(sess));
+}
diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_private.h b/drivers/crypto/ipsec_mb/ipsec_mb_private.h
index fc7d6f5f06..5509dc21f3 100644
--- a/drivers/crypto/ipsec_mb/ipsec_mb_private.h
+++ b/drivers/crypto/ipsec_mb/ipsec_mb_private.h
@@ -356,6 +356,12 @@ typedef int (*ipsec_mb_session_configure_t)(IMB_MGR *mbr_mgr,
void *session_private,
const struct rte_crypto_sym_xform *xform);
+typedef int (*ipsec_mb_asym_session_configure_t)(IMB_MGR *mbr_mgr,
+ void *session_private,
+ const struct rte_crypto_asym_xform *xform);
+
+typedef void (*ipsec_mb_asym_session_clear_t)(void *session_private);
+
/** Configure internals PMD type specific data */
typedef int (*ipsec_mb_dev_configure_t)(struct rte_cryptodev *dev);
@@ -366,6 +372,9 @@ struct ipsec_mb_internals {
ipsec_mb_dev_configure_t dev_config;
ipsec_mb_queue_pair_configure_t queue_pair_configure;
ipsec_mb_session_configure_t session_configure;
+ ipsec_mb_asym_session_configure_t asym_session_configure;
+ ipsec_mb_asym_session_clear_t asym_session_clear;
+ bool asym_session_created;
const struct rte_cryptodev_capabilities *caps;
struct rte_cryptodev_ops *ops;
struct rte_security_ops *security_ops;
@@ -423,11 +432,22 @@ int ipsec_mb_sym_session_configure(
struct rte_crypto_sym_xform *xform,
struct rte_cryptodev_sym_session *sess);
+/** Configure an aesni multi-buffer asym session from a crypto xform chain */
+int ipsec_mb_asym_session_configure(
+ struct rte_cryptodev *dev,
+ struct rte_crypto_asym_xform *xform,
+ struct rte_cryptodev_asym_session *sess);
+
/** Clear the memory of session so it does not leave key material behind */
void
ipsec_mb_sym_session_clear(struct rte_cryptodev *dev,
struct rte_cryptodev_sym_session *sess);
+/** Clear the asym session memory */
+void
+ipsec_mb_asym_session_clear(struct rte_cryptodev *dev,
+ struct rte_cryptodev_asym_session *sess);
+
/** Get session from op. If sessionless create a session */
static __rte_always_inline void *
ipsec_mb_get_session_private(struct ipsec_mb_qp *qp, struct rte_crypto_op *op)
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
index a49a82d561..ab65316978 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
@@ -874,7 +874,7 @@ aesni_mb_set_session_aead_parameters(IMB_MGR *mb_mgr,
}
/** Configure a aesni multi-buffer session from a crypto xform chain */
-int
+static int
aesni_mb_session_configure(IMB_MGR *mb_mgr,
void *priv_sess,
const struct rte_crypto_sym_xform *xform)
@@ -965,6 +965,97 @@ aesni_mb_session_configure(IMB_MGR *mb_mgr,
return 0;
}
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+/** Configure a aesni multi-buffer asym session from a crypto xform chain */
+static int
+aesni_mb_asym_session_configure(IMB_MGR *mb_mgr,
+ void *priv_sess,
+ const struct rte_crypto_asym_xform *xform)
+{
+ (void)mb_mgr;
+ struct aesni_mb_session *sess = (struct aesni_mb_session *) priv_sess;
+ int ret = 0;
+
+ sess->asym.xform_type = xform->xform_type;
+
+ switch (sess->asym.xform_type) {
+ case RTE_CRYPTO_ASYM_XFORM_ML_KEM:
+ {
+ switch (xform->mlkem.type) {
+ case RTE_CRYPTO_ML_KEM_512:
+ sess->asym.ml_kem.alg = IMB_ML_KEM_512;
+ break;
+ case RTE_CRYPTO_ML_KEM_768:
+ sess->asym.ml_kem.alg = IMB_ML_KEM_768;
+ break;
+ case RTE_CRYPTO_ML_KEM_1024:
+ sess->asym.ml_kem.alg = IMB_ML_KEM_1024;
+ break;
+ default:
+ ret = -ENOTSUP;
+ IPSEC_MB_LOG(ERR, "Unsupported type");
+ return ret;
+ }
+ ret = imb_ml_kem_new(mb_mgr, sess->asym.ml_kem.alg, &sess->asym.ml_kem.ctx);
+ if (ret) {
+ IPSEC_MB_LOG(ERR, "Failed to create ML KEM session");
+ return ret;
+ }
+ IMB_ML_KEM_KEYGEN_PARAMS_INIT(&sess->asym.ml_kem.keygen_params);
+ IMB_ML_KEM_ENCAP_PARAMS_INIT(&sess->asym.ml_kem.encap_params);
+ }
+ break;
+ case RTE_CRYPTO_ASYM_XFORM_ML_DSA:
+ {
+ switch (xform->mldsa.type) {
+ case RTE_CRYPTO_ML_DSA_44:
+ sess->asym.ml_dsa.alg = IMB_ML_DSA_44;
+ break;
+ case RTE_CRYPTO_ML_DSA_65:
+ sess->asym.ml_dsa.alg = IMB_ML_DSA_65;
+ break;
+ case RTE_CRYPTO_ML_DSA_87:
+ sess->asym.ml_dsa.alg = IMB_ML_DSA_87;
+ break;
+ default:
+ ret = -ENOTSUP;
+ IPSEC_MB_LOG(ERR, "Unsupported type");
+ return ret;
+ }
+ ret = imb_ml_dsa_new(mb_mgr, sess->asym.ml_dsa.alg, &sess->asym.ml_dsa.ctx);
+ if (ret) {
+ IPSEC_MB_LOG(ERR, "Failed to create ML DSA session");
+ return ret;
+ }
+ IMB_ML_DSA_KEYGEN_PARAMS_INIT(&sess->asym.ml_dsa.keygen_params);
+ IMB_ML_DSA_SIGN_PARAMS_INIT(&sess->asym.ml_dsa.sign_params);
+ IMB_ML_DSA_VERIFY_PARAMS_INIT(&sess->asym.ml_dsa.verify_params);
+ }
+ break;
+ default:
+ ret = -ENOTSUP;
+ IPSEC_MB_LOG(ERR, "Unsupported algorithm");
+ return ret;
+ };
+
+ return ret;
+}
+
+static void
+aesni_mb_asym_session_clear(void *priv_sess)
+{
+ struct aesni_mb_session *sess = (struct aesni_mb_session *) priv_sess;
+ if (sess->asym.xform_type == RTE_CRYPTO_ASYM_XFORM_ML_KEM) {
+ imb_ml_kem_free(sess->asym.ml_kem.ctx);
+ sess->asym.ml_kem.ctx = NULL;
+ } else if (sess->asym.xform_type == RTE_CRYPTO_ASYM_XFORM_ML_DSA) {
+ imb_ml_dsa_free(sess->asym.ml_dsa.ctx);
+ sess->asym.ml_dsa.ctx = NULL;
+ }
+}
+
+#endif
+
/** Check DOCSIS security session configuration is valid */
static int
check_docsis_sec_session(struct rte_security_session_conf *conf)
@@ -2391,7 +2482,237 @@ set_job_null_op(IMB_JOB *job, struct rte_crypto_op *op)
return job;
}
-uint16_t
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+static inline size_t aesni_mb_ml_kem_ek_sz(const IMB_ML_KEM_ALG alg)
+{
+ switch (alg) {
+ case IMB_ML_KEM_512:
+ return IMB_ML_KEM_512_PUBKEY_BYTES;
+ case IMB_ML_KEM_768:
+ return IMB_ML_KEM_768_PUBKEY_BYTES;
+ case IMB_ML_KEM_1024:
+ return IMB_ML_KEM_1024_PUBKEY_BYTES;
+ }
+ return 0;
+}
+static inline size_t aesni_mb_ml_kem_dk_sz(const IMB_ML_KEM_ALG alg)
+{
+ switch (alg) {
+ case IMB_ML_KEM_512:
+ return IMB_ML_KEM_512_PRIVKEY_BYTES;
+ case IMB_ML_KEM_768:
+ return IMB_ML_KEM_768_PRIVKEY_BYTES;
+ case IMB_ML_KEM_1024:
+ return IMB_ML_KEM_1024_PRIVKEY_BYTES;
+ }
+ return 0;
+}
+static inline size_t aesni_mb_ml_kem_ct_sz(const IMB_ML_KEM_ALG alg)
+{
+ switch (alg) {
+ case IMB_ML_KEM_512:
+ return IMB_ML_KEM_512_CIPHERTEXT_BYTES;
+ case IMB_ML_KEM_768:
+ return IMB_ML_KEM_768_CIPHERTEXT_BYTES;
+ case IMB_ML_KEM_1024:
+ return IMB_ML_KEM_1024_CIPHERTEXT_BYTES;
+ }
+ return 0;
+}
+static inline size_t aesni_mb_ml_dsa_pk_sz(const IMB_ML_DSA_ALG alg)
+{
+ switch (alg) {
+ case IMB_ML_DSA_44:
+ return IMB_ML_DSA_44_PUBKEY_BYTES;
+ case IMB_ML_DSA_65:
+ return IMB_ML_DSA_65_PUBKEY_BYTES;
+ case IMB_ML_DSA_87:
+ return IMB_ML_DSA_87_PUBKEY_BYTES;
+ }
+ return 0;
+}
+static inline size_t aesni_mb_ml_dsa_sk_sz(const IMB_ML_DSA_ALG alg)
+{
+ switch (alg) {
+ case IMB_ML_DSA_44:
+ return IMB_ML_DSA_44_PRIVKEY_BYTES;
+ case IMB_ML_DSA_65:
+ return IMB_ML_DSA_65_PRIVKEY_BYTES;
+ case IMB_ML_DSA_87:
+ return IMB_ML_DSA_87_PRIVKEY_BYTES;
+ }
+ return 0;
+}
+static inline size_t aesni_mb_ml_dsa_sig_sz(const IMB_ML_DSA_ALG alg)
+{
+ switch (alg) {
+ case IMB_ML_DSA_44:
+ return IMB_ML_DSA_44_SIG_BYTES;
+ case IMB_ML_DSA_65:
+ return IMB_ML_DSA_65_SIG_BYTES;
+ case IMB_ML_DSA_87:
+ return IMB_ML_DSA_87_SIG_BYTES;
+ }
+ return 0;
+}
+
+static void
+aesni_mb_asym_process(struct rte_crypto_op *op)
+{
+ struct rte_crypto_asym_op *aop = op->asym;
+ struct aesni_mb_session *sess =
+ CRYPTODEV_GET_ASYM_SESS_PRIV(aop->session);
+ op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
+ int rc = 0;
+
+ switch (sess->asym.xform_type) {
+ case RTE_CRYPTO_ASYM_XFORM_ML_KEM:
+ {
+ struct rte_crypto_ml_kem_op *mlkem = &aop->mlkem;
+ switch (mlkem->op) {
+ case RTE_CRYPTO_ML_KEM_OP_KEYGEN:
+ {
+ uint8_t seed_d_z[64];
+ struct rte_crypto_ml_kem_keygen_op *keygen = &mlkem->keygen;
+ memcpy(seed_d_z, keygen->d.data, 32);
+ memcpy(&seed_d_z[32], keygen->z.data, 32);
+ sess->asym.ml_kem.keygen_params.seed_d_z = seed_d_z;
+ sess->asym.ml_kem.keygen_params.seed_d_z_len = 64;
+ keygen->ek.length = aesni_mb_ml_kem_ek_sz(sess->asym.ml_kem.alg);
+ keygen->dk.length = aesni_mb_ml_kem_dk_sz(sess->asym.ml_kem.alg);
+ rc = imb_ml_kem_keypair(sess->asym.ml_kem.ctx,
+ keygen->ek.data, keygen->ek.length,
+ keygen->dk.data, keygen->dk.length,
+ &sess->asym.ml_kem.keygen_params);
+ }
+ break;
+ case RTE_CRYPTO_ML_KEM_OP_ENCAP:
+ {
+ struct rte_crypto_ml_kem_encap_op *encap = &mlkem->encap;
+ sess->asym.ml_kem.encap_params.m_32 = encap->message.data;
+ sess->asym.ml_kem.encap_params.m_len = encap->message.length;
+ encap->ek.length = aesni_mb_ml_kem_ek_sz(sess->asym.ml_kem.alg);
+ encap->cipher.length = aesni_mb_ml_kem_ct_sz(sess->asym.ml_kem.alg);
+ rc = imb_ml_kem_set_pubkey(sess->asym.ml_kem.ctx,
+ encap->ek.data, encap->ek.length);
+ if (rc == 0)
+ rc = imb_ml_kem_encap(sess->asym.ml_kem.ctx,
+ encap->cipher.data, encap->cipher.length,
+ encap->sk.data, encap->sk.length,
+ &sess->asym.ml_kem.encap_params);
+ }
+ break;
+ case RTE_CRYPTO_ML_KEM_OP_DECAP:
+ {
+ struct rte_crypto_ml_kem_decap_op *decap = &mlkem->decap;
+ decap->dk.length = aesni_mb_ml_kem_dk_sz(sess->asym.ml_kem.alg);
+ decap->cipher.length = aesni_mb_ml_kem_ct_sz(sess->asym.ml_kem.alg);
+ rc = imb_ml_kem_set_privkey(sess->asym.ml_kem.ctx,
+ decap->dk.data, decap->dk.length);
+ if (rc == 0)
+ rc = imb_ml_kem_decap(sess->asym.ml_kem.ctx,
+ decap->sk.data, decap->sk.length,
+ decap->cipher.data, decap->cipher.length,
+ NULL);
+ }
+ break;
+ default:
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ break;
+ };
+ }
+ break;
+ case RTE_CRYPTO_ASYM_XFORM_ML_DSA:
+ {
+ struct rte_crypto_ml_dsa_op *mldsa = &aop->mldsa;
+ switch (mldsa->op) {
+ case RTE_CRYPTO_ML_DSA_OP_KEYGEN:
+ {
+ struct rte_crypto_ml_dsa_keygen_op *keygen = &mldsa->keygen;
+ sess->asym.ml_dsa.keygen_params.xi_32 = keygen->seed.data;
+ sess->asym.ml_dsa.keygen_params.xi_len = keygen->seed.length;
+ keygen->pubkey.length =
+ aesni_mb_ml_dsa_pk_sz(sess->asym.ml_dsa.alg);
+ keygen->privkey.length =
+ aesni_mb_ml_dsa_sk_sz(sess->asym.ml_dsa.alg);
+ rc = imb_ml_dsa_keypair(sess->asym.ml_dsa.ctx,
+ keygen->pubkey.data, keygen->pubkey.length,
+ keygen->privkey.data, keygen->privkey.length,
+ &sess->asym.ml_dsa.keygen_params);
+ }
+ break;
+ case RTE_CRYPTO_ML_DSA_OP_SIGN:
+ {
+ struct rte_crypto_ml_dsa_siggen_op *siggen = &mldsa->siggen;
+ void *md;
+ size_t md_len;
+ sess->asym.ml_dsa.sign_params.ctx = siggen->ctx.data;
+ sess->asym.ml_dsa.sign_params.ctx_len = siggen->ctx.length;
+ sess->asym.ml_dsa.sign_params.rnd_32 = siggen->seed.data;
+ sess->asym.ml_dsa.sign_params.rnd_len = siggen->seed.length;
+ if (siggen->mu.length) {
+ md = siggen->mu.data;
+ md_len = siggen->mu.length;
+ sess->asym.ml_dsa.sign_params.msg_is_mu = 1;
+ } else {
+ md = siggen->message.data;
+ md_len = siggen->message.length;
+ sess->asym.ml_dsa.sign_params.msg_is_mu = 0;
+ }
+ rc = imb_ml_dsa_set_privkey(sess->asym.ml_dsa.ctx,
+ siggen->privkey.data, siggen->privkey.length);
+ if (rc == 0)
+ rc = imb_ml_dsa_sign(sess->asym.ml_dsa.ctx,
+ siggen->sign.data, &siggen->sign.length,
+ md, md_len, &sess->asym.ml_dsa.sign_params);
+ }
+ break;
+ case RTE_CRYPTO_ML_DSA_OP_VERIFY:
+ {
+ struct rte_crypto_ml_dsa_sigver_op *sigver = &mldsa->sigver;
+ void *md;
+ size_t md_len;
+ sess->asym.ml_dsa.verify_params.ctx = sigver->ctx.data;
+ sess->asym.ml_dsa.verify_params.ctx_len = sigver->ctx.length;
+ if (sigver->mu.length) {
+ md = sigver->mu.data;
+ md_len = sigver->mu.length;
+ sess->asym.ml_dsa.verify_params.msg_is_mu = 1;
+ } else {
+ md = sigver->message.data;
+ md_len = sigver->message.length;
+ sess->asym.ml_dsa.verify_params.msg_is_mu = 0;
+ }
+ rc = imb_ml_dsa_set_pubkey(sess->asym.ml_dsa.ctx,
+ sigver->pubkey.data, sigver->pubkey.length);
+ if (rc == 0)
+ rc = imb_ml_dsa_verify(sess->asym.ml_dsa.ctx,
+ md, md_len, sigver->sign.data, sigver->sign.length,
+ &sess->asym.ml_dsa.verify_params);
+ }
+ break;
+ default:
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ break;
+ };
+
+ }
+ break;
+ default:
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ break;
+ }
+ if (op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED)
+ op->status = (rc == 0) ? RTE_CRYPTO_OP_STATUS_SUCCESS :
+ RTE_CRYPTO_OP_STATUS_ERROR;
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+ if (rc)
+ IPSEC_MB_LOG(ERR, "%s", imb_get_strerror(rc));
+#endif
+}
+#endif
+
+static uint16_t
aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
uint16_t nb_ops)
{
@@ -2456,8 +2777,38 @@ aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
* the IMB_MGR can use that pointer again in next
* get_next calls.
*/
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ {
+ struct ipsec_mb_internals *pmd_data =
+ &ipsec_mb_pmds[qp->pmd_type];
+ /* If there is at least one asym session created process
+ * the asym ops here and copy the sym ops to deqd_ops array
+ */
+ if (pmd_data->asym_session_created) {
+ struct rte_crypto_op *tmp_dq[IMB_MAX_BURST_SIZE];
+ uint16_t sym_cnt = 0;
+ nb_submit_ops = rte_ring_dequeue_burst(qp->ingress_queue,
+ (void **)tmp_dq, n, NULL);
+ for (i = 0; i < nb_submit_ops; i++) {
+ if (tmp_dq[i]->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) {
+ aesni_mb_asym_process(tmp_dq[i]);
+ ops[processed_jobs++] = tmp_dq[i];
+ } else {
+ deqd_ops[sym_cnt++] = tmp_dq[i];
+ }
+ }
+ /* What remains are only symmetric ops. */
+ nb_submit_ops = sym_cnt;
+ } else {
+ /* No asym session created, all ops are sym*/
+ nb_submit_ops = rte_ring_dequeue_burst(qp->ingress_queue,
+ (void **)deqd_ops, n, NULL);
+ }
+ }
+#else
nb_submit_ops = rte_ring_dequeue_burst(qp->ingress_queue,
(void **)deqd_ops, n, NULL);
+#endif
for (i = 0; i < nb_submit_ops; i++) {
job = jobs[i];
op = deqd_ops[i];
@@ -2587,7 +2938,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec,
return k;
}
-uint32_t
+static uint32_t
aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused,
struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
struct rte_crypto_sym_vec *vec)
@@ -2674,7 +3025,12 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {
.sym_session_get_size = ipsec_mb_sym_session_get_size,
.sym_session_configure = ipsec_mb_sym_session_configure,
- .sym_session_clear = ipsec_mb_sym_session_clear
+ .sym_session_clear = ipsec_mb_sym_session_clear,
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ .asym_session_get_size = ipsec_mb_sym_session_get_size,
+ .asym_session_configure = ipsec_mb_asym_session_configure,
+ .asym_session_clear = ipsec_mb_asym_session_clear,
+#endif
};
/**
@@ -2800,7 +3156,11 @@ RTE_INIT(ipsec_mb_register_aesni_mb)
RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO |
#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ /* removed in 3.0 */
RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA |
+#else
+ /* added in 3.0 */
+ RTE_CRYPTODEV_FF_ASYMMETRIC_CRYPTO |
#endif
RTE_CRYPTODEV_FF_SYM_SESSIONLESS |
RTE_CRYPTODEV_FF_IN_PLACE_SGL |
@@ -2817,5 +3177,9 @@ RTE_INIT(ipsec_mb_register_aesni_mb)
aesni_mb_data->security_ops = &aesni_mb_pmd_sec_ops;
aesni_mb_data->dev_config = aesni_mb_configure_dev;
aesni_mb_data->session_configure = aesni_mb_session_configure;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ aesni_mb_data->asym_session_configure = aesni_mb_asym_session_configure;
+ aesni_mb_data->asym_session_clear = aesni_mb_asym_session_clear;
+#endif
aesni_mb_data->session_priv_size = sizeof(struct aesni_mb_session);
}
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
index 9390bad208..99ca4e49e6 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
@@ -28,18 +28,6 @@
#define IMB_AUTH_ZUC256_EIA3 IMB_AUTH_ZUC256_EIA3_BITLEN
#endif
-int
-aesni_mb_session_configure(IMB_MGR * m __rte_unused, void *priv_sess,
- const struct rte_crypto_sym_xform *xform);
-
-uint16_t
-aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
- uint16_t nb_ops);
-
-uint32_t
-aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused,
- struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs,
- struct rte_crypto_sym_vec *vec);
static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
{ /* MD5 HMAC */
@@ -1108,6 +1096,60 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
}, }
}, }
},
+ {
+ /* ML-KEM */
+ .op = RTE_CRYPTO_OP_TYPE_ASYMMETRIC,
+ .asym = {
+ .xform_capa = {
+ .xform_type = RTE_CRYPTO_ASYM_XFORM_ML_KEM,
+ .op_types =
+ ((1 << RTE_CRYPTO_ML_KEM_OP_KEYGEN) |
+ (1 << RTE_CRYPTO_ML_KEM_OP_ENCAP) |
+ (1 << RTE_CRYPTO_ML_KEM_OP_DECAP)),
+ .mlkem_capa = {
+ [RTE_CRYPTO_ML_KEM_OP_KEYGEN] =
+ (1 << RTE_CRYPTO_ML_KEM_512) |
+ (1 << RTE_CRYPTO_ML_KEM_768) |
+ (1 << RTE_CRYPTO_ML_KEM_1024),
+ [RTE_CRYPTO_ML_KEM_OP_ENCAP] =
+ (1 << RTE_CRYPTO_ML_KEM_512) |
+ (1 << RTE_CRYPTO_ML_KEM_768) |
+ (1 << RTE_CRYPTO_ML_KEM_1024),
+ [RTE_CRYPTO_ML_KEM_OP_DECAP] =
+ (1 << RTE_CRYPTO_ML_KEM_512) |
+ (1 << RTE_CRYPTO_ML_KEM_768) |
+ (1 << RTE_CRYPTO_ML_KEM_1024)
+ }
+ }
+ }
+ },
+ {
+ /* ML-DSA */
+ .op = RTE_CRYPTO_OP_TYPE_ASYMMETRIC,
+ .asym = {
+ .xform_capa = {
+ .xform_type = RTE_CRYPTO_ASYM_XFORM_ML_DSA,
+ .op_types =
+ ((1 << RTE_CRYPTO_ML_DSA_OP_SIGN) |
+ (1 << RTE_CRYPTO_ML_DSA_OP_KEYGEN) |
+ (1 << RTE_CRYPTO_ML_DSA_OP_VERIFY)),
+ .mldsa_capa = {
+ [RTE_CRYPTO_ML_DSA_OP_KEYGEN] =
+ (1 << RTE_CRYPTO_ML_DSA_44) |
+ (1 << RTE_CRYPTO_ML_DSA_65) |
+ (1 << RTE_CRYPTO_ML_DSA_87),
+ [RTE_CRYPTO_ML_DSA_OP_SIGN] =
+ (1 << RTE_CRYPTO_ML_DSA_44) |
+ (1 << RTE_CRYPTO_ML_DSA_65) |
+ (1 << RTE_CRYPTO_ML_DSA_87),
+ [RTE_CRYPTO_ML_DSA_OP_VERIFY] =
+ (1 << RTE_CRYPTO_ML_DSA_44) |
+ (1 << RTE_CRYPTO_ML_DSA_65) |
+ (1 << RTE_CRYPTO_ML_DSA_87)
+ }
+ }
+ }
+ },
#endif
RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
};
@@ -1339,6 +1381,26 @@ struct __rte_cache_aligned aesni_mb_session {
uint16_t req_digest_len;
} auth;
+#if IMB_VERSION(3, 0, 0) <= IMB_VERSION_NUM
+ struct {
+ enum rte_crypto_asym_xform_type xform_type;
+ union {
+ struct {
+ IMB_ML_KEM *ctx;
+ IMB_ML_KEM_ALG alg;
+ IMB_ML_KEM_KEYGEN_PARAMS keygen_params;
+ IMB_ML_KEM_ENCAP_PARAMS encap_params;
+ } ml_kem;
+ struct {
+ IMB_ML_DSA *ctx;
+ IMB_ML_DSA_ALG alg;
+ IMB_ML_DSA_KEYGEN_PARAMS keygen_params;
+ IMB_ML_DSA_SIGN_PARAMS sign_params;
+ IMB_ML_DSA_VERIFY_PARAMS verify_params;
+ } ml_dsa;
+ };
+ } asym;
+ #endif
};
typedef void (*hash_one_block_t)(const void *data, void *digest);
--
2.52.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB
2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau
` (2 preceding siblings ...)
2026-10-02 9:56 ` [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA Radu Nicolau
@ 2026-10-02 9:56 ` Radu Nicolau
2026-10-06 13:08 ` [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Ji, Kai
4 siblings, 0 replies; 7+ messages in thread
From: Radu Nicolau @ 2026-10-02 9:56 UTC (permalink / raw)
To: dev; +Cc: Radu Nicolau, Akhil Goyal, Fan Zhang
Enable asymmetric crypto unit test suite for AESNI_MB PMD.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
app/test/test_cryptodev_asym.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 4f0408c87a..1870011610 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -6181,6 +6181,12 @@ test_cryptodev_virtio_user_asym(void)
return run_cryptodev_asym_testsuite(RTE_STR(CRYPTODEV_NAME_VIRTIO_USER_PMD));
}
+static int
+test_cryptodev_aesni_mb_asym(void)
+{
+ return run_cryptodev_asym_testsuite(RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+}
+
REGISTER_DRIVER_TEST(cryptodev_openssl_asym_autotest, test_cryptodev_openssl_asym);
REGISTER_DRIVER_TEST(cryptodev_qat_asym_autotest, test_cryptodev_qat_asym);
REGISTER_DRIVER_TEST(cryptodev_octeontx_asym_autotest, test_cryptodev_octeontx_asym);
@@ -6189,3 +6195,4 @@ REGISTER_DRIVER_TEST(cryptodev_cn10k_asym_autotest, test_cryptodev_cn10k_asym);
REGISTER_DRIVER_TEST(cryptodev_cn20k_asym_autotest, test_cryptodev_cn20k_asym);
REGISTER_DRIVER_TEST(cryptodev_virtio_asym_autotest, test_cryptodev_virtio_asym);
REGISTER_DRIVER_TEST(cryptodev_virtio_user_asym_autotest, test_cryptodev_virtio_user_asym);
+REGISTER_DRIVER_TEST(cryptodev_aesni_mb_asym_autotest, test_cryptodev_aesni_mb_asym);
--
2.52.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/4] crypto/ipsec_mb: add support for IPsecMB 3.0
2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau
@ 2026-10-06 12:06 ` Ji, Kai
0 siblings, 0 replies; 7+ messages in thread
From: Ji, Kai @ 2026-10-06 12:06 UTC (permalink / raw)
To: Nicolau, Radu, dev@dpdk.org; +Cc: De Lara Guarch, Pablo
[-- Attachment #1: Type: text/plain, Size: 45720 bytes --]
Series-acked-by: Kai Ji <kai.ji@intel.com>
________________________________
From: Nicolau, Radu <radu.nicolau@intel.com>
Sent: 02 October 2026 10:56
To: dev@dpdk.org <dev@dpdk.org>
Cc: Nicolau, Radu <radu.nicolau@intel.com>; Ji, Kai <kai.ji@intel.com>; De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
Subject: [PATCH 1/4] crypto/ipsec_mb: add support for IPsecMB 3.0
Add support for IPsecMB release 3.0
- removed all _BITLEN enums, all lengths are in bytes
- removed feature flag
- use non const IMB_MGR
- use IMB API instead of function pointers in the GCM PMD
- remove ZUC 256 support
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
---
doc/guides/cryptodevs/aesni_gcm.rst | 4 +-
doc/guides/cryptodevs/aesni_mb.rst | 11 +-
doc/guides/rel_notes/release_26_11.rst | 4 +
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++++++++++++----
drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +--
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 107 +++++---
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 47 +++-
7 files changed, 349 insertions(+), 122 deletions(-)
diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst
index aefc1cedcb..e68dc4eb1e 100644
--- a/doc/guides/cryptodevs/aesni_gcm.rst
+++ b/doc/guides/cryptodevs/aesni_gcm.rst
@@ -40,8 +40,8 @@ Installation
To build DPDK with the AESNI_GCM_PMD the user is required to download the multi-buffer
library from `here <https://github.com/01org/intel-ipsec-mb>`_
and compile it on their user system before building DPDK.
-The latest version of the library supported by this PMD is v2.0, which
-can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_.
+The latest version of the library supported by this PMD is v3.0, which
+can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_.
For Arm system, ARM64 port of the multi-buffer library can be downloaded from
https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/.
diff --git a/doc/guides/cryptodevs/aesni_mb.rst b/doc/guides/cryptodevs/aesni_mb.rst
index add3c03b74..4851997fd9 100644
--- a/doc/guides/cryptodevs/aesni_mb.rst
+++ b/doc/guides/cryptodevs/aesni_mb.rst
@@ -73,6 +73,13 @@ Protocol offloads:
* RTE_SECURITY_PROTOCOL_DOCSIS
+Note
+----
+Version 3.0 of the multi-buffer library removed support for:
+
+* the 256 bit versions of ZUC EEA3 and ZUC EIA3 algorithms.
+* non byte aligned bit length (``RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA`` removed).
+
Limitations
-----------
@@ -96,8 +103,8 @@ Installation
To build DPDK with the AESNI_MB_PMD the user is required to download the multi-buffer
library from `here <https://github.com/01org/intel-ipsec-mb>`_
and compile it on their user system before building DPDK.
-The latest version of the library supported by this PMD is v2.0, which
-can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v2.0.zip>`_.
+The latest version of the library supported by this PMD is v3.0, which
+can be downloaded from `<https://github.com/01org/intel-ipsec-mb/archive/v3.0.zip>`_.
For Arm system, ARM64 port of the multi-buffer library can be downloaded from
https://gitlab.arm.com/arm-reference-solutions/ipsec-mb/-/tree/main/.
diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst
index 69ab5af8a3..ea126aa9bb 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -33,6 +33,10 @@ New Features
``pss_salt`` field in ``rte_crypto_rsa_op_param`` to carry it per
sign operation.
+* **Updated AESNI_MB crypto driver.**
+
+ * Added support for version 3.0 of IPsec MB Library.
+
.. This section should contain new features added in this release.
Sample format:
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c
index 3c266cc47e..0c4358a7b1 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c
@@ -6,56 +6,205 @@
#include "pmd_aesni_gcm_priv.h"
+/* wrapper functions for library versions < 3.0.0 */
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_pre(const void *key, \
+ struct gcm_key_data *key_data, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_pre(key, key_data); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_init( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, const void *iv, \
+ const void *aad, uint64_t aad_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_init(key_data, context_data, iv, aad, aad_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, uint64_t len, const void *iv, \
+ const void *aad, uint64_t aad_len, void *auth_tag, \
+ uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_enc(key_data, context_data, out, in, len, iv, \
+ aad, aad_len, auth_tag, auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_update( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, const uint64_t len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_enc_update(key_data, context_data, out, in, len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_enc_finalize( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *auth_tag, \
+ const uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_enc_finalize(key_data, context_data, auth_tag, \
+ auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, uint64_t len, const void *iv, \
+ const void *aad, uint64_t aad_len, void *auth_tag, \
+ uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_dec(key_data, context_data, out, in, len, iv, \
+ aad, aad_len, auth_tag, auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_update( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *out, \
+ const void *in, const uint64_t len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_dec_update(key_data, context_data, out, in, len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gcm_dec_finalize( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *auth_tag, \
+ const uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gcm##bitlen##_dec_finalize(key_data, context_data, auth_tag, \
+ auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gmac_init( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, const void *iv, \
+ const uint64_t iv_len, IMB_MGR *state) \
+{ \
+ state->gmac##bitlen##_init(key_data, context_data, iv, iv_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gmac_update( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, const void *in, \
+ const uint64_t len, IMB_MGR *state) \
+{ \
+ state->gmac##bitlen##_update(key_data, context_data, in, len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#define _fn(bitlen) static void imb_aes##bitlen##_gmac_finalize( \
+ const struct gcm_key_data *key_data, \
+ struct gcm_context_data *context_data, void *auth_tag, \
+ const uint64_t auth_tag_len, IMB_MGR *state) \
+{ \
+ state->gmac##bitlen##_finalize(key_data, context_data, auth_tag, \
+ auth_tag_len); \
+}
+_fn(128)
+_fn(192)
+_fn(256)
+#undef _fn
+
+#endif
+
static void
aesni_gcm_set_ops(struct aesni_gcm_ops *ops, IMB_MGR *mb_mgr)
{
+ /* Set IMB_MGR pointers. */
+ ops[GCM_KEY_128].mb_mgr = mb_mgr;
+ ops[GCM_KEY_192].mb_mgr = mb_mgr;
+ ops[GCM_KEY_256].mb_mgr = mb_mgr;
+
/* Set 128 bit function pointers. */
- ops[GCM_KEY_128].pre = mb_mgr->gcm128_pre;
- ops[GCM_KEY_128].init = mb_mgr->gcm128_init;
+ ops[GCM_KEY_128].pre = imb_aes128_gcm_pre;
+ ops[GCM_KEY_128].init = imb_aes128_gcm_init;
- ops[GCM_KEY_128].enc = mb_mgr->gcm128_enc;
- ops[GCM_KEY_128].update_enc = mb_mgr->gcm128_enc_update;
- ops[GCM_KEY_128].finalize_enc = mb_mgr->gcm128_enc_finalize;
+ ops[GCM_KEY_128].enc = imb_aes128_gcm_enc;
+ ops[GCM_KEY_128].update_enc = imb_aes128_gcm_enc_update;
+ ops[GCM_KEY_128].finalize_enc = imb_aes128_gcm_enc_finalize;
- ops[GCM_KEY_128].dec = mb_mgr->gcm128_dec;
- ops[GCM_KEY_128].update_dec = mb_mgr->gcm128_dec_update;
- ops[GCM_KEY_128].finalize_dec = mb_mgr->gcm128_dec_finalize;
+ ops[GCM_KEY_128].dec = imb_aes128_gcm_dec;
+ ops[GCM_KEY_128].update_dec = imb_aes128_gcm_dec_update;
+ ops[GCM_KEY_128].finalize_dec = imb_aes128_gcm_dec_finalize;
- ops[GCM_KEY_128].gmac_init = mb_mgr->gmac128_init;
- ops[GCM_KEY_128].gmac_update = mb_mgr->gmac128_update;
- ops[GCM_KEY_128].gmac_finalize = mb_mgr->gmac128_finalize;
+ ops[GCM_KEY_128].gmac_init = imb_aes128_gmac_init;
+ ops[GCM_KEY_128].gmac_update = imb_aes128_gmac_update;
+ ops[GCM_KEY_128].gmac_finalize = imb_aes128_gmac_finalize;
/* Set 192 bit function pointers. */
- ops[GCM_KEY_192].pre = mb_mgr->gcm192_pre;
- ops[GCM_KEY_192].init = mb_mgr->gcm192_init;
+ ops[GCM_KEY_192].pre = imb_aes192_gcm_pre;
+ ops[GCM_KEY_192].init = imb_aes192_gcm_init;
- ops[GCM_KEY_192].enc = mb_mgr->gcm192_enc;
- ops[GCM_KEY_192].update_enc = mb_mgr->gcm192_enc_update;
- ops[GCM_KEY_192].finalize_enc = mb_mgr->gcm192_enc_finalize;
+ ops[GCM_KEY_192].enc = imb_aes192_gcm_enc;
+ ops[GCM_KEY_192].update_enc = imb_aes192_gcm_enc_update;
+ ops[GCM_KEY_192].finalize_enc = imb_aes192_gcm_enc_finalize;
- ops[GCM_KEY_192].dec = mb_mgr->gcm192_dec;
- ops[GCM_KEY_192].update_dec = mb_mgr->gcm192_dec_update;
- ops[GCM_KEY_192].finalize_dec = mb_mgr->gcm192_dec_finalize;
+ ops[GCM_KEY_192].dec = imb_aes192_gcm_dec;
+ ops[GCM_KEY_192].update_dec = imb_aes192_gcm_dec_update;
+ ops[GCM_KEY_192].finalize_dec = imb_aes192_gcm_dec_finalize;
- ops[GCM_KEY_192].gmac_init = mb_mgr->gmac192_init;
- ops[GCM_KEY_192].gmac_update = mb_mgr->gmac192_update;
- ops[GCM_KEY_192].gmac_finalize = mb_mgr->gmac192_finalize;
+ ops[GCM_KEY_192].gmac_init = imb_aes192_gmac_init;
+ ops[GCM_KEY_192].gmac_update = imb_aes192_gmac_update;
+ ops[GCM_KEY_192].gmac_finalize = imb_aes192_gmac_finalize;
/* Set 256 bit function pointers. */
- ops[GCM_KEY_256].pre = mb_mgr->gcm256_pre;
- ops[GCM_KEY_256].init = mb_mgr->gcm256_init;
+ ops[GCM_KEY_256].pre = imb_aes256_gcm_pre;
+ ops[GCM_KEY_256].init = imb_aes256_gcm_init;
- ops[GCM_KEY_256].enc = mb_mgr->gcm256_enc;
- ops[GCM_KEY_256].update_enc = mb_mgr->gcm256_enc_update;
- ops[GCM_KEY_256].finalize_enc = mb_mgr->gcm256_enc_finalize;
+ ops[GCM_KEY_256].enc = imb_aes256_gcm_enc;
+ ops[GCM_KEY_256].update_enc = imb_aes256_gcm_enc_update;
+ ops[GCM_KEY_256].finalize_enc = imb_aes256_gcm_enc_finalize;
- ops[GCM_KEY_256].dec = mb_mgr->gcm256_dec;
- ops[GCM_KEY_256].update_dec = mb_mgr->gcm256_dec_update;
- ops[GCM_KEY_256].finalize_dec = mb_mgr->gcm256_dec_finalize;
+ ops[GCM_KEY_256].dec = imb_aes256_gcm_dec;
+ ops[GCM_KEY_256].update_dec = imb_aes256_gcm_dec_update;
+ ops[GCM_KEY_256].finalize_dec = imb_aes256_gcm_dec_finalize;
- ops[GCM_KEY_256].gmac_init = mb_mgr->gmac256_init;
- ops[GCM_KEY_256].gmac_update = mb_mgr->gmac256_update;
- ops[GCM_KEY_256].gmac_finalize = mb_mgr->gmac256_finalize;
+ ops[GCM_KEY_256].gmac_init = imb_aes256_gmac_init;
+ ops[GCM_KEY_256].gmac_update = imb_aes256_gmac_update;
+ ops[GCM_KEY_256].gmac_finalize = imb_aes256_gmac_finalize;
}
static int
@@ -135,15 +284,15 @@ aesni_gcm_session_configure(IMB_MGR *mb_mgr, void *session,
switch (key_length) {
case 16:
sess->key_length = GCM_KEY_128;
- mb_mgr->gcm128_pre(key, &sess->gdata_key);
+ imb_aes128_gcm_pre(key, &sess->gdata_key, mb_mgr);
break;
case 24:
sess->key_length = GCM_KEY_192;
- mb_mgr->gcm192_pre(key, &sess->gdata_key);
+ imb_aes192_gcm_pre(key, &sess->gdata_key, mb_mgr);
break;
case 32:
sess->key_length = GCM_KEY_256;
- mb_mgr->gcm256_pre(key, &sess->gdata_key);
+ imb_aes256_gcm_pre(key, &sess->gdata_key, mb_mgr);
break;
default:
IPSEC_MB_LOG(ERR, "Invalid key length");
@@ -324,10 +473,10 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT) {
ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr,
sym_op->aead.aad.data,
- (uint64_t)session->aad_length);
+ (uint64_t)session->aad_length, ops->mb_mgr);
ops->update_enc(&session->gdata_key, &qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -345,7 +494,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->update_enc(&session->gdata_key,
&qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
@@ -355,14 +504,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
tag = sym_op->aead.digest.data;
ops->finalize_enc(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
} else if (session->op == IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT) {
ops->init(&session->gdata_key, &qp_data->gcm_ctx_data, iv_ptr,
sym_op->aead.aad.data,
- (uint64_t)session->aad_length);
+ (uint64_t)session->aad_length, ops->mb_mgr);
ops->update_dec(&session->gdata_key, &qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -380,19 +529,19 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->update_dec(&session->gdata_key,
&qp_data->gcm_ctx_data,
- dst, src, (uint64_t)part_len);
+ dst, src, (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
tag = qp_data->temp_digest;
ops->finalize_dec(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
} else if (session->op == IPSEC_MB_OP_HASH_GEN_ONLY) {
ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data,
- iv_ptr, session->iv.length);
+ iv_ptr, session->iv.length, ops->mb_mgr);
ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data,
- src, (uint64_t)part_len);
+ src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -406,7 +555,7 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->gmac_update(&session->gdata_key,
&qp_data->gcm_ctx_data, src,
- (uint64_t)part_len);
+ (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
@@ -416,13 +565,13 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
tag = sym_op->auth.digest.data;
ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
} else { /* IPSEC_MB_OP_HASH_VERIFY_ONLY */
ops->gmac_init(&session->gdata_key, &qp_data->gcm_ctx_data,
- iv_ptr, session->iv.length);
+ iv_ptr, session->iv.length, ops->mb_mgr);
ops->gmac_update(&session->gdata_key, &qp_data->gcm_ctx_data,
- src, (uint64_t)part_len);
+ src, (uint64_t)part_len, ops->mb_mgr);
total_len = data_length - part_len;
while (total_len) {
@@ -436,14 +585,14 @@ process_gcm_crypto_op(struct ipsec_mb_qp *qp, struct rte_crypto_op *op,
ops->gmac_update(&session->gdata_key,
&qp_data->gcm_ctx_data, src,
- (uint64_t)part_len);
+ (uint64_t)part_len, ops->mb_mgr);
total_len -= part_len;
}
tag = qp_data->temp_digest;
ops->gmac_finalize(&session->gdata_key, &qp_data->gcm_ctx_data,
- tag, session->gen_digest_length);
+ tag, session->gen_digest_length, ops->mb_mgr);
}
return 0;
}
@@ -540,11 +689,11 @@ aesni_gcm_sgl_op_finalize_encryption(const struct aesni_gcm_session *s,
uint8_t tmpdigest[s->gen_digest_length];
ops.finalize_enc(&s->gdata_key, gdata_ctx, tmpdigest,
- s->gen_digest_length);
+ s->gen_digest_length, ops.mb_mgr);
memcpy(digest, tmpdigest, s->req_digest_length);
} else {
ops.finalize_enc(&s->gdata_key, gdata_ctx, digest,
- s->gen_digest_length);
+ s->gen_digest_length, ops.mb_mgr);
}
return 0;
@@ -558,7 +707,7 @@ aesni_gcm_sgl_op_finalize_decryption(const struct aesni_gcm_session *s,
uint8_t tmpdigest[s->gen_digest_length];
ops.finalize_dec(&s->gdata_key, gdata_ctx, tmpdigest,
- s->gen_digest_length);
+ s->gen_digest_length, ops.mb_mgr);
return rte_memeq_timingsafe(digest, tmpdigest, s->req_digest_length) ? 0 : EBADMSG;
}
@@ -573,7 +722,7 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s,
/* init crypto operation */
ops.init(&s->gdata_key, gdata_ctx, iv, aad,
- (uint64_t)s->aad_length);
+ (uint64_t)s->aad_length, ops.mb_mgr);
/* update with sgl data */
for (i = 0; i < sgl->num; i++) {
@@ -582,11 +731,11 @@ aesni_gcm_process_gcm_sgl_op(const struct aesni_gcm_session *s,
switch (s->op) {
case IPSEC_MB_OP_AEAD_AUTHENTICATED_ENCRYPT:
ops.update_enc(&s->gdata_key, gdata_ctx,
- vec->base, vec->base, vec->len);
+ vec->base, vec->base, vec->len, ops.mb_mgr);
break;
case IPSEC_MB_OP_AEAD_AUTHENTICATED_DECRYPT:
ops.update_dec(&s->gdata_key, gdata_ctx,
- vec->base, vec->base, vec->len);
+ vec->base, vec->base, vec->len, ops.mb_mgr);
break;
default:
IPSEC_MB_LOG(ERR, "Invalid session op");
@@ -603,7 +752,7 @@ aesni_gcm_process_gmac_sgl_op(const struct aesni_gcm_session *s,
struct aesni_gcm_ops ops)
{
ops.init(&s->gdata_key, gdata_ctx, iv, sgl->vec[0].base,
- sgl->vec[0].len);
+ sgl->vec[0].len, ops.mb_mgr);
}
static inline uint32_t
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h
index 8a5a41fb19..4074430d6e 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h
@@ -79,41 +79,41 @@ enum aesni_gcm_key_length {
typedef void (*aesni_gcm_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *out, const uint8_t *in,
- uint64_t plaintext_len, const uint8_t *iv,
- const uint8_t *aad, uint64_t aad_len,
- uint8_t *auth_tag, uint64_t auth_tag_len);
+ void *out, const void *in,
+ uint64_t plaintext_len, const void *iv,
+ const void *aad, uint64_t aad_len,
+ void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state);
typedef void (*aesni_gcm_pre_t)(const void *key,
- struct gcm_key_data *gcm_data);
+ struct gcm_key_data *gcm_data, IMB_MGR *state);
typedef void (*aesni_gcm_init_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- const uint8_t *iv, uint8_t const *aad,
- uint64_t aad_len);
+ const void *iv, const void *aad,
+ uint64_t aad_len, IMB_MGR *state);
typedef void (*aesni_gcm_update_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *out, const uint8_t *in,
- uint64_t plaintext_len);
+ void *out, const void *in,
+ uint64_t plaintext_len, IMB_MGR *state);
typedef void (*aesni_gcm_finalize_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *auth_tag, uint64_t auth_tag_len);
+ void *auth_tag, uint64_t auth_tag_len, IMB_MGR *state);
typedef void (*aesni_gmac_init_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- const uint8_t *iv, const uint64_t iv_len);
+ const void *iv, const uint64_t iv_len, IMB_MGR *state);
typedef void (*aesni_gmac_update_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- const uint8_t *in,
- const uint64_t plaintext_len);
+ const void *in,
+ const uint64_t plaintext_len, IMB_MGR *state);
typedef void (*aesni_gmac_finalize_t)(const struct gcm_key_data *gcm_key_data,
struct gcm_context_data *gcm_ctx_data,
- uint8_t *auth_tag,
- const uint64_t auth_tag_len);
+ void *auth_tag,
+ const uint64_t auth_tag_len, IMB_MGR *state);
/** GCM operation handlers */
struct aesni_gcm_ops {
@@ -128,6 +128,7 @@ struct aesni_gcm_ops {
aesni_gmac_init_t gmac_init;
aesni_gmac_update_t gmac_update;
aesni_gmac_finalize_t gmac_finalize;
+ IMB_MGR *mb_mgr;
};
RTE_DEFINE_PER_LCORE(struct aesni_gcm_ops, gcm_ops[GCM_NUM_KEY_TYPES]);
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
index 02bacd020b..e8f6e77d21 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c
@@ -161,20 +161,22 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
if (xform->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) {
if (xform->auth.key.length == 16) {
- sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3_BITLEN;
+ sess->template_job.hash_alg = IMB_AUTH_ZUC_EIA3;
if (sess->auth.req_digest_len != 4) {
IPSEC_MB_LOG(ERR, "Invalid digest size");
return -EINVAL;
}
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
} else if (xform->auth.key.length == 32) {
- sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3_BITLEN;
+ sess->template_job.hash_alg = IMB_AUTH_ZUC256_EIA3;
if (sess->auth.req_digest_len != 4 &&
sess->auth.req_digest_len != 8 &&
sess->auth.req_digest_len != 16) {
IPSEC_MB_LOG(ERR, "Invalid digest size");
return -EINVAL;
}
+#endif
} else {
IPSEC_MB_LOG(ERR, "Invalid authentication key length");
return -EINVAL;
@@ -187,10 +189,10 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
sess->template_job.u.ZUC_EIA3._key = sess->auth.zuc_auth_key;
return 0;
} else if (xform->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) {
- sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2_BITLEN;
+ sess->template_job.hash_alg = IMB_AUTH_SNOW3G_UIA2;
uint16_t snow3g_uia2_digest_len =
get_truncated_digest_byte_length(
- IMB_AUTH_SNOW3G_UIA2_BITLEN);
+ IMB_AUTH_SNOW3G_UIA2);
if (sess->auth.req_digest_len != snow3g_uia2_digest_len) {
IPSEC_MB_LOG(ERR, "Invalid digest size");
return -EINVAL;
@@ -339,7 +341,7 @@ aesni_mb_set_session_auth_parameters(IMB_MGR *mb_mgr,
/** Set session cipher parameters */
static int
-aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr,
+aesni_mb_set_session_cipher_parameters(IMB_MGR *mb_mgr,
struct aesni_mb_session *sess,
const struct rte_crypto_sym_xform *xform)
{
@@ -407,11 +409,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr,
is_zuc = 1;
break;
case RTE_CRYPTO_CIPHER_SNOW3G_UEA2:
- sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2_BITLEN;
+ sess->template_job.cipher_mode = IMB_CIPHER_SNOW3G_UEA2;
is_snow3g = 1;
break;
case RTE_CRYPTO_CIPHER_KASUMI_F8:
- sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1_BITLEN;
+ sess->template_job.cipher_mode = IMB_CIPHER_KASUMI_UEA1;
is_kasumi = 1;
break;
case RTE_CRYPTO_CIPHER_NULL:
@@ -536,8 +538,11 @@ aesni_mb_set_session_cipher_parameters(const IMB_MGR *mb_mgr,
sess->template_job.dec_keys = sess->cipher.exp_3des_keys.ks_ptr;
sess->template_job.key_len_in_bytes = 24;
} else if (is_zuc) {
- if (xform->cipher.key.length != 16 &&
- xform->cipher.key.length != 32) {
+ if (xform->cipher.key.length != 16
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ && xform->cipher.key.length != 32
+#endif
+ ) {
IPSEC_MB_LOG(ERR, "Invalid cipher key length");
return -EINVAL;
}
@@ -1177,19 +1182,23 @@ sgl_linear_cipher_auth_len(IMB_JOB *job, uint64_t *auth_len)
{
uint64_t cipher_len;
- if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN ||
- job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN)
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ job->cipher_mode == IMB_CIPHER_KASUMI_UEA1)
cipher_len = (job->msg_len_to_cipher_in_bits >> 3) +
(job->cipher_start_src_offset_in_bits >> 3);
else
+#endif
cipher_len = job->msg_len_to_cipher_in_bytes +
job->cipher_start_src_offset_in_bytes;
- if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN ||
- job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN)
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ if (job->hash_alg == IMB_AUTH_SNOW3G_UIA2 ||
+ job->hash_alg == IMB_AUTH_ZUC_EIA3)
*auth_len = (job->msg_len_to_hash_in_bits >> 3) +
job->hash_start_src_offset_in_bytes;
else
+#endif
*auth_len = job->msg_len_to_hash_in_bytes +
job->hash_start_src_offset_in_bytes;
@@ -1457,18 +1466,20 @@ aesni_mb_digest_appended_in_src(struct rte_crypto_op *op, IMB_JOB *job,
return NULL;
if (job->cipher_mode == IMB_CIPHER_ZUC_EEA3 ||
- job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN ||
- job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN) {
+ job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ job->cipher_mode == IMB_CIPHER_KASUMI_UEA1) {
cipher_size = (op->sym->cipher.data.offset >> 3) +
(op->sym->cipher.data.length >> 3);
} else {
cipher_size = (op->sym->cipher.data.offset) +
(op->sym->cipher.data.length);
}
- if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN ||
- job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN ||
- job->hash_alg == IMB_AUTH_KASUMI_UIA1 ||
- job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) {
+ if (job->hash_alg == IMB_AUTH_ZUC_EIA3 ||
+ job->hash_alg == IMB_AUTH_SNOW3G_UIA2 ||
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ job->hash_alg == IMB_AUTH_ZUC256_EIA3 ||
+#endif
+ job->hash_alg == IMB_AUTH_KASUMI_UIA1) {
auth_size = (op->sym->auth.data.offset >> 3) +
(op->sym->auth.data.length >> 3);
} else {
@@ -1585,12 +1596,14 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
job->u.GMAC._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->auth_iv.offset);
break;
- case IMB_AUTH_ZUC_EIA3_BITLEN:
- case IMB_AUTH_ZUC256_EIA3_BITLEN:
+ case IMB_AUTH_ZUC_EIA3:
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ case IMB_AUTH_ZUC256_EIA3:
+#endif
job->u.ZUC_EIA3._iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->auth_iv.offset);
break;
- case IMB_AUTH_SNOW3G_UIA2_BITLEN:
+ case IMB_AUTH_SNOW3G_UIA2:
job->u.SNOW3G_UIA2._iv =
rte_crypto_op_ctod_offset(op, uint8_t *,
session->auth_iv.offset);
@@ -1616,12 +1629,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
else
m_offset = op->sym->cipher.data.offset;
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
if (cipher_mode == IMB_CIPHER_ZUC_EEA3)
m_offset >>= 3;
- else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN)
+ else if (cipher_mode == IMB_CIPHER_SNOW3G_UEA2)
m_offset = 0;
- else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN)
+ else if (cipher_mode == IMB_CIPHER_KASUMI_UEA1)
m_offset = 0;
+#else
+ if (cipher_mode == IMB_CIPHER_ZUC_EEA3 ||
+ cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ cipher_mode == IMB_CIPHER_KASUMI_UEA1)
+ m_offset >>= 3;
+#endif
/* Set digest output location */
if (job->hash_alg != IMB_AUTH_NULL &&
@@ -1701,9 +1721,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
session->iv.offset);
break;
/* ZUC and SNOW3G require length in bits and offset in bytes */
- case IMB_AUTH_ZUC_EIA3_BITLEN:
- case IMB_AUTH_ZUC256_EIA3_BITLEN:
- case IMB_AUTH_SNOW3G_UIA2_BITLEN:
+ case IMB_AUTH_ZUC_EIA3:
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ case IMB_AUTH_ZUC256_EIA3:
+#endif
+ case IMB_AUTH_SNOW3G_UIA2:
auth_off_in_bytes = op->sym->auth.data.offset >> 3;
ciph_off_in_bytes = op->sym->cipher.data.offset >> 3;
auth_len_in_bytes = op->sym->auth.data.length >> 3;
@@ -1714,7 +1736,11 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
session, oop, auth_off_in_bytes,
ciph_off_in_bytes, auth_len_in_bytes,
ciph_len_in_bytes, lb_sgl);
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
job->msg_len_to_hash_in_bits = op->sym->auth.data.length;
+#else
+ job->msg_len_to_hash_in_bytes = op->sym->auth.data.length >> 3;
+#endif
job->iv = rte_crypto_op_ctod_offset(op, uint8_t *,
session->iv.offset);
@@ -1772,12 +1798,19 @@ set_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp,
op->sym->cipher.data.length >> 3;
break;
/* ZUC and SNOW3G require length and offset in bits */
- case IMB_CIPHER_SNOW3G_UEA2_BITLEN:
- case IMB_CIPHER_KASUMI_UEA1_BITLEN:
+ case IMB_CIPHER_SNOW3G_UEA2:
+ case IMB_CIPHER_KASUMI_UEA1:
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
job->cipher_start_src_offset_in_bits =
op->sym->cipher.data.offset;
job->msg_len_to_cipher_in_bits =
op->sym->cipher.data.length;
+#else
+ job->cipher_start_src_offset_in_bytes =
+ op->sym->cipher.data.offset >> 3;
+ job->msg_len_to_cipher_in_bytes =
+ op->sym->cipher.data.length >> 3;
+#endif
break;
case IMB_CIPHER_GCM:
job->cipher_start_src_offset_in_bytes =
@@ -2047,8 +2080,8 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)
aesni_mb_digest_appended_in_src(op, job, oop) != NULL) {
unsigned int auth_size, cipher_size;
int unencrypted_bytes = 0;
- if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2_BITLEN ||
- job->cipher_mode == IMB_CIPHER_KASUMI_UEA1_BITLEN ||
+ if (job->cipher_mode == IMB_CIPHER_SNOW3G_UEA2 ||
+ job->cipher_mode == IMB_CIPHER_KASUMI_UEA1 ||
job->cipher_mode == IMB_CIPHER_ZUC_EEA3) {
cipher_size = (op->sym->cipher.data.offset >> 3) +
(op->sym->cipher.data.length >> 3);
@@ -2056,10 +2089,12 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)
cipher_size = (op->sym->cipher.data.offset) +
(op->sym->cipher.data.length);
}
- if (job->hash_alg == IMB_AUTH_ZUC_EIA3_BITLEN ||
- job->hash_alg == IMB_AUTH_SNOW3G_UIA2_BITLEN ||
- job->hash_alg == IMB_AUTH_KASUMI_UIA1 ||
- job->hash_alg == IMB_AUTH_ZUC256_EIA3_BITLEN) {
+ if (job->hash_alg == IMB_AUTH_ZUC_EIA3 ||
+ job->hash_alg == IMB_AUTH_SNOW3G_UIA2 ||
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+ job->hash_alg == IMB_AUTH_ZUC256_EIA3 ||
+#endif
+ job->hash_alg == IMB_AUTH_KASUMI_UIA1) {
auth_size = (op->sym->auth.data.offset >> 3) +
(op->sym->auth.data.length >> 3);
} else {
@@ -2550,7 +2585,9 @@ RTE_INIT(ipsec_mb_register_aesni_mb)
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
RTE_CRYPTODEV_FF_SYM_CPU_CRYPTO |
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA |
+#endif
RTE_CRYPTODEV_FF_SYM_SESSIONLESS |
RTE_CRYPTODEV_FF_IN_PLACE_SGL |
RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT |
diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
index ad38a1e28a..6afda45414 100644
--- a/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
+++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h
@@ -19,6 +19,15 @@
#define MAX_NUM_SEGS 16
+/* bit len removed in 3.0 */
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
+#define IMB_CIPHER_SNOW3G_UEA2 IMB_CIPHER_SNOW3G_UEA2_BITLEN
+#define IMB_CIPHER_KASUMI_UEA1 IMB_CIPHER_KASUMI_UEA1_BITLEN
+#define IMB_AUTH_ZUC_EIA3 IMB_AUTH_ZUC_EIA3_BITLEN
+#define IMB_AUTH_SNOW3G_UIA2 IMB_AUTH_SNOW3G_UIA2_BITLEN
+#define IMB_AUTH_ZUC256_EIA3 IMB_AUTH_ZUC256_EIA3_BITLEN
+#endif
+
int
aesni_mb_session_configure(IMB_MGR * m __rte_unused, void *priv_sess,
const struct rte_crypto_sym_xform *xform);
@@ -573,8 +582,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
.block_size = 16,
.key_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 32,
.increment = 16
+#else
+ .max = 16,
+ .increment = 0
+#endif
},
.digest_size = {
.min = 4,
@@ -583,8 +597,13 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
},
.iv_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 25,
.increment = 9
+#else
+ .max = 16,
+ .increment = 0
+#endif
}
}, }
}, }
@@ -598,13 +617,23 @@ static const struct rte_cryptodev_capabilities aesni_mb_capabilities[] = {
.block_size = 16,
.key_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 32,
.increment = 16
+#else
+ .max = 16,
+ .increment = 0
+#endif
},
.iv_size = {
.min = 16,
+#if IMB_VERSION(3, 0, 0) > IMB_VERSION_NUM
.max = 25,
.increment = 9
+#else
+ .max = 16,
+ .increment = 0
+#endif
},
}, }
}, }
@@ -888,9 +917,9 @@ static const unsigned int auth_blocksize[] = {
[IMB_AUTH_SHA_256] = 64,
[IMB_AUTH_SHA_384] = 128,
[IMB_AUTH_SHA_512] = 128,
- [IMB_AUTH_ZUC_EIA3_BITLEN] = 16,
- [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 16,
- [IMB_AUTH_KASUMI_UIA1] = 16
+ [IMB_AUTH_ZUC_EIA3] = 16,
+ [IMB_AUTH_SNOW3G_UIA2] = 16,
+ [IMB_AUTH_KASUMI_UIA1] = 16
};
/**
@@ -922,9 +951,9 @@ static const unsigned int auth_truncated_digest_byte_lengths[] = {
[IMB_AUTH_SHA_256] = 32,
[IMB_AUTH_SHA_384] = 48,
[IMB_AUTH_SHA_512] = 64,
- [IMB_AUTH_ZUC_EIA3_BITLEN] = 4,
- [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4,
- [IMB_AUTH_KASUMI_UIA1] = 4
+ [IMB_AUTH_ZUC_EIA3] = 4,
+ [IMB_AUTH_SNOW3G_UIA2] = 4,
+ [IMB_AUTH_KASUMI_UIA1] = 4
};
/**
@@ -957,9 +986,9 @@ static const unsigned int auth_digest_byte_lengths[] = {
[IMB_AUTH_SHA_256] = 32,
[IMB_AUTH_SHA_384] = 48,
[IMB_AUTH_SHA_512] = 64,
- [IMB_AUTH_ZUC_EIA3_BITLEN] = 4,
- [IMB_AUTH_SNOW3G_UIA2_BITLEN] = 4,
- [IMB_AUTH_KASUMI_UIA1] = 4,
+ [IMB_AUTH_ZUC_EIA3] = 4,
+ [IMB_AUTH_SNOW3G_UIA2] = 4,
+ [IMB_AUTH_KASUMI_UIA1] = 4,
[IMB_AUTH_SM3] = 32,
[IMB_AUTH_HMAC_SM3] = 32,
/**< Vector mode dependent pointer table of the multi-buffer APIs */
--
2.52.0
[-- Attachment #2: Type: text/html, Size: 110255 bytes --]
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0
2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau
` (3 preceding siblings ...)
2026-10-02 9:56 ` [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB Radu Nicolau
@ 2026-10-06 13:08 ` Ji, Kai
4 siblings, 0 replies; 7+ messages in thread
From: Ji, Kai @ 2026-10-06 13:08 UTC (permalink / raw)
To: Nicolau, Radu, dev@dpdk.org
[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]
Series-acked-by: Kai Ji <kai.ji@intel.com>
________________________________
From: Radu Nicolau <radu.nicolau@intel.com>
Sent: 02 October 2026 10:56
To: dev@dpdk.org <dev@dpdk.org>
Cc: Nicolau, Radu <radu.nicolau@intel.com>
Subject: [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0
Intel(R) Multi-Buffer Crypto for IPsec Library released version 3.0.0
codenamed Túnel Pablo
https://github.com/intel/intel-ipsec-mb/releases/tag/v3.0.0
This patchset adds support for the new library version and support
for the new algorithms included in this release.
Radu Nicolau (4):
crypto/ipsec_mb: add support for IPsecMB 3.0
crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms
crypto/ipsec_mb: add support for ML-KEM and ML-DSA
test/crypto: add asym test suite for AESNI_MB
app/test/test_cryptodev_asym.c | 7 +
doc/guides/cryptodevs/aesni_gcm.rst | 4 +-
doc/guides/cryptodevs/aesni_mb.rst | 26 +-
doc/guides/cryptodevs/features/aesni_mb.ini | 12 +
doc/guides/rel_notes/release_26_11.rst | 12 +
drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 58 ++
drivers/crypto/ipsec_mb/ipsec_mb_private.h | 20 +
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 267 +++++--
drivers/crypto/ipsec_mb/pmd_aesni_gcm_priv.h | 31 +-
drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 695 +++++++++++++++++--
drivers/crypto/ipsec_mb/pmd_aesni_mb_priv.h | 360 +++++++++-
11 files changed, 1354 insertions(+), 138 deletions(-)
--
2.52.0
[-- Attachment #2: Type: text/html, Size: 3034 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-10-06 13:08 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-02 9:56 [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Radu Nicolau
2026-10-02 9:56 ` [PATCH 1/4] " Radu Nicolau
2026-10-06 12:06 ` Ji, Kai
2026-10-02 9:56 ` [PATCH 2/4] crypto/ipsec_mb: add support for 256-NxA4/5/6 algorithms Radu Nicolau
2026-10-02 9:56 ` [PATCH 3/4] crypto/ipsec_mb: add support for ML-KEM and ML-DSA Radu Nicolau
2026-10-02 9:56 ` [PATCH 4/4] test/crypto: add asym test suite for AESNI_MB Radu Nicolau
2026-10-06 13:08 ` [PATCH 0/4] crypto/ipsec_mb: add support for IPsecMB 3.0 Ji, Kai
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).