From: Declan Doherty <declan.doherty@intel.com>
To: dev@dpdk.org
Subject: [PATCH v2 1/6] cryptodev: Initial DPDK Crypto APIs and device framework release
Date: Fri, 30 Oct 2015 12:59:41 +0000 [thread overview]
Message-ID: <1446209986-29137-2-git-send-email-declan.doherty@intel.com> (raw)
In-Reply-To: <1446209986-29137-1-git-send-email-declan.doherty@intel.com>
This patch contains the initial proposed APIs and device framework for
integrating crypto packet processing into DPDK.
features include:
- Crypto device configuration / management APIs
- Definitions of supported cipher algorithms and operations.
- Definitions of supported hash/authentication algorithms and
operations.
- Crypto session management APIs
- Crypto operation data structures and APIs allocation of crypto
operation structure used to specify the crypto operations to
be performed on a particular mbuf.
- Extension of mbuf to contain crypto operation data pointer and
extra flags.
- Burst enqueue / dequeue APIs for processing of crypto operations.
changes from RFC:
- Session management API changes to support specification of crypto
transform(xform) chains using linked list of xforms.
- Changes to the crypto operation struct as a result of session
management changes.
- Some movement of common MACROS shared by cryptodevs and ethdevs to
common headers
Signed-off-by: Des O Dea <des.j.o.dea@intel.com>
Signed-off-by: John Griffin <john.griffin@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
---
config/common_bsdapp | 10 +-
config/common_linuxapp | 10 +-
doc/api/doxy-api-index.md | 1 +
doc/api/doxy-api.conf | 1 +
lib/Makefile | 1 +
lib/librte_cryptodev/Makefile | 60 ++
lib/librte_cryptodev/rte_crypto.h | 604 ++++++++++++++
lib/librte_cryptodev/rte_cryptodev.c | 1065 ++++++++++++++++++++++++
lib/librte_cryptodev/rte_cryptodev.h | 619 ++++++++++++++
lib/librte_cryptodev/rte_cryptodev_pmd.h | 529 ++++++++++++
lib/librte_cryptodev/rte_cryptodev_version.map | 41 +
lib/librte_eal/common/include/rte_common.h | 15 +
lib/librte_eal/common/include/rte_eal.h | 14 +
lib/librte_eal/common/include/rte_log.h | 1 +
lib/librte_eal/common/include/rte_memory.h | 14 +-
lib/librte_ether/rte_ethdev.c | 30 -
lib/librte_mbuf/rte_mbuf.h | 30 +-
mk/rte.app.mk | 1 +
18 files changed, 3011 insertions(+), 35 deletions(-)
create mode 100644 lib/librte_cryptodev/Makefile
create mode 100644 lib/librte_cryptodev/rte_crypto.h
create mode 100644 lib/librte_cryptodev/rte_cryptodev.c
create mode 100644 lib/librte_cryptodev/rte_cryptodev.h
create mode 100644 lib/librte_cryptodev/rte_cryptodev_pmd.h
create mode 100644 lib/librte_cryptodev/rte_cryptodev_version.map
diff --git a/config/common_bsdapp b/config/common_bsdapp
index b37dcf4..8ce6af5 100644
--- a/config/common_bsdapp
+++ b/config/common_bsdapp
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -147,6 +147,14 @@ CONFIG_RTE_ETHDEV_QUEUE_STAT_CNTRS=16
CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y
#
+# Compile generic Crypto device library
+#
+CONFIG_RTE_LIBRTE_CRYPTODEV=y
+CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=n
+CONFIG_RTE_CRYPTO_MAX_DEVS=64
+CONFIG_RTE_CRYPTODEV_NAME_LEN=64
+
+#
# Support NIC bypass logic
#
CONFIG_RTE_NIC_BYPASS=n
diff --git a/config/common_linuxapp b/config/common_linuxapp
index 0de43d5..e7b9b25 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -145,6 +145,14 @@ CONFIG_RTE_ETHDEV_QUEUE_STAT_CNTRS=16
CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y
#
+# Compile generic Crypto device library
+#
+CONFIG_RTE_LIBRTE_CRYPTODEV=y
+CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=n
+CONFIG_RTE_CRYPTO_MAX_DEVS=64
+CONFIG_RTE_CRYPTODEV_NAME_LEN=64
+
+#
# Support NIC bypass logic
#
CONFIG_RTE_NIC_BYPASS=n
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 72ac3c4..bdb6130 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -39,6 +39,7 @@ There are many libraries, so their headers may be grouped by topics:
[dev] (@ref rte_dev.h),
[ethdev] (@ref rte_ethdev.h),
[ethctrl] (@ref rte_eth_ctrl.h),
+ [cryptodev] (@ref rte_cryptodev.h),
[devargs] (@ref rte_devargs.h),
[bond] (@ref rte_eth_bond.h),
[vhost] (@ref rte_virtio_net.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index cfb4627..7244b8f 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -37,6 +37,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_cfgfile \
lib/librte_cmdline \
lib/librte_compat \
+ lib/librte_cryptodev \
lib/librte_distributor \
lib/librte_ether \
lib/librte_hash \
diff --git a/lib/Makefile b/lib/Makefile
index 9727b83..4c5c1b4 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -40,6 +40,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += librte_timer
DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile
DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline
DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ether
+DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += librte_cryptodev
DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += librte_vhost
DIRS-$(CONFIG_RTE_LIBRTE_HASH) += librte_hash
DIRS-$(CONFIG_RTE_LIBRTE_LPM) += librte_lpm
diff --git a/lib/librte_cryptodev/Makefile b/lib/librte_cryptodev/Makefile
new file mode 100644
index 0000000..b434a53
--- /dev/null
+++ b/lib/librte_cryptodev/Makefile
@@ -0,0 +1,60 @@
+# BSD LICENSE
+#
+# Copyright(c) 2015 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_cryptodev.a
+
+# library version
+LIBABIVER := 1
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+# library source files
+SRCS-y += rte_cryptodev.c
+
+# export include files
+SYMLINK-y-include += rte_crypto.h
+SYMLINK-y-include += rte_cryptodev.h
+SYMLINK-y-include += rte_cryptodev_pmd.h
+
+# versioning export map
+EXPORT_MAP := rte_cryptodev_version.map
+
+# library dependencies
+DEPDIRS-y += lib/librte_eal
+DEPDIRS-y += lib/librte_mempool
+DEPDIRS-y += lib/librte_ring
+DEPDIRS-y += lib/librte_mbuf
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_cryptodev/rte_crypto.h b/lib/librte_cryptodev/rte_crypto.h
new file mode 100644
index 0000000..bd6748d
--- /dev/null
+++ b/lib/librte_cryptodev/rte_crypto.h
@@ -0,0 +1,604 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTO_H_
+#define _RTE_CRYPTO_H_
+
+/**
+ * @file rte_crypto.h
+ *
+ * RTE Cryptographic Definitions
+ *
+ * Defines symmetric cipher and authentication algorithms and modes, as well
+ * as supported symmetric crypto operation combinations.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <rte_mbuf.h>
+#include <rte_memory.h>
+#include <rte_mempool.h>
+
+/** Symmetric Cipher Algorithms */
+enum rte_crypto_cipher_algorithm {
+ RTE_CRYPTO_CIPHER_NULL = 1,
+ /**< NULL cipher algorithm. No mode applies to the NULL algorithm. */
+
+ RTE_CRYPTO_CIPHER_3DES_CBC,
+ /**< Triple DES algorithm in CBC mode */
+ RTE_CRYPTO_CIPHER_3DES_CTR,
+ /**< Triple DES algorithm in CTR mode */
+ RTE_CRYPTO_CIPHER_3DES_ECB,
+ /**< Triple DES algorithm in ECB mode */
+
+ RTE_CRYPTO_CIPHER_AES_CBC,
+ /**< AES algorithm in CBC mode */
+ RTE_CRYPTO_CIPHER_AES_CCM,
+ /**< AES algorithm in CCM mode. When this cipher algorithm is used the
+ * *RTE_CRYPTO_AUTH_AES_CCM* element of the
+ * *rte_crypto_hash_algorithm* enum MUST be used to set up the related
+ * *rte_crypto_hash_setup_data* structure in the session context or in
+ * the op_params of the crypto operation structure in the case of a
+ * session-less crypto operation
+ */
+ RTE_CRYPTO_CIPHER_AES_CTR,
+ /**< AES algorithm in Counter mode */
+ RTE_CRYPTO_CIPHER_AES_ECB,
+ /**< AES algorithm in ECB mode */
+ RTE_CRYPTO_CIPHER_AES_F8,
+ /**< AES algorithm in F8 mode */
+ RTE_CRYPTO_CIPHER_AES_GCM,
+ /**< AES algorithm in GCM mode. When this cipher algorithm is used the
+ * *RTE_CRYPTO_AUTH_AES_GCM* element of the
+ * *rte_crypto_hash_algorithm* enum MUST be used to set up the related
+ * *rte_crypto_hash_setup_data* structure in the session context or in
+ * the op_params of the crypto operation structure in the case of a
+ * session-less crypto operation.
+ */
+ RTE_CRYPTO_CIPHER_AES_XTS,
+ /**< AES algorithm in XTS mode */
+
+ RTE_CRYPTO_CIPHER_ARC4,
+ /**< (A)RC4 cipher algorithm */
+
+ RTE_CRYPTO_CIPHER_KASUMI_F8,
+ /**< Kasumi algorithm in F8 mode */
+
+ RTE_CRYPTO_CIPHER_SNOW3G_UEA2,
+ /**< SNOW3G algorithm in UEA2 mode */
+
+ RTE_CRYPTO_CIPHER_ZUC_EEA3
+ /**< ZUC algorithm in EEA3 mode */
+};
+
+/** Symmetric Cipher Direction */
+enum rte_crypto_cipher_operation {
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ /**< Encrypt cipher operation */
+ RTE_CRYPTO_CIPHER_OP_DECRYPT
+ /**< Decrypt cipher operation */
+};
+
+/** Crypto key structure */
+struct rte_crypto_key {
+ uint8_t *data; /**< pointer to key data */
+ phys_addr_t phys_addr;
+ size_t length; /**< key length in bytes */
+};
+
+/**
+ * Symmetric Cipher Setup Data.
+ *
+ * This structure contains data relating to Cipher (Encryption and Decryption)
+ * use to create a session.
+ */
+struct rte_crypto_cipher_xform {
+ enum rte_crypto_cipher_operation op;
+ /**< This parameter determines if the cipher operation is an encrypt or
+ * a decrypt operation. For the RC4 algorithm and the F8/CTR modes,
+ * only encrypt operations are valid. */
+ enum rte_crypto_cipher_algorithm algo;
+ /**< Cipher algorithm */
+
+ struct rte_crypto_key key;
+ /**< Cipher key
+ *
+ * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.data will
+ * point to a concatenation of the AES encryption key followed by a
+ * keymask. As per RFC3711, the keymask should be padded with trailing
+ * bytes to match the length of the encryption key used.
+ *
+ * For AES-XTS mode of operation, two keys must be provided and
+ * key.data must point to the two keys concatenated together (Key1 ||
+ * Key2). The cipher key length will contain the total size of both keys.
+ *
+ * Cipher key length is in bytes. For AES it can be 128 bits (16 bytes),
+ * 192 bits (24 bytes) or 256 bits (32 bytes).
+ *
+ * For the CCM mode of operation, the only supported key length is 128
+ * bits (16 bytes).
+ *
+ * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.length
+ * should be set to the combined length of the encryption key and the
+ * keymask. Since the keymask and the encryption key are the same size,
+ * key.length should be set to 2 x the AES encryption key length.
+ *
+ * For the AES-XTS mode of operation:
+ * - Two keys must be provided and key.length refers to total length of
+ * the two keys.
+ * - Each key can be either 128 bits (16 bytes) or 256 bits (32 bytes).
+ * - Both keys must have the same size.
+ **/
+};
+
+/** Symmetric Authentication / Hash Algorithms */
+enum rte_crypto_auth_algorithm {
+ RTE_CRYPTO_AUTH_NULL = 1,
+ /**< NULL hash algorithm. */
+
+ RTE_CRYPTO_AUTH_AES_CBC_MAC,
+ /**< AES-CBC-MAC algorithm. Only 128-bit keys are supported. */
+ RTE_CRYPTO_AUTH_AES_CCM,
+ /**< AES algorithm in CCM mode. This is an authenticated cipher. When
+ * this hash algorithm is used, the *RTE_CRYPTO_CIPHER_AES_CCM*
+ * element of the *rte_crypto_cipher_algorithm* enum MUST be used to
+ * set up the related rte_crypto_cipher_setup_data structure in the
+ * session context or the corresponding parameter in the crypto operation
+ * data structures op_params parameter MUST be set for a session-less
+ * crypto operation.
+ * */
+ RTE_CRYPTO_AUTH_AES_CMAC,
+ /**< AES CMAC algorithm. */
+ RTE_CRYPTO_AUTH_AES_GCM,
+ /**< AES algorithm in GCM mode. When this hash algorithm
+ * is used, the RTE_CRYPTO_CIPHER_AES_GCM element of the
+ * rte_crypto_cipher_algorithm enum MUST be used to set up the related
+ * rte_crypto_cipher_setup_data structure in the session context, or
+ * the corresponding parameter in the crypto operation data structures
+ * op_params parameter MUST be set for a session-less crypto operation.
+ */
+ RTE_CRYPTO_AUTH_AES_GMAC,
+ /**< AES GMAC algorithm. When this hash algorithm
+ * is used, the RTE_CRYPTO_CIPHER_AES_GCM element of the
+ * rte_crypto_cipher_algorithm enum MUST be used to set up the related
+ * rte_crypto_cipher_setup_data structure in the session context, or
+ * the corresponding parameter in the crypto operation data structures
+ * op_params parameter MUST be set for a session-less crypto operation.
+ */
+ RTE_CRYPTO_AUTH_AES_XCBC_MAC,
+ /**< AES XCBC algorithm. */
+
+ RTE_CRYPTO_AUTH_KASUMI_F9,
+ /**< Kasumi algorithm in F9 mode. */
+
+ RTE_CRYPTO_AUTH_MD5,
+ /**< MD5 algorithm */
+ RTE_CRYPTO_AUTH_MD5_HMAC,
+ /**< HMAC using MD5 algorithm */
+
+ RTE_CRYPTO_AUTH_SHA1,
+ /**< 128 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA1_HMAC,
+ /**< HMAC using 128 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA224,
+ /**< 224 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA224_HMAC,
+ /**< HMAC using 224 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA256,
+ /**< 256 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA256_HMAC,
+ /**< HMAC using 256 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA384,
+ /**< 384 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA384_HMAC,
+ /**< HMAC using 384 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA512,
+ /**< 512 bit SHA algorithm. */
+ RTE_CRYPTO_AUTH_SHA512_HMAC,
+ /**< HMAC using 512 bit SHA algorithm. */
+
+ RTE_CRYPTO_AUTH_SNOW3G_UIA2,
+ /**< SNOW3G algorithm in UIA2 mode. */
+
+ RTE_CRYPTO_AUTH_ZUC_EIA3,
+ /**< ZUC algorithm in EIA3 mode */
+};
+
+/** Symmetric Authentication / Hash Operations */
+enum rte_crypto_auth_operation {
+ RTE_CRYPTO_AUTH_OP_VERIFY, /**< Verify authentication digest */
+ RTE_CRYPTO_AUTH_OP_GENERATE /**< Generate authentication digest */
+};
+
+/**
+ * Authentication / Hash transform data.
+ *
+ * This structure contains data relating to an authentication/hash crypto
+ * transforms. The fields op, algo and digest_length are common to all
+ * authentication transforms and MUST be set.
+ */
+struct rte_crypto_auth_xform {
+ enum rte_crypto_auth_operation op; /**< Authentication operation type */
+ enum rte_crypto_auth_algorithm algo; /**< Authentication algorithm selection */
+
+ struct rte_crypto_key key; /**< Authentication key data.
+ * The authentication key length MUST be less than or equal to the
+ * block size of the algorithm. It is the callers responsibility to
+ * ensure that the key length is compliant with the standard being used
+ * (for example RFC 2104, FIPS 198a).
+ */
+
+ uint32_t digest_length;
+ /**< Length of the digest to be returned. If the verify option is set,
+ * this specifies the length of the digest to be compared for the
+ * session.
+ *
+ * If the value is less than the maximum length allowed by the hash,
+ * the result shall be truncated. If the value is greater than the
+ * maximum length allowed by the hash then an error will be generated
+ * by *rte_cryptodev_session_create* or by the
+ * *rte_cryptodev_enqueue_burst* if using session-less APIs.
+ */
+
+ uint32_t add_auth_data_length;
+ /**< The length of the additional authenticated data (AAD) in bytes.
+ * The maximum permitted value is 240 bytes, unless otherwise specified
+ * below.
+ *
+ * This field must be specified when the hash algorithm is one of the
+ * following:
+ *
+ * - For SNOW3G (@ref RTE_CRYPTO_AUTH_SNOW3G_UIA2), this is the
+ * length of the IV (which should be 16).
+ *
+ * - For GCM (@ref RTE_CRYPTO_AUTH_AES_GCM). In this case, this is
+ * the length of the Additional Authenticated Data (called A, in NIST
+ * SP800-38D).
+ *
+ * - For CCM (@ref RTE_CRYPTO_AUTH_AES_CCM). In this case, this is
+ * the length of the associated data (called A, in NIST SP800-38C).
+ * Note that this does NOT include the length of any padding, or the
+ * 18 bytes reserved at the start of the above field to store the
+ * block B0 and the encoded length. The maximum permitted value in
+ * this case is 222 bytes.
+ *
+ * @note
+ * For AES-GMAC (@ref RTE_CRYPTO_AUTH_AES_GMAC) mode of operation
+ * this field is not used and should be set to 0. Instead the length
+ * of the AAD data is specified in the message length to hash field of
+ * the rte_crypto_op_data structure.
+ */
+};
+
+/** Crypto transformation types */
+enum rte_crypto_xform_type {
+ RTE_CRYPTO_XFORM_NOT_SPECIFIED = 0, /**< No xform specified */
+ RTE_CRYPTO_XFORM_AUTH, /**< Authentication xform */
+ RTE_CRYPTO_XFORM_CIPHER /**< Cipher xform */
+};
+
+/**
+ * Crypto transform structure.
+ *
+ * This is used to specify the crypto transforms required, multiple transforms
+ * can be chained together to specify a chain transforms such as authentication
+ * then cipher, or cipher then authentication. Each transform structure can
+ * hold a single transform, the type field is used to specify which transform
+ * is contained within the union */
+struct rte_crypto_xform {
+ struct rte_crypto_xform *next; /**< next xform in chain */
+
+ enum rte_crypto_xform_type type; /**< xform type */
+ union {
+ struct rte_crypto_auth_xform auth;
+ /**< Authentication / hash xform */
+ struct rte_crypto_cipher_xform cipher;
+ /**< Cipher xform */
+ };
+};
+
+/**
+ * Crypto operation session type. This is used to specify whether a crypto
+ * operation has session structure attached for immutable parameters or if all
+ * operation information is included in the operation data structure.
+ */
+enum rte_crypto_op_sess_type {
+ RTE_CRYPTO_OP_WITH_SESSION, /**< Session based crypto operation */
+ RTE_CRYPTO_OP_SESSIONLESS /**< Session-less crypto operation */
+};
+
+/** Status of crypto operation */
+enum rte_crypto_op_status {
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ /**< Operation completed successfully */
+ RTE_CRYPTO_OP_STATUS_NO_SUBMITTED,
+ /**< Operation not yet submitted to a cryptodev */
+ RTE_CRYPTO_OP_STATUS_ENQUEUED,
+ /**< Operation is enqueued on device */
+ RTE_CRYPTO_OP_STATUS_AUTH_FAILED,
+ /**< Authentication verification failed */
+ RTE_CRYPTO_OP_STATUS_INVALID_ARGS,
+ /**< Operation failed due to invalid arguments in request */
+ RTE_CRYPTO_OP_STATUS_ERROR,
+ /**< Error handling operation */
+};
+
+/**
+ * Cryptographic Operation Data.
+ *
+ * This structure contains data relating to performing cryptographic processing
+ * on a data buffer. This request is used with rte_crypto_enqueue_burst() call
+ * for performing cipher, hash, or a combined hash and cipher operations.
+ */
+struct rte_crypto_op {
+ enum rte_crypto_op_sess_type type;
+ enum rte_crypto_op_status status;
+
+ struct {
+ struct rte_mbuf *m; /**< Destination mbuf */
+ uint8_t offset; /**< Data offset */
+ } dst;
+
+ union {
+ struct rte_cryptodev_session *session;
+ /**< Handle for the initialised session context */
+ struct rte_crypto_xform *xform;
+ /**< Session-less API crypto operation parameters */
+ };
+
+ struct {
+ struct {
+ uint32_t offset;
+ /**< Starting point for cipher processing, specified
+ * as number of bytes from start of data in the source
+ * buffer. The result of the cipher operation will be
+ * written back into the output buffer starting at
+ * this location. */
+
+ uint32_t length;
+ /**< The message length, in bytes, of the source buffer
+ * on which the cryptographic operation will be
+ * computed. This must be a multiple of the block size
+ * if a block cipher is being used. This is also the
+ * same as the result length.
+ *
+ * @note
+ * In the case of CCM @ref RTE_CRYPTO_AUTH_AES_CCM,
+ * this value should not include the length of the
+ * padding or the length of the MAC; the driver will
+ * compute the actual number of bytes over which the
+ * encryption will occur, which will include these
+ * values.
+ *
+ * @note
+ * For AES-GMAC @ref RTE_CRYPTO_AUTH_AES_GMAC, this
+ * field should be set to 0.
+ */
+ } to_cipher; /**< Data offsets and length for ciphering */
+
+ struct {
+ uint32_t offset;
+ /**< Starting point for hash processing, specified as
+ * number of bytes from start of packet in source
+ * buffer.
+ *
+ * @note
+ * For CCM and GCM modes of operation, this field is
+ * ignored. The field @ref additional_auth field
+ * should be set instead.
+ *
+ * @note For AES-GMAC (@ref RTE_CRYPTO_AUTH_AES_GMAC) mode of
+ * operation, this field specifies the start of the AAD data in
+ * the source buffer.
+ */
+
+ uint32_t length;
+ /**< The message length, in bytes, of the source buffer that
+ * the hash will be computed on.
+ *
+ * @note
+ * For CCM and GCM modes of operation, this field is
+ * ignored. The field @ref additional_auth field should
+ * be set instead.
+ *
+ * @note
+ * For AES-GMAC @ref RTE_CRYPTO_AUTH_AES_GMAC mode
+ * of operation, this field specifies the length of
+ * the AAD data in the source buffer.
+ */
+ } to_hash; /**< Data offsets and length for authentication */
+ } data; /**< Details of data to be operated on */
+
+ struct {
+ uint8_t *data;
+ /**< Initialisation Vector or Counter.
+ *
+ * - For block ciphers in CBC or F8 mode, or for Kasumi in F8
+ * mode, or for SNOW3G in UEA2 mode, this is the Initialisation
+ * Vector (IV) value.
+ *
+ * - For block ciphers in CTR mode, this is the counter.
+ *
+ * - For GCM mode, this is either the IV (if the length is 96
+ * bits) or J0 (for other sizes), where J0 is as defined by
+ * NIST SP800-38D. Regardless of the IV length, a full 16 bytes
+ * needs to be allocated.
+ *
+ * - For CCM mode, the first byte is reserved, and the nonce
+ * should be written starting at &iv[1] (to allow space for the
+ * implementation to write in the flags in the first byte).
+ * Note that a full 16 bytes should be allocated, even though
+ * the length field will have a value less than this.
+ *
+ * - For AES-XTS, this is the 128bit tweak, i, from IEEE Std
+ * 1619-2007.
+ *
+ * For optimum performance, the data pointed to SHOULD be
+ * 8-byte aligned.
+ */
+ phys_addr_t phys_addr;
+ size_t length;
+ /**< Length of valid IV data.
+ *
+ * - For block ciphers in CBC or F8 mode, or for Kasumi in F8
+ * mode, or for SNOW3G in UEA2 mode, this is the length of the
+ * IV (which must be the same as the block length of the
+ * cipher).
+ *
+ * - For block ciphers in CTR mode, this is the length of the
+ * counter (which must be the same as the block length of the
+ * cipher).
+ *
+ * - For GCM mode, this is either 12 (for 96-bit IVs) or 16, in
+ * which case data points to J0.
+ *
+ * - For CCM mode, this is the length of the nonce, which can
+ * be in the range 7 to 13 inclusive.
+ */
+ } iv; /**< Initialisation vector parameters */
+
+ struct {
+ uint8_t *data;
+ /**< If this member of this structure is set this is a
+ * pointer to the location where the digest result should be
+ * inserted (in the case of digest generation) or where the
+ * purported digest exists (in the case of digest
+ * verification).
+ *
+ * At session creation time, the client specified the digest
+ * result length with the digest_length member of the @ref
+ * rte_crypto_hash_setup_data structure. For physical crypto
+ * devices the caller must allocate at least digest_length of
+ * physically contiguous memory at this location.
+ *
+ * For digest generation, the digest result will overwrite
+ * any data at this location.
+ *
+ * @note
+ * For GCM (@ref RTE_CRYPTO_AUTH_AES_GCM), for
+ * "digest result" read "authentication tag T".
+ *
+ * If this member is not set the digest result is understood
+ * to be in the destination buffer for digest generation, and
+ * in the source buffer for digest verification. The location
+ * of the digest result in this case is immediately following
+ * the region over which the digest is computed.
+ */
+ phys_addr_t phys_addr; /**< Physical address of digest */
+ uint32_t length; /**< Length of digest */
+ } digest; /**< Digest parameters */
+
+ struct {
+ uint8_t *data;
+ /**< Pointer to Additional Authenticated Data (AAD) needed for
+ * authenticated cipher mechanisms (CCM and GCM), and to the IV
+ * for SNOW3G authentication
+ * (@ref RTE_CRYPTO_AUTH_SNOW3G_UIA2). For other
+ * authentication mechanisms this pointer is ignored.
+ *
+ * The length of the data pointed to by this field is set up for
+ * the session in the @ref rte_crypto_hash_params structure
+ * as part of the @ref rte_cryptodev_session_create function
+ * call. This length must not exceed 240 bytes.
+ *
+ * Specifically for CCM (@ref RTE_CRYPTO_AUTH_AES_CCM), the
+ * caller should setup this field as follows:
+ *
+ * - the nonce should be written starting at an offset of one
+ * byte into the array, leaving room for the implementation
+ * to write in the flags to the first byte.
+ *
+ * - the additional authentication data itself should be
+ * written starting at an offset of 18 bytes into the array,
+ * leaving room for the length encoding in the first two
+ * bytes of the second block.
+ *
+ * - the array should be big enough to hold the above fields,
+ * plus any padding to round this up to the nearest multiple
+ * of the block size (16 bytes). Padding will be added by the
+ * implementation.
+ *
+ * Finally, for GCM (@ref RTE_CRYPTO_AUTH_AES_GCM), the
+ * caller should setup this field as follows:
+ *
+ * - the AAD is written in starting at byte 0
+ * - the array must be big enough to hold the AAD, plus any
+ * space to round this up to the nearest multiple of the
+ * block size (16 bytes).
+ *
+ * @note
+ * For AES-GMAC (@ref RTE_CRYPTO_AUTH_AES_GMAC) mode of
+ * operation, this field is not used and should be set to 0.
+ * Instead the AAD data should be placed in the source buffer.
+ */
+ phys_addr_t phys_addr; /**< physical address */
+ uint32_t length; /**< Length of digest */
+ } additional_auth; /**< Additional authentication parameters */
+
+ struct rte_mempool *pool; /**< mempool used to allocate crypto op */
+
+ void *user_data; /**< opaque pointer for user data */
+};
+
+
+/**
+ * Reset the fields of a packet mbuf to their default values.
+ *
+ * The given mbuf must have only one segment.
+ *
+ * @param m
+ * The packet mbuf to be resetted.
+ */
+static inline void
+__rte_crypto_op_reset(struct rte_crypto_op *op)
+{
+ op->type = RTE_CRYPTO_OP_SESSIONLESS;
+ op->dst.m = NULL;
+ op->dst.offset = 0;
+}
+
+/** Attach a session to a crypto operation */
+static inline void
+rte_crypto_op_attach_session(struct rte_crypto_op *op,
+ struct rte_cryptodev_session *sess)
+{
+ op->session = sess;
+ op->type = RTE_CRYPTO_OP_WITH_SESSION;
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_CRYPTO_H_ */
diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
new file mode 100644
index 0000000..c5610e8
--- /dev/null
+++ b/lib/librte_cryptodev/rte_cryptodev.c
@@ -0,0 +1,1065 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <sys/types.h>
+#include <sys/queue.h>
+#include <ctype.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdarg.h>
+#include <errno.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <netinet/in.h>
+
+#include <rte_byteorder.h>
+#include <rte_log.h>
+#include <rte_debug.h>
+#include <rte_dev.h>
+#include <rte_interrupts.h>
+#include <rte_pci.h>
+#include <rte_memory.h>
+#include <rte_memcpy.h>
+#include <rte_memzone.h>
+#include <rte_launch.h>
+#include <rte_tailq.h>
+#include <rte_eal.h>
+#include <rte_per_lcore.h>
+#include <rte_lcore.h>
+#include <rte_atomic.h>
+#include <rte_branch_prediction.h>
+#include <rte_common.h>
+#include <rte_ring.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_mbuf.h>
+#include <rte_errno.h>
+#include <rte_spinlock.h>
+#include <rte_string_fns.h>
+
+#include "rte_crypto.h"
+#include "rte_cryptodev.h"
+#include "rte_cryptodev_pmd.h"
+
+struct rte_cryptodev rte_crypto_devices[RTE_CRYPTO_MAX_DEVS];
+
+struct rte_cryptodev *rte_cryptodevs = &rte_crypto_devices[0];
+
+static struct rte_cryptodev_global cryptodev_globals = {
+ .devs = &rte_crypto_devices[0],
+ .data = { NULL },
+ .nb_devs = 0,
+ .max_devs = RTE_CRYPTO_MAX_DEVS
+};
+
+struct rte_cryptodev_global *rte_cryptodev_globals = &cryptodev_globals;
+
+/* spinlock for crypto device callbacks */
+static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER;
+
+
+/**
+ * The user application callback description.
+ *
+ * It contains callback address to be registered by user application,
+ * the pointer to the parameters for callback, and the event type.
+ */
+struct rte_cryptodev_callback {
+ TAILQ_ENTRY(rte_cryptodev_callback) next; /**< Callbacks list */
+ rte_cryptodev_cb_fn cb_fn; /**< Callback address */
+ void *cb_arg; /**< Parameter for callback */
+ enum rte_cryptodev_event_type event; /**< Interrupt event type */
+ uint32_t active; /**< Callback is executing */
+};
+
+int
+rte_cryptodev_create_vdev(const char *name, const char *args)
+{
+ return rte_eal_vdev_init(name, args);
+}
+
+int
+rte_cryptodev_get_dev_id(const char *name) {
+ unsigned i;
+
+ if (name == NULL)
+ return -1;
+
+ for (i = 0; i < rte_cryptodev_globals->max_devs; i++)
+ if (strcmp(rte_cryptodev_globals->devs[i].data->name, name) == 0 &&
+ rte_cryptodev_globals->devs[i].attached ==
+ RTE_CRYPTODEV_ATTACHED)
+ return i;
+
+ return -1;
+}
+
+uint8_t
+rte_cryptodev_count(void)
+{
+ return rte_cryptodev_globals->nb_devs;
+}
+
+uint8_t
+rte_cryptodev_count_devtype(enum rte_cryptodev_type type)
+{
+ uint8_t i, dev_count = 0;
+
+ for (i = 0; i < rte_cryptodev_globals->max_devs; i++)
+ if (rte_cryptodev_globals->devs[i].dev_type == type &&
+ rte_cryptodev_globals->devs[i].attached ==
+ RTE_CRYPTODEV_ATTACHED)
+ dev_count++;
+
+ return dev_count;
+}
+
+int
+rte_cryptodev_socket_id(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev;
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+ return -1;
+
+ dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+ return dev->data->socket_id;
+}
+
+static inline int
+rte_cryptodev_data_alloc(uint8_t dev_id, struct rte_cryptodev_data **data,
+ int socket_id)
+{
+ char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ const struct rte_memzone *mz;
+ int n;
+
+ /* generate memzone name */
+ n = snprintf(mz_name, sizeof(mz_name), "rte_cryptodev_data_%u", dev_id);
+ if (n >= (int)sizeof(mz_name))
+ return -EINVAL;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ mz = rte_memzone_reserve(mz_name,
+ sizeof(struct rte_cryptodev_data),
+ socket_id, 0);
+ } else
+ mz = rte_memzone_lookup(mz_name);
+
+ if (mz == NULL)
+ return -ENOMEM;
+
+ *data = mz->addr;
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ memset(*data, 0, sizeof(struct rte_cryptodev_data));
+
+ return 0;
+}
+
+static uint8_t
+rte_cryptodev_find_free_device_index(void)
+{
+ uint8_t dev_id;
+
+ for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) {
+ if (rte_crypto_devices[dev_id].attached == RTE_CRYPTODEV_DETACHED)
+ return dev_id;
+ }
+ return RTE_CRYPTO_MAX_DEVS;
+}
+
+struct rte_cryptodev *
+rte_cryptodev_pmd_allocate(const char *name, enum pmd_type type, int socket_id)
+{
+ struct rte_cryptodev *cryptodev;
+ uint8_t dev_id;
+
+ if (rte_cryptodev_pmd_get_named_dev(name) != NULL) {
+ CDEV_LOG_ERR("Crypto device with name %s already "
+ "allocated!", name);
+ return NULL;
+ }
+
+ dev_id = rte_cryptodev_find_free_device_index();
+ if (dev_id == RTE_CRYPTO_MAX_DEVS) {
+ CDEV_LOG_ERR("Reached maximum number of crypto devices");
+ return NULL;
+ }
+
+ cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
+
+ if (cryptodev->data == NULL) {
+ struct rte_cryptodev_data *cryptodev_data =
+ cryptodev_globals.data[dev_id];
+
+ int retval = rte_cryptodev_data_alloc(dev_id, &cryptodev_data,
+ socket_id);
+
+ if (retval < 0 || cryptodev_data == NULL)
+ return NULL;
+
+ cryptodev->data = cryptodev_data;
+
+ snprintf(cryptodev->data->name, RTE_CRYPTODEV_NAME_MAX_LEN,
+ "%s", name);
+
+ cryptodev->data->dev_id = dev_id;
+ cryptodev->data->socket_id = socket_id;
+ cryptodev->data->dev_started = 0;
+
+ cryptodev->attached = RTE_CRYPTODEV_ATTACHED;
+ cryptodev->pmd_type = type;
+
+ cryptodev_globals.nb_devs++;
+ }
+
+ return cryptodev;
+}
+
+static inline int
+rte_cryptodev_create_unique_device_name(char *name, size_t size,
+ struct rte_pci_device *pci_dev)
+{
+ int ret;
+
+ if ((name == NULL) || (pci_dev == NULL))
+ return -EINVAL;
+
+ ret = snprintf(name, size, "%d:%d.%d",
+ pci_dev->addr.bus, pci_dev->addr.devid,
+ pci_dev->addr.function);
+ if (ret < 0)
+ return ret;
+ return 0;
+}
+
+int
+rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev)
+{
+ int ret;
+ if (cryptodev == NULL)
+ return -EINVAL;
+
+ ret = rte_cryptodev_close(cryptodev->data->dev_id);
+ if (ret < 0)
+ return ret;
+
+ cryptodev->attached = RTE_CRYPTODEV_DETACHED;
+ cryptodev_globals.nb_devs--;
+ return 0;
+}
+
+struct rte_cryptodev *
+rte_cryptodev_pmd_virtual_dev_init(const char *name, size_t dev_private_size,
+ int socket_id)
+{
+ struct rte_cryptodev *cryptodev;
+
+ /* allocate device structure */
+ cryptodev = rte_cryptodev_pmd_allocate(name, PMD_VDEV, socket_id);
+ if (cryptodev == NULL)
+ return NULL;
+
+ /* allocate private device structure */
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ cryptodev->data->dev_private =
+ rte_zmalloc_socket("cryptodev device private",
+ dev_private_size,
+ RTE_CACHE_LINE_SIZE,
+ socket_id);
+
+ if (cryptodev->data->dev_private == NULL)
+ rte_panic("Cannot allocate memzone for private device"
+ " data");
+ }
+
+ /* initialise user call-back tail queue */
+ TAILQ_INIT(&(cryptodev->link_intr_cbs));
+
+ return cryptodev;
+}
+
+static int
+rte_cryptodev_init(struct rte_pci_driver *pci_drv,
+ struct rte_pci_device *pci_dev)
+{
+ struct rte_cryptodev_driver *cryptodrv;
+ struct rte_cryptodev *cryptodev;
+
+ char cryptodev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+
+ int retval;
+
+ cryptodrv = (struct rte_cryptodev_driver *)pci_drv;
+ if (cryptodrv == NULL)
+ return -ENODEV;
+
+ /* Create unique Crypto device name using PCI address */
+ rte_cryptodev_create_unique_device_name(cryptodev_name,
+ sizeof(cryptodev_name), pci_dev);
+
+ cryptodev = rte_cryptodev_pmd_allocate(cryptodev_name, PMD_PDEV, rte_socket_id());
+ if (cryptodev == NULL)
+ return -ENOMEM;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ cryptodev->data->dev_private =
+ rte_zmalloc_socket("cryptodev private structure",
+ cryptodrv->dev_private_size,
+ RTE_CACHE_LINE_SIZE, rte_socket_id());
+
+ if (cryptodev->data->dev_private == NULL)
+ rte_panic("Cannot allocate memzone for private device data");
+ }
+
+ cryptodev->pci_dev = pci_dev;
+ cryptodev->driver = cryptodrv;
+
+ /* init user callbacks */
+ TAILQ_INIT(&(cryptodev->link_intr_cbs));
+
+ /* Invoke PMD device initialization function */
+ retval = (*cryptodrv->cryptodev_init)(cryptodrv, cryptodev);
+ if (retval == 0)
+ return 0;
+
+ CDEV_LOG_ERR("driver %s: crypto_dev_init(vendor_id=0x%u device_id=0x%x)"
+ " failed", pci_drv->name,
+ (unsigned) pci_dev->id.vendor_id,
+ (unsigned) pci_dev->id.device_id);
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ rte_free(cryptodev->data->dev_private);
+
+ cryptodev->attached = RTE_CRYPTODEV_DETACHED;
+ cryptodev_globals.nb_devs--;
+
+ return -ENXIO;
+}
+
+static int
+rte_cryptodev_uninit(struct rte_pci_device *pci_dev)
+{
+ const struct rte_cryptodev_driver *cryptodrv;
+ struct rte_cryptodev *cryptodev;
+ char cryptodev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ int ret;
+
+ if (pci_dev == NULL)
+ return -EINVAL;
+
+ /* Create unique device name using PCI address */
+ rte_cryptodev_create_unique_device_name(cryptodev_name,
+ sizeof(cryptodev_name), pci_dev);
+
+ cryptodev = rte_cryptodev_pmd_get_named_dev(cryptodev_name);
+ if (cryptodev == NULL)
+ return -ENODEV;
+
+ cryptodrv = (const struct rte_cryptodev_driver *)pci_dev->driver;
+ if (cryptodrv == NULL)
+ return -ENODEV;
+
+ /* Invoke PMD device uninit function */
+ if (*cryptodrv->cryptodev_uninit) {
+ ret = (*cryptodrv->cryptodev_uninit)(cryptodrv, cryptodev);
+ if (ret)
+ return ret;
+ }
+
+ /* free crypto device */
+ rte_cryptodev_pmd_release_device(cryptodev);
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ rte_free(cryptodev->data->dev_private);
+
+ cryptodev->pci_dev = NULL;
+ cryptodev->driver = NULL;
+ cryptodev->data = NULL;
+
+ return 0;
+}
+
+int
+rte_cryptodev_pmd_driver_register(struct rte_cryptodev_driver *cryptodrv,
+ enum pmd_type type)
+{
+ /* Call crypto device initialization directly if device is virtual */
+ if (type == PMD_VDEV)
+ return rte_cryptodev_init((struct rte_pci_driver *)cryptodrv,
+ NULL);
+
+ /* Register PCI driver for physical device intialisation during
+ * PCI probing */
+ cryptodrv->pci_drv.devinit = rte_cryptodev_init;
+ cryptodrv->pci_drv.devuninit = rte_cryptodev_uninit;
+
+ rte_eal_pci_register(&cryptodrv->pci_drv);
+
+ return 0;
+}
+
+
+uint16_t
+rte_cryptodev_queue_pair_count(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev;
+
+ dev = &rte_crypto_devices[dev_id];
+ return dev->data->nb_queue_pairs;
+}
+
+static int
+rte_cryptodev_queue_pairs_config(struct rte_cryptodev *dev, uint16_t nb_qpairs, int socket_id)
+{
+ struct rte_cryptodev_info dev_info;
+ void **qp;
+ unsigned i;
+
+ if ((dev == NULL) || (nb_qpairs < 1)) {
+ CDEV_LOG_ERR("invalid param: dev %p, nb_queues %u",
+ dev, nb_qpairs);
+ return -EINVAL;
+ }
+
+ CDEV_LOG_DEBUG("Setup %d queues pairs on device %u",
+ nb_qpairs, dev->data->dev_id);
+
+ memset(&dev_info, 0, sizeof(struct rte_cryptodev_info));
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP);
+ (*dev->dev_ops->dev_infos_get)(dev, &dev_info);
+
+ if (nb_qpairs > (dev_info.max_queue_pairs)) {
+ CDEV_LOG_ERR("Invalid num queue_pairs (%u) for dev %u",
+ nb_qpairs, dev->data->dev_id);
+ return (-EINVAL);
+ }
+
+ if (dev->data->queue_pairs == NULL) { /* first time configuration */
+ dev->data->queue_pairs = rte_zmalloc_socket(
+ "cryptodev->queue_pairs",
+ sizeof(dev->data->queue_pairs[0]) * nb_qpairs,
+ RTE_CACHE_LINE_SIZE, socket_id);
+
+ if (dev->data->queue_pairs == NULL) {
+ dev->data->nb_queue_pairs = 0;
+ CDEV_LOG_ERR("failed to get memory for qp meta data, "
+ "nb_queues %u", nb_qpairs);
+ return -(ENOMEM);
+ }
+ } else { /* re-configure */
+ int ret;
+ uint16_t old_nb_queues = dev->data->nb_queue_pairs;
+ qp = dev->data->queue_pairs;
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_release, -ENOTSUP);
+
+ for (i = nb_qpairs; i < old_nb_queues; i++) {
+ ret = (*dev->dev_ops->queue_pair_release)(dev, i);
+ if (ret < 0)
+ return ret;
+ }
+ qp = rte_realloc(qp, sizeof(qp[0]) * nb_qpairs,
+ RTE_CACHE_LINE_SIZE);
+ if (qp == NULL) {
+ CDEV_LOG_ERR("failed to realloc qp meta data,"
+ " nb_queues %u", nb_qpairs);
+ return -(ENOMEM);
+ }
+ if (nb_qpairs > old_nb_queues) {
+ uint16_t new_qs = nb_qpairs - old_nb_queues;
+
+ memset(qp + old_nb_queues, 0,
+ sizeof(qp[0]) * new_qs);
+ }
+
+ dev->data->queue_pairs = qp;
+
+ }
+ dev->data->nb_queue_pairs = nb_qpairs;
+ return 0;
+}
+
+int
+rte_cryptodev_queue_pair_start(uint8_t dev_id, uint16_t queue_pair_id)
+{
+ struct rte_cryptodev *dev;
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return -EINVAL;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+ if (queue_pair_id >= dev->data->nb_queue_pairs) {
+ CDEV_LOG_ERR("Invalid queue_pair_id=%d", queue_pair_id);
+ return -EINVAL;
+ }
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_start, -ENOTSUP);
+
+ return dev->dev_ops->queue_pair_start(dev, queue_pair_id);
+
+}
+
+int
+rte_cryptodev_queue_pair_stop(uint8_t dev_id, uint16_t queue_pair_id)
+{
+ struct rte_cryptodev *dev;
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return -EINVAL;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+ if (queue_pair_id >= dev->data->nb_queue_pairs) {
+ CDEV_LOG_ERR("Invalid queue_pair_id=%d", queue_pair_id);
+ return -EINVAL;
+ }
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_stop, -ENOTSUP);
+
+ return dev->dev_ops->queue_pair_stop(dev, queue_pair_id);
+
+}
+
+static int
+rte_crypto_session_pool_create(struct rte_cryptodev *dev, unsigned nb_objs,
+ unsigned obj_cache_size, int socket_id);
+
+int
+rte_cryptodev_configure(uint8_t dev_id, struct rte_cryptodev_config *config)
+{
+ struct rte_cryptodev *dev;
+ int diag;
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return (-EINVAL);
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ if (dev->data->dev_started) {
+ CDEV_LOG_ERR(
+ "device %d must be stopped to allow configuration", dev_id);
+ return (-EBUSY);
+ }
+
+ /* Setup new number of queue pairs and reconfigure device. */
+ diag = rte_cryptodev_queue_pairs_config(dev, config->nb_queue_pairs,
+ config->socket_id);
+ if (diag != 0) {
+ CDEV_LOG_ERR("dev%d rte_crypto_dev_queue_pairs_config = %d",
+ dev_id, diag);
+ return diag;
+ }
+
+ /* Setup Session mempool for device */
+ return rte_crypto_session_pool_create(dev, config->session_mp.nb_objs,
+ config->session_mp.cache_size, config->socket_id);
+}
+
+
+int
+rte_cryptodev_start(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev;
+ int diag;
+
+ CDEV_LOG_DEBUG("Start dev_id=%" PRIu8, dev_id);
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return (-EINVAL);
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP);
+
+ if (dev->data->dev_started != 0) {
+ CDEV_LOG_ERR("Device with dev_id=%" PRIu8 " already started",
+ dev_id);
+ return 0;
+ }
+
+ diag = (*dev->dev_ops->dev_start)(dev);
+ if (diag == 0)
+ dev->data->dev_started = 1;
+ else
+ return diag;
+
+ return 0;
+}
+
+void
+rte_cryptodev_stop(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev;
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_RET();
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop);
+
+ if (dev->data->dev_started == 0) {
+ CDEV_LOG_ERR("Device with dev_id=%" PRIu8 " already stopped",
+ dev_id);
+ return;
+ }
+
+ dev->data->dev_started = 0;
+ (*dev->dev_ops->dev_stop)(dev);
+}
+
+int
+rte_cryptodev_close(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev;
+ int retval;
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_ERR_RET(-EINVAL);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return -1;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ /* Device must be stopped before it can be closed */
+ if (dev->data->dev_started == 1) {
+ CDEV_LOG_ERR("Device %" PRIu8 " must be stopped before closing",
+ dev_id);
+ return -EBUSY;
+ }
+
+ /* We can't close the device if there are outstanding sessions in
+ * existence */
+ if (dev->data->session_pool != NULL) {
+ if (!rte_mempool_full(dev->data->session_pool)) {
+ CDEV_LOG_ERR("dev_id=%u close failed, session mempool "
+ "has sessions still in use, free "
+ "all sessions before calling close",
+ (unsigned)dev_id);
+ return -EBUSY;
+ }
+ }
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP);
+ retval = (*dev->dev_ops->dev_close)(dev);
+
+ if (retval < 0)
+ return retval;
+
+ return 0;
+}
+
+int
+rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id,
+ const struct rte_cryptodev_qp_conf *qp_conf, int socket_id)
+{
+ struct rte_cryptodev *dev;
+
+ /* This function is only safe when called from the primary process
+ * in a multi-process setup*/
+ PROC_PRIMARY_OR_ERR_RET(-E_RTE_SECONDARY);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return (-EINVAL);
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+ if (queue_pair_id >= dev->data->nb_queue_pairs) {
+ CDEV_LOG_ERR("Invalid queue_pair_id=%d", queue_pair_id);
+ return (-EINVAL);
+ }
+
+ if (dev->data->dev_started) {
+ CDEV_LOG_ERR(
+ "device %d must be stopped to allow configuration", dev_id);
+ return -EBUSY;
+ }
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_setup, -ENOTSUP);
+
+ return (*dev->dev_ops->queue_pair_setup)(dev, queue_pair_id, qp_conf,
+ socket_id);
+}
+
+
+int
+rte_cryptodev_stats_get(uint8_t dev_id, struct rte_cryptodev_stats *stats)
+{
+ struct rte_cryptodev *dev;
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%d", dev_id);
+ return (-ENODEV);
+ }
+
+ if (stats == NULL) {
+ CDEV_LOG_ERR("Invalid stats ptr");
+ return -EINVAL;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+ memset(stats, 0, sizeof(*stats));
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stats_get, -ENOTSUP);
+ (*dev->dev_ops->stats_get)(dev, stats);
+ return 0;
+}
+
+void
+rte_cryptodev_stats_reset(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev;
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset);
+ (*dev->dev_ops->stats_reset)(dev);
+}
+
+
+void
+rte_cryptodev_info_get(uint8_t dev_id, struct rte_cryptodev_info *dev_info)
+{
+ struct rte_cryptodev *dev;
+
+ if (dev_id >= cryptodev_globals.nb_devs) {
+ CDEV_LOG_ERR("Invalid dev_id=%d", dev_id);
+ return;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ memset(dev_info, 0, sizeof(struct rte_cryptodev_info));
+
+ FUNC_PTR_OR_RET(*dev->dev_ops->dev_infos_get);
+ (*dev->dev_ops->dev_infos_get)(dev, dev_info);
+
+ dev_info->pci_dev = dev->pci_dev;
+ if (dev->driver)
+ dev_info->driver_name = dev->driver->pci_drv.name;
+}
+
+
+int
+rte_cryptodev_callback_register(uint8_t dev_id,
+ enum rte_cryptodev_event_type event,
+ rte_cryptodev_cb_fn cb_fn, void *cb_arg)
+{
+ struct rte_cryptodev *dev;
+ struct rte_cryptodev_callback *user_cb;
+
+ if (!cb_fn)
+ return (-EINVAL);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return (-EINVAL);
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+ rte_spinlock_lock(&rte_cryptodev_cb_lock);
+
+ TAILQ_FOREACH(user_cb, &(dev->link_intr_cbs), next) {
+ if (user_cb->cb_fn == cb_fn &&
+ user_cb->cb_arg == cb_arg &&
+ user_cb->event == event) {
+ break;
+ }
+ }
+
+ /* create a new callback. */
+ if (user_cb == NULL) {
+ user_cb = rte_zmalloc("INTR_USER_CALLBACK",
+ sizeof(struct rte_cryptodev_callback), 0);
+ if (user_cb != NULL) {
+ user_cb->cb_fn = cb_fn;
+ user_cb->cb_arg = cb_arg;
+ user_cb->event = event;
+ TAILQ_INSERT_TAIL(&(dev->link_intr_cbs), user_cb, next);
+ }
+ }
+
+ rte_spinlock_unlock(&rte_cryptodev_cb_lock);
+ return ((user_cb == NULL) ? -ENOMEM : 0);
+}
+
+int
+rte_cryptodev_callback_unregister(uint8_t dev_id,
+ enum rte_cryptodev_event_type event,
+ rte_cryptodev_cb_fn cb_fn, void *cb_arg)
+{
+ int ret;
+ struct rte_cryptodev *dev;
+ struct rte_cryptodev_callback *cb, *next;
+
+ if (!cb_fn)
+ return (-EINVAL);
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id);
+ return (-EINVAL);
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+ rte_spinlock_lock(&rte_cryptodev_cb_lock);
+
+ ret = 0;
+ for (cb = TAILQ_FIRST(&dev->link_intr_cbs); cb != NULL; cb = next) {
+
+ next = TAILQ_NEXT(cb, next);
+
+ if (cb->cb_fn != cb_fn || cb->event != event ||
+ (cb->cb_arg != (void *)-1 &&
+ cb->cb_arg != cb_arg))
+ continue;
+
+ /*
+ * if this callback is not executing right now,
+ * then remove it.
+ */
+ if (cb->active == 0) {
+ TAILQ_REMOVE(&(dev->link_intr_cbs), cb, next);
+ rte_free(cb);
+ } else {
+ ret = -EAGAIN;
+ }
+ }
+
+ rte_spinlock_unlock(&rte_cryptodev_cb_lock);
+ return ret;
+}
+
+void
+rte_cryptodev_pmd_callback_process(struct rte_cryptodev *dev,
+ enum rte_cryptodev_event_type event)
+{
+ struct rte_cryptodev_callback *cb_lst;
+ struct rte_cryptodev_callback dev_cb;
+
+ rte_spinlock_lock(&rte_cryptodev_cb_lock);
+ TAILQ_FOREACH(cb_lst, &(dev->link_intr_cbs), next) {
+ if (cb_lst->cb_fn == NULL || cb_lst->event != event)
+ continue;
+ dev_cb = *cb_lst;
+ cb_lst->active = 1;
+ rte_spinlock_unlock(&rte_cryptodev_cb_lock);
+ dev_cb.cb_fn(dev->data->dev_id, dev_cb.event,
+ dev_cb.cb_arg);
+ rte_spinlock_lock(&rte_cryptodev_cb_lock);
+ cb_lst->active = 0;
+ }
+ rte_spinlock_unlock(&rte_cryptodev_cb_lock);
+}
+
+
+static void
+rte_crypto_session_init(struct rte_mempool *mp,
+ void *opaque_arg,
+ void *_sess,
+ __rte_unused unsigned i)
+{
+ struct rte_cryptodev_session *sess = _sess;
+ struct rte_cryptodev *dev = opaque_arg;
+
+ memset(sess, 0, mp->elt_size);
+
+ sess->dev_id = dev->data->dev_id;
+ sess->type = dev->dev_type;
+ sess->mp = mp;
+
+ if (dev->dev_ops->session_initialize)
+ (*dev->dev_ops->session_initialize)(mp, sess->_private);
+}
+
+static int
+rte_crypto_session_pool_create(struct rte_cryptodev *dev, unsigned nb_objs,
+ unsigned obj_cache_size, int socket_id)
+{
+ char mp_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ unsigned priv_sess_size;
+
+ unsigned n = snprintf(mp_name, sizeof(mp_name), "cdev_%d_sess_mp",
+ dev->data->dev_id);
+ if (n > sizeof(mp_name)) {
+ CDEV_LOG_ERR("Unable to create unique name for session mempool");
+ return -ENOMEM;
+ }
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->session_get_size, -ENOTSUP);
+ priv_sess_size = (*dev->dev_ops->session_get_size)(dev);
+ if (priv_sess_size == 0) {
+ CDEV_LOG_ERR("%s returned and invalid private session size ",
+ dev->data->name);
+ return -ENOMEM;
+ }
+
+ unsigned elt_size = sizeof(struct rte_cryptodev_session) + priv_sess_size;
+
+ dev->data->session_pool = rte_mempool_lookup(mp_name);
+ if (dev->data->session_pool != NULL) {
+ if (dev->data->session_pool->elt_size != elt_size ||
+ dev->data->session_pool->cache_size < obj_cache_size ||
+ dev->data->session_pool->size < nb_objs) {
+
+ CDEV_LOG_ERR("%s mempool already exists with different "
+ "initialization parameters", mp_name);
+ dev->data->session_pool = NULL;
+ return -ENOMEM;
+ }
+ } else {
+ dev->data->session_pool = rte_mempool_create(
+ mp_name, /* mempool name */
+ nb_objs, /* number of elements*/
+ elt_size, /* element size*/
+ obj_cache_size, /* Cache size*/
+ 0, /* private data size */
+ NULL, /* obj initialization constructor */
+ NULL, /* obj initialization constructor arg */
+ rte_crypto_session_init, /* obj constructor */
+ dev, /* obj constructor arg */
+ socket_id, /* socket id */
+ 0); /* flags */
+
+ if (dev->data->session_pool == NULL) {
+ CDEV_LOG_ERR("%s mempool allocation failed", mp_name);
+ return -ENOMEM;
+ }
+ }
+
+ CDEV_LOG_DEBUG("%s mempool created!", mp_name);
+ return 0;
+}
+
+struct rte_cryptodev_session *
+rte_cryptodev_session_create(uint8_t dev_id, struct rte_crypto_xform *xform)
+{
+ struct rte_cryptodev *dev;
+ struct rte_cryptodev_session *sess;
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%d", dev_id);
+ return NULL;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ /* Allocate a session structure from the session pool */
+ if (rte_mempool_get(dev->data->session_pool, (void **)&sess)) {
+ CDEV_LOG_ERR("Couldn't get object from session mempool");
+ return NULL;
+ }
+
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->session_configure, NULL);
+ if (dev->dev_ops->session_configure(dev, xform, sess->_private) ==
+ NULL) {
+ CDEV_LOG_ERR("dev_id %d failed to configure session details",
+ dev_id);
+
+ /* Return session to mempool */
+ rte_mempool_put(sess->mp, (void *)sess);
+ return NULL;
+ }
+
+ return sess;
+}
+
+struct rte_cryptodev_session *
+rte_cryptodev_session_free(uint8_t dev_id, struct rte_cryptodev_session *sess)
+{
+ struct rte_cryptodev *dev;
+
+ if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
+ CDEV_LOG_ERR("Invalid dev_id=%d", dev_id);
+ return sess;
+ }
+
+ dev = &rte_crypto_devices[dev_id];
+
+ /* Check the session belongs to this device type */
+ if (sess->type != dev->dev_type)
+ return sess;
+
+ /* Let device implementation clear session material */
+ FUNC_PTR_OR_ERR_RET(*dev->dev_ops->session_clear, sess);
+ dev->dev_ops->session_clear(dev, (void *)sess->_private);
+
+ /* Return session to mempool */
+ rte_mempool_put(sess->mp, (void *)sess);
+
+ return NULL;
+}
+
+
+
+
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
new file mode 100644
index 0000000..9b6ee14
--- /dev/null
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -0,0 +1,619 @@
+/*-
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTODEV_H_
+#define _RTE_CRYPTODEV_H_
+
+/**
+ * @file rte_cryptodev.h
+ *
+ * RTE Cryptographic Device APIs
+ *
+ * Defines RTE Crypto Device APIs for the provisioning of cipher and
+ * authentication operations.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "stddef.h"
+
+#include "rte_crypto.h"
+#include "rte_dev.h"
+
+#define CRYPTODEV_NAME_NULL_PMD ("cryptodev_null_pmd")
+/**< Null crypto PMD device name */
+#define CRYPTODEV_NAME_AESNI_MB_PMD ("cryptodev_aesni_mb_pmd")
+/**< AES-NI Multi buffer PMD device name */
+#define CRYPTODEV_NAME_QAT_PMD ("cryptodev_qat_pmd")
+/**< Intel QAT PMD device name */
+
+/** Crypto device type */
+enum rte_cryptodev_type {
+ RTE_CRYPTODEV_NULL_PMD = 1, /**< Null crypto PMD */
+ RTE_CRYPTODEV_AESNI_MB_PMD, /**< AES-NI multi buffer PMD */
+ RTE_CRYPTODEV_QAT_PMD, /**< QAT PMD */
+};
+
+/* Logging Macros */
+
+#define CDEV_LOG_ERR(fmt, args...) \
+ RTE_LOG(ERR, CRYPTODEV, "%s() line %u: " fmt "\n", \
+ __func__, __LINE__, ## args)
+
+#define CDEV_PMD_LOG_ERR(dev, fmt, args...) \
+ RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ dev, __func__, __LINE__, ## args)
+
+#ifdef RTE_LIBRTE_CRYPTODEV_DEBUG
+#define CDEV_LOG_DEBUG(fmt, args...) \
+ RTE_LOG(DEBUG, CRYPTODEV, "%s() line %u: " fmt "\n", \
+ __func__, __LINE__, ## args) \
+
+#define CDEV_PMD_TRACE(fmt, args...) \
+ RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s: " fmt "\n", \
+ dev, __func__, ## args)
+
+#else
+#define CDEV_LOG_DEBUG(fmt, args...)
+#define CDEV_PMD_TRACE(fmt, args...)
+#endif
+
+/** Crypto device information */
+struct rte_cryptodev_info {
+ const char *driver_name; /**< Driver name. */
+ enum rte_cryptodev_type dev_type; /**< Device type */
+ struct rte_pci_device *pci_dev; /**< PCI information. */
+ uint16_t max_queue_pairs; /**< Maximum number of queues
+ * pairs supported by device.
+ */
+};
+
+#define RTE_CRYPTODEV_DETACHED (0)
+#define RTE_CRYPTODEV_ATTACHED (1)
+
+/** Definitions of Crypto device event types */
+enum rte_cryptodev_event_type {
+ RTE_CRYPTODEV_EVENT_UNKNOWN, /**< unknown event type */
+ RTE_CRYPTODEV_EVENT_ERROR, /**< error interrupt event */
+ RTE_CRYPTODEV_EVENT_MAX /**< max value of this enum */
+};
+
+/** Crypto device queue pair configuration structure. */
+struct rte_cryptodev_qp_conf {
+ uint32_t nb_descriptors; /**< Number of descriptors per queue pair */
+};
+
+/**
+ * Typedef for application callback function to be registered by application
+ * software for notification of device events
+ *
+ * @param dev_id Crypto device identifier
+ * @param event Crypto device event to register for notification of.
+ * @param cb_arg User specified parameter to be passed as to passed to
+ * users callback function.
+ */
+typedef void (*rte_cryptodev_cb_fn)(uint8_t dev_id,
+ enum rte_cryptodev_event_type event, void *cb_arg);
+
+#ifdef RTE_CRYPTODEV_PERF
+/**
+ * Crypto Device performance counter statistics structure. This structure is
+ * used for RDTSC counters for measuring crypto operations.
+ */
+struct rte_cryptodev_perf_stats {
+ uint64_t t_accumlated; /**< Accumulated time processing operation */
+ uint64_t t_min; /**< Max time */
+ uint64_t t_max; /**< Min time */
+};
+#endif
+
+/** Crypto Device statistics */
+struct rte_cryptodev_stats {
+ uint64_t enqueued_count; /**< Count of all operations enqueued */
+ uint64_t dequeued_count; /**< Count of all operations dequeued */
+
+ uint64_t enqueue_err_count; /**< Total error count on operations enqueued */
+ uint64_t dequeue_err_count; /**< Total error count on operations dequeued */
+
+#ifdef RTE_CRYPTODEV_DETAILED_STATS
+ struct {
+ uint64_t encrypt_ops; /**< Count of encrypt operations */
+ uint64_t encrypt_bytes; /**< Number of bytes encrypted */
+
+ uint64_t decrypt_ops; /**< Count of decrypt operations */
+ uint64_t decrypt_bytes; /**< Number of bytes decrypted */
+ } cipher; /**< Cipher operations stats */
+
+ struct {
+ uint64_t generate_ops; /**< Count of generate operations */
+ uint64_t bytes_hashed; /**< Number of bytes hashed */
+
+ uint64_t verify_ops; /**< Count of verify operations */
+ uint64_t bytes_verified;/**< Number of bytes verified */
+ } hash; /**< Hash operations stats */
+#endif
+
+#ifdef RTE_CRYPTODEV_PERF
+ struct rte_cryptodev_perf_stats op_perf; /**< Operations stats */
+#endif
+} __rte_cache_aligned;
+
+/**
+ * Create a virtual crypto device
+ *
+ * @param name Cryptodev PMD name of device to be created.
+ * @param args Options arguments for device.
+ *
+ * @return
+ * - On successful creation of the cryptodev the device index is returned,
+ * which will be between 0 and rte_cryptodev_count().
+ * - In the case of a failure, returns -1.
+ */
+extern int
+rte_cryptodev_create_vdev(const char *name, const char *args);
+
+/**
+ * Get the device identifier for the named crypto device.
+ *
+ * @param name device name to select the device structure.
+ *
+ * @return
+ * - Returns crypto device identifier on success.
+ * - Return -1 on failure to find named crypto device.
+ */
+extern int
+rte_cryptodev_get_dev_id(const char *name);
+
+/**
+ * Get the total number of crypto devices that have been successfully
+ * initialised.
+ *
+ * @return
+ * - The total number of usable crypto devices.
+ */
+extern uint8_t
+rte_cryptodev_count(void);
+
+extern uint8_t
+rte_cryptodev_count_devtype(enum rte_cryptodev_type type);
+/*
+ * Return the NUMA socket to which a device is connected
+ *
+ * @param dev_id
+ * The identifier of the device
+ * @return
+ * The NUMA socket id to which the device is connected or
+ * a default of zero if the socket could not be determined.
+ * -1 if returned is the dev_id value is out of range.
+ */
+extern int
+rte_cryptodev_socket_id(uint8_t dev_id);
+
+/** Crypto device configuration structure */
+struct rte_cryptodev_config {
+ int socket_id; /**< Socket to allocate resources on */
+ uint16_t nb_queue_pairs; /**< Number of queue pairs to configure
+ * on device */
+
+ struct {
+ uint32_t nb_objs; /**< Number of objects in mempool */
+ uint32_t cache_size; /**< l-core object cache size */
+ } session_mp; /**< Session mempool configuration */
+};
+
+/**
+ * Configure a device.
+ *
+ * This function must be invoked first before any other function in the
+ * API. This function can also be re-invoked when a device is in the
+ * stopped state.
+ *
+ * @param dev_id The identifier of the device to configure.
+ * @param nb_qp_queue The number of queue pairs to set up for the device.
+ *
+ * @return
+ * - 0: Success, device configured.
+ * - <0: Error code returned by the driver configuration function.
+ */
+extern int
+rte_cryptodev_configure(uint8_t dev_id, struct rte_cryptodev_config *config);
+
+/**
+ * Start an device.
+ *
+ * The device start step is the last one and consists of setting the configured
+ * offload features and in starting the transmit and the receive units of the
+ * device.
+ * On success, all basic functions exported by the API (link status,
+ * receive/transmit, and so on) can be invoked.
+ *
+ * @param dev_id
+ * The identifier of the device.
+ * @return
+ * - 0: Success, device started.
+ * - <0: Error code of the driver device start function.
+ */
+extern int
+rte_cryptodev_start(uint8_t dev_id);
+
+/**
+ * Stop an device. The device can be restarted with a call to
+ * rte_cryptodev_start()
+ *
+ * @param dev_id The identifier of the device.
+ */
+extern void
+rte_cryptodev_stop(uint8_t dev_id);
+
+/**
+ * Close an device. The device cannot be restarted!
+ *
+ * @param dev_id The identifier of the device.
+ *
+ * @return
+ * - 0 on successfully closing device
+ * - <0 on failure to close device
+ */
+extern int
+rte_cryptodev_close(uint8_t dev_id);
+
+/**
+ * Allocate and set up a receive queue pair for a device.
+ *
+ *
+ * @param dev_id The identifier of the device.
+ * @param queue_pair_id The index of the queue pairs to set up. The
+ * value must be in the range [0, nb_queue_pair
+ * - 1] previously supplied to
+ * rte_cryptodev_configure().
+ * @param qp_conf The pointer to the configuration data to be
+ * used for the queue pair. NULL value is
+ * allowed, in which case default configuration
+ * will be used.
+ * @param socket_id The *socket_id* argument is the socket
+ * identifier in case of NUMA. The value can be
+ * *SOCKET_ID_ANY* if there is no NUMA constraint
+ * for the DMA memory allocated for the receive
+ * queue pair.
+ *
+ * @return
+ * - 0: Success, queue pair correctly set up.
+ * - <0: Queue pair configuration failed
+ */
+extern int
+rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id,
+ const struct rte_cryptodev_qp_conf *qp_conf, int socket_id);
+
+/**
+ * Start a specified queue pair of a device. It is used
+ * when deferred_start flag of the specified queue is true.
+ *
+ * @param dev_id The identifier of the device
+ * @param queue_pair_id The index of the queue pair to start. The value
+ * must be in the range [0, nb_queue_pair - 1]
+ * previously supplied to rte_crypto_dev_configure().
+ * @return
+ * - 0: Success, the transmit queue is correctly set up.
+ * - -EINVAL: The dev_id or the queue_id out of range.
+ * - -ENOTSUP: The function not supported in PMD driver.
+ */
+extern int
+rte_cryptodev_queue_pair_start(uint8_t dev_id, uint16_t queue_pair_id);
+
+/**
+ * Stop specified queue pair of a device
+ *
+ * @param dev_id The identifier of the device
+ * @param queue_pair_id The index of the queue pair to stop. The value
+ * must be in the range [0, nb_queue_pair - 1]
+ * previously supplied to rte_cryptodev_configure().
+ * @return
+ * - 0: Success, the transmit queue is correctly set up.
+ * - -EINVAL: The dev_id or the queue_id out of range.
+ * - -ENOTSUP: The function not supported in PMD driver.
+ */
+extern int
+rte_cryptodev_queue_pair_stop(uint8_t dev_id, uint16_t queue_pair_id);
+
+/**
+ * Get the number of queue pairs on a specific crypto device
+ *
+ * @param dev_id Crypto device identifier.
+ * @return
+ * - The number of configured queue pairs.
+ */
+extern uint16_t
+rte_cryptodev_queue_pair_count(uint8_t dev_id);
+
+
+/**
+ * Retrieve the general I/O statistics of a device.
+ *
+ * @param dev_id The identifier of the device.
+ * @param stats A pointer to a structure of type
+ * *rte_cryptodev_stats* to be filled with the
+ * values of device counters.
+ * @return
+ * - Zero if successful.
+ * - Non-zero otherwise.
+ */
+extern int
+rte_cryptodev_stats_get(uint8_t dev_id, struct rte_cryptodev_stats *stats);
+
+/**
+ * Reset the general I/O statistics of a device.
+ *
+ * @param dev_id The identifier of the device.
+ */
+extern void
+rte_cryptodev_stats_reset(uint8_t dev_id);
+
+/**
+ * Retrieve the contextual information of a device.
+ *
+ * @param dev_id The identifier of the device.
+ * @param dev_info A pointer to a structure of type
+ * *rte_cryptodev_info* to be filled with the
+ * contextual information of the device.
+ */
+extern void
+rte_cryptodev_info_get(uint8_t dev_id, struct rte_cryptodev_info *dev_info);
+
+
+/**
+ * Register a callback function for specific device id.
+ *
+ * @param dev_id Device id.
+ * @param event Event interested.
+ * @param cb_fn User supplied callback function to be called.
+ * @param cb_arg Pointer to the parameters for the registered callback.
+ *
+ * @return
+ * - On success, zero.
+ * - On failure, a negative value.
+ */
+extern int
+rte_cryptodev_callback_register(uint8_t dev_id,
+ enum rte_cryptodev_event_type event,
+ rte_cryptodev_cb_fn cb_fn, void *cb_arg);
+
+/**
+ * Unregister a callback function for specific device id.
+ *
+ * @param dev_id The device identifier.
+ * @param event Event interested.
+ * @param cb_fn User supplied callback function to be called.
+ * @param cb_arg Pointer to the parameters for the registered callback.
+ *
+ * @return
+ * - On success, zero.
+ * - On failure, a negative value.
+ */
+extern int
+rte_cryptodev_callback_unregister(uint8_t dev_id,
+ enum rte_cryptodev_event_type event,
+ rte_cryptodev_cb_fn cb_fn, void *cb_arg);
+
+
+typedef uint16_t (*dequeue_pkt_burst_t)(void *qp, struct rte_mbuf **pkts,
+ uint16_t nb_pkts);
+/**< Dequeue processed packets from queue pair of a device. */
+
+typedef uint16_t (*enqueue_pkt_burst_t)(void *qp, struct rte_mbuf **pkts,
+ uint16_t nb_pkts);
+/**< Enqueue packets for processing on queue pair of a device. */
+
+
+struct rte_cryptodev_callback;
+
+/** Structure to keep track of registered callbacks */
+TAILQ_HEAD(rte_cryptodev_cb_list, rte_cryptodev_callback);
+
+/** The data structure associated with each crypto device. */
+struct rte_cryptodev {
+ dequeue_pkt_burst_t dequeue_burst; /**< Pointer to PMD receive function. */
+ enqueue_pkt_burst_t enqueue_burst; /**< Pointer to PMD transmit function. */
+
+ const struct rte_cryptodev_driver *driver; /**< Driver for this device */
+ struct rte_cryptodev_data *data; /**< Pointer to device data */
+ struct rte_cryptodev_ops *dev_ops; /**< Functions exported by PMD */
+ struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
+
+ enum rte_cryptodev_type dev_type; /**< Crypto device type */
+ enum pmd_type pmd_type; /**< PMD type - PDEV / VDEV */
+
+ struct rte_cryptodev_cb_list link_intr_cbs;
+ /**< User application callback for interrupts if present */
+
+ uint8_t attached : 1; /**< Flag indicating the device is attached */
+} __rte_cache_aligned;
+
+
+#define RTE_CRYPTODEV_NAME_MAX_LEN (64)
+/**< Max length of name of crypto PMD */
+
+/**
+ *
+ * The data part, with no function pointers, associated with each crypto device.
+ *
+ * This structure is safe to place in shared memory to be common among different
+ * processes in a multi-process configuration.
+ */
+struct rte_cryptodev_data {
+ uint8_t dev_id; /**< Device ID for this instance */
+ uint8_t socket_id; /**< Socket ID where memory is allocated */
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN]; /**< Unique identifier name */
+
+ uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0) */
+
+ struct rte_mempool *session_pool; /**< Session memory pool */
+ void **queue_pairs; /**< Array of pointers to queue pairs. */
+ uint16_t nb_queue_pairs; /**< Number of device queue pairs. */
+
+ void *dev_private; /**< PMD-specific private data */
+} __rte_cache_aligned;
+
+extern struct rte_cryptodev *rte_cryptodevs;
+/**
+ *
+ * Dequeue a burst of processed packets from a queue of the crypto device.
+ * The dequeued packets are stored in *rte_mbuf* structures whose pointers are
+ * supplied in the *pkts* array.
+ *
+ * The rte_crypto_dequeue_burst() function returns the number of packets
+ * actually dequeued, which is the number of *rte_mbuf* data structures
+ * effectively supplied into the *pkts* array.
+ *
+ * A return value equal to *nb_pkts* indicates that the queue contained
+ * at least *rx_pkts* packets, and this is likely to signify that other
+ * received packets remain in the input queue. Applications implementing
+ * a "retrieve as much received packets as possible" policy can check this
+ * specific case and keep invoking the rte_crypto_dequeue_burst() function until
+ * a value less than *nb_pkts* is returned.
+ *
+ * The rte_crypto_dequeue_burst() function does not provide any error
+ * notification to avoid the corresponding overhead.
+ *
+ * @param dev_id The identifier of the device.
+ * @param qp_id The index of the queue pair from which to
+ * retrieve processed packets. The value must be
+ * in the range [0, nb_queue_pair - 1] previously
+ * supplied to rte_cryptodev_configure().
+ * @param pkts The address of an array of pointers to
+ * *rte_mbuf* structures that must be large enough
+ * to store *nb_pkts* pointers in it.
+ * @param nb_pkts The maximum number of packets to dequeue.
+ *
+ * @return
+ * - The number of packets actually dequeued, which is the number
+ * of pointers to *rte_mbuf* structures effectively supplied to the
+ * *pkts* array.
+ */
+static inline uint16_t
+rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id,
+ struct rte_mbuf **pkts, uint16_t nb_pkts)
+{
+ struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
+
+ nb_pkts = (*dev->dequeue_burst)
+ (dev->data->queue_pairs[qp_id], pkts, nb_pkts);
+
+ return nb_pkts;
+}
+
+/**
+ * Enqueue a burst of packets for processing on a crypto device.
+ *
+ * The rte_crypto_enqueue_burst() function is invoked to place packets
+ * on the queue *queue_id* of the device designated by its *dev_id*.
+ *
+ * The *nb_pkts* parameter is the number of packets to process which are
+ * supplied in the *pkts* array of *rte_mbuf* structures.
+ *
+ * The rte_crypto_enqueue_burst() function returns the number of packets it
+ * actually sent. A return value equal to *nb_pkts* means that all packets
+ * have been sent.
+ * *
+ * @param dev_id The identifier of the device.
+ * @param queue_id The index of the transmit queue through
+ * which output packets must be sent. The value
+ * must be in the range [0, nb_queue_pairs - 1]
+ * previously supplied to rte_cryptodev_configure().
+ * @param tx_pkts The address of an array of *nb_pkts* pointers
+ * to *rte_mbuf* structures which contain the
+ * output packets.
+ * @param nb_pkts The number of packets to transmit.
+ *
+ * @return
+ * The number of packets actually enqueued on the crypto device. The return
+ * value can be less than the value of the *nb_pkts* parameter when the
+ * crypto devices queue is full or has been filled up.
+ * The number of packets is 0 if the device hasn't been started.
+ */
+static inline uint16_t
+rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id,
+ struct rte_mbuf **pkts, uint16_t nb_pkts)
+{
+ struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
+
+ return (*dev->enqueue_burst)(dev->data->queue_pairs[qp_id], pkts, nb_pkts);
+}
+
+
+/**
+ * Initialise a session for symmetric cryptographic operations.
+ *
+ * This function is used by the client to initialize immutable
+ * parameters of symmetric cryptographic operation.
+ * To perform the operation the rte_cryptodev_enqueue_burst function is
+ * used. Each mbuf should contain a reference to the session
+ * pointer returned from this function contained within it's crypto_op if a
+ * session-based operation is being provisioned. Memory to contain the session
+ * information is allocated from within mempool managed by the cryptodev.
+ *
+ * The rte_cryptodev_session_free must be called to free allocated
+ * memory when the session is no longer required.
+ *
+ * @param dev_id The device identifier.
+ * @param xform Crypto transform chain.
+
+ *
+ * @return
+ * Pointer to the created session or NULL
+ */
+extern struct rte_cryptodev_session *
+rte_cryptodev_session_create(uint8_t dev_id,
+ struct rte_crypto_xform *xform);
+
+
+/**
+ * Free the memory associated with a previously allocated session.
+ *
+ * @param dev_id The device identifier.
+ * @param session Session pointer previously allocated by
+ * *rte_cryptodev_session_create*.
+ *
+ * @return
+ * NULL on successful freeing of session.
+ * Session pointer on failure to free session.
+ */
+extern struct rte_cryptodev_session *
+rte_cryptodev_session_free(uint8_t dev_id,
+ struct rte_cryptodev_session *session);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_CRYPTODEV_H_ */
diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h b/lib/librte_cryptodev/rte_cryptodev_pmd.h
new file mode 100644
index 0000000..36191a6
--- /dev/null
+++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h
@@ -0,0 +1,529 @@
+/*-
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTODEV_PMD_H_
+#define _RTE_CRYPTODEV_PMD_H_
+
+/** @file
+ * RTE Crypto PMD APIs
+ *
+ * @note
+ * These API are from crypto PMD only and user applications should not call them
+ * directly.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <string.h>
+
+#include <rte_dev.h>
+#include <rte_pci.h>
+#include <rte_malloc.h>
+#include <rte_mbuf.h>
+#include <rte_mempool.h>
+#include <rte_log.h>
+
+#include "rte_crypto.h"
+#include "rte_cryptodev.h"
+
+struct rte_cryptodev_stats;
+struct rte_cryptodev_info;
+struct rte_cryptodev_qp_conf;
+
+enum rte_cryptodev_event_type;
+
+
+struct rte_cryptodev_session {
+ struct {
+ uint8_t dev_id;
+ enum rte_cryptodev_type type;
+ struct rte_mempool *mp;
+ } __rte_aligned(8);
+
+ char _private[];
+};
+
+struct rte_cryptodev_driver;
+struct rte_cryptodev;
+
+/**
+ * Initialisation function of a crypto driver invoked for each matching
+ * crypto PCI device detected during the PCI probing phase.
+ *
+ * @param drv The pointer to the [matching] crypto driver structure
+ * supplied by the PMD when it registered itself.
+ * @param dev The dev pointer is the address of the *rte_cryptodev*
+ * structure associated with the matching device and which
+ * has been [automatically] allocated in the
+ * *rte_crypto_devices* array.
+ *
+ * @return
+ * - 0: Success, the device is properly initialised by the driver.
+ * In particular, the driver MUST have set up the *dev_ops* pointer
+ * of the *dev* structure.
+ * - <0: Error code of the device initialisation failure.
+ */
+typedef int (*cryptodev_init_t)(struct rte_cryptodev_driver *drv,
+ struct rte_cryptodev *dev);
+
+/**
+ * Finalisation function of a driver invoked for each matching
+ * PCI device detected during the PCI closing phase.
+ *
+ * @param drv The pointer to the [matching] driver structure supplied
+ * by the PMD when it registered itself.
+ * @param dev The dev pointer is the address of the *rte_cryptodev*
+ * structure associated with the matching device and which
+ * has been [automatically] allocated in the
+ * *rte_crypto_devices* array.
+ *
+ * * @return
+ * - 0: Success, the device is properly finalised by the driver.
+ * In particular, the driver MUST free the *dev_ops* pointer
+ * of the *dev* structure.
+ * - <0: Error code of the device initialisation failure.
+ */
+typedef int (*cryptodev_uninit_t)(const struct rte_cryptodev_driver *drv,
+ struct rte_cryptodev *dev);
+
+/**
+ * The structure associated with a PMD driver.
+ *
+ * Each driver acts as a PCI driver and is represented by a generic
+ * *crypto_driver* structure that holds:
+ *
+ * - An *rte_pci_driver* structure (which must be the first field).
+ *
+ * - The *cryptodev_init* function invoked for each matching PCI device.
+ *
+ * - The size of the private data to allocate for each matching device.
+ */
+struct rte_cryptodev_driver {
+ struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */
+ unsigned dev_private_size; /**< Size of device private data. */
+
+ cryptodev_init_t cryptodev_init; /**< Device init function. */
+ cryptodev_uninit_t cryptodev_uninit; /**< Device uninit function. */
+};
+
+
+/** Global structure used for maintaining state of allocated crypto devices */
+struct rte_cryptodev_global {
+ struct rte_cryptodev *devs; /**< Device information array */
+ struct rte_cryptodev_data *data[RTE_CRYPTO_MAX_DEVS]; /**< Device private data */
+ uint8_t nb_devs; /**< Number of devices found */
+ uint8_t max_devs; /**< Max number of devices */
+};
+
+/** pointer to global crypto devices data structure. */
+extern struct rte_cryptodev_global *rte_cryptodev_globals;
+
+/**
+ * Get the rte_cryptodev structure device pointer for the device. Assumes a
+ * valid device index.
+ *
+ * @param dev_id Device ID value to select the device structure.
+ *
+ * @return
+ * - The rte_cryptodev structure pointer for the given device ID.
+ */
+static inline struct rte_cryptodev *
+rte_cryptodev_pmd_get_dev(uint8_t dev_id)
+{
+ return &rte_cryptodev_globals->devs[dev_id];
+}
+
+/**
+ * Get the rte_cryptodev structure device pointer for the named device.
+ *
+ * @param name device name to select the device structure.
+ *
+ * @return
+ * - The rte_cryptodev structure pointer for the given device ID.
+ */
+static inline struct rte_cryptodev *
+rte_cryptodev_pmd_get_named_dev(const char *name)
+{
+ unsigned i;
+
+ if (name == NULL)
+ return NULL;
+
+ for (i = 0; i < rte_cryptodev_globals->max_devs; i++) {
+ if (rte_cryptodev_globals->devs[i].attached == RTE_CRYPTODEV_ATTACHED &&
+ strcmp(rte_cryptodev_globals->devs[i].data->name, name) == 0)
+ return &rte_cryptodev_globals->devs[i];
+ }
+
+ return NULL;
+}
+
+/**
+ * Validate if the crypto device index is valid attached crypto device.
+ *
+ * @param dev_id Crypto device index.
+ *
+ * @return
+ * - If the device index is valid (1) or not (0).
+ */
+static inline unsigned
+rte_cryptodev_pmd_is_valid_dev(uint8_t dev_id)
+{
+ struct rte_cryptodev *dev = NULL;
+
+ if (dev_id >= rte_cryptodev_globals->nb_devs)
+ return 0;
+
+ dev = rte_cryptodev_pmd_get_dev(dev_id);
+ if (dev->attached != RTE_CRYPTODEV_ATTACHED)
+ return 0;
+ else
+ return 1;
+}
+
+/**
+ * The pool of rte_cryptodev structures.
+ */
+extern struct rte_cryptodev *rte_cryptodevs;
+
+
+/**
+ * Definitions of all functions exported by a driver through the
+ * the generic structure of type *crypto_dev_ops* supplied in the
+ * *rte_cryptodev* structure associated with a device.
+ */
+
+/**
+ * Function used to configure device.
+ *
+ * @param dev Crypto device pointer
+ *
+ * @return Returns 0 on success
+ */
+typedef int (*cryptodev_configure_t)(struct rte_cryptodev *dev);
+
+/**
+ * Function used to start a configured device.
+ *
+ * @param dev Crypto device pointer
+ *
+ * @return Returns 0 on success
+ */
+typedef int (*cryptodev_start_t)(struct rte_cryptodev *dev);
+
+/**
+ * Function used to stop a configured device.
+ *
+ * @param dev Crypto device pointer
+ */
+typedef void (*cryptodev_stop_t)(struct rte_cryptodev *dev);
+
+/**
+ Function used to close a configured device.
+ *
+ * @param dev Crypto device pointer
+ * @return
+ * - 0 on success.
+ * - EAGAIN if can't close as device is busy
+ */
+typedef int (*cryptodev_close_t)(struct rte_cryptodev *dev);
+
+
+/**
+ * Function used to get statistics of a device.
+ *
+ * @param dev Crypto device pointer
+ * @param stats Pointer to crypto device stats structure to populate
+ */
+typedef void (*cryptodev_stats_get_t)(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats);
+
+
+/**
+ * Function used to reset statistics of a device.
+ *
+ * @param dev Crypto device pointer
+ */
+typedef void (*cryptodev_stats_reset_t)(struct rte_cryptodev *dev);
+
+
+/**
+ * Function used to get specific information of a device.
+ *
+ * @param dev Crypto device pointer
+ */
+typedef void (*cryptodev_info_get_t)(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *dev_info);
+
+/**
+ * Start queue pair of a device.
+ *
+ * @param dev Crypto device pointer
+ * @param qp_id Queue Pair Index
+ *
+ * @return Returns 0 on success.
+ */
+typedef int (*cryptodev_queue_pair_start_t)(struct rte_cryptodev *dev,
+ uint16_t qp_id);
+
+/**
+ * Stop queue pair of a device.
+ *
+ * @param dev Crypto device pointer
+ * @param qp_id Queue Pair Index
+ *
+ * @return Returns 0 on success.
+ */
+typedef int (*cryptodev_queue_pair_stop_t)(struct rte_cryptodev *dev,
+ uint16_t qp_id);
+
+/**
+ * Setup a queue pair for a device.
+ *
+ * @param dev Crypto device pointer
+ * @param qp_id Queue Pair Index
+ * @param qp_conf Queue configuration structure
+ * @param socket_id Socket Index
+ *
+ * @return Returns 0 on success.
+ */
+typedef int (*cryptodev_queue_pair_setup_t)(struct rte_cryptodev *dev,
+ uint16_t qp_id, const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id);
+
+/**
+ * Release memory resources allocated by given queue pair.
+ *
+ * @param dev Crypto device pointer
+ * @param qp_id Queue Pair Index
+ *
+ * @return
+ * - 0 on success.
+ * - EAGAIN if can't close as device is busy
+ */
+typedef int (*cryptodev_queue_pair_release_t)(struct rte_cryptodev *dev,
+ uint16_t qp_id);
+
+/**
+ * Get number of available queue pairs of a device.
+ *
+ * @param dev Crypto device pointer
+ *
+ * @return Returns number of queue pairs on success.
+ */
+typedef uint32_t (*cryptodev_queue_pair_count_t)(struct rte_cryptodev *dev);
+
+/**
+ * Create a session mempool to allocate sessions from
+ *
+ * @param dev Crypto device pointer
+ * @param nb_objs number of sessions objects in mempool
+ * @param obj_cache l-core object cache size, see *rte_ring_create*
+ * @param socket_id Socket Id to allocate mempool on.
+ *
+ * @return
+ * - On success returns a pointer to a rte_mempool
+ * - On failure returns a NULL pointer
+ * */
+typedef int (*cryptodev_create_session_pool_t)(
+ struct rte_cryptodev *dev, unsigned nb_objs,
+ unsigned obj_cache_size, int socket_id);
+
+
+/**
+ * Get the size of a cryptodev session
+ *
+ * @param dev Crypto device pointer
+ *
+ * @return
+ * - On success returns the size of the session structure for device
+ * - On failure returns 0
+ * */
+typedef unsigned (*cryptodev_get_session_private_size_t)(
+ struct rte_cryptodev *dev);
+
+/**
+ * Initialize a Crypto session on a device.
+ *
+ * @param dev Crypto device pointer
+ * @param xform Single or chain of crypto xforms
+ * @param priv_sess Pointer to cryptodev's private session structure
+ *
+ * @return
+ * - Returns private session structure on success.
+ * - Returns NULL on failure.
+ * */
+typedef void (*cryptodev_initialize_session_t)(struct rte_mempool *mempool,
+ void *session_private);
+
+/**
+ * Configure a Crypto session on a device.
+ *
+ * @param dev Crypto device pointer
+ * @param xform Single or chain of crypto xforms
+ * @param priv_sess Pointer to cryptodev's private session structure
+ *
+ * @return
+ * - Returns private session structure on success.
+ * - Returns NULL on failure.
+ * */
+typedef void * (*cryptodev_configure_session_t)(struct rte_cryptodev *dev,
+ struct rte_crypto_xform *xform, void *session_private);
+
+/**
+ * Free Crypto session.
+ * @param session Cryptodev session structure to free
+ * */
+typedef void (*cryptodev_free_session_t)(struct rte_cryptodev *dev,
+ void *session_private);
+
+
+/** Crypto device operations function pointer table */
+struct rte_cryptodev_ops {
+ cryptodev_configure_t dev_configure; /**< Configure device. */
+ cryptodev_start_t dev_start; /**< Start device. */
+ cryptodev_stop_t dev_stop; /**< Stop device. */
+ cryptodev_close_t dev_close; /**< Close device. */
+
+ cryptodev_info_get_t dev_infos_get; /**< Get device info. */
+
+ cryptodev_stats_get_t stats_get; /**< Get generic device statistics. */
+ cryptodev_stats_reset_t stats_reset; /**< Reset generic device statistics. */
+
+ cryptodev_queue_pair_setup_t queue_pair_setup; /**< Set up a device queue pair. */
+ cryptodev_queue_pair_release_t queue_pair_release; /**< Release a queue pair. */
+ cryptodev_queue_pair_start_t queue_pair_start; /**< Start a queue pair. */
+ cryptodev_queue_pair_stop_t queue_pair_stop; /**< Stop a queue pair. */
+ cryptodev_queue_pair_count_t queue_pair_count; /**< Get count of the queue pairs. */
+
+ cryptodev_get_session_private_size_t session_get_size; /**< Return private session. */
+ cryptodev_initialize_session_t session_initialize; /**< Initialization function for private session data */
+ cryptodev_configure_session_t session_configure; /**< Configure a Crypto session. */
+ cryptodev_free_session_t session_clear; /**< Clear a Crypto sessions private data. */
+};
+
+
+/**
+ * Function for internal use by dummy drivers primarily, e.g. ring-based
+ * driver.
+ * Allocates a new cryptodev slot for an crypto device and returns the pointer
+ * to that slot for the driver to use.
+ *
+ * @param name Unique identifier name for each device
+ * @param type Device type of this Crypto device
+ * @param socket_id Socket to allocate resources on.
+ * @return
+ * - Slot in the rte_dev_devices array for a new device;
+ */
+struct rte_cryptodev *
+rte_cryptodev_pmd_allocate(const char *name, enum pmd_type type, int socket_id);
+
+/**
+ * Creates a new virtual crypto device and returns the pointer
+ * to that device.
+ *
+ * @param name PMD type name
+ * @param dev_private_size Size of crypto PMDs private data
+ * @param socket_id Socket to allocate resources on.
+ *
+ * @return
+ * - Cryptodev pointer if device is successfully created.
+ * - NULL if device cannot be created.
+ */
+struct rte_cryptodev *
+rte_cryptodev_pmd_virtual_dev_init(const char *name, size_t dev_private_size,
+ int socket_id);
+
+
+/**
+ * Function for internal use by dummy drivers primarily, e.g. ring-based
+ * driver.
+ * Release the specified cryptodev device.
+ *
+ * @param cryptodev
+ * The *cryptodev* pointer is the address of the *rte_cryptodev* structure.
+ * @return
+ * - 0 on success, negative on error
+ */
+extern int
+rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev);
+
+
+/**
+ * Register a Crypto [Poll Mode] driver.
+ *
+ * Function invoked by the initialization function of a Crypto driver
+ * to simultaneously register itself as Crypto Poll Mode Driver and to either:
+ *
+ * a - register itself as PCI driver if the crypto device is a physical
+ * device, by invoking the rte_eal_pci_register() function to
+ * register the *pci_drv* structure embedded in the *crypto_drv*
+ * structure, after having stored the address of the
+ * rte_cryptodev_init() function in the *devinit* field of the
+ * *pci_drv* structure.
+ *
+ * During the PCI probing phase, the rte_cryptodev_init()
+ * function is invoked for each PCI [device] matching the
+ * embedded PCI identifiers provided by the driver.
+ *
+ * b, complete the initialization sequence if the device is a virtual
+ * device by calling the rte_cryptodev_init() directly passing a
+ * NULL parameter for the rte_pci_device structure.
+ *
+ * @param crypto_drv crypto_driver structure associated with the crypto
+ * driver.
+ * @param type pmd type
+ */
+extern int
+rte_cryptodev_pmd_driver_register(struct rte_cryptodev_driver *crypto_drv,
+ enum pmd_type type);
+
+/**
+ * Executes all the user application registered callbacks for the specific
+ * device.
+ * *
+ * @param dev Pointer to cryptodev struct
+ * @param event Crypto device interrupt event type.
+ *
+ * @return
+ * void
+ */
+void rte_cryptodev_pmd_callback_process(struct rte_cryptodev *dev,
+ enum rte_cryptodev_event_type event);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_CRYPTODEV_PMD_H_ */
diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_cryptodev/rte_cryptodev_version.map
new file mode 100644
index 0000000..979d7eb
--- /dev/null
+++ b/lib/librte_cryptodev/rte_cryptodev_version.map
@@ -0,0 +1,41 @@
+DPDK_2.2 {
+ global:
+
+ rte_cryptodevs;
+ rte_cryptodev_callback_register;
+ rte_cryptodev_callback_unregister;
+ rte_cryptodev_close;
+ rte_cryptodev_count;
+ rte_cryptodev_count_devtype;
+ rte_cryptodev_configure;
+ rte_cryptodev_create_vdev;
+ rte_cryptodev_enqueue_burst;
+ rte_cryptodev_dequeue_burst;
+ rte_cryptodev_get_dev_id;
+ rte_cryptodev_info_get;
+ rte_cryptodev_session_create;
+ rte_cryptodev_session_free;
+ rte_cryptodev_socket_id;
+ rte_cryptodev_start;
+ rte_cryptodev_stats_get;
+ rte_cryptodev_stats_reset;
+ rte_cryptodev_stop;
+ rte_cryptodev_queue_pair_setup;
+ rte_cryptodev_queue_pair_start;
+ rte_cryptodev_queue_pair_stop;
+ rte_cryptodev_queue_pair_count;
+
+ rte_cryptodev_pmd_allocate;
+ rte_cryptodev_pmd_attach;
+ rte_cryptodev_pmd_callback_process;
+ rte_cryptodev_pmd_detach;
+ rte_cryptodev_pmd_driver_register;
+ rte_cryptodev_pmd_get_dev;
+ rte_cryptodev_pmd_get_named_dev;
+ rte_cryptodev_pmd_is_valid_dev;
+ rte_cryptodev_pmd_release_device;
+ rte_cryptodev_pmd_socket_id;
+ rte_cryptodev_pmd_virtual_dev_init;
+
+ local: *;
+};
diff --git a/lib/librte_eal/common/include/rte_common.h b/lib/librte_eal/common/include/rte_common.h
index 3121314..bae4054 100644
--- a/lib/librte_eal/common/include/rte_common.h
+++ b/lib/librte_eal/common/include/rte_common.h
@@ -270,8 +270,23 @@ rte_align64pow2(uint64_t v)
_a > _b ? _a : _b; \
})
+
/*********** Other general functions / macros ********/
+#define FUNC_PTR_OR_ERR_RET(func, retval) do { \
+ if ((func) == NULL) { \
+ RTE_LOG(ERR, PMD, "Function not supported"); \
+ return retval; \
+ } \
+} while (0)
+
+#define FUNC_PTR_OR_RET(func) do { \
+ if ((func) == NULL) { \
+ RTE_LOG(ERR, PMD, "Function not supported"); \
+ return; \
+ } \
+} while (0)
+
#ifdef __SSE2__
#include <emmintrin.h>
/**
diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h
index f36a792..948cc0a 100644
--- a/lib/librte_eal/common/include/rte_eal.h
+++ b/lib/librte_eal/common/include/rte_eal.h
@@ -115,6 +115,20 @@ enum rte_lcore_role_t rte_eal_lcore_role(unsigned lcore_id);
*/
enum rte_proc_type_t rte_eal_process_type(void);
+#define PROC_PRIMARY_OR_RET() do { \
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \
+ RTE_LOG(ERR, PMD, "Cannot run in secondary processes"); \
+ return; \
+ } \
+} while (0)
+
+#define PROC_PRIMARY_OR_ERR_RET(retval) do { \
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \
+ RTE_LOG(ERR, PMD, "Cannot run in secondary processes"); \
+ return retval; \
+ } \
+} while (0)
+
/**
* Request iopl privilege for all RPL.
*
diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h
index ede0dca..2e47e7f 100644
--- a/lib/librte_eal/common/include/rte_log.h
+++ b/lib/librte_eal/common/include/rte_log.h
@@ -78,6 +78,7 @@ extern struct rte_logs rte_logs;
#define RTE_LOGTYPE_TABLE 0x00004000 /**< Log related to table. */
#define RTE_LOGTYPE_PIPELINE 0x00008000 /**< Log related to pipeline. */
#define RTE_LOGTYPE_MBUF 0x00010000 /**< Log related to mbuf. */
+#define RTE_LOGTYPE_CRYPTODEV 0x00020000 /**< Log related to cryptodev. */
/* these log types can be used in an application */
#define RTE_LOGTYPE_USER1 0x01000000 /**< User-defined log type 1. */
diff --git a/lib/librte_eal/common/include/rte_memory.h b/lib/librte_eal/common/include/rte_memory.h
index 1bed415..40e8d43 100644
--- a/lib/librte_eal/common/include/rte_memory.h
+++ b/lib/librte_eal/common/include/rte_memory.h
@@ -76,9 +76,19 @@ enum rte_page_sizes {
/**< Return the first cache-aligned value greater or equal to size. */
/**
+ * Force alignment.
+ */
+#define __rte_aligned(a) __attribute__((__aligned__(a)))
+
+/**
* Force alignment to cache line.
*/
-#define __rte_cache_aligned __attribute__((__aligned__(RTE_CACHE_LINE_SIZE)))
+#define __rte_cache_aligned __rte_aligned(RTE_CACHE_LINE_SIZE)
+
+/**
+ * Force a structure to be packed
+ */
+#define __rte_packed __attribute__((__packed__))
typedef uint64_t phys_addr_t; /**< Physical address definition. */
#define RTE_BAD_PHYS_ADDR ((phys_addr_t)-1)
@@ -104,7 +114,7 @@ struct rte_memseg {
/**< store segment MFNs */
uint64_t mfn[DOM0_NUM_MEMBLOCK];
#endif
-} __attribute__((__packed__));
+} __rte_packed;
/**
* Lock page in physical memory and prevent from swapping.
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index f593f6e..16fde77 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -77,36 +77,6 @@
#define PMD_DEBUG_TRACE(fmt, args...)
#endif
-/* Macros for checking for restricting functions to primary instance only */
-#define PROC_PRIMARY_OR_ERR_RET(retval) do { \
- if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \
- PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \
- return (retval); \
- } \
-} while (0)
-
-#define PROC_PRIMARY_OR_RET() do { \
- if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \
- PMD_DEBUG_TRACE("Cannot run in secondary processes\n"); \
- return; \
- } \
-} while (0)
-
-/* Macros to check for invalid function pointers in dev_ops structure */
-#define FUNC_PTR_OR_ERR_RET(func, retval) do { \
- if ((func) == NULL) { \
- PMD_DEBUG_TRACE("Function not supported\n"); \
- return (retval); \
- } \
-} while (0)
-
-#define FUNC_PTR_OR_RET(func) do { \
- if ((func) == NULL) { \
- PMD_DEBUG_TRACE("Function not supported\n"); \
- return; \
- } \
-} while (0)
-
/* Macros to check for valid port */
#define VALID_PORTID_OR_ERR_RET(port_id, retval) do { \
if (!rte_eth_dev_is_valid_port(port_id)) { \
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 4a93189..710086a 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -98,6 +98,7 @@ extern "C" {
#define PKT_RX_FDIR_ID (1ULL << 13) /**< FD id reported if FDIR match. */
#define PKT_RX_FDIR_FLX (1ULL << 14) /**< Flexible bytes reported if FDIR match. */
#define PKT_RX_QINQ_PKT (1ULL << 15) /**< RX packet with double VLAN stripped. */
+
/* add new RX flags here */
/* add new TX flags here */
@@ -105,7 +106,7 @@ extern "C" {
/**
* Second VLAN insertion (QinQ) flag.
*/
-#define PKT_TX_QINQ_PKT (1ULL << 49) /**< TX packet with double VLAN inserted. */
+#define PKT_TX_QINQ_PKT (1ULL << 49) /**< TX packet with double VLAN inserted. */
/**
* TCP segmentation offload. To enable this offload feature for a
@@ -1622,6 +1623,33 @@ static inline struct rte_mbuf *rte_pktmbuf_lastseg(struct rte_mbuf *m)
#define rte_pktmbuf_mtod(m, t) rte_pktmbuf_mtod_offset(m, t, 0)
/**
+ * A macro that returns the physical address of the data in the mbuf.
+ *
+ * The returned pointer is cast to type t. Before using this
+ * function, the user must ensure that m_headlen(m) is large enough to
+ * read its data.
+ *
+ * @param m
+ * The packet mbuf.
+ * @param o
+ * The offset into the data to calculate address from.
+ */
+#define rte_pktmbuf_mtophys_offset(m, o) ((phys_addr_t)((char *)(m)->buf_physaddr + (m)->data_off) + (o))
+
+/**
+ * A macro that returns the physical address of the data in the mbuf.
+ *
+ * The returned pointer is cast to type t. Before using this
+ * function, the user must ensure that m_headlen(m) is large enough to
+ * read its data.
+ *
+ * @param m
+ * The packet mbuf.
+ * @param o
+ * The offset into the data to calculate address from.
+ */
+#define rte_pktmbuf_mtophys(m) rte_pktmbuf_mtophys_offset(m, 0)
+/**
* A macro that returns the length of the packet.
*
* The value can be read or assigned.
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 9e1909e..80f68bb 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -114,6 +114,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs
_LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf
_LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += -lrte_ip_frag
_LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lethdev
+_LDLIBS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += -lrte_cryptodev
_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += -lrte_mempool
_LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring
_LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal
--
2.4.3
next prev parent reply other threads:[~2015-10-30 12:55 UTC|newest]
Thread overview: 115+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-02 23:01 [PATCH 0/6] Crypto API and device framework Declan Doherty
2015-10-02 23:01 ` [PATCH 1/6] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-10-21 9:24 ` Thomas Monjalon
2015-10-21 11:16 ` Declan Doherty
2015-10-02 23:01 ` [PATCH 2/6] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-10-02 23:01 ` [PATCH 3/6] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-10-02 23:01 ` [PATCH 4/6] docs: add getting started guides for multi-buffer pmd and qat pmd Declan Doherty
2015-10-21 11:34 ` Thomas Monjalon
2015-10-02 23:01 ` [PATCH 5/6] app/test: add cryptodev unit and performance tests Declan Doherty
2015-10-02 23:01 ` [PATCH 6/6] l2fwd-crypto: crypto Declan Doherty
2015-10-21 9:11 ` [PATCH 0/6] Crypto API and device framework Declan Doherty
2015-10-30 12:59 ` [PATCH v2 " Declan Doherty
2015-10-30 12:59 ` Declan Doherty [this message]
2015-10-30 12:59 ` [PATCH v2 2/6] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-10-30 12:59 ` [PATCH v2 3/6] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-10-30 12:59 ` [PATCH v2 4/6] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-10-30 12:59 ` [PATCH v2 5/6] app/test: add cryptodev unit and performance tests Declan Doherty
2015-10-30 12:59 ` [PATCH v2 6/6] l2fwd-crypto: crypto Declan Doherty
2015-10-30 16:08 ` [PATCH v3 0/6] Crypto API and device framework Declan Doherty
2015-10-30 16:08 ` [PATCH v3 1/6] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-10-30 16:08 ` [PATCH v3 2/6] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-10-30 16:34 ` Ananyev, Konstantin
2015-10-30 16:08 ` [PATCH v3 3/6] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-10-30 16:08 ` [PATCH v3 4/6] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-10-30 16:08 ` [PATCH v3 5/6] app/test: add cryptodev unit and performance tests Declan Doherty
2015-10-30 16:08 ` [PATCH v3 6/6] l2fwd-crypto: crypto Declan Doherty
2015-11-03 17:45 ` [PATCH v4 0/6] Crypto API and device framework Declan Doherty
2015-11-03 17:45 ` [PATCH v4 1/6] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-11-03 17:45 ` [PATCH v4 2/6] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-11-03 17:45 ` [PATCH v4 3/6] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-11-03 17:45 ` [PATCH v4 4/6] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-11-03 17:45 ` [PATCH v4 5/6] app/test: add cryptodev unit and performance tests Declan Doherty
2015-11-03 17:45 ` [PATCH v4 6/6] l2fwd-crypto: crypto Declan Doherty
2015-11-03 21:20 ` [PATCH v4 0/6] Crypto API and device framework Sergio Gonzalez Monroy
2015-11-09 20:34 ` [PATCH v5 00/10] " Declan Doherty
2015-11-09 20:34 ` [PATCH v5 01/10] ethdev: rename macros to have RTE_ prefix Declan Doherty
2015-11-10 10:30 ` Bruce Richardson
2015-11-09 20:34 ` [PATCH v5 02/10] ethdev: make error checking macros public Declan Doherty
2015-11-10 10:32 ` Bruce Richardson
2015-11-10 15:50 ` Adrien Mazarguil
2015-11-10 17:00 ` Declan Doherty
2015-11-09 20:34 ` [PATCH v5 03/10] eal: add __rte_packed /__rte_aligned macros Declan Doherty
2015-11-09 20:34 ` [PATCH v5 04/10] mbuf: add new marcos to get the physical address of data Declan Doherty
2015-11-09 20:34 ` [PATCH v5 05/10] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-11-09 20:34 ` [PATCH v5 06/10] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-11-09 20:34 ` [PATCH v5 07/10] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-11-09 20:34 ` [PATCH v5 08/10] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-11-09 20:34 ` [PATCH v5 09/10] app/test: add cryptodev unit and performance tests Declan Doherty
2015-11-09 20:34 ` [PATCH v5 10/10] l2fwd-crypto: crypto Declan Doherty
2015-11-10 17:32 ` [PATCH v6 00/10] Crypto API and device framework Declan Doherty
2015-11-10 17:32 ` [PATCH v6 01/10] ethdev: rename macros to have RTE_ prefix Declan Doherty
2015-11-10 17:32 ` [PATCH v6 02/10] ethdev: make error checking macros public Declan Doherty
2015-11-10 17:38 ` Adrien Mazarguil
2015-11-10 17:32 ` [PATCH v6 03/10] eal: add __rte_packed /__rte_aligned macros Declan Doherty
2015-11-13 15:35 ` Thomas Monjalon
2015-11-13 15:41 ` Declan Doherty
2015-11-10 17:32 ` [PATCH v6 04/10] mbuf: add new marcos to get the physical address of data Declan Doherty
2015-11-10 17:32 ` [PATCH v6 05/10] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-11-13 15:44 ` Thomas Monjalon
2015-11-10 17:32 ` [PATCH v6 06/10] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-11-13 15:59 ` Thomas Monjalon
2015-11-13 16:11 ` Thomas Monjalon
2015-11-10 17:32 ` [PATCH v6 07/10] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-11-13 16:00 ` Thomas Monjalon
2015-11-13 16:25 ` Declan Doherty
2015-11-10 17:32 ` [PATCH v6 08/10] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-11-10 17:32 ` [PATCH v6 09/10] app/test: add cryptodev unit and performance tests Declan Doherty
2015-11-10 17:32 ` [PATCH v6 10/10] l2fwd-crypto: crypto Declan Doherty
2015-11-13 16:03 ` Thomas Monjalon
2015-11-13 18:58 ` [PATCH v7 00/10] Crypto API and device framework Declan Doherty
2015-11-13 18:58 ` [PATCH v7 01/10] ethdev: rename macros to have RTE_ prefix Declan Doherty
2015-11-17 14:44 ` Declan Doherty
2015-11-17 15:39 ` Thomas Monjalon
2015-11-17 16:04 ` [PATCH v7.1 " Declan Doherty
2015-11-13 18:58 ` [PATCH v7 02/10] ethdev: make error checking macros public Declan Doherty
2015-11-13 18:58 ` [PATCH v7 03/10] eal: add __rte_packed /__rte_aligned macros Declan Doherty
2015-11-13 18:58 ` [PATCH v7 04/10] mbuf: add new marcos to get the physical address of data Declan Doherty
2015-11-25 0:25 ` Thomas Monjalon
2015-11-13 18:58 ` [PATCH v7 05/10] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-11-25 0:32 ` Thomas Monjalon
2015-11-13 18:58 ` [PATCH v7 06/10] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-11-20 15:27 ` Olivier MATZ
2015-11-20 17:26 ` Declan Doherty
2015-11-23 9:10 ` Olivier MATZ
2015-11-23 11:52 ` Ananyev, Konstantin
2015-11-23 12:16 ` Declan Doherty
2015-11-23 13:08 ` Olivier MATZ
2015-11-23 14:17 ` Thomas Monjalon
2015-11-23 14:46 ` Thomas Monjalon
2015-11-23 15:47 ` Declan Doherty
2015-11-23 14:33 ` Declan Doherty
2015-11-13 18:58 ` [PATCH v7 07/10] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-11-25 1:00 ` Thomas Monjalon
2015-11-25 9:16 ` Mcnamara, John
2015-11-25 10:34 ` Thomas Monjalon
2015-11-25 10:49 ` Thomas Monjalon
2015-11-25 10:59 ` Declan Doherty
2015-11-25 12:01 ` Mcnamara, John
2015-11-13 18:58 ` [PATCH v7 08/10] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-11-25 10:32 ` Thomas Monjalon
2015-11-13 18:58 ` [PATCH v7 09/10] app/test: add cryptodev unit and performance tests Declan Doherty
2015-11-13 18:58 ` [PATCH v7 10/10] l2fwd-crypto: crypto Declan Doherty
2015-11-25 1:03 ` Thomas Monjalon
2015-11-25 13:25 ` [PATCH v8 00/10] Crypto API and device framework Declan Doherty
2015-11-25 13:25 ` [PATCH v8 01/10] ethdev: rename macros to have RTE_ prefix Declan Doherty
2015-11-25 13:25 ` [PATCH v8 02/10] ethdev: make error checking macros public Declan Doherty
2015-11-25 13:25 ` [PATCH v8 03/10] eal: add __rte_packed /__rte_aligned macros Declan Doherty
2015-11-25 13:25 ` [PATCH v8 04/10] mbuf: add new marcos to get the physical address of data Declan Doherty
2015-11-25 13:25 ` [PATCH v8 05/10] cryptodev: Initial DPDK Crypto APIs and device framework release Declan Doherty
2015-11-25 13:25 ` [PATCH v8 06/10] mbuf_offload: library to support attaching offloads to a mbuf Declan Doherty
2015-11-25 13:25 ` [PATCH v8 07/10] qat_crypto_pmd: Addition of a new QAT DPDK PMD Declan Doherty
2015-11-25 13:25 ` [PATCH v8 08/10] aesni_mb_pmd: Initial implementation of multi buffer based crypto device Declan Doherty
2015-11-25 13:25 ` [PATCH v8 09/10] app/test: add cryptodev unit and performance tests Declan Doherty
2015-11-25 13:25 ` [PATCH v8 10/10] l2fwd-crypto: crypto Declan Doherty
2015-11-25 17:44 ` [PATCH v8 00/10] Crypto API and device framework Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1446209986-29137-2-git-send-email-declan.doherty@intel.com \
--to=declan.doherty@intel.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.