* [PATCH 4/5] vmxnet3: Avoid memory leak in vmxnet3_dev_tx_queue_setup.
From: nickcooper-zhangtonghao @ 2017-01-05 10:43 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
In-Reply-To: <1483612983-3545-1-git-send-email-nic@opencloud.tech>
This patch will check the "nb_desc" parameter for tx queue,
release the txq and re-allocation it soon.
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
drivers/net/vmxnet3/vmxnet3_rxtx.c | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index 9822fa0..077baac 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -828,6 +828,23 @@
return -EINVAL;
}
+ /* Tx vmxnet ring length should be between 512-4096 */
+ if (nb_desc < VMXNET3_DEF_TX_RING_SIZE) {
+ PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Min: %u",
+ VMXNET3_DEF_TX_RING_SIZE);
+ return -EINVAL;
+ } else if (nb_desc > VMXNET3_TX_RING_MAX_SIZE) {
+ PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Max: %u",
+ VMXNET3_TX_RING_MAX_SIZE);
+ return -EINVAL;
+ }
+
+ /* Free memory prior to re-allocation if needed... */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ vmxnet3_dev_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
txq = rte_zmalloc("ethdev_tx_queue", sizeof(struct vmxnet3_tx_queue),
RTE_CACHE_LINE_SIZE);
if (txq == NULL) {
@@ -846,19 +863,8 @@
comp_ring = &txq->comp_ring;
data_ring = &txq->data_ring;
- /* Tx vmxnet ring length should be between 512-4096 */
- if (nb_desc < VMXNET3_DEF_TX_RING_SIZE) {
- PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Min: %u",
- VMXNET3_DEF_TX_RING_SIZE);
- return -EINVAL;
- } else if (nb_desc > VMXNET3_TX_RING_MAX_SIZE) {
- PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Max: %u",
- VMXNET3_TX_RING_MAX_SIZE);
- return -EINVAL;
- } else {
- ring->size = nb_desc;
- ring->size &= ~VMXNET3_RING_SIZE_MASK;
- }
+ ring->size = nb_desc;
+ ring->size &= ~VMXNET3_RING_SIZE_MASK;
comp_ring->size = data_ring->size = ring->size;
/* Tx vmxnet rings structure initialization*/
--
1.8.3.1
^ permalink raw reply related
* [PATCH 5/5] vmxnet3: Avoid segfault caused by vmxnet3_dev_tx_queue_setup.
From: nickcooper-zhangtonghao @ 2017-01-05 10:43 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
In-Reply-To: <1483612983-3545-1-git-send-email-nic@opencloud.tech>
We should allocate Tx ring for max possible mumber of hardware descriptors.
If we config Tx queue with 2048 Tx queue size, and 4096 soon,
there will be segment fault.
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
drivers/net/vmxnet3/vmxnet3_rxtx.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index 077baac..2302c8a 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -874,9 +874,10 @@
comp_ring->next2proc = 0;
comp_ring->gen = VMXNET3_INIT_GEN;
- size = sizeof(struct Vmxnet3_TxDesc) * ring->size;
- size += sizeof(struct Vmxnet3_TxCompDesc) * comp_ring->size;
- size += sizeof(struct Vmxnet3_TxDataDesc) * data_ring->size;
+ /* Allocate Tx ring for max possible mumber of hardware descriptors. */
+ size = sizeof(struct Vmxnet3_TxDesc) * VMXNET3_TX_RING_MAX_SIZE;
+ size += sizeof(struct Vmxnet3_TxCompDesc) * VMXNET3_TX_RING_MAX_SIZE;
+ size += sizeof(struct Vmxnet3_TxDataDesc) * VMXNET3_TX_RING_MAX_SIZE;
mz = ring_dma_zone_reserve(dev, "txdesc", queue_idx, size, socket_id);
if (mz == NULL) {
--
1.8.3.1
^ permalink raw reply related
* [PATCH v1] doc: announce API and ABI change for ethdev
From: Bernard Iremonger @ 2017-01-05 10:44 UTC (permalink / raw)
To: dev, john.mcnamara; +Cc: Bernard Iremonger
In 17.05 nine rte_eth_dev_* functions will be removed from
librte_ether, renamed and moved to the ixgbe PMD.
Signed-off-by: Bernard Iremonger <bernard.iremonger@intel.com>
---
doc/guides/rel_notes/deprecation.rst | 61 ++++++++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 1438c77..f3d79d8 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -79,3 +79,64 @@ Deprecation Notices
PMDs that implement the latter.
Target release for removal of the legacy API will be defined once most
PMDs have switched to rte_flow.
+
+* ethdev: for 17.05 it is planned to deprecate the following nine rte_eth_dev_* functions
+ and move them into the ixgbe PMD:
+
+ ``rte_eth_dev_bypass_init``
+
+ ``rte_eth_dev_bypass_state_set``
+
+ ``rte_eth_dev_bypass_state_show``
+
+ ``rte_eth_dev_bypass_event_store``
+
+ ``rte_eth_dev_bypass_event_show``
+
+ ``rte_eth_dev_wd_timeout_store``
+
+ ``rte_eth_dev_bypass_wd_timeout_show``
+
+ ``rte_eth_dev_bypass_ver_show``
+
+ ``rte_eth_dev_bypass_wd_reset``
+
+ The following fields will be removed from ``struct eth_dev_ops``:
+
+ ``bypass_init_t``
+
+ ``bypass_state_set_t``
+
+ ``bypass_state_show_t``
+
+ ``bypass_event_set_t``
+
+ ``bypass_event_show_t``
+
+ ``bypass_wd_timeout_set_t``
+
+ ``bypass_wd_timeout_show_t``
+
+ ``bypass_ver_show_t``
+
+ ``bypass_wd_reset_t``
+
+ The functions will be renamed to the following, and moved to the ``ixgbe`` PMD:
+
+ ``rte_pmd_ixgbe_bypass_init``
+
+ ``rte_pmd_ixgbe_bypass_state_set``
+
+ ``rte_pmd_ixgbe_bypass_state_show``
+
+ ``rte_pmd_ixgbe_bypass_event_set``
+
+ ``rte_pmd_ixgbe_bypass_event_show``
+
+ ``rte_pmd_ixgbe_bypass_wd_timeout_set``
+
+ ``rte_pmd_ixgbe_bypass_wd_timeout_show``
+
+ ``rte_pmd_ixgbe_bypass_ver_show``
+
+ ``rte_pmd_ixgbe_bypass_wd_reset``
--
2.10.1
^ permalink raw reply related
* Re: [PATCH v5 07/17] net/i40e: add flow validate function
From: Ferruh Yigit @ 2017-01-05 11:16 UTC (permalink / raw)
To: Xing, Beilei, Wu, Jingjing, Zhang, Helin; +Cc: dev@dpdk.org, Zhao1, Wei
In-Reply-To: <94479800C636CB44BD422CB454846E013158E038@SHSMSX101.ccr.corp.intel.com>
On 1/5/2017 6:08 AM, Xing, Beilei wrote:
> Hi Ferruh,
>
>> -----Original Message-----
>> From: Yigit, Ferruh
>> Sent: Thursday, January 5, 2017 2:57 AM
>> To: Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing
>> <jingjing.wu@intel.com>; Zhang, Helin <helin.zhang@intel.com>
>> Cc: dev@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH v5 07/17] net/i40e: add flow validate
>> function
>>
>> On 1/4/2017 3:22 AM, Beilei Xing wrote:
>>> This patch adds i40e_flow_validation function to check if a flow is
>>> valid according to the flow pattern.
>>> i40e_parse_ethertype_filter is added first, it also gets the ethertype
>>> info.
>>> i40e_flow.c is added to handle all generic filter events.
>>>
>>> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
>>> ---
>>
>> <...>
>>
>>> diff --git a/drivers/net/i40e/i40e_ethdev.c
>>> b/drivers/net/i40e/i40e_ethdev.c index 153322a..edfd52b 100644
>>> --- a/drivers/net/i40e/i40e_ethdev.c
>>> +++ b/drivers/net/i40e/i40e_ethdev.c
>>> @@ -8426,6 +8426,8 @@ i40e_ethertype_filter_handle(struct rte_eth_dev
>> *dev,
>>> return ret;
>>> }
>>>
>>> +const struct rte_flow_ops i40e_flow_ops;
>>
>> Is this intentional (instead of using extern) ?
>> Because i40e_flow.c has a global variable definition with same name, it looks
>> like this is not causing a build error, but I think confusing.
>>
>
> Actually it's the global variable definition in i40e_flow.c. I thought gcc would add extern automatically during compiling, as I checked the address of the variable is the same in different files.
> To avoid confusion, I will add extern in next version.
>
>> <...>
>>
>>> +static int i40e_parse_ethertype_act(struct rte_eth_dev *dev,
>>> + const struct rte_flow_action *actions,
>>> + struct rte_flow_error *error,
>>> + struct rte_eth_ethertype_filter *filter);
>>
>> In API naming, I would prefer full "action" instead of shorten "act", but it is
>> your call.
>
> I will change the API name in next version. Thanks.
>
>>
>> <...>
>>
>>> +
>>> +union i40e_filter_t cons_filter;
>>
>> Why this cons_filter is required. I can see this is saving some state related
>> rule during validate function.
>> If the plan is to use this during rule creation, is user has to call validate before
>> each create?
>
> You are right, cons_filter will get filter info during validation, and it's for flow_create function.
> User needn't to call the flow_validate function, as validate function will be called in i40e_flow_create.
Ok then.
>
>>
>> <...>
>>
>>> +
>>> +static int
>>> +i40e_parse_ethertype_filter(struct rte_eth_dev *dev,
>>> + const struct rte_flow_attr *attr,
>>> + const struct rte_flow_item pattern[],
>>> + const struct rte_flow_action actions[],
>>> + struct rte_flow_error *error,
>>> + union i40e_filter_t *filter)
>>> +{
>>> + struct rte_eth_ethertype_filter *ethertype_filter =
>>> + &filter->ethertype_filter;
>>> + int ret;
>>> +
>>> + ret = i40e_parse_ethertype_pattern(dev, pattern, error,
>>> + ethertype_filter);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + ret = i40e_parse_ethertype_act(dev, actions, error,
>>> + ethertype_filter);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + ret = i40e_parse_attr(attr, error);
>>
>> It is your call, but I would suggest using a specific namespace for all rte_flow
>> related functions, something like "i40e_flow_".
>> In this context it is clear what this function is, but in whole driver code, the
>> function name is too generic to understand what it does.
>
> Make sense. I'll update the function names.
>
>>
>>> + if (ret)
>>> + return ret;
>>> +
>>> + return ret;
>>> +}
>>> +
>>
>> <...>
>>
>>> +
>>> +static int
>>> +i40e_parse_ethertype_pattern(__rte_unused struct rte_eth_dev *dev,
>>> + const struct rte_flow_item *pattern,
>>> + struct rte_flow_error *error,
>>> + struct rte_eth_ethertype_filter *filter)
>>
>> I think it is good idea to comment what pattern is recognized in to function
>> comment, instead of reading code every time to figure out.
>
> In fact, the array of i40e_supported_patterns has listed all supported patterns for each filter type.
> i40e_supported_patterns is also defined in this patch.
i40e_supported_patterns only shows item->type values, I think it is good
to documents expected/valid mask (.dst, .src, .type) and last values for
this type.
>
>>
>>> +{
>>> + const struct rte_flow_item *item = pattern;
>>> + const struct rte_flow_item_eth *eth_spec;
>>> + const struct rte_flow_item_eth *eth_mask;
>>> + enum rte_flow_item_type item_type;
>>> +
>>> + for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
>>> + if (item->last) {
>>> + rte_flow_error_set(error, EINVAL,
>>> + RTE_FLOW_ERROR_TYPE_ITEM,
>>> + item,
>>> + "Not support range");
>>> + return -rte_errno;
>>> + }
>>> + item_type = item->type;
>>> + switch (item_type) {
>>> + case RTE_FLOW_ITEM_TYPE_ETH:
>>> + eth_spec = (const struct rte_flow_item_eth *)item-
>>> spec;
>>> + eth_mask = (const struct rte_flow_item_eth *)item-
>>> mask;
>>> + /* Get the MAC info. */
>>> + if (!eth_spec || !eth_mask) {
>>
>> Why an eth_mask is required?
> Yes, since eth_type mask in eth_mask should be UINT16_MAX.
>
>> Can't driver support drop/queue packets from specific src to specific dst with
>> specific eth_type?
> No, we support specific dst with specific eth_type, or only specific eth_type. Perfect match.
Thanks for clarification.
>
>>
>>> + rte_flow_error_set(error, EINVAL,
>>> +
>> RTE_FLOW_ERROR_TYPE_ITEM,
>>> + item,
>>> + "NULL ETH spec/mask");
>>> + return -rte_errno;
>>> + }
>>> +
>>> + /* Mask bits of source MAC address must be full of 0.
>>> + * Mask bits of destination MAC address must be full
>>> + * of 1 or full of 0.
>>> + */
>>> + if (!is_zero_ether_addr(ð_mask->src) ||
>>> + (!is_zero_ether_addr(ð_mask->dst) &&
>>> + !is_broadcast_ether_addr(ð_mask->dst))) {
>>> + rte_flow_error_set(error, EINVAL,
>>> +
>> RTE_FLOW_ERROR_TYPE_ITEM,
>>> + item,
>>> + "Invalid MAC_addr mask");
>>> + return -rte_errno;
>>> + }
>>> +
>>> + if ((eth_mask->type & UINT16_MAX) !=
>> UINT16_MAX) {
>>> + rte_flow_error_set(error, EINVAL,
>>> +
>> RTE_FLOW_ERROR_TYPE_ITEM,
>>> + item,
>>> + "Invalid ethertype mask");
>>
>> Why returning error here?
>> Can't we say drop packets to specific MAC address, independent from the
>> ether_type?
>
> No. as I said above, we support specific dst with specific eth_type, or only specific eth_type for ethertype_filter.
>
>>
>>> + return -rte_errno;
>>> + }
>>> +
>>> + /* If mask bits of destination MAC address
>>> + * are full of 1, set RTE_ETHTYPE_FLAGS_MAC.
>>> + */
>>> + if (is_broadcast_ether_addr(ð_mask->dst)) {
>>> + filter->mac_addr = eth_spec->dst;
>>> + filter->flags |= RTE_ETHTYPE_FLAGS_MAC;
>>> + } else {
>>> + filter->flags &= ~RTE_ETHTYPE_FLAGS_MAC;
>>> + }
>>> + filter->ether_type = rte_be_to_cpu_16(eth_spec-
>>> type);
>>> +
>>> + if (filter->ether_type == ETHER_TYPE_IPv4 ||
>>> + filter->ether_type == ETHER_TYPE_IPv6) {
>>> + rte_flow_error_set(error, EINVAL,
>>> +
>> RTE_FLOW_ERROR_TYPE_ITEM,
>>> + item,
>>> + "Unsupported ether_type
>> in"
>>> + " control packet filter.");
>>
>> Can't we create a drop rule based on dst MAC address if eth_type is ip ?
>
> No, we don't support drop MAC_addr + eth_type_IP for ethertype filter.
>
>>
>>> + return -rte_errno;
>>> + }
>>> + if (filter->ether_type == ETHER_TYPE_VLAN)
>>> + PMD_DRV_LOG(WARNING, "filter vlan
>> ether_type in"
>>> + " first tag is not supported.");
>>
>> Who is the target of this message?
>> To the caller, this API is responding as this is supported.
>> The end user, the user of the application, can see this message, how this
>> message will help to end user?
>
> Actually I add this warning according to the original processing in i40e_dev_eythertype_filter_set.
> After checing datasheet, "The ethertype programmed by this command should not be one of the L2 tags ethertype (VLAN, E-tag, S-tag, etc.) and should not be IP or IPv6" is descripted.
> But if QinQ is disabled, and inner vlan is ETHER_TYPE_VLAN, the filter works. So the message is "vlan ether_type in outer tag is not supported".
> I want to simplify it in next version, don't support the situation above, and return error if (filter->ether_type == ETHER_TYPE_VLAN), because HW only recognizes ETH when QinQ is diabled. What do you think?
I think it is better.
And this can be fine tuned in the future to check QinQ and return
accordingly.
>
>>
>>> +
>>> + break;
>>> + default:
>>> + break;
>>> + }
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int
>>> +i40e_parse_ethertype_act(struct rte_eth_dev *dev,
>>> + const struct rte_flow_action *actions,
>>> + struct rte_flow_error *error,
>>> + struct rte_eth_ethertype_filter *filter)
>>
>> I think it would be good to comment this functions to say only DROP and
>> QUEUE actions are supported.
>
> Yes, will update in next version.
>
>>
>> <...>
>>
>>> +
>>> +static int
>>> +i40e_flow_validate(struct rte_eth_dev *dev,
>>> + const struct rte_flow_attr *attr,
>>> + const struct rte_flow_item pattern[],
>>> + const struct rte_flow_action actions[],
>>> + struct rte_flow_error *error)
>>> +{
>>> + struct rte_flow_item *items; /* internal pattern w/o VOID items */
>>> + parse_filter_t parse_filter;
>>> + uint32_t item_num = 0; /* non-void item number of pattern*/
>>> + uint32_t i = 0;
>>> + int ret;
>>> +
>>> + if (!pattern) {
>>> + rte_flow_error_set(error, EINVAL,
>> RTE_FLOW_ERROR_TYPE_ITEM_NUM,
>>> + NULL, "NULL pattern.");
>>> + return -rte_errno;
>>> + }
>>> +
>>> + if (!actions) {
>>> + rte_flow_error_set(error, EINVAL,
>>> + RTE_FLOW_ERROR_TYPE_ACTION_NUM,
>>> + NULL, "NULL action.");
>>> + return -rte_errno;
>>> + }
>>
>> It may be good to validate attr too, if it is NULL or not. It is accessed without
>> check in later stages of the call stack.
>
> Yes. Thanks for reminder.
>
> Best Regards,
> Beilei
>
>>
>> <...>
>>
>
^ permalink raw reply
* Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
From: Ananyev, Konstantin @ 2017-01-05 11:32 UTC (permalink / raw)
To: Adrien Mazarguil, Bie, Tiwei
Cc: dev@dpdk.org, Lu, Wenzhuo, Mcnamara, John, olivier.matz@6wind.com,
thomas.monjalon@6wind.com, Zhang, Helin, Dai, Wei, Wang, Xiao W
In-Reply-To: <20170105083322.GK12822@6wind.com>
Hi Adrien,
>
> On Thu, Jan 05, 2017 at 07:56:08AM +0800, Tiwei Bie wrote:
> > On Thu, Jan 05, 2017 at 01:44:18AM +0800, Ananyev, Konstantin wrote:
> > [...]
> > > > >
> > > > > I understand that.
> > > > > My question was: suppose user would like to create a bonded device over 2 NICs.
> > > > > One of them is ixgbe, while other would be some other type.
> > > > > In future get_dev_info() for each of them might return DEV_RX_OFFLOAD_RESERVED_0 bit as set.
> > > > > But it would mean completely different thing.
> > > > > How bonded device would know that to deal properly?
> > > > >
> > > > > Another example - user has 2 NICs of different type and would like to send the same packet on both of them simultaneously.
> > > > > As PKT_TX_RESERVED might mean different things for these devices, and user would like to use let say
> > > > > PKT_TX_IXGBE_MACSEC on one of them, he would need to do a copy of them, instead just increment a refcnt.
> > > > >
> > > > > Similar issues might arise at RX handling: user got a packet with PKT_RX_RESERVED_0 set.
> > > > > What does it really mean if there are different NIC types in the system?
> > > > > The only way to answer that question, as I can see, is to keep track from what NIC that packet was received.
> > > > > Which I suppose, is not always convenient.
> > > > >
> > > >
> > > > The main purpose is to put the PMD-specific APIs in a separate
> > > > namespace instead of mixing the PMD-specific APIs and global APIs
> > > > up, and also save the bits in mbuf.ol_flags.
> > > >
> > > > There are other ways to achieve this goal, such as introducing
> > > > the PMD specific ol_flags in mbuf second cache line as you said.
> > > > I just thought defining some reserved bits seems to be the most
> > > > simple way which won't introduce many changes.
> > > >
> > > > What's your suggestions? Should I just revert the changes and
> > > > define the generic flags directly?
> > >
> > > Yes, that would be my preference.
> > > As I said above - spending extra bit in ol_flags doesn't look like a big problem to me.
> > > In return there would be no need to consider how to handle all that confusing scenarios in future.
> >
> > Okay. I'll update my patches. Thanks a lot for your comments.
>
> Well, I do not agree with Konstantin (no one saw this coming eh?)
:)
>and do not think you need to update your series again.
>
> PMD-specific symbols have nothing to do in the global namespace in my
> opinion, they are not versioned and may evolve without notice. Neither
> applications nor the bonding PMD can rely on them. That's the trade-off.
Not sure I do understand your reasoning.
For me MACSEC offload is just one of many HW offloads that we support
and should be treated in exactly the same way.
Applications should be able to use it in a transparent and reliable way,
not only under some limited conditions.
Otherwise what is the point to introduce it at all?
Yes, right now it is supported only by ixgbe PMD, but why that should be the
reason to treat is as second-class citizen?
Let say PKT_TX_TUNNEL_* offloads also are supported only by one PMD right now.
>
> Therefore until APIs are made global, the safe compromise is to define
> neutral, reserved symbols that any PMD can use to implement their own
> temporary APIs for testing purposes. These can be renamed later without
> changing their value as long as a single PMD uses them.
Ok, so what we'll gain by introducing PKT_TX_RESERVED instead of PKT_TX_MACSEC?
As I said in my previous mail the redefinition for the same ol_flag bit (and dev capabilities)
by different PMD might create a lot of confusion in future.
Does the potential saving of 1 bit really worth it?
Konstantin
^ permalink raw reply
* Re: [PATCH v5 07/17] net/i40e: add flow validate function
From: Xing, Beilei @ 2017-01-05 11:52 UTC (permalink / raw)
To: Yigit, Ferruh, Wu, Jingjing, Zhang, Helin; +Cc: dev@dpdk.org, Zhao1, Wei
In-Reply-To: <8d74dd50-2eba-37b6-996d-3d97dd043f2d@intel.com>
> -----Original Message-----
> From: Yigit, Ferruh
> Sent: Thursday, January 5, 2017 7:16 PM
> To: Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing
> <jingjing.wu@intel.com>; Zhang, Helin <helin.zhang@intel.com>
> Cc: dev@dpdk.org; Zhao1, Wei <wei.zhao1@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v5 07/17] net/i40e: add flow validate
> function
>
> On 1/5/2017 6:08 AM, Xing, Beilei wrote:
> > Hi Ferruh,
> >
> >> -----Original Message-----
> >> From: Yigit, Ferruh
> >> Sent: Thursday, January 5, 2017 2:57 AM
> >> To: Xing, Beilei <beilei.xing@intel.com>; Wu, Jingjing
> >> <jingjing.wu@intel.com>; Zhang, Helin <helin.zhang@intel.com>
> >> Cc: dev@dpdk.org
> >> Subject: Re: [dpdk-dev] [PATCH v5 07/17] net/i40e: add flow validate
> >> function
> >>
> >> On 1/4/2017 3:22 AM, Beilei Xing wrote:
> >>> This patch adds i40e_flow_validation function to check if a flow is
> >>> valid according to the flow pattern.
> >>> i40e_parse_ethertype_filter is added first, it also gets the
> >>> ethertype info.
> >>> i40e_flow.c is added to handle all generic filter events.
> >>>
> >>> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> >>> ---
> >>
> >> <...>
> >>
> >>> diff --git a/drivers/net/i40e/i40e_ethdev.c
> >>> b/drivers/net/i40e/i40e_ethdev.c index 153322a..edfd52b 100644
> >>> --- a/drivers/net/i40e/i40e_ethdev.c
> >>> +++ b/drivers/net/i40e/i40e_ethdev.c
> >>> @@ -8426,6 +8426,8 @@ i40e_ethertype_filter_handle(struct
> >>> rte_eth_dev
> >> *dev,
> >>> return ret;
> >>> }
> >>>
> >>> +const struct rte_flow_ops i40e_flow_ops;
> >>
> >> Is this intentional (instead of using extern) ?
> >> Because i40e_flow.c has a global variable definition with same name,
> >> it looks like this is not causing a build error, but I think confusing.
> >>
> >
> > Actually it's the global variable definition in i40e_flow.c. I thought gcc
> would add extern automatically during compiling, as I checked the address of
> the variable is the same in different files.
> > To avoid confusion, I will add extern in next version.
> >
> >> <...>
> >>
> >>> +static int i40e_parse_ethertype_act(struct rte_eth_dev *dev,
> >>> + const struct rte_flow_action *actions,
> >>> + struct rte_flow_error *error,
> >>> + struct rte_eth_ethertype_filter *filter);
> >>
> >> In API naming, I would prefer full "action" instead of shorten "act",
> >> but it is your call.
> >
> > I will change the API name in next version. Thanks.
> >
> >>
> >> <...>
> >>
> >>> +
> >>> +union i40e_filter_t cons_filter;
> >>
> >> Why this cons_filter is required. I can see this is saving some state
> >> related rule during validate function.
> >> If the plan is to use this during rule creation, is user has to call
> >> validate before each create?
> >
> > You are right, cons_filter will get filter info during validation, and it's for
> flow_create function.
> > User needn't to call the flow_validate function, as validate function will be
> called in i40e_flow_create.
>
> Ok then.
>
> >
> >>
> >> <...>
> >>
> >>> +
> >>> +static int
> >>> +i40e_parse_ethertype_filter(struct rte_eth_dev *dev,
> >>> + const struct rte_flow_attr *attr,
> >>> + const struct rte_flow_item pattern[],
> >>> + const struct rte_flow_action actions[],
> >>> + struct rte_flow_error *error,
> >>> + union i40e_filter_t *filter) {
> >>> + struct rte_eth_ethertype_filter *ethertype_filter =
> >>> + &filter->ethertype_filter;
> >>> + int ret;
> >>> +
> >>> + ret = i40e_parse_ethertype_pattern(dev, pattern, error,
> >>> + ethertype_filter);
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + ret = i40e_parse_ethertype_act(dev, actions, error,
> >>> + ethertype_filter);
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + ret = i40e_parse_attr(attr, error);
> >>
> >> It is your call, but I would suggest using a specific namespace for
> >> all rte_flow related functions, something like "i40e_flow_".
> >> In this context it is clear what this function is, but in whole
> >> driver code, the function name is too generic to understand what it does.
> >
> > Make sense. I'll update the function names.
> >
> >>
> >>> + if (ret)
> >>> + return ret;
> >>> +
> >>> + return ret;
> >>> +}
> >>> +
> >>
> >> <...>
> >>
> >>> +
> >>> +static int
> >>> +i40e_parse_ethertype_pattern(__rte_unused struct rte_eth_dev
> *dev,
> >>> + const struct rte_flow_item *pattern,
> >>> + struct rte_flow_error *error,
> >>> + struct rte_eth_ethertype_filter *filter)
> >>
> >> I think it is good idea to comment what pattern is recognized in to
> >> function comment, instead of reading code every time to figure out.
> >
> > In fact, the array of i40e_supported_patterns has listed all supported
> patterns for each filter type.
> > i40e_supported_patterns is also defined in this patch.
>
> i40e_supported_patterns only shows item->type values, I think it is good to
> documents expected/valid mask (.dst, .src, .type) and last values for this
> type.
OK, I see, will add the comments in the function.
>
> >
> >>
> >>> +{
> >>> + const struct rte_flow_item *item = pattern;
> >>> + const struct rte_flow_item_eth *eth_spec;
> >>> + const struct rte_flow_item_eth *eth_mask;
> >>> + enum rte_flow_item_type item_type;
> >>> +
> >>> + for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
> >>> + if (item->last) {
> >>> + rte_flow_error_set(error, EINVAL,
> >>> + RTE_FLOW_ERROR_TYPE_ITEM,
> >>> + item,
> >>> + "Not support range");
> >>> + return -rte_errno;
> >>> + }
> >>> + item_type = item->type;
> >>> + switch (item_type) {
> >>> + case RTE_FLOW_ITEM_TYPE_ETH:
> >>> + eth_spec = (const struct rte_flow_item_eth *)item-
> >>> spec;
> >>> + eth_mask = (const struct rte_flow_item_eth *)item-
> >>> mask;
> >>> + /* Get the MAC info. */
> >>> + if (!eth_spec || !eth_mask) {
> >>
> >> Why an eth_mask is required?
> > Yes, since eth_type mask in eth_mask should be UINT16_MAX.
> >
> >> Can't driver support drop/queue packets from specific src to specific
> >> dst with specific eth_type?
> > No, we support specific dst with specific eth_type, or only specific
> eth_type. Perfect match.
>
> Thanks for clarification.
>
> >
> >>
> >>> + rte_flow_error_set(error, EINVAL,
> >>> +
> >> RTE_FLOW_ERROR_TYPE_ITEM,
> >>> + item,
> >>> + "NULL ETH spec/mask");
> >>> + return -rte_errno;
> >>> + }
> >>> +
> >>> + /* Mask bits of source MAC address must be full of 0.
> >>> + * Mask bits of destination MAC address must be full
> >>> + * of 1 or full of 0.
> >>> + */
> >>> + if (!is_zero_ether_addr(ð_mask->src) ||
> >>> + (!is_zero_ether_addr(ð_mask->dst) &&
> >>> + !is_broadcast_ether_addr(ð_mask->dst))) {
> >>> + rte_flow_error_set(error, EINVAL,
> >>> +
> >> RTE_FLOW_ERROR_TYPE_ITEM,
> >>> + item,
> >>> + "Invalid MAC_addr mask");
> >>> + return -rte_errno;
> >>> + }
> >>> +
> >>> + if ((eth_mask->type & UINT16_MAX) !=
> >> UINT16_MAX) {
> >>> + rte_flow_error_set(error, EINVAL,
> >>> +
> >> RTE_FLOW_ERROR_TYPE_ITEM,
> >>> + item,
> >>> + "Invalid ethertype mask");
> >>
> >> Why returning error here?
> >> Can't we say drop packets to specific MAC address, independent from
> >> the ether_type?
> >
> > No. as I said above, we support specific dst with specific eth_type, or only
> specific eth_type for ethertype_filter.
> >
> >>
> >>> + return -rte_errno;
> >>> + }
> >>> +
> >>> + /* If mask bits of destination MAC address
> >>> + * are full of 1, set RTE_ETHTYPE_FLAGS_MAC.
> >>> + */
> >>> + if (is_broadcast_ether_addr(ð_mask->dst)) {
> >>> + filter->mac_addr = eth_spec->dst;
> >>> + filter->flags |= RTE_ETHTYPE_FLAGS_MAC;
> >>> + } else {
> >>> + filter->flags &= ~RTE_ETHTYPE_FLAGS_MAC;
> >>> + }
> >>> + filter->ether_type = rte_be_to_cpu_16(eth_spec-
> >>> type);
> >>> +
> >>> + if (filter->ether_type == ETHER_TYPE_IPv4 ||
> >>> + filter->ether_type == ETHER_TYPE_IPv6) {
> >>> + rte_flow_error_set(error, EINVAL,
> >>> +
> >> RTE_FLOW_ERROR_TYPE_ITEM,
> >>> + item,
> >>> + "Unsupported ether_type
> >> in"
> >>> + " control packet filter.");
> >>
> >> Can't we create a drop rule based on dst MAC address if eth_type is ip ?
> >
> > No, we don't support drop MAC_addr + eth_type_IP for ethertype filter.
> >
> >>
> >>> + return -rte_errno;
> >>> + }
> >>> + if (filter->ether_type == ETHER_TYPE_VLAN)
> >>> + PMD_DRV_LOG(WARNING, "filter vlan
> >> ether_type in"
> >>> + " first tag is not supported.");
> >>
> >> Who is the target of this message?
> >> To the caller, this API is responding as this is supported.
> >> The end user, the user of the application, can see this message, how
> >> this message will help to end user?
> >
> > Actually I add this warning according to the original processing in
> i40e_dev_eythertype_filter_set.
> > After checing datasheet, "The ethertype programmed by this command
> should not be one of the L2 tags ethertype (VLAN, E-tag, S-tag, etc.) and
> should not be IP or IPv6" is descripted.
> > But if QinQ is disabled, and inner vlan is ETHER_TYPE_VLAN, the filter works.
> So the message is "vlan ether_type in outer tag is not supported".
> > I want to simplify it in next version, don't support the situation above, and
> return error if (filter->ether_type == ETHER_TYPE_VLAN), because HW only
> recognizes ETH when QinQ is diabled. What do you think?
>
> I think it is better.
> And this can be fine tuned in the future to check QinQ and return accordingly.
I have tuned QinQ, and it will not work when ether_type is equal to the outer vlan. So I will update.
>
> >
> >>
> >>> +
> >>> + break;
> >>> + default:
> >>> + break;
> >>> + }
> >>> + }
> >>> +
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static int
> >>> +i40e_parse_ethertype_act(struct rte_eth_dev *dev,
> >>> + const struct rte_flow_action *actions,
> >>> + struct rte_flow_error *error,
> >>> + struct rte_eth_ethertype_filter *filter)
> >>
> >> I think it would be good to comment this functions to say only DROP
> >> and QUEUE actions are supported.
> >
> > Yes, will update in next version.
> >
> >>
> >> <...>
> >>
> >>> +
> >>> +static int
> >>> +i40e_flow_validate(struct rte_eth_dev *dev,
> >>> + const struct rte_flow_attr *attr,
> >>> + const struct rte_flow_item pattern[],
> >>> + const struct rte_flow_action actions[],
> >>> + struct rte_flow_error *error)
> >>> +{
> >>> + struct rte_flow_item *items; /* internal pattern w/o VOID items */
> >>> + parse_filter_t parse_filter;
> >>> + uint32_t item_num = 0; /* non-void item number of pattern*/
> >>> + uint32_t i = 0;
> >>> + int ret;
> >>> +
> >>> + if (!pattern) {
> >>> + rte_flow_error_set(error, EINVAL,
> >> RTE_FLOW_ERROR_TYPE_ITEM_NUM,
> >>> + NULL, "NULL pattern.");
> >>> + return -rte_errno;
> >>> + }
> >>> +
> >>> + if (!actions) {
> >>> + rte_flow_error_set(error, EINVAL,
> >>> + RTE_FLOW_ERROR_TYPE_ACTION_NUM,
> >>> + NULL, "NULL action.");
> >>> + return -rte_errno;
> >>> + }
> >>
> >> It may be good to validate attr too, if it is NULL or not. It is
> >> accessed without check in later stages of the call stack.
> >
> > Yes. Thanks for reminder.
> >
> > Best Regards,
> > Beilei
> >
> >>
> >> <...>
> >>
> >
^ permalink raw reply
* [PATCH v2] net/mlx5: add support for ConnectX-5 NICs
From: Yongseok Koh @ 2017-01-05 11:52 UTC (permalink / raw)
To: ferruh.yigit; +Cc: dev, adrien.mazarguil, Yongseok Koh
In-Reply-To: <20170105023219.10146-1-yskoh@mellanox.com>
Add PCI device ID for ConnectX-5 and enable multi-packet send for PF and VF
along with changing documentation and release note.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
---
ConnectX-5 is a newly announced NIC of Mellanox. This patch includes basic
enablement of ConnectX-5 as well as documentation.
config/common_base | 2 +-
doc/guides/nics/mlx5.rst | 37 +++++++++++++++++-------------
doc/guides/rel_notes/release_17_02.rst | 5 ++++
drivers/net/mlx5/mlx5.c | 42 ++++++++++++++++++++++++++++++----
drivers/net/mlx5/mlx5.h | 4 ++++
drivers/net/mlx5/mlx5_ethdev.c | 7 ++----
drivers/net/mlx5/mlx5_txq.c | 2 +-
7 files changed, 71 insertions(+), 28 deletions(-)
diff --git a/config/common_base b/config/common_base
index faee944c8..bb7beacf6 100644
--- a/config/common_base
+++ b/config/common_base
@@ -204,7 +204,7 @@ CONFIG_RTE_LIBRTE_MLX4_TX_MP_CACHE=8
CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS=1
#
-# Compile burst-oriented Mellanox ConnectX-4 (MLX5) PMD
+# Compile burst-oriented Mellanox ConnectX-4 & ConnectX-5 (MLX5) PMD
#
CONFIG_RTE_LIBRTE_MLX5_PMD=n
CONFIG_RTE_LIBRTE_MLX5_DEBUG=n
diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 98d134190..514232bc8 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -30,10 +30,10 @@
MLX5 poll mode driver
=====================
-The MLX5 poll mode driver library (**librte_pmd_mlx5**) provides support for
-**Mellanox ConnectX-4** and **Mellanox ConnectX-4 Lx** families of
-10/25/40/50/100 Gb/s adapters as well as their virtual functions (VF) in
-SR-IOV context.
+The MLX5 poll mode driver library (**librte_pmd_mlx5**) provides support
+for **Mellanox ConnectX-4**, **Mellanox ConnectX-4 Lx** and **Mellanox
+ConnectX-5** families of 10/25/40/50/100 Gb/s adapters as well as their
+virtual functions (VF) in SR-IOV context.
Information and documentation about these adapters can be found on the
`Mellanox website <http://www.mellanox.com>`__. Help is also provided by the
@@ -185,8 +185,8 @@ Run-time configuration
save PCI bandwidth and improve performance at the cost of a slightly
higher CPU usage.
- It is currently only supported on the ConnectX-4 Lx family of adapters.
- Enabled by default.
+ It is currently only supported on the ConnectX-4 Lx and ConnectX-5
+ families of adapters. Enabled by default.
Prerequisites
-------------
@@ -207,8 +207,8 @@ DPDK and must be installed separately:
- **libmlx5**
- Low-level user space driver library for Mellanox ConnectX-4 devices,
- it is automatically loaded by libibverbs.
+ Low-level user space driver library for Mellanox ConnectX-4/ConnectX-5
+ devices, it is automatically loaded by libibverbs.
This library basically implements send/receive calls to the hardware
queues.
@@ -222,14 +222,15 @@ DPDK and must be installed separately:
Unlike most other PMDs, these modules must remain loaded and bound to
their devices:
- - mlx5_core: hardware driver managing Mellanox ConnectX-4 devices and
- related Ethernet kernel network devices.
+ - mlx5_core: hardware driver managing Mellanox ConnectX-4/ConnectX-5
+ devices and related Ethernet kernel network devices.
- mlx5_ib: InifiniBand device driver.
- ib_uverbs: user space driver for Verbs (entry point for libibverbs).
- **Firmware update**
- Mellanox OFED releases include firmware updates for ConnectX-4 adapters.
+ Mellanox OFED releases include firmware updates for ConnectX-4/ConnectX-5
+ adapters.
Because each release provides new features, these updates must be applied to
match the kernel modules and libraries they come with.
@@ -241,12 +242,16 @@ DPDK and must be installed separately:
Currently supported by DPDK:
-- Mellanox OFED **3.4-1.0.0.0**.
+- Mellanox OFED version:
+ - ConnectX-4: **3.4-1.0.0.0** or higher
+ - ConnectX-4 Lx: **3.4-1.0.0.0** or higher
+ - ConnectX-5: **4.0-0.0.8.1** or higher
- firmware version:
- - ConnectX-4: **12.17.1010**
- - ConnectX-4 Lx: **14.17.1010**
+ - ConnectX-4: **12.17.1010** or higher
+ - ConnectX-4 Lx: **14.17.1010** or higher
+ - ConnectX-5: **16.18.0296** or higher
Getting Mellanox OFED
~~~~~~~~~~~~~~~~~~~~~
@@ -288,8 +293,8 @@ behavior as librte_pmd_mlx4:
Usage example
-------------
-This section demonstrates how to launch **testpmd** with Mellanox ConnectX-4
-devices managed by librte_pmd_mlx5.
+This section demonstrates how to launch **testpmd** with Mellanox
+ConnectX-4/ConnectX-5 devices managed by librte_pmd_mlx5.
#. Load the kernel modules:
diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst
index 699487dfe..6da4d0f8d 100644
--- a/doc/guides/rel_notes/release_17_02.rst
+++ b/doc/guides/rel_notes/release_17_02.rst
@@ -61,6 +61,11 @@ New Features
A new network PMD which supports Solarflare SFN7xxx and SFN8xxx family
of 10/40 Gbps adapters has been added.
+* **Added support for Mellanox ConnectX-5 adpaters (mlx5).**
+
+ Support for Mellanox ConnectX-5 family of 10/25/40/50/100 Gbps adapters
+ has been added to the existing mlx5 PMD.
+
Resolved Issues
---------------
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index b97b6d16a..6293c1fda 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -286,7 +286,7 @@ mlx5_args_check(const char *key, const char *val, void *opaque)
} else if (strcmp(MLX5_TXQS_MIN_INLINE, key) == 0) {
priv->txqs_inline = tmp;
} else if (strcmp(MLX5_TXQ_MPW_EN, key) == 0) {
- priv->mps = !!tmp;
+ priv->mps &= !!tmp; /* Enable MPW only if HW supports */
} else {
WARN("%s: unknown parameter", key);
return -EINVAL;
@@ -408,10 +408,26 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
sriov = ((pci_dev->id.device_id ==
PCI_DEVICE_ID_MELLANOX_CONNECTX4VF) ||
(pci_dev->id.device_id ==
- PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF));
- /* Multi-packet send is only supported by ConnectX-4 Lx PF. */
- mps = (pci_dev->id.device_id ==
- PCI_DEVICE_ID_MELLANOX_CONNECTX4LX);
+ PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF) ||
+ (pci_dev->id.device_id ==
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5VF) ||
+ (pci_dev->id.device_id ==
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5EXVF));
+ /*
+ * Multi-packet send is supported by ConnectX-4 Lx PF as well
+ * as all ConnectX-5 devices.
+ */
+ switch (pci_dev->id.device_id) {
+ case PCI_DEVICE_ID_MELLANOX_CONNECTX4LX:
+ case PCI_DEVICE_ID_MELLANOX_CONNECTX5:
+ case PCI_DEVICE_ID_MELLANOX_CONNECTX5VF:
+ case PCI_DEVICE_ID_MELLANOX_CONNECTX5EX:
+ case PCI_DEVICE_ID_MELLANOX_CONNECTX5EXVF:
+ mps = 1;
+ break;
+ default:
+ mps = 0;
+ }
INFO("PCI information matches, using device \"%s\""
" (SR-IOV: %s, MPS: %s)",
list[i]->name,
@@ -719,6 +735,22 @@ static const struct rte_pci_id mlx5_pci_id_map[] = {
PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF)
},
{
+ RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5)
+ },
+ {
+ RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5VF)
+ },
+ {
+ RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5EX)
+ },
+ {
+ RTE_PCI_DEVICE(PCI_VENDOR_ID_MELLANOX,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5EXVF)
+ },
+ {
.vendor_id = 0
}
};
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index c415ce32c..ee62e044e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -83,6 +83,10 @@ enum {
PCI_DEVICE_ID_MELLANOX_CONNECTX4VF = 0x1014,
PCI_DEVICE_ID_MELLANOX_CONNECTX4LX = 0x1015,
PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF = 0x1016,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5 = 0x1017,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5VF = 0x1018,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5EX = 0x1019,
+ PCI_DEVICE_ID_MELLANOX_CONNECTX5EXVF = 0x101a,
};
struct priv {
diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c
index 65228d5f9..fbb1b6566 100644
--- a/drivers/net/mlx5/mlx5_ethdev.c
+++ b/drivers/net/mlx5/mlx5_ethdev.c
@@ -1517,14 +1517,11 @@ void
priv_select_tx_function(struct priv *priv)
{
priv->dev->tx_pkt_burst = mlx5_tx_burst;
- /* Display warning for unsupported configurations. */
- if (priv->sriov && priv->mps)
- WARN("multi-packet send WQE cannot be used on a SR-IOV setup");
/* Select appropriate TX function. */
- if ((priv->sriov == 0) && priv->mps && priv->txq_inline) {
+ if (priv->mps && priv->txq_inline) {
priv->dev->tx_pkt_burst = mlx5_tx_burst_mpw_inline;
DEBUG("selected MPW inline TX function");
- } else if ((priv->sriov == 0) && priv->mps) {
+ } else if (priv->mps) {
priv->dev->tx_pkt_burst = mlx5_tx_burst_mpw;
DEBUG("selected MPW TX function");
}
diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c
index 053665d55..4f36402eb 100644
--- a/drivers/net/mlx5/mlx5_txq.c
+++ b/drivers/net/mlx5/mlx5_txq.c
@@ -412,7 +412,7 @@ txq_ctrl_setup(struct rte_eth_dev *dev, struct txq_ctrl *txq_ctrl,
.obj = tmpl.qp,
/* Enable multi-packet send if supported. */
.family_flags =
- ((priv->mps && !priv->sriov) ?
+ (priv->mps ?
IBV_EXP_QP_BURST_CREATE_ENABLE_MULTI_PACKET_SEND_WR :
0),
};
--
2.11.0
^ permalink raw reply related
* [PATCH v2 1/5] eal: Set numa node value for system which not support NUMA.
From: nickcooper-zhangtonghao @ 2017-01-05 12:01 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
The NUMA node information for PCI devices provided through
sysfs is invalid for AMD Opteron(TM) Processor 62xx and 63xx
on Red Hat Enterprise Linux 6, and VMs on some hypervisors.
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
lib/librte_eal/linuxapp/eal/eal_pci.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 4350134..5dfdbe9 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -317,7 +317,13 @@
free(dev);
return -1;
}
- dev->device.numa_node = tmp;
+ /* The NUMA node information for PCI devices provided through
+ * sysfs is invalid for AMD Opteron(TM) Processor 62xx and 63xx
+ * on Red Hat Enterprise Linux 6, and VMs on some hypervisors.
+ * In the upstream linux kernel, the numa_node is an integer,
+ * which data type is int, not unsigned long.
+ */
+ dev->device.numa_node = (int)tmp > 0 ? (int)tmp : 0;
}
/* parse resources */
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 2/5] vmxnet3: Avoid memory leak in vmxnet3_dev_rx_queue_setup.
From: nickcooper-zhangtonghao @ 2017-01-05 12:01 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
In-Reply-To: <1483617709-7088-1-git-send-email-nic@opencloud.tech>
This patch will check the "nb_desc" parameter for rx queue,
release the rxq and re-allocation it soon.
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
drivers/net/vmxnet3/vmxnet3_rxtx.c | 30 ++++++++++++++++++------------
1 file changed, 18 insertions(+), 12 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index b109168..e77374f 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -926,6 +926,21 @@
PMD_INIT_FUNC_TRACE();
+ /* Rx vmxnet rings length should be between 128-4096 */
+ if (nb_desc < VMXNET3_DEF_RX_RING_SIZE) {
+ PMD_INIT_LOG(ERR, "VMXNET3 Rx Ring Size Min: 128");
+ return -EINVAL;
+ } else if (nb_desc > VMXNET3_RX_RING_MAX_SIZE) {
+ PMD_INIT_LOG(ERR, "VMXNET3 Rx Ring Size Max: 4096");
+ return -EINVAL;
+ }
+
+ /* Free memory prior to re-allocation if needed. */
+ if (dev->data->rx_queues[queue_idx] != NULL) {
+ vmxnet3_dev_rx_queue_release(dev->data->rx_queues[queue_idx]);
+ dev->data->rx_queues[queue_idx] = NULL;
+ }
+
rxq = rte_zmalloc("ethdev_rx_queue", sizeof(struct vmxnet3_rx_queue),
RTE_CACHE_LINE_SIZE);
if (rxq == NULL) {
@@ -946,18 +961,9 @@
ring1 = &rxq->cmd_ring[1];
comp_ring = &rxq->comp_ring;
- /* Rx vmxnet rings length should be between 256-4096 */
- if (nb_desc < VMXNET3_DEF_RX_RING_SIZE) {
- PMD_INIT_LOG(ERR, "VMXNET3 Rx Ring Size Min: 256");
- return -EINVAL;
- } else if (nb_desc > VMXNET3_RX_RING_MAX_SIZE) {
- PMD_INIT_LOG(ERR, "VMXNET3 Rx Ring Size Max: 4096");
- return -EINVAL;
- } else {
- ring0->size = nb_desc;
- ring0->size &= ~VMXNET3_RING_SIZE_MASK;
- ring1->size = ring0->size;
- }
+ ring0->size = nb_desc;
+ ring0->size &= ~VMXNET3_RING_SIZE_MASK;
+ ring1->size = ring0->size;
comp_ring->size = ring0->size + ring1->size;
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 4/5] vmxnet3: Avoid memory leak in vmxnet3_dev_tx_queue_setup.
From: nickcooper-zhangtonghao @ 2017-01-05 12:01 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
In-Reply-To: <1483617709-7088-1-git-send-email-nic@opencloud.tech>
This patch will check the "nb_desc" parameter for tx queue,
release the txq and re-allocation it soon.
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
drivers/net/vmxnet3/vmxnet3_rxtx.c | 32 +++++++++++++++++++-------------
1 file changed, 19 insertions(+), 13 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index d5d7c33..f00b3b9 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -828,6 +828,23 @@
return -EINVAL;
}
+ /* Tx vmxnet ring length should be between 512-4096 */
+ if (nb_desc < VMXNET3_DEF_TX_RING_SIZE) {
+ PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Min: %u",
+ VMXNET3_DEF_TX_RING_SIZE);
+ return -EINVAL;
+ } else if (nb_desc > VMXNET3_TX_RING_MAX_SIZE) {
+ PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Max: %u",
+ VMXNET3_TX_RING_MAX_SIZE);
+ return -EINVAL;
+ }
+
+ /* Free memory prior to re-allocation if needed... */
+ if (dev->data->tx_queues[queue_idx] != NULL) {
+ vmxnet3_dev_tx_queue_release(dev->data->tx_queues[queue_idx]);
+ dev->data->tx_queues[queue_idx] = NULL;
+ }
+
txq = rte_zmalloc("ethdev_tx_queue", sizeof(struct vmxnet3_tx_queue),
RTE_CACHE_LINE_SIZE);
if (txq == NULL) {
@@ -846,19 +863,8 @@
comp_ring = &txq->comp_ring;
data_ring = &txq->data_ring;
- /* Tx vmxnet ring length should be between 512-4096 */
- if (nb_desc < VMXNET3_DEF_TX_RING_SIZE) {
- PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Min: %u",
- VMXNET3_DEF_TX_RING_SIZE);
- return -EINVAL;
- } else if (nb_desc > VMXNET3_TX_RING_MAX_SIZE) {
- PMD_INIT_LOG(ERR, "VMXNET3 Tx Ring Size Max: %u",
- VMXNET3_TX_RING_MAX_SIZE);
- return -EINVAL;
- } else {
- ring->size = nb_desc;
- ring->size &= ~VMXNET3_RING_SIZE_MASK;
- }
+ ring->size = nb_desc;
+ ring->size &= ~VMXNET3_RING_SIZE_MASK;
comp_ring->size = data_ring->size = ring->size;
/* Tx vmxnet rings structure initialization*/
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 3/5] vmxnet3: Avoid segfault caused by vmxnet3_dev_rx_queue_setup.
From: nickcooper-zhangtonghao @ 2017-01-05 12:01 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
In-Reply-To: <1483617709-7088-1-git-send-email-nic@opencloud.tech>
We should allocate RX ring for max possible number of hardware
descriptors. If we config RX queue with 2048 RX queue size,
and 4096 soon, there will be segment fault when calling other
ethernet API (e.g. rte_eth_dev_start).
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
drivers/net/vmxnet3/vmxnet3_rxtx.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index e77374f..d5d7c33 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -977,8 +977,11 @@
comp_ring->next2proc = 0;
comp_ring->gen = VMXNET3_INIT_GEN;
- size = sizeof(struct Vmxnet3_RxDesc) * (ring0->size + ring1->size);
- size += sizeof(struct Vmxnet3_RxCompDesc) * comp_ring->size;
+ /* Allocate RX ring for max possible number of hardware descriptors. */
+ size = sizeof(struct Vmxnet3_RxDesc) *
+ (VMXNET3_RX_RING_MAX_SIZE * VMXNET3_RX_CMDRING_SIZE);
+ size += sizeof(struct Vmxnet3_RxCompDesc) *
+ (VMXNET3_RX_RING_MAX_SIZE * VMXNET3_RX_CMDRING_SIZE);
mz = ring_dma_zone_reserve(dev, "rxdesc", queue_idx, size, socket_id);
if (mz == NULL) {
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 5/5] vmxnet3: Avoid segfault caused by vmxnet3_dev_tx_queue_setup.
From: nickcooper-zhangtonghao @ 2017-01-05 12:01 UTC (permalink / raw)
To: dev; +Cc: nickcooper-zhangtonghao
In-Reply-To: <1483617709-7088-1-git-send-email-nic@opencloud.tech>
We should allocate Tx ring for max possible mumber of hardware descriptors.
If we config Tx queue with 2048 Tx queue size, and 4096 soon,
there will be segment fault.
Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
---
drivers/net/vmxnet3/vmxnet3_rxtx.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
index f00b3b9..5f35a2e 100644
--- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
+++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
@@ -874,9 +874,10 @@
comp_ring->next2proc = 0;
comp_ring->gen = VMXNET3_INIT_GEN;
- size = sizeof(struct Vmxnet3_TxDesc) * ring->size;
- size += sizeof(struct Vmxnet3_TxCompDesc) * comp_ring->size;
- size += sizeof(struct Vmxnet3_TxDataDesc) * data_ring->size;
+ /* Allocate Tx ring for max possible number of hardware descriptors. */
+ size = sizeof(struct Vmxnet3_TxDesc) * VMXNET3_TX_RING_MAX_SIZE;
+ size += sizeof(struct Vmxnet3_TxCompDesc) * VMXNET3_TX_RING_MAX_SIZE;
+ size += sizeof(struct Vmxnet3_TxDataDesc) * VMXNET3_TX_RING_MAX_SIZE;
mz = ring_dma_zone_reserve(dev, "txdesc", queue_idx, size, socket_id);
if (mz == NULL) {
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH v2 00/26] net/qede: update qede pmd to 2.0.0.1
From: Ferruh Yigit @ 2017-01-05 13:18 UTC (permalink / raw)
To: Rasesh Mody; +Cc: dev, Dept-EngDPDKDev
In-Reply-To: <1483599848-7714-1-git-send-email-rasesh.mody@cavium.com>
On 1/5/2017 7:03 AM, Rasesh Mody wrote:
> Hi,
>
> This patch set consists of enhancements, new 8.14.x.x firmware support
> and semantic changes. It update the QEDE PMD version to 2.0.0.1.
>
> v1..v2 - addresses all the review comments
>
> Please include in DPDK 17.02 release.
>
> Thanks!
> Rasesh
>
> Rasesh Mody (26):
> net/qede/base: add request for PF FLR before load request
> net/qede/base: improve set field macro
> net/qede/base: add handling of malicious VF
> net/qede/base: change return codes in SR-IOV
> net/qede/base: make API non-static
> net/qede/base: rename macro
> net/qede/base: add check to validate txq
> net/qede/base: fix updating VF queue zone id
> net/qede/base: improve Tx-switching performance
> net/qede/base: semantic change
> net/qede: remove unused struct member
> net/qede/base: enhance resource info set printouts
> net/qede/base: add new enum member to status codes
> net/qede/base: add macros for converting pointer
> net/qede: add new host ring type option
> net/qede/base: add check for get nvm info return code
> net/qede/base: retrieve FW crash dump info
> net/qede/base: add support for external PHY
> net/qede/base: add support for 2x10G mode
> net/qede: add PCI ids for new chip variant
> net/qede: add 50G device PCI id
> net/qede/base: add support for new firmware
> net/qede/base: semantic/formatting changes
> net/qede/base: refactor some code bits
> net/qede/base: dcbx changes for base driver
> net/qede: update PMD version to 2.0.0.1
>
<...>
Series applied to dpdk-next-net/master, thanks.
^ permalink raw reply
* Re: [dpdk-stable] [PATCH 1/5] net/qede: fix scatter-gather issue
From: Ferruh Yigit @ 2017-01-05 13:26 UTC (permalink / raw)
To: Rasesh Mody, dev; +Cc: Harish Patil, stable, Dept-EngDPDKDev
In-Reply-To: <1483172217-30186-1-git-send-email-rasesh.mody@cavium.com>
On 12/31/2016 8:16 AM, Rasesh Mody wrote:
> From: Harish Patil <harish.patil@qlogic.com>
>
> - Make qede_process_sg_pkts() inline and add unlikely check
> - Fix mbuf segment chaining logic in qede_process_sg_pkts()
> - Change qede_encode_sg_bd() to return total segments required
> - Fix first TX buffer descriptor's length
> - Replace repeatitive code using a macro
>
> Fixes: bec0228816c0 ("net/qede: support scatter gather")
>
> Signed-off-by: Harish Patil <harish.patil@qlogic.com>
> ---
Hi Harish,
This patch doesn't apply cleanly on top next-net, after pci_dev changes
rebased into next-net tree.
Can you please send the new version of the patchset rebased on top of
latest next-net?
Thanks,
ferruh
<...>
^ permalink raw reply
* Re: [PATCH v1] doc: announce API and ABI change for ethdev
From: Thomas Monjalon @ 2017-01-05 13:31 UTC (permalink / raw)
To: Bernard Iremonger; +Cc: dev, john.mcnamara
In-Reply-To: <1483613049-25155-1-git-send-email-bernard.iremonger@intel.com>
2017-01-05 10:44, Bernard Iremonger:
> In 17.05 nine rte_eth_dev_* functions will be removed from
> librte_ether, renamed and moved to the ixgbe PMD.
I agree it is a good move to clean up ethdev API.
> --- a/doc/guides/rel_notes/deprecation.rst
> +++ b/doc/guides/rel_notes/deprecation.rst
> +* ethdev: for 17.05 it is planned to deprecate the following nine rte_eth_dev_* functions
> + and move them into the ixgbe PMD:
> +
> + ``rte_eth_dev_bypass_init``
> +
> + ``rte_eth_dev_bypass_state_set``
> +
> + ``rte_eth_dev_bypass_state_show``
> +
> + ``rte_eth_dev_bypass_event_store``
> +
> + ``rte_eth_dev_bypass_event_show``
> +
> + ``rte_eth_dev_wd_timeout_store``
> +
> + ``rte_eth_dev_bypass_wd_timeout_show``
> +
> + ``rte_eth_dev_bypass_ver_show``
> +
> + ``rte_eth_dev_bypass_wd_reset``
> +
> + The following fields will be removed from ``struct eth_dev_ops``:
> +
> + ``bypass_init_t``
> +
> + ``bypass_state_set_t``
> +
> + ``bypass_state_show_t``
> +
> + ``bypass_event_set_t``
> +
> + ``bypass_event_show_t``
> +
> + ``bypass_wd_timeout_set_t``
> +
> + ``bypass_wd_timeout_show_t``
> +
> + ``bypass_ver_show_t``
> +
> + ``bypass_wd_reset_t``
> +
> + The functions will be renamed to the following, and moved to the ``ixgbe`` PMD:
> +
> + ``rte_pmd_ixgbe_bypass_init``
> +
> + ``rte_pmd_ixgbe_bypass_state_set``
> +
> + ``rte_pmd_ixgbe_bypass_state_show``
> +
> + ``rte_pmd_ixgbe_bypass_event_set``
> +
> + ``rte_pmd_ixgbe_bypass_event_show``
> +
> + ``rte_pmd_ixgbe_bypass_wd_timeout_set``
> +
> + ``rte_pmd_ixgbe_bypass_wd_timeout_show``
> +
> + ``rte_pmd_ixgbe_bypass_ver_show``
> +
> + ``rte_pmd_ixgbe_bypass_wd_reset``
>
Please could you make it shorter by using commas for listing?
^ permalink raw reply
* Re: [PATCH v4 1/5] ethdev: add firmware version get
From: Thomas Monjalon @ 2017-01-05 13:44 UTC (permalink / raw)
To: Qiming Yang; +Cc: dev, ferruh.yigit, helin.zhang, remy.horton
In-Reply-To: <1483531428-14481-2-git-send-email-qiming.yang@intel.com>
2017-01-04 20:03, Qiming Yang:
> This patch adds a new API 'rte_eth_dev_fw_version_get' for
> fetching firmware version related information by a given device.
[...]
> /**
> + * Retrieve the firmware version of a device.
> + *
> + * @param port_id
> + * The port identifier of the device.
> + * @param fw_major
> + * A pointer to store the major firmware version of a device.
> + * @param fw_minor
> + * A pointer to store the minor firmware version of a device.
> + * @param fw_patch
> + * A pointer to store the firmware patch number of a device.
> + * @param etrack_id
> + * A pointer to store the nvm version of a device.
> + */
> +void rte_eth_dev_fw_version_get(uint8_t port_id, uint32_t *fw_major,
> + uint32_t *fw_minor, uint32_t *fw_patch, uint32_t *etrack_id);
After reading few comments, I think it should just fill a string.
There is no way the firmware version can be generalized or standardized.
If the application wants to do some processing like number comparisons,
it has to be aware of the specific firmware version string format.
If you want to help the application to parse this string, it should be
a PMD specific API.
^ permalink raw reply
* [PATCH v4] crypto/aesni_gcm: migration from MB library to ISA-L
From: Piotr Azarewicz @ 2017-01-05 13:51 UTC (permalink / raw)
To: pablo.de.lara.guarch, dev
In-Reply-To: <1483448523-150510-1-git-send-email-piotrx.t.azarewicz@intel.com>
Current Cryptodev AES-NI GCM PMD is implemented using Multi Buffer
Crypto library.This patch reimplement the device using ISA-L Crypto
library: https://github.com/01org/isa-l_crypto.
The migration entailed the following additional support for:
* GMAC algorithm.
* 256-bit cipher key.
* Session-less mode.
* Out-of place processing
* Scatter-gatter support for chained mbufs (only out-of place and
destination mbuf must be contiguous)
Verified current unit tests and added new unit tests to verify new
functionalities.
PERFORMANCE COMPARISON
----------------------
Comparison the new and old implementation is made by running app/test
and calling cryptodev_aesni_gcm_perftest.
As we may see below, the new implementation has small performance drop
when buffer size is above 64B.
Old implementation with MB library:
Cipher algo: AES_GCM Cipher hash: AES_GCM ciphr key: 128b burst size: 32
Buffer Size(B) OPS(M) Throughput(Gbps) Retries EmptyPolls
64 4.57 2.34 0 0
128 4.28 4.39 0 0
256 2.76 5.66 0 0
512 1.60 6.56 0 0
1024 0.90 7.34 0 0
1536 0.62 7.66 0 0
2048 0.48 7.84 0 0
New implementation with ISA-L library:
Cipher algo: AES_GCM Cipher hash: AES_GCM ciphr key: 128b burst size: 32
Buffer Size(B) OPS(M) Throughput(Gbps) Retries EmptyPolls
64 4.62 2.37 0 0
128 4.06 4.16 0 0
256 2.65 5.44 0 0
512 1.57 6.45 0 0
1024 0.89 7.26 0 0
1536 0.62 7.58 0 0
2048 0.47 7.77 0 0
Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
---
To be applied on top of:
[dpdk-dev] [PATCH v3 0/5] Chained Mbufs support in SW PMDs
v4 changes:
- rebase on top of dpdk-next-crypto
- update the script test-build.sh
v3 changes:
- rebase on top of dpdk-next-crypto
v2 changes:
- implement native scatter-gatter support for chained mbufs (only out-of
place and destination mbuf must be contiguous)
- write unit test for session-less mode
- write unit test for out-of place processing
- add support for GMAC authentication algorithm
app/test/test_cryptodev.c | 753 +++++++++++++++++++---
app/test/test_cryptodev_gcm_test_vectors.h | 491 +++++++++++++-
doc/guides/cryptodevs/aesni_gcm.rst | 23 +-
doc/guides/rel_notes/release_17_02.rst | 12 +
drivers/crypto/aesni_gcm/Makefile | 8 +-
drivers/crypto/aesni_gcm/aesni_gcm_ops.h | 95 +--
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c | 324 +++++-----
drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c | 49 +-
drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h | 15 +-
mk/rte.app.mk | 3 +-
scripts/test-build.sh | 4 +-
11 files changed, 1363 insertions(+), 414 deletions(-)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 4c9a54f..1d5c67d 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -4317,16 +4317,48 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+create_gcm_xforms(struct rte_crypto_op *op,
+ enum rte_crypto_cipher_operation cipher_op,
+ uint8_t *key, const uint8_t key_len,
+ const uint8_t aad_len, const uint8_t auth_len,
+ enum rte_crypto_auth_operation auth_op)
+{
+ TEST_ASSERT_NOT_NULL(rte_crypto_op_sym_xforms_alloc(op, 2),
+ "failed to allocate space for crypto transforms");
+
+ struct rte_crypto_sym_op *sym_op = op->sym;
+
+ /* Setup Cipher Parameters */
+ sym_op->xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+ sym_op->xform->cipher.algo = RTE_CRYPTO_CIPHER_AES_GCM;
+ sym_op->xform->cipher.op = cipher_op;
+ sym_op->xform->cipher.key.data = key;
+ sym_op->xform->cipher.key.length = key_len;
+
+ TEST_HEXDUMP(stdout, "key:", key, key_len);
+
+ /* Setup Authentication Parameters */
+ sym_op->xform->next->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+ sym_op->xform->next->auth.algo = RTE_CRYPTO_AUTH_AES_GCM;
+ sym_op->xform->next->auth.op = auth_op;
+ sym_op->xform->next->auth.digest_length = auth_len;
+ sym_op->xform->next->auth.add_auth_data_length = aad_len;
+ sym_op->xform->next->auth.key.length = 0;
+ sym_op->xform->next->auth.key.data = NULL;
+ sym_op->xform->next->next = NULL;
+
+ return 0;
+}
+
+static int
create_gcm_operation(enum rte_crypto_cipher_operation op,
- const uint8_t *auth_tag, const unsigned auth_tag_len,
- const uint8_t *iv, const unsigned iv_len,
- const uint8_t *aad, const unsigned aad_len,
- const unsigned data_len, unsigned data_pad_len)
+ const struct gcm_test_data *tdata)
{
struct crypto_testsuite_params *ts_params = &testsuite_params;
struct crypto_unittest_params *ut_params = &unittest_params;
- unsigned iv_pad_len = 0, aad_buffer_len;
+ uint8_t *plaintext, *ciphertext;
+ unsigned int iv_pad_len, aad_pad_len, plaintext_pad_len;
/* Generate Crypto op data structure */
ut_params->op = rte_crypto_op_alloc(ts_params->op_mpool,
@@ -4336,77 +4368,118 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
struct rte_crypto_sym_op *sym_op = ut_params->op->sym;
- if (ut_params->obuf) {
- sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
- ut_params->obuf, auth_tag_len);
- TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
- "no room to append digest");
- sym_op->auth.digest.phys_addr = sgl_pktmbuf_mtophys_offset(
- ut_params->obuf, data_pad_len);
- } else {
- sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
- ut_params->ibuf, auth_tag_len);
- TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
- "no room to append digest");
- sym_op->auth.digest.phys_addr = sgl_pktmbuf_mtophys_offset(
- ut_params->ibuf, data_pad_len);
- }
- sym_op->auth.digest.length = auth_tag_len;
-
- if (op == RTE_CRYPTO_CIPHER_OP_DECRYPT) {
- rte_memcpy(sym_op->auth.digest.data, auth_tag, auth_tag_len);
- TEST_HEXDUMP(stdout, "digest:",
- sym_op->auth.digest.data,
- sym_op->auth.digest.length);
- }
+ /* Append aad data */
+ aad_pad_len = RTE_ALIGN_CEIL(tdata->aad.len, 16);
+ sym_op->auth.aad.data = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf,
+ aad_pad_len);
+ TEST_ASSERT_NOT_NULL(sym_op->auth.aad.data,
+ "no room to append aad");
- /* iv */
- iv_pad_len = RTE_ALIGN_CEIL(iv_len, 16);
+ sym_op->auth.aad.length = tdata->aad.len;
+ sym_op->auth.aad.phys_addr =
+ rte_pktmbuf_mtophys(ut_params->ibuf);
+ memcpy(sym_op->auth.aad.data, tdata->aad.data, tdata->aad.len);
+ TEST_HEXDUMP(stdout, "aad:", sym_op->auth.aad.data,
+ sym_op->auth.aad.length);
+ /* Prepend iv */
+ iv_pad_len = RTE_ALIGN_CEIL(tdata->iv.len, 16);
sym_op->cipher.iv.data = (uint8_t *)rte_pktmbuf_prepend(
ut_params->ibuf, iv_pad_len);
TEST_ASSERT_NOT_NULL(sym_op->cipher.iv.data, "no room to prepend iv");
memset(sym_op->cipher.iv.data, 0, iv_pad_len);
sym_op->cipher.iv.phys_addr = rte_pktmbuf_mtophys(ut_params->ibuf);
- sym_op->cipher.iv.length = iv_len;
+ sym_op->cipher.iv.length = tdata->iv.len;
- rte_memcpy(sym_op->cipher.iv.data, iv, iv_len);
+ rte_memcpy(sym_op->cipher.iv.data, tdata->iv.data, tdata->iv.len);
+ TEST_HEXDUMP(stdout, "iv:", sym_op->cipher.iv.data,
+ sym_op->cipher.iv.length);
- /*
- * Always allocate the aad up to the block size.
- * The cryptodev API calls out -
- * - the array must be big enough to hold the AAD, plus any
- * space to round this up to the nearest multiple of the
- * block size (16 bytes).
- */
- aad_buffer_len = ALIGN_POW2_ROUNDUP(aad_len, 16);
+ /* Append plaintext/ciphertext */
+ if (op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
+ plaintext_pad_len = RTE_ALIGN_CEIL(tdata->plaintext.len, 16);
+ plaintext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf,
+ plaintext_pad_len);
+ TEST_ASSERT_NOT_NULL(plaintext, "no room to append plaintext");
- sym_op->auth.aad.data = (uint8_t *)rte_pktmbuf_prepend(
- ut_params->ibuf, aad_buffer_len);
- TEST_ASSERT_NOT_NULL(sym_op->auth.aad.data,
- "no room to prepend aad");
- sym_op->auth.aad.phys_addr = rte_pktmbuf_mtophys(
- ut_params->ibuf);
- sym_op->auth.aad.length = aad_len;
+ memcpy(plaintext, tdata->plaintext.data, tdata->plaintext.len);
+ TEST_HEXDUMP(stdout, "plaintext:", plaintext,
+ tdata->plaintext.len);
- memset(sym_op->auth.aad.data, 0, aad_buffer_len);
- rte_memcpy(sym_op->auth.aad.data, aad, aad_len);
+ if (ut_params->obuf) {
+ ciphertext = (uint8_t *)rte_pktmbuf_append(
+ ut_params->obuf,
+ plaintext_pad_len + aad_pad_len +
+ iv_pad_len);
+ TEST_ASSERT_NOT_NULL(ciphertext,
+ "no room to append ciphertext");
- TEST_HEXDUMP(stdout, "iv:", sym_op->cipher.iv.data, iv_pad_len);
- TEST_HEXDUMP(stdout, "aad:",
- sym_op->auth.aad.data, aad_len);
+ memset(ciphertext + aad_pad_len + iv_pad_len, 0,
+ tdata->ciphertext.len);
+ }
+ } else {
+ plaintext_pad_len = RTE_ALIGN_CEIL(tdata->ciphertext.len, 16);
+ ciphertext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf,
+ plaintext_pad_len);
+ TEST_ASSERT_NOT_NULL(ciphertext,
+ "no room to append ciphertext");
- if (ut_params->obuf) {
- rte_pktmbuf_prepend(ut_params->obuf, iv_pad_len);
- rte_pktmbuf_prepend(ut_params->obuf, aad_buffer_len);
+ memcpy(ciphertext, tdata->ciphertext.data,
+ tdata->ciphertext.len);
+ TEST_HEXDUMP(stdout, "ciphertext:", ciphertext,
+ tdata->ciphertext.len);
+
+ if (ut_params->obuf) {
+ plaintext = (uint8_t *)rte_pktmbuf_append(
+ ut_params->obuf,
+ plaintext_pad_len + aad_pad_len +
+ iv_pad_len);
+ TEST_ASSERT_NOT_NULL(plaintext,
+ "no room to append plaintext");
+
+ memset(plaintext + aad_pad_len + iv_pad_len, 0,
+ tdata->plaintext.len);
+ }
}
- sym_op->cipher.data.length = data_len;
- sym_op->cipher.data.offset = aad_buffer_len + iv_pad_len;
+ /* Append digest data */
+ if (op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
+ sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
+ ut_params->obuf ? ut_params->obuf :
+ ut_params->ibuf,
+ tdata->auth_tag.len);
+ TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
+ "no room to append digest");
+ memset(sym_op->auth.digest.data, 0, tdata->auth_tag.len);
+ sym_op->auth.digest.phys_addr = rte_pktmbuf_mtophys_offset(
+ ut_params->obuf ? ut_params->obuf :
+ ut_params->ibuf,
+ plaintext_pad_len +
+ aad_pad_len + iv_pad_len);
+ sym_op->auth.digest.length = tdata->auth_tag.len;
+ } else {
+ sym_op->auth.digest.data = (uint8_t *)rte_pktmbuf_append(
+ ut_params->ibuf, tdata->auth_tag.len);
+ TEST_ASSERT_NOT_NULL(sym_op->auth.digest.data,
+ "no room to append digest");
+ sym_op->auth.digest.phys_addr = rte_pktmbuf_mtophys_offset(
+ ut_params->ibuf,
+ plaintext_pad_len + aad_pad_len + iv_pad_len);
+ sym_op->auth.digest.length = tdata->auth_tag.len;
- sym_op->auth.data.offset = aad_buffer_len + iv_pad_len;
- sym_op->auth.data.length = data_len;
+ rte_memcpy(sym_op->auth.digest.data, tdata->auth_tag.data,
+ tdata->auth_tag.len);
+ TEST_HEXDUMP(stdout, "digest:",
+ sym_op->auth.digest.data,
+ sym_op->auth.digest.length);
+ }
+
+ sym_op->cipher.data.length = tdata->plaintext.len;
+ sym_op->cipher.data.offset = aad_pad_len + iv_pad_len;
+
+ sym_op->auth.data.length = tdata->plaintext.len;
+ sym_op->auth.data.offset = aad_pad_len + iv_pad_len;
return 0;
}
@@ -4418,9 +4491,9 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
struct crypto_unittest_params *ut_params = &unittest_params;
int retval;
-
- uint8_t *plaintext, *ciphertext, *auth_tag;
+ uint8_t *ciphertext, *auth_tag;
uint16_t plaintext_pad_len;
+ uint32_t i;
/* Create GCM session */
retval = create_gcm_session(ts_params->valid_devs[0],
@@ -4431,31 +4504,20 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
if (retval < 0)
return retval;
-
- ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+ if (tdata->aad.len > MBUF_SIZE) {
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->large_mbuf_pool);
+ /* Populate full size of add data */
+ for (i = 32; i < GMC_MAX_AAD_LENGTH; i += 32)
+ memcpy(&tdata->aad.data[i], &tdata->aad.data[0], 32);
+ } else
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
/* clear mbuf payload */
memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
rte_pktmbuf_tailroom(ut_params->ibuf));
- /*
- * Append data which is padded to a multiple
- * of the algorithms block size
- */
- plaintext_pad_len = RTE_ALIGN_CEIL(tdata->plaintext.len, 16);
-
- plaintext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf,
- plaintext_pad_len);
- memcpy(plaintext, tdata->plaintext.data, tdata->plaintext.len);
-
- TEST_HEXDUMP(stdout, "plaintext:", plaintext, tdata->plaintext.len);
-
- /* Create GCM opertaion */
- retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_ENCRYPT,
- tdata->auth_tag.data, tdata->auth_tag.len,
- tdata->iv.data, tdata->iv.len,
- tdata->aad.data, tdata->aad.len,
- tdata->plaintext.len, plaintext_pad_len);
+ /* Create GCM operation */
+ retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_ENCRYPT, tdata);
if (retval < 0)
return retval;
@@ -4470,14 +4532,18 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
"crypto op processing failed");
+ plaintext_pad_len = RTE_ALIGN_CEIL(tdata->plaintext.len, 16);
+
if (ut_params->op->sym->m_dst) {
ciphertext = rte_pktmbuf_mtod(ut_params->op->sym->m_dst,
uint8_t *);
auth_tag = rte_pktmbuf_mtod_offset(ut_params->op->sym->m_dst,
uint8_t *, plaintext_pad_len);
} else {
- ciphertext = plaintext;
- auth_tag = plaintext + plaintext_pad_len;
+ ciphertext = rte_pktmbuf_mtod_offset(ut_params->op->sym->m_src,
+ uint8_t *,
+ ut_params->op->sym->cipher.data.offset);
+ auth_tag = ciphertext + plaintext_pad_len;
}
TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, tdata->ciphertext.len);
@@ -4543,15 +4609,68 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+test_mb_AES_GCM_auth_encryption_test_case_256_1(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_1);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_256_2(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_2);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_256_3(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_3);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_256_4(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_4);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_256_5(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_5);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_256_6(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_6);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_256_7(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_256_7);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_aad_1(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_aad_1);
+}
+
+static int
+test_mb_AES_GCM_auth_encryption_test_case_aad_2(void)
+{
+ return test_mb_AES_GCM_authenticated_encryption(&gcm_test_case_aad_2);
+}
+
+static int
test_mb_AES_GCM_authenticated_decryption(const struct gcm_test_data *tdata)
{
struct crypto_testsuite_params *ts_params = &testsuite_params;
struct crypto_unittest_params *ut_params = &unittest_params;
int retval;
-
- uint8_t *plaintext, *ciphertext;
- uint16_t ciphertext_pad_len;
+ uint8_t *plaintext;
+ uint32_t i;
/* Create GCM session */
retval = create_gcm_session(ts_params->valid_devs[0],
@@ -4562,31 +4681,23 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
if (retval < 0)
return retval;
-
/* alloc mbuf and set payload */
- ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+ if (tdata->aad.len > MBUF_SIZE) {
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->large_mbuf_pool);
+ /* Populate full size of add data */
+ for (i = 32; i < GMC_MAX_AAD_LENGTH; i += 32)
+ memcpy(&tdata->aad.data[i], &tdata->aad.data[0], 32);
+ } else
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
rte_pktmbuf_tailroom(ut_params->ibuf));
- ciphertext_pad_len = RTE_ALIGN_CEIL(tdata->ciphertext.len, 16);
-
- ciphertext = (uint8_t *)rte_pktmbuf_append(ut_params->ibuf,
- ciphertext_pad_len);
- memcpy(ciphertext, tdata->ciphertext.data, tdata->ciphertext.len);
-
- TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, tdata->ciphertext.len);
-
- /* Create GCM opertaion */
- retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_DECRYPT,
- tdata->auth_tag.data, tdata->auth_tag.len,
- tdata->iv.data, tdata->iv.len,
- tdata->aad.data, tdata->aad.len,
- tdata->ciphertext.len, ciphertext_pad_len);
+ /* Create GCM operation */
+ retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_DECRYPT, tdata);
if (retval < 0)
return retval;
-
rte_crypto_op_attach_sym_session(ut_params->op, ut_params->sess);
ut_params->op->sym->m_src = ut_params->ibuf;
@@ -4602,7 +4713,9 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
plaintext = rte_pktmbuf_mtod(ut_params->op->sym->m_dst,
uint8_t *);
else
- plaintext = ciphertext;
+ plaintext = rte_pktmbuf_mtod_offset(ut_params->op->sym->m_src,
+ uint8_t *,
+ ut_params->op->sym->cipher.data.offset);
TEST_HEXDUMP(stdout, "plaintext:", plaintext, tdata->ciphertext.len);
@@ -4662,6 +4775,358 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
}
static int
+test_mb_AES_GCM_auth_decryption_test_case_256_1(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_1);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_256_2(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_2);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_256_3(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_3);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_256_4(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_4);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_256_5(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_5);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_256_6(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_6);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_256_7(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_256_7);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_aad_1(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_aad_1);
+}
+
+static int
+test_mb_AES_GCM_auth_decryption_test_case_aad_2(void)
+{
+ return test_mb_AES_GCM_authenticated_decryption(&gcm_test_case_aad_2);
+}
+
+static int
+test_AES_GCM_authenticated_encryption_oop(const struct gcm_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+ uint8_t *ciphertext, *auth_tag;
+ uint16_t plaintext_pad_len;
+
+ /* Create GCM session */
+ retval = create_gcm_session(ts_params->valid_devs[0],
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ tdata->key.data, tdata->key.len,
+ tdata->aad.len, tdata->auth_tag.len,
+ RTE_CRYPTO_AUTH_OP_GENERATE);
+ if (retval < 0)
+ return retval;
+
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+ ut_params->obuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+
+ /* clear mbuf payload */
+ memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
+ rte_pktmbuf_tailroom(ut_params->ibuf));
+ memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
+ rte_pktmbuf_tailroom(ut_params->obuf));
+
+ /* Create GCM operation */
+ retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_ENCRYPT, tdata);
+ if (retval < 0)
+ return retval;
+
+ rte_crypto_op_attach_sym_session(ut_params->op, ut_params->sess);
+
+ ut_params->op->sym->m_src = ut_params->ibuf;
+ ut_params->op->sym->m_dst = ut_params->obuf;
+
+ /* Process crypto operation */
+ TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op), "failed to process sym crypto op");
+
+ TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "crypto op processing failed");
+
+ plaintext_pad_len = RTE_ALIGN_CEIL(tdata->plaintext.len, 16);
+
+ ciphertext = rte_pktmbuf_mtod_offset(ut_params->obuf, uint8_t *,
+ ut_params->op->sym->cipher.data.offset);
+ auth_tag = ciphertext + plaintext_pad_len;
+
+ TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, tdata->ciphertext.len);
+ TEST_HEXDUMP(stdout, "auth tag:", auth_tag, tdata->auth_tag.len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(
+ ciphertext,
+ tdata->ciphertext.data,
+ tdata->ciphertext.len,
+ "GCM Ciphertext data not as expected");
+
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(
+ auth_tag,
+ tdata->auth_tag.data,
+ tdata->auth_tag.len,
+ "GCM Generated auth tag not as expected");
+
+ return 0;
+
+}
+
+static int
+test_mb_AES_GCM_authenticated_encryption_oop(void)
+{
+ return test_AES_GCM_authenticated_encryption_oop(&gcm_test_case_5);
+}
+
+static int
+test_AES_GCM_authenticated_decryption_oop(const struct gcm_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+ uint8_t *plaintext;
+
+ /* Create GCM session */
+ retval = create_gcm_session(ts_params->valid_devs[0],
+ RTE_CRYPTO_CIPHER_OP_DECRYPT,
+ tdata->key.data, tdata->key.len,
+ tdata->aad.len, tdata->auth_tag.len,
+ RTE_CRYPTO_AUTH_OP_VERIFY);
+ if (retval < 0)
+ return retval;
+
+ /* alloc mbuf and set payload */
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+ ut_params->obuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+
+ memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
+ rte_pktmbuf_tailroom(ut_params->ibuf));
+ memset(rte_pktmbuf_mtod(ut_params->obuf, uint8_t *), 0,
+ rte_pktmbuf_tailroom(ut_params->obuf));
+
+ /* Create GCM operation */
+ retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_DECRYPT, tdata);
+ if (retval < 0)
+ return retval;
+
+ rte_crypto_op_attach_sym_session(ut_params->op, ut_params->sess);
+
+ ut_params->op->sym->m_src = ut_params->ibuf;
+ ut_params->op->sym->m_dst = ut_params->obuf;
+
+ /* Process crypto operation */
+ TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op), "failed to process sym crypto op");
+
+ TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "crypto op processing failed");
+
+ plaintext = rte_pktmbuf_mtod_offset(ut_params->obuf, uint8_t *,
+ ut_params->op->sym->cipher.data.offset);
+
+ TEST_HEXDUMP(stdout, "plaintext:", plaintext, tdata->ciphertext.len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(
+ plaintext,
+ tdata->plaintext.data,
+ tdata->plaintext.len,
+ "GCM plaintext data not as expected");
+
+ TEST_ASSERT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "GCM authentication failed");
+ return 0;
+}
+
+static int
+test_mb_AES_GCM_authenticated_decryption_oop(void)
+{
+ return test_AES_GCM_authenticated_decryption_oop(&gcm_test_case_5);
+}
+
+static int
+test_AES_GCM_authenticated_encryption_sessionless(
+ const struct gcm_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+ uint8_t *ciphertext, *auth_tag;
+ uint16_t plaintext_pad_len;
+ uint8_t key[tdata->key.len + 1];
+
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+
+ /* clear mbuf payload */
+ memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
+ rte_pktmbuf_tailroom(ut_params->ibuf));
+
+ /* Create GCM operation */
+ retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_ENCRYPT, tdata);
+ if (retval < 0)
+ return retval;
+
+ /* Create GCM xforms */
+ memcpy(key, tdata->key.data, tdata->key.len);
+ retval = create_gcm_xforms(ut_params->op,
+ RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ key, tdata->key.len,
+ tdata->aad.len, tdata->auth_tag.len,
+ RTE_CRYPTO_AUTH_OP_GENERATE);
+ if (retval < 0)
+ return retval;
+
+ ut_params->op->sym->m_src = ut_params->ibuf;
+
+ TEST_ASSERT_EQUAL(ut_params->op->sym->sess_type,
+ RTE_CRYPTO_SYM_OP_SESSIONLESS,
+ "crypto op session type not sessionless");
+
+ /* Process crypto operation */
+ TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op), "failed to process sym crypto op");
+
+ TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
+
+ TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "crypto op status not success");
+
+ plaintext_pad_len = RTE_ALIGN_CEIL(tdata->plaintext.len, 16);
+
+ ciphertext = rte_pktmbuf_mtod_offset(ut_params->ibuf, uint8_t *,
+ ut_params->op->sym->cipher.data.offset);
+ auth_tag = ciphertext + plaintext_pad_len;
+
+ TEST_HEXDUMP(stdout, "ciphertext:", ciphertext, tdata->ciphertext.len);
+ TEST_HEXDUMP(stdout, "auth tag:", auth_tag, tdata->auth_tag.len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(
+ ciphertext,
+ tdata->ciphertext.data,
+ tdata->ciphertext.len,
+ "GCM Ciphertext data not as expected");
+
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(
+ auth_tag,
+ tdata->auth_tag.data,
+ tdata->auth_tag.len,
+ "GCM Generated auth tag not as expected");
+
+ return 0;
+
+}
+
+static int
+test_mb_AES_GCM_authenticated_encryption_sessionless(void)
+{
+ return test_AES_GCM_authenticated_encryption_sessionless(
+ &gcm_test_case_5);
+}
+
+static int
+test_AES_GCM_authenticated_decryption_sessionless(
+ const struct gcm_test_data *tdata)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ struct crypto_unittest_params *ut_params = &unittest_params;
+
+ int retval;
+ uint8_t *plaintext;
+ uint8_t key[tdata->key.len + 1];
+
+ /* alloc mbuf and set payload */
+ ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
+
+ memset(rte_pktmbuf_mtod(ut_params->ibuf, uint8_t *), 0,
+ rte_pktmbuf_tailroom(ut_params->ibuf));
+
+ /* Create GCM operation */
+ retval = create_gcm_operation(RTE_CRYPTO_CIPHER_OP_DECRYPT, tdata);
+ if (retval < 0)
+ return retval;
+
+ /* Create GCM xforms */
+ memcpy(key, tdata->key.data, tdata->key.len);
+ retval = create_gcm_xforms(ut_params->op,
+ RTE_CRYPTO_CIPHER_OP_DECRYPT,
+ key, tdata->key.len,
+ tdata->aad.len, tdata->auth_tag.len,
+ RTE_CRYPTO_AUTH_OP_VERIFY);
+ if (retval < 0)
+ return retval;
+
+ ut_params->op->sym->m_src = ut_params->ibuf;
+
+ TEST_ASSERT_EQUAL(ut_params->op->sym->sess_type,
+ RTE_CRYPTO_SYM_OP_SESSIONLESS,
+ "crypto op session type not sessionless");
+
+ /* Process crypto operation */
+ TEST_ASSERT_NOT_NULL(process_crypto_request(ts_params->valid_devs[0],
+ ut_params->op), "failed to process sym crypto op");
+
+ TEST_ASSERT_NOT_NULL(ut_params->op, "failed crypto process");
+
+ TEST_ASSERT_EQUAL(ut_params->op->status, RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "crypto op status not success");
+
+ plaintext = rte_pktmbuf_mtod_offset(ut_params->ibuf, uint8_t *,
+ ut_params->op->sym->cipher.data.offset);
+
+ TEST_HEXDUMP(stdout, "plaintext:", plaintext, tdata->ciphertext.len);
+
+ /* Validate obuf */
+ TEST_ASSERT_BUFFERS_ARE_EQUAL(
+ plaintext,
+ tdata->plaintext.data,
+ tdata->plaintext.len,
+ "GCM plaintext data not as expected");
+
+ TEST_ASSERT_EQUAL(ut_params->op->status,
+ RTE_CRYPTO_OP_STATUS_SUCCESS,
+ "GCM authentication failed");
+ return 0;
+}
+
+static int
+test_mb_AES_GCM_authenticated_decryption_sessionless(void)
+{
+ return test_AES_GCM_authenticated_decryption_sessionless(
+ &gcm_test_case_5);
+}
+
+static int
test_stats(void)
{
struct crypto_testsuite_params *ts_params = &testsuite_params;
@@ -7102,6 +7567,86 @@ struct test_crypto_vector {
TEST_CASE_ST(ut_setup, ut_teardown,
test_mb_AES_GCM_authenticated_decryption_test_case_7),
+ /** AES GCM Authenticated Encryption 256 bits key */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_1),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_2),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_3),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_4),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_5),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_6),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_256_7),
+
+ /** AES GCM Authenticated Decryption 256 bits key */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_1),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_2),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_3),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_4),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_5),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_6),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_256_7),
+
+ /** AES GCM Authenticated Encryption big aad size */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_aad_1),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_encryption_test_case_aad_2),
+
+ /** AES GCM Authenticated Decryption big aad size */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_aad_1),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_auth_decryption_test_case_aad_2),
+
+ /** AES GMAC Authentication */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GMAC_authentication_test_case_1),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GMAC_authentication_verify_test_case_1),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GMAC_authentication_test_case_3),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GMAC_authentication_verify_test_case_3),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GMAC_authentication_test_case_4),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GMAC_authentication_verify_test_case_4),
+
+ /** Negative tests */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ authentication_verify_AES128_GMAC_fail_data_corrupt),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ authentication_verify_AES128_GMAC_fail_tag_corrupt),
+
+ /** Out of place tests */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_authenticated_encryption_oop),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_authenticated_decryption_oop),
+
+ /** Session-less tests */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_authenticated_encryption_sessionless),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_mb_AES_GCM_authenticated_decryption_sessionless),
+
+ /** Scatter-Gather */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_AES_GCM_auth_encrypt_SGL_out_of_place_400B_1seg),
+
TEST_CASES_END() /**< NULL terminate unit test array */
}
};
diff --git a/app/test/test_cryptodev_gcm_test_vectors.h b/app/test/test_cryptodev_gcm_test_vectors.h
index 45ea3d4..81b4755 100644
--- a/app/test/test_cryptodev_gcm_test_vectors.h
+++ b/app/test/test_cryptodev_gcm_test_vectors.h
@@ -33,7 +33,17 @@
#ifndef TEST_CRYPTODEV_GCM_TEST_VECTORS_H_
#define TEST_CRYPTODEV_GCM_TEST_VECTORS_H_
-#define GMAC_LARGE_PLAINTEXT_LENGTH 65376
+#define GMAC_LARGE_PLAINTEXT_LENGTH 65344
+#define GMC_MAX_AAD_LENGTH 65536
+#define GMC_LARGE_AAD_LENGTH 65296
+
+static uint8_t gcm_aad_zero_text[GMC_MAX_AAD_LENGTH] = { 0 };
+
+static uint8_t gcm_aad_text[GMC_MAX_AAD_LENGTH] = {
+ 0xfe, 0xed, 0xfa, 0xce, 0xde, 0xad, 0xbe, 0xef,
+ 0xfe, 0xed, 0xfa, 0xce, 0xde, 0xad, 0xbe, 0xef,
+ 0x00, 0xf1, 0xe2, 0xd3, 0xc4, 0xb5, 0xa6, 0x97,
+ 0x88, 0x79, 0x6a, 0x5b, 0x4c, 0x3d, 0x2e, 0x1f };
struct gcm_test_data {
@@ -48,7 +58,7 @@ struct gcm_test_data {
} iv;
struct {
- uint8_t data[64];
+ uint8_t *data;
unsigned len;
} aad;
@@ -111,7 +121,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = { 0 },
+ .data = gcm_aad_zero_text,
.len = 0
},
.plaintext = {
@@ -148,7 +158,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = { 0 },
+ .data = gcm_aad_zero_text,
.len = 0
},
.plaintext = {
@@ -186,7 +196,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = { 0 },
+ .data = gcm_aad_zero_text,
.len = 0
},
.plaintext = {
@@ -238,8 +248,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = {
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 },
+ .data = gcm_aad_zero_text,
.len = 8
},
.plaintext = {
@@ -294,8 +303,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = {
- 0xfe, 0xed, 0xfa, 0xce, 0xde, 0xad, 0xbe, 0xef },
+ .data = gcm_aad_text,
.len = 8
},
.plaintext = {
@@ -351,10 +359,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = {
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x00, 0x00, 0x00, 0x00
- },
+ .data = gcm_aad_zero_text,
.len = 12
},
.plaintext = {
@@ -409,10 +414,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = {
- 0xfe, 0xed, 0xfa, 0xce, 0xde, 0xad, 0xbe, 0xef,
- 0xfe, 0xed, 0xfa, 0xce
- },
+ .data = gcm_aad_text,
.len = 12
},
.plaintext = {
@@ -466,10 +468,7 @@ struct gmac_test_data {
.len = 12
},
.aad = {
- .data = {
- 0xfe, 0xed, 0xfa, 0xce, 0xde, 0xad, 0xbe, 0xef,
- 0xfe, 0xed, 0xfa, 0xce
- },
+ .data = gcm_aad_text,
.len = 12
},
.plaintext = {
@@ -1003,6 +1002,450 @@ struct gmac_test_data {
}
};
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_1 = {
+ .key = {
+ .data = {
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_zero_text,
+ .len = 0
+ },
+ .plaintext = {
+ .data = { 0x00 },
+ .len = 0
+ },
+ .ciphertext = {
+ .data = { 0x00 },
+ .len = 0
+ },
+ .auth_tag = {
+ .data = {
+ 0x53, 0x0F, 0x8A, 0xFB, 0xC7, 0x45, 0x36, 0xB9,
+ 0xA9, 0x63, 0xB4, 0xF1, 0xC4, 0xCB, 0x73, 0x8B },
+ .len = 16
+ }
+};
+
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_2 = {
+ .key = {
+ .data = {
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_zero_text,
+ .len = 0
+ },
+ .plaintext = {
+ .data = {
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 },
+ .len = 16
+ },
+ .ciphertext = {
+ .data = {
+ 0xCE, 0xA7, 0x40, 0x3D, 0x4D, 0x60, 0x6B, 0x6E,
+ 0x07, 0x4E, 0xC5, 0xD3, 0xBA, 0xF3, 0x9D, 0x18 },
+ .len = 16
+ },
+ .auth_tag = {
+ .data = {
+ 0xD0, 0xD1, 0xC8, 0xA7, 0x99, 0x99, 0x6B, 0xF0,
+ 0x26, 0x5B, 0x98, 0xB5, 0xD4, 0x8A, 0xB9, 0x19 },
+ .len = 16
+ }
+};
+
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_3 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08,
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_zero_text,
+ .len = 0
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39, 0x1a, 0xaf, 0xd2, 0x55 },
+ .len = 64
+ },
+ .ciphertext = {
+ .data = {
+ 0x05, 0xA2, 0x39, 0xA5, 0xE1, 0x1A, 0x74, 0xEA,
+ 0x6B, 0x2A, 0x55, 0xF6, 0xD7, 0x88, 0x44, 0x7E,
+ 0x93, 0x7E, 0x23, 0x64, 0x8D, 0xF8, 0xD4, 0x04,
+ 0x3B, 0x40, 0xEF, 0x6D, 0x7C, 0x6B, 0xF3, 0xB9,
+ 0x50, 0x15, 0x97, 0x5D, 0xB8, 0x28, 0xA1, 0xD5,
+ 0x22, 0xDE, 0x36, 0x26, 0xD0, 0x6A, 0x7A, 0xC0,
+ 0xB5, 0x14, 0x36, 0xAF, 0x3A, 0xC6, 0x50, 0xAB,
+ 0xFA, 0x47, 0xC8, 0x2E, 0xF0, 0x68, 0xE1, 0x3E },
+ .len = 64
+ },
+ .auth_tag = {
+ .data = {
+ 0x64, 0xAF, 0x1D, 0xFB, 0xE8, 0x0D, 0x37, 0xD8,
+ 0x92, 0xC3, 0xB9, 0x1D, 0xD3, 0x08, 0xAB, 0xFC },
+ .len = 16
+ }
+};
+
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_4 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08,
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_zero_text,
+ .len = 8
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39 },
+ .len = 60
+ },
+ .ciphertext = {
+ .data = {
+ 0x05, 0xA2, 0x39, 0xA5, 0xE1, 0x1A, 0x74, 0xEA,
+ 0x6B, 0x2A, 0x55, 0xF6, 0xD7, 0x88, 0x44, 0x7E,
+ 0x93, 0x7E, 0x23, 0x64, 0x8D, 0xF8, 0xD4, 0x04,
+ 0x3B, 0x40, 0xEF, 0x6D, 0x7C, 0x6B, 0xF3, 0xB9,
+ 0x50, 0x15, 0x97, 0x5D, 0xB8, 0x28, 0xA1, 0xD5,
+ 0x22, 0xDE, 0x36, 0x26, 0xD0, 0x6A, 0x7A, 0xC0,
+ 0xB5, 0x14, 0x36, 0xAF, 0x3A, 0xC6, 0x50, 0xAB,
+ 0xFA, 0x47, 0xC8, 0x2E },
+ .len = 60
+ },
+ .auth_tag = {
+ .data = {
+ 0x63, 0x16, 0x91, 0xAE, 0x17, 0x05, 0x5E, 0xA6,
+ 0x6D, 0x0A, 0x51, 0xE2, 0x50, 0x21, 0x85, 0x4A },
+ .len = 16
+ }
+
+};
+
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_5 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08,
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_text,
+ .len = 8
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39 },
+ .len = 60
+ },
+ .ciphertext = {
+ .data = {
+ 0x05, 0xA2, 0x39, 0xA5, 0xE1, 0x1A, 0x74, 0xEA,
+ 0x6B, 0x2A, 0x55, 0xF6, 0xD7, 0x88, 0x44, 0x7E,
+ 0x93, 0x7E, 0x23, 0x64, 0x8D, 0xF8, 0xD4, 0x04,
+ 0x3B, 0x40, 0xEF, 0x6D, 0x7C, 0x6B, 0xF3, 0xB9,
+ 0x50, 0x15, 0x97, 0x5D, 0xB8, 0x28, 0xA1, 0xD5,
+ 0x22, 0xDE, 0x36, 0x26, 0xD0, 0x6A, 0x7A, 0xC0,
+ 0xB5, 0x14, 0x36, 0xAF, 0x3A, 0xC6, 0x50, 0xAB,
+ 0xFA, 0x47, 0xC8, 0x2E },
+ .len = 60
+ },
+ .auth_tag = {
+ .data = {
+ 0xA7, 0x99, 0xAC, 0xB8, 0x27, 0xDA, 0xB1, 0x82,
+ 0x79, 0xFD, 0x83, 0x73, 0x52, 0x4D, 0xDB, 0xF1 },
+ .len = 16
+ }
+
+};
+
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_6 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08,
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_zero_text,
+ .len = 12
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39 },
+ .len = 60
+ },
+ .ciphertext = {
+ .data = {
+ 0x05, 0xA2, 0x39, 0xA5, 0xE1, 0x1A, 0x74, 0xEA,
+ 0x6B, 0x2A, 0x55, 0xF6, 0xD7, 0x88, 0x44, 0x7E,
+ 0x93, 0x7E, 0x23, 0x64, 0x8D, 0xF8, 0xD4, 0x04,
+ 0x3B, 0x40, 0xEF, 0x6D, 0x7C, 0x6B, 0xF3, 0xB9,
+ 0x50, 0x15, 0x97, 0x5D, 0xB8, 0x28, 0xA1, 0xD5,
+ 0x22, 0xDE, 0x36, 0x26, 0xD0, 0x6A, 0x7A, 0xC0,
+ 0xB5, 0x14, 0x36, 0xAF, 0x3A, 0xC6, 0x50, 0xAB,
+ 0xFA, 0x47, 0xC8, 0x2E },
+ .len = 60
+ },
+ .auth_tag = {
+ .data = {
+ 0x5D, 0xA5, 0x0E, 0x53, 0x64, 0x7F, 0x3F, 0xAE,
+ 0x1A, 0x1F, 0xC0, 0xB0, 0xD8, 0xBE, 0xF2, 0x64 },
+ .len = 16
+ }
+};
+
+/** AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_256_7 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08,
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_text,
+ .len = 12
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39 },
+ .len = 60
+ },
+ .ciphertext = {
+ .data = {
+ 0x05, 0xA2, 0x39, 0xA5, 0xE1, 0x1A, 0x74, 0xEA,
+ 0x6B, 0x2A, 0x55, 0xF6, 0xD7, 0x88, 0x44, 0x7E,
+ 0x93, 0x7E, 0x23, 0x64, 0x8D, 0xF8, 0xD4, 0x04,
+ 0x3B, 0x40, 0xEF, 0x6D, 0x7C, 0x6B, 0xF3, 0xB9,
+ 0x50, 0x15, 0x97, 0x5D, 0xB8, 0x28, 0xA1, 0xD5,
+ 0x22, 0xDE, 0x36, 0x26, 0xD0, 0x6A, 0x7A, 0xC0,
+ 0xB5, 0x14, 0x36, 0xAF, 0x3A, 0xC6, 0x50, 0xAB,
+ 0xFA, 0x47, 0xC8, 0x2E },
+ .len = 60
+ },
+ .auth_tag = {
+ .data = {
+ 0x4E, 0xD0, 0x91, 0x95, 0x83, 0xA9, 0x38, 0x72,
+ 0x09, 0xA9, 0xCE, 0x5F, 0x89, 0x06, 0x4E, 0xC8 },
+ .len = 16
+ }
+};
+
+/** variable AAD AES-128 Test Vectors */
+static const struct gcm_test_data gcm_test_case_aad_1 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08 },
+ .len = 16
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_text,
+ .len = GMC_LARGE_AAD_LENGTH
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39, 0x1a, 0xaf, 0xd2, 0x55 },
+ .len = 64
+ },
+ .ciphertext = {
+ .data = {
+ 0x42, 0x83, 0x1E, 0xC2, 0x21, 0x77, 0x74, 0x24,
+ 0x4B, 0x72, 0x21, 0xB7, 0x84, 0xD0, 0xD4, 0x9C,
+ 0xE3, 0xAA, 0x21, 0x2F, 0x2C, 0x02, 0xA4, 0xE0,
+ 0x35, 0xC1, 0x7E, 0x23, 0x29, 0xAC, 0xA1, 0x2E,
+ 0x21, 0xD5, 0x14, 0xB2, 0x54, 0x66, 0x93, 0x1C,
+ 0x7D, 0x8F, 0x6A, 0x5A, 0xAC, 0x84, 0xAA, 0x05,
+ 0x1B, 0xA3, 0x0B, 0x39, 0x6A, 0x0A, 0xAC, 0x97,
+ 0x3D, 0x58, 0xE0, 0x91, 0x47, 0x3F, 0x59, 0x85
+ },
+ .len = 64
+ },
+ .auth_tag = {
+ .data = {
+ 0xCA, 0x70, 0xAF, 0x96, 0xA8, 0x5D, 0x40, 0x47,
+ 0x0C, 0x3C, 0x48, 0xF5, 0xF0, 0xF5, 0xA5, 0x7D
+ },
+ .len = 16
+ }
+};
+
+/** variable AAD AES-256 Test Vectors */
+static const struct gcm_test_data gcm_test_case_aad_2 = {
+ .key = {
+ .data = {
+ 0xfe, 0xff, 0xe9, 0x92, 0x86, 0x65, 0x73, 0x1c,
+ 0x6d, 0x6a, 0x8f, 0x94, 0x67, 0x30, 0x83, 0x08,
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a },
+ .len = 32
+ },
+ .iv = {
+ .data = {
+ 0xca, 0xfe, 0xba, 0xbe, 0xfa, 0xce, 0xdb, 0xad,
+ 0xde, 0xca, 0xf8, 0x88 },
+ .len = 12
+ },
+ .aad = {
+ .data = gcm_aad_text,
+ .len = GMC_LARGE_AAD_LENGTH
+ },
+ .plaintext = {
+ .data = {
+ 0xd9, 0x31, 0x32, 0x25, 0xf8, 0x84, 0x06, 0xe5,
+ 0xa5, 0x59, 0x09, 0xc5, 0xaf, 0xf5, 0x26, 0x9a,
+ 0x86, 0xa7, 0xa9, 0x53, 0x15, 0x34, 0xf7, 0xda,
+ 0x2e, 0x4c, 0x30, 0x3d, 0x8a, 0x31, 0x8a, 0x72,
+ 0x1c, 0x3c, 0x0c, 0x95, 0x95, 0x68, 0x09, 0x53,
+ 0x2f, 0xcf, 0x0e, 0x24, 0x49, 0xa6, 0xb5, 0x25,
+ 0xb1, 0x6a, 0xed, 0xf5, 0xaa, 0x0d, 0xe6, 0x57,
+ 0xba, 0x63, 0x7b, 0x39, 0x1a, 0xaf, 0xd2, 0x55 },
+ .len = 64
+ },
+ .ciphertext = {
+ .data = {
+ 0x05, 0xA2, 0x39, 0xA5, 0xE1, 0x1A, 0x74, 0xEA,
+ 0x6B, 0x2A, 0x55, 0xF6, 0xD7, 0x88, 0x44, 0x7E,
+ 0x93, 0x7E, 0x23, 0x64, 0x8D, 0xF8, 0xD4, 0x04,
+ 0x3B, 0x40, 0xEF, 0x6D, 0x7C, 0x6B, 0xF3, 0xB9,
+ 0x50, 0x15, 0x97, 0x5D, 0xB8, 0x28, 0xA1, 0xD5,
+ 0x22, 0xDE, 0x36, 0x26, 0xD0, 0x6A, 0x7A, 0xC0,
+ 0xB5, 0x14, 0x36, 0xAF, 0x3A, 0xC6, 0x50, 0xAB,
+ 0xFA, 0x47, 0xC8, 0x2E, 0xF0, 0x68, 0xE1, 0x3E
+ },
+ .len = 64
+ },
+ .auth_tag = {
+ .data = {
+ 0xBA, 0x06, 0xDA, 0xA1, 0x91, 0xE1, 0xFE, 0x22,
+ 0x59, 0xDA, 0x67, 0xAF, 0x9D, 0xA5, 0x43, 0x94
+ },
+ .len = 16
+ }
+};
+
/** GMAC Test Vectors */
static uint8_t gmac_plaintext[GMAC_LARGE_PLAINTEXT_LENGTH] = {
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
@@ -1781,8 +2224,8 @@ struct cryptodev_perf_test_data {
},
.gmac_tag = {
.data = {
- 0x88, 0x82, 0xb4, 0x93, 0x8f, 0x04, 0xcd, 0x06,
- 0xfd, 0xac, 0x6d, 0x8b, 0x9c, 0x9e, 0x8f, 0xec
+ 0x3f, 0x07, 0xcb, 0xb9, 0x86, 0x3a, 0xea, 0xc2,
+ 0x2f, 0x3a, 0x2a, 0x93, 0xd8, 0x09, 0x6b, 0xda
},
.len = 16
}
@@ -1802,7 +2245,7 @@ struct cryptodev_perf_test_data {
.len = 12
},
.aad = {
- .data = { 0 },
+ .data = gcm_aad_zero_text,
.len = 0
},
.plaintext = {
diff --git a/doc/guides/cryptodevs/aesni_gcm.rst b/doc/guides/cryptodevs/aesni_gcm.rst
index 04bf43c..184b71c 100644
--- a/doc/guides/cryptodevs/aesni_gcm.rst
+++ b/doc/guides/cryptodevs/aesni_gcm.rst
@@ -32,10 +32,8 @@ AES-NI GCM Crypto Poll Mode Driver
The AES-NI GCM PMD (**librte_pmd_aesni_gcm**) provides poll mode crypto driver
-support for utilizing Intel multi buffer library (see AES-NI Multi-buffer PMD documentation
-to learn more about it, including installation).
-
-The AES-NI GCM PMD has current only been tested on Fedora 21 64-bit with gcc.
+support for utilizing Intel ISA-L crypto library, which provides operation acceleration
+through the AES-NI instruction sets for AES-GCM authenticated cipher algorithm.
Features
--------
@@ -49,16 +47,21 @@ Cipher algorithms:
Authentication algorithms:
* RTE_CRYPTO_AUTH_AES_GCM
+* RTE_CRYPTO_AUTH_AES_GMAC
+
+Installation
+------------
+
+To build DPDK with the AESNI_GCM_PMD the user is required to install
+the ``libisal_crypto`` library in the build environment.
+For download and more details please visit `<https://github.com/01org/isa-l_crypto>`_.
Initialization
--------------
In order to enable this virtual crypto PMD, user must:
-* Export the environmental variable AESNI_MULTI_BUFFER_LIB_PATH with the path where
- the library was extracted.
-
-* Build the multi buffer library (go to Installation section in AES-NI MB PMD documentation).
+* Install the ISA-L crypto library (explained in Installation section).
* Set CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=y in config/common_base.
@@ -86,9 +89,7 @@ Example:
Limitations
-----------
-* Chained mbufs are not supported.
+* Chained mbufs are supported but only out-of-place (destination mbuf must be contiguous).
* Hash only is not supported.
* Cipher only is not supported.
-* Only in-place is currently supported (destination address is the same as source address).
-* Only supports session-oriented API implementation (session-less APIs are not supported).
* Not performance tuned.
diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst
index 5ab7019..3b982b2 100644
--- a/doc/guides/rel_notes/release_17_02.rst
+++ b/doc/guides/rel_notes/release_17_02.rst
@@ -67,6 +67,18 @@ New Features
* Support for single operations (cipher only and authentication only).
+* **Updated the AES-NI GCM PMD.**
+
+ The AES-NI GCM PMD was migrated from MB library to ISA-L library.
+ The migration entailed the following additional support for:
+
+ * GMAC algorithm.
+ * 256-bit cipher key.
+ * Session-less mode.
+ * Out-of place processing
+ * Scatter-gatter support for chained mbufs (only out-of place and destination
+ mbuf must be contiguous)
+
Resolved Issues
---------------
diff --git a/drivers/crypto/aesni_gcm/Makefile b/drivers/crypto/aesni_gcm/Makefile
index 5898cae..fb17fbf 100644
--- a/drivers/crypto/aesni_gcm/Makefile
+++ b/drivers/crypto/aesni_gcm/Makefile
@@ -31,9 +31,6 @@
include $(RTE_SDK)/mk/rte.vars.mk
ifneq ($(MAKECMDGOALS),clean)
-ifeq ($(AESNI_MULTI_BUFFER_LIB_PATH),)
-$(error "Please define AESNI_MULTI_BUFFER_LIB_PATH environment variable")
-endif
endif
# library name
@@ -50,10 +47,7 @@ LIBABIVER := 1
EXPORT_MAP := rte_pmd_aesni_gcm_version.map
# external library dependencies
-CFLAGS += -I$(AESNI_MULTI_BUFFER_LIB_PATH)
-CFLAGS += -I$(AESNI_MULTI_BUFFER_LIB_PATH)/include
-LDLIBS += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-LDLIBS += -lcrypto
+LDLIBS += -lisal_crypto
# library source files
SRCS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM) += aesni_gcm_pmd.c
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_ops.h b/drivers/crypto/aesni_gcm/aesni_gcm_ops.h
index c399068..e9de654 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_ops.h
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_ops.h
@@ -37,91 +37,26 @@
#define LINUX
#endif
-#include <gcm_defines.h>
-#include <aux_funcs.h>
+#include <isa-l_crypto/aes_gcm.h>
-/** Supported vector modes */
-enum aesni_gcm_vector_mode {
- RTE_AESNI_GCM_NOT_SUPPORTED = 0,
- RTE_AESNI_GCM_SSE,
- RTE_AESNI_GCM_AVX,
- RTE_AESNI_GCM_AVX2
-};
-
-typedef void (*aes_keyexp_128_enc_t)(void *key, void *enc_exp_keys);
+typedef void (*aesni_gcm_init_t)(struct gcm_data *my_ctx_data,
+ uint8_t *iv,
+ uint8_t const *aad,
+ uint64_t aad_len);
-typedef void (*aesni_gcm_t)(gcm_data *my_ctx_data, u8 *out, const u8 *in,
- u64 plaintext_len, u8 *iv, const u8 *aad, u64 aad_len,
- u8 *auth_tag, u64 auth_tag_len);
+typedef void (*aesni_gcm_update_t)(struct gcm_data *my_ctx_data,
+ uint8_t *out,
+ const uint8_t *in,
+ uint64_t plaintext_len);
-typedef void (*aesni_gcm_precomp_t)(gcm_data *my_ctx_data, u8 *hash_subkey);
+typedef void (*aesni_gcm_finalize_t)(struct gcm_data *my_ctx_data,
+ uint8_t *auth_tag,
+ uint64_t auth_tag_len);
-/** GCM library function pointer table */
struct aesni_gcm_ops {
- struct {
- struct {
- aes_keyexp_128_enc_t aes128_enc;
- /**< AES128 enc key expansion */
- } keyexp;
- /**< Key expansion functions */
- } aux; /**< Auxiliary functions */
-
- struct {
- aesni_gcm_t enc; /**< GCM encode function pointer */
- aesni_gcm_t dec; /**< GCM decode function pointer */
- aesni_gcm_precomp_t precomp; /**< GCM pre-compute */
- } gcm; /**< GCM functions */
+ aesni_gcm_init_t init;
+ aesni_gcm_update_t update;
+ aesni_gcm_finalize_t finalize;
};
-
-static const struct aesni_gcm_ops gcm_ops[] = {
- [RTE_AESNI_GCM_NOT_SUPPORTED] = {
- .aux = {
- .keyexp = {
- NULL
- }
- },
- .gcm = {
- NULL
- }
- },
- [RTE_AESNI_GCM_SSE] = {
- .aux = {
- .keyexp = {
- aes_keyexp_128_enc_sse
- }
- },
- .gcm = {
- aesni_gcm_enc_sse,
- aesni_gcm_dec_sse,
- aesni_gcm_precomp_sse
- }
- },
- [RTE_AESNI_GCM_AVX] = {
- .aux = {
- .keyexp = {
- aes_keyexp_128_enc_avx,
- }
- },
- .gcm = {
- aesni_gcm_enc_avx_gen2,
- aesni_gcm_dec_avx_gen2,
- aesni_gcm_precomp_avx_gen2
- }
- },
- [RTE_AESNI_GCM_AVX2] = {
- .aux = {
- .keyexp = {
- aes_keyexp_128_enc_avx2,
- }
- },
- .gcm = {
- aesni_gcm_enc_avx_gen4,
- aesni_gcm_dec_avx_gen4,
- aesni_gcm_precomp_avx_gen4
- }
- }
-};
-
-
#endif /* _AESNI_GCM_OPS_H_ */
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
index 5af22f7..8b2d792 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c
@@ -30,8 +30,6 @@
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
-#include <openssl/aes.h>
-
#include <rte_common.h>
#include <rte_config.h>
#include <rte_hexdump.h>
@@ -44,6 +42,34 @@
#include "aesni_gcm_pmd_private.h"
+/** GCM encode functions pointer table */
+static const struct aesni_gcm_ops aesni_gcm_enc[] = {
+ [AESNI_GCM_KEY_128] = {
+ aesni_gcm128_init,
+ aesni_gcm128_enc_update,
+ aesni_gcm128_enc_finalize
+ },
+ [AESNI_GCM_KEY_256] = {
+ aesni_gcm256_init,
+ aesni_gcm256_enc_update,
+ aesni_gcm256_enc_finalize
+ }
+};
+
+/** GCM decode functions pointer table */
+static const struct aesni_gcm_ops aesni_gcm_dec[] = {
+ [AESNI_GCM_KEY_128] = {
+ aesni_gcm128_init,
+ aesni_gcm128_dec_update,
+ aesni_gcm128_dec_finalize
+ },
+ [AESNI_GCM_KEY_256] = {
+ aesni_gcm256_init,
+ aesni_gcm256_dec_update,
+ aesni_gcm256_dec_finalize
+ }
+};
+
/**
* Global static parameter used to create a unique name for each AES-NI multi
* buffer crypto device.
@@ -65,112 +91,68 @@
return 0;
}
-static int
-aesni_gcm_calculate_hash_sub_key(uint8_t *hsubkey, unsigned hsubkey_length,
- uint8_t *aeskey, unsigned aeskey_length)
-{
- uint8_t key[aeskey_length] __rte_aligned(16);
- AES_KEY enc_key;
-
- if (hsubkey_length % 16 != 0 && aeskey_length % 16 != 0)
- return -EFAULT;
-
- memcpy(key, aeskey, aeskey_length);
-
- if (AES_set_encrypt_key(key, aeskey_length << 3, &enc_key) != 0)
- return -EFAULT;
-
- AES_encrypt(hsubkey, hsubkey, &enc_key);
-
- return 0;
-}
-
-/** Get xform chain order */
-static int
-aesni_gcm_get_mode(const struct rte_crypto_sym_xform *xform)
-{
- /*
- * GCM only supports authenticated encryption or authenticated
- * decryption, all other options are invalid, so we must have exactly
- * 2 xform structs chained together
- */
- if (xform->next == NULL || xform->next->next != NULL)
- return -1;
-
- if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
- return AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION;
- }
-
- if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
- xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
- return AESNI_GCM_OP_AUTHENTICATED_DECRYPTION;
- }
-
- return -1;
-}
-
/** Parse crypto xform chain and set private session parameters */
int
-aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *gcm_ops,
- struct aesni_gcm_session *sess,
+aesni_gcm_set_session_parameters(struct aesni_gcm_session *sess,
const struct rte_crypto_sym_xform *xform)
{
- const struct rte_crypto_sym_xform *auth_xform = NULL;
- const struct rte_crypto_sym_xform *cipher_xform = NULL;
+ const struct rte_crypto_sym_xform *auth_xform;
+ const struct rte_crypto_sym_xform *cipher_xform;
- uint8_t hsubkey[16] __rte_aligned(16) = { 0 };
-
- /* Select Crypto operation - hash then cipher / cipher then hash */
- switch (aesni_gcm_get_mode(xform)) {
- case AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION:
- sess->op = AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION;
+ if (xform->next == NULL || xform->next->next != NULL) {
+ GCM_LOG_ERR("Two and only two chained xform required");
+ return -EINVAL;
+ }
- cipher_xform = xform;
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
auth_xform = xform->next;
- break;
- case AESNI_GCM_OP_AUTHENTICATED_DECRYPTION:
- sess->op = AESNI_GCM_OP_AUTHENTICATED_DECRYPTION;
-
+ cipher_xform = xform;
+ } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH &&
+ xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
auth_xform = xform;
cipher_xform = xform->next;
- break;
- default:
- GCM_LOG_ERR("Unsupported operation chain order parameter");
+ } else {
+ GCM_LOG_ERR("Cipher and auth xform required");
return -EINVAL;
}
- /* We only support AES GCM */
- if (cipher_xform->cipher.algo != RTE_CRYPTO_CIPHER_AES_GCM &&
- auth_xform->auth.algo != RTE_CRYPTO_AUTH_AES_GCM)
+ if (!(cipher_xform->cipher.algo == RTE_CRYPTO_CIPHER_AES_GCM &&
+ (auth_xform->auth.algo == RTE_CRYPTO_AUTH_AES_GCM ||
+ auth_xform->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC))) {
+ GCM_LOG_ERR("We only support AES GCM and AES GMAC");
return -EINVAL;
+ }
- /* Select cipher direction */
- if (sess->op == AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION &&
- cipher_xform->cipher.op !=
- RTE_CRYPTO_CIPHER_OP_ENCRYPT) {
- GCM_LOG_ERR("xform chain (CIPHER/AUTH) and cipher operation "
- "(DECRYPT) specified are an invalid selection");
- return -EINVAL;
- } else if (sess->op == AESNI_GCM_OP_AUTHENTICATED_DECRYPTION &&
- cipher_xform->cipher.op !=
- RTE_CRYPTO_CIPHER_OP_DECRYPT) {
- GCM_LOG_ERR("xform chain (AUTH/CIPHER) and cipher operation "
- "(ENCRYPT) specified are an invalid selection");
+ /* Select Crypto operation */
+ if (cipher_xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT &&
+ auth_xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE)
+ sess->op = AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION;
+ else if (cipher_xform->cipher.op == RTE_CRYPTO_CIPHER_OP_DECRYPT &&
+ auth_xform->auth.op == RTE_CRYPTO_AUTH_OP_VERIFY)
+ sess->op = AESNI_GCM_OP_AUTHENTICATED_DECRYPTION;
+ else {
+ GCM_LOG_ERR("Cipher/Auth operations: Encrypt/Generate or"
+ " Decrypt/Verify are valid only");
return -EINVAL;
}
- /* Expand GCM AES128 key */
- (*gcm_ops->aux.keyexp.aes128_enc)(cipher_xform->cipher.key.data,
- sess->gdata.expanded_keys);
+ /* Check key length and calculate GCM pre-compute. */
+ switch (cipher_xform->cipher.key.length) {
+ case 16:
+ aesni_gcm128_pre(cipher_xform->cipher.key.data, &sess->gdata);
+ sess->key = AESNI_GCM_KEY_128;
- /* Calculate hash sub key here */
- aesni_gcm_calculate_hash_sub_key(hsubkey, sizeof(hsubkey),
- cipher_xform->cipher.key.data,
- cipher_xform->cipher.key.length);
+ break;
+ case 32:
+ aesni_gcm256_pre(cipher_xform->cipher.key.data, &sess->gdata);
+ sess->key = AESNI_GCM_KEY_256;
- /* Calculate GCM pre-compute */
- (*gcm_ops->gcm.precomp)(&sess->gdata, hsubkey);
+ break;
+ default:
+ GCM_LOG_ERR("Unsupported cipher key length");
+ return -EINVAL;
+ }
return 0;
}
@@ -194,10 +176,10 @@
return sess;
sess = (struct aesni_gcm_session *)
- ((struct rte_cryptodev_session *)_sess)->_private;
+ ((struct rte_cryptodev_sym_session *)_sess)->_private;
- if (unlikely(aesni_gcm_set_session_parameters(qp->ops,
- sess, op->xform) != 0)) {
+ if (unlikely(aesni_gcm_set_session_parameters(sess,
+ op->xform) != 0)) {
rte_mempool_put(qp->sess_mp, _sess);
sess = NULL;
}
@@ -217,19 +199,45 @@
*
*/
static int
-process_gcm_crypto_op(struct aesni_gcm_qp *qp, struct rte_crypto_sym_op *op,
+process_gcm_crypto_op(struct rte_crypto_sym_op *op,
struct aesni_gcm_session *session)
{
uint8_t *src, *dst;
- struct rte_mbuf *m = op->m_src;
+ struct rte_mbuf *m_src = op->m_src;
+ uint32_t offset = op->cipher.data.offset;
+ uint32_t part_len, total_len, data_len;
+
+ RTE_ASSERT(m_src != NULL);
+
+ while (offset >= m_src->data_len) {
+ offset -= m_src->data_len;
+ m_src = m_src->next;
+
+ RTE_ASSERT(m_src != NULL);
+ }
+
+ data_len = m_src->data_len - offset;
+ part_len = (data_len < op->cipher.data.length) ? data_len :
+ op->cipher.data.length;
+
+ /* Destination buffer is required when segmented source buffer */
+ RTE_ASSERT((part_len == op->cipher.data.length) ||
+ ((part_len != op->cipher.data.length) &&
+ (op->m_dst != NULL)));
+ /* Segmented destination buffer is not supported */
+ RTE_ASSERT((op->m_dst == NULL) ||
+ ((op->m_dst != NULL) &&
+ rte_pktmbuf_is_contiguous(op->m_dst)));
+
- src = rte_pktmbuf_mtod(m, uint8_t *) + op->cipher.data.offset;
dst = op->m_dst ?
rte_pktmbuf_mtod_offset(op->m_dst, uint8_t *,
op->cipher.data.offset) :
- rte_pktmbuf_mtod_offset(m, uint8_t *,
+ rte_pktmbuf_mtod_offset(op->m_src, uint8_t *,
op->cipher.data.offset);
+ src = rte_pktmbuf_mtod_offset(m_src, uint8_t *, offset);
+
/* sanity checks */
if (op->cipher.iv.length != 16 && op->cipher.iv.length != 12 &&
op->cipher.iv.length != 0) {
@@ -246,48 +254,81 @@
*iv_padd = rte_bswap32(1);
}
- if (op->auth.aad.length != 12 && op->auth.aad.length != 8 &&
- op->auth.aad.length != 0) {
- GCM_LOG_ERR("iv");
- return -1;
- }
-
if (op->auth.digest.length != 16 &&
op->auth.digest.length != 12 &&
- op->auth.digest.length != 8 &&
- op->auth.digest.length != 0) {
- GCM_LOG_ERR("iv");
+ op->auth.digest.length != 8) {
+ GCM_LOG_ERR("digest");
return -1;
}
if (session->op == AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION) {
- (*qp->ops->gcm.enc)(&session->gdata, dst, src,
- (uint64_t)op->cipher.data.length,
+ aesni_gcm_enc[session->key].init(&session->gdata,
op->cipher.iv.data,
op->auth.aad.data,
- (uint64_t)op->auth.aad.length,
+ (uint64_t)op->auth.aad.length);
+
+ aesni_gcm_enc[session->key].update(&session->gdata, dst, src,
+ (uint64_t)part_len);
+ total_len = op->cipher.data.length - part_len;
+
+ while (total_len) {
+ dst += part_len;
+ m_src = m_src->next;
+
+ RTE_ASSERT(m_src != NULL);
+
+ src = rte_pktmbuf_mtod(m_src, uint8_t *);
+ part_len = (m_src->data_len < total_len) ?
+ m_src->data_len : total_len;
+
+ aesni_gcm_enc[session->key].update(&session->gdata,
+ dst, src,
+ (uint64_t)part_len);
+ total_len -= part_len;
+ }
+
+ aesni_gcm_enc[session->key].finalize(&session->gdata,
op->auth.digest.data,
(uint64_t)op->auth.digest.length);
- } else if (session->op == AESNI_GCM_OP_AUTHENTICATED_DECRYPTION) {
- uint8_t *auth_tag = (uint8_t *)rte_pktmbuf_append(m,
+ } else { /* session->op == AESNI_GCM_OP_AUTHENTICATED_DECRYPTION */
+ uint8_t *auth_tag = (uint8_t *)rte_pktmbuf_append(op->m_dst ?
+ op->m_dst : op->m_src,
op->auth.digest.length);
if (!auth_tag) {
- GCM_LOG_ERR("iv");
+ GCM_LOG_ERR("auth_tag");
return -1;
}
- (*qp->ops->gcm.dec)(&session->gdata, dst, src,
- (uint64_t)op->cipher.data.length,
+ aesni_gcm_dec[session->key].init(&session->gdata,
op->cipher.iv.data,
op->auth.aad.data,
- (uint64_t)op->auth.aad.length,
+ (uint64_t)op->auth.aad.length);
+
+ aesni_gcm_dec[session->key].update(&session->gdata, dst, src,
+ (uint64_t)part_len);
+ total_len = op->cipher.data.length - part_len;
+
+ while (total_len) {
+ dst += part_len;
+ m_src = m_src->next;
+
+ RTE_ASSERT(m_src != NULL);
+
+ src = rte_pktmbuf_mtod(m_src, uint8_t *);
+ part_len = (m_src->data_len < total_len) ?
+ m_src->data_len : total_len;
+
+ aesni_gcm_dec[session->key].update(&session->gdata,
+ dst, src,
+ (uint64_t)part_len);
+ total_len -= part_len;
+ }
+
+ aesni_gcm_dec[session->key].finalize(&session->gdata,
auth_tag,
(uint64_t)op->auth.digest.length);
- } else {
- GCM_LOG_ERR("iv");
- return -1;
}
return 0;
@@ -377,21 +418,7 @@
break;
}
-#ifdef RTE_LIBRTE_PMD_AESNI_GCM_DEBUG
- if (!rte_pktmbuf_is_contiguous(ops[i]->sym->m_src) ||
- (ops[i]->sym->m_dst != NULL &&
- !rte_pktmbuf_is_contiguous(
- ops[i]->sym->m_dst))) {
- ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
- GCM_LOG_ERR("PMD supports only contiguous mbufs, "
- "op (%p) provides noncontiguous mbuf as "
- "source/destination buffer.\n", ops[i]);
- qp->qp_stats.enqueue_err_count++;
- break;
- }
-#endif
-
- retval = process_gcm_crypto_op(qp, ops[i]->sym, sess);
+ retval = process_gcm_crypto_op(ops[i]->sym, sess);
if (retval < 0) {
ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
qp->qp_stats.enqueue_err_count++;
@@ -429,7 +456,6 @@
struct rte_cryptodev *dev;
char crypto_dev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
struct aesni_gcm_private *internals;
- enum aesni_gcm_vector_mode vector_mode;
/* Check CPU for support for AES instruction set */
if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -437,18 +463,6 @@
return -EFAULT;
}
- /* Check CPU for supported vector instruction set */
- if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2))
- vector_mode = RTE_AESNI_GCM_AVX2;
- else if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX))
- vector_mode = RTE_AESNI_GCM_AVX;
- else if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SSE4_1))
- vector_mode = RTE_AESNI_GCM_SSE;
- else {
- GCM_LOG_ERR("Vector instructions are not supported by CPU");
- return -EFAULT;
- }
-
/* create a unique device name */
if (create_unique_device_name(crypto_dev_name,
RTE_CRYPTODEV_NAME_MAX_LEN) != 0) {
@@ -473,27 +487,11 @@
dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
- RTE_CRYPTODEV_FF_CPU_AESNI;
+ RTE_CRYPTODEV_FF_CPU_AESNI |
+ RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER;
- switch (vector_mode) {
- case RTE_AESNI_GCM_SSE:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE;
- break;
- case RTE_AESNI_GCM_AVX:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX;
- break;
- case RTE_AESNI_GCM_AVX2:
- dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2;
- break;
- default:
- break;
- }
-
- /* Set vector instructions mode supported */
internals = dev->data->dev_private;
- internals->vector_mode = vector_mode;
-
internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
internals->max_nb_sessions = init_params->max_nb_sessions;
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c
index c51f82a..2362006 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c
@@ -39,17 +39,17 @@
#include "aesni_gcm_pmd_private.h"
static const struct rte_cryptodev_capabilities aesni_gcm_pmd_capabilities[] = {
- { /* AES GCM (AUTH) */
+ { /* AES GMAC (AUTH) */
.op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
{.sym = {
.xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
{.auth = {
- .algo = RTE_CRYPTO_AUTH_AES_GCM,
+ .algo = RTE_CRYPTO_AUTH_AES_GMAC,
.block_size = 16,
.key_size = {
.min = 16,
- .max = 16,
- .increment = 0
+ .max = 32,
+ .increment = 16
},
.digest_size = {
.min = 8,
@@ -57,9 +57,34 @@
.increment = 4
},
.aad_size = {
+ .min = 0,
+ .max = 65535,
+ .increment = 1
+ }
+ }, }
+ }, }
+ },
+ { /* AES GCM (AUTH) */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+ {.auth = {
+ .algo = RTE_CRYPTO_AUTH_AES_GCM,
+ .block_size = 16,
+ .key_size = {
+ .min = 16,
+ .max = 32,
+ .increment = 16
+ },
+ .digest_size = {
.min = 8,
- .max = 12,
+ .max = 16,
.increment = 4
+ },
+ .aad_size = {
+ .min = 0,
+ .max = 65535,
+ .increment = 1
}
}, }
}, }
@@ -73,8 +98,8 @@
.block_size = 16,
.key_size = {
.min = 16,
- .max = 16,
- .increment = 0
+ .max = 32,
+ .increment = 16
},
.iv_size = {
.min = 12,
@@ -221,7 +246,6 @@
int socket_id)
{
struct aesni_gcm_qp *qp = NULL;
- struct aesni_gcm_private *internals = dev->data->dev_private;
/* Free memory prior to re-allocation if needed. */
if (dev->data->queue_pairs[qp_id] != NULL)
@@ -239,8 +263,6 @@
if (aesni_gcm_pmd_qp_set_unique_name(dev, qp))
goto qp_setup_cleanup;
- qp->ops = &gcm_ops[internals->vector_mode];
-
qp->processed_pkts = aesni_gcm_pmd_qp_create_processed_pkts_ring(qp,
qp_conf->nb_descriptors, socket_id);
if (qp->processed_pkts == NULL)
@@ -291,18 +313,15 @@
/** Configure a aesni gcm session from a crypto xform chain */
static void *
-aesni_gcm_pmd_session_configure(struct rte_cryptodev *dev,
+aesni_gcm_pmd_session_configure(struct rte_cryptodev *dev __rte_unused,
struct rte_crypto_sym_xform *xform, void *sess)
{
- struct aesni_gcm_private *internals = dev->data->dev_private;
-
if (unlikely(sess == NULL)) {
GCM_LOG_ERR("invalid session struct");
return NULL;
}
- if (aesni_gcm_set_session_parameters(&gcm_ops[internals->vector_mode],
- sess, xform) != 0) {
+ if (aesni_gcm_set_session_parameters(sess, xform) != 0) {
GCM_LOG_ERR("failed configure session parameters");
return NULL;
}
diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
index 9878d6e..0496b44 100644
--- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
+++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h
@@ -58,8 +58,6 @@
/** private data structure for each virtual AESNI GCM device */
struct aesni_gcm_private {
- enum aesni_gcm_vector_mode vector_mode;
- /**< Vector mode */
unsigned max_nb_queue_pairs;
/**< Max number of queue pairs supported by device */
unsigned max_nb_sessions;
@@ -71,8 +69,6 @@ struct aesni_gcm_qp {
/**< Queue Pair Identifier */
char name[RTE_CRYPTODEV_NAME_LEN];
/**< Unique Queue Pair Name */
- const struct aesni_gcm_ops *ops;
- /**< Architecture dependent function pointer table of the gcm APIs */
struct rte_ring *processed_pkts;
/**< Ring for placing process packets */
struct rte_mempool *sess_mp;
@@ -87,10 +83,17 @@ enum aesni_gcm_operation {
AESNI_GCM_OP_AUTHENTICATED_DECRYPTION
};
+enum aesni_gcm_key {
+ AESNI_GCM_KEY_128,
+ AESNI_GCM_KEY_256
+};
+
/** AESNI GCM private session structure */
struct aesni_gcm_session {
enum aesni_gcm_operation op;
/**< GCM operation type */
+ enum aesni_gcm_key key;
+ /**< GCM key type */
struct gcm_data gdata __rte_cache_aligned;
/**< GCM parameters */
};
@@ -98,7 +101,6 @@ struct aesni_gcm_session {
/**
* Setup GCM session parameters
- * @param ops gcm ops function pointer table
* @param sess aesni gcm session structure
* @param xform crypto transform chain
*
@@ -107,8 +109,7 @@ struct aesni_gcm_session {
* - On failure returns error code < 0
*/
extern int
-aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *ops,
- struct aesni_gcm_session *sess,
+aesni_gcm_set_session_parameters(struct aesni_gcm_session *sess,
const struct rte_crypto_sym_xform *xform);
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index f75f0e2..ed3eab5 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -134,8 +134,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += -lrte_pmd_vmxnet3_uio
ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += -lrte_pmd_aesni_mb
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM) += -lrte_pmd_aesni_gcm -lcrypto
-_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM) += -L$(AESNI_MULTI_BUFFER_LIB_PATH) -lIPSec_MB
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM) += -lrte_pmd_aesni_gcm -lisal_crypto
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_OPENSSL) += -lrte_pmd_openssl -lcrypto
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += -lrte_pmd_null_crypto
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += -lrte_pmd_qat -lcrypto
diff --git a/scripts/test-build.sh b/scripts/test-build.sh
index a979309..d0c1a34 100755
--- a/scripts/test-build.sh
+++ b/scripts/test-build.sh
@@ -44,6 +44,7 @@ default_path=$PATH
# - DPDK_DEP_SSL (y/[n])
# - DPDK_DEP_SZE (y/[n])
# - DPDK_DEP_ZLIB (y/[n])
+# - DPDK_DEP_ISAL (y/[n])
# - DPDK_MAKE_JOBS (int)
# - DPDK_NOTIFY (notify-send)
# - LIBSSO_SNOW3G_PATH
@@ -126,6 +127,7 @@ reset_env ()
unset DPDK_DEP_SSL
unset DPDK_DEP_SZE
unset DPDK_DEP_ZLIB
+ unset DPDK_DEP_ISAL
unset AESNI_MULTI_BUFFER_LIB_PATH
unset LIBSSO_SNOW3G_PATH
unset LIBSSO_KASUMI_PATH
@@ -176,7 +178,7 @@ config () # <directory> <target> <options>
sed -ri 's,(PCAP=)n,\1y,' $1/.config
test -z "$AESNI_MULTI_BUFFER_LIB_PATH" || \
sed -ri 's,(PMD_AESNI_MB=)n,\1y,' $1/.config
- test -z "$AESNI_MULTI_BUFFER_LIB_PATH" || \
+ test "$DPDK_DEP_ISAL" != y || \
sed -ri 's,(PMD_AESNI_GCM=)n,\1y,' $1/.config
test -z "$LIBSSO_SNOW3G_PATH" || \
sed -ri 's,(PMD_SNOW3G=)n,\1y,' $1/.config
--
1.7.9.5
^ permalink raw reply related
* [PATCH v2 1/4] net/af_packet: add iface name to internals
From: Charles (Chas) Williams @ 2017-01-05 13:53 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
This will be used by later changes to determine the underlying linux
interface.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 17b4ffe..4ef61a2 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -99,6 +99,7 @@ struct pmd_internals {
unsigned nb_queues;
int if_index;
+ char *if_name;
struct ether_addr eth_addr;
struct tpacket_req req;
@@ -532,6 +533,7 @@ rte_pmd_init_internals(const char *name,
name);
goto error_early;
}
+ (*internals)->if_name = strdup(pair->value);
(*internals)->if_index = ifr.ifr_ifindex;
if (ioctl(sockfd, SIOCGIFHWADDR, &ifr) == -1) {
@@ -723,6 +725,7 @@ rte_pmd_init_internals(const char *name,
((*internals)->rx_queue[q].sockfd != qsockfd))
close((*internals)->rx_queue[q].sockfd);
}
+ free((*internals)->if_name);
rte_free(*internals);
error_early:
rte_free(data);
@@ -891,6 +894,7 @@ rte_pmd_af_packet_remove(const char *name)
rte_free(internals->rx_queue[q].rd);
rte_free(internals->tx_queue[q].rd);
}
+ free(internals->if_name);
rte_free(eth_dev->data->dev_private);
rte_free(eth_dev->data);
--
2.1.4
^ permalink raw reply related
* [PATCH v2 2/4] net/af_packet: add support to change mtu
From: Charles (Chas) Williams @ 2017-01-05 13:53 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
In-Reply-To: <1483624424-2806-1-git-send-email-ciwillia@brocade.com>
The underlying linux device's MTU is changed subject to the frame size
limitations during device creation.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 4ef61a2..a700b96 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -412,12 +412,41 @@ eth_tx_queue_setup(struct rte_eth_dev *dev,
return 0;
}
+static int
+eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+ struct ifreq ifr = { .ifr_mtu = mtu };
+ int ret;
+ int s;
+ unsigned int data_size = internals->req.tp_frame_size -
+ TPACKET2_HDRLEN -
+ sizeof(struct sockaddr_ll);
+
+ if (mtu > data_size)
+ return -EINVAL;
+
+ s = socket(PF_INET, SOCK_DGRAM, 0);
+ if (s < 0)
+ return -EINVAL;
+
+ strncpy(ifr.ifr_name, internals->if_name, IFNAMSIZ);
+ ret = ioctl(s, SIOCSIFMTU, &ifr);
+ close(s);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
.dev_close = eth_dev_close,
.dev_configure = eth_dev_configure,
.dev_infos_get = eth_dev_info,
+ .mtu_set = eth_dev_mtu_set,
.rx_queue_setup = eth_rx_queue_setup,
.tx_queue_setup = eth_tx_queue_setup,
.rx_queue_release = eth_queue_release,
--
2.1.4
^ permalink raw reply related
* [PATCH v2 3/4] net/af_packet: promisicuous support
From: Charles (Chas) Williams @ 2017-01-05 13:53 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
In-Reply-To: <1483624424-2806-1-git-send-email-ciwillia@brocade.com>
Add promiscuous support to the AF_PACKET PMD. The underlying linux
device's IF_PROMISC flag is toggled to enable or disable.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 39 +++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index a700b96..6aa8132 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -440,6 +440,43 @@ eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)
return 0;
}
+static void
+eth_dev_change_flags(char *if_name, uint32_t flags, uint32_t mask)
+{
+ struct ifreq ifr;
+ int s;
+
+ s = socket(PF_INET, SOCK_DGRAM, 0);
+ if (s < 0)
+ return;
+
+ strncpy(ifr.ifr_name, if_name, IFNAMSIZ);
+ if (ioctl(s, SIOCGIFFLAGS, &ifr) < 0)
+ goto out;
+ ifr.ifr_flags &= mask;
+ ifr.ifr_flags |= flags;
+ if (ioctl(s, SIOCSIFFLAGS, &ifr) < 0)
+ goto out;
+out:
+ close(s);
+}
+
+static void
+eth_dev_promiscuous_enable(struct rte_eth_dev *dev)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+
+ eth_dev_change_flags(internals->if_name, IFF_PROMISC, ~0);
+}
+
+static void
+eth_dev_promiscuous_disable(struct rte_eth_dev *dev)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+
+ eth_dev_change_flags(internals->if_name, 0, ~IFF_PROMISC);
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -447,6 +484,8 @@ static const struct eth_dev_ops ops = {
.dev_configure = eth_dev_configure,
.dev_infos_get = eth_dev_info,
.mtu_set = eth_dev_mtu_set,
+ .promiscuous_enable = eth_dev_promiscuous_enable,
+ .promiscuous_disable = eth_dev_promiscuous_disable,
.rx_queue_setup = eth_rx_queue_setup,
.tx_queue_setup = eth_tx_queue_setup,
.rx_queue_release = eth_queue_release,
--
2.1.4
^ permalink raw reply related
* [PATCH v2 4/4] net/af_packet: add 802.1Q (VLAN) support
From: Charles (Chas) Williams @ 2017-01-05 13:53 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
In-Reply-To: <1483624424-2806-1-git-send-email-ciwillia@brocade.com>
AF_PACKET has some flags to check on the receive side for 802.1Q
information. If present, we copy into the mbuf. For transmit, we
insert any 802.1Q information into the packet before copying to the ring.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 6aa8132..ee51c17 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -161,6 +161,12 @@ eth_af_packet_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
pbuf = (uint8_t *) ppd + ppd->tp_mac;
memcpy(rte_pktmbuf_mtod(mbuf, void *), pbuf, rte_pktmbuf_data_len(mbuf));
+ /* check for vlan info */
+ if (ppd->tp_status & TP_STATUS_VLAN_VALID) {
+ mbuf->vlan_tci = ppd->tp_vlan_tci;
+ mbuf->ol_flags |= (PKT_RX_VLAN_PKT | PKT_RX_VLAN_STRIPPED);
+ }
+
/* release incoming frame and advance ring buffer */
ppd->tp_status = TP_STATUS_KERNEL;
if (++framenum >= framecount)
@@ -214,6 +220,14 @@ eth_af_packet_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
continue;
}
+ /* insert vlan info if necessary */
+ if (mbuf->ol_flags & PKT_TX_VLAN_PKT) {
+ if (rte_vlan_insert(&mbuf)) {
+ rte_pktmbuf_free(mbuf);
+ continue;
+ }
+ }
+
/* point at the next incoming frame */
if ((ppd->tp_status != TP_STATUS_AVAILABLE) &&
(poll(&pfd, 1, -1) < 0))
--
2.1.4
^ permalink raw reply related
* Re: [PATCH v2 1/4] net/af_packet: add iface name to internals
From: Ferruh Yigit @ 2017-01-05 14:04 UTC (permalink / raw)
To: Charles (Chas) Williams, dev; +Cc: linville
In-Reply-To: <1483624424-2806-1-git-send-email-ciwillia@brocade.com>
On 1/5/2017 1:53 PM, Charles (Chas) Williams wrote:
> This will be used by later changes to determine the underlying linux
> interface.
>
> Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
Hi Charles,
Independent from the patch content,
it will be useful if you can send new patchset as reply to previous one,
this way all patches will be in same mail thread, this will help to
trace/review them, also it will be easier to find them in mail archive
in the future.
Another thing is, if you can add a version diff in the comment log,
after "---", that helps others to understand what has been changed in
this new version.
Thanks,
ferruh
<...>
^ permalink raw reply
* Re: [PATCH 0/7] net/mlx5: improve single core performance
From: Ferruh Yigit @ 2017-01-05 14:13 UTC (permalink / raw)
To: Nelio Laranjeiro, dev; +Cc: Thomas Monjalon, Adrien Mazarguil
In-Reply-To: <cover.1479995764.git.nelio.laranjeiro@6wind.com>
On 11/24/2016 4:03 PM, Nelio Laranjeiro wrote:
> This series applies on top of
> "[PATCH] eal: define generic vector types" [1][2]
>
> Using built-in vector types forces compilers to consider SIMD instructions in
> specific places in order to improve performance on both IBM POWER8 and Intel
> architectures.
>
> For example, testpmd single-thread I/O forwarding packets per second
> performance is improved by 6% on Intel platforms.
>
> [1] http://dpdk.org/ml/archives/dev/2016-November/050261.html
> [2] http://dpdk.org/dev/patchwork/patch/17024/
>
> Nelio Laranjeiro (7):
> net/mlx5: prepare Tx vectorization
> net/mlx5: use work queue buffer as a raw buffer
> net/mlx5: use vector types to speed up processing
> net/mlx5: fix missing inline attributes
> net/mlx5: move static prototype
> net/mlx5: optimize copy of Ethernet header
> net/mlx5: remove inefficient prefetching
>
<...>
Series applied to dpdk-next-net/master, thanks.
^ permalink raw reply
* Re: [PATCH v2 1/5] eal: Set numa node value for system which not support NUMA.
From: Ferruh Yigit @ 2017-01-05 14:23 UTC (permalink / raw)
To: nickcooper-zhangtonghao, dev, Yong Wang
In-Reply-To: <1483617709-7088-1-git-send-email-nic@opencloud.tech>
On 1/5/2017 12:01 PM, nickcooper-zhangtonghao wrote:
> The NUMA node information for PCI devices provided through
> sysfs is invalid for AMD Opteron(TM) Processor 62xx and 63xx
> on Red Hat Enterprise Linux 6, and VMs on some hypervisors.
>
> Signed-off-by: nickcooper-zhangtonghao <nic@opencloud.tech>
Hi nickcooper-zhangtonghao,
The patches in the patchset are individual patches, right? Is there any
dependency between them?
And CC'ed vmxnet3 driver maintainer: Yong Wang <yongwang@vmware.com>
Thanks,
ferruh
> ---
> lib/librte_eal/linuxapp/eal/eal_pci.c | 8 +++++++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index 4350134..5dfdbe9 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -317,7 +317,13 @@
> free(dev);
> return -1;
> }
> - dev->device.numa_node = tmp;
> + /* The NUMA node information for PCI devices provided through
> + * sysfs is invalid for AMD Opteron(TM) Processor 62xx and 63xx
> + * on Red Hat Enterprise Linux 6, and VMs on some hypervisors.
> + * In the upstream linux kernel, the numa_node is an integer,
> + * which data type is int, not unsigned long.
> + */
> + dev->device.numa_node = (int)tmp > 0 ? (int)tmp : 0;
> }
>
> /* parse resources */
>
^ permalink raw reply
* [PATCH] net/af_packet: fix fd use after free
From: Timmons C. Player @ 2017-01-05 14:33 UTC (permalink / raw)
To: linville; +Cc: dev, Timmons C. Player
When using the same file descriptor for both rx and tx, the
eth_dev_stop function would close the same fd twice. This
change prevents that from happening.
Signed-off-by: Timmons C. Player <timmons.player@spirent.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 2951f86..c44b8b9 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -261,9 +261,16 @@ eth_dev_stop(struct rte_eth_dev *dev)
sockfd = internals->rx_queue[i].sockfd;
if (sockfd != -1)
close(sockfd);
- sockfd = internals->tx_queue[i].sockfd;
- if (sockfd != -1)
- close(sockfd);
+
+ /* Prevent use after free in case tx fd == rx fd */
+ if (sockfd != internals->tx_queue[i].sockfd) {
+ sockfd = internals->tx_queue[i].sockfd;
+ if (sockfd != -1)
+ close(sockfd);
+ }
+
+ internals->rx_queue[i].sockfd = -1;
+ internals->tx_queue[i].sockfd = -1;
}
dev->data->dev_link.link_status = ETH_LINK_DOWN;
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox