* Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
From: Tiwei Bie @ 2017-01-04 14:39 UTC (permalink / raw)
To: Ananyev, Konstantin
Cc: dev@dpdk.org, adrien.mazarguil@6wind.com, Lu, Wenzhuo,
Mcnamara, John, olivier.matz@6wind.com, thomas.monjalon@6wind.com,
Zhang, Helin, Dai, Wei, Wang, Xiao W
In-Reply-To: <2601191342CEEE43887BDE71AB9772583F0FEE0C@irsmsx105.ger.corp.intel.com>
On Wed, Jan 04, 2017 at 10:21:08PM +0800, Ananyev, Konstantin wrote:
> Hi Twei,
>
> > -----Original Message-----
> > From: Bie, Tiwei
> > Sent: Wednesday, January 4, 2017 7:22 AM
> > To: dev@dpdk.org
> > Cc: adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> > Subject: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> >
> > Reserve a Tx capability flag and a Rx capability flag, that can be
> > used by PMD to define its own capability flags when implementing the
> > PMD-specific API.
> >
> > Suggested-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
> > Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > ---
> > lib/librte_ether/rte_ethdev.h | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> > index d465825..8800b39 100644
> > --- a/lib/librte_ether/rte_ethdev.h
> > +++ b/lib/librte_ether/rte_ethdev.h
> > @@ -857,6 +857,7 @@ struct rte_eth_conf {
> > #define DEV_RX_OFFLOAD_TCP_LRO 0x00000010
> > #define DEV_RX_OFFLOAD_QINQ_STRIP 0x00000020
> > #define DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM 0x00000040
> > +#define DEV_RX_OFFLOAD_RESERVED_0 0x00000080 /**< Used for PMD-specific API. */
> >
> > /**
> > * TX offload capabilities of a device.
> > @@ -874,6 +875,7 @@ struct rte_eth_conf {
> > #define DEV_TX_OFFLOAD_GRE_TNL_TSO 0x00000400 /**< Used for tunneling packet. */
> > #define DEV_TX_OFFLOAD_IPIP_TNL_TSO 0x00000800 /**< Used for tunneling packet. */
> > #define DEV_TX_OFFLOAD_GENEVE_TNL_TSO 0x00001000 /**< Used for tunneling packet. */
> > +#define DEV_TX_OFFLOAD_RESERVED_0 0x00002000 /**< Used for PMD-specific API. */
> >
> > /**
> > * Ethernet device information
> > --
> > 2.7.4
>
> I am not sure how that supposed to work and how user should know that DEV_RX_OFFLOAD_RESERVED_0
> is actually a MACSEC for ixgbe?
Users are not supposed to use DEV_RX_OFFLOAD_RESERVED_0, instead, they
should use the capabilities and the likes defined in rte_pmd_ixgbe.h
where the PMD-specifics APIs are declared:
/**
* If these flags are advertised by the PMD, the NIC supports the MACsec
* offload. The incoming MACsec traffics can be offloaded transparently
* after the MACsec offload is configured correctly by the application.
* And the application can set the PKT_TX_IXGBE_MACSEC flag in mbufs to
* enable the MACsec offload for the packets to be transmitted.
*/
#define DEV_RX_OFFLOAD_IXGBE_MACSEC_STRIP DEV_RX_OFFLOAD_RESERVED_0
#define DEV_TX_OFFLOAD_IXGBE_MACSEC_INSERT DEV_TX_OFFLOAD_RESERVED_0
/**
* This event will occur when the PN counter in a MACsec connection
* reach the exhaustion threshold.
*/
#define RTE_ETH_EVENT_IXGBE_MACSEC RTE_ETH_EVENT_RESERVED_0
/**
* Offload the MACsec. This flag must be set by the application in mbuf
* to enable this offload feature for a packet to be transmitted.
*/
#define PKT_TX_IXGBE_MACSEC PKT_TX_RESERVED_0
PMD-specific APIs can only be used on the corresponding driver/device,
so different PMD can share the same reserved bit to represent different
things when implementing their own PMD-specific APIs.
> Another question what to do if you would like to create a bonded device over two devices with different NIC types?
> As I understand you can end up in situation when DEV_RX_OFFLOAD_RESERVED_0 would mean different capabilities.
> Why not to have this MACSEC capability and ol_flag value as generic ones, as you have them in previous versions of your patch?
Those flags are only used in PMD-specific APIs. I don't think we could
use the PMD-specific APIs provided by a certain PMD on a bonded device.
Thanks & regards,
Tiwei Bie
^ permalink raw reply
* Re: [PATCH] nfp: add support for new metadata api
From: Ferruh Yigit @ 2017-01-04 14:48 UTC (permalink / raw)
To: Alejandro Lucero; +Cc: dev
In-Reply-To: <CAD+H990+t61m3Tc6Q4xi-pHB-O0ezonnQG15Qt50hWhke9A2=w@mail.gmail.com>
On 1/4/2017 2:43 PM, Alejandro Lucero wrote:
> Hi Ferruh,
>
> On Wed, Jan 4, 2017 at 3:15 PM, Ferruh Yigit <ferruh.yigit@intel.com
> <mailto:ferruh.yigit@intel.com>> wrote:
>
> On 12/20/2016 2:13 PM, Alejandro Lucero wrote:
> > NFP is a smart programmable NIC and firmware is deployed for specific
> > system needs, like offloading OVS, vRouter, contrack or eBPF into the
> > hardware. This often requires to give metadata to the host within
> > packets delivered. Last NFP firmware implementations support richer
> > metadata api facilitating interaction between firmware and host code.
> >
> > Old way of handling metadata needs to be still there for supporting
> > old firmware.
> >
> > Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com <mailto:alejandro.lucero@netronome.com>>
> > ---
>
> <...>
>
> > +
> > + } else if (NFP_DESC_META_LEN(rxd)) {
> > + meta_offset = (uint8_t *)mbuf->buf_addr;
> > + meta_info = rte_be_to_cpu_32(*(uint32_t *)meta_offset);
> > + meta_offset += 4;
> > + /* NFP PMD just supports metadata for hashing */
> > + switch (meta_info & NFP_NET_META_FIELD_MASK) {
> > + case NFP_NET_META_HASH:
> > + meta_info >>= NFP_NET_META_FIELD_SIZE;
> > + hash = rte_be_to_cpu_32(*(uint32_t *)meta_offset);
> > + hash_type = meta_info && NFP_NET_META_FIELD_MASK;
>
> I already applied this patch but above "&&" looks wrong.
> Most probably intention is "bitwise AND" (&), do you want me fix this as
> "&" or remove the patch completely to replace with new version?
>
>
> Yes, that is wrong. I wonder how related tests did not fail. I'll check
> that right now.
>
> Maybe it is better to wait for another patch version or at least to be
> sure that simple change is good enough.
> Let me to peer into those tests and re-run them with that fix applied.
Removed from next-net, patchwork status updated as "Change Requested".
>
>
> Thanks,
> ferruh
>
>
^ permalink raw reply
* Re: [PATCH v5 0/6] net/mlx5: support flow API
From: Ferruh Yigit @ 2017-01-04 14:48 UTC (permalink / raw)
To: Nelio Laranjeiro, dev; +Cc: Adrien Mazarguil
In-Reply-To: <cover.1483022600.git.nelio.laranjeiro@6wind.com>
On 12/29/2016 3:15 PM, Nelio Laranjeiro wrote:
> Changes in v5:
>
> - Fix masking when only spec is present in item structure.
> - Fix first element of flow items array.
>
> Changes in v4:
>
> - Simplify flow parsing by using a graph.
> - Add VXLAN flow item.
> - Add mark flow action.
> - Extend IPv4 filter item (Type of service, Next Protocol ID).
>
> Changes in v3:
>
> - Fix Ethernet ether type issue.
>
> Changes in v2:
>
> - Fix several issues.
> - Support VLAN filtering.
>
> Nelio Laranjeiro (6):
> net/mlx5: add preliminary flow API support
> net/mlx5: support basic flow items and actions
> net/mlx5: support VLAN flow item
> net/mlx5: support VXLAN flow item
> net/mlx5: support mark flow action
> net/mlx5: extend IPv4 flow item
>
Series applied to dpdk-next-net/master, thanks.
^ permalink raw reply
* Re: [PATCH v4] ethtool: dispaly bus information
From: Mcnamara, John @ 2017-01-04 14:49 UTC (permalink / raw)
To: Yang, Qiming, dev@dpdk.org
Cc: Yigit, Ferruh, Zhang, Helin, Horton, Remy, Yang, Qiming
In-Reply-To: <1483532295-15460-1-git-send-email-qiming.yang@intel.com>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Qiming Yang
> Sent: Wednesday, January 4, 2017 12:18 PM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; Horton, Remy <remy.horton@intel.com>; Yang,
> Qiming <qiming.yang@intel.com>
> Subject: [dpdk-dev] [PATCH v4] ethtool: dispaly bus information
Typo: s/dispaly/display in this and other patches.
^ permalink raw reply
* Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
From: Ananyev, Konstantin @ 2017-01-04 15:14 UTC (permalink / raw)
To: Bie, Tiwei
Cc: dev@dpdk.org, adrien.mazarguil@6wind.com, Lu, Wenzhuo,
Mcnamara, John, olivier.matz@6wind.com, thomas.monjalon@6wind.com,
Zhang, Helin, Dai, Wei, Wang, Xiao W
In-Reply-To: <20170104143923.GA57552@dpdk19>
> -----Original Message-----
> From: Bie, Tiwei
> Sent: Wednesday, January 4, 2017 2:39 PM
> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Cc: dev@dpdk.org; adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> olivier.matz@6wind.com; thomas.monjalon@6wind.com; Zhang, Helin <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang,
> Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
>
> On Wed, Jan 04, 2017 at 10:21:08PM +0800, Ananyev, Konstantin wrote:
> > Hi Twei,
> >
> > > -----Original Message-----
> > > From: Bie, Tiwei
> > > Sent: Wednesday, January 4, 2017 7:22 AM
> > > To: dev@dpdk.org
> > > Cc: adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > > <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> > > Subject: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> > >
> > > Reserve a Tx capability flag and a Rx capability flag, that can be
> > > used by PMD to define its own capability flags when implementing the
> > > PMD-specific API.
> > >
> > > Suggested-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > > Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
> > > Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > ---
> > > lib/librte_ether/rte_ethdev.h | 2 ++
> > > 1 file changed, 2 insertions(+)
> > >
> > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> > > index d465825..8800b39 100644
> > > --- a/lib/librte_ether/rte_ethdev.h
> > > +++ b/lib/librte_ether/rte_ethdev.h
> > > @@ -857,6 +857,7 @@ struct rte_eth_conf {
> > > #define DEV_RX_OFFLOAD_TCP_LRO 0x00000010
> > > #define DEV_RX_OFFLOAD_QINQ_STRIP 0x00000020
> > > #define DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM 0x00000040
> > > +#define DEV_RX_OFFLOAD_RESERVED_0 0x00000080 /**< Used for PMD-specific API. */
> > >
> > > /**
> > > * TX offload capabilities of a device.
> > > @@ -874,6 +875,7 @@ struct rte_eth_conf {
> > > #define DEV_TX_OFFLOAD_GRE_TNL_TSO 0x00000400 /**< Used for tunneling packet. */
> > > #define DEV_TX_OFFLOAD_IPIP_TNL_TSO 0x00000800 /**< Used for tunneling packet. */
> > > #define DEV_TX_OFFLOAD_GENEVE_TNL_TSO 0x00001000 /**< Used for tunneling packet. */
> > > +#define DEV_TX_OFFLOAD_RESERVED_0 0x00002000 /**< Used for PMD-specific API. */
> > >
> > > /**
> > > * Ethernet device information
> > > --
> > > 2.7.4
> >
> > I am not sure how that supposed to work and how user should know that DEV_RX_OFFLOAD_RESERVED_0
> > is actually a MACSEC for ixgbe?
>
> Users are not supposed to use DEV_RX_OFFLOAD_RESERVED_0, instead, they
> should use the capabilities and the likes defined in rte_pmd_ixgbe.h
> where the PMD-specifics APIs are declared:
>
> /**
> * If these flags are advertised by the PMD, the NIC supports the MACsec
> * offload. The incoming MACsec traffics can be offloaded transparently
> * after the MACsec offload is configured correctly by the application.
> * And the application can set the PKT_TX_IXGBE_MACSEC flag in mbufs to
> * enable the MACsec offload for the packets to be transmitted.
> */
> #define DEV_RX_OFFLOAD_IXGBE_MACSEC_STRIP DEV_RX_OFFLOAD_RESERVED_0
> #define DEV_TX_OFFLOAD_IXGBE_MACSEC_INSERT DEV_TX_OFFLOAD_RESERVED_0
>
> /**
> * This event will occur when the PN counter in a MACsec connection
> * reach the exhaustion threshold.
> */
> #define RTE_ETH_EVENT_IXGBE_MACSEC RTE_ETH_EVENT_RESERVED_0
>
> /**
> * Offload the MACsec. This flag must be set by the application in mbuf
> * to enable this offload feature for a packet to be transmitted.
> */
> #define PKT_TX_IXGBE_MACSEC PKT_TX_RESERVED_0
>
> PMD-specific APIs can only be used on the corresponding driver/device,
> so different PMD can share the same reserved bit to represent different
> things when implementing their own PMD-specific APIs.
Ok, and why do we need it?
Why we can't just have PKT_TX_MACSEC straightway?
What are you trying to gain here?
Is it just for future opportunity to save an extra bit in mbuf.ol_flags?
I don't think we are short of bits here right now, and we don't consume them exra-fast
to start to worry about it.
Again, if it *really* would be for ixgbe only forever, and y don't want to waste a bit in tx_olflags,
why not to introduce device specific ol_flags in mbuf second cache line?
Probably uint16_t would be enough for that.
>
> > Another question what to do if you would like to create a bonded device over two devices with different NIC types?
> > As I understand you can end up in situation when DEV_RX_OFFLOAD_RESERVED_0 would mean different capabilities.
> > Why not to have this MACSEC capability and ol_flag value as generic ones, as you have them in previous versions of your patch?
>
> Those flags are only used in PMD-specific APIs. I don't think we could
> use the PMD-specific APIs provided by a certain PMD on a bonded device.
I understand that.
My question was: suppose user would like to create a bonded device over 2 NICs.
One of them is ixgbe, while other would be some other type.
In future get_dev_info() for each of them might return DEV_RX_OFFLOAD_RESERVED_0 bit as set.
But it would mean completely different thing.
How bonded device would know that to deal properly?
Another example - user has 2 NICs of different type and would like to send the same packet on both of them simultaneously.
As PKT_TX_RESERVED might mean different things for these devices, and user would like to use let say
PKT_TX_IXGBE_MACSEC on one of them, he would need to do a copy of them, instead just increment a refcnt.
Similar issues might arise at RX handling: user got a packet with PKT_RX_RESERVED_0 set.
What does it really mean if there are different NIC types in the system?
The only way to answer that question, as I can see, is to keep track from what NIC that packet was received.
Which I suppose, is not always convenient.
Konstantin
^ permalink raw reply
* Re: [PATCH v2 23/29] net/i40e: use eal I/O device memory read/write API
From: Santosh Shukla @ 2017-01-04 15:22 UTC (permalink / raw)
To: Tiwei Bie
Cc: Jerin Jacob, dev, konstantin.ananyev, thomas.monjalon,
bruce.richardson, jianbo.liu, viktorin, Helin Zhang, Jingjing Wu,
Satha Rao
In-Reply-To: <20170104135339.GA56511@dpdk19>
On Wed, Jan 04, 2017 at 09:53:40PM +0800, Tiwei Bie wrote:
> On Tue, Dec 27, 2016 at 03:19:29PM +0530, Jerin Jacob wrote:
> > From: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> > /* Update the tx tail register */
> > rte_wmb();
> > - I40E_PCI_REG_WRITE(txq->qtx_tail, txq->tx_tail);
> > + I40E_PCI_REG_WRITE_RELAXED(txq->qtx_tail, txq->tx_tail);
> >
> > return nb_pkts;
> > }
>
> Besides i40e_xmit_pkts() and tx_xmit_pkts(), i40e_rx_alloc_bufs() which is
> called by rx_recv_pkts() is also in the fast path. So I40E_PCI_REG_WRITE()
> called by it should also be replaced by the relaxed version:
>
> diff --git i/drivers/net/i40e/i40e_rxtx.c w/drivers/net/i40e/i40e_rxtx.c
> index 7ae7d9f..55a707a 100644
> --- i/drivers/net/i40e/i40e_rxtx.c
> +++ w/drivers/net/i40e/i40e_rxtx.c
> @@ -581,7 +581,7 @@ i40e_rx_alloc_bufs(struct i40e_rx_queue *rxq)
>
> /* Update rx tail regsiter */
> rte_wmb();
> - I40E_PCI_REG_WRITE(rxq->qrx_tail, rxq->rx_free_trigger);
> + I40E_PCI_REG_WRITE_RELAXED(rxq->qrx_tail, rxq->rx_free_trigger);
>
> rxq->rx_free_trigger =
> (uint16_t)(rxq->rx_free_trigger + rxq->rx_free_thresh);
>
Yes.
Will queue it in v3.
> > --
> > 2.5.5
> >
^ permalink raw reply
* Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
From: Tiwei Bie @ 2017-01-04 17:00 UTC (permalink / raw)
To: Ananyev, Konstantin
Cc: dev@dpdk.org, adrien.mazarguil@6wind.com, Lu, Wenzhuo,
Mcnamara, John, olivier.matz@6wind.com, thomas.monjalon@6wind.com,
Zhang, Helin, Dai, Wei, Wang, Xiao W
In-Reply-To: <2601191342CEEE43887BDE71AB9772583F0FEE6D@irsmsx105.ger.corp.intel.com>
On Wed, Jan 04, 2017 at 11:14:25PM +0800, Ananyev, Konstantin wrote:
>
>
> > -----Original Message-----
> > From: Bie, Tiwei
> > Sent: Wednesday, January 4, 2017 2:39 PM
> > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > Cc: dev@dpdk.org; adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Zhang, Helin <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang,
> > Xiao W <xiao.w.wang@intel.com>
> > Subject: Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> >
> > On Wed, Jan 04, 2017 at 10:21:08PM +0800, Ananyev, Konstantin wrote:
> > > Hi Twei,
> > >
> > > > -----Original Message-----
> > > > From: Bie, Tiwei
> > > > Sent: Wednesday, January 4, 2017 7:22 AM
> > > > To: dev@dpdk.org
> > > > Cc: adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > > > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > > > <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> > > > Subject: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> > > >
> > > > Reserve a Tx capability flag and a Rx capability flag, that can be
> > > > used by PMD to define its own capability flags when implementing the
> > > > PMD-specific API.
> > > >
> > > > Suggested-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > > > Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
> > > > Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > > ---
> > > > lib/librte_ether/rte_ethdev.h | 2 ++
> > > > 1 file changed, 2 insertions(+)
> > > >
> > > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> > > > index d465825..8800b39 100644
> > > > --- a/lib/librte_ether/rte_ethdev.h
> > > > +++ b/lib/librte_ether/rte_ethdev.h
> > > > @@ -857,6 +857,7 @@ struct rte_eth_conf {
> > > > #define DEV_RX_OFFLOAD_TCP_LRO 0x00000010
> > > > #define DEV_RX_OFFLOAD_QINQ_STRIP 0x00000020
> > > > #define DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM 0x00000040
> > > > +#define DEV_RX_OFFLOAD_RESERVED_0 0x00000080 /**< Used for PMD-specific API. */
> > > >
> > > > /**
> > > > * TX offload capabilities of a device.
> > > > @@ -874,6 +875,7 @@ struct rte_eth_conf {
> > > > #define DEV_TX_OFFLOAD_GRE_TNL_TSO 0x00000400 /**< Used for tunneling packet. */
> > > > #define DEV_TX_OFFLOAD_IPIP_TNL_TSO 0x00000800 /**< Used for tunneling packet. */
> > > > #define DEV_TX_OFFLOAD_GENEVE_TNL_TSO 0x00001000 /**< Used for tunneling packet. */
> > > > +#define DEV_TX_OFFLOAD_RESERVED_0 0x00002000 /**< Used for PMD-specific API. */
> > > >
> > > > /**
> > > > * Ethernet device information
> > > > --
> > > > 2.7.4
> > >
> > > I am not sure how that supposed to work and how user should know that DEV_RX_OFFLOAD_RESERVED_0
> > > is actually a MACSEC for ixgbe?
> >
> > Users are not supposed to use DEV_RX_OFFLOAD_RESERVED_0, instead, they
> > should use the capabilities and the likes defined in rte_pmd_ixgbe.h
> > where the PMD-specifics APIs are declared:
> >
> > /**
> > * If these flags are advertised by the PMD, the NIC supports the MACsec
> > * offload. The incoming MACsec traffics can be offloaded transparently
> > * after the MACsec offload is configured correctly by the application.
> > * And the application can set the PKT_TX_IXGBE_MACSEC flag in mbufs to
> > * enable the MACsec offload for the packets to be transmitted.
> > */
> > #define DEV_RX_OFFLOAD_IXGBE_MACSEC_STRIP DEV_RX_OFFLOAD_RESERVED_0
> > #define DEV_TX_OFFLOAD_IXGBE_MACSEC_INSERT DEV_TX_OFFLOAD_RESERVED_0
> >
> > /**
> > * This event will occur when the PN counter in a MACsec connection
> > * reach the exhaustion threshold.
> > */
> > #define RTE_ETH_EVENT_IXGBE_MACSEC RTE_ETH_EVENT_RESERVED_0
> >
> > /**
> > * Offload the MACsec. This flag must be set by the application in mbuf
> > * to enable this offload feature for a packet to be transmitted.
> > */
> > #define PKT_TX_IXGBE_MACSEC PKT_TX_RESERVED_0
> >
> > PMD-specific APIs can only be used on the corresponding driver/device,
> > so different PMD can share the same reserved bit to represent different
> > things when implementing their own PMD-specific APIs.
>
> Ok, and why do we need it?
> Why we can't just have PKT_TX_MACSEC straightway?
> What are you trying to gain here?
> Is it just for future opportunity to save an extra bit in mbuf.ol_flags?
> I don't think we are short of bits here right now, and we don't consume them exra-fast
> to start to worry about it.
> Again, if it *really* would be for ixgbe only forever, and y don't want to waste a bit in tx_olflags,
> why not to introduce device specific ol_flags in mbuf second cache line?
> Probably uint16_t would be enough for that.
>
> >
> > > Another question what to do if you would like to create a bonded device over two devices with different NIC types?
> > > As I understand you can end up in situation when DEV_RX_OFFLOAD_RESERVED_0 would mean different capabilities.
> > > Why not to have this MACSEC capability and ol_flag value as generic ones, as you have them in previous versions of your patch?
> >
> > Those flags are only used in PMD-specific APIs. I don't think we could
> > use the PMD-specific APIs provided by a certain PMD on a bonded device.
>
> I understand that.
> My question was: suppose user would like to create a bonded device over 2 NICs.
> One of them is ixgbe, while other would be some other type.
> In future get_dev_info() for each of them might return DEV_RX_OFFLOAD_RESERVED_0 bit as set.
> But it would mean completely different thing.
> How bonded device would know that to deal properly?
>
> Another example - user has 2 NICs of different type and would like to send the same packet on both of them simultaneously.
> As PKT_TX_RESERVED might mean different things for these devices, and user would like to use let say
> PKT_TX_IXGBE_MACSEC on one of them, he would need to do a copy of them, instead just increment a refcnt.
>
> Similar issues might arise at RX handling: user got a packet with PKT_RX_RESERVED_0 set.
> What does it really mean if there are different NIC types in the system?
> The only way to answer that question, as I can see, is to keep track from what NIC that packet was received.
> Which I suppose, is not always convenient.
>
The main purpose is to put the PMD-specific APIs in a separate
namespace instead of mixing the PMD-specific APIs and global APIs
up, and also save the bits in mbuf.ol_flags.
There are other ways to achieve this goal, such as introducing
the PMD specific ol_flags in mbuf second cache line as you said.
I just thought defining some reserved bits seems to be the most
simple way which won't introduce many changes.
What's your suggestions? Should I just revert the changes and
define the generic flags directly?
Thanks & regards,
Tiwei Bie
^ permalink raw reply
* [PATCH v3 0/8] Add crypto PMD optimized for ARMv8
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1481077985-4224-2-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Introduce crypto poll mode driver using ARMv8
cryptographic extensions. This PMD is optimized
to provide performance boost for chained
crypto operations processing, such as:
* encryption + HMAC generation
* decryption + HMAC validation.
In particular, cipher only or hash only
operations are not provided.
Performance gain can be observed in tests
against OpenSSL PMD which also uses ARM
crypto extensions for packets processing.
Exemplary crypto performance tests comparison:
cipher_hash. cipher algo: AES_CBC
auth algo: SHA1_HMAC cipher key size=16.
burst_size: 64 ops
ARMv8 PMD improvement over OpenSSL PMD
(Optimized for ARMv8 cipher only and hash
only cases):
Buffer
Size(B) OPS(M) Throughput(Gbps)
64 729 % 742 %
128 577 % 592 %
256 483 % 476 %
512 336 % 351 %
768 300 % 286 %
1024 263 % 250 %
1280 225 % 229 %
1536 214 % 213 %
1792 186 % 203 %
2048 200 % 193 %
The driver currently supports AES-128-CBC
in combination with: SHA256 HMAC and SHA1 HMAC.
The core crypto functionality of this driver is
provided by the external armv8_crypto library
that can be downloaded from the Cavium repository:
https://github.com/caviumnetworks/armv8_crypto
CPU compatibility with this virtual device
is detected in run-time and virtual crypto
device will not be created if CPU doesn't
provide AES, SHA1, SHA2 and NEON.
The functionality and performance of this
code can be tested using generic test application
with the following commands:
* cryptodev_sw_armv8_autotest
* cryptodev_sw_armv8_perftest
New test vectors and cases have been added
to the general pool. In particular SHA1 and
SHA256 HMAC for short cases were introduced.
This is because low-level ARM assembly code
is using different code paths for long and
short data sets, so in order to test the
mentioned driver correctly, two different
data sets need to be provided.
---
v3:
* Addressed review remarks
* Moved low-level assembly code to the external library
* Removed SHA256 MAC cases
* Various fixes: interface to the library, digest destination
and source address interpreting, missing mbuf manipulations.
v2:
* Fixed checkpatch warnings
* Divide patches into smaller logical parts
Zbigniew Bodek (8):
mk: fix build of assembly files for ARM64
lib: add cryptodev type for the upcoming ARMv8 PMD
crypto/armv8: add PMD optimized for ARMv8 processors
mk/crypto/armv8: add PMD to the build system
doc/armv8: update documentation about crypto PMD
crypto/armv8: enable ARMv8 PMD in the configuration
crypto/armv8: update MAINTAINERS entry for ARMv8 crypto
app/test: add ARMv8 crypto tests and test vectors
MAINTAINERS | 6 +
app/test/test_cryptodev.c | 63 ++
app/test/test_cryptodev_aes_test_vectors.h | 144 +++-
app/test/test_cryptodev_blockcipher.c | 4 +
app/test/test_cryptodev_blockcipher.h | 1 +
app/test/test_cryptodev_perf.c | 480 +++++++++++++
config/common_base | 6 +
doc/guides/cryptodevs/armv8.rst | 96 +++
doc/guides/cryptodevs/index.rst | 1 +
doc/guides/rel_notes/release_17_02.rst | 5 +
drivers/crypto/Makefile | 1 +
drivers/crypto/armv8/Makefile | 73 ++
drivers/crypto/armv8/rte_armv8_pmd.c | 926 +++++++++++++++++++++++++
drivers/crypto/armv8/rte_armv8_pmd_ops.c | 369 ++++++++++
drivers/crypto/armv8/rte_armv8_pmd_private.h | 211 ++++++
drivers/crypto/armv8/rte_armv8_pmd_version.map | 3 +
lib/librte_cryptodev/rte_cryptodev.h | 3 +
mk/arch/arm64/rte.vars.mk | 1 -
mk/rte.app.mk | 2 +
mk/toolchain/gcc/rte.vars.mk | 6 +-
20 files changed, 2390 insertions(+), 11 deletions(-)
create mode 100644 doc/guides/cryptodevs/armv8.rst
create mode 100644 drivers/crypto/armv8/Makefile
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd.c
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd_ops.c
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd_private.h
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd_version.map
--
1.9.1
^ permalink raw reply
* [PATCH v3 1/8] mk: fix build of assembly files for ARM64
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Avoid using incorrect assembler (nasm) and unsupported flags
when building for ARM64.
Fixes: af75078fece3 ("first public release")
b3ce00e5fe36 ("mk: introduce ARMv8 architecture")
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
mk/arch/arm64/rte.vars.mk | 1 -
mk/toolchain/gcc/rte.vars.mk | 6 ++++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/mk/arch/arm64/rte.vars.mk b/mk/arch/arm64/rte.vars.mk
index c168426..3b1178a 100644
--- a/mk/arch/arm64/rte.vars.mk
+++ b/mk/arch/arm64/rte.vars.mk
@@ -53,7 +53,6 @@ CROSS ?=
CPU_CFLAGS ?=
CPU_LDFLAGS ?=
-CPU_ASFLAGS ?= -felf
export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS
diff --git a/mk/toolchain/gcc/rte.vars.mk b/mk/toolchain/gcc/rte.vars.mk
index ff70f3d..94f6412 100644
--- a/mk/toolchain/gcc/rte.vars.mk
+++ b/mk/toolchain/gcc/rte.vars.mk
@@ -41,9 +41,11 @@
CC = $(CROSS)gcc
KERNELCC = $(CROSS)gcc
CPP = $(CROSS)cpp
-# for now, we don't use as but nasm.
-# AS = $(CROSS)as
+ifeq ($(CONFIG_RTE_ARCH_X86),y)
AS = nasm
+else
+AS = $(CROSS)as
+endif
AR = $(CROSS)ar
LD = $(CROSS)ld
OBJCOPY = $(CROSS)objcopy
--
1.9.1
^ permalink raw reply related
* [PATCH v3 2/8] lib: add cryptodev type for the upcoming ARMv8 PMD
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Add type and name for ARMv8 crypto PMD
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
lib/librte_cryptodev/rte_cryptodev.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 8f63e8f..6f34f22 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -66,6 +66,8 @@
/**< KASUMI PMD device name */
#define CRYPTODEV_NAME_ZUC_PMD crypto_zuc
/**< KASUMI PMD device name */
+#define CRYPTODEV_NAME_ARMV8_PMD crypto_armv8
+/**< ARMv8 Crypto PMD device name */
/** Crypto device type */
enum rte_cryptodev_type {
@@ -77,6 +79,7 @@ enum rte_cryptodev_type {
RTE_CRYPTODEV_KASUMI_PMD, /**< KASUMI PMD */
RTE_CRYPTODEV_ZUC_PMD, /**< ZUC PMD */
RTE_CRYPTODEV_OPENSSL_PMD, /**< OpenSSL PMD */
+ RTE_CRYPTODEV_ARMV8_PMD, /**< ARMv8 crypto PMD */
};
extern const char **rte_cyptodev_names;
--
1.9.1
^ permalink raw reply related
* [PATCH v3 3/8] crypto/armv8: add PMD optimized for ARMv8 processors
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
This patch introduces crypto poll mode driver
using ARMv8 cryptographic extensions.
CPU compatibility with this driver is detected in
run-time and virtual crypto device will not be
created if CPU doesn't provide:
AES, SHA1, SHA2 and NEON.
This PMD is optimized to provide performance boost
for chained crypto operations processing,
such as encryption + HMAC generation,
decryption + HMAC validation. In particular,
cipher only or hash only operations are
not provided.
The driver currently supports AES-128-CBC
in combination with: SHA256 HMAC and SHA1 HMAC
and relies on the external armv8_crypto library:
https://github.com/caviumnetworks/armv8_crypto
This patch adds driver's code only and does
not include it in the build system.
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
drivers/crypto/armv8/Makefile | 73 ++
drivers/crypto/armv8/rte_armv8_pmd.c | 926 +++++++++++++++++++++++++
drivers/crypto/armv8/rte_armv8_pmd_ops.c | 369 ++++++++++
drivers/crypto/armv8/rte_armv8_pmd_private.h | 211 ++++++
drivers/crypto/armv8/rte_armv8_pmd_version.map | 3 +
5 files changed, 1582 insertions(+)
create mode 100644 drivers/crypto/armv8/Makefile
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd.c
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd_ops.c
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd_private.h
create mode 100644 drivers/crypto/armv8/rte_armv8_pmd_version.map
diff --git a/drivers/crypto/armv8/Makefile b/drivers/crypto/armv8/Makefile
new file mode 100644
index 0000000..dc5ea02
--- /dev/null
+++ b/drivers/crypto/armv8/Makefile
@@ -0,0 +1,73 @@
+#
+# BSD LICENSE
+#
+# Copyright (C) Cavium networks Ltd. 2017.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Cavium networks nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(MAKECMDGOALS),clean)
+ifneq ($(MAKECMDGOALS),config)
+ifeq ($(ARMV8_CRYPTO_LIB_PATH),)
+$(error "Please define ARMV8_CRYPTO_LIB_PATH environment variable")
+endif
+endif
+endif
+
+# library name
+LIB = librte_pmd_armv8.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -L$(RTE_SDK)/../openssl -I$(RTE_SDK)/../openssl/include
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_armv8_pmd_version.map
+
+# external library dependencies
+CFLAGS += -I$(ARMV8_CRYPTO_LIB_PATH)
+CFLAGS += -I$(ARMV8_CRYPTO_LIB_PATH)/asm/include
+LDLIBS += -L$(ARMV8_CRYPTO_LIB_PATH) -larmv8_crypto
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += rte_armv8_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += rte_armv8_pmd_ops.c
+
+# library dependencies
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += lib/librte_eal
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += lib/librte_mbuf
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += lib/librte_mempool
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += lib/librte_ring
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += lib/librte_cryptodev
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/crypto/armv8/rte_armv8_pmd.c b/drivers/crypto/armv8/rte_armv8_pmd.c
new file mode 100644
index 0000000..39433bb
--- /dev/null
+++ b/drivers/crypto/armv8/rte_armv8_pmd.c
@@ -0,0 +1,926 @@
+/*
+ * BSD LICENSE
+ *
+ * Copyright (C) Cavium networks Ltd. 2017.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Cavium networks nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdbool.h>
+
+#include <rte_common.h>
+#include <rte_hexdump.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_vdev.h>
+#include <rte_malloc.h>
+#include <rte_cpuflags.h>
+
+#include "armv8_crypto_defs.h"
+
+#include "rte_armv8_pmd_private.h"
+
+static int cryptodev_armv8_crypto_uninit(const char *name);
+
+/**
+ * Pointers to the supported combined mode crypto functions are stored
+ * in the static tables. Each combined (chained) cryptographic operation
+ * can be decribed by a set of numbers:
+ * - order: order of operations (cipher, auth) or (auth, cipher)
+ * - direction: encryption or decryption
+ * - calg: cipher algorithm such as AES_CBC, AES_CTR, etc.
+ * - aalg: authentication algorithm such as SHA1, SHA256, etc.
+ * - keyl: cipher key length, for example 128, 192, 256 bits
+ *
+ * In order to quickly acquire each function pointer based on those numbers,
+ * a hierarchy of arrays is maintained. The final level, 3D array is indexed
+ * by the combined mode function parameters only (cipher algorithm,
+ * authentication algorithm and key length).
+ *
+ * This gives 3 memory accesses to obtain a function pointer instead of
+ * traversing the array manually and comparing function parameters on each loop.
+ *
+ * +--+CRYPTO_FUNC
+ * +--+ENC|
+ * +--+CA|
+ * | +--+DEC
+ * ORDER|
+ * | +--+ENC
+ * +--+AC|
+ * +--+DEC
+ *
+ */
+
+/**
+ * 3D array type for ARM Combined Mode crypto functions pointers.
+ * CRYPTO_CIPHER_MAX: max cipher ID number
+ * CRYPTO_AUTH_MAX: max auth ID number
+ * CRYPTO_CIPHER_KEYLEN_MAX: max key length ID number
+ */
+typedef const crypto_func_t
+crypto_func_tbl_t[CRYPTO_CIPHER_MAX][CRYPTO_AUTH_MAX][CRYPTO_CIPHER_KEYLEN_MAX];
+
+/* Evaluate to key length definition */
+#define KEYL(keyl) (ARMV8_CRYPTO_CIPHER_KEYLEN_ ## keyl)
+
+/* Local aliases for supported ciphers */
+#define CIPH_AES_CBC RTE_CRYPTO_CIPHER_AES_CBC
+/* Local aliases for supported hashes */
+#define AUTH_SHA1_HMAC RTE_CRYPTO_AUTH_SHA1_HMAC
+#define AUTH_SHA256 RTE_CRYPTO_AUTH_SHA256
+#define AUTH_SHA256_HMAC RTE_CRYPTO_AUTH_SHA256_HMAC
+
+/**
+ * Arrays containing pointers to particular cryptographic,
+ * combined mode functions.
+ * crypto_op_ca_encrypt: cipher (encrypt), authenticate
+ * crypto_op_ca_decrypt: cipher (decrypt), authenticate
+ * crypto_op_ac_encrypt: authenticate, cipher (encrypt)
+ * crypto_op_ac_decrypt: authenticate, cipher (decrypt)
+ */
+static const crypto_func_tbl_t
+crypto_op_ca_encrypt = {
+ /* [cipher alg][auth alg][key length] = crypto_function, */
+ [CIPH_AES_CBC][AUTH_SHA1_HMAC][KEYL(128)] = aes128cbc_sha1_hmac,
+ [CIPH_AES_CBC][AUTH_SHA256_HMAC][KEYL(128)] = aes128cbc_sha256_hmac,
+};
+
+static const crypto_func_tbl_t
+crypto_op_ca_decrypt = {
+ NULL
+};
+
+static const crypto_func_tbl_t
+crypto_op_ac_encrypt = {
+ NULL
+};
+
+static const crypto_func_tbl_t
+crypto_op_ac_decrypt = {
+ /* [cipher alg][auth alg][key length] = crypto_function, */
+ [CIPH_AES_CBC][AUTH_SHA1_HMAC][KEYL(128)] = sha1_hmac_aes128cbc_dec,
+ [CIPH_AES_CBC][AUTH_SHA256_HMAC][KEYL(128)] = sha256_hmac_aes128cbc_dec,
+};
+
+/**
+ * Arrays containing pointers to particular cryptographic function sets,
+ * covering given cipher operation directions (encrypt, decrypt)
+ * for each order of cipher and authentication pairs.
+ */
+static const crypto_func_tbl_t *
+crypto_cipher_auth[] = {
+ &crypto_op_ca_encrypt,
+ &crypto_op_ca_decrypt,
+ NULL
+};
+
+static const crypto_func_tbl_t *
+crypto_auth_cipher[] = {
+ &crypto_op_ac_encrypt,
+ &crypto_op_ac_decrypt,
+ NULL
+};
+
+/**
+ * Top level array containing pointers to particular cryptographic
+ * function sets, covering given order of chained operations.
+ * crypto_cipher_auth: cipher first, authenticate after
+ * crypto_auth_cipher: authenticate first, cipher after
+ */
+static const crypto_func_tbl_t **
+crypto_chain_order[] = {
+ crypto_cipher_auth,
+ crypto_auth_cipher,
+ NULL
+};
+
+/**
+ * Extract particular combined mode crypto function from the 3D array.
+ */
+#define CRYPTO_GET_ALGO(order, cop, calg, aalg, keyl) \
+({ \
+ crypto_func_tbl_t *func_tbl = \
+ (crypto_chain_order[(order)])[(cop)]; \
+ \
+ ((*func_tbl)[(calg)][(aalg)][KEYL(keyl)]); \
+})
+
+/*----------------------------------------------------------------------------*/
+
+/**
+ * 2D array type for ARM key schedule functions pointers.
+ * CRYPTO_CIPHER_MAX: max cipher ID number
+ * CRYPTO_CIPHER_KEYLEN_MAX: max key length ID number
+ */
+typedef const crypto_key_sched_t
+crypto_key_sched_tbl_t[CRYPTO_CIPHER_MAX][CRYPTO_CIPHER_KEYLEN_MAX];
+
+static const crypto_key_sched_tbl_t
+crypto_key_sched_encrypt = {
+ /* [cipher alg][key length] = key_expand_func, */
+ [CIPH_AES_CBC][KEYL(128)] = aes128_key_sched_enc,
+};
+
+static const crypto_key_sched_tbl_t
+crypto_key_sched_decrypt = {
+ /* [cipher alg][key length] = key_expand_func, */
+ [CIPH_AES_CBC][KEYL(128)] = aes128_key_sched_dec,
+};
+
+/**
+ * Top level array containing pointers to particular key generation
+ * function sets, covering given operation direction.
+ * crypto_key_sched_encrypt: keys for encryption
+ * crypto_key_sched_decrypt: keys for decryption
+ */
+static const crypto_key_sched_tbl_t *
+crypto_key_sched_dir[] = {
+ &crypto_key_sched_encrypt,
+ &crypto_key_sched_decrypt,
+ NULL
+};
+
+/**
+ * Extract particular combined mode crypto function from the 3D array.
+ */
+#define CRYPTO_GET_KEY_SCHED(cop, calg, keyl) \
+({ \
+ crypto_key_sched_tbl_t *ks_tbl = crypto_key_sched_dir[(cop)]; \
+ \
+ ((*ks_tbl)[(calg)][KEYL(keyl)]); \
+})
+
+/*----------------------------------------------------------------------------*/
+
+/**
+ * Global static parameter used to create a unique name for each
+ * ARMV8 crypto device.
+ */
+static unsigned int unique_name_id;
+
+static inline int
+create_unique_device_name(char *name, size_t size)
+{
+ int ret;
+
+ if (name == NULL)
+ return -EINVAL;
+
+ ret = snprintf(name, size, "%s_%u", RTE_STR(CRYPTODEV_NAME_ARMV8_PMD),
+ unique_name_id++);
+ if (ret < 0)
+ return ret;
+ return 0;
+}
+
+/*
+ *------------------------------------------------------------------------------
+ * Session Prepare
+ *------------------------------------------------------------------------------
+ */
+
+/** Get xform chain order */
+static enum armv8_crypto_chain_order
+armv8_crypto_get_chain_order(const struct rte_crypto_sym_xform *xform)
+{
+
+ /*
+ * This driver currently covers only chained operations.
+ * Ignore only cipher or only authentication operations
+ * or chains longer than 2 xform structures.
+ */
+ if (xform->next == NULL || xform->next->next != NULL)
+ return ARMV8_CRYPTO_CHAIN_NOT_SUPPORTED;
+
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
+ if (xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER)
+ return ARMV8_CRYPTO_CHAIN_AUTH_CIPHER;
+ }
+
+ if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) {
+ if (xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH)
+ return ARMV8_CRYPTO_CHAIN_CIPHER_AUTH;
+ }
+
+ return ARMV8_CRYPTO_CHAIN_NOT_SUPPORTED;
+}
+
+static inline void
+auth_hmac_pad_prepare(struct armv8_crypto_session *sess,
+ const struct rte_crypto_sym_xform *xform)
+{
+ size_t i;
+
+ /* Generate i_key_pad and o_key_pad */
+ memset(sess->auth.hmac.i_key_pad, 0, sizeof(sess->auth.hmac.i_key_pad));
+ rte_memcpy(sess->auth.hmac.i_key_pad, sess->auth.hmac.key,
+ xform->auth.key.length);
+ memset(sess->auth.hmac.o_key_pad, 0, sizeof(sess->auth.hmac.o_key_pad));
+ rte_memcpy(sess->auth.hmac.o_key_pad, sess->auth.hmac.key,
+ xform->auth.key.length);
+ /*
+ * XOR key with IPAD/OPAD values to obtain i_key_pad
+ * and o_key_pad.
+ * Byte-by-byte operation may seem to be the less efficient
+ * here but in fact it's the opposite.
+ * The result ASM code is likely operate on NEON registers
+ * (load auth key to Qx, load IPAD/OPAD to multiple
+ * elements of Qy, eor 128 bits at once).
+ */
+ for (i = 0; i < SHA_BLOCK_MAX; i++) {
+ sess->auth.hmac.i_key_pad[i] ^= HMAC_IPAD_VALUE;
+ sess->auth.hmac.o_key_pad[i] ^= HMAC_OPAD_VALUE;
+ }
+}
+
+static inline int
+auth_set_prerequisites(struct armv8_crypto_session *sess,
+ const struct rte_crypto_sym_xform *xform)
+{
+ uint8_t partial[64] = { 0 };
+ int error;
+
+ switch (xform->auth.algo) {
+ case RTE_CRYPTO_AUTH_SHA1_HMAC:
+ /*
+ * Generate authentication key, i_key_pad and o_key_pad.
+ */
+ /* Zero memory under key */
+ memset(sess->auth.hmac.key, 0, SHA1_AUTH_KEY_LENGTH);
+
+ if (xform->auth.key.length > SHA1_AUTH_KEY_LENGTH) {
+ /*
+ * In case the key is longer than 160 bits
+ * the algorithm will use SHA1(key) instead.
+ */
+ error = sha1_block(NULL, xform->auth.key.data,
+ sess->auth.hmac.key, xform->auth.key.length);
+ if (error != 0)
+ return -1;
+ } else {
+ /*
+ * Now copy the given authentication key to the session
+ * key assuming that the session key is zeroed there is
+ * no need for additional zero padding if the key is
+ * shorter than SHA1_AUTH_KEY_LENGTH.
+ */
+ rte_memcpy(sess->auth.hmac.key, xform->auth.key.data,
+ xform->auth.key.length);
+ }
+
+ /* Prepare HMAC padding: key|pattern */
+ auth_hmac_pad_prepare(sess, xform);
+ /*
+ * Calculate partial hash values for i_key_pad and o_key_pad.
+ * Will be used as initialization state for final HMAC.
+ */
+ error = sha1_block_partial(NULL, sess->auth.hmac.i_key_pad,
+ partial, SHA1_BLOCK_SIZE);
+ if (error != 0)
+ return -1;
+ memcpy(sess->auth.hmac.i_key_pad, partial, SHA1_BLOCK_SIZE);
+
+ error = sha1_block_partial(NULL, sess->auth.hmac.o_key_pad,
+ partial, SHA1_BLOCK_SIZE);
+ if (error != 0)
+ return -1;
+ memcpy(sess->auth.hmac.o_key_pad, partial, SHA1_BLOCK_SIZE);
+
+ break;
+ case RTE_CRYPTO_AUTH_SHA256_HMAC:
+ /*
+ * Generate authentication key, i_key_pad and o_key_pad.
+ */
+ /* Zero memory under key */
+ memset(sess->auth.hmac.key, 0, SHA256_AUTH_KEY_LENGTH);
+
+ if (xform->auth.key.length > SHA256_AUTH_KEY_LENGTH) {
+ /*
+ * In case the key is longer than 256 bits
+ * the algorithm will use SHA256(key) instead.
+ */
+ error = sha256_block(NULL, xform->auth.key.data,
+ sess->auth.hmac.key, xform->auth.key.length);
+ if (error != 0)
+ return -1;
+ } else {
+ /*
+ * Now copy the given authentication key to the session
+ * key assuming that the session key is zeroed there is
+ * no need for additional zero padding if the key is
+ * shorter than SHA256_AUTH_KEY_LENGTH.
+ */
+ rte_memcpy(sess->auth.hmac.key, xform->auth.key.data,
+ xform->auth.key.length);
+ }
+
+ /* Prepare HMAC padding: key|pattern */
+ auth_hmac_pad_prepare(sess, xform);
+ /*
+ * Calculate partial hash values for i_key_pad and o_key_pad.
+ * Will be used as initialization state for final HMAC.
+ */
+ error = sha256_block_partial(NULL, sess->auth.hmac.i_key_pad,
+ partial, SHA256_BLOCK_SIZE);
+ if (error != 0)
+ return -1;
+ memcpy(sess->auth.hmac.i_key_pad, partial, SHA256_BLOCK_SIZE);
+
+ error = sha256_block_partial(NULL, sess->auth.hmac.o_key_pad,
+ partial, SHA256_BLOCK_SIZE);
+ if (error != 0)
+ return -1;
+ memcpy(sess->auth.hmac.o_key_pad, partial, SHA256_BLOCK_SIZE);
+
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static inline int
+cipher_set_prerequisites(struct armv8_crypto_session *sess,
+ const struct rte_crypto_sym_xform *xform)
+{
+ crypto_key_sched_t cipher_key_sched;
+
+ cipher_key_sched = sess->cipher.key_sched;
+ if (likely(cipher_key_sched != NULL)) {
+ /* Set up cipher session key */
+ cipher_key_sched(sess->cipher.key.data, xform->cipher.key.data);
+ }
+
+ return 0;
+}
+
+static int
+armv8_crypto_set_session_chained_parameters(struct armv8_crypto_session *sess,
+ const struct rte_crypto_sym_xform *cipher_xform,
+ const struct rte_crypto_sym_xform *auth_xform)
+{
+ enum armv8_crypto_chain_order order;
+ enum armv8_crypto_cipher_operation cop;
+ enum rte_crypto_cipher_algorithm calg;
+ enum rte_crypto_auth_algorithm aalg;
+
+ /* Validate and prepare scratch order of combined operations */
+ switch (sess->chain_order) {
+ case ARMV8_CRYPTO_CHAIN_CIPHER_AUTH:
+ case ARMV8_CRYPTO_CHAIN_AUTH_CIPHER:
+ order = sess->chain_order;
+ break;
+ default:
+ return -EINVAL;
+ }
+ /* Select cipher direction */
+ sess->cipher.direction = cipher_xform->cipher.op;
+ /* Select cipher key */
+ sess->cipher.key.length = cipher_xform->cipher.key.length;
+ /* Set cipher direction */
+ cop = sess->cipher.direction;
+ /* Set cipher algorithm */
+ calg = cipher_xform->cipher.algo;
+
+ /* Select cipher algo */
+ switch (calg) {
+ /* Cover supported cipher algorithms */
+ case RTE_CRYPTO_CIPHER_AES_CBC:
+ sess->cipher.algo = calg;
+ /* IV len is always 16 bytes (block size) for AES CBC */
+ sess->cipher.iv_len = 16;
+ break;
+ default:
+ return -EINVAL;
+ }
+ /* Select auth generate/verify */
+ sess->auth.operation = auth_xform->auth.op;
+
+ /* Select auth algo */
+ switch (auth_xform->auth.algo) {
+ /* Cover supported hash algorithms */
+ case RTE_CRYPTO_AUTH_SHA256:
+ aalg = auth_xform->auth.algo;
+ sess->auth.mode = ARMV8_CRYPTO_AUTH_AS_AUTH;
+ break;
+ case RTE_CRYPTO_AUTH_SHA1_HMAC:
+ case RTE_CRYPTO_AUTH_SHA256_HMAC: /* Fall through */
+ aalg = auth_xform->auth.algo;
+ sess->auth.mode = ARMV8_CRYPTO_AUTH_AS_HMAC;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /* Verify supported key lengths and extract proper algorithm */
+ switch (cipher_xform->cipher.key.length << 3) {
+ case 128:
+ sess->crypto_func =
+ CRYPTO_GET_ALGO(order, cop, calg, aalg, 128);
+ sess->cipher.key_sched =
+ CRYPTO_GET_KEY_SCHED(cop, calg, 128);
+ break;
+ case 192:
+ sess->crypto_func =
+ CRYPTO_GET_ALGO(order, cop, calg, aalg, 192);
+ sess->cipher.key_sched =
+ CRYPTO_GET_KEY_SCHED(cop, calg, 192);
+ break;
+ case 256:
+ sess->crypto_func =
+ CRYPTO_GET_ALGO(order, cop, calg, aalg, 256);
+ sess->cipher.key_sched =
+ CRYPTO_GET_KEY_SCHED(cop, calg, 256);
+ break;
+ default:
+ sess->crypto_func = NULL;
+ sess->cipher.key_sched = NULL;
+ return -EINVAL;
+ }
+
+ if (unlikely(sess->crypto_func == NULL)) {
+ /*
+ * If we got here that means that there must be a bug
+ * in the algorithms selection above. Nevertheless keep
+ * it here to catch bug immediately and avoid NULL pointer
+ * dereference in OPs processing.
+ */
+ ARMV8_CRYPTO_LOG_ERR(
+ "No appropriate crypto function for given parameters");
+ return -EINVAL;
+ }
+
+ /* Set up cipher session prerequisites */
+ if (cipher_set_prerequisites(sess, cipher_xform) != 0)
+ return -EINVAL;
+
+ /* Set up authentication session prerequisites */
+ if (auth_set_prerequisites(sess, auth_xform) != 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+/** Parse crypto xform chain and set private session parameters */
+int
+armv8_crypto_set_session_parameters(struct armv8_crypto_session *sess,
+ const struct rte_crypto_sym_xform *xform)
+{
+ const struct rte_crypto_sym_xform *cipher_xform = NULL;
+ const struct rte_crypto_sym_xform *auth_xform = NULL;
+ bool is_chained_op;
+ int ret;
+
+ /* Filter out spurious/broken requests */
+ if (xform == NULL)
+ return -EINVAL;
+
+ sess->chain_order = armv8_crypto_get_chain_order(xform);
+ switch (sess->chain_order) {
+ case ARMV8_CRYPTO_CHAIN_CIPHER_AUTH:
+ cipher_xform = xform;
+ auth_xform = xform->next;
+ is_chained_op = true;
+ break;
+ case ARMV8_CRYPTO_CHAIN_AUTH_CIPHER:
+ auth_xform = xform;
+ cipher_xform = xform->next;
+ is_chained_op = true;
+ break;
+ default:
+ is_chained_op = false;
+ return -EINVAL;
+ }
+
+ if (is_chained_op) {
+ ret = armv8_crypto_set_session_chained_parameters(sess,
+ cipher_xform, auth_xform);
+ if (unlikely(ret != 0)) {
+ ARMV8_CRYPTO_LOG_ERR(
+ "Invalid/unsupported chained (cipher/auth) parameters");
+ return -EINVAL;
+ }
+ } else {
+ ARMV8_CRYPTO_LOG_ERR("Invalid/unsupported operation");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/** Provide session for operation */
+static struct armv8_crypto_session *
+get_session(struct armv8_crypto_qp *qp, struct rte_crypto_op *op)
+{
+ struct armv8_crypto_session *sess = NULL;
+
+ if (op->sym->sess_type == RTE_CRYPTO_SYM_OP_WITH_SESSION) {
+ /* get existing session */
+ if (likely(op->sym->session != NULL &&
+ op->sym->session->dev_type ==
+ RTE_CRYPTODEV_ARMV8_PMD)) {
+ sess = (struct armv8_crypto_session *)
+ op->sym->session->_private;
+ }
+ } else {
+ /* provide internal session */
+ void *_sess = NULL;
+
+ if (!rte_mempool_get(qp->sess_mp, (void **)&_sess)) {
+ sess = (struct armv8_crypto_session *)
+ ((struct rte_cryptodev_sym_session *)_sess)
+ ->_private;
+
+ if (unlikely(armv8_crypto_set_session_parameters(
+ sess, op->sym->xform) != 0)) {
+ rte_mempool_put(qp->sess_mp, _sess);
+ sess = NULL;
+ } else
+ op->sym->session = _sess;
+ }
+ }
+
+ if (sess == NULL)
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION;
+
+ return sess;
+}
+
+/*
+ *------------------------------------------------------------------------------
+ * Process Operations
+ *------------------------------------------------------------------------------
+ */
+
+/*----------------------------------------------------------------------------*/
+
+/** Process cipher operation */
+static void
+process_armv8_chained_op
+ (struct rte_crypto_op *op, struct armv8_crypto_session *sess,
+ struct rte_mbuf *mbuf_src, struct rte_mbuf *mbuf_dst)
+{
+ crypto_func_t crypto_func;
+ crypto_arg_t arg;
+ struct rte_mbuf *m_asrc, *m_adst;
+ uint8_t *csrc, *cdst;
+ uint8_t *adst, *asrc;
+ uint64_t clen, alen __rte_unused;
+ int error;
+
+ clen = op->sym->cipher.data.length;
+ alen = op->sym->auth.data.length;
+
+ csrc = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
+ op->sym->cipher.data.offset);
+ cdst = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
+ op->sym->cipher.data.offset);
+
+ switch (sess->chain_order) {
+ case ARMV8_CRYPTO_CHAIN_CIPHER_AUTH:
+ m_asrc = m_adst = mbuf_dst;
+ break;
+ case ARMV8_CRYPTO_CHAIN_AUTH_CIPHER:
+ m_asrc = mbuf_src;
+ m_adst = mbuf_dst;
+ break;
+ default:
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return;
+ }
+ asrc = rte_pktmbuf_mtod_offset(m_asrc, uint8_t *,
+ op->sym->auth.data.offset);
+
+ switch (sess->auth.mode) {
+ case ARMV8_CRYPTO_AUTH_AS_AUTH:
+ /* Nothing to do here, just verify correct option */
+ break;
+ case ARMV8_CRYPTO_AUTH_AS_HMAC:
+ arg.digest.hmac.key = sess->auth.hmac.key;
+ arg.digest.hmac.i_key_pad = sess->auth.hmac.i_key_pad;
+ arg.digest.hmac.o_key_pad = sess->auth.hmac.o_key_pad;
+ break;
+ default:
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return;
+ }
+
+ if (sess->auth.operation == RTE_CRYPTO_AUTH_OP_GENERATE) {
+ adst = op->sym->auth.digest.data;
+ if (adst == NULL) {
+ adst = rte_pktmbuf_mtod_offset(m_adst,
+ uint8_t *,
+ op->sym->auth.data.offset +
+ op->sym->auth.data.length);
+ }
+ } else {
+ adst = (uint8_t *)rte_pktmbuf_append(m_asrc,
+ op->sym->auth.digest.length);
+ }
+
+ if (unlikely(op->sym->cipher.iv.length != sess->cipher.iv_len)) {
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return;
+ }
+
+ arg.cipher.iv = op->sym->cipher.iv.data;
+ arg.cipher.key = sess->cipher.key.data;
+ /* Acquire combined mode function */
+ crypto_func = sess->crypto_func;
+ ARMV8_CRYPTO_ASSERT(crypto_func != NULL);
+ error = crypto_func(csrc, cdst, clen, asrc, adst, alen, &arg);
+ if (error != 0) {
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return;
+ }
+
+ op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+ if (sess->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) {
+ if (memcmp(adst, op->sym->auth.digest.data,
+ op->sym->auth.digest.length) != 0) {
+ op->status = RTE_CRYPTO_OP_STATUS_AUTH_FAILED;
+ }
+ /* Trim area used for digest from mbuf. */
+ rte_pktmbuf_trim(m_asrc,
+ op->sym->auth.digest.length);
+ }
+}
+
+/** Process crypto operation for mbuf */
+static int
+process_op(const struct armv8_crypto_qp *qp, struct rte_crypto_op *op,
+ struct armv8_crypto_session *sess)
+{
+ struct rte_mbuf *msrc, *mdst;
+ int retval;
+
+ msrc = op->sym->m_src;
+ mdst = op->sym->m_dst ? op->sym->m_dst : op->sym->m_src;
+
+ op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
+
+ switch (sess->chain_order) {
+ case ARMV8_CRYPTO_CHAIN_CIPHER_AUTH:
+ case ARMV8_CRYPTO_CHAIN_AUTH_CIPHER: /* Fall through */
+ process_armv8_chained_op(op, sess, msrc, mdst);
+ break;
+ default:
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ break;
+ }
+
+ /* Free session if a session-less crypto op */
+ if (op->sym->sess_type == RTE_CRYPTO_SYM_OP_SESSIONLESS) {
+ memset(sess, 0, sizeof(struct armv8_crypto_session));
+ rte_mempool_put(qp->sess_mp, op->sym->session);
+ op->sym->session = NULL;
+ }
+
+ if (op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED)
+ op->status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+
+ if (op->status != RTE_CRYPTO_OP_STATUS_ERROR)
+ retval = rte_ring_enqueue(qp->processed_ops, (void *)op);
+ else
+ retval = -1;
+
+ return retval;
+}
+
+/*
+ *------------------------------------------------------------------------------
+ * PMD Framework
+ *------------------------------------------------------------------------------
+ */
+
+/** Enqueue burst */
+static uint16_t
+armv8_crypto_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct armv8_crypto_session *sess;
+ struct armv8_crypto_qp *qp = queue_pair;
+ int i, retval;
+
+ for (i = 0; i < nb_ops; i++) {
+ sess = get_session(qp, ops[i]);
+ if (unlikely(sess == NULL))
+ goto enqueue_err;
+
+ retval = process_op(qp, ops[i], sess);
+ if (unlikely(retval < 0))
+ goto enqueue_err;
+ }
+
+ qp->stats.enqueued_count += i;
+ return i;
+
+enqueue_err:
+ if (ops[i] != NULL)
+ ops[i]->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+
+ qp->stats.enqueue_err_count++;
+ return i;
+}
+
+/** Dequeue burst */
+static uint16_t
+armv8_crypto_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct armv8_crypto_qp *qp = queue_pair;
+
+ unsigned int nb_dequeued = 0;
+
+ nb_dequeued = rte_ring_dequeue_burst(qp->processed_ops,
+ (void **)ops, nb_ops);
+ qp->stats.dequeued_count += nb_dequeued;
+
+ return nb_dequeued;
+}
+
+/** Create ARMv8 crypto device */
+static int
+cryptodev_armv8_crypto_create(const char *name,
+ struct rte_crypto_vdev_init_params *init_params)
+{
+ struct rte_cryptodev *dev;
+ char crypto_dev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct armv8_crypto_private *internals;
+
+ /* Check CPU for support for AES instruction set */
+ if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
+ ARMV8_CRYPTO_LOG_ERR(
+ "AES instructions not supported by CPU");
+ return -EFAULT;
+ }
+
+ /* Check CPU for support for SHA instruction set */
+ if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_SHA1) ||
+ !rte_cpu_get_flag_enabled(RTE_CPUFLAG_SHA2)) {
+ ARMV8_CRYPTO_LOG_ERR(
+ "SHA1/SHA2 instructions not supported by CPU");
+ return -EFAULT;
+ }
+
+ /* Check CPU for support for Advance SIMD instruction set */
+ if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) {
+ ARMV8_CRYPTO_LOG_ERR(
+ "Advanced SIMD instructions not supported by CPU");
+ return -EFAULT;
+ }
+
+ /* create a unique device name */
+ if (create_unique_device_name(crypto_dev_name,
+ RTE_CRYPTODEV_NAME_MAX_LEN) != 0) {
+ ARMV8_CRYPTO_LOG_ERR("failed to create unique cryptodev name");
+ return -EINVAL;
+ }
+
+ dev = rte_cryptodev_pmd_virtual_dev_init(crypto_dev_name,
+ sizeof(struct armv8_crypto_private),
+ init_params->socket_id);
+ if (dev == NULL) {
+ ARMV8_CRYPTO_LOG_ERR("failed to create cryptodev vdev");
+ goto init_error;
+ }
+
+ dev->dev_type = RTE_CRYPTODEV_ARMV8_PMD;
+ dev->dev_ops = rte_armv8_crypto_pmd_ops;
+
+ /* register rx/tx burst functions for data path */
+ dev->dequeue_burst = armv8_crypto_pmd_dequeue_burst;
+ dev->enqueue_burst = armv8_crypto_pmd_enqueue_burst;
+
+ dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
+ RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING;
+
+ /* Set vector instructions mode supported */
+ internals = dev->data->dev_private;
+
+ internals->max_nb_qpairs = init_params->max_nb_queue_pairs;
+ internals->max_nb_sessions = init_params->max_nb_sessions;
+
+ return 0;
+
+init_error:
+ ARMV8_CRYPTO_LOG_ERR(
+ "driver %s: cryptodev_armv8_crypto_create failed", name);
+
+ cryptodev_armv8_crypto_uninit(crypto_dev_name);
+ return -EFAULT;
+}
+
+/** Initialise ARMv8 crypto device */
+static int
+cryptodev_armv8_crypto_init(const char *name,
+ const char *input_args)
+{
+ struct rte_crypto_vdev_init_params init_params = {
+ RTE_CRYPTODEV_VDEV_DEFAULT_MAX_NB_QUEUE_PAIRS,
+ RTE_CRYPTODEV_VDEV_DEFAULT_MAX_NB_SESSIONS,
+ rte_socket_id()
+ };
+
+ rte_cryptodev_parse_vdev_init_params(&init_params, input_args);
+
+ RTE_LOG(INFO, PMD, "Initialising %s on NUMA node %d\n", name,
+ init_params.socket_id);
+ RTE_LOG(INFO, PMD, " Max number of queue pairs = %d\n",
+ init_params.max_nb_queue_pairs);
+ RTE_LOG(INFO, PMD, " Max number of sessions = %d\n",
+ init_params.max_nb_sessions);
+
+ return cryptodev_armv8_crypto_create(name, &init_params);
+}
+
+/** Uninitialise ARMv8 crypto device */
+static int
+cryptodev_armv8_crypto_uninit(const char *name)
+{
+ if (name == NULL)
+ return -EINVAL;
+
+ RTE_LOG(INFO, PMD,
+ "Closing ARMv8 crypto device %s on numa socket %u\n",
+ name, rte_socket_id());
+
+ return 0;
+}
+
+static struct rte_vdev_driver armv8_crypto_drv = {
+ .probe = cryptodev_armv8_crypto_init,
+ .remove = cryptodev_armv8_crypto_uninit
+};
+
+RTE_PMD_REGISTER_VDEV(CRYPTODEV_NAME_ARMV8_PMD, armv8_crypto_drv);
+RTE_PMD_REGISTER_ALIAS(CRYPTODEV_NAME_ARMV8_PMD, cryptodev_armv8_pmd);
+RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_ARMV8_PMD,
+ "max_nb_queue_pairs=<int> "
+ "max_nb_sessions=<int> "
+ "socket_id=<int>");
diff --git a/drivers/crypto/armv8/rte_armv8_pmd_ops.c b/drivers/crypto/armv8/rte_armv8_pmd_ops.c
new file mode 100644
index 0000000..2bf6475
--- /dev/null
+++ b/drivers/crypto/armv8/rte_armv8_pmd_ops.c
@@ -0,0 +1,369 @@
+/*
+ * BSD LICENSE
+ *
+ * Copyright (C) Cavium networks Ltd. 2017.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Cavium networks nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_cryptodev_pmd.h>
+
+#include "armv8_crypto_defs.h"
+
+#include "rte_armv8_pmd_private.h"
+
+static const struct rte_cryptodev_capabilities
+ armv8_crypto_pmd_capabilities[] = {
+ { /* SHA1 HMAC */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+ {.auth = {
+ .algo = RTE_CRYPTO_AUTH_SHA1_HMAC,
+ .block_size = 64,
+ .key_size = {
+ .min = 16,
+ .max = 128,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 20,
+ .max = 20,
+ .increment = 0
+ },
+ .aad_size = { 0 }
+ }, }
+ }, }
+ },
+ { /* SHA256 HMAC */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH,
+ {.auth = {
+ .algo = RTE_CRYPTO_AUTH_SHA256_HMAC,
+ .block_size = 64,
+ .key_size = {
+ .min = 16,
+ .max = 128,
+ .increment = 0
+ },
+ .digest_size = {
+ .min = 32,
+ .max = 32,
+ .increment = 0
+ },
+ .aad_size = { 0 }
+ }, }
+ }, }
+ },
+ { /* AES CBC */
+ .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+ {.sym = {
+ .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER,
+ {.cipher = {
+ .algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .block_size = 16,
+ .key_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ },
+ .iv_size = {
+ .min = 16,
+ .max = 16,
+ .increment = 0
+ }
+ }, }
+ }, }
+ },
+
+ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST()
+};
+
+
+/** Configure device */
+static int
+armv8_crypto_pmd_config(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+/** Start device */
+static int
+armv8_crypto_pmd_start(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+/** Stop device */
+static void
+armv8_crypto_pmd_stop(__rte_unused struct rte_cryptodev *dev)
+{
+}
+
+/** Close device */
+static int
+armv8_crypto_pmd_close(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+
+/** Get device statistics */
+static void
+armv8_crypto_pmd_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ int qp_id;
+
+ for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+ struct armv8_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ stats->enqueued_count += qp->stats.enqueued_count;
+ stats->dequeued_count += qp->stats.dequeued_count;
+
+ stats->enqueue_err_count += qp->stats.enqueue_err_count;
+ stats->dequeue_err_count += qp->stats.dequeue_err_count;
+ }
+}
+
+/** Reset device statistics */
+static void
+armv8_crypto_pmd_stats_reset(struct rte_cryptodev *dev)
+{
+ int qp_id;
+
+ for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+ struct armv8_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ memset(&qp->stats, 0, sizeof(qp->stats));
+ }
+}
+
+
+/** Get device info */
+static void
+armv8_crypto_pmd_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *dev_info)
+{
+ struct armv8_crypto_private *internals = dev->data->dev_private;
+
+ if (dev_info != NULL) {
+ dev_info->dev_type = dev->dev_type;
+ dev_info->feature_flags = dev->feature_flags;
+ dev_info->capabilities = armv8_crypto_pmd_capabilities;
+ dev_info->max_nb_queue_pairs = internals->max_nb_qpairs;
+ dev_info->sym.max_nb_sessions = internals->max_nb_sessions;
+ }
+}
+
+/** Release queue pair */
+static int
+armv8_crypto_pmd_qp_release(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+
+ if (dev->data->queue_pairs[qp_id] != NULL) {
+ rte_free(dev->data->queue_pairs[qp_id]);
+ dev->data->queue_pairs[qp_id] = NULL;
+ }
+
+ return 0;
+}
+
+/** set a unique name for the queue pair based on it's name, dev_id and qp_id */
+static int
+armv8_crypto_pmd_qp_set_unique_name(struct rte_cryptodev *dev,
+ struct armv8_crypto_qp *qp)
+{
+ unsigned int n;
+
+ n = snprintf(qp->name, sizeof(qp->name), "armv8_crypto_pmd_%u_qp_%u",
+ dev->data->dev_id, qp->id);
+
+ if (n > sizeof(qp->name))
+ return -1;
+
+ return 0;
+}
+
+
+/** Create a ring to place processed operations on */
+static struct rte_ring *
+armv8_crypto_pmd_qp_create_processed_ops_ring(struct armv8_crypto_qp *qp,
+ unsigned int ring_size, int socket_id)
+{
+ struct rte_ring *r;
+
+ r = rte_ring_lookup(qp->name);
+ if (r) {
+ if (r->prod.size >= ring_size) {
+ ARMV8_CRYPTO_LOG_INFO(
+ "Reusing existing ring %s for processed ops",
+ qp->name);
+ return r;
+ }
+
+ ARMV8_CRYPTO_LOG_ERR(
+ "Unable to reuse existing ring %s for processed ops",
+ qp->name);
+ return NULL;
+ }
+
+ return rte_ring_create(qp->name, ring_size, socket_id,
+ RING_F_SP_ENQ | RING_F_SC_DEQ);
+}
+
+
+/** Setup a queue pair */
+static int
+armv8_crypto_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ const struct rte_cryptodev_qp_conf *qp_conf,
+ int socket_id)
+{
+ struct armv8_crypto_qp *qp = NULL;
+
+ /* Free memory prior to re-allocation if needed. */
+ if (dev->data->queue_pairs[qp_id] != NULL)
+ armv8_crypto_pmd_qp_release(dev, qp_id);
+
+ /* Allocate the queue pair data structure. */
+ qp = rte_zmalloc_socket("ARMv8 PMD Queue Pair", sizeof(*qp),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (qp == NULL)
+ return -ENOMEM;
+
+ qp->id = qp_id;
+ dev->data->queue_pairs[qp_id] = qp;
+
+ if (armv8_crypto_pmd_qp_set_unique_name(dev, qp) != 0)
+ goto qp_setup_cleanup;
+
+ qp->processed_ops = armv8_crypto_pmd_qp_create_processed_ops_ring(qp,
+ qp_conf->nb_descriptors, socket_id);
+ if (qp->processed_ops == NULL)
+ goto qp_setup_cleanup;
+
+ qp->sess_mp = dev->data->session_pool;
+
+ memset(&qp->stats, 0, sizeof(qp->stats));
+
+ return 0;
+
+qp_setup_cleanup:
+ if (qp)
+ rte_free(qp);
+
+ return -1;
+}
+
+/** Start queue pair */
+static int
+armv8_crypto_pmd_qp_start(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint16_t queue_pair_id)
+{
+ return -ENOTSUP;
+}
+
+/** Stop queue pair */
+static int
+armv8_crypto_pmd_qp_stop(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint16_t queue_pair_id)
+{
+ return -ENOTSUP;
+}
+
+/** Return the number of allocated queue pairs */
+static uint32_t
+armv8_crypto_pmd_qp_count(struct rte_cryptodev *dev)
+{
+ return dev->data->nb_queue_pairs;
+}
+
+/** Returns the size of the session structure */
+static unsigned
+armv8_crypto_pmd_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct armv8_crypto_session);
+}
+
+/** Configure the session from a crypto xform chain */
+static void *
+armv8_crypto_pmd_session_configure(struct rte_cryptodev *dev __rte_unused,
+ struct rte_crypto_sym_xform *xform, void *sess)
+{
+ if (unlikely(sess == NULL)) {
+ ARMV8_CRYPTO_LOG_ERR("invalid session struct");
+ return NULL;
+ }
+
+ if (armv8_crypto_set_session_parameters(
+ sess, xform) != 0) {
+ ARMV8_CRYPTO_LOG_ERR("failed configure session parameters");
+ return NULL;
+ }
+
+ return sess;
+}
+
+/** Clear the memory of session so it doesn't leave key material behind */
+static void
+armv8_crypto_pmd_session_clear(struct rte_cryptodev *dev __rte_unused,
+ void *sess)
+{
+
+ /* Zero out the whole structure */
+ if (sess)
+ memset(sess, 0, sizeof(struct armv8_crypto_session));
+}
+
+struct rte_cryptodev_ops armv8_crypto_pmd_ops = {
+ .dev_configure = armv8_crypto_pmd_config,
+ .dev_start = armv8_crypto_pmd_start,
+ .dev_stop = armv8_crypto_pmd_stop,
+ .dev_close = armv8_crypto_pmd_close,
+
+ .stats_get = armv8_crypto_pmd_stats_get,
+ .stats_reset = armv8_crypto_pmd_stats_reset,
+
+ .dev_infos_get = armv8_crypto_pmd_info_get,
+
+ .queue_pair_setup = armv8_crypto_pmd_qp_setup,
+ .queue_pair_release = armv8_crypto_pmd_qp_release,
+ .queue_pair_start = armv8_crypto_pmd_qp_start,
+ .queue_pair_stop = armv8_crypto_pmd_qp_stop,
+ .queue_pair_count = armv8_crypto_pmd_qp_count,
+
+ .session_get_size = armv8_crypto_pmd_session_get_size,
+ .session_configure = armv8_crypto_pmd_session_configure,
+ .session_clear = armv8_crypto_pmd_session_clear
+};
+
+struct rte_cryptodev_ops *rte_armv8_crypto_pmd_ops = &armv8_crypto_pmd_ops;
diff --git a/drivers/crypto/armv8/rte_armv8_pmd_private.h b/drivers/crypto/armv8/rte_armv8_pmd_private.h
new file mode 100644
index 0000000..fe46cde
--- /dev/null
+++ b/drivers/crypto/armv8/rte_armv8_pmd_private.h
@@ -0,0 +1,211 @@
+/*
+ * BSD LICENSE
+ *
+ * Copyright (C) Cavium networks Ltd. 2017.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Cavium networks nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_ARMV8_PMD_PRIVATE_H_
+#define _RTE_ARMV8_PMD_PRIVATE_H_
+
+#define ARMV8_CRYPTO_LOG_ERR(fmt, args...) \
+ RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_ARMV8_CRYPTO_PMD), \
+ __func__, __LINE__, ## args)
+
+#ifdef RTE_LIBRTE_ARMV8_CRYPTO_DEBUG
+#define ARMV8_CRYPTO_LOG_INFO(fmt, args...) \
+ RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_ARMV8_CRYPTO_PMD), \
+ __func__, __LINE__, ## args)
+
+#define ARMV8_CRYPTO_LOG_DBG(fmt, args...) \
+ RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_ARMV8_CRYPTO_PMD), \
+ __func__, __LINE__, ## args)
+
+#define ARMV8_CRYPTO_ASSERT(con) \
+do { \
+ if (!(con)) { \
+ rte_panic("%s(): " \
+ con "condition failed, line %u", __func__); \
+ } \
+} while (0)
+
+#else
+#define ARMV8_CRYPTO_LOG_INFO(fmt, args...)
+#define ARMV8_CRYPTO_LOG_DBG(fmt, args...)
+#define ARMV8_CRYPTO_ASSERT(con)
+#endif
+
+#define NBBY 8 /* Number of bits in a byte */
+#define BYTE_LENGTH(x) ((x) / 8) /* Number of bytes in x (roun down) */
+
+/** ARMv8 operation order mode enumerator */
+enum armv8_crypto_chain_order {
+ ARMV8_CRYPTO_CHAIN_CIPHER_AUTH,
+ ARMV8_CRYPTO_CHAIN_AUTH_CIPHER,
+ ARMV8_CRYPTO_CHAIN_NOT_SUPPORTED,
+ ARMV8_CRYPTO_CHAIN_LIST_END = ARMV8_CRYPTO_CHAIN_NOT_SUPPORTED
+};
+
+/** ARMv8 cipher operation enumerator */
+enum armv8_crypto_cipher_operation {
+ ARMV8_CRYPTO_CIPHER_OP_ENCRYPT = RTE_CRYPTO_CIPHER_OP_ENCRYPT,
+ ARMV8_CRYPTO_CIPHER_OP_DECRYPT = RTE_CRYPTO_CIPHER_OP_DECRYPT,
+ ARMV8_CRYPTO_CIPHER_OP_NOT_SUPPORTED,
+ ARMV8_CRYPTO_CIPHER_OP_LIST_END = ARMV8_CRYPTO_CIPHER_OP_NOT_SUPPORTED
+};
+
+enum armv8_crypto_cipher_keylen {
+ ARMV8_CRYPTO_CIPHER_KEYLEN_128,
+ ARMV8_CRYPTO_CIPHER_KEYLEN_192,
+ ARMV8_CRYPTO_CIPHER_KEYLEN_256,
+ ARMV8_CRYPTO_CIPHER_KEYLEN_NOT_SUPPORTED,
+ ARMV8_CRYPTO_CIPHER_KEYLEN_LIST_END =
+ ARMV8_CRYPTO_CIPHER_KEYLEN_NOT_SUPPORTED
+};
+
+/** ARMv8 auth mode enumerator */
+enum armv8_crypto_auth_mode {
+ ARMV8_CRYPTO_AUTH_AS_AUTH,
+ ARMV8_CRYPTO_AUTH_AS_HMAC,
+ ARMV8_CRYPTO_AUTH_AS_CIPHER,
+ ARMV8_CRYPTO_AUTH_NOT_SUPPORTED,
+ ARMV8_CRYPTO_AUTH_LIST_END = ARMV8_CRYPTO_AUTH_NOT_SUPPORTED
+};
+
+#define CRYPTO_ORDER_MAX ARMV8_CRYPTO_CHAIN_LIST_END
+#define CRYPTO_CIPHER_OP_MAX ARMV8_CRYPTO_CIPHER_OP_LIST_END
+#define CRYPTO_CIPHER_KEYLEN_MAX ARMV8_CRYPTO_CIPHER_KEYLEN_LIST_END
+#define CRYPTO_CIPHER_MAX RTE_CRYPTO_CIPHER_LIST_END
+#define CRYPTO_AUTH_MAX RTE_CRYPTO_AUTH_LIST_END
+
+#define HMAC_IPAD_VALUE (0x36)
+#define HMAC_OPAD_VALUE (0x5C)
+
+#define SHA256_AUTH_KEY_LENGTH (BYTE_LENGTH(256))
+#define SHA256_BLOCK_SIZE (BYTE_LENGTH(512))
+
+#define SHA1_AUTH_KEY_LENGTH (BYTE_LENGTH(160))
+#define SHA1_BLOCK_SIZE (BYTE_LENGTH(512))
+
+#define SHA_AUTH_KEY_MAX SHA256_AUTH_KEY_LENGTH
+#define SHA_BLOCK_MAX SHA256_BLOCK_SIZE
+
+typedef int (*crypto_func_t)(uint8_t *, uint8_t *, uint64_t,
+ uint8_t *, uint8_t *, uint64_t,
+ crypto_arg_t *);
+
+typedef void (*crypto_key_sched_t)(uint8_t *, const uint8_t *);
+
+/** private data structure for each ARMv8 crypto device */
+struct armv8_crypto_private {
+ unsigned int max_nb_qpairs;
+ /**< Max number of queue pairs */
+ unsigned int max_nb_sessions;
+ /**< Max number of sessions */
+};
+
+/** ARMv8 crypto queue pair */
+struct armv8_crypto_qp {
+ uint16_t id;
+ /**< Queue Pair Identifier */
+ char name[RTE_CRYPTODEV_NAME_LEN];
+ /**< Unique Queue Pair Name */
+ struct rte_ring *processed_ops;
+ /**< Ring for placing process packets */
+ struct rte_mempool *sess_mp;
+ /**< Session Mempool */
+ struct rte_cryptodev_stats stats;
+ /**< Queue pair statistics */
+} __rte_cache_aligned;
+
+/** ARMv8 crypto private session structure */
+struct armv8_crypto_session {
+ enum armv8_crypto_chain_order chain_order;
+ /**< chain order mode */
+ crypto_func_t crypto_func;
+ /**< cryptographic function to use for this session */
+
+ /** Cipher Parameters */
+ struct {
+ enum rte_crypto_cipher_operation direction;
+ /**< cipher operation direction */
+ enum rte_crypto_cipher_algorithm algo;
+ /**< cipher algorithm */
+ int iv_len;
+ /**< IV length */
+
+ struct {
+ uint8_t data[256];
+ /**< key data */
+ size_t length;
+ /**< key length in bytes */
+ } key;
+
+ crypto_key_sched_t key_sched;
+ /**< Key schedule function */
+ } cipher;
+
+ /** Authentication Parameters */
+ struct {
+ enum rte_crypto_auth_operation operation;
+ /**< auth operation generate or verify */
+ enum armv8_crypto_auth_mode mode;
+ /**< auth operation mode */
+
+ union {
+ struct {
+ /* Add data if needed */
+ } auth;
+
+ struct {
+ uint8_t i_key_pad[SHA_BLOCK_MAX]
+ __rte_cache_aligned;
+ /**< inner pad (max supported block length) */
+ uint8_t o_key_pad[SHA_BLOCK_MAX]
+ __rte_cache_aligned;
+ /**< outer pad (max supported block length) */
+ uint8_t key[SHA_AUTH_KEY_MAX];
+ /**< HMAC key (max supported length)*/
+ } hmac;
+ };
+ } auth;
+
+} __rte_cache_aligned;
+
+/** Set and validate ARMv8 crypto session parameters */
+extern int armv8_crypto_set_session_parameters(
+ struct armv8_crypto_session *sess,
+ const struct rte_crypto_sym_xform *xform);
+
+/** device specific operations function pointer structure */
+extern struct rte_cryptodev_ops *rte_armv8_crypto_pmd_ops;
+
+#endif /* _RTE_ARMV8_PMD_PRIVATE_H_ */
diff --git a/drivers/crypto/armv8/rte_armv8_pmd_version.map b/drivers/crypto/armv8/rte_armv8_pmd_version.map
new file mode 100644
index 0000000..1f84b68
--- /dev/null
+++ b/drivers/crypto/armv8/rte_armv8_pmd_version.map
@@ -0,0 +1,3 @@
+DPDK_17.02 {
+ local: *;
+};
--
1.9.1
^ permalink raw reply related
* [PATCH v3 4/8] mk/crypto/armv8: add PMD to the build system
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Build ARMv8 crypto PMD if compiling for ARM64
and CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO option
is enable in the configuration file.
ARMV8_CRYPTO_LIB_PATH environment variable will
point to the appropriate library directory.
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
drivers/crypto/Makefile | 1 +
mk/rte.app.mk | 2 ++
2 files changed, 3 insertions(+)
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index 745c614..77b02cf 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -33,6 +33,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
DIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_GCM) += aesni_gcm
DIRS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += aesni_mb
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += armv8
DIRS-$(CONFIG_RTE_LIBRTE_PMD_OPENSSL) += openssl
DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat
DIRS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G) += snow3g
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index f75f0e2..bbb5265 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -145,6 +145,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += -lrte_pmd_zuc
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += -L$(LIBSSO_ZUC_PATH)/build -lsso_zuc
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += -lrte_pmd_armv8
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO) += -L$(ARMV8_CRYPTO_LIB_PATH) -larmv8_crypto
endif # CONFIG_RTE_LIBRTE_CRYPTODEV
endif # !CONFIG_RTE_BUILD_SHARED_LIBS
--
1.9.1
^ permalink raw reply related
* [PATCH v3 5/8] doc/armv8: update documentation about crypto PMD
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Add documentation about the driver and update
release notes.
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
doc/guides/cryptodevs/armv8.rst | 96 ++++++++++++++++++++++++++++++++++
doc/guides/cryptodevs/index.rst | 1 +
doc/guides/rel_notes/release_17_02.rst | 5 ++
3 files changed, 102 insertions(+)
create mode 100644 doc/guides/cryptodevs/armv8.rst
diff --git a/doc/guides/cryptodevs/armv8.rst b/doc/guides/cryptodevs/armv8.rst
new file mode 100644
index 0000000..ca8781e
--- /dev/null
+++ b/doc/guides/cryptodevs/armv8.rst
@@ -0,0 +1,96 @@
+.. BSD LICENSE
+ Copyright (C) Cavium networks Ltd. 2017.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Cavium networks nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+ARMv8 Crypto Poll Mode Driver
+================================
+
+This code provides the initial implementation of the ARMv8 crypto PMD.
+The driver uses ARMv8 cryptographic extensions to process chained crypto
+operations in an optimized way. The core functionality is provided by
+a low-level library, written in the assembly code.
+
+Features
+--------
+
+ARMv8 Crypto PMD has support for the following algorithm pairs:
+
+Supported cipher algorithms:
+* ``RTE_CRYPTO_CIPHER_AES_CBC``
+
+Supported authentication algorithms:
+* ``RTE_CRYPTO_AUTH_SHA1_HMAC``
+* ``RTE_CRYPTO_AUTH_SHA256_HMAC``
+
+Installation
+------------
+
+In order to enable this virtual crypto PMD, user must:
+
+* Download ARMv8 crypto library source code from
+ `here <https://github.com/caviumnetworks/armv8_crypto>`_
+
+* Export the environmental variable ARMV8_CRYPTO_LIB_PATH with
+ the path where the ``armv8_crypto`` library was downloaded
+ or cloned.
+
+* Build the library by invoking:
+
+.. code-block:: console
+
+ make -C $ARMV8_CRYPTO_LIB_PATH/
+
+* Set CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO=y in
+ config/defconfig_arm64-armv8a-linuxapp-gcc
+
+The corresponding device can be created only if the following features
+are supported by the CPU:
+
+* ``RTE_CPUFLAG_AES``
+* ``RTE_CPUFLAG_SHA1``
+* ``RTE_CPUFLAG_SHA2``
+* ``RTE_CPUFLAG_NEON``
+
+Initialization
+--------------
+
+User can use app/test application to check how to use this PMD and to verify
+crypto processing.
+
+Test name is cryptodev_sw_armv8_autotest.
+For performance test cryptodev_sw_armv8_perftest can be used.
+
+Limitations
+-----------
+
+* Maximum number of sessions is 2048.
+* Only chained operations are supported.
+* AES-128-CBC is the only supported cipher variant.
+* Cipher input data has to be a multiple of 16 bytes.
+* Digest input data has to be a multiple of 8 bytes.
diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst
index a6a9f23..06c3f6e 100644
--- a/doc/guides/cryptodevs/index.rst
+++ b/doc/guides/cryptodevs/index.rst
@@ -38,6 +38,7 @@ Crypto Device Drivers
overview
aesni_mb
aesni_gcm
+ armv8
kasumi
openssl
null
diff --git a/doc/guides/rel_notes/release_17_02.rst b/doc/guides/rel_notes/release_17_02.rst
index 180af82..c3e1f56 100644
--- a/doc/guides/rel_notes/release_17_02.rst
+++ b/doc/guides/rel_notes/release_17_02.rst
@@ -52,6 +52,11 @@ New Features
See the :ref:`Generic flow API <Generic_flow_API>` documentation for more
information.
+* **Added armv8 crypto PMD.**
+
+ A new crypto PMD has been added, which provides combined mode cryptografic
+ operations optimized for ARMv8 processors. The driver can be used to enhance
+ performance in processing chained operations such as cipher + HMAC.
Resolved Issues
---------------
--
1.9.1
^ permalink raw reply related
* [PATCH v3 6/8] crypto/armv8: enable ARMv8 PMD in the configuration
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Add CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO option to
the common configuration file. Don't enable it by
default for ARM64 as it requires external library
to build.
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
config/common_base | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/config/common_base b/config/common_base
index edb6a54..e0c0c0a 100644
--- a/config/common_base
+++ b/config/common_base
@@ -407,6 +407,12 @@ CONFIG_RTE_LIBRTE_PMD_ZUC=n
CONFIG_RTE_LIBRTE_PMD_ZUC_DEBUG=n
#
+# Compile PMD for ARMv8 Crypto device
+#
+CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO=n
+CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO_DEBUG=n
+
+#
# Compile PMD for NULL Crypto device
#
CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y
--
1.9.1
^ permalink raw reply related
* [PATCH v3 7/8] crypto/armv8: update MAINTAINERS entry for ARMv8 crypto
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
MAINTAINERS | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index ebc97b8..89b5179 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -447,6 +447,12 @@ M: Declan Doherty <declan.doherty@intel.com>
F: drivers/crypto/openssl/
F: doc/guides/cryptodevs/openssl.rst
+ARMv8 Crypto PMD
+M: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
+M: Jerin Jacob <jerin.jacob@caviumnetworks.com>
+F: drivers/crypto/armv8/
+F: doc/guides/cryptodevs/armv8.rst
+
Null Crypto PMD
M: Declan Doherty <declan.doherty@intel.com>
F: drivers/crypto/null/
--
1.9.1
^ permalink raw reply related
* [PATCH v3 8/8] app/test: add ARMv8 crypto tests and test vectors
From: zbigniew.bodek @ 2017-01-04 17:33 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, jerin.jacob, Zbigniew Bodek
In-Reply-To: <1483551207-18236-1-git-send-email-zbigniew.bodek@caviumnetworks.com>
From: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Introduce unit tests for ARMv8 crypto PMD.
Add test vectors for short cases such as 160 bytes.
These test cases are ARMv8 specific since the code provides
different processing paths for different input data sizes.
User can validate correctness of algorithms' implementation using:
* cryptodev_sw_armv8_autotest
For performance test one can use:
* cryptodev_sw_armv8_perftest
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
---
app/test/test_cryptodev.c | 63 ++++
app/test/test_cryptodev_aes_test_vectors.h | 144 ++++++++-
app/test/test_cryptodev_blockcipher.c | 4 +
app/test/test_cryptodev_blockcipher.h | 1 +
app/test/test_cryptodev_perf.c | 480 +++++++++++++++++++++++++++++
5 files changed, 684 insertions(+), 8 deletions(-)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 872f8b4..a0540d6 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -348,6 +348,27 @@ struct crypto_unittest_params {
}
}
+ /* Create 2 ARMv8 devices if required */
+ if (gbl_cryptodev_type == RTE_CRYPTODEV_ARMV8_PMD) {
+#ifndef RTE_LIBRTE_PMD_ARMV8_CRYPTO
+ RTE_LOG(ERR, USER1, "CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO must be"
+ " enabled in config file to run this testsuite.\n");
+ return TEST_FAILED;
+#endif
+ nb_devs = rte_cryptodev_count_devtype(RTE_CRYPTODEV_ARMV8_PMD);
+ if (nb_devs < 2) {
+ for (i = nb_devs; i < 2; i++) {
+ ret = rte_eal_vdev_init(
+ RTE_STR(CRYPTODEV_NAME_ARMV8_PMD),
+ NULL);
+
+ TEST_ASSERT(ret == 0, "Failed to create "
+ "instance %u of pmd : %s", i,
+ RTE_STR(CRYPTODEV_NAME_ARMV8_PMD));
+ }
+ }
+ }
+
#ifndef RTE_LIBRTE_PMD_QAT
if (gbl_cryptodev_type == RTE_CRYPTODEV_QAT_SYM_PMD) {
RTE_LOG(ERR, USER1, "CONFIG_RTE_LIBRTE_PMD_QAT must be enabled "
@@ -1545,6 +1566,22 @@ struct crypto_unittest_params {
return TEST_SUCCESS;
}
+static int
+test_AES_chain_armv8_all(void)
+{
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+ int status;
+
+ status = test_blockcipher_all_tests(ts_params->mbuf_pool,
+ ts_params->op_mpool, ts_params->valid_devs[0],
+ RTE_CRYPTODEV_ARMV8_PMD,
+ BLKCIPHER_AES_CHAIN_TYPE);
+
+ TEST_ASSERT_EQUAL(status, 0, "Test failed");
+
+ return TEST_SUCCESS;
+}
+
/* ***** SNOW 3G Tests ***** */
static int
create_wireless_algo_hash_session(uint8_t dev_id,
@@ -6504,6 +6541,23 @@ struct test_crypto_vector {
}
};
+static struct unit_test_suite cryptodev_armv8_testsuite = {
+ .suite_name = "Crypto Device ARMv8 Unit Test Suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_armv8_all),
+
+ /** Negative tests */
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ auth_decryption_AES128CBC_HMAC_SHA1_fail_data_corrupt),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ auth_decryption_AES128CBC_HMAC_SHA1_fail_tag_corrupt),
+
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
static int
test_cryptodev_qat(void /*argv __rte_unused, int argc __rte_unused*/)
{
@@ -6567,6 +6621,14 @@ struct test_crypto_vector {
return unit_test_suite_runner(&cryptodev_sw_zuc_testsuite);
}
+static int
+test_cryptodev_armv8(void)
+{
+ gbl_cryptodev_type = RTE_CRYPTODEV_ARMV8_PMD;
+
+ return unit_test_suite_runner(&cryptodev_armv8_testsuite);
+}
+
REGISTER_TEST_COMMAND(cryptodev_qat_autotest, test_cryptodev_qat);
REGISTER_TEST_COMMAND(cryptodev_aesni_mb_autotest, test_cryptodev_aesni_mb);
REGISTER_TEST_COMMAND(cryptodev_openssl_autotest, test_cryptodev_openssl);
@@ -6575,3 +6637,4 @@ struct test_crypto_vector {
REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_autotest, test_cryptodev_sw_snow3g);
REGISTER_TEST_COMMAND(cryptodev_sw_kasumi_autotest, test_cryptodev_sw_kasumi);
REGISTER_TEST_COMMAND(cryptodev_sw_zuc_autotest, test_cryptodev_sw_zuc);
+REGISTER_TEST_COMMAND(cryptodev_sw_armv8_autotest, test_cryptodev_armv8);
diff --git a/app/test/test_cryptodev_aes_test_vectors.h b/app/test/test_cryptodev_aes_test_vectors.h
index 1c68f93..5683406 100644
--- a/app/test/test_cryptodev_aes_test_vectors.h
+++ b/app/test/test_cryptodev_aes_test_vectors.h
@@ -825,6 +825,98 @@
}
};
+/** AES-128-CBC SHA256 HMAC test vector (160 bytes) */
+static const struct blockcipher_test_data aes_test_data_12 = {
+ .crypto_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key = {
+ .data = {
+ 0xE4, 0x23, 0x33, 0x8A, 0x35, 0x64, 0x61, 0xE2,
+ 0x49, 0x03, 0xDD, 0xC6, 0xB8, 0xCA, 0x55, 0x7A
+ },
+ .len = 16
+ },
+ .iv = {
+ .data = {
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F
+ },
+ .len = 16
+ },
+ .plaintext = {
+ .data = plaintext_aes_common,
+ .len = 160
+ },
+ .ciphertext = {
+ .data = ciphertext512_aes128cbc,
+ .len = 160
+ },
+ .auth_algo = RTE_CRYPTO_AUTH_SHA256_HMAC,
+ .auth_key = {
+ .data = {
+ 0x42, 0x1A, 0x7D, 0x3D, 0xF5, 0x82, 0x80, 0xF1,
+ 0xF1, 0x35, 0x5C, 0x3B, 0xDD, 0x9A, 0x65, 0xBA,
+ 0x58, 0x34, 0x85, 0x61, 0x1C, 0x42, 0x10, 0x76,
+ 0x9A, 0x4F, 0x88, 0x1B, 0xB6, 0x8F, 0xD8, 0x60
+ },
+ .len = 32
+ },
+ .digest = {
+ .data = {
+ 0x92, 0xEC, 0x65, 0x9A, 0x52, 0xCC, 0x50, 0xA5,
+ 0xEE, 0x0E, 0xDF, 0x1E, 0xA4, 0xC9, 0xC1, 0x04,
+ 0xD5, 0xDC, 0x78, 0x90, 0xF4, 0xE3, 0x35, 0x62,
+ 0xAD, 0x95, 0x45, 0x28, 0x5C, 0xF8, 0x8C, 0x0B
+ },
+ .len = 32,
+ .truncated_len = 16
+ }
+};
+
+/** AES-128-CBC SHA1 HMAC test vector (160 bytes) */
+static const struct blockcipher_test_data aes_test_data_13 = {
+ .crypto_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key = {
+ .data = {
+ 0xE4, 0x23, 0x33, 0x8A, 0x35, 0x64, 0x61, 0xE2,
+ 0x49, 0x03, 0xDD, 0xC6, 0xB8, 0xCA, 0x55, 0x7A
+ },
+ .len = 16
+ },
+ .iv = {
+ .data = {
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F
+ },
+ .len = 16
+ },
+ .plaintext = {
+ .data = plaintext_aes_common,
+ .len = 160
+ },
+ .ciphertext = {
+ .data = ciphertext512_aes128cbc,
+ .len = 160
+ },
+ .auth_algo = RTE_CRYPTO_AUTH_SHA1_HMAC,
+ .auth_key = {
+ .data = {
+ 0xF8, 0x2A, 0xC7, 0x54, 0xDB, 0x96, 0x18, 0xAA,
+ 0xC3, 0xA1, 0x53, 0xF6, 0x1F, 0x17, 0x60, 0xBD,
+ 0xDE, 0xF4, 0xDE, 0xAD
+ },
+ .len = 20
+ },
+ .digest = {
+ .data = {
+ 0x4F, 0x16, 0xEA, 0xF7, 0x4A, 0x88, 0xD3, 0xE0,
+ 0x0E, 0x12, 0x8B, 0xE7, 0x05, 0xD0, 0x86, 0x48,
+ 0x22, 0x43, 0x30, 0xA7
+ },
+ .len = 20,
+ .truncated_len = 12
+ }
+};
+
static const struct blockcipher_test_case aes_chain_test_cases[] = {
{
.test_descr = "AES-128-CTR HMAC-SHA1 Encryption Digest",
@@ -878,37 +970,69 @@
.test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest",
.test_data = &aes_test_data_4,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_MB |
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL |
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA1 Encryption Digest "
+ "(short buffers)",
+ .test_data = &aes_test_data_13,
+ .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8
+ },
+ {
.test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest "
"Verify",
.test_data = &aes_test_data_4,
.op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_MB |
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL |
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA1 Decryption Digest "
+ "Verify (short buffers)",
+ .test_data = &aes_test_data_13,
+ .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8
+ },
+ {
.test_descr = "AES-128-CBC HMAC-SHA256 Encryption Digest",
.test_data = &aes_test_data_5,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_MB |
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL |
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA256 Encryption Digest "
+ "(short buffers)",
+ .test_data = &aes_test_data_12,
+ .op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8
+ },
+ {
.test_descr = "AES-128-CBC HMAC-SHA256 Decryption Digest "
"Verify",
.test_data = &aes_test_data_5,
.op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_MB |
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_MB |
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL |
BLOCKCIPHER_TEST_TARGET_PMD_QAT
},
{
+ .test_descr = "AES-128-CBC HMAC-SHA256 Decryption Digest "
+ "Verify (short buffers)",
+ .test_data = &aes_test_data_12,
+ .op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8
+ },
+ {
.test_descr = "AES-128-CBC HMAC-SHA512 Encryption Digest",
.test_data = &aes_test_data_6,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
@@ -954,7 +1078,8 @@
.test_data = &aes_test_data_4,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
.feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_QAT |
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_QAT |
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
},
{
@@ -963,7 +1088,8 @@
.test_data = &aes_test_data_4,
.op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
.feature_mask = BLOCKCIPHER_TEST_FEATURE_OOP,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_QAT |
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_QAT |
BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
},
{
@@ -1006,7 +1132,8 @@
.test_data = &aes_test_data_4,
.op_mask = BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN,
.feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
},
{
.test_descr =
@@ -1015,7 +1142,8 @@
.test_data = &aes_test_data_4,
.op_mask = BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC,
.feature_mask = BLOCKCIPHER_TEST_FEATURE_SESSIONLESS,
- .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
+ .pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 |
+ BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL
},
};
diff --git a/app/test/test_cryptodev_blockcipher.c b/app/test/test_cryptodev_blockcipher.c
index 37b10cf..6963241 100644
--- a/app/test/test_cryptodev_blockcipher.c
+++ b/app/test/test_cryptodev_blockcipher.c
@@ -82,6 +82,7 @@
switch (cryptodev_type) {
case RTE_CRYPTODEV_QAT_SYM_PMD:
case RTE_CRYPTODEV_OPENSSL_PMD:
+ case RTE_CRYPTODEV_ARMV8_PMD: /* Fall through */
digest_len = tdata->digest.len;
break;
case RTE_CRYPTODEV_AESNI_MB_PMD:
@@ -508,6 +509,9 @@
case RTE_CRYPTODEV_OPENSSL_PMD:
target_pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL;
break;
+ case RTE_CRYPTODEV_ARMV8_PMD:
+ target_pmd_mask = BLOCKCIPHER_TEST_TARGET_PMD_ARMV8;
+ break;
default:
TEST_ASSERT(0, "Unrecognized cryptodev type");
break;
diff --git a/app/test/test_cryptodev_blockcipher.h b/app/test/test_cryptodev_blockcipher.h
index 04ff1ee..bd362c7 100644
--- a/app/test/test_cryptodev_blockcipher.h
+++ b/app/test/test_cryptodev_blockcipher.h
@@ -49,6 +49,7 @@
#define BLOCKCIPHER_TEST_TARGET_PMD_MB 0x0001 /* Multi-buffer flag */
#define BLOCKCIPHER_TEST_TARGET_PMD_QAT 0x0002 /* QAT flag */
#define BLOCKCIPHER_TEST_TARGET_PMD_OPENSSL 0x0004 /* SW OPENSSL flag */
+#define BLOCKCIPHER_TEST_TARGET_PMD_ARMV8 0x0008 /* ARMv8 flag */
#define BLOCKCIPHER_TEST_OP_CIPHER (BLOCKCIPHER_TEST_OP_ENCRYPT | \
BLOCKCIPHER_TEST_OP_DECRYPT)
diff --git a/app/test/test_cryptodev_perf.c b/app/test/test_cryptodev_perf.c
index 59a6891..827cf4d 100644
--- a/app/test/test_cryptodev_perf.c
+++ b/app/test/test_cryptodev_perf.c
@@ -157,6 +157,12 @@ struct crypto_unittest_params {
enum rte_crypto_cipher_algorithm cipher_algo,
unsigned int cipher_key_len,
enum rte_crypto_auth_algorithm auth_algo);
+static struct rte_cryptodev_sym_session *
+test_perf_create_armv8_session(uint8_t dev_id, enum chain_mode chain,
+ enum rte_crypto_cipher_algorithm cipher_algo,
+ unsigned int cipher_key_len,
+ enum rte_crypto_auth_algorithm auth_algo);
+
static struct rte_mbuf *
test_perf_create_pktmbuf(struct rte_mempool *mpool, unsigned buf_sz);
static inline struct rte_crypto_op *
@@ -397,6 +403,27 @@ static const char *auth_algo_name(enum rte_crypto_auth_algorithm auth_algo)
}
}
+ /* Create 2 ARMv8 devices if required */
+ if (gbl_cryptodev_perftest_devtype == RTE_CRYPTODEV_ARMV8_PMD) {
+#ifndef RTE_LIBRTE_PMD_ARMV8_CRYPTO
+ RTE_LOG(ERR, USER1, "CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO must be"
+ " enabled in config file to run this testsuite.\n");
+ return TEST_FAILED;
+#endif
+ nb_devs = rte_cryptodev_count_devtype(RTE_CRYPTODEV_ARMV8_PMD);
+ if (nb_devs < 2) {
+ for (i = nb_devs; i < 2; i++) {
+ ret = rte_eal_vdev_init(
+ RTE_STR(CRYPTODEV_NAME_ARMV8_PMD),
+ NULL);
+
+ TEST_ASSERT(ret == 0, "Failed to create "
+ "instance %u of pmd : %s", i,
+ RTE_STR(CRYPTODEV_NAME_ARMV8_PMD));
+ }
+ }
+ }
+
#ifndef RTE_LIBRTE_PMD_QAT
if (gbl_cryptodev_perftest_devtype == RTE_CRYPTODEV_QAT_SYM_PMD) {
RTE_LOG(ERR, USER1, "CONFIG_RTE_LIBRTE_PMD_QAT must be enabled "
@@ -2422,6 +2449,136 @@ struct crypto_data_params aes_cbc_hmac_sha256_output[MAX_PACKET_SIZE_INDEX] = {
return TEST_SUCCESS;
}
+static int
+test_perf_armv8_optimise_cyclecount(struct perf_test_params *pparams)
+{
+ uint32_t num_to_submit = pparams->total_operations;
+ struct rte_crypto_op *c_ops[num_to_submit];
+ struct rte_crypto_op *proc_ops[num_to_submit];
+ uint64_t failed_polls, retries, start_cycles, end_cycles,
+ total_cycles = 0;
+ uint32_t burst_sent = 0, burst_received = 0;
+ uint32_t i, burst_size, num_sent, num_ops_received;
+
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+
+ static struct rte_cryptodev_sym_session *sess;
+
+ unsigned int digest_length = get_auth_digest_length(pparams->auth_algo);
+
+ if (rte_cryptodev_count() == 0) {
+ printf("\nNo crypto devices found. Is PMD build configured?\n");
+ return TEST_FAILED;
+ }
+
+ /* Create Crypto session*/
+ sess = test_perf_create_armv8_session(ts_params->dev_id,
+ pparams->chain, pparams->cipher_algo,
+ pparams->cipher_key_length, pparams->auth_algo);
+ TEST_ASSERT_NOT_NULL(sess, "Session creation failed");
+
+ /* Generate Crypto op data structure(s)*/
+ for (i = 0; i < num_to_submit ; i++) {
+ struct rte_mbuf *m = test_perf_create_pktmbuf(
+ ts_params->mbuf_mp,
+ pparams->buf_size);
+ TEST_ASSERT_NOT_NULL(m, "Failed to allocate tx_buf");
+
+ struct rte_crypto_op *op =
+ rte_crypto_op_alloc(ts_params->op_mpool,
+ RTE_CRYPTO_OP_TYPE_SYMMETRIC);
+ TEST_ASSERT_NOT_NULL(op, "Failed to allocate op");
+
+ op = test_perf_set_crypto_op_aes(op, m, sess, pparams->buf_size,
+ digest_length);
+ TEST_ASSERT_NOT_NULL(op, "Failed to attach op to session");
+
+ c_ops[i] = op;
+ }
+
+ printf("\nOn %s dev%u qp%u, %s, cipher algo:%s, cipher key length:%u, "
+ "auth_algo:%s, Packet Size %u bytes",
+ pmd_name(gbl_cryptodev_perftest_devtype),
+ ts_params->dev_id, 0,
+ chain_mode_name(pparams->chain),
+ cipher_algo_name(pparams->cipher_algo),
+ pparams->cipher_key_length,
+ auth_algo_name(pparams->auth_algo),
+ pparams->buf_size);
+ printf("\nOps Tx\tOps Rx\tOps/burst ");
+ printf("Retries "
+ "EmptyPolls\tIACycles/CyOp\tIACycles/Burst\tIACycles/Byte");
+
+ for (i = 2; i <= 128 ; i *= 2) {
+ num_sent = 0;
+ num_ops_received = 0;
+ retries = 0;
+ failed_polls = 0;
+ burst_size = i;
+ total_cycles = 0;
+ while (num_sent < num_to_submit) {
+ start_cycles = rte_rdtsc_precise();
+ burst_sent = rte_cryptodev_enqueue_burst(
+ ts_params->dev_id,
+ 0, &c_ops[num_sent],
+ ((num_to_submit - num_sent) < burst_size) ?
+ num_to_submit - num_sent : burst_size);
+ end_cycles = rte_rdtsc_precise();
+ if (burst_sent == 0)
+ retries++;
+ num_sent += burst_sent;
+ total_cycles += (end_cycles - start_cycles);
+
+ /* Wait until requests have been sent. */
+ rte_delay_ms(1);
+
+ start_cycles = rte_rdtsc_precise();
+ burst_received = rte_cryptodev_dequeue_burst(
+ ts_params->dev_id, 0, proc_ops,
+ burst_size);
+ end_cycles = rte_rdtsc_precise();
+ if (burst_received < burst_sent)
+ failed_polls++;
+ num_ops_received += burst_received;
+
+ total_cycles += end_cycles - start_cycles;
+ }
+
+ while (num_ops_received != num_to_submit) {
+ /* Sending 0 length burst to flush sw crypto device */
+ rte_cryptodev_enqueue_burst(
+ ts_params->dev_id, 0, NULL, 0);
+
+ start_cycles = rte_rdtsc_precise();
+ burst_received = rte_cryptodev_dequeue_burst(
+ ts_params->dev_id, 0, proc_ops, burst_size);
+ end_cycles = rte_rdtsc_precise();
+
+ total_cycles += end_cycles - start_cycles;
+ if (burst_received == 0)
+ failed_polls++;
+ num_ops_received += burst_received;
+ }
+
+ printf("\n%u\t%u\t%u", num_sent, num_ops_received, burst_size);
+ printf("\t\t%"PRIu64, retries);
+ printf("\t%"PRIu64, failed_polls);
+ printf("\t\t%"PRIu64, total_cycles/num_ops_received);
+ printf("\t\t%"PRIu64,
+ (total_cycles/num_ops_received)*burst_size);
+ printf("\t\t%"PRIu64,
+ total_cycles/(num_ops_received*pparams->buf_size));
+ }
+ printf("\n");
+
+ for (i = 0; i < num_to_submit ; i++) {
+ rte_pktmbuf_free(c_ops[i]->sym->m_src);
+ rte_crypto_op_free(c_ops[i]);
+ }
+
+ return TEST_SUCCESS;
+}
+
static uint32_t get_auth_key_max_length(enum rte_crypto_auth_algorithm algo)
{
switch (algo) {
@@ -2683,6 +2840,56 @@ static uint32_t get_auth_digest_length(enum rte_crypto_auth_algorithm algo)
}
}
+static struct rte_cryptodev_sym_session *
+test_perf_create_armv8_session(uint8_t dev_id, enum chain_mode chain,
+ enum rte_crypto_cipher_algorithm cipher_algo,
+ unsigned int cipher_key_len,
+ enum rte_crypto_auth_algorithm auth_algo)
+{
+ struct rte_crypto_sym_xform cipher_xform = { 0 };
+ struct rte_crypto_sym_xform auth_xform = { 0 };
+
+ /* Setup Cipher Parameters */
+ cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+ cipher_xform.cipher.algo = cipher_algo;
+
+ switch (cipher_algo) {
+ case RTE_CRYPTO_CIPHER_AES_CBC:
+ cipher_xform.cipher.key.data = aes_cbc_128_key;
+ break;
+ default:
+ return NULL;
+ }
+
+ cipher_xform.cipher.key.length = cipher_key_len;
+
+ /* Setup Auth Parameters */
+ auth_xform.type = RTE_CRYPTO_SYM_XFORM_AUTH;
+ auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+ auth_xform.auth.algo = auth_algo;
+
+ auth_xform.auth.digest_length = get_auth_digest_length(auth_algo);
+
+ switch (chain) {
+ case CIPHER_HASH:
+ cipher_xform.next = &auth_xform;
+ auth_xform.next = NULL;
+ /* Encrypt and hash the result */
+ cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+ /* Create Crypto session*/
+ return rte_cryptodev_sym_session_create(dev_id, &cipher_xform);
+ case HASH_CIPHER:
+ auth_xform.next = &cipher_xform;
+ cipher_xform.next = NULL;
+ /* Hash encrypted message and decrypt */
+ cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
+ /* Create Crypto session*/
+ return rte_cryptodev_sym_session_create(dev_id, &auth_xform);
+ default:
+ return NULL;
+ }
+}
+
#define AES_BLOCK_SIZE 16
#define AES_CIPHER_IV_LENGTH 16
@@ -3356,6 +3563,138 @@ static uint32_t get_auth_digest_length(enum rte_crypto_auth_algorithm algo)
return TEST_SUCCESS;
}
+static int
+test_perf_armv8(uint8_t dev_id, uint16_t queue_id,
+ struct perf_test_params *pparams)
+{
+ uint16_t i, k, l, m;
+ uint16_t j = 0;
+ uint16_t ops_unused = 0;
+ uint16_t burst_size;
+ uint16_t ops_needed;
+
+ uint64_t burst_enqueued = 0, total_enqueued = 0, burst_dequeued = 0;
+ uint64_t processed = 0, failed_polls = 0, retries = 0;
+ uint64_t tsc_start = 0, tsc_end = 0;
+
+ unsigned int digest_length = get_auth_digest_length(pparams->auth_algo);
+
+ struct rte_crypto_op *ops[pparams->burst_size];
+ struct rte_crypto_op *proc_ops[pparams->burst_size];
+
+ struct rte_mbuf *mbufs[pparams->burst_size * NUM_MBUF_SETS];
+
+ struct crypto_testsuite_params *ts_params = &testsuite_params;
+
+ static struct rte_cryptodev_sym_session *sess;
+
+ if (rte_cryptodev_count() == 0) {
+ printf("\nNo crypto devices found. Is PMD build configured?\n");
+ return TEST_FAILED;
+ }
+
+ /* Create Crypto session*/
+ sess = test_perf_create_armv8_session(ts_params->dev_id,
+ pparams->chain, pparams->cipher_algo,
+ pparams->cipher_key_length, pparams->auth_algo);
+ TEST_ASSERT_NOT_NULL(sess, "Session creation failed");
+
+ /* Generate a burst of crypto operations */
+ for (i = 0; i < (pparams->burst_size * NUM_MBUF_SETS); i++) {
+ mbufs[i] = test_perf_create_pktmbuf(
+ ts_params->mbuf_mp,
+ pparams->buf_size);
+
+ if (mbufs[i] == NULL) {
+ printf("\nFailed to get mbuf - freeing the rest.\n");
+ for (k = 0; k < i; k++)
+ rte_pktmbuf_free(mbufs[k]);
+ return -1;
+ }
+ }
+
+ tsc_start = rte_rdtsc_precise();
+
+ while (total_enqueued < pparams->total_operations) {
+ if ((total_enqueued + pparams->burst_size) <=
+ pparams->total_operations)
+ burst_size = pparams->burst_size;
+ else
+ burst_size = pparams->total_operations - total_enqueued;
+
+ ops_needed = burst_size - ops_unused;
+
+ if (ops_needed != rte_crypto_op_bulk_alloc(ts_params->op_mpool,
+ RTE_CRYPTO_OP_TYPE_SYMMETRIC, ops, ops_needed)){
+ printf("\nFailed to alloc enough ops, finish dequeuing "
+ "and free ops below.");
+ } else {
+ for (i = 0; i < ops_needed; i++)
+ ops[i] = test_perf_set_crypto_op_aes(ops[i],
+ mbufs[i + (pparams->burst_size *
+ (j % NUM_MBUF_SETS))],
+ sess, pparams->buf_size, digest_length);
+
+ /* enqueue burst */
+ burst_enqueued = rte_cryptodev_enqueue_burst(dev_id,
+ queue_id, ops, burst_size);
+
+ if (burst_enqueued < burst_size)
+ retries++;
+
+ ops_unused = burst_size - burst_enqueued;
+ total_enqueued += burst_enqueued;
+ }
+
+ /* dequeue burst */
+ burst_dequeued = rte_cryptodev_dequeue_burst(dev_id, queue_id,
+ proc_ops, pparams->burst_size);
+ if (burst_dequeued == 0)
+ failed_polls++;
+ else {
+ processed += burst_dequeued;
+
+ for (l = 0; l < burst_dequeued; l++)
+ rte_crypto_op_free(proc_ops[l]);
+ }
+ j++;
+ }
+
+ /* Dequeue any operations still in the crypto device */
+ while (processed < pparams->total_operations) {
+ /* Sending 0 length burst to flush sw crypto device */
+ rte_cryptodev_enqueue_burst(dev_id, queue_id, NULL, 0);
+
+ /* dequeue burst */
+ burst_dequeued = rte_cryptodev_dequeue_burst(dev_id, queue_id,
+ proc_ops, pparams->burst_size);
+ if (burst_dequeued == 0)
+ failed_polls++;
+ else {
+ processed += burst_dequeued;
+
+ for (m = 0; m < burst_dequeued; m++)
+ rte_crypto_op_free(proc_ops[m]);
+ }
+ }
+
+ tsc_end = rte_rdtsc_precise();
+
+ double ops_s = ((double)processed / (tsc_end - tsc_start))
+ * rte_get_tsc_hz();
+ double throughput = (ops_s * pparams->buf_size * NUM_MBUF_SETS)
+ / 1000000000;
+
+ printf("\t%u\t%6.2f\t%10.2f\t%8"PRIu64"\t%8"PRIu64, pparams->buf_size,
+ ops_s / 1000000, throughput, retries, failed_polls);
+
+ for (i = 0; i < pparams->burst_size * NUM_MBUF_SETS; i++)
+ rte_pktmbuf_free(mbufs[i]);
+
+ printf("\n");
+ return TEST_SUCCESS;
+}
+
/*
perf_test_aes_sha("avx2", HASH_CIPHER, 16, CBC, SHA1);
@@ -3664,6 +4003,125 @@ static uint32_t get_auth_digest_length(enum rte_crypto_auth_algorithm algo)
}
static int
+test_perf_armv8_vary_pkt_size(void)
+{
+ unsigned int total_operations = 100000;
+ unsigned int burst_size = { 64 };
+ unsigned int buf_lengths[] = { 64, 128, 256, 512, 768, 1024, 1280, 1536,
+ 1792, 2048 };
+ uint8_t i, j;
+
+ struct perf_test_params params_set[] = {
+ {
+ .chain = CIPHER_HASH,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA1_HMAC
+ },
+ {
+ .chain = HASH_CIPHER,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA1_HMAC
+ },
+ {
+ .chain = CIPHER_HASH,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA256_HMAC
+ },
+ {
+ .chain = HASH_CIPHER,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA256_HMAC
+ },
+ };
+
+ for (i = 0; i < RTE_DIM(params_set); i++) {
+ params_set[i].total_operations = total_operations;
+ params_set[i].burst_size = burst_size;
+ printf("\n%s. cipher algo: %s auth algo: %s cipher key size=%u."
+ " burst_size: %d ops\n",
+ chain_mode_name(params_set[i].chain),
+ cipher_algo_name(params_set[i].cipher_algo),
+ auth_algo_name(params_set[i].auth_algo),
+ params_set[i].cipher_key_length,
+ burst_size);
+ printf("\nBuffer Size(B)\tOPS(M)\tThroughput(Gbps)\tRetries\t"
+ "EmptyPolls\n");
+ for (j = 0; j < RTE_DIM(buf_lengths); j++) {
+ params_set[i].buf_size = buf_lengths[j];
+ test_perf_armv8(testsuite_params.dev_id, 0,
+ ¶ms_set[i]);
+ }
+ }
+
+ return 0;
+}
+
+static int
+test_perf_armv8_vary_burst_size(void)
+{
+ unsigned int total_operations = 4096;
+ uint16_t buf_lengths[] = { 64 };
+ uint8_t i, j;
+
+ struct perf_test_params params_set[] = {
+ {
+ .chain = CIPHER_HASH,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA1_HMAC
+ },
+ {
+ .chain = HASH_CIPHER,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA1_HMAC
+ },
+ {
+ .chain = CIPHER_HASH,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA256_HMAC
+ },
+ {
+ .chain = HASH_CIPHER,
+
+ .cipher_algo = RTE_CRYPTO_CIPHER_AES_CBC,
+ .cipher_key_length = 16,
+ .auth_algo = RTE_CRYPTO_AUTH_SHA256_HMAC
+ },
+ };
+
+ printf("\n\nStart %s.", __func__);
+ printf("\nThis Test measures the average IA cycle cost using a "
+ "constant request(packet) size. ");
+ printf("Cycle cost is only valid when indicators show device is "
+ "not busy, i.e. Retries and EmptyPolls = 0");
+
+ for (i = 0; i < RTE_DIM(params_set); i++) {
+ printf("\n");
+ params_set[i].total_operations = total_operations;
+
+ for (j = 0; j < RTE_DIM(buf_lengths); j++) {
+ params_set[i].buf_size = buf_lengths[j];
+ test_perf_armv8_optimise_cyclecount(¶ms_set[i]);
+ }
+ }
+
+ return 0;
+}
+
+static int
test_perf_aes_cbc_vary_burst_size(void)
{
return test_perf_crypto_qp_vary_burst_size(testsuite_params.dev_id);
@@ -4214,6 +4672,19 @@ static int test_continual_perf_AES_GCM(void)
}
};
+static struct unit_test_suite cryptodev_armv8_testsuite = {
+ .suite_name = "Crypto Device ARMv8 Unit Test Suite",
+ .setup = testsuite_setup,
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_perf_armv8_vary_pkt_size),
+ TEST_CASE_ST(ut_setup, ut_teardown,
+ test_perf_armv8_vary_burst_size),
+ TEST_CASES_END() /**< NULL terminate unit test array */
+ }
+};
+
static int
perftest_aesni_gcm_cryptodev(void)
{
@@ -4270,6 +4741,14 @@ static int test_continual_perf_AES_GCM(void)
return unit_test_suite_runner(&cryptodev_qat_continual_testsuite);
}
+static int
+perftest_sw_armv8_cryptodev(void /*argv __rte_unused, int argc __rte_unused*/)
+{
+ gbl_cryptodev_perftest_devtype = RTE_CRYPTODEV_ARMV8_PMD;
+
+ return unit_test_suite_runner(&cryptodev_armv8_testsuite);
+}
+
REGISTER_TEST_COMMAND(cryptodev_aesni_mb_perftest, perftest_aesni_mb_cryptodev);
REGISTER_TEST_COMMAND(cryptodev_qat_perftest, perftest_qat_cryptodev);
REGISTER_TEST_COMMAND(cryptodev_sw_snow3g_perftest, perftest_sw_snow3g_cryptodev);
@@ -4279,3 +4758,4 @@ static int test_continual_perf_AES_GCM(void)
perftest_openssl_cryptodev);
REGISTER_TEST_COMMAND(cryptodev_qat_continual_perftest,
perftest_qat_continual_cryptodev);
+REGISTER_TEST_COMMAND(cryptodev_sw_armv8_perftest, perftest_sw_armv8_cryptodev);
--
1.9.1
^ permalink raw reply related
* Re: [PATCH v2 1/6] ethdev: fix port data mismatched in multiple process model
From: Thomas Monjalon @ 2017-01-04 17:34 UTC (permalink / raw)
To: Yuanhan Liu; +Cc: dev, Bruce Richardson, Ferruh Yigit, Gonzalez Monroy, Sergio
In-Reply-To: <1482922962-21036-2-git-send-email-yuanhan.liu@linux.intel.com>
+Cc Sergio (maintainer of the secondary process thing)
2016-12-28 19:02, Yuanhan Liu:
> --- a/lib/librte_ether/rte_ethdev.c
> +++ b/lib/librte_ether/rte_ethdev.c
> @@ -201,9 +201,6 @@ rte_eth_dev_allocate(const char *name)
> return NULL;
> }
>
> - if (rte_eth_dev_data == NULL)
> - rte_eth_dev_data_alloc();
> -
It is dangerous to move this to rte_eth_dev_pci_probe.
Please keep it here and duplicate it in eth_dev_attach.
[...]
> +/*
> + * Attach to a port already registered by the primary process, which
> + * makes sure that the same device would both have the same port id
> + * in the primary and secondary process.
> + */
> +static struct rte_eth_dev *
> +eth_dev_attach(const char *name)
Maybe that the word "secondary" could help to differentiate of
the function rte_eth_dev_attach().
> +{
> + uint8_t i;
> + struct rte_eth_dev *eth_dev;
> +
> + for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
> + if (strcmp(rte_eth_dev_data[i].name, name) == 0)
> + break;
> + }
> + if (i == RTE_MAX_ETHPORTS) {
> + RTE_PMD_DEBUG_TRACE(
> + "device %s is not driven by the primary process\n",
> + name);
> + return NULL;
> + }
> +
> + RTE_ASSERT(eth_dev->data->port_id == i);
> +
> + eth_dev = &rte_eth_devices[i];
> + eth_dev->data = &rte_eth_dev_data[i];
> + eth_dev->attached = DEV_ATTACHED;
> + nb_ports++;
I am a bit nervous when I see these lines duplicated from rte_eth_dev_allocate.
Not sure whether it deserves a common function or not.
[...]
> @@ -246,9 +275,26 @@ rte_eth_dev_pci_probe(struct rte_pci_driver *pci_drv,
> - eth_dev = rte_eth_dev_allocate(ethdev_name);
> - if (eth_dev == NULL)
> - return -ENOMEM;
> + if (rte_eth_dev_data == NULL)
> + rte_eth_dev_data_alloc();
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + eth_dev = rte_eth_dev_allocate(ethdev_name);
> + if (eth_dev == NULL)
> + return -ENOMEM;
> + } else {
> + /*
> + * if we failed to attach a device, it means that
> + * device is skipped, due to some errors. Take
> + * virtio-net device as example, it could be the
> + * device is managed by virtio-net kernel driver.
> + * For such case, we return a positive value, to
> + * let EAL skip it as well.
> + */
This comment (a bit too long) should be placed between "if" and "return".
> + eth_dev = eth_dev_attach(ethdev_name);
> + if (eth_dev == NULL)
> + return 1;
> + }
^ permalink raw reply
* Re: [PATCH v2 1/7] pci: If a driver's probe function fails, unmap resources.
From: Thomas Monjalon @ 2017-01-04 17:39 UTC (permalink / raw)
To: Ben Walker; +Cc: Shreyansh Jain, dev
In-Reply-To: <2631459.aun9V5NdHs@xps13>
2016-12-21 17:19, Thomas Monjalon:
> 2016-11-25 14:51, Shreyansh Jain:
> > On Thursday 24 November 2016 01:37 AM, Ben Walker wrote:
> > > If resources were mapped prior to probe, unmap them
> > > if probe fails.
> > >
> > > This does not handle the case where the kernel driver was
> > > forcibly unbound prior to probe.
> > >
> > > Signed-off-by: Ben Walker <benjamin.walker@intel.com>
> >
> > Acked-by: Shreyansh Jain <shreyansh.jain@nxp.com>
>
> The 3 first patches of this series seems good and easy to agree,
> except some minor nits.
>
> Please Ben or Shreyansh, could you respin only them to help make
> progress before applying a PCI/bus rework from Shreyansh?
> As soon as one of you shoot a partial v3, I'll apply them.
> Thanks
Please Ben, do you want to make a partial v3?
^ permalink raw reply
* Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
From: Ananyev, Konstantin @ 2017-01-04 17:44 UTC (permalink / raw)
To: Bie, Tiwei
Cc: dev@dpdk.org, adrien.mazarguil@6wind.com, Lu, Wenzhuo,
Mcnamara, John, olivier.matz@6wind.com, thomas.monjalon@6wind.com,
Zhang, Helin, Dai, Wei, Wang, Xiao W
In-Reply-To: <20170104170011.GB56511@dpdk19>
> -----Original Message-----
> From: Bie, Tiwei
> Sent: Wednesday, January 4, 2017 5:00 PM
> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Cc: dev@dpdk.org; adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> olivier.matz@6wind.com; thomas.monjalon@6wind.com; Zhang, Helin <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang,
> Xiao W <xiao.w.wang@intel.com>
> Subject: Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
>
> On Wed, Jan 04, 2017 at 11:14:25PM +0800, Ananyev, Konstantin wrote:
> >
> >
> > > -----Original Message-----
> > > From: Bie, Tiwei
> > > Sent: Wednesday, January 4, 2017 2:39 PM
> > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > > Cc: dev@dpdk.org; adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>;
> > > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Zhang, Helin <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>;
> Wang,
> > > Xiao W <xiao.w.wang@intel.com>
> > > Subject: Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> > >
> > > On Wed, Jan 04, 2017 at 10:21:08PM +0800, Ananyev, Konstantin wrote:
> > > > Hi Twei,
> > > >
> > > > > -----Original Message-----
> > > > > From: Bie, Tiwei
> > > > > Sent: Wednesday, January 4, 2017 7:22 AM
> > > > > To: dev@dpdk.org
> > > > > Cc: adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > > > > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > > > > <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> > > > > Subject: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> > > > >
> > > > > Reserve a Tx capability flag and a Rx capability flag, that can be
> > > > > used by PMD to define its own capability flags when implementing the
> > > > > PMD-specific API.
> > > > >
> > > > > Suggested-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > > > > Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
> > > > > Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > > > ---
> > > > > lib/librte_ether/rte_ethdev.h | 2 ++
> > > > > 1 file changed, 2 insertions(+)
> > > > >
> > > > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> > > > > index d465825..8800b39 100644
> > > > > --- a/lib/librte_ether/rte_ethdev.h
> > > > > +++ b/lib/librte_ether/rte_ethdev.h
> > > > > @@ -857,6 +857,7 @@ struct rte_eth_conf {
> > > > > #define DEV_RX_OFFLOAD_TCP_LRO 0x00000010
> > > > > #define DEV_RX_OFFLOAD_QINQ_STRIP 0x00000020
> > > > > #define DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM 0x00000040
> > > > > +#define DEV_RX_OFFLOAD_RESERVED_0 0x00000080 /**< Used for PMD-specific API. */
> > > > >
> > > > > /**
> > > > > * TX offload capabilities of a device.
> > > > > @@ -874,6 +875,7 @@ struct rte_eth_conf {
> > > > > #define DEV_TX_OFFLOAD_GRE_TNL_TSO 0x00000400 /**< Used for tunneling packet. */
> > > > > #define DEV_TX_OFFLOAD_IPIP_TNL_TSO 0x00000800 /**< Used for tunneling packet. */
> > > > > #define DEV_TX_OFFLOAD_GENEVE_TNL_TSO 0x00001000 /**< Used for tunneling packet. */
> > > > > +#define DEV_TX_OFFLOAD_RESERVED_0 0x00002000 /**< Used for PMD-specific API. */
> > > > >
> > > > > /**
> > > > > * Ethernet device information
> > > > > --
> > > > > 2.7.4
> > > >
> > > > I am not sure how that supposed to work and how user should know that DEV_RX_OFFLOAD_RESERVED_0
> > > > is actually a MACSEC for ixgbe?
> > >
> > > Users are not supposed to use DEV_RX_OFFLOAD_RESERVED_0, instead, they
> > > should use the capabilities and the likes defined in rte_pmd_ixgbe.h
> > > where the PMD-specifics APIs are declared:
> > >
> > > /**
> > > * If these flags are advertised by the PMD, the NIC supports the MACsec
> > > * offload. The incoming MACsec traffics can be offloaded transparently
> > > * after the MACsec offload is configured correctly by the application.
> > > * And the application can set the PKT_TX_IXGBE_MACSEC flag in mbufs to
> > > * enable the MACsec offload for the packets to be transmitted.
> > > */
> > > #define DEV_RX_OFFLOAD_IXGBE_MACSEC_STRIP DEV_RX_OFFLOAD_RESERVED_0
> > > #define DEV_TX_OFFLOAD_IXGBE_MACSEC_INSERT DEV_TX_OFFLOAD_RESERVED_0
> > >
> > > /**
> > > * This event will occur when the PN counter in a MACsec connection
> > > * reach the exhaustion threshold.
> > > */
> > > #define RTE_ETH_EVENT_IXGBE_MACSEC RTE_ETH_EVENT_RESERVED_0
> > >
> > > /**
> > > * Offload the MACsec. This flag must be set by the application in mbuf
> > > * to enable this offload feature for a packet to be transmitted.
> > > */
> > > #define PKT_TX_IXGBE_MACSEC PKT_TX_RESERVED_0
> > >
> > > PMD-specific APIs can only be used on the corresponding driver/device,
> > > so different PMD can share the same reserved bit to represent different
> > > things when implementing their own PMD-specific APIs.
> >
> > Ok, and why do we need it?
> > Why we can't just have PKT_TX_MACSEC straightway?
> > What are you trying to gain here?
> > Is it just for future opportunity to save an extra bit in mbuf.ol_flags?
> > I don't think we are short of bits here right now, and we don't consume them exra-fast
> > to start to worry about it.
> > Again, if it *really* would be for ixgbe only forever, and y don't want to waste a bit in tx_olflags,
> > why not to introduce device specific ol_flags in mbuf second cache line?
> > Probably uint16_t would be enough for that.
> >
> > >
> > > > Another question what to do if you would like to create a bonded device over two devices with different NIC types?
> > > > As I understand you can end up in situation when DEV_RX_OFFLOAD_RESERVED_0 would mean different capabilities.
> > > > Why not to have this MACSEC capability and ol_flag value as generic ones, as you have them in previous versions of your patch?
> > >
> > > Those flags are only used in PMD-specific APIs. I don't think we could
> > > use the PMD-specific APIs provided by a certain PMD on a bonded device.
> >
> > I understand that.
> > My question was: suppose user would like to create a bonded device over 2 NICs.
> > One of them is ixgbe, while other would be some other type.
> > In future get_dev_info() for each of them might return DEV_RX_OFFLOAD_RESERVED_0 bit as set.
> > But it would mean completely different thing.
> > How bonded device would know that to deal properly?
> >
> > Another example - user has 2 NICs of different type and would like to send the same packet on both of them simultaneously.
> > As PKT_TX_RESERVED might mean different things for these devices, and user would like to use let say
> > PKT_TX_IXGBE_MACSEC on one of them, he would need to do a copy of them, instead just increment a refcnt.
> >
> > Similar issues might arise at RX handling: user got a packet with PKT_RX_RESERVED_0 set.
> > What does it really mean if there are different NIC types in the system?
> > The only way to answer that question, as I can see, is to keep track from what NIC that packet was received.
> > Which I suppose, is not always convenient.
> >
>
> The main purpose is to put the PMD-specific APIs in a separate
> namespace instead of mixing the PMD-specific APIs and global APIs
> up, and also save the bits in mbuf.ol_flags.
>
> There are other ways to achieve this goal, such as introducing
> the PMD specific ol_flags in mbuf second cache line as you said.
> I just thought defining some reserved bits seems to be the most
> simple way which won't introduce many changes.
>
> What's your suggestions? Should I just revert the changes and
> define the generic flags directly?
Yes, that would be my preference.
As I said above - spending extra bit in ol_flags doesn't look like a big problem to me.
In return there would be no need to consider how to handle all that confusing scenarios in future.
Konstantin
>
> Thanks & regards,
> Tiwei Bie
^ permalink raw reply
* Re: [PATCH v5 2/6] net/mlx5: support basic flow items and actions
From: Ferruh Yigit @ 2017-01-04 17:49 UTC (permalink / raw)
To: Nelio Laranjeiro, dev; +Cc: Adrien Mazarguil
In-Reply-To: <025e66266f9baba7515ad4ba54a4a6addc9e2345.1483022600.git.nelio.laranjeiro@6wind.com>
Hi Nelio,
A quick question.
On 12/29/2016 3:15 PM, Nelio Laranjeiro wrote:
> Introduce initial software for rte_flow rules.
>
> VLAN, VXLAN are still not supported.
>
> Signed-off-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>
> Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
<...>
> +static int
> +priv_flow_validate(struct priv *priv,
> + const struct rte_flow_attr *attr,
> + const struct rte_flow_item items[],
> + const struct rte_flow_action actions[],
> + struct rte_flow_error *error,
> + struct mlx5_flow *flow)
> +{
> + const struct mlx5_flow_items *cur_item = mlx5_flow_items;
<...>
> + for (; items->type != RTE_FLOW_ITEM_TYPE_END; ++items) {
<...>
> + }
> + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; ++actions) {
<...>
> + }
Is it guarantied in somewhere that items or actions terminated with
TYPE_END?
And these fields are direct inputs from user.
Is there a way to verify user provided values are with TYPE_END terminated?
<...>
^ permalink raw reply
* Re: [PATCH v2 2/2] ethdev: clarify xstats Api documentation
From: Thomas Monjalon @ 2017-01-04 18:10 UTC (permalink / raw)
To: Olivier Matz; +Cc: Remy Horton, dev, john.mcnamara, stable
In-Reply-To: <ccad48f5-3ab1-61ec-20f3-9cf890694186@intel.com>
2017-01-03 10:03, Remy Horton:
>
> On 23/12/2016 20:35, Olivier Matz wrote:
> > Reword the Api documentation of xstats ethdev.
> >
> > CC: stable@dpdk.org
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
>
> Acked-by: Remy Horton <remy.horton@intel.com>
Series applied, thanks
^ permalink raw reply
* Re: [PATCH v2 00/25] Generic flow API (rte_flow)
From: Adrien Mazarguil @ 2017-01-04 18:12 UTC (permalink / raw)
To: Simon Horman; +Cc: dev
In-Reply-To: <20170104095347.GA24762@penelope.horms.nl>
On Wed, Jan 04, 2017 at 10:53:50AM +0100, Simon Horman wrote:
> On Thu, Dec 22, 2016 at 01:48:04PM +0100, Adrien Mazarguil wrote:
> > On Wed, Dec 21, 2016 at 05:19:16PM +0100, Simon Horman wrote:
> > > On Fri, Dec 16, 2016 at 05:24:57PM +0100, Adrien Mazarguil wrote:
[...]
> > > I would like to ask some high level questions on the proposal.
> > > I apologise in advance if any of these questions are based on a
> > > misunderstanding on my part.
> > >
> > > * I am wondering about provisions for actions to modify packet data or
> > > metadata. I do see support for marking packets. Is the implication of
> > > this that the main focus is to provide a mechanism for classification
> > > with the assumption that any actions - other than drop and variants of
> > > output - would be performed elsewhere?
> >
> > I'm not sure to understand what you mean by "elsewhere" here. Packet marking
> > as currently defined is a purely ingress action, i.e. HW matches some packet
> > and returns a user-defined tag in related meta-data that the PMD copies to
> > the appropriate mbuf structure field before returning it to the application.
>
> By elsewhere I meant in the application, sorry for being unclear.
OK, then a high level definition would be that PMDs perform all actions part
of a flow rule, and applications are left to handle what they did not
explicitly request to be offloaded.
> > There is provision for egress rules and I wrote down a few ideas describing
> > how they could be useful (as above), however they remain to be defined.
> >
> > > If so I would observe that this seems somewhat limiting in the case of
> > > hardware that can perform a richer set of actions. And seems particularly
> > > limiting on egress as there doesn't seem anywhere else that other actions
> > > could be performed after classification is performed by this API.
> >
> > A single flow rule may contain any number of distinct actions. For egress,
> > it means you could wrap matching packets in VLAN and VXLAN at once.
> >
> > If you wanted to perform the same action twice on matching packets, you'd
> > have to provide two rules with defined priorities and use a non-terminating
> > action for the first one:
> >
> > - Rule with priority 0: match UDP -> add VLAN 42, passthrough
> > - Rule with priority 1: match UDP -> add VLAN 64, terminating
> >
> > This is how automatic QinQ would be defined for outgoing UDP packets.
>
> Ok understood. I have two follow-up questions:
>
> 1. Is the "add VLAN" action included at this time, I was not able to find it
It has not been defined yet. All egress offload actions remain to be
defined.
> 2. Was consideration given to allowing multiple actions in a single rule?
> I see there would be some advantage to that if classification is
> expensive.
Yes, it is supported as described in the documentation (now available
on-line since the series has been applied [3]).
What is not supported however is requesting the same action to be performed
multiple times in a single flow rule, because order is not guaranteed by
design. Actions may all happen simultaneously.
This scenario can be handled with multiple rules using priorities as in my
QinQ example above, or through a new action performing several things at
once in a defined order (e.g. a single "QinQ" action).
> > > * I am curious to know what considerations have been given to supporting support for tunnelling (encapsulation and decapsulation of e.g. VXLAN),
> > > tagging (pushing and popping e.g. VLANs), and labels (pushing or popping
> > > e.g. MPLS).
> > >
> > > Such features seem would useful for application of this work in a variety
> > > of situations including overlay networks and VNFs.
> >
> > This is also what I had in mind and we'd only have to define specific
> > ingress/egress actions for these. Currently rte_flow only implements a basic
> > set of existing features from the legacy filtering framework, but is meant
> > to be extended.
>
> Thanks. I think that answers most of my questions: what I see as missing
> in terms of actions can be added.
>
> > > * I am wondering if any thought has gone into supporting matching on the
> > > n-th instance of a field that may appear more than once: e.g. VLAN tag.
> >
> > Sure, please see the latest documentation [1] and testpmd examples [2].
> > Pattern items being stacked in the same order as protocol layers, maching
> > specific QinQ traffic and redirecting it to some queue could be expressed
> > with something like:
> >
> > testpmd> flow create 0 ingress pattern eth / vlan vid is 64 / vlan vid is 42 / end
> > actions queue 6 / end
> >
> > Such a rule is translated as-is to rte_flow pattern items and action
> > structures.
>
> Thanks, I will look over that.
>
> > > With the above questions in mind I am curious to know what use-cases
> > > the proposal is targeted at.
> >
> > Well, it should be easier to answer if you have a specific use-case in mind
> > you would like to support but that cannot be expressed with the API as
> > defined in [1], in which case please share it with the community.
>
> A use-case would be implementing OvS DPIF flow offload using this API.
OK, OVS has been mentioned several times in this thread and my understanding
is that rte_flow seems to accommodate most of its needs according to people
familiar with it. Perhaps ML archives can answer the remaining questions you
may have about combining rte_flow with OVS.
> > [1] http://dpdk.org/ml/archives/dev/2016-December/052954.html
> > [2] http://dpdk.org/ml/archives/dev/2016-December/052975.html
[3] http://dpdk.org/doc/guides/prog_guide/rte_flow.html
--
Adrien Mazarguil
6WIND
^ permalink raw reply
* Re: [PATCH v5 2/6] net/mlx5: support basic flow items and actions
From: Adrien Mazarguil @ 2017-01-04 18:42 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: Nelio Laranjeiro, dev
In-Reply-To: <2973bbd2-8db3-f9e4-da9a-359e399527b3@intel.com>
Hi Ferruh,
On Wed, Jan 04, 2017 at 05:49:46PM +0000, Ferruh Yigit wrote:
> Hi Nelio,
>
> A quick question.
I'll reply since it's related to the API.
> On 12/29/2016 3:15 PM, Nelio Laranjeiro wrote:
> > Introduce initial software for rte_flow rules.
> >
> > VLAN, VXLAN are still not supported.
> >
> > Signed-off-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>
> > Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
>
> <...>
>
> > +static int
> > +priv_flow_validate(struct priv *priv,
> > + const struct rte_flow_attr *attr,
> > + const struct rte_flow_item items[],
> > + const struct rte_flow_action actions[],
> > + struct rte_flow_error *error,
> > + struct mlx5_flow *flow)
> > +{
> > + const struct mlx5_flow_items *cur_item = mlx5_flow_items;
>
> <...>
>
> > + for (; items->type != RTE_FLOW_ITEM_TYPE_END; ++items) {
> <...>
> > + }
> > + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; ++actions) {
> <...>
> > + }
>
> Is it guarantied in somewhere that items or actions terminated with
> TYPE_END?
Yes, since it's now the only way to terminate items/actions lists [1][2].
There used to be a "max" value in the original draft but it seemed redundant
and proved annoying to use, and was therefore dropped.
END items/actions behave like a NUL terminator for C strings. They are
likewise defined with value 0 for convenience.
> And these fields are direct inputs from user.
> Is there a way to verify user provided values are with TYPE_END terminated?
No, applications must check for its presence (they normally add it
themselves) before feeding these lists to PMDs. I think that's safe enough.
Note the testpmd flow command does not allow entering a flow rule without
"end" tokens in both lists, there is no way around this restriction.
[1] http://dpdk.org/doc/guides/prog_guide/rte_flow.html#matching-pattern
[2] http://dpdk.org/doc/guides/prog_guide/rte_flow.html#actions
--
Adrien Mazarguil
6WIND
^ permalink raw reply
* Re: [PATCH v5 07/17] net/i40e: add flow validate function
From: Ferruh Yigit @ 2017-01-04 18:57 UTC (permalink / raw)
To: Beilei Xing, jingjing.wu, helin.zhang; +Cc: dev
In-Reply-To: <1483500187-124740-8-git-send-email-beilei.xing@intel.com>
On 1/4/2017 3:22 AM, Beilei Xing wrote:
> This patch adds i40e_flow_validation function to check if
> a flow is valid according to the flow pattern.
> i40e_parse_ethertype_filter is added first, it also gets
> the ethertype info.
> i40e_flow.c is added to handle all generic filter events.
>
> Signed-off-by: Beilei Xing <beilei.xing@intel.com>
> ---
<...>
> diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
> index 153322a..edfd52b 100644
> --- a/drivers/net/i40e/i40e_ethdev.c
> +++ b/drivers/net/i40e/i40e_ethdev.c
> @@ -8426,6 +8426,8 @@ i40e_ethertype_filter_handle(struct rte_eth_dev *dev,
> return ret;
> }
>
> +const struct rte_flow_ops i40e_flow_ops;
Is this intentional (instead of using extern) ?
Because i40e_flow.c has a global variable definition with same name, it
looks like this is not causing a build error, but I think confusing.
<...>
> +static int i40e_parse_ethertype_act(struct rte_eth_dev *dev,
> + const struct rte_flow_action *actions,
> + struct rte_flow_error *error,
> + struct rte_eth_ethertype_filter *filter);
In API naming, I would prefer full "action" instead of shorten "act",
but it is your call.
<...>
> +
> +union i40e_filter_t cons_filter;
Why this cons_filter is required. I can see this is saving some state
related rule during validate function.
If the plan is to use this during rule creation, is user has to call
validate before each create?
<...>
> +
> +static int
> +i40e_parse_ethertype_filter(struct rte_eth_dev *dev,
> + const struct rte_flow_attr *attr,
> + const struct rte_flow_item pattern[],
> + const struct rte_flow_action actions[],
> + struct rte_flow_error *error,
> + union i40e_filter_t *filter)
> +{
> + struct rte_eth_ethertype_filter *ethertype_filter =
> + &filter->ethertype_filter;
> + int ret;
> +
> + ret = i40e_parse_ethertype_pattern(dev, pattern, error,
> + ethertype_filter);
> + if (ret)
> + return ret;
> +
> + ret = i40e_parse_ethertype_act(dev, actions, error,
> + ethertype_filter);
> + if (ret)
> + return ret;
> +
> + ret = i40e_parse_attr(attr, error);
It is your call, but I would suggest using a specific namespace for all
rte_flow related functions, something like "i40e_flow_".
In this context it is clear what this function is, but in whole driver
code, the function name is too generic to understand what it does.
> + if (ret)
> + return ret;
> +
> + return ret;
> +}
> +
<...>
> +
> +static int
> +i40e_parse_ethertype_pattern(__rte_unused struct rte_eth_dev *dev,
> + const struct rte_flow_item *pattern,
> + struct rte_flow_error *error,
> + struct rte_eth_ethertype_filter *filter)
I think it is good idea to comment what pattern is recognized in to
function comment, instead of reading code every time to figure out.
> +{
> + const struct rte_flow_item *item = pattern;
> + const struct rte_flow_item_eth *eth_spec;
> + const struct rte_flow_item_eth *eth_mask;
> + enum rte_flow_item_type item_type;
> +
> + for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
> + if (item->last) {
> + rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + item,
> + "Not support range");
> + return -rte_errno;
> + }
> + item_type = item->type;
> + switch (item_type) {
> + case RTE_FLOW_ITEM_TYPE_ETH:
> + eth_spec = (const struct rte_flow_item_eth *)item->spec;
> + eth_mask = (const struct rte_flow_item_eth *)item->mask;
> + /* Get the MAC info. */
> + if (!eth_spec || !eth_mask) {
Why an eth_mask is required?
Can't driver support drop/queue packets from specific src to specific
dst with specific eth_type?
> + rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + item,
> + "NULL ETH spec/mask");
> + return -rte_errno;
> + }
> +
> + /* Mask bits of source MAC address must be full of 0.
> + * Mask bits of destination MAC address must be full
> + * of 1 or full of 0.
> + */
> + if (!is_zero_ether_addr(ð_mask->src) ||
> + (!is_zero_ether_addr(ð_mask->dst) &&
> + !is_broadcast_ether_addr(ð_mask->dst))) {
> + rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + item,
> + "Invalid MAC_addr mask");
> + return -rte_errno;
> + }
> +
> + if ((eth_mask->type & UINT16_MAX) != UINT16_MAX) {
> + rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + item,
> + "Invalid ethertype mask");
Why returning error here?
Can't we say drop packets to specific MAC address, independent from the
ether_type?
> + return -rte_errno;
> + }
> +
> + /* If mask bits of destination MAC address
> + * are full of 1, set RTE_ETHTYPE_FLAGS_MAC.
> + */
> + if (is_broadcast_ether_addr(ð_mask->dst)) {
> + filter->mac_addr = eth_spec->dst;
> + filter->flags |= RTE_ETHTYPE_FLAGS_MAC;
> + } else {
> + filter->flags &= ~RTE_ETHTYPE_FLAGS_MAC;
> + }
> + filter->ether_type = rte_be_to_cpu_16(eth_spec->type);
> +
> + if (filter->ether_type == ETHER_TYPE_IPv4 ||
> + filter->ether_type == ETHER_TYPE_IPv6) {
> + rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ITEM,
> + item,
> + "Unsupported ether_type in"
> + " control packet filter.");
Can't we create a drop rule based on dst MAC address if eth_type is ip ?
> + return -rte_errno;
> + }
> + if (filter->ether_type == ETHER_TYPE_VLAN)
> + PMD_DRV_LOG(WARNING, "filter vlan ether_type in"
> + " first tag is not supported.");
Who is the target of this message?
To the caller, this API is responding as this is supported.
The end user, the user of the application, can see this message, how
this message will help to end user?
> +
> + break;
> + default:
> + break;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int
> +i40e_parse_ethertype_act(struct rte_eth_dev *dev,
> + const struct rte_flow_action *actions,
> + struct rte_flow_error *error,
> + struct rte_eth_ethertype_filter *filter)
I think it would be good to comment this functions to say only DROP and
QUEUE actions are supported.
<...>
> +
> +static int
> +i40e_flow_validate(struct rte_eth_dev *dev,
> + const struct rte_flow_attr *attr,
> + const struct rte_flow_item pattern[],
> + const struct rte_flow_action actions[],
> + struct rte_flow_error *error)
> +{
> + struct rte_flow_item *items; /* internal pattern w/o VOID items */
> + parse_filter_t parse_filter;
> + uint32_t item_num = 0; /* non-void item number of pattern*/
> + uint32_t i = 0;
> + int ret;
> +
> + if (!pattern) {
> + rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
> + NULL, "NULL pattern.");
> + return -rte_errno;
> + }
> +
> + if (!actions) {
> + rte_flow_error_set(error, EINVAL,
> + RTE_FLOW_ERROR_TYPE_ACTION_NUM,
> + NULL, "NULL action.");
> + return -rte_errno;
> + }
It may be good to validate attr too, if it is NULL or not. It is
accessed without check in later stages of the call stack.
<...>
^ permalink raw reply
* Re: [PATCH v2 00/25] Generic flow API (rte_flow)
From: John Fastabend @ 2017-01-04 19:34 UTC (permalink / raw)
To: Adrien Mazarguil, Simon Horman; +Cc: dev
In-Reply-To: <20170104181219.GH12822@6wind.com>
[...]
>>> Well, it should be easier to answer if you have a specific use-case in mind
>>> you would like to support but that cannot be expressed with the API as
>>> defined in [1], in which case please share it with the community.
>>
>> A use-case would be implementing OvS DPIF flow offload using this API.
>
> OK, OVS has been mentioned several times in this thread and my understanding
> is that rte_flow seems to accommodate most of its needs according to people
> familiar with it. Perhaps ML archives can answer the remaining questions you
> may have about combining rte_flow with OVS.
For what its worth. I reviewed this and believe it should be sufficient
to support the OVS SR-IOV offload use case with action/classifier extensions
but without any fundamental changes to the design. We built a prototype
OVS offload on top of another API we dubbed Flow-API a year+ ago and there
seems to be a 1:1 mapping between that older API and the one now in DPDK
so I'm happy. And the missing things seem to fit nicely into extensions.
Also I believe the partial pre-classify use cases should be easily handled
as well although I'm not as familiar with the bit-level details of that
implementation.
At some point capability discovery will be useful but we certainly don't need
those in first iteration and rte_flow doesn't preclude these type of extensions
so that is good.
By the way thanks for doing this work Adrien, glad to see it being accepted
and drivers picking it up.
Thanks,
John
>
>>> [1] http://dpdk.org/ml/archives/dev/2016-December/052954.html
>>> [2] http://dpdk.org/ml/archives/dev/2016-December/052975.html
>
> [3] http://dpdk.org/doc/guides/prog_guide/rte_flow.html
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox