* Re: [PATCH v2 01/18] net/ixgbe: store SYN filter
From: Zhao1, Wei @ 2017-01-04 1:46 UTC (permalink / raw)
To: Dai, Wei, dev@dpdk.org; +Cc: Lu, Wenzhuo
In-Reply-To: <49759EB36A64CF4892C1AFEC9231E8D63A322663@PGSMSX106.gar.corp.intel.com>
Hi, Daiwei
> -----Original Message-----
> From: Dai, Wei
> Sent: Tuesday, January 3, 2017 10:33 PM
> To: Zhao1, Wei <wei.zhao1@intel.com>; dev@dpdk.org
> Cc: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Zhao1, Wei
> <wei.zhao1@intel.com>
> Subject: RE: [dpdk-dev] [PATCH v2 01/18] net/ixgbe: store SYN filter
>
> Hi, Wei Zhao
>
> I think that you had better give a cover letter for such a series of patches.
> You can give the changes between v2 and v1 in cover letter and maybe no
> need describe it in each one.
>
Ok, I will add cover letter in v3
> Thanks &Best Regards
> -Wei
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wei Zhao
> > Sent: Friday, December 30, 2016 3:53 PM
> > To: dev@dpdk.org
> > Cc: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Zhao1, Wei
> > <wei.zhao1@intel.com>
> > Subject: [dpdk-dev] [PATCH v2 01/18] net/ixgbe: store SYN filter
> >
> > Add support for storing SYN filter in SW.
> >
> > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
> > ---
> >
> > v2:
> > --synqf assignment location change
> > ---
> > drivers/net/ixgbe/ixgbe_ethdev.c | 14 +++++++++++---
> > drivers/net/ixgbe/ixgbe_ethdev.h | 2 ++
> > 2 files changed, 13 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> > b/drivers/net/ixgbe/ixgbe_ethdev.c
> > index a25bac8..316e560 100644
> > --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> > @@ -1274,6 +1274,8 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev)
> > memset(filter_info->fivetuple_mask, 0,
> > sizeof(uint32_t) * IXGBE_5TUPLE_ARRAY_SIZE);
> >
> > + /* initialize SYN filter */
> > + filter_info->syn_info = 0;
> > return 0;
> > }
> >
> > @@ -5580,15 +5582,18 @@ ixgbe_syn_filter_set(struct rte_eth_dev *dev,
> > bool add)
> > {
> > struct ixgbe_hw *hw =
> > IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> > + struct ixgbe_filter_info *filter_info =
> > + IXGBE_DEV_PRIVATE_TO_FILTER_INFO(dev->data-
> >dev_private);
> > + uint32_t syn_info;
> > uint32_t synqf;
> >
> > if (filter->queue >= IXGBE_MAX_RX_QUEUE_NUM)
> > return -EINVAL;
> >
> > - synqf = IXGBE_READ_REG(hw, IXGBE_SYNQF);
> > + syn_info = filter_info->syn_info;
> >
> > if (add) {
> > - if (synqf & IXGBE_SYN_FILTER_ENABLE)
> > + if (syn_info & IXGBE_SYN_FILTER_ENABLE)
> > return -EINVAL;
> > synqf = (uint32_t)(((filter->queue <<
> > IXGBE_SYN_FILTER_QUEUE_SHIFT) &
> > IXGBE_SYN_FILTER_QUEUE) |
> IXGBE_SYN_FILTER_ENABLE); @@ -5598,10
> > +5603,13 @@ ixgbe_syn_filter_set(struct rte_eth_dev *dev,
> > else
> > synqf &= ~IXGBE_SYN_FILTER_SYNQFP;
> > } else {
> > - if (!(synqf & IXGBE_SYN_FILTER_ENABLE))
> > + synqf = IXGBE_READ_REG(hw, IXGBE_SYNQF);
> > + if (!(syn_info & IXGBE_SYN_FILTER_ENABLE))
> > return -ENOENT;
> > synqf &= ~(IXGBE_SYN_FILTER_QUEUE |
> IXGBE_SYN_FILTER_ENABLE);
> > }
> > +
> > + filter_info->syn_info = synqf;
> > IXGBE_WRITE_REG(hw, IXGBE_SYNQF, synqf);
> > IXGBE_WRITE_FLUSH(hw);
> > return 0;
> > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h
> > b/drivers/net/ixgbe/ixgbe_ethdev.h
> > index 4ff6338..827026c 100644
> > --- a/drivers/net/ixgbe/ixgbe_ethdev.h
> > +++ b/drivers/net/ixgbe/ixgbe_ethdev.h
> > @@ -262,6 +262,8 @@ struct ixgbe_filter_info {
> > /* Bit mask for every used 5tuple filter */
> > uint32_t fivetuple_mask[IXGBE_5TUPLE_ARRAY_SIZE];
> > struct ixgbe_5tuple_filter_list fivetuple_list;
> > + /* store the SYN filter info */
> > + uint32_t syn_info;
> > };
> >
> > /*
> > --
> > 2.5.5
^ permalink raw reply
* [PATCH v2] eal: Support running as unprivileged user
From: Ben Walker @ 2017-01-03 22:56 UTC (permalink / raw)
To: dev; +Cc: Ben Walker
In-Reply-To: <20170103223033.26313-1-benjamin.walker@intel.com>
For Linux kernel 4.0 and newer, the ability to obtain
physical page frame numbers for unprivileged users from
/proc/self/pagemap was removed. Instead, when an IOMMU
is present, simply choose our own DMA addresses instead.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
---
lib/librte_eal/common/eal_private.h | 12 +++++
lib/librte_eal/linuxapp/eal/eal_memory.c | 75 +++++++++++++++++++++++---------
lib/librte_eal/linuxapp/eal/eal_pci.c | 6 ++-
3 files changed, 71 insertions(+), 22 deletions(-)
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 9e7d8f6..8b2d323 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -34,6 +34,7 @@
#ifndef _EAL_PRIVATE_H_
#define _EAL_PRIVATE_H_
+#include <stdbool.h>
#include <stdio.h>
#include <rte_pci.h>
@@ -301,4 +302,15 @@ int rte_eal_hugepage_init(void);
*/
int rte_eal_hugepage_attach(void);
+/**
+ * Returns true if the system is able to obtain
+ * physical addresses. Return false if using DMA
+ * addresses through an IOMMU.
+ *
+ * Drivers based on uio will not load unless physical
+ * addresses are obtainable. It is only possible to get
+ * physical addresses when running as a privileged user.
+ */
+bool rte_eal_using_phys_addrs(void);
+
#endif /* _EAL_PRIVATE_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index a956bb2..8678ae9 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -64,6 +64,7 @@
#define _FILE_OFFSET_BITS 64
#include <errno.h>
#include <stdarg.h>
+#include <stdbool.h>
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
@@ -122,26 +123,24 @@ int rte_xen_dom0_supported(void)
static uint64_t baseaddr_offset;
-static unsigned proc_pagemap_readable;
+static bool phys_addrs_available = true;
#define RANDOMIZE_VA_SPACE_FILE "/proc/sys/kernel/randomize_va_space"
static void
-test_proc_pagemap_readable(void)
+test_phys_addrs_available(void)
{
- int fd = open("/proc/self/pagemap", O_RDONLY);
+ uint64_t tmp;
+ phys_addr_t physaddr;
- if (fd < 0) {
+ physaddr = rte_mem_virt2phy(&tmp);
+ if (physaddr == RTE_BAD_PHYS_ADDR) {
RTE_LOG(ERR, EAL,
- "Cannot open /proc/self/pagemap: %s. "
- "virt2phys address translation will not work\n",
+ "Cannot obtain physical addresses: %s. "
+ "Only vfio will function.\n",
strerror(errno));
- return;
+ phys_addrs_available = false;
}
-
- /* Is readable */
- close(fd);
- proc_pagemap_readable = 1;
}
/* Lock page in physical memory and prevent from swapping. */
@@ -190,7 +189,7 @@ rte_mem_virt2phy(const void *virtaddr)
}
/* Cannot parse /proc/self/pagemap, no need to log errors everywhere */
- if (!proc_pagemap_readable)
+ if (!phys_addrs_available)
return RTE_BAD_PHYS_ADDR;
/* standard page size */
@@ -229,6 +228,9 @@ rte_mem_virt2phy(const void *virtaddr)
* the pfn (page frame number) are bits 0-54 (see
* pagemap.txt in linux Documentation)
*/
+ if ((page & 0x7fffffffffffffULL) == 0)
+ return RTE_BAD_PHYS_ADDR;
+
physaddr = ((page & 0x7fffffffffffffULL) * page_size)
+ ((unsigned long)virtaddr % page_size);
@@ -242,7 +244,7 @@ rte_mem_virt2phy(const void *virtaddr)
static int
find_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
{
- unsigned i;
+ unsigned int i;
phys_addr_t addr;
for (i = 0; i < hpi->num_pages[0]; i++) {
@@ -255,6 +257,22 @@ find_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
}
/*
+ * For each hugepage in hugepg_tbl, fill the physaddr value sequentially.
+ */
+static int
+set_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
+{
+ unsigned int i;
+ phys_addr_t addr = 0;
+
+ for (i = 0; i < hpi->num_pages[0]; i++) {
+ hugepg_tbl[i].physaddr = addr;
+ addr += hugepg_tbl[i].size;
+ }
+ return 0;
+}
+
+/*
* Check whether address-space layout randomization is enabled in
* the kernel. This is important for multi-process as it can prevent
* two processes mapping data to the same virtual address
@@ -951,7 +969,7 @@ rte_eal_hugepage_init(void)
int nr_hugefiles, nr_hugepages = 0;
void *addr;
- test_proc_pagemap_readable();
+ test_phys_addrs_available();
memset(used_hp, 0, sizeof(used_hp));
@@ -1043,11 +1061,22 @@ rte_eal_hugepage_init(void)
continue;
}
- /* find physical addresses and sockets for each hugepage */
- if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0){
- RTE_LOG(DEBUG, EAL, "Failed to find phys addr for %u MB pages\n",
- (unsigned)(hpi->hugepage_sz / 0x100000));
- goto fail;
+ if (phys_addrs_available) {
+ /* find physical addresses for each hugepage */
+ if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0) {
+ RTE_LOG(DEBUG, EAL, "Failed to find phys addr "
+ "for %u MB pages\n",
+ (unsigned int)(hpi->hugepage_sz / 0x100000));
+ goto fail;
+ }
+ } else {
+ /* set physical addresses for each hugepage */
+ if (set_physaddrs(&tmp_hp[hp_offset], hpi) < 0) {
+ RTE_LOG(DEBUG, EAL, "Failed to set phys addr "
+ "for %u MB pages\n",
+ (unsigned int)(hpi->hugepage_sz / 0x100000));
+ goto fail;
+ }
}
if (find_numasocket(&tmp_hp[hp_offset], hpi) < 0){
@@ -1289,7 +1318,7 @@ rte_eal_hugepage_attach(void)
"into secondary processes\n");
}
- test_proc_pagemap_readable();
+ test_phys_addrs_available();
if (internal_config.xen_dom0_support) {
#ifdef RTE_LIBRTE_XEN_DOM0
@@ -1426,3 +1455,9 @@ rte_eal_hugepage_attach(void)
close(fd_hugepage);
return -1;
}
+
+bool
+rte_eal_using_phys_addrs(void)
+{
+ return phys_addrs_available;
+}
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 4350134..cfbfedf 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -99,8 +99,10 @@ rte_eal_pci_map_device(struct rte_pci_device *dev)
break;
case RTE_KDRV_IGB_UIO:
case RTE_KDRV_UIO_GENERIC:
- /* map resources for devices that use uio */
- ret = pci_uio_map_resource(dev);
+ if (rte_eal_using_phys_addrs()) {
+ /* map resources for devices that use uio */
+ ret = pci_uio_map_resource(dev);
+ }
break;
default:
RTE_LOG(DEBUG, EAL,
--
2.9.3
^ permalink raw reply related
* Re: Running DPDK as an unprivileged user
From: Walker, Benjamin @ 2017-01-03 22:50 UTC (permalink / raw)
To: stephen@networkplumber.org, thomas.monjalon@6wind.com; +Cc: dev@dpdk.org
In-Reply-To: <20170102114709.0a2f5546@xeon-e3>
On Thu, 2016-12-29 at 17:14 -0800, Stephen Hemminger wrote:
> If kernel broke pinning of hugepages, then it is an upstream kernel bug.
The kernel, under a myriad of circumstances, will change the mapping of virtual
to physical addresses for hugepages. This behavior began somewhere around kernel
3.16 and with each release more cases where the mapping can change are
introduced. DPDK should not be relying on that mapping staying static, and
instead should be using vfio to explicitly pin the pages. I've consulted the
relevant kernel developers who write the code in this area and they are
universally in agreement that this is not a kernel bug and the mappings will get
less static over time.
On Mon, 2017-01-02 at 11:47 -0800, Stephen Hemminger wrote:
> On Mon, 02 Jan 2017 15:32:08 +0100
> Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
>
> > 2016-12-29 17:14, Stephen Hemminger:
> > > On Thu, 29 Dec 2016 20:41:21 +0000
> > > "Walker, Benjamin" <benjamin.walker@intel.com> wrote:
> > > > My second question is whether the user should be allowed to
> > > > mix uio and vfio usage simultaneously. For vfio, the
> > > > physical addresses are really DMA addresses and are best
> > > > when arbitrarily chosen to appear sequential relative to
> > > > their virtual addresses. For uio, they are physical
> > > > addresses and are not chosen at all. It seems that these two
> > > > things are in conflict and that it will be difficult, ugly,
> > > > and maybe impossible to resolve the simultaneous use of
> > > > both.
> > >
> > > Unless application is running as privileged user (ie root), UIO
> > > is not going to work. Therefore don't worry about mixed environment.
> >
> > Yes, mixing UIO and VFIO is possible only as root.
> > However, what is the benefit of mixing them?
>
> One possible case where this could be used, Hyper-V/Azure and SR-IOV.
> The VF interface will show up on an isolated PCI bus and the virtual NIC
> is on VMBUS. It is possible to use VFIO on the PCI to get MSI-X per queue
> interrupts, but there is no support for VFIO on VMBUS.
I sent out a patch a little while ago that makes DPDK work when running as an
unprivileged user with an IOMMU. I allow mixing of uio/vfio when root (I choose
the DMA address to be the physical address), but only vfio when unprivileged (I
choose the DMA addresses to start at 0).
Unfortunately, there are a few more wrinkles for systems that do not have an
IOMMU. These systems still need to explicitly pin memory, but they need to use
physical addresses instead of DMA addresses. There are two concerns with this:
1) Physical addresses cannot be exposed to unprivileged users due to security
concerns (the fallout of rowhammer). Therefore, systems without an IOMMU can
only support privileged users. I think this is probably fine.
2) The IOCTL from vfio to pin the memory is tied to specifying the DMA address
and programming the IOMMU. This is unfortunate - systems without an IOMMU still
want to do the pinning, but they need to be given the physical address instead
of specifying a DMA address.
3) Not all device types, particularly in virtualization environments, support
vfio today. These devices have no way to explicitly pin memory.
I think this is going to take a kernel patch or two to resolve, unless someone
has a good idea.
^ permalink raw reply
* [PATCH] eal: Support running as unprivileged user
From: Ben Walker @ 2017-01-03 22:30 UTC (permalink / raw)
To: dev; +Cc: Ben Walker
For Linux kernel 4.0 and newer, the ability to obtain
physical page frame numbers for unprivileged users from
/proc/self/pagemap was removed. Instead, when an IOMMU
is present, simply choose our own DMA addresses instead.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
---
lib/librte_eal/common/eal_private.h | 12 ++++++
lib/librte_eal/linuxapp/eal/eal_memory.c | 71 +++++++++++++++++++++++---------
lib/librte_eal/linuxapp/eal/eal_pci.c | 6 ++-
3 files changed, 68 insertions(+), 21 deletions(-)
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 9e7d8f6..8b2d323 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -34,6 +34,7 @@
#ifndef _EAL_PRIVATE_H_
#define _EAL_PRIVATE_H_
+#include <stdbool.h>
#include <stdio.h>
#include <rte_pci.h>
@@ -301,4 +302,15 @@ int rte_eal_hugepage_init(void);
*/
int rte_eal_hugepage_attach(void);
+/**
+ * Returns true if the system is able to obtain
+ * physical addresses. Return false if using DMA
+ * addresses through an IOMMU.
+ *
+ * Drivers based on uio will not load unless physical
+ * addresses are obtainable. It is only possible to get
+ * physical addresses when running as a privileged user.
+ */
+bool rte_eal_using_phys_addrs(void);
+
#endif /* _EAL_PRIVATE_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index a956bb2..33c66c1 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -64,6 +64,7 @@
#define _FILE_OFFSET_BITS 64
#include <errno.h>
#include <stdarg.h>
+#include <stdbool.h>
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
@@ -122,26 +123,24 @@ int rte_xen_dom0_supported(void)
static uint64_t baseaddr_offset;
-static unsigned proc_pagemap_readable;
+static bool phys_addrs_available = true;
#define RANDOMIZE_VA_SPACE_FILE "/proc/sys/kernel/randomize_va_space"
static void
-test_proc_pagemap_readable(void)
+test_phys_addrs_available(void)
{
- int fd = open("/proc/self/pagemap", O_RDONLY);
+ uint64_t tmp;
+ phys_addr_t physaddr;
- if (fd < 0) {
+ physaddr = rte_mem_virt2phy(&tmp);
+ if (physaddr == RTE_BAD_PHYS_ADDR) {
RTE_LOG(ERR, EAL,
- "Cannot open /proc/self/pagemap: %s. "
- "virt2phys address translation will not work\n",
+ "Cannot obtain physical addresses: %s. "
+ "Only vfio will function.\n",
strerror(errno));
- return;
+ phys_addrs_available = false;
}
-
- /* Is readable */
- close(fd);
- proc_pagemap_readable = 1;
}
/* Lock page in physical memory and prevent from swapping. */
@@ -190,7 +189,7 @@ rte_mem_virt2phy(const void *virtaddr)
}
/* Cannot parse /proc/self/pagemap, no need to log errors everywhere */
- if (!proc_pagemap_readable)
+ if (!phys_addrs_available)
return RTE_BAD_PHYS_ADDR;
/* standard page size */
@@ -229,6 +228,9 @@ rte_mem_virt2phy(const void *virtaddr)
* the pfn (page frame number) are bits 0-54 (see
* pagemap.txt in linux Documentation)
*/
+ if ((page & 0x7fffffffffffffULL) == 0)
+ return RTE_BAD_PHYS_ADDR;
+
physaddr = ((page & 0x7fffffffffffffULL) * page_size)
+ ((unsigned long)virtaddr % page_size);
@@ -255,6 +257,22 @@ find_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
}
/*
+ * For each hugepage in hugepg_tbl, fill the physaddr value sequentially.
+ */
+static int
+set_physaddrs(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
+{
+ unsigned i;
+ phys_addr_t addr = 0;
+
+ for (i = 0; i < hpi->num_pages[0]; i++) {
+ hugepg_tbl[i].physaddr = addr;
+ addr += hugepg_tbl[i].size;
+ }
+ return 0;
+}
+
+/*
* Check whether address-space layout randomization is enabled in
* the kernel. This is important for multi-process as it can prevent
* two processes mapping data to the same virtual address
@@ -951,7 +969,7 @@ rte_eal_hugepage_init(void)
int nr_hugefiles, nr_hugepages = 0;
void *addr;
- test_proc_pagemap_readable();
+ test_phys_addrs_available();
memset(used_hp, 0, sizeof(used_hp));
@@ -1043,11 +1061,20 @@ rte_eal_hugepage_init(void)
continue;
}
- /* find physical addresses and sockets for each hugepage */
- if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0){
- RTE_LOG(DEBUG, EAL, "Failed to find phys addr for %u MB pages\n",
- (unsigned)(hpi->hugepage_sz / 0x100000));
- goto fail;
+ if (phys_addrs_available) {
+ /* find physical addresses for each hugepage */
+ if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0){
+ RTE_LOG(DEBUG, EAL, "Failed to find phys addr for %u MB pages\n",
+ (unsigned)(hpi->hugepage_sz / 0x100000));
+ goto fail;
+ }
+ } else {
+ /* set physical addresses for each hugepage */
+ if (set_physaddrs(&tmp_hp[hp_offset], hpi) < 0) {
+ RTE_LOG(DEBUG, EAL, "Failed to set phys addr for %u MB pages\n",
+ (unsigned)(hpi->hugepage_sz / 0x100000));
+ goto fail;
+ }
}
if (find_numasocket(&tmp_hp[hp_offset], hpi) < 0){
@@ -1289,7 +1316,7 @@ rte_eal_hugepage_attach(void)
"into secondary processes\n");
}
- test_proc_pagemap_readable();
+ test_phys_addrs_available();
if (internal_config.xen_dom0_support) {
#ifdef RTE_LIBRTE_XEN_DOM0
@@ -1426,3 +1453,9 @@ rte_eal_hugepage_attach(void)
close(fd_hugepage);
return -1;
}
+
+bool
+rte_eal_using_phys_addrs(void)
+{
+ return phys_addrs_available;
+}
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 4350134..cfbfedf 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -99,8 +99,10 @@ rte_eal_pci_map_device(struct rte_pci_device *dev)
break;
case RTE_KDRV_IGB_UIO:
case RTE_KDRV_UIO_GENERIC:
- /* map resources for devices that use uio */
- ret = pci_uio_map_resource(dev);
+ if (rte_eal_using_phys_addrs()) {
+ /* map resources for devices that use uio */
+ ret = pci_uio_map_resource(dev);
+ }
break;
default:
RTE_LOG(DEBUG, EAL,
--
2.9.3
^ permalink raw reply related
* Re: [PATCH v5 00/12] Introducing EAL Bus-Device-Driver Model
From: Thomas Monjalon @ 2017-01-03 22:22 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-1-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:53, Shreyansh Jain:
> Link to v1: [10]
> Link to v2: [11]
> Link to v3: [13]
> Link to v4: [14]
>
> :: Introduction ::
>
> DPDK has been inherently a PCI inclined framework. Because of this, the
> design of device tree (or list) within DPDK is also PCI inclined. A
> non-PCI device doesn't have a way of being expressed without using hooks
> started from EAL to PMD.
It is a very important work to make DPDK growing.
I am sorry to not have done a lot of public comments before today.
I have sent some thoughts about moving some things from generic objects to
specialized ones. I think they are not so much big changes in your work
and I hope we could converge to something in the git tree really soon.
Thanks Shreyansh.
PS: reviews from others are more than welcome!
^ permalink raw reply
* Re: [PATCH v5 12/12] eal/bus: add bus iteration macros
From: Thomas Monjalon @ 2017-01-03 22:15 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-13-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:54, Shreyansh Jain:
> Three macros:
> FOREACH_BUS
> FOREACH_DEVICE_ON_BUS
> FOREACH_DRIVER_ON_BUS
> are introduced to make looping over bus (on global list), devices and
> drivers (on a specific bus) prettier.
Nice
^ permalink raw reply
* Re: [PATCH v5 08/12] eal/pci: generalize args of PCI scan/match towards RTE device/driver
From: Thomas Monjalon @ 2017-01-03 22:13 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-9-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:54, Shreyansh Jain:
> PCI scan and match now work on rte_device/rte_driver rather than PCI
> specific objects. These functions can now be plugged to the generic
> bus callbacks for scanning and matching devices/drivers.
These sentences looks weird :)
PCI functions must work with PCI objects, it's simpler.
However I agree to register PCI scan, match, init and remove functions
with the generic rte_bus. Then the rte_device object is casted into
rte_pci_device inside these functions.
^ permalink raw reply
* Re: [PATCH v5 07/12] pci: split match and probe function
From: Thomas Monjalon @ 2017-01-03 22:08 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-8-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:54, Shreyansh Jain:
> --- a/lib/librte_eal/common/include/rte_pci.h
> +++ b/lib/librte_eal/common/include/rte_pci.h
> @@ -373,6 +373,21 @@ rte_eal_compare_pci_addr(const struct rte_pci_addr *addr,
> int rte_eal_pci_scan(void);
>
> /**
> + * Match the PCI Driver and Device using the ID Table
> + *
> + * @param pci_drv
> + * PCI driver from which ID table would be extracted
> + * @param pci_dev
> + * PCI device to match against the driver
> + * @return
> + * 0 for successful match
> + * !0 for unsuccessful match
> + */
> +int
> +rte_eal_pci_match(struct rte_pci_driver *pci_drv,
> + struct rte_pci_device *pci_dev);
Yes we definitely need this function.
^ permalink raw reply
* Re: [PATCH v5 05/12] eal: add probe and remove support for rte_driver
From: Thomas Monjalon @ 2017-01-03 22:05 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-6-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:53, Shreyansh Jain:
> --- a/lib/librte_eal/common/include/rte_dev.h
> +++ b/lib/librte_eal/common/include/rte_dev.h
> @@ -152,6 +162,8 @@ struct rte_driver {
> struct rte_bus *bus; /**< Bus serviced by this driver */
> const char *name; /**< Driver name. */
> const char *alias; /**< Driver alias. */
> + driver_probe_t *probe; /**< Probe the device */
> + driver_remove_t *remove; /**< Remove/hotplugging the device */
> };
If I understand well, this probe function does neither scan nor match.
So it could be named init.
I think the probe (init) and remove ops must be specific to the bus.
We can have them in rte_bus, and as an example, the pci implementation
would call the pci probe and remove ops of rte_pci_driver.
Please use rte_ prefix in public headers.
^ permalink raw reply
* Re: [PATCH v5 01/12] eal/bus: introduce bus abstraction
From: Thomas Monjalon @ 2017-01-03 21:52 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-2-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:53, Shreyansh Jain:
> +DPDK_17.02 {
> + global:
> +
> + rte_bus_list;
> + rte_eal_bus_add_device;
> + rte_eal_bus_add_driver;
> + rte_eal_bus_get;
> + rte_eal_bus_dump;
> + rte_eal_bus_register;
> + rte_eal_bus_insert_device;
> + rte_eal_bus_remove_device;
> + rte_eal_bus_remove_driver;
> + rte_eal_bus_unregister;
I think the prefix can be just rte_bus_ instead of rte_eal_bus_.
> +/** Double linked list of buses */
> +TAILQ_HEAD(rte_bus_list, rte_bus);
> +
> +/* Global Bus list */
> +extern struct rte_bus_list rte_bus_list;
Why the bus list is public?
> +/**
> + * A structure describing a generic bus.
> + */
> +struct rte_bus {
> + TAILQ_ENTRY(rte_bus) next; /**< Next bus object in linked list */
> + struct rte_driver_list driver_list;
> + /**< List of all drivers on bus */
> + struct rte_device_list device_list;
> + /**< List of all devices on bus */
> + const char *name; /**< Name of the bus */
> +};
I am not convinced we should link a generic bus to drivers and devices.
What do you think of having rte_pci_bus being a rte_bus and linking
with rte_pci_device and rte_pci_driver lists?
I'm thinking to something like that:
struct rte_bus {
TAILQ_ENTRY(rte_bus) next;
const char *name;
rte_bus_scan_t scan;
rte_bus_match_t match;
};
struct rte_pci_bus {
struct rte_bus bus;
struct rte_pci_driver_list pci_drivers;
struct rte_pci_device_list pci_devices;
};
> +/** Helper for Bus registration. The constructor has higher priority than
> + * PMD constructors
> + */
> +#define RTE_REGISTER_BUS(nm, bus) \
> +static void __attribute__((constructor(101), used)) businitfn_ ##nm(void) \
> +{\
> + (bus).name = RTE_STR(nm);\
> + rte_eal_bus_register(&bus); \
> +}
By removing the lists from rte_bus as suggested above, do you still need
a priority for this constructor?
> struct rte_device {
> TAILQ_ENTRY(rte_device) next; /**< Next device */
> + struct rte_bus *bus; /**< Device connected to this bus */
> const struct rte_driver *driver;/**< Associated driver */
> int numa_node; /**< NUMA node connection */
> struct rte_devargs *devargs; /**< Device user arguments */
> @@ -148,6 +149,7 @@ void rte_eal_device_remove(struct rte_device *dev);
> */
> struct rte_driver {
> TAILQ_ENTRY(rte_driver) next; /**< Next in list. */
> + struct rte_bus *bus; /**< Bus serviced by this driver */
> const char *name; /**< Driver name. */
> const char *alias; /**< Driver alias. */
> };
Do we need to know the bus associated to a driver in rte_driver?
Bus and driver are already associated in rte_device.
^ permalink raw reply
* Re: [PATCH v5 04/12] eal: integrate bus scan and probe with EAL
From: Thomas Monjalon @ 2017-01-03 21:46 UTC (permalink / raw)
To: Shreyansh Jain; +Cc: david.marchand, dev
In-Reply-To: <1482758645-23057-5-git-send-email-shreyansh.jain@nxp.com>
2016-12-26 18:53, Shreyansh Jain:
> --- a/lib/librte_eal/linuxapp/eal/eal.c
> +++ b/lib/librte_eal/linuxapp/eal/eal.c
> @@ -844,6 +845,9 @@ rte_eal_init(int argc, char **argv)
> if (rte_eal_intr_init() < 0)
> rte_panic("Cannot init interrupt-handling thread\n");
>
> + if (rte_eal_bus_scan())
> + rte_panic("Cannot scan the buses for devices\n");
Yes, definitely. Just one scan functions which scan registered bus.
> @@ -884,6 +888,9 @@ rte_eal_init(int argc, char **argv)
> if (rte_eal_pci_probe())
> rte_panic("Cannot probe PCI\n");
>
> + if (rte_eal_bus_probe())
> + rte_panic("Cannot probe devices\n");
> +
> if (rte_eal_dev_init() < 0)
> rte_panic("Cannot init pmd devices\n");
What is the benefit of initializing (probe) a device outside of the scan?
Currently, it is done in two steps, so you are keeping the same behaviour.
I imagine a model where the scan function decide to initialize the
device and can require some help from a callback to make this decision.
So the whitelist/blacklist policy can be implemented with callbacks at
the scan level and possibly the responsibility of the application.
Note that the callback model would be a change for a next release.
^ permalink raw reply
* [PATCH 4/4] net/af_packet: add 802.1Q (VLAN) support
From: Charles (Chas) Williams @ 2017-01-03 20:26 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
In-Reply-To: <1483475171-20387-1-git-send-email-ciwillia@brocade.com>
AF_PACKET has some flags to check on the receive side for 802.1Q
information. If present, we copy into the mbuf. For transmit, we
insert any 802.1Q information into the packet before copying to the ring.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index b01a8d1..7f1df92 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -161,6 +161,12 @@ eth_af_packet_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
pbuf = (uint8_t *) ppd + ppd->tp_mac;
memcpy(rte_pktmbuf_mtod(mbuf, void *), pbuf, rte_pktmbuf_data_len(mbuf));
+ /* check for vlan info */
+ if (ppd->tp_status & TP_STATUS_VLAN_VALID) {
+ mbuf->vlan_tci = ppd->tp_vlan_tci;
+ mbuf->ol_flags |= (PKT_RX_VLAN_PKT | PKT_RX_VLAN_STRIPPED);
+ }
+
/* release incoming frame and advance ring buffer */
ppd->tp_status = TP_STATUS_KERNEL;
if (++framenum >= framecount)
@@ -214,6 +220,14 @@ eth_af_packet_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
continue;
}
+ /* insert vlan info if necessary */
+ if (mbuf->ol_flags & PKT_TX_VLAN_PKT) {
+ if (rte_vlan_insert(&mbuf)) {
+ rte_pktmbuf_free(mbuf);
+ continue;
+ }
+ }
+
/* point at the next incoming frame */
if ((ppd->tp_status != TP_STATUS_AVAILABLE) &&
(poll(&pfd, 1, -1) < 0))
--
2.1.4
^ permalink raw reply related
* [PATCH 3/4] net/af_packet: promisicuous support
From: Charles (Chas) Williams @ 2017-01-03 20:26 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
In-Reply-To: <1483475171-20387-1-git-send-email-ciwillia@brocade.com>
Add promiscuous support to the AF_PACKET PMD. The underlying linux
device's IF_PROMISC flag is toggled to enable or disable.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 40 +++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index d8ac5c6..b01a8d1 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -441,6 +441,44 @@ eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)
return 0;
}
+static void
+eth_dev_change_flags(char *if_name, uint32_t flags, uint32_t mask)
+{
+ struct ifreq ifr;
+ int s;
+
+ s = socket(PF_INET, SOCK_DGRAM, 0);
+ if (s < 0)
+ return;
+
+ strncpy(ifr.ifr_name, if_name, IFNAMSIZ);
+ if (ioctl(s, SIOCGIFFLAGS, &ifr) < 0)
+ goto out;
+ ifr.ifr_flags &= mask;
+ ifr.ifr_flags |= flags;
+ if (ioctl(s, SIOCSIFFLAGS, &ifr) < 0)
+ goto out;
+out:
+ close(s);
+ return;
+}
+
+static void
+eth_dev_promiscuous_enable(struct rte_eth_dev *dev)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+
+ eth_dev_change_flags(internals->if_name, IFF_PROMISC, ~0);
+}
+
+static void
+eth_dev_promiscuous_disable(struct rte_eth_dev *dev)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+
+ eth_dev_change_flags(internals->if_name, 0, ~IFF_PROMISC);
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -448,6 +486,8 @@ static const struct eth_dev_ops ops = {
.dev_configure = eth_dev_configure,
.dev_infos_get = eth_dev_info,
.mtu_set = eth_dev_mtu_set,
+ .promiscuous_disable = eth_dev_promiscuous_disable,
+ .promiscuous_enable = eth_dev_promiscuous_enable,
.rx_queue_setup = eth_rx_queue_setup,
.tx_queue_setup = eth_tx_queue_setup,
.rx_queue_release = eth_queue_release,
--
2.1.4
^ permalink raw reply related
* [PATCH 2/4] net/af_packet: add support to change mtu
From: Charles (Chas) Williams @ 2017-01-03 20:26 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
In-Reply-To: <1483475171-20387-1-git-send-email-ciwillia@brocade.com>
The underlying linux device's MTU is changed subject to the frame size
limitations during device creation.
Signed-off-by: Charles (Chas) Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index 5541fd7..d8ac5c6 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -413,12 +413,41 @@ eth_tx_queue_setup(struct rte_eth_dev *dev,
return 0;
}
+static int
+eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)
+{
+ struct pmd_internals *internals = dev->data->dev_private;
+ struct ifreq ifr = { .ifr_mtu = mtu };
+ int ret;
+ int s;
+ unsigned int data_size = internals->req.tp_frame_size -
+ TPACKET2_HDRLEN -
+ sizeof(struct sockaddr_ll);
+
+ if (mtu > data_size)
+ return -EINVAL;
+
+ s = socket(PF_INET, SOCK_DGRAM, 0);
+ if (s < 0)
+ return -EINVAL;
+
+ strncpy(ifr.ifr_name, internals->if_name, IFNAMSIZ);
+ ret = ioctl(s, SIOCSIFMTU, &ifr);
+ close(s);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
.dev_close = eth_dev_close,
.dev_configure = eth_dev_configure,
.dev_infos_get = eth_dev_info,
+ .mtu_set = eth_dev_mtu_set,
.rx_queue_setup = eth_rx_queue_setup,
.tx_queue_setup = eth_tx_queue_setup,
.rx_queue_release = eth_queue_release,
--
2.1.4
^ permalink raw reply related
* [PATCH 1/4] net/af_packet: add iface name to internals
From: Charles (Chas) Williams @ 2017-01-03 20:26 UTC (permalink / raw)
To: dev; +Cc: linville, Charles (Chas) Williams
This will be used by later changes to determine the underlying linux
interface.
Signed-off-by: Chas Williams <ciwillia@brocade.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index a1e13ff..5541fd7 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -99,6 +99,7 @@ struct pmd_internals {
unsigned nb_queues;
int if_index;
+ char *if_name;
struct ether_addr eth_addr;
struct tpacket_req req;
@@ -533,6 +534,7 @@ rte_pmd_init_internals(const char *name,
name);
goto error_early;
}
+ (*internals)->if_name = strdup(pair->value);
(*internals)->if_index = ifr.ifr_ifindex;
if (ioctl(sockfd, SIOCGIFHWADDR, &ifr) == -1) {
@@ -724,6 +726,7 @@ rte_pmd_init_internals(const char *name,
((*internals)->rx_queue[q].sockfd != qsockfd))
close((*internals)->rx_queue[q].sockfd);
}
+ rte_free((*internals)->if_name);
rte_free(*internals);
error_early:
rte_free(data);
@@ -892,6 +895,7 @@ rte_pmd_af_packet_remove(const char *name)
rte_free(internals->rx_queue[q].rd);
rte_free(internals->tx_queue[q].rd);
}
+ rte_free(internals->if_name);
rte_free(eth_dev->data->dev_private);
rte_free(eth_dev->data);
--
2.1.4
^ permalink raw reply related
* Re: [PATCH RFC 0/2] Allow vectorized Rx with 4096 desc ring size on Intel NICs.
From: Ferruh Yigit @ 2017-01-03 17:24 UTC (permalink / raw)
To: Thomas Monjalon, Ilya Maximets
Cc: dev, Helin Zhang, Konstantin Ananyev, Jingjing Wu, Heetae Ahn,
Bruce Richardson, Wenzhuo Lu
In-Reply-To: <2020651.oVKGRqHCud@xps13>
On 1/2/2017 3:40 PM, Thomas Monjalon wrote:
> 2016-12-27 08:03, Ilya Maximets:
>> Hello.
>> Ferruh, Thomas, is there a chance for this to be accepted to 17.02?
>> Maybe I should resend this patch-set without 'RFC' tag?
>
> Yes it should be integrated in 17.02.
> Ferruh, any news?
>
I was waiting for a review from driver maintainers
Ilya,
If you don't mind, can you please send again as patches instead of RFC,
if patches also don't get any objection, they can be merged.
Thanks,
ferruh
^ permalink raw reply
* Re: [PATCH] mk: disable ICC warning 188
From: Adrien Mazarguil @ 2017-01-03 17:19 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: dev, Thomas Monjalon
In-Reply-To: <20170103161542.27303-1-ferruh.yigit@intel.com>
On Tue, Jan 03, 2017 at 04:15:42PM +0000, Ferruh Yigit wrote:
> error #188: enumerated type mixed with another type
>
> This is get when an integer assigned to an enum variable.
>
> Since this usage is common and causing many ICC compilation errors, and
> other compilers accept this usage. Disabling the warning.
>
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> mk/toolchain/icc/rte.vars.mk | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/mk/toolchain/icc/rte.vars.mk b/mk/toolchain/icc/rte.vars.mk
> index ba69f1f..86d9ef7 100644
> --- a/mk/toolchain/icc/rte.vars.mk
> +++ b/mk/toolchain/icc/rte.vars.mk
> @@ -71,6 +71,7 @@ TOOLCHAIN_ASFLAGS =
> # was declared "deprecated"
> WERROR_FLAGS := -Wall -w2 -diag-disable 271 -diag-warning 1478
> WERROR_FLAGS += -diag-disable 13368 -diag-disable 15527
> +WERROR_FLAGS += -diag-disable 188
>
> ifeq ($(RTE_DEVEL_BUILD),y)
> WERROR_FLAGS += -Werror-all
> --
> 2.9.3
I also think this warning may be useful but is not worth the trouble in many
cases, thus:
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
--
Adrien Mazarguil
6WIND
^ permalink raw reply
* [PATCH v3] Scheduler: add driver for scheduler crypto pmd
From: Fan Zhang @ 2017-01-03 17:16 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, roy.fan.zhang
In-Reply-To: <1480688123-39494-1-git-send-email-roy.fan.zhang@intel.com>
This patch provides the initial implementation of the scheduler poll mode
driver using DPDK cryptodev framework.
Scheduler PMD is used to schedule and enqueue the crypto ops to the
hardware and/or software crypto devices attached to it (slaves). The
dequeue operation from the slave(s), and the possible dequeued crypto op
reordering, are then carried out by the scheduler.
As the initial version, the scheduler PMD currently supports only the
Round-robin mode, which distributes the enqueued burst of crypto ops
among its slaves in a round-robin manner. This mode may help to fill
the throughput gap between the physical core and the existing cryptodevs
to increase the overall performance. Moreover, the scheduler PMD is
provided the APIs for user to create his/her own scheduler.
Build instructions:
To build DPDK with CRYTPO_SCHEDULER_PMD the user is required to set
CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y in config/common_base
Notice:
- Scheduler PMD shares same EAL commandline options as other cryptodevs.
However, apart from socket_id, the rest of cryptodev options are
ignored. The scheduler PMD's max_nb_queue_pairs and max_nb_sessions
options are set as the minimum values of the attached slaves'. For
example, a scheduler cryptodev is attached 2 cryptodevs with
max_nb_queue_pairs of 2 and 8, respectively. The scheduler cryptodev's
max_nb_queue_pairs will be automatically updated as 2.
- The scheduler cryptodev cannot be started unless the scheduling mode
is set and at least one slave is attached. Also, to configure the
scheduler in the run-time, like attach/detach slave(s), change
scheduling mode, or enable/disable crypto op ordering, one should stop
the scheduler first, otherwise an error will be returned.
Changes in v3:
Fixed config/common_base.
Changes in v2:
New approaches in API to suit future scheduling modes.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
---
config/common_base | 6 +
drivers/crypto/Makefile | 1 +
drivers/crypto/scheduler/Makefile | 67 +++
drivers/crypto/scheduler/rte_cryptodev_scheduler.c | 598 +++++++++++++++++++++
drivers/crypto/scheduler/rte_cryptodev_scheduler.h | 183 +++++++
.../scheduler/rte_cryptodev_scheduler_ioctls.h | 92 ++++
.../scheduler/rte_cryptodev_scheduler_operations.h | 71 +++
.../scheduler/rte_pmd_crypto_scheduler_version.map | 12 +
drivers/crypto/scheduler/scheduler_pmd.c | 168 ++++++
drivers/crypto/scheduler/scheduler_pmd_ops.c | 495 +++++++++++++++++
drivers/crypto/scheduler/scheduler_pmd_private.h | 122 +++++
drivers/crypto/scheduler/scheduler_roundrobin.c | 419 +++++++++++++++
lib/librte_cryptodev/rte_cryptodev.h | 4 +
mk/rte.app.mk | 3 +-
14 files changed, 2240 insertions(+), 1 deletion(-)
create mode 100644 drivers/crypto/scheduler/Makefile
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler.c
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler.h
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
create mode 100644 drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map
create mode 100644 drivers/crypto/scheduler/scheduler_pmd.c
create mode 100644 drivers/crypto/scheduler/scheduler_pmd_ops.c
create mode 100644 drivers/crypto/scheduler/scheduler_pmd_private.h
create mode 100644 drivers/crypto/scheduler/scheduler_roundrobin.c
diff --git a/config/common_base b/config/common_base
index 4bff83a..79d120d 100644
--- a/config/common_base
+++ b/config/common_base
@@ -400,6 +400,12 @@ CONFIG_RTE_LIBRTE_PMD_KASUMI=n
CONFIG_RTE_LIBRTE_PMD_KASUMI_DEBUG=n
#
+# Compile PMD for Crypto Scheduler device
+#
+CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=n
+CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER_DEBUG=n
+
+#
# Compile PMD for ZUC device
#
CONFIG_RTE_LIBRTE_PMD_ZUC=n
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index 745c614..cdd3c94 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -38,6 +38,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat
DIRS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G) += snow3g
DIRS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += kasumi
DIRS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += zuc
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler
DIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += null
include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/scheduler/Makefile b/drivers/crypto/scheduler/Makefile
new file mode 100644
index 0000000..976a565
--- /dev/null
+++ b/drivers/crypto/scheduler/Makefile
@@ -0,0 +1,67 @@
+# BSD LICENSE
+#
+# Copyright(c) 2015 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_crypto_scheduler.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_crypto_scheduler_version.map
+
+#
+# Export include files
+#
+SYMLINK-y-include += rte_cryptodev_scheduler_ioctls.h
+SYMLINK-y-include += rte_cryptodev_scheduler_operations.h
+SYMLINK-y-include += rte_cryptodev_scheduler.h
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler_pmd_ops.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += rte_cryptodev_scheduler.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler_roundrobin.c
+
+# library dependencies
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_eal
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_mbuf
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_mempool
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_ring
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_reorder
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_cryptodev
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler.c b/drivers/crypto/scheduler/rte_cryptodev_scheduler.c
new file mode 100644
index 0000000..d2d068c
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler.c
@@ -0,0 +1,598 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <rte_jhash.h>
+#include <rte_reorder.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_cryptodev_scheduler.h>
+#include <rte_malloc.h>
+
+#include "scheduler_pmd_private.h"
+
+/** update the scheduler pmd's capability with attaching device's
+ * capability.
+ * For each device to be attached, the scheduler's capability should be
+ * the common capability set of all slaves
+ **/
+static uint32_t
+sync_caps(struct rte_cryptodev_capabilities *caps,
+ uint32_t nb_caps,
+ const struct rte_cryptodev_capabilities *slave_caps)
+{
+ uint32_t sync_nb_caps = nb_caps, nb_slave_caps = 0;
+ uint32_t i;
+
+ while (slave_caps[nb_slave_caps].op != RTE_CRYPTO_OP_TYPE_UNDEFINED)
+ nb_slave_caps++;
+
+ if (nb_caps == 0) {
+ rte_memcpy(caps, slave_caps, sizeof(*caps) * nb_slave_caps);
+ return nb_slave_caps;
+ }
+
+ for (i = 0; i < sync_nb_caps; i++) {
+ struct rte_cryptodev_capabilities *cap = &caps[i];
+ uint32_t j;
+
+ for (j = 0; j < nb_slave_caps; j++) {
+ const struct rte_cryptodev_capabilities *s_cap =
+ &slave_caps[i];
+
+ if (s_cap->op != cap->op || s_cap->sym.xform_type !=
+ cap->sym.xform_type)
+ continue;
+
+ if (s_cap->sym.xform_type ==
+ RTE_CRYPTO_SYM_XFORM_AUTH) {
+ if (s_cap->sym.auth.algo !=
+ cap->sym.auth.algo)
+ continue;
+
+ cap->sym.auth.digest_size.min =
+ s_cap->sym.auth.digest_size.min <
+ cap->sym.auth.digest_size.min ?
+ s_cap->sym.auth.digest_size.min :
+ cap->sym.auth.digest_size.min;
+ cap->sym.auth.digest_size.max =
+ s_cap->sym.auth.digest_size.max <
+ cap->sym.auth.digest_size.max ?
+ s_cap->sym.auth.digest_size.max :
+ cap->sym.auth.digest_size.max;
+
+ }
+
+ if (s_cap->sym.xform_type ==
+ RTE_CRYPTO_SYM_XFORM_CIPHER)
+ if (s_cap->sym.cipher.algo !=
+ cap->sym.cipher.algo)
+ continue;
+
+ /* no common cap found */
+ break;
+ }
+
+ if (j < nb_slave_caps)
+ continue;
+
+ /* remove a uncommon cap from the array */
+ for (j = i; j < sync_nb_caps - 1; j++)
+ rte_memcpy(&caps[j], &caps[j+1], sizeof(*cap));
+
+ memset(&caps[sync_nb_caps - 1], 0, sizeof(*cap));
+ sync_nb_caps--;
+ }
+
+ return sync_nb_caps;
+}
+
+static int
+update_scheduler_capability(struct scheduler_ctx *sched_ctx)
+{
+ struct rte_cryptodev_capabilities tmp_caps[256] = {0};
+ uint32_t nb_caps = 0, i;
+
+ if (sched_ctx->capabilities)
+ rte_free(sched_ctx->capabilities);
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(sched_ctx->slaves[i].dev_id, &dev_info);
+
+ nb_caps = sync_caps(tmp_caps, nb_caps, dev_info.capabilities);
+ if (nb_caps == 0)
+ return -1;
+ }
+
+ sched_ctx->capabilities = rte_zmalloc_socket(NULL,
+ sizeof(struct rte_cryptodev_capabilities) *
+ (nb_caps + 1), 0, SOCKET_ID_ANY);
+ if (!sched_ctx->capabilities)
+ return -ENOMEM;
+
+ rte_memcpy(sched_ctx->capabilities, tmp_caps,
+ sizeof(struct rte_cryptodev_capabilities) * nb_caps);
+
+ return 0;
+}
+
+static void
+update_scheduler_feature_flag(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ uint32_t i;
+
+ dev->feature_flags = 0;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(sched_ctx->slaves[i].dev_id, &dev_info);
+
+ dev->feature_flags |= dev_info.feature_flags;
+ }
+}
+
+static void
+update_max_nb_qp(struct scheduler_ctx *sched_ctx)
+{
+ uint32_t i;
+ uint32_t max_nb_qp;
+
+ if (!sched_ctx->nb_slaves)
+ return;
+
+ max_nb_qp = sched_ctx->nb_slaves ? UINT32_MAX : 0;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(sched_ctx->slaves[i].dev_id, &dev_info);
+ max_nb_qp = dev_info.max_nb_queue_pairs < max_nb_qp ?
+ dev_info.max_nb_queue_pairs : max_nb_qp;
+ }
+
+ sched_ctx->max_nb_queue_pairs = max_nb_qp;
+}
+
+/** Attach a device to the scheduler. */
+int
+rte_cryptodev_scheduler_slave_attach(uint8_t scheduler_id, uint8_t slave_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ struct scheduler_slave *slave;
+ struct rte_cryptodev_info dev_info;
+ uint32_t i;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+ if (sched_ctx->nb_slaves >= MAX_SLAVES_NUM) {
+ CS_LOG_ERR("Too many slaves attached");
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++)
+ if (sched_ctx->slaves[i].dev_id == slave_id) {
+ CS_LOG_ERR("Slave already added");
+ return -ENOTSUP;
+ }
+
+ slave = &sched_ctx->slaves[sched_ctx->nb_slaves];
+
+ rte_cryptodev_info_get(slave_id, &dev_info);
+
+ slave->dev_id = slave_id;
+ slave->dev_type = dev_info.dev_type;
+ sched_ctx->nb_slaves++;
+
+ if (update_scheduler_capability(sched_ctx) < 0) {
+ slave->dev_id = 0;
+ slave->dev_type = 0;
+ sched_ctx->nb_slaves--;
+
+ CS_LOG_ERR("capabilities update failed");
+ return -ENOTSUP;
+ }
+
+ update_scheduler_feature_flag(dev);
+
+ update_max_nb_qp(sched_ctx);
+
+ return 0;
+}
+
+int
+rte_cryptodev_scheduler_slave_detach(uint8_t scheduler_id, uint8_t slave_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ uint32_t i, slave_pos;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ for (slave_pos = 0; slave_pos < sched_ctx->nb_slaves; slave_pos++)
+ if (sched_ctx->slaves[slave_pos].dev_id == slave_id)
+ break;
+ if (slave_pos == sched_ctx->nb_slaves) {
+ CS_LOG_ERR("Cannot find slave");
+ return -ENOTSUP;
+ }
+
+ if (sched_ctx->ops.slave_detach(dev, slave_id) < 0) {
+ CS_LOG_ERR("Failed to detach slave");
+ return -ENOTSUP;
+ }
+
+ for (i = slave_pos; i < sched_ctx->nb_slaves - 1; i++) {
+ memcpy(&sched_ctx->slaves[i], &sched_ctx->slaves[i+1],
+ sizeof(struct scheduler_slave));
+ }
+ memset(&sched_ctx->slaves[sched_ctx->nb_slaves - 1], 0,
+ sizeof(struct scheduler_slave));
+ sched_ctx->nb_slaves--;
+
+ if (update_scheduler_capability(sched_ctx) < 0) {
+ CS_LOG_ERR("capabilities update failed");
+ return -ENOTSUP;
+ }
+
+ update_scheduler_feature_flag(dev);
+
+ update_max_nb_qp(sched_ctx);
+
+ return 0;
+}
+
+int
+rte_crpytodev_scheduler_mode_set(uint8_t scheduler_id,
+ enum rte_cryptodev_scheduler_mode mode)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ int ret;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ if (mode == sched_ctx->mode && mode != CDEV_SCHED_MODE_USERDEFINED)
+ return 0;
+
+ switch (mode) {
+ case CDEV_SCHED_MODE_ROUNDROBIN:
+ if (rte_cryptodev_scheduler_load_user_scheduler(scheduler_id,
+ roundrobin_scheduler) < 0) {
+ CS_LOG_ERR("Failed to load scheduler");
+ return -1;
+ }
+ break;
+ case CDEV_SCHED_MODE_MIGRATION:
+ case CDEV_SCHED_MODE_FALLBACK:
+ default:
+ CS_LOG_ERR("Not yet supported");
+ return -ENOTSUP;
+ }
+
+ if (sched_ctx->private_ctx)
+ rte_free(sched_ctx->private_ctx);
+
+ ret = (*sched_ctx->ops.create_private_ctx)(dev);
+ if (ret < 0) {
+ CS_LOG_ERR("Unable to create scheduler private context");
+ return ret;
+ }
+
+ sched_ctx->mode = mode;
+
+ return 0;
+}
+
+enum rte_cryptodev_scheduler_mode
+rte_crpytodev_scheduler_mode_get(uint8_t scheduler_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ return sched_ctx->mode;
+}
+
+int
+rte_cryptodev_scheduler_ordering_set(uint8_t scheduler_id,
+ uint32_t enable_reorder)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ sched_ctx->reordering_enabled = enable_reorder;
+
+ return 0;
+}
+
+int
+rte_cryptodev_scheduler_ordering_get(uint8_t scheduler_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ return (int)sched_ctx->reordering_enabled;
+}
+
+int
+rte_cryptodev_scheduler_ioctl(uint8_t scheduler_id, uint16_t ioctl_id,
+ void *ioctl_param) {
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ if (ioctl_id >= sched_ctx->ioctl_count) {
+ CS_LOG_ERR("Invalid IOCTL ID");
+ return -EINVAL;
+ }
+
+ return (*(sched_ctx->ioctls[ioctl_id]->ioctl))(ioctl_param);
+}
+
+int
+rte_cryptodev_scheduler_ioctl_count(uint8_t scheduler_id) {
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ return sched_ctx->ioctl_count;
+}
+
+int
+rte_cryptodev_scheduler_ioctl_list(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler_ioctl_description **ioctls_desc,
+ uint16_t nb_ioctls)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ uint32_t i;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ if (nb_ioctls > sched_ctx->ioctl_count) {
+ CS_LOG_ERR("Invalid IOCTL number");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < nb_ioctls; i++) {
+ ioctls_desc[i]->request_id = sched_ctx->ioctls[i]->id;
+ ioctls_desc[i]->name = sched_ctx->ioctls[i]->name;
+ ioctls_desc[i]->description = sched_ctx->ioctls[i]->description;
+ }
+
+ return 0;
+}
+
+int
+rte_cryptodev_scheduler_load_user_scheduler(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler *scheduler) {
+
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ uint32_t i;
+ size_t size;
+
+ /* check device stopped */
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Device should be stopped before loading scheduler");
+ return -EBUSY;
+ }
+
+ strncpy(sched_ctx->name, scheduler->name,
+ RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN);
+ strncpy(sched_ctx->description, scheduler->description,
+ RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN);
+
+ /* load scheduler instance ioctls */
+ if (sched_ctx->ioctls)
+ rte_free(sched_ctx->ioctls);
+ if (scheduler->nb_ioctls) {
+ size = sizeof(struct rte_cryptodev_scheduler_ioctl) *
+ scheduler->nb_ioctls;
+ sched_ctx->ioctls = rte_zmalloc_socket(NULL, size, 0,
+ SOCKET_ID_ANY);
+ if (!sched_ctx->ioctls) {
+ CS_LOG_ERR("Failed to allocate memory");
+ return -ENOMEM;
+ }
+ }
+
+
+ for (i = 0; i < scheduler->nb_ioctls; i++) {
+ struct rte_cryptodev_scheduler_ioctl *ioctl =
+ sched_ctx->ioctls[scheduler->ioctls[i]->id];
+
+ strncpy(ioctl->name, scheduler->ioctls[i]->name,
+ RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN);
+ strncpy(ioctl->description, scheduler->ioctls[i]->description,
+ RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN);
+ ioctl->ioctl = scheduler->ioctls[i]->ioctl;
+ }
+
+ sched_ctx->ioctl_count = scheduler->nb_ioctls;
+
+ /* load scheduler instance options */
+ if (sched_ctx->options)
+ rte_free(sched_ctx->options);
+ if (scheduler->nb_options) {
+ size = sizeof(struct rte_cryptodev_scheduler_option) *
+ scheduler->nb_options;
+ sched_ctx->options = rte_zmalloc_socket(NULL, size, 0,
+ SOCKET_ID_ANY);
+ if (!sched_ctx->options) {
+ CS_LOG_ERR("Failed to allocate memory");
+ return -ENOMEM;
+ }
+ }
+
+ for (i = 0; i < scheduler->nb_options; i++) {
+ struct rte_cryptodev_scheduler_option *option =
+ sched_ctx->options[i];
+
+ strncpy(option->name, scheduler->options[i]->name,
+ RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN);
+ strncpy(option->description, scheduler->options[i]->description,
+ RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN);
+ option->option_parser = scheduler->options[i]->option_parser;
+ }
+ sched_ctx->nb_options = scheduler->nb_options;
+
+ /* load scheduler instance operations functions */
+ sched_ctx->ops.config_queue_pair = scheduler->ops->config_queue_pair;
+ sched_ctx->ops.create_private_ctx = scheduler->ops->create_private_ctx;
+ sched_ctx->ops.scheduler_start = scheduler->ops->scheduler_start;
+ sched_ctx->ops.scheduler_stop = scheduler->ops->scheduler_stop;
+ sched_ctx->ops.slave_attach = scheduler->ops->slave_attach;
+ sched_ctx->ops.slave_detach = scheduler->ops->slave_detach;
+
+ return 0;
+}
+
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler.h b/drivers/crypto/scheduler/rte_cryptodev_scheduler.h
new file mode 100644
index 0000000..ee5eeb4
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler.h
@@ -0,0 +1,183 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTO_SCHEDULER_H
+#define _RTE_CRYPTO_SCHEDULER_H
+
+#include <rte_cryptodev_scheduler_ioctls.h>
+#include <rte_cryptodev_scheduler_operations.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Crypto scheduler PMD operation modes
+ */
+enum rte_cryptodev_scheduler_mode {
+ CDEV_SCHED_MODE_NOT_SET = 0,
+ CDEV_SCHED_MODE_USERDEFINED,
+ CDEV_SCHED_MODE_ROUNDROBIN,
+ CDEV_SCHED_MODE_MIGRATION,
+ CDEV_SCHED_MODE_FALLBACK,
+ CDEV_SCHED_MODE_MULTICORE,
+
+ CDEV_SCHED_MODE_COUNT /* number of modes */
+};
+
+#define RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN (64)
+#define RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN (256)
+
+struct rte_cryptodev_scheduler;
+
+/**
+ * Load a user defined scheduler
+ *
+ * @param scheduler_id The target scheduler device ID
+ * scheduler Pointer to the user defined scheduler
+ *
+ * @return
+ * 0 if loading successful, negative integer if otherwise.
+ */
+int
+rte_cryptodev_scheduler_load_user_scheduler(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler *scheduler);
+
+/**
+ * Attach a pre-configured crypto device to the scheduler
+ *
+ * @param scheduler_id The target scheduler device ID
+ * slave_id crypto device ID to be attached
+ *
+ * @return
+ * 0 if attaching successful, negative int if otherwise.
+ */
+int
+rte_cryptodev_scheduler_slave_attach(uint8_t scheduler_id, uint8_t slave_id);
+
+/**
+ * Detach a attached crypto device to the scheduler
+ *
+ * @param scheduler_id The target scheduler device ID
+ * slave_id crypto device ID to be detached
+ *
+ * @return
+ * 0 if detaching successful, negative int if otherwise.
+ */
+int
+rte_cryptodev_scheduler_slave_detach(uint8_t scheduler_id, uint8_t slave_id);
+
+/**
+ * Set the scheduling mode
+ *
+ * @param scheduler_id The target scheduler device ID
+ * mode The scheduling mode
+ *
+ * @return
+ * 0 if attaching successful, negative integer if otherwise.
+ */
+int
+rte_crpytodev_scheduler_mode_set(uint8_t scheduler_id,
+ enum rte_cryptodev_scheduler_mode mode);
+
+/**
+ * Get the current scheduling mode
+ *
+ * @param scheduler_id The target scheduler device ID
+ * mode Pointer to write the scheduling mode
+ */
+enum rte_cryptodev_scheduler_mode
+rte_crpytodev_scheduler_mode_get(uint8_t scheduler_id);
+
+/**
+ * Set the crypto ops reordering feature on/off
+ *
+ * @param dev_id The target scheduler device ID
+ * enable_reorder set the crypto op reordering feature
+ * 0: disable reordering
+ * 1: enable reordering
+ *
+ * @return
+ * 0 if setting successful, negative integer if otherwise.
+ */
+int
+rte_cryptodev_scheduler_ordering_set(uint8_t scheduler_id,
+ uint32_t enable_reorder);
+
+/**
+ * Get the current crypto ops reordering feature
+ *
+ * @param dev_id The target scheduler device ID
+ *
+ * @return
+ * 0 if reordering is disabled
+ * 1 if reordering is enabled
+ * negative integer if otherwise.
+ */
+int
+rte_cryptodev_scheduler_ordering_get(uint8_t scheduler_id);
+
+typedef int (*rte_cryptodev_scheduler_option_parser)(
+ const char *key, const char *value, void *extra_args);
+
+typedef uint16_t (*rte_cryptodev_scheduler_burst_enqueue_t)(void *qp_ctx,
+ struct rte_crypto_op **ops, uint16_t nb_ops);
+
+typedef uint16_t (*rte_cryptodev_scheduler_burst_dequeue_t)(void *qp_ctx,
+ struct rte_crypto_op **ops, uint16_t nb_ops);
+
+struct rte_cryptodev_scheduler_option {
+ char name[RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN];
+ char description[RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN];
+
+ rte_cryptodev_scheduler_option_parser option_parser;
+};
+
+struct rte_cryptodev_scheduler {
+ const char *name;
+ const char *description;
+ struct rte_cryptodev_scheduler_option **options;
+ unsigned nb_options;
+
+ struct rte_cryptodev_scheduler_ioctl **ioctls;
+ unsigned nb_ioctls;
+
+ struct rte_cryptodev_scheduler_ops *ops;
+};
+
+extern struct rte_cryptodev_scheduler *roundrobin_scheduler;
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* _RTE_CRYPTO_SCHEDULER_H */
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h b/drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h
new file mode 100644
index 0000000..c19a9d3
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h
@@ -0,0 +1,92 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef _RTE_CRYPTODEV_SCHEDULER_IOCTLS
+#define _RTE_CRYPTODEV_SCHEDULER_IOCTLS
+
+#include <rte_cryptodev_scheduler.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define RTE_CRYPTODEV_SCHEDULER_IOCTL_NAME_MAX_LEN (64)
+#define RTE_CRYPTODEV_SCHEDULER_IOCTL_DESC_MAX_LEN (256)
+
+#define RTE_CRYPTODEV_SCHEDULER_MAX_NB_IOCTLS (8)
+
+#define CDEV_SCHED_IOCTL_LIVE_MIGRATION_SCHED_STATE_GET (1)
+#define CDEV_SCHED_IOCTL_LIVE_MIGRATION_SCHED_MIGRATE (2)
+#define CDEV_SCHED_IOCTL_FALLBACK_SCHED_SET_PRIMARY (3)
+
+struct ioctl_migration_scheduler_state_get {
+ uint8_t slave_id;
+ /**< Active crypto device id */
+ enum migration_scheduler_state {
+ MIGRATION_SCHEDULER_STATE_ACTIVE,
+ MIGRATION_SCHEDULER_STATE_AWAITING_MIGRATE,
+ MIGRATION_SCHEDULER_STATE_MIGRATION
+ } state;
+ /**< Migration Scheduler State */
+};
+
+int
+rte_cryptodev_scheduler_ioctl(uint8_t scheduler_id, uint16_t request_id,
+ void *request_params);
+
+int
+rte_cryptodev_scheduler_ioctl_count(uint8_t scheduler_id);
+
+struct rte_cryptodev_scheduler_ioctl_description {
+ uint16_t request_id;
+ const char *name;
+ const char *description;
+};
+
+int
+rte_cryptodev_scheduler_ioctl_list(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler_ioctl_description **ioctls_desc,
+ uint16_t nb_ioctls);
+
+typedef int (*rte_cryptodev_scheduler_ioctl_fn)(void *request_params);
+
+struct rte_cryptodev_scheduler_ioctl {
+ int id;
+ char name[RTE_CRYPTODEV_SCHEDULER_IOCTL_NAME_MAX_LEN];
+ char description[RTE_CRYPTODEV_SCHEDULER_IOCTL_DESC_MAX_LEN];
+
+ rte_cryptodev_scheduler_ioctl_fn ioctl;
+};
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* _RTE_CRYPTODEV_SCHEDULER_IOCTLS */
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h b/drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
new file mode 100644
index 0000000..ab8595b
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
@@ -0,0 +1,71 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTO_SCHEDULER_OPERATIONS_H
+#define _RTE_CRYPTO_SCHEDULER_OPERATIONS_H
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef int (*rte_cryptodev_scheduler_slave_attach_t)(
+ struct rte_cryptodev *dev, uint8_t slave_id);
+typedef int (*rte_cryptodev_scheduler_slave_detach_t)(
+ struct rte_cryptodev *dev, uint8_t slave_id);
+
+typedef int (*rte_cryptodev_scheduler_start_t)(struct rte_cryptodev *dev);
+typedef int (*rte_cryptodev_scheduler_stop_t)(struct rte_cryptodev *dev);
+
+typedef int (*rte_cryptodev_scheduler_config_queue_pair)(
+ struct rte_cryptodev *dev, uint16_t qp_id);
+
+typedef int (*rte_cryptodev_scheduler_create_private_ctx)(
+ struct rte_cryptodev *dev);
+
+struct rte_cryptodev_scheduler_ops {
+ rte_cryptodev_scheduler_slave_attach_t slave_attach;
+ rte_cryptodev_scheduler_slave_attach_t slave_detach;
+
+ rte_cryptodev_scheduler_start_t scheduler_start;
+ rte_cryptodev_scheduler_stop_t scheduler_stop;
+
+ rte_cryptodev_scheduler_config_queue_pair config_queue_pair;
+
+ rte_cryptodev_scheduler_create_private_ctx create_private_ctx;
+};
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* _RTE_CRYPTO_SCHEDULER_OPERATIONS_H */
diff --git a/drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map b/drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map
new file mode 100644
index 0000000..0510f68
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map
@@ -0,0 +1,12 @@
+DPDK_17.02 {
+ global:
+
+ rte_cryptodev_scheduler_load_user_scheduler;
+ rte_cryptodev_scheduler_slave_attach;
+ rte_cryptodev_scheduler_slave_detach;
+ rte_crpytodev_scheduler_mode_set;
+ rte_crpytodev_scheduler_mode_get;
+ rte_cryptodev_scheduler_ordering_set;
+ rte_cryptodev_scheduler_ordering_get;
+
+} DPDK_17.02;
\ No newline at end of file
diff --git a/drivers/crypto/scheduler/scheduler_pmd.c b/drivers/crypto/scheduler/scheduler_pmd.c
new file mode 100644
index 0000000..0c13b55
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_pmd.c
@@ -0,0 +1,168 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <rte_common.h>
+#include <rte_hexdump.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_vdev.h>
+#include <rte_malloc.h>
+#include <rte_cpuflags.h>
+#include <rte_reorder.h>
+#include <rte_cryptodev_scheduler.h>
+
+#include "scheduler_pmd_private.h"
+
+static uint16_t
+scheduler_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *qp_ctx = queue_pair;
+ uint16_t processed_ops;
+
+ processed_ops = (*qp_ctx->schedule_enqueue)(qp_ctx, ops,
+ nb_ops);
+
+ return processed_ops;
+}
+
+static uint16_t
+scheduler_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *qp_ctx = queue_pair;
+ uint16_t processed_ops;
+
+ processed_ops = (*qp_ctx->schedule_dequeue)(qp_ctx, ops,
+ nb_ops);
+
+ return processed_ops;
+}
+
+static uint32_t unique_name_id;
+
+static int
+cryptodev_scheduler_create(const char *name,
+ struct rte_crypto_vdev_init_params *init_params)
+{
+ char crypto_dev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct rte_cryptodev *dev;
+ struct scheduler_ctx *sched_ctx;
+
+ if (snprintf(crypto_dev_name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%u",
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), unique_name_id++) < 0) {
+ CS_LOG_ERR("driver %s: failed to create unique cryptodev "
+ "name", name);
+ return -EFAULT;
+ }
+
+ dev = rte_cryptodev_pmd_virtual_dev_init(crypto_dev_name,
+ sizeof(struct scheduler_ctx),
+ init_params->socket_id);
+ if (dev == NULL) {
+ CS_LOG_ERR("driver %s: failed to create cryptodev vdev",
+ name);
+ return -EFAULT;
+ }
+
+ dev->dev_type = RTE_CRYPTODEV_SCHEDULER_PMD;
+ dev->dev_ops = rte_crypto_scheduler_pmd_ops;
+
+ dev->enqueue_burst = scheduler_enqueue_burst;
+ dev->dequeue_burst = scheduler_dequeue_burst;
+
+ sched_ctx = dev->data->dev_private;
+ sched_ctx->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
+
+ return 0;
+}
+
+static int
+cryptodev_scheduler_remove(const char *name)
+{
+ struct rte_cryptodev *dev;
+ struct scheduler_ctx *sched_ctx;
+
+ if (name == NULL)
+ return -EINVAL;
+
+ dev = rte_cryptodev_pmd_get_named_dev(name);
+ if (dev == NULL)
+ return -EINVAL;
+
+ sched_ctx = dev->data->dev_private;
+
+ if (sched_ctx->nb_slaves) {
+ uint32_t i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++)
+ rte_cryptodev_scheduler_slave_detach(dev->data->dev_id,
+ sched_ctx->slaves[i].dev_id);
+ }
+
+ RTE_LOG(INFO, PMD, "Closing Crypto Scheduler device %s on numa "
+ "socket %u\n", name, rte_socket_id());
+
+ return 0;
+}
+
+static int
+cryptodev_scheduler_probe(const char *name, const char *input_args)
+{
+ struct rte_crypto_vdev_init_params init_params = {
+ RTE_CRYPTODEV_VDEV_DEFAULT_MAX_NB_QUEUE_PAIRS,
+ RTE_CRYPTODEV_VDEV_DEFAULT_MAX_NB_SESSIONS,
+ rte_socket_id()
+ };
+
+ rte_cryptodev_parse_vdev_init_params(&init_params, input_args);
+
+ RTE_LOG(INFO, PMD, "Initialising %s on NUMA node %d\n", name,
+ init_params.socket_id);
+ RTE_LOG(INFO, PMD, " Max number of queue pairs = %d\n",
+ init_params.max_nb_queue_pairs);
+ RTE_LOG(INFO, PMD, " Max number of sessions = %d\n",
+ init_params.max_nb_sessions);
+
+ return cryptodev_scheduler_create(name, &init_params);
+}
+
+static struct rte_vdev_driver cryptodev_scheduler_pmd_drv = {
+ .probe = cryptodev_scheduler_probe,
+ .remove = cryptodev_scheduler_remove
+};
+
+RTE_PMD_REGISTER_VDEV(CRYPTODEV_NAME_SCHEDULER_PMD,
+ cryptodev_scheduler_pmd_drv);
+RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_SCHEDULER_PMD,
+ "max_nb_queue_pairs=<int> "
+ "max_nb_sessions=<int> "
+ "socket_id=<int>");
diff --git a/drivers/crypto/scheduler/scheduler_pmd_ops.c b/drivers/crypto/scheduler/scheduler_pmd_ops.c
new file mode 100644
index 0000000..972a355
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_pmd_ops.c
@@ -0,0 +1,495 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <string.h>
+
+#include <rte_config.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_dev.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_reorder.h>
+
+#include "scheduler_pmd_private.h"
+
+/** Configure device */
+static int
+scheduler_pmd_config(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+ int ret = 0;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ ret = (*slave_dev->dev_ops->dev_configure)(slave_dev);
+ if (ret < 0)
+ break;
+ }
+
+ return ret;
+}
+
+static int
+update_reorder_buff(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[qp_id];
+
+ if (sched_ctx->reordering_enabled) {
+ char reorder_buff_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ uint32_t buff_size = sched_ctx->nb_slaves * PER_SLAVE_BUFF_SIZE;
+
+ if (qp_ctx->reorder_buf) {
+ rte_reorder_free(qp_ctx->reorder_buf);
+ qp_ctx->reorder_buf = NULL;
+ }
+
+ if (!buff_size)
+ return 0;
+
+ if (snprintf(reorder_buff_name, RTE_CRYPTODEV_NAME_MAX_LEN,
+ "%s_rb_%u_%u", RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD),
+ dev->data->dev_id, qp_id) < 0) {
+ CS_LOG_ERR("failed to create unique reorder buffer "
+ "name");
+ return -ENOMEM;
+ }
+
+ qp_ctx->reorder_buf = rte_reorder_create(reorder_buff_name,
+ rte_socket_id(), buff_size);
+ if (!qp_ctx->reorder_buf) {
+ CS_LOG_ERR("failed to create reorder buffer");
+ return -ENOMEM;
+ }
+ } else {
+ if (qp_ctx->reorder_buf) {
+ rte_reorder_free(qp_ctx->reorder_buf);
+ qp_ctx->reorder_buf = NULL;
+ }
+ }
+
+ return 0;
+}
+
+/** Start device */
+static int
+scheduler_pmd_start(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ uint32_t i;
+ int ret;
+
+ if (dev->data->dev_started)
+ return 0;
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = update_reorder_buff(dev, i);
+ if (ret < 0) {
+ CS_LOG_ERR("Failed to update reorder buffer");
+ return ret;
+ }
+ }
+
+ if (sched_ctx->mode == CDEV_SCHED_MODE_NOT_SET) {
+ CS_LOG_ERR("Scheduler mode is not set");
+ return -1;
+ }
+
+ if (!sched_ctx->nb_slaves) {
+ CS_LOG_ERR("No slave in the scheduler");
+ return -1;
+ }
+
+ RTE_FUNC_PTR_OR_ERR_RET(*sched_ctx->ops.slave_attach, -ENOTSUP);
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+
+ if ((*sched_ctx->ops.slave_attach)(dev, slave_dev_id) < 0) {
+ CS_LOG_ERR("Failed to attach slave");
+ return -ENOTSUP;
+ }
+ }
+
+ RTE_FUNC_PTR_OR_ERR_RET(*sched_ctx->ops.scheduler_start, -ENOTSUP);
+
+ if ((*sched_ctx->ops.scheduler_start)(dev) < 0) {
+ CS_LOG_ERR("Scheduler start failed");
+ return -1;
+ }
+
+ /* start all slaves */
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ ret = (*slave_dev->dev_ops->dev_start)(slave_dev);
+ if (ret < 0) {
+ CS_LOG_ERR("Failed to start slave dev %u",
+ slave_dev_id);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+/** Stop device */
+static void
+scheduler_pmd_stop(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+
+ if (!dev->data->dev_started)
+ return;
+
+ /* stop all slaves first */
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ (*slave_dev->dev_ops->dev_stop)(slave_dev);
+ }
+
+ if (*sched_ctx->ops.scheduler_stop)
+ (*sched_ctx->ops.scheduler_stop)(dev);
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+
+ if (*sched_ctx->ops.slave_detach)
+ (*sched_ctx->ops.slave_detach)(dev, slave_dev_id);
+ }
+}
+
+/** Close device */
+static int
+scheduler_pmd_close(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+ int ret;
+
+ /* the dev should be stopped before being closed */
+ if (dev->data->dev_started)
+ return -EBUSY;
+
+ /* close all slaves first */
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ ret = (*slave_dev->dev_ops->dev_close)(slave_dev);
+ if (ret < 0)
+ return ret;
+ }
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[i];
+
+ if (qp_ctx->reorder_buf) {
+ rte_reorder_free(qp_ctx->reorder_buf);
+ qp_ctx->reorder_buf = NULL;
+ }
+
+ if (qp_ctx->private_qp_ctx) {
+ rte_free(qp_ctx->private_qp_ctx);
+ qp_ctx->private_qp_ctx = NULL;
+ }
+ }
+
+ if (sched_ctx->private_ctx)
+ rte_free(sched_ctx->private_ctx);
+
+ if (sched_ctx->capabilities)
+ rte_free(sched_ctx->capabilities);
+
+ if (sched_ctx->ioctls)
+ rte_free(sched_ctx->ioctls);
+
+ if (sched_ctx->options)
+ rte_free(sched_ctx->options);
+
+ return 0;
+}
+
+/** Get device statistics */
+static void
+scheduler_pmd_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+ struct rte_cryptodev_stats slave_stats = {0};
+
+ (*slave_dev->dev_ops->stats_get)(slave_dev, &slave_stats);
+
+ stats->enqueued_count += slave_stats.enqueued_count;
+ stats->dequeued_count += slave_stats.dequeued_count;
+
+ stats->enqueue_err_count += slave_stats.enqueue_err_count;
+ stats->dequeue_err_count += slave_stats.dequeue_err_count;
+ }
+}
+
+/** Reset device statistics */
+static void
+scheduler_pmd_stats_reset(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ (*slave_dev->dev_ops->stats_reset)(slave_dev);
+ }
+}
+
+/** Get device info */
+static void
+scheduler_pmd_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *dev_info)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned max_nb_sessions = sched_ctx->nb_slaves ? UINT32_MAX : 0;
+ unsigned i;
+
+ if (!dev_info)
+ return;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev_info slave_info;
+
+ rte_cryptodev_info_get(slave_dev_id, &slave_info);
+ max_nb_sessions = slave_info.sym.max_nb_sessions <
+ max_nb_sessions ?
+ slave_info.sym.max_nb_sessions :
+ max_nb_sessions;
+ }
+
+ dev_info->dev_type = dev->dev_type;
+ dev_info->feature_flags = dev->feature_flags;
+ dev_info->capabilities = sched_ctx->capabilities;
+ dev_info->max_nb_queue_pairs = sched_ctx->max_nb_queue_pairs;
+ dev_info->sym.max_nb_sessions = max_nb_sessions;
+}
+
+/** Release queue pair */
+static int
+scheduler_pmd_qp_release(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[qp_id];
+
+ if (!qp_ctx)
+ return 0;
+
+ if (qp_ctx->reorder_buf)
+ rte_reorder_free(qp_ctx->reorder_buf);
+ if (qp_ctx->private_qp_ctx)
+ rte_free(qp_ctx->private_qp_ctx);
+
+ rte_free(qp_ctx);
+ dev->data->queue_pairs[qp_id] = NULL;
+
+ return 0;
+}
+
+/** Setup a queue pair */
+static int
+scheduler_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ __rte_unused const struct rte_cryptodev_qp_conf *qp_conf, int socket_id)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ struct scheduler_qp_ctx *qp_ctx;
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+
+ if (snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN,
+ "CRYTO_SCHE PMD %u QP %u",
+ dev->data->dev_id, qp_id) < 0) {
+ CS_LOG_ERR("Failed to create unique queue pair name");
+ return -EFAULT;
+ }
+
+ /* Free memory prior to re-allocation if needed. */
+ if (dev->data->queue_pairs[qp_id] != NULL)
+ scheduler_pmd_qp_release(dev, qp_id);
+
+ /* Allocate the queue pair data structure. */
+ qp_ctx = rte_zmalloc_socket(name, sizeof(*qp_ctx), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (qp_ctx == NULL)
+ return -ENOMEM;
+
+ dev->data->queue_pairs[qp_id] = qp_ctx;
+
+ if (*sched_ctx->ops.config_queue_pair) {
+ if ((*sched_ctx->ops.config_queue_pair)(dev, qp_id) < 0) {
+ CS_LOG_ERR("Unable to configure queue pair");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/** Start queue pair */
+static int
+scheduler_pmd_qp_start(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint16_t queue_pair_id)
+{
+ return -ENOTSUP;
+}
+
+/** Stop queue pair */
+static int
+scheduler_pmd_qp_stop(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint16_t queue_pair_id)
+{
+ return -ENOTSUP;
+}
+
+/** Return the number of allocated queue pairs */
+static uint32_t
+scheduler_pmd_qp_count(struct rte_cryptodev *dev)
+{
+ return dev->data->nb_queue_pairs;
+}
+
+static unsigned
+scheduler_pmd_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct scheduler_session);
+}
+
+static int
+config_slave_sess(struct scheduler_ctx *sched_ctx,
+ struct rte_crypto_sym_xform *xform,
+ struct scheduler_session *sess,
+ uint32_t create)
+{
+ unsigned i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct scheduler_slave *slave = &sched_ctx->slaves[i];
+ struct rte_cryptodev *dev = &rte_cryptodev_globals->
+ devs[slave->dev_id];
+
+ if (sess->sessions[i]) {
+ if (create)
+ continue;
+ /* !create */
+ (*dev->dev_ops->session_clear)(dev,
+ (void *)sess->sessions[i]);
+ sess->sessions[i] = NULL;
+ } else {
+ if (!create)
+ continue;
+ /* create */
+ sess->sessions[i] =
+ rte_cryptodev_sym_session_create(
+ slave->dev_id, xform);
+ if (!sess->sessions[i]) {
+ config_slave_sess(sched_ctx, NULL, sess, 0);
+ return -1;
+ }
+ }
+ }
+
+ return 0;
+}
+
+/** Clear the memory of session so it doesn't leave key material behind */
+static void
+scheduler_pmd_session_clear(struct rte_cryptodev *dev,
+ void *sess)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+
+ config_slave_sess(sched_ctx, NULL, sess, 0);
+
+ memset(sess, 0, sizeof(struct scheduler_session));
+}
+
+static void *
+scheduler_pmd_session_configure(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *sess)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+
+ if (config_slave_sess(sched_ctx, xform, sess, 1) < 0) {
+ CS_LOG_ERR("unabled to config sym session");
+ return NULL;
+ }
+
+ return sess;
+}
+
+struct rte_cryptodev_ops scheduler_pmd_ops = {
+ .dev_configure = scheduler_pmd_config,
+ .dev_start = scheduler_pmd_start,
+ .dev_stop = scheduler_pmd_stop,
+ .dev_close = scheduler_pmd_close,
+
+ .stats_get = scheduler_pmd_stats_get,
+ .stats_reset = scheduler_pmd_stats_reset,
+
+ .dev_infos_get = scheduler_pmd_info_get,
+
+ .queue_pair_setup = scheduler_pmd_qp_setup,
+ .queue_pair_release = scheduler_pmd_qp_release,
+ .queue_pair_start = scheduler_pmd_qp_start,
+ .queue_pair_stop = scheduler_pmd_qp_stop,
+ .queue_pair_count = scheduler_pmd_qp_count,
+
+ .session_get_size = scheduler_pmd_session_get_size,
+ .session_configure = scheduler_pmd_session_configure,
+ .session_clear = scheduler_pmd_session_clear,
+};
+
+struct rte_cryptodev_ops *rte_crypto_scheduler_pmd_ops = &scheduler_pmd_ops;
diff --git a/drivers/crypto/scheduler/scheduler_pmd_private.h b/drivers/crypto/scheduler/scheduler_pmd_private.h
new file mode 100644
index 0000000..550fdcc
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_pmd_private.h
@@ -0,0 +1,122 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _SCHEDULER_PMD_PRIVATE_H
+#define _SCHEDULER_PMD_PRIVATE_H
+
+#include <rte_hash.h>
+#include <rte_reorder.h>
+#include <rte_cryptodev_scheduler.h>
+#include <rte_cryptodev_scheduler_ioctls.h>
+
+/**< Maximum number of bonded devices per devices */
+#ifndef MAX_SLAVES_NUM
+#define MAX_SLAVES_NUM (8)
+#endif
+
+#define PER_SLAVE_BUFF_SIZE (256)
+
+#define CS_LOG_ERR(fmt, args...) \
+ RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), \
+ __func__, __LINE__, ## args)
+
+#ifdef RTE_LIBRTE_CRYPTO_SCHEDULER_DEBUG
+#define CS_LOG_INFO(fmt, args...) \
+ RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), \
+ __func__, __LINE__, ## args)
+
+#define CS_LOG_DBG(fmt, args...) \
+ RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), \
+ __func__, __LINE__, ## args)
+#else
+#define CS_LOG_INFO(fmt, args...)
+#define CS_LOG_DBG(fmt, args...)
+#endif
+
+struct scheduler_slave {
+ uint8_t dev_id;
+ uint16_t qp_id;
+ uint32_t nb_inflight_cops;
+
+ enum rte_cryptodev_type dev_type;
+};
+
+struct scheduler_ctx {
+ void *private_ctx;
+ /**< private scheduler context pointer */
+
+ struct rte_cryptodev_capabilities *capabilities;
+ unsigned nb_capabilities;
+
+ unsigned max_nb_queue_pairs;
+
+ struct scheduler_slave slaves[MAX_SLAVES_NUM];
+ unsigned nb_slaves;
+
+ enum rte_cryptodev_scheduler_mode mode;
+
+ uint32_t ioctl_count;
+ struct rte_cryptodev_scheduler_ioctl **ioctls;
+
+ uint32_t nb_options;
+ struct rte_cryptodev_scheduler_option **options;
+
+ struct rte_cryptodev_scheduler_ops ops;
+
+ uint8_t reordering_enabled;
+
+ char name[RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN];
+ char description[RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN];
+} __rte_cache_aligned;
+
+struct scheduler_qp_ctx {
+ void *private_qp_ctx;
+
+ rte_cryptodev_scheduler_burst_enqueue_t schedule_enqueue;
+ rte_cryptodev_scheduler_burst_dequeue_t schedule_dequeue;
+
+ struct rte_reorder_buffer *reorder_buf;
+ uint32_t seqn;
+} __rte_cache_aligned;
+
+struct scheduler_session {
+ struct rte_cryptodev_sym_session *sessions[MAX_SLAVES_NUM];
+};
+
+/** device specific operations function pointer structure */
+extern struct rte_cryptodev_ops *rte_crypto_scheduler_pmd_ops;
+
+#endif /* _SCHEDULER_PMD_PRIVATE_H */
diff --git a/drivers/crypto/scheduler/scheduler_roundrobin.c b/drivers/crypto/scheduler/scheduler_roundrobin.c
new file mode 100644
index 0000000..be0b7fd
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_roundrobin.c
@@ -0,0 +1,419 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_cryptodev.h>
+#include <rte_malloc.h>
+#include <rte_cryptodev_scheduler_operations.h>
+
+#include "scheduler_pmd_private.h"
+
+struct roundround_scheduler_ctx {
+};
+
+struct rr_scheduler_qp_ctx {
+ struct scheduler_slave slaves[MAX_SLAVES_NUM];
+ unsigned nb_slaves;
+
+ unsigned last_enq_slave_idx;
+ unsigned last_deq_slave_idx;
+};
+
+static uint16_t
+schedule_enqueue(void *qp_ctx, struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ ((struct scheduler_qp_ctx *)qp_ctx)->private_qp_ctx;
+ uint32_t slave_idx = rr_qp_ctx->last_enq_slave_idx;
+ struct scheduler_slave *slave = &rr_qp_ctx->slaves[slave_idx];
+ uint16_t i, processed_ops;
+ struct scheduler_session *sess0, *sess1, *sess2, *sess3;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ for (i = 0; i < nb_ops && i < 4; i++)
+ rte_prefetch0(ops[i]->sym->session);
+
+ for (i = 0; i < nb_ops - 8; i += 4) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ sess1 = (struct scheduler_session *)
+ ops[i+1]->sym->session->_private;
+ sess2 = (struct scheduler_session *)
+ ops[i+2]->sym->session->_private;
+ sess3 = (struct scheduler_session *)
+ ops[i+3]->sym->session->_private;
+
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ ops[i + 1]->sym->session = sess1->sessions[slave_idx];
+ ops[i + 2]->sym->session = sess2->sessions[slave_idx];
+ ops[i + 3]->sym->session = sess3->sessions[slave_idx];
+
+ rte_prefetch0(ops[i + 4]->sym->session);
+ rte_prefetch0(ops[i + 5]->sym->session);
+ rte_prefetch0(ops[i + 6]->sym->session);
+ rte_prefetch0(ops[i + 7]->sym->session);
+ }
+
+ for (; i < nb_ops; i++) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ }
+
+ processed_ops = rte_cryptodev_enqueue_burst(slave->dev_id,
+ slave->qp_id, ops, nb_ops);
+
+ slave->nb_inflight_cops += processed_ops;
+
+ rr_qp_ctx->last_enq_slave_idx += 1;
+ if (unlikely(rr_qp_ctx->last_enq_slave_idx >= rr_qp_ctx->nb_slaves))
+ rr_qp_ctx->last_enq_slave_idx = 0;
+
+ return processed_ops;
+}
+
+static uint16_t
+schedule_enqueue_ordering(void *qp_ctx, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *gen_qp_ctx = qp_ctx;
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ gen_qp_ctx->private_qp_ctx;
+ uint32_t slave_idx = rr_qp_ctx->last_enq_slave_idx;
+ struct scheduler_slave *slave = &rr_qp_ctx->slaves[slave_idx];
+ uint16_t i, processed_ops;
+ struct scheduler_session *sess0, *sess1, *sess2, *sess3;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ for (i = 0; i < nb_ops && i < 4; i++) {
+ rte_prefetch0(ops[i]->sym->session);
+ rte_prefetch0(ops[i]->sym->m_src);
+ }
+
+ for (i = 0; i < nb_ops - 8; i += 4) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ sess1 = (struct scheduler_session *)
+ ops[i+1]->sym->session->_private;
+ sess2 = (struct scheduler_session *)
+ ops[i+2]->sym->session->_private;
+ sess3 = (struct scheduler_session *)
+ ops[i+3]->sym->session->_private;
+
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ ops[i]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ ops[i + 1]->sym->session = sess1->sessions[slave_idx];
+ ops[i + 1]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ ops[i + 2]->sym->session = sess2->sessions[slave_idx];
+ ops[i + 2]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ ops[i + 3]->sym->session = sess3->sessions[slave_idx];
+ ops[i + 3]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+
+ rte_prefetch0(ops[i + 4]->sym->session);
+ rte_prefetch0(ops[i + 4]->sym->m_src);
+ rte_prefetch0(ops[i + 5]->sym->session);
+ rte_prefetch0(ops[i + 5]->sym->m_src);
+ rte_prefetch0(ops[i + 6]->sym->session);
+ rte_prefetch0(ops[i + 6]->sym->m_src);
+ rte_prefetch0(ops[i + 7]->sym->session);
+ rte_prefetch0(ops[i + 7]->sym->m_src);
+ }
+
+ for (; i < nb_ops; i++) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ ops[i]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ }
+
+ processed_ops = rte_cryptodev_enqueue_burst(slave->dev_id,
+ slave->qp_id, ops, nb_ops);
+
+ slave->nb_inflight_cops += processed_ops;
+
+ rr_qp_ctx->last_enq_slave_idx += 1;
+ if (unlikely(rr_qp_ctx->last_enq_slave_idx >= rr_qp_ctx->nb_slaves))
+ rr_qp_ctx->last_enq_slave_idx = 0;
+
+ return processed_ops;
+}
+
+
+static uint16_t
+schedule_dequeue(void *qp_ctx, struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ ((struct scheduler_qp_ctx *)qp_ctx)->private_qp_ctx;
+ struct scheduler_slave *slave;
+ uint32_t last_slave_idx = rr_qp_ctx->last_deq_slave_idx;
+ uint16_t nb_deq_ops;
+
+ if (unlikely(rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops == 0)) {
+ do {
+ last_slave_idx += 1;
+
+ if (unlikely(last_slave_idx >= rr_qp_ctx->nb_slaves))
+ last_slave_idx = 0;
+ /* looped back, means no inflight cops in the queue */
+ if (last_slave_idx == rr_qp_ctx->last_deq_slave_idx)
+ return 0;
+ } while (rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops
+ == 0);
+ }
+
+ slave = &rr_qp_ctx->slaves[last_slave_idx];
+
+ nb_deq_ops = rte_cryptodev_dequeue_burst(slave->dev_id,
+ slave->qp_id, ops, nb_ops);
+
+ last_slave_idx += 1;
+ if (unlikely(last_slave_idx >= rr_qp_ctx->nb_slaves))
+ last_slave_idx = 0;
+
+ rr_qp_ctx->last_deq_slave_idx = last_slave_idx;
+
+ slave->nb_inflight_cops -= nb_deq_ops;
+
+ return nb_deq_ops;
+}
+
+static uint16_t
+schedule_dequeue_ordering(void *qp_ctx, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *gen_qp_ctx = (struct scheduler_qp_ctx *)qp_ctx;
+ struct rr_scheduler_qp_ctx *rr_qp_ctx = (gen_qp_ctx->private_qp_ctx);
+ struct scheduler_slave *slave;
+ struct rte_reorder_buffer *reorder_buff = gen_qp_ctx->reorder_buf;
+ struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3;
+ uint16_t nb_deq_ops, nb_drained_mbufs;
+ const uint16_t nb_op_ops = nb_ops;
+ struct rte_crypto_op *op_ops[nb_op_ops];
+ struct rte_mbuf *reorder_mbufs[nb_op_ops];
+ uint32_t last_slave_idx = rr_qp_ctx->last_deq_slave_idx;
+ uint16_t i;
+
+ if (unlikely(rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops == 0)) {
+ do {
+ last_slave_idx += 1;
+
+ if (unlikely(last_slave_idx >= rr_qp_ctx->nb_slaves))
+ last_slave_idx = 0;
+ /* looped back, means no inflight cops in the queue */
+ if (last_slave_idx == rr_qp_ctx->last_deq_slave_idx)
+ return 0;
+ } while (rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops
+ == 0);
+ }
+
+ slave = &rr_qp_ctx->slaves[last_slave_idx];
+
+ nb_deq_ops = rte_cryptodev_dequeue_burst(slave->dev_id,
+ slave->qp_id, op_ops, nb_ops);
+
+ rr_qp_ctx->last_deq_slave_idx += 1;
+ if (unlikely(rr_qp_ctx->last_deq_slave_idx >= rr_qp_ctx->nb_slaves))
+ rr_qp_ctx->last_deq_slave_idx = 0;
+
+ slave->nb_inflight_cops -= nb_deq_ops;
+
+ for (i = 0; i < nb_deq_ops && i < 4; i++)
+ rte_prefetch0(op_ops[i]->sym->m_src);
+
+ for (i = 0; i < nb_deq_ops - 8; i += 4) {
+ mbuf0 = op_ops[i]->sym->m_src;
+ mbuf1 = op_ops[i + 1]->sym->m_src;
+ mbuf2 = op_ops[i + 2]->sym->m_src;
+ mbuf3 = op_ops[i + 3]->sym->m_src;
+
+ rte_memcpy(mbuf0->buf_addr, &op_ops[i], sizeof(op_ops[i]));
+ rte_memcpy(mbuf1->buf_addr, &op_ops[i+1], sizeof(op_ops[i+1]));
+ rte_memcpy(mbuf2->buf_addr, &op_ops[i+2], sizeof(op_ops[i+2]));
+ rte_memcpy(mbuf3->buf_addr, &op_ops[i+3], sizeof(op_ops[i+3]));
+
+ rte_reorder_insert(reorder_buff, mbuf0);
+ rte_reorder_insert(reorder_buff, mbuf1);
+ rte_reorder_insert(reorder_buff, mbuf2);
+ rte_reorder_insert(reorder_buff, mbuf3);
+
+ rte_prefetch0(op_ops[i + 4]->sym->m_src);
+ rte_prefetch0(op_ops[i + 5]->sym->m_src);
+ rte_prefetch0(op_ops[i + 6]->sym->m_src);
+ rte_prefetch0(op_ops[i + 7]->sym->m_src);
+ }
+
+ for (; i < nb_deq_ops; i++) {
+ mbuf0 = op_ops[i]->sym->m_src;
+ rte_memcpy(mbuf0->buf_addr, &op_ops[i], sizeof(op_ops[i]));
+ rte_reorder_insert(reorder_buff, mbuf0);
+ }
+
+ nb_drained_mbufs = rte_reorder_drain(reorder_buff, reorder_mbufs,
+ nb_ops);
+ for (i = 0; i < nb_drained_mbufs && i < 4; i++)
+ rte_prefetch0(reorder_mbufs[i]);
+
+ for (i = 0; i < nb_drained_mbufs - 8; i += 4) {
+ ops[i] = *(struct rte_crypto_op **)reorder_mbufs[i]->buf_addr;
+ ops[i + 1] = *(struct rte_crypto_op **)
+ reorder_mbufs[i + 1]->buf_addr;
+ ops[i + 2] = *(struct rte_crypto_op **)
+ reorder_mbufs[i + 2]->buf_addr;
+ ops[i + 3] = *(struct rte_crypto_op **)
+ reorder_mbufs[i + 3]->buf_addr;
+
+ *(struct rte_crypto_op **)reorder_mbufs[i]->buf_addr = NULL;
+ *(struct rte_crypto_op **)reorder_mbufs[i + 1]->buf_addr = NULL;
+ *(struct rte_crypto_op **)reorder_mbufs[i + 2]->buf_addr = NULL;
+ *(struct rte_crypto_op **)reorder_mbufs[i + 3]->buf_addr = NULL;
+
+ rte_prefetch0(reorder_mbufs[i + 4]);
+ rte_prefetch0(reorder_mbufs[i + 5]);
+ rte_prefetch0(reorder_mbufs[i + 6]);
+ rte_prefetch0(reorder_mbufs[i + 7]);
+ }
+
+ for (; i < nb_drained_mbufs; i++) {
+ ops[i] = *(struct rte_crypto_op **)
+ reorder_mbufs[i]->buf_addr;
+ *(struct rte_crypto_op **)reorder_mbufs[i]->buf_addr = NULL;
+ }
+
+ return nb_drained_mbufs;
+}
+
+static int
+slave_attach(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint8_t slave_id)
+{
+ return 0;
+}
+
+static int
+slave_detach(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint8_t slave_id)
+{
+ return 0;
+}
+
+static int
+scheduler_start(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[i];
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ qp_ctx->private_qp_ctx;
+ uint32_t j;
+ uint16_t qp_id = rr_qp_ctx->slaves[0].qp_id;
+
+ memset(rr_qp_ctx->slaves, 0, MAX_SLAVES_NUM *
+ sizeof(struct scheduler_slave));
+ for (j = 0; j < sched_ctx->nb_slaves; j++) {
+ rr_qp_ctx->slaves[j].dev_id =
+ sched_ctx->slaves[i].dev_id;
+ rr_qp_ctx->slaves[j].qp_id = qp_id;
+ }
+
+ rr_qp_ctx->nb_slaves = sched_ctx->nb_slaves;
+
+ rr_qp_ctx->last_enq_slave_idx = 0;
+ rr_qp_ctx->last_deq_slave_idx = 0;
+
+ if (sched_ctx->reordering_enabled) {
+ qp_ctx->schedule_enqueue = &schedule_enqueue_ordering;
+ qp_ctx->schedule_dequeue = &schedule_dequeue_ordering;
+ } else {
+ qp_ctx->schedule_enqueue = &schedule_enqueue;
+ qp_ctx->schedule_dequeue = &schedule_dequeue;
+ }
+ }
+
+ return 0;
+}
+
+static int
+scheduler_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+static int
+scheduler_config_qp(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[qp_id];
+ struct rr_scheduler_qp_ctx *rr_qp_ctx;
+
+ rr_qp_ctx = rte_zmalloc_socket(NULL, sizeof(*rr_qp_ctx), 0,
+ rte_socket_id());
+ if (!rr_qp_ctx) {
+ CS_LOG_ERR("failed allocate memory for private queue pair");
+ return -ENOMEM;
+ }
+
+ qp_ctx->private_qp_ctx = (void *)rr_qp_ctx;
+
+ return 0;
+}
+
+static int
+scheduler_create_private_ctx(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+struct rte_cryptodev_scheduler_ops ops = {
+ slave_attach,
+ slave_detach,
+ scheduler_start,
+ scheduler_stop,
+ scheduler_config_qp,
+ scheduler_create_private_ctx
+};
+
+struct rte_cryptodev_scheduler scheduler = {
+ .name = "roundrobin-scheduler",
+ .description = "scheduler which will round robin burst across "
+ "slave crypto devices",
+ .options = NULL,
+ .ops = &ops,
+ .ioctls = NULL
+};
+
+
+struct rte_cryptodev_scheduler *roundrobin_scheduler = &scheduler;
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 8f63e8f..61a3ce0 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -66,6 +66,7 @@ extern "C" {
/**< KASUMI PMD device name */
#define CRYPTODEV_NAME_ZUC_PMD crypto_zuc
/**< KASUMI PMD device name */
+#define CRYPTODEV_NAME_SCHEDULER_PMD crypto_scheduler
/** Crypto device type */
enum rte_cryptodev_type {
@@ -77,6 +78,9 @@ enum rte_cryptodev_type {
RTE_CRYPTODEV_KASUMI_PMD, /**< KASUMI PMD */
RTE_CRYPTODEV_ZUC_PMD, /**< ZUC PMD */
RTE_CRYPTODEV_OPENSSL_PMD, /**< OpenSSL PMD */
+ RTE_CRYPTODEV_SCHEDULER_PMD, /**< Crypto Scheduler PMD */
+
+ RTE_CRYPTODEV_TYPE_COUNT
};
extern const char **rte_cyptodev_names;
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index f75f0e2..ee34688 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -70,7 +70,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PORT) += -lrte_port
_LDLIBS-$(CONFIG_RTE_LIBRTE_PDUMP) += -lrte_pdump
_LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += -lrte_distributor
-_LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder
_LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += -lrte_ip_frag
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lrte_meter
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched
@@ -98,6 +97,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring
_LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal
_LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline
_LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
+_LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += -lrte_pmd_bond
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += -lrte_pmd_xenvirt -lxenstore
@@ -145,6 +145,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += -lrte_pmd_zuc
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += -L$(LIBSSO_ZUC_PATH)/build -lsso_zuc
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += -lrte_pmd_crypto_scheduler
endif # CONFIG_RTE_LIBRTE_CRYPTODEV
endif # !CONFIG_RTE_BUILD_SHARED_LIBS
--
2.7.4
^ permalink raw reply related
* Re: [PATCH] net/i40e: fix wrong return value when handling PF message
From: Ferruh Yigit @ 2017-01-03 17:13 UTC (permalink / raw)
To: Wenzhuo Lu, dev, dpdk stable
In-Reply-To: <1482308980-76777-1-git-send-email-wenzhuo.lu@intel.com>
On 12/21/2016 8:29 AM, Wenzhuo Lu wrote:
> When VF receives a message from PF, it should check the return
> value. But in i40evf_execute_vf_cmd the value is ignored and not
> returned to the caller.
>
> Fixes: 95cd21f45d1b ("i40evf: allocate virtchnl commands buffer per VF")
>
> Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
CC: stable@dpdk.org
Applied to dpdk-next-net/master, thanks.
^ permalink raw reply
* [PATCH v2] Scheduler: add driver for scheduler crypto pmd
From: Fan Zhang @ 2017-01-03 17:08 UTC (permalink / raw)
To: dev; +Cc: pablo.de.lara.guarch, declan.doherty, roy.fan.zhang
In-Reply-To: <1480688123-39494-1-git-send-email-roy.fan.zhang@intel.com>
This patch provides the initial implementation of the scheduler poll mode
driver using DPDK cryptodev framework.
Scheduler PMD is used to schedule and enqueue the crypto ops to the
hardware and/or software crypto devices attached to it (slaves). The
dequeue operation from the slave(s), and the possible dequeued crypto op
reordering, are then carried out by the scheduler.
As the initial version, the scheduler PMD currently supports only the
Round-robin mode, which distributes the enqueued burst of crypto ops
among its slaves in a round-robin manner. This mode may help to fill
the throughput gap between the physical core and the existing cryptodevs
to increase the overall performance. Moreover, the scheduler PMD is
provided the APIs for user to create his/her own scheduler.
Build instructions:
To build DPDK with CRYTPO_SCHEDULER_PMD the user is required to set
CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y in config/common_base
Notice:
- Scheduler PMD shares same EAL commandline options as other cryptodevs.
However, apart from socket_id, the rest of cryptodev options are
ignored. The scheduler PMD's max_nb_queue_pairs and max_nb_sessions
options are set as the minimum values of the attached slaves'. For
example, a scheduler cryptodev is attached 2 cryptodevs with
max_nb_queue_pairs of 2 and 8, respectively. The scheduler cryptodev's
max_nb_queue_pairs will be automatically updated as 2.
- The scheduler cryptodev cannot be started unless the scheduling mode
is set and at least one slave is attached. Also, to configure the
scheduler in the run-time, like attach/detach slave(s), change
scheduling mode, or enable/disable crypto op ordering, one should stop
the scheduler first, otherwise an error will be returned.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
---
config/common_base | 10 +-
drivers/crypto/Makefile | 1 +
drivers/crypto/scheduler/Makefile | 67 +++
drivers/crypto/scheduler/rte_cryptodev_scheduler.c | 598 +++++++++++++++++++++
drivers/crypto/scheduler/rte_cryptodev_scheduler.h | 183 +++++++
.../scheduler/rte_cryptodev_scheduler_ioctls.h | 92 ++++
.../scheduler/rte_cryptodev_scheduler_operations.h | 71 +++
.../scheduler/rte_pmd_crypto_scheduler_version.map | 12 +
drivers/crypto/scheduler/scheduler_pmd.c | 168 ++++++
drivers/crypto/scheduler/scheduler_pmd_ops.c | 495 +++++++++++++++++
drivers/crypto/scheduler/scheduler_pmd_private.h | 122 +++++
drivers/crypto/scheduler/scheduler_roundrobin.c | 419 +++++++++++++++
lib/librte_cryptodev/rte_cryptodev.h | 4 +
mk/rte.app.mk | 3 +-
14 files changed, 2242 insertions(+), 3 deletions(-)
create mode 100644 drivers/crypto/scheduler/Makefile
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler.c
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler.h
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h
create mode 100644 drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
create mode 100644 drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map
create mode 100644 drivers/crypto/scheduler/scheduler_pmd.c
create mode 100644 drivers/crypto/scheduler/scheduler_pmd_ops.c
create mode 100644 drivers/crypto/scheduler/scheduler_pmd_private.h
create mode 100644 drivers/crypto/scheduler/scheduler_roundrobin.c
diff --git a/config/common_base b/config/common_base
index 4bff83a..a3783a6 100644
--- a/config/common_base
+++ b/config/common_base
@@ -358,7 +358,7 @@ CONFIG_RTE_CRYPTODEV_NAME_LEN=64
#
# Compile PMD for QuickAssist based devices
#
-CONFIG_RTE_LIBRTE_PMD_QAT=n
+CONFIG_RTE_LIBRTE_PMD_QAT=y
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n
@@ -372,7 +372,7 @@ CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
#
# Compile PMD for AESNI backed device
#
-CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n
+CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y
CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n
#
@@ -400,6 +400,12 @@ CONFIG_RTE_LIBRTE_PMD_KASUMI=n
CONFIG_RTE_LIBRTE_PMD_KASUMI_DEBUG=n
#
+# Compile PMD for Crypto Scheduler device
+#
+CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y
+CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER_DEBUG=n
+
+#
# Compile PMD for ZUC device
#
CONFIG_RTE_LIBRTE_PMD_ZUC=n
diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile
index 745c614..cdd3c94 100644
--- a/drivers/crypto/Makefile
+++ b/drivers/crypto/Makefile
@@ -38,6 +38,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += qat
DIRS-$(CONFIG_RTE_LIBRTE_PMD_SNOW3G) += snow3g
DIRS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += kasumi
DIRS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += zuc
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler
DIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO) += null
include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/crypto/scheduler/Makefile b/drivers/crypto/scheduler/Makefile
new file mode 100644
index 0000000..976a565
--- /dev/null
+++ b/drivers/crypto/scheduler/Makefile
@@ -0,0 +1,67 @@
+# BSD LICENSE
+#
+# Copyright(c) 2015 Intel Corporation. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_crypto_scheduler.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_crypto_scheduler_version.map
+
+#
+# Export include files
+#
+SYMLINK-y-include += rte_cryptodev_scheduler_ioctls.h
+SYMLINK-y-include += rte_cryptodev_scheduler_operations.h
+SYMLINK-y-include += rte_cryptodev_scheduler.h
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler_pmd_ops.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += rte_cryptodev_scheduler.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += scheduler_roundrobin.c
+
+# library dependencies
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_eal
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_mbuf
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_mempool
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_ring
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_reorder
+DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += lib/librte_cryptodev
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler.c b/drivers/crypto/scheduler/rte_cryptodev_scheduler.c
new file mode 100644
index 0000000..d2d068c
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler.c
@@ -0,0 +1,598 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <rte_jhash.h>
+#include <rte_reorder.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_cryptodev_scheduler.h>
+#include <rte_malloc.h>
+
+#include "scheduler_pmd_private.h"
+
+/** update the scheduler pmd's capability with attaching device's
+ * capability.
+ * For each device to be attached, the scheduler's capability should be
+ * the common capability set of all slaves
+ **/
+static uint32_t
+sync_caps(struct rte_cryptodev_capabilities *caps,
+ uint32_t nb_caps,
+ const struct rte_cryptodev_capabilities *slave_caps)
+{
+ uint32_t sync_nb_caps = nb_caps, nb_slave_caps = 0;
+ uint32_t i;
+
+ while (slave_caps[nb_slave_caps].op != RTE_CRYPTO_OP_TYPE_UNDEFINED)
+ nb_slave_caps++;
+
+ if (nb_caps == 0) {
+ rte_memcpy(caps, slave_caps, sizeof(*caps) * nb_slave_caps);
+ return nb_slave_caps;
+ }
+
+ for (i = 0; i < sync_nb_caps; i++) {
+ struct rte_cryptodev_capabilities *cap = &caps[i];
+ uint32_t j;
+
+ for (j = 0; j < nb_slave_caps; j++) {
+ const struct rte_cryptodev_capabilities *s_cap =
+ &slave_caps[i];
+
+ if (s_cap->op != cap->op || s_cap->sym.xform_type !=
+ cap->sym.xform_type)
+ continue;
+
+ if (s_cap->sym.xform_type ==
+ RTE_CRYPTO_SYM_XFORM_AUTH) {
+ if (s_cap->sym.auth.algo !=
+ cap->sym.auth.algo)
+ continue;
+
+ cap->sym.auth.digest_size.min =
+ s_cap->sym.auth.digest_size.min <
+ cap->sym.auth.digest_size.min ?
+ s_cap->sym.auth.digest_size.min :
+ cap->sym.auth.digest_size.min;
+ cap->sym.auth.digest_size.max =
+ s_cap->sym.auth.digest_size.max <
+ cap->sym.auth.digest_size.max ?
+ s_cap->sym.auth.digest_size.max :
+ cap->sym.auth.digest_size.max;
+
+ }
+
+ if (s_cap->sym.xform_type ==
+ RTE_CRYPTO_SYM_XFORM_CIPHER)
+ if (s_cap->sym.cipher.algo !=
+ cap->sym.cipher.algo)
+ continue;
+
+ /* no common cap found */
+ break;
+ }
+
+ if (j < nb_slave_caps)
+ continue;
+
+ /* remove a uncommon cap from the array */
+ for (j = i; j < sync_nb_caps - 1; j++)
+ rte_memcpy(&caps[j], &caps[j+1], sizeof(*cap));
+
+ memset(&caps[sync_nb_caps - 1], 0, sizeof(*cap));
+ sync_nb_caps--;
+ }
+
+ return sync_nb_caps;
+}
+
+static int
+update_scheduler_capability(struct scheduler_ctx *sched_ctx)
+{
+ struct rte_cryptodev_capabilities tmp_caps[256] = {0};
+ uint32_t nb_caps = 0, i;
+
+ if (sched_ctx->capabilities)
+ rte_free(sched_ctx->capabilities);
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(sched_ctx->slaves[i].dev_id, &dev_info);
+
+ nb_caps = sync_caps(tmp_caps, nb_caps, dev_info.capabilities);
+ if (nb_caps == 0)
+ return -1;
+ }
+
+ sched_ctx->capabilities = rte_zmalloc_socket(NULL,
+ sizeof(struct rte_cryptodev_capabilities) *
+ (nb_caps + 1), 0, SOCKET_ID_ANY);
+ if (!sched_ctx->capabilities)
+ return -ENOMEM;
+
+ rte_memcpy(sched_ctx->capabilities, tmp_caps,
+ sizeof(struct rte_cryptodev_capabilities) * nb_caps);
+
+ return 0;
+}
+
+static void
+update_scheduler_feature_flag(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ uint32_t i;
+
+ dev->feature_flags = 0;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(sched_ctx->slaves[i].dev_id, &dev_info);
+
+ dev->feature_flags |= dev_info.feature_flags;
+ }
+}
+
+static void
+update_max_nb_qp(struct scheduler_ctx *sched_ctx)
+{
+ uint32_t i;
+ uint32_t max_nb_qp;
+
+ if (!sched_ctx->nb_slaves)
+ return;
+
+ max_nb_qp = sched_ctx->nb_slaves ? UINT32_MAX : 0;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct rte_cryptodev_info dev_info;
+
+ rte_cryptodev_info_get(sched_ctx->slaves[i].dev_id, &dev_info);
+ max_nb_qp = dev_info.max_nb_queue_pairs < max_nb_qp ?
+ dev_info.max_nb_queue_pairs : max_nb_qp;
+ }
+
+ sched_ctx->max_nb_queue_pairs = max_nb_qp;
+}
+
+/** Attach a device to the scheduler. */
+int
+rte_cryptodev_scheduler_slave_attach(uint8_t scheduler_id, uint8_t slave_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ struct scheduler_slave *slave;
+ struct rte_cryptodev_info dev_info;
+ uint32_t i;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+ if (sched_ctx->nb_slaves >= MAX_SLAVES_NUM) {
+ CS_LOG_ERR("Too many slaves attached");
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++)
+ if (sched_ctx->slaves[i].dev_id == slave_id) {
+ CS_LOG_ERR("Slave already added");
+ return -ENOTSUP;
+ }
+
+ slave = &sched_ctx->slaves[sched_ctx->nb_slaves];
+
+ rte_cryptodev_info_get(slave_id, &dev_info);
+
+ slave->dev_id = slave_id;
+ slave->dev_type = dev_info.dev_type;
+ sched_ctx->nb_slaves++;
+
+ if (update_scheduler_capability(sched_ctx) < 0) {
+ slave->dev_id = 0;
+ slave->dev_type = 0;
+ sched_ctx->nb_slaves--;
+
+ CS_LOG_ERR("capabilities update failed");
+ return -ENOTSUP;
+ }
+
+ update_scheduler_feature_flag(dev);
+
+ update_max_nb_qp(sched_ctx);
+
+ return 0;
+}
+
+int
+rte_cryptodev_scheduler_slave_detach(uint8_t scheduler_id, uint8_t slave_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ uint32_t i, slave_pos;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ for (slave_pos = 0; slave_pos < sched_ctx->nb_slaves; slave_pos++)
+ if (sched_ctx->slaves[slave_pos].dev_id == slave_id)
+ break;
+ if (slave_pos == sched_ctx->nb_slaves) {
+ CS_LOG_ERR("Cannot find slave");
+ return -ENOTSUP;
+ }
+
+ if (sched_ctx->ops.slave_detach(dev, slave_id) < 0) {
+ CS_LOG_ERR("Failed to detach slave");
+ return -ENOTSUP;
+ }
+
+ for (i = slave_pos; i < sched_ctx->nb_slaves - 1; i++) {
+ memcpy(&sched_ctx->slaves[i], &sched_ctx->slaves[i+1],
+ sizeof(struct scheduler_slave));
+ }
+ memset(&sched_ctx->slaves[sched_ctx->nb_slaves - 1], 0,
+ sizeof(struct scheduler_slave));
+ sched_ctx->nb_slaves--;
+
+ if (update_scheduler_capability(sched_ctx) < 0) {
+ CS_LOG_ERR("capabilities update failed");
+ return -ENOTSUP;
+ }
+
+ update_scheduler_feature_flag(dev);
+
+ update_max_nb_qp(sched_ctx);
+
+ return 0;
+}
+
+int
+rte_crpytodev_scheduler_mode_set(uint8_t scheduler_id,
+ enum rte_cryptodev_scheduler_mode mode)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ int ret;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ if (mode == sched_ctx->mode && mode != CDEV_SCHED_MODE_USERDEFINED)
+ return 0;
+
+ switch (mode) {
+ case CDEV_SCHED_MODE_ROUNDROBIN:
+ if (rte_cryptodev_scheduler_load_user_scheduler(scheduler_id,
+ roundrobin_scheduler) < 0) {
+ CS_LOG_ERR("Failed to load scheduler");
+ return -1;
+ }
+ break;
+ case CDEV_SCHED_MODE_MIGRATION:
+ case CDEV_SCHED_MODE_FALLBACK:
+ default:
+ CS_LOG_ERR("Not yet supported");
+ return -ENOTSUP;
+ }
+
+ if (sched_ctx->private_ctx)
+ rte_free(sched_ctx->private_ctx);
+
+ ret = (*sched_ctx->ops.create_private_ctx)(dev);
+ if (ret < 0) {
+ CS_LOG_ERR("Unable to create scheduler private context");
+ return ret;
+ }
+
+ sched_ctx->mode = mode;
+
+ return 0;
+}
+
+enum rte_cryptodev_scheduler_mode
+rte_crpytodev_scheduler_mode_get(uint8_t scheduler_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ return sched_ctx->mode;
+}
+
+int
+rte_cryptodev_scheduler_ordering_set(uint8_t scheduler_id,
+ uint32_t enable_reorder)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Illegal operation");
+ return -EBUSY;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ sched_ctx->reordering_enabled = enable_reorder;
+
+ return 0;
+}
+
+int
+rte_cryptodev_scheduler_ordering_get(uint8_t scheduler_id)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ return (int)sched_ctx->reordering_enabled;
+}
+
+int
+rte_cryptodev_scheduler_ioctl(uint8_t scheduler_id, uint16_t ioctl_id,
+ void *ioctl_param) {
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ if (ioctl_id >= sched_ctx->ioctl_count) {
+ CS_LOG_ERR("Invalid IOCTL ID");
+ return -EINVAL;
+ }
+
+ return (*(sched_ctx->ioctls[ioctl_id]->ioctl))(ioctl_param);
+}
+
+int
+rte_cryptodev_scheduler_ioctl_count(uint8_t scheduler_id) {
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ return sched_ctx->ioctl_count;
+}
+
+int
+rte_cryptodev_scheduler_ioctl_list(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler_ioctl_description **ioctls_desc,
+ uint16_t nb_ioctls)
+{
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx;
+ uint32_t i;
+
+ if (!dev) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ if (dev->dev_type != RTE_CRYPTODEV_SCHEDULER_PMD) {
+ CS_LOG_ERR("Operation not supported");
+ return -ENOTSUP;
+ }
+
+ sched_ctx = dev->data->dev_private;
+
+ if (nb_ioctls > sched_ctx->ioctl_count) {
+ CS_LOG_ERR("Invalid IOCTL number");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < nb_ioctls; i++) {
+ ioctls_desc[i]->request_id = sched_ctx->ioctls[i]->id;
+ ioctls_desc[i]->name = sched_ctx->ioctls[i]->name;
+ ioctls_desc[i]->description = sched_ctx->ioctls[i]->description;
+ }
+
+ return 0;
+}
+
+int
+rte_cryptodev_scheduler_load_user_scheduler(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler *scheduler) {
+
+ struct rte_cryptodev *dev = rte_cryptodev_pmd_get_dev(scheduler_id);
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ uint32_t i;
+ size_t size;
+
+ /* check device stopped */
+ if (dev->data->dev_started) {
+ CS_LOG_ERR("Device should be stopped before loading scheduler");
+ return -EBUSY;
+ }
+
+ strncpy(sched_ctx->name, scheduler->name,
+ RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN);
+ strncpy(sched_ctx->description, scheduler->description,
+ RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN);
+
+ /* load scheduler instance ioctls */
+ if (sched_ctx->ioctls)
+ rte_free(sched_ctx->ioctls);
+ if (scheduler->nb_ioctls) {
+ size = sizeof(struct rte_cryptodev_scheduler_ioctl) *
+ scheduler->nb_ioctls;
+ sched_ctx->ioctls = rte_zmalloc_socket(NULL, size, 0,
+ SOCKET_ID_ANY);
+ if (!sched_ctx->ioctls) {
+ CS_LOG_ERR("Failed to allocate memory");
+ return -ENOMEM;
+ }
+ }
+
+
+ for (i = 0; i < scheduler->nb_ioctls; i++) {
+ struct rte_cryptodev_scheduler_ioctl *ioctl =
+ sched_ctx->ioctls[scheduler->ioctls[i]->id];
+
+ strncpy(ioctl->name, scheduler->ioctls[i]->name,
+ RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN);
+ strncpy(ioctl->description, scheduler->ioctls[i]->description,
+ RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN);
+ ioctl->ioctl = scheduler->ioctls[i]->ioctl;
+ }
+
+ sched_ctx->ioctl_count = scheduler->nb_ioctls;
+
+ /* load scheduler instance options */
+ if (sched_ctx->options)
+ rte_free(sched_ctx->options);
+ if (scheduler->nb_options) {
+ size = sizeof(struct rte_cryptodev_scheduler_option) *
+ scheduler->nb_options;
+ sched_ctx->options = rte_zmalloc_socket(NULL, size, 0,
+ SOCKET_ID_ANY);
+ if (!sched_ctx->options) {
+ CS_LOG_ERR("Failed to allocate memory");
+ return -ENOMEM;
+ }
+ }
+
+ for (i = 0; i < scheduler->nb_options; i++) {
+ struct rte_cryptodev_scheduler_option *option =
+ sched_ctx->options[i];
+
+ strncpy(option->name, scheduler->options[i]->name,
+ RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN);
+ strncpy(option->description, scheduler->options[i]->description,
+ RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN);
+ option->option_parser = scheduler->options[i]->option_parser;
+ }
+ sched_ctx->nb_options = scheduler->nb_options;
+
+ /* load scheduler instance operations functions */
+ sched_ctx->ops.config_queue_pair = scheduler->ops->config_queue_pair;
+ sched_ctx->ops.create_private_ctx = scheduler->ops->create_private_ctx;
+ sched_ctx->ops.scheduler_start = scheduler->ops->scheduler_start;
+ sched_ctx->ops.scheduler_stop = scheduler->ops->scheduler_stop;
+ sched_ctx->ops.slave_attach = scheduler->ops->slave_attach;
+ sched_ctx->ops.slave_detach = scheduler->ops->slave_detach;
+
+ return 0;
+}
+
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler.h b/drivers/crypto/scheduler/rte_cryptodev_scheduler.h
new file mode 100644
index 0000000..ee5eeb4
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler.h
@@ -0,0 +1,183 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTO_SCHEDULER_H
+#define _RTE_CRYPTO_SCHEDULER_H
+
+#include <rte_cryptodev_scheduler_ioctls.h>
+#include <rte_cryptodev_scheduler_operations.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Crypto scheduler PMD operation modes
+ */
+enum rte_cryptodev_scheduler_mode {
+ CDEV_SCHED_MODE_NOT_SET = 0,
+ CDEV_SCHED_MODE_USERDEFINED,
+ CDEV_SCHED_MODE_ROUNDROBIN,
+ CDEV_SCHED_MODE_MIGRATION,
+ CDEV_SCHED_MODE_FALLBACK,
+ CDEV_SCHED_MODE_MULTICORE,
+
+ CDEV_SCHED_MODE_COUNT /* number of modes */
+};
+
+#define RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN (64)
+#define RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN (256)
+
+struct rte_cryptodev_scheduler;
+
+/**
+ * Load a user defined scheduler
+ *
+ * @param scheduler_id The target scheduler device ID
+ * scheduler Pointer to the user defined scheduler
+ *
+ * @return
+ * 0 if loading successful, negative integer if otherwise.
+ */
+int
+rte_cryptodev_scheduler_load_user_scheduler(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler *scheduler);
+
+/**
+ * Attach a pre-configured crypto device to the scheduler
+ *
+ * @param scheduler_id The target scheduler device ID
+ * slave_id crypto device ID to be attached
+ *
+ * @return
+ * 0 if attaching successful, negative int if otherwise.
+ */
+int
+rte_cryptodev_scheduler_slave_attach(uint8_t scheduler_id, uint8_t slave_id);
+
+/**
+ * Detach a attached crypto device to the scheduler
+ *
+ * @param scheduler_id The target scheduler device ID
+ * slave_id crypto device ID to be detached
+ *
+ * @return
+ * 0 if detaching successful, negative int if otherwise.
+ */
+int
+rte_cryptodev_scheduler_slave_detach(uint8_t scheduler_id, uint8_t slave_id);
+
+/**
+ * Set the scheduling mode
+ *
+ * @param scheduler_id The target scheduler device ID
+ * mode The scheduling mode
+ *
+ * @return
+ * 0 if attaching successful, negative integer if otherwise.
+ */
+int
+rte_crpytodev_scheduler_mode_set(uint8_t scheduler_id,
+ enum rte_cryptodev_scheduler_mode mode);
+
+/**
+ * Get the current scheduling mode
+ *
+ * @param scheduler_id The target scheduler device ID
+ * mode Pointer to write the scheduling mode
+ */
+enum rte_cryptodev_scheduler_mode
+rte_crpytodev_scheduler_mode_get(uint8_t scheduler_id);
+
+/**
+ * Set the crypto ops reordering feature on/off
+ *
+ * @param dev_id The target scheduler device ID
+ * enable_reorder set the crypto op reordering feature
+ * 0: disable reordering
+ * 1: enable reordering
+ *
+ * @return
+ * 0 if setting successful, negative integer if otherwise.
+ */
+int
+rte_cryptodev_scheduler_ordering_set(uint8_t scheduler_id,
+ uint32_t enable_reorder);
+
+/**
+ * Get the current crypto ops reordering feature
+ *
+ * @param dev_id The target scheduler device ID
+ *
+ * @return
+ * 0 if reordering is disabled
+ * 1 if reordering is enabled
+ * negative integer if otherwise.
+ */
+int
+rte_cryptodev_scheduler_ordering_get(uint8_t scheduler_id);
+
+typedef int (*rte_cryptodev_scheduler_option_parser)(
+ const char *key, const char *value, void *extra_args);
+
+typedef uint16_t (*rte_cryptodev_scheduler_burst_enqueue_t)(void *qp_ctx,
+ struct rte_crypto_op **ops, uint16_t nb_ops);
+
+typedef uint16_t (*rte_cryptodev_scheduler_burst_dequeue_t)(void *qp_ctx,
+ struct rte_crypto_op **ops, uint16_t nb_ops);
+
+struct rte_cryptodev_scheduler_option {
+ char name[RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN];
+ char description[RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN];
+
+ rte_cryptodev_scheduler_option_parser option_parser;
+};
+
+struct rte_cryptodev_scheduler {
+ const char *name;
+ const char *description;
+ struct rte_cryptodev_scheduler_option **options;
+ unsigned nb_options;
+
+ struct rte_cryptodev_scheduler_ioctl **ioctls;
+ unsigned nb_ioctls;
+
+ struct rte_cryptodev_scheduler_ops *ops;
+};
+
+extern struct rte_cryptodev_scheduler *roundrobin_scheduler;
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* _RTE_CRYPTO_SCHEDULER_H */
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h b/drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h
new file mode 100644
index 0000000..c19a9d3
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler_ioctls.h
@@ -0,0 +1,92 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef _RTE_CRYPTODEV_SCHEDULER_IOCTLS
+#define _RTE_CRYPTODEV_SCHEDULER_IOCTLS
+
+#include <rte_cryptodev_scheduler.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define RTE_CRYPTODEV_SCHEDULER_IOCTL_NAME_MAX_LEN (64)
+#define RTE_CRYPTODEV_SCHEDULER_IOCTL_DESC_MAX_LEN (256)
+
+#define RTE_CRYPTODEV_SCHEDULER_MAX_NB_IOCTLS (8)
+
+#define CDEV_SCHED_IOCTL_LIVE_MIGRATION_SCHED_STATE_GET (1)
+#define CDEV_SCHED_IOCTL_LIVE_MIGRATION_SCHED_MIGRATE (2)
+#define CDEV_SCHED_IOCTL_FALLBACK_SCHED_SET_PRIMARY (3)
+
+struct ioctl_migration_scheduler_state_get {
+ uint8_t slave_id;
+ /**< Active crypto device id */
+ enum migration_scheduler_state {
+ MIGRATION_SCHEDULER_STATE_ACTIVE,
+ MIGRATION_SCHEDULER_STATE_AWAITING_MIGRATE,
+ MIGRATION_SCHEDULER_STATE_MIGRATION
+ } state;
+ /**< Migration Scheduler State */
+};
+
+int
+rte_cryptodev_scheduler_ioctl(uint8_t scheduler_id, uint16_t request_id,
+ void *request_params);
+
+int
+rte_cryptodev_scheduler_ioctl_count(uint8_t scheduler_id);
+
+struct rte_cryptodev_scheduler_ioctl_description {
+ uint16_t request_id;
+ const char *name;
+ const char *description;
+};
+
+int
+rte_cryptodev_scheduler_ioctl_list(uint8_t scheduler_id,
+ struct rte_cryptodev_scheduler_ioctl_description **ioctls_desc,
+ uint16_t nb_ioctls);
+
+typedef int (*rte_cryptodev_scheduler_ioctl_fn)(void *request_params);
+
+struct rte_cryptodev_scheduler_ioctl {
+ int id;
+ char name[RTE_CRYPTODEV_SCHEDULER_IOCTL_NAME_MAX_LEN];
+ char description[RTE_CRYPTODEV_SCHEDULER_IOCTL_DESC_MAX_LEN];
+
+ rte_cryptodev_scheduler_ioctl_fn ioctl;
+};
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* _RTE_CRYPTODEV_SCHEDULER_IOCTLS */
diff --git a/drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h b/drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
new file mode 100644
index 0000000..ab8595b
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_cryptodev_scheduler_operations.h
@@ -0,0 +1,71 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_CRYPTO_SCHEDULER_OPERATIONS_H
+#define _RTE_CRYPTO_SCHEDULER_OPERATIONS_H
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef int (*rte_cryptodev_scheduler_slave_attach_t)(
+ struct rte_cryptodev *dev, uint8_t slave_id);
+typedef int (*rte_cryptodev_scheduler_slave_detach_t)(
+ struct rte_cryptodev *dev, uint8_t slave_id);
+
+typedef int (*rte_cryptodev_scheduler_start_t)(struct rte_cryptodev *dev);
+typedef int (*rte_cryptodev_scheduler_stop_t)(struct rte_cryptodev *dev);
+
+typedef int (*rte_cryptodev_scheduler_config_queue_pair)(
+ struct rte_cryptodev *dev, uint16_t qp_id);
+
+typedef int (*rte_cryptodev_scheduler_create_private_ctx)(
+ struct rte_cryptodev *dev);
+
+struct rte_cryptodev_scheduler_ops {
+ rte_cryptodev_scheduler_slave_attach_t slave_attach;
+ rte_cryptodev_scheduler_slave_attach_t slave_detach;
+
+ rte_cryptodev_scheduler_start_t scheduler_start;
+ rte_cryptodev_scheduler_stop_t scheduler_stop;
+
+ rte_cryptodev_scheduler_config_queue_pair config_queue_pair;
+
+ rte_cryptodev_scheduler_create_private_ctx create_private_ctx;
+};
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* _RTE_CRYPTO_SCHEDULER_OPERATIONS_H */
diff --git a/drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map b/drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map
new file mode 100644
index 0000000..0510f68
--- /dev/null
+++ b/drivers/crypto/scheduler/rte_pmd_crypto_scheduler_version.map
@@ -0,0 +1,12 @@
+DPDK_17.02 {
+ global:
+
+ rte_cryptodev_scheduler_load_user_scheduler;
+ rte_cryptodev_scheduler_slave_attach;
+ rte_cryptodev_scheduler_slave_detach;
+ rte_crpytodev_scheduler_mode_set;
+ rte_crpytodev_scheduler_mode_get;
+ rte_cryptodev_scheduler_ordering_set;
+ rte_cryptodev_scheduler_ordering_get;
+
+} DPDK_17.02;
\ No newline at end of file
diff --git a/drivers/crypto/scheduler/scheduler_pmd.c b/drivers/crypto/scheduler/scheduler_pmd.c
new file mode 100644
index 0000000..0c13b55
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_pmd.c
@@ -0,0 +1,168 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <rte_common.h>
+#include <rte_hexdump.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_vdev.h>
+#include <rte_malloc.h>
+#include <rte_cpuflags.h>
+#include <rte_reorder.h>
+#include <rte_cryptodev_scheduler.h>
+
+#include "scheduler_pmd_private.h"
+
+static uint16_t
+scheduler_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *qp_ctx = queue_pair;
+ uint16_t processed_ops;
+
+ processed_ops = (*qp_ctx->schedule_enqueue)(qp_ctx, ops,
+ nb_ops);
+
+ return processed_ops;
+}
+
+static uint16_t
+scheduler_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *qp_ctx = queue_pair;
+ uint16_t processed_ops;
+
+ processed_ops = (*qp_ctx->schedule_dequeue)(qp_ctx, ops,
+ nb_ops);
+
+ return processed_ops;
+}
+
+static uint32_t unique_name_id;
+
+static int
+cryptodev_scheduler_create(const char *name,
+ struct rte_crypto_vdev_init_params *init_params)
+{
+ char crypto_dev_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ struct rte_cryptodev *dev;
+ struct scheduler_ctx *sched_ctx;
+
+ if (snprintf(crypto_dev_name, RTE_CRYPTODEV_NAME_MAX_LEN, "%s_%u",
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), unique_name_id++) < 0) {
+ CS_LOG_ERR("driver %s: failed to create unique cryptodev "
+ "name", name);
+ return -EFAULT;
+ }
+
+ dev = rte_cryptodev_pmd_virtual_dev_init(crypto_dev_name,
+ sizeof(struct scheduler_ctx),
+ init_params->socket_id);
+ if (dev == NULL) {
+ CS_LOG_ERR("driver %s: failed to create cryptodev vdev",
+ name);
+ return -EFAULT;
+ }
+
+ dev->dev_type = RTE_CRYPTODEV_SCHEDULER_PMD;
+ dev->dev_ops = rte_crypto_scheduler_pmd_ops;
+
+ dev->enqueue_burst = scheduler_enqueue_burst;
+ dev->dequeue_burst = scheduler_dequeue_burst;
+
+ sched_ctx = dev->data->dev_private;
+ sched_ctx->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
+
+ return 0;
+}
+
+static int
+cryptodev_scheduler_remove(const char *name)
+{
+ struct rte_cryptodev *dev;
+ struct scheduler_ctx *sched_ctx;
+
+ if (name == NULL)
+ return -EINVAL;
+
+ dev = rte_cryptodev_pmd_get_named_dev(name);
+ if (dev == NULL)
+ return -EINVAL;
+
+ sched_ctx = dev->data->dev_private;
+
+ if (sched_ctx->nb_slaves) {
+ uint32_t i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++)
+ rte_cryptodev_scheduler_slave_detach(dev->data->dev_id,
+ sched_ctx->slaves[i].dev_id);
+ }
+
+ RTE_LOG(INFO, PMD, "Closing Crypto Scheduler device %s on numa "
+ "socket %u\n", name, rte_socket_id());
+
+ return 0;
+}
+
+static int
+cryptodev_scheduler_probe(const char *name, const char *input_args)
+{
+ struct rte_crypto_vdev_init_params init_params = {
+ RTE_CRYPTODEV_VDEV_DEFAULT_MAX_NB_QUEUE_PAIRS,
+ RTE_CRYPTODEV_VDEV_DEFAULT_MAX_NB_SESSIONS,
+ rte_socket_id()
+ };
+
+ rte_cryptodev_parse_vdev_init_params(&init_params, input_args);
+
+ RTE_LOG(INFO, PMD, "Initialising %s on NUMA node %d\n", name,
+ init_params.socket_id);
+ RTE_LOG(INFO, PMD, " Max number of queue pairs = %d\n",
+ init_params.max_nb_queue_pairs);
+ RTE_LOG(INFO, PMD, " Max number of sessions = %d\n",
+ init_params.max_nb_sessions);
+
+ return cryptodev_scheduler_create(name, &init_params);
+}
+
+static struct rte_vdev_driver cryptodev_scheduler_pmd_drv = {
+ .probe = cryptodev_scheduler_probe,
+ .remove = cryptodev_scheduler_remove
+};
+
+RTE_PMD_REGISTER_VDEV(CRYPTODEV_NAME_SCHEDULER_PMD,
+ cryptodev_scheduler_pmd_drv);
+RTE_PMD_REGISTER_PARAM_STRING(CRYPTODEV_NAME_SCHEDULER_PMD,
+ "max_nb_queue_pairs=<int> "
+ "max_nb_sessions=<int> "
+ "socket_id=<int>");
diff --git a/drivers/crypto/scheduler/scheduler_pmd_ops.c b/drivers/crypto/scheduler/scheduler_pmd_ops.c
new file mode 100644
index 0000000..972a355
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_pmd_ops.c
@@ -0,0 +1,495 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#include <string.h>
+
+#include <rte_config.h>
+#include <rte_common.h>
+#include <rte_malloc.h>
+#include <rte_dev.h>
+#include <rte_cryptodev.h>
+#include <rte_cryptodev_pmd.h>
+#include <rte_reorder.h>
+
+#include "scheduler_pmd_private.h"
+
+/** Configure device */
+static int
+scheduler_pmd_config(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+ int ret = 0;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ ret = (*slave_dev->dev_ops->dev_configure)(slave_dev);
+ if (ret < 0)
+ break;
+ }
+
+ return ret;
+}
+
+static int
+update_reorder_buff(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[qp_id];
+
+ if (sched_ctx->reordering_enabled) {
+ char reorder_buff_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+ uint32_t buff_size = sched_ctx->nb_slaves * PER_SLAVE_BUFF_SIZE;
+
+ if (qp_ctx->reorder_buf) {
+ rte_reorder_free(qp_ctx->reorder_buf);
+ qp_ctx->reorder_buf = NULL;
+ }
+
+ if (!buff_size)
+ return 0;
+
+ if (snprintf(reorder_buff_name, RTE_CRYPTODEV_NAME_MAX_LEN,
+ "%s_rb_%u_%u", RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD),
+ dev->data->dev_id, qp_id) < 0) {
+ CS_LOG_ERR("failed to create unique reorder buffer "
+ "name");
+ return -ENOMEM;
+ }
+
+ qp_ctx->reorder_buf = rte_reorder_create(reorder_buff_name,
+ rte_socket_id(), buff_size);
+ if (!qp_ctx->reorder_buf) {
+ CS_LOG_ERR("failed to create reorder buffer");
+ return -ENOMEM;
+ }
+ } else {
+ if (qp_ctx->reorder_buf) {
+ rte_reorder_free(qp_ctx->reorder_buf);
+ qp_ctx->reorder_buf = NULL;
+ }
+ }
+
+ return 0;
+}
+
+/** Start device */
+static int
+scheduler_pmd_start(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ uint32_t i;
+ int ret;
+
+ if (dev->data->dev_started)
+ return 0;
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ ret = update_reorder_buff(dev, i);
+ if (ret < 0) {
+ CS_LOG_ERR("Failed to update reorder buffer");
+ return ret;
+ }
+ }
+
+ if (sched_ctx->mode == CDEV_SCHED_MODE_NOT_SET) {
+ CS_LOG_ERR("Scheduler mode is not set");
+ return -1;
+ }
+
+ if (!sched_ctx->nb_slaves) {
+ CS_LOG_ERR("No slave in the scheduler");
+ return -1;
+ }
+
+ RTE_FUNC_PTR_OR_ERR_RET(*sched_ctx->ops.slave_attach, -ENOTSUP);
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+
+ if ((*sched_ctx->ops.slave_attach)(dev, slave_dev_id) < 0) {
+ CS_LOG_ERR("Failed to attach slave");
+ return -ENOTSUP;
+ }
+ }
+
+ RTE_FUNC_PTR_OR_ERR_RET(*sched_ctx->ops.scheduler_start, -ENOTSUP);
+
+ if ((*sched_ctx->ops.scheduler_start)(dev) < 0) {
+ CS_LOG_ERR("Scheduler start failed");
+ return -1;
+ }
+
+ /* start all slaves */
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ ret = (*slave_dev->dev_ops->dev_start)(slave_dev);
+ if (ret < 0) {
+ CS_LOG_ERR("Failed to start slave dev %u",
+ slave_dev_id);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+/** Stop device */
+static void
+scheduler_pmd_stop(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+
+ if (!dev->data->dev_started)
+ return;
+
+ /* stop all slaves first */
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ (*slave_dev->dev_ops->dev_stop)(slave_dev);
+ }
+
+ if (*sched_ctx->ops.scheduler_stop)
+ (*sched_ctx->ops.scheduler_stop)(dev);
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+
+ if (*sched_ctx->ops.slave_detach)
+ (*sched_ctx->ops.slave_detach)(dev, slave_dev_id);
+ }
+}
+
+/** Close device */
+static int
+scheduler_pmd_close(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+ int ret;
+
+ /* the dev should be stopped before being closed */
+ if (dev->data->dev_started)
+ return -EBUSY;
+
+ /* close all slaves first */
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ ret = (*slave_dev->dev_ops->dev_close)(slave_dev);
+ if (ret < 0)
+ return ret;
+ }
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[i];
+
+ if (qp_ctx->reorder_buf) {
+ rte_reorder_free(qp_ctx->reorder_buf);
+ qp_ctx->reorder_buf = NULL;
+ }
+
+ if (qp_ctx->private_qp_ctx) {
+ rte_free(qp_ctx->private_qp_ctx);
+ qp_ctx->private_qp_ctx = NULL;
+ }
+ }
+
+ if (sched_ctx->private_ctx)
+ rte_free(sched_ctx->private_ctx);
+
+ if (sched_ctx->capabilities)
+ rte_free(sched_ctx->capabilities);
+
+ if (sched_ctx->ioctls)
+ rte_free(sched_ctx->ioctls);
+
+ if (sched_ctx->options)
+ rte_free(sched_ctx->options);
+
+ return 0;
+}
+
+/** Get device statistics */
+static void
+scheduler_pmd_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+ struct rte_cryptodev_stats slave_stats = {0};
+
+ (*slave_dev->dev_ops->stats_get)(slave_dev, &slave_stats);
+
+ stats->enqueued_count += slave_stats.enqueued_count;
+ stats->dequeued_count += slave_stats.dequeued_count;
+
+ stats->enqueue_err_count += slave_stats.enqueue_err_count;
+ stats->dequeue_err_count += slave_stats.dequeue_err_count;
+ }
+}
+
+/** Reset device statistics */
+static void
+scheduler_pmd_stats_reset(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev *slave_dev =
+ rte_cryptodev_pmd_get_dev(slave_dev_id);
+
+ (*slave_dev->dev_ops->stats_reset)(slave_dev);
+ }
+}
+
+/** Get device info */
+static void
+scheduler_pmd_info_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *dev_info)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ unsigned max_nb_sessions = sched_ctx->nb_slaves ? UINT32_MAX : 0;
+ unsigned i;
+
+ if (!dev_info)
+ return;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ uint8_t slave_dev_id = sched_ctx->slaves[i].dev_id;
+ struct rte_cryptodev_info slave_info;
+
+ rte_cryptodev_info_get(slave_dev_id, &slave_info);
+ max_nb_sessions = slave_info.sym.max_nb_sessions <
+ max_nb_sessions ?
+ slave_info.sym.max_nb_sessions :
+ max_nb_sessions;
+ }
+
+ dev_info->dev_type = dev->dev_type;
+ dev_info->feature_flags = dev->feature_flags;
+ dev_info->capabilities = sched_ctx->capabilities;
+ dev_info->max_nb_queue_pairs = sched_ctx->max_nb_queue_pairs;
+ dev_info->sym.max_nb_sessions = max_nb_sessions;
+}
+
+/** Release queue pair */
+static int
+scheduler_pmd_qp_release(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[qp_id];
+
+ if (!qp_ctx)
+ return 0;
+
+ if (qp_ctx->reorder_buf)
+ rte_reorder_free(qp_ctx->reorder_buf);
+ if (qp_ctx->private_qp_ctx)
+ rte_free(qp_ctx->private_qp_ctx);
+
+ rte_free(qp_ctx);
+ dev->data->queue_pairs[qp_id] = NULL;
+
+ return 0;
+}
+
+/** Setup a queue pair */
+static int
+scheduler_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+ __rte_unused const struct rte_cryptodev_qp_conf *qp_conf, int socket_id)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+ struct scheduler_qp_ctx *qp_ctx;
+ char name[RTE_CRYPTODEV_NAME_MAX_LEN];
+
+ if (snprintf(name, RTE_CRYPTODEV_NAME_MAX_LEN,
+ "CRYTO_SCHE PMD %u QP %u",
+ dev->data->dev_id, qp_id) < 0) {
+ CS_LOG_ERR("Failed to create unique queue pair name");
+ return -EFAULT;
+ }
+
+ /* Free memory prior to re-allocation if needed. */
+ if (dev->data->queue_pairs[qp_id] != NULL)
+ scheduler_pmd_qp_release(dev, qp_id);
+
+ /* Allocate the queue pair data structure. */
+ qp_ctx = rte_zmalloc_socket(name, sizeof(*qp_ctx), RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (qp_ctx == NULL)
+ return -ENOMEM;
+
+ dev->data->queue_pairs[qp_id] = qp_ctx;
+
+ if (*sched_ctx->ops.config_queue_pair) {
+ if ((*sched_ctx->ops.config_queue_pair)(dev, qp_id) < 0) {
+ CS_LOG_ERR("Unable to configure queue pair");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/** Start queue pair */
+static int
+scheduler_pmd_qp_start(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint16_t queue_pair_id)
+{
+ return -ENOTSUP;
+}
+
+/** Stop queue pair */
+static int
+scheduler_pmd_qp_stop(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint16_t queue_pair_id)
+{
+ return -ENOTSUP;
+}
+
+/** Return the number of allocated queue pairs */
+static uint32_t
+scheduler_pmd_qp_count(struct rte_cryptodev *dev)
+{
+ return dev->data->nb_queue_pairs;
+}
+
+static unsigned
+scheduler_pmd_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct scheduler_session);
+}
+
+static int
+config_slave_sess(struct scheduler_ctx *sched_ctx,
+ struct rte_crypto_sym_xform *xform,
+ struct scheduler_session *sess,
+ uint32_t create)
+{
+ unsigned i;
+
+ for (i = 0; i < sched_ctx->nb_slaves; i++) {
+ struct scheduler_slave *slave = &sched_ctx->slaves[i];
+ struct rte_cryptodev *dev = &rte_cryptodev_globals->
+ devs[slave->dev_id];
+
+ if (sess->sessions[i]) {
+ if (create)
+ continue;
+ /* !create */
+ (*dev->dev_ops->session_clear)(dev,
+ (void *)sess->sessions[i]);
+ sess->sessions[i] = NULL;
+ } else {
+ if (!create)
+ continue;
+ /* create */
+ sess->sessions[i] =
+ rte_cryptodev_sym_session_create(
+ slave->dev_id, xform);
+ if (!sess->sessions[i]) {
+ config_slave_sess(sched_ctx, NULL, sess, 0);
+ return -1;
+ }
+ }
+ }
+
+ return 0;
+}
+
+/** Clear the memory of session so it doesn't leave key material behind */
+static void
+scheduler_pmd_session_clear(struct rte_cryptodev *dev,
+ void *sess)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+
+ config_slave_sess(sched_ctx, NULL, sess, 0);
+
+ memset(sess, 0, sizeof(struct scheduler_session));
+}
+
+static void *
+scheduler_pmd_session_configure(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform, void *sess)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+
+ if (config_slave_sess(sched_ctx, xform, sess, 1) < 0) {
+ CS_LOG_ERR("unabled to config sym session");
+ return NULL;
+ }
+
+ return sess;
+}
+
+struct rte_cryptodev_ops scheduler_pmd_ops = {
+ .dev_configure = scheduler_pmd_config,
+ .dev_start = scheduler_pmd_start,
+ .dev_stop = scheduler_pmd_stop,
+ .dev_close = scheduler_pmd_close,
+
+ .stats_get = scheduler_pmd_stats_get,
+ .stats_reset = scheduler_pmd_stats_reset,
+
+ .dev_infos_get = scheduler_pmd_info_get,
+
+ .queue_pair_setup = scheduler_pmd_qp_setup,
+ .queue_pair_release = scheduler_pmd_qp_release,
+ .queue_pair_start = scheduler_pmd_qp_start,
+ .queue_pair_stop = scheduler_pmd_qp_stop,
+ .queue_pair_count = scheduler_pmd_qp_count,
+
+ .session_get_size = scheduler_pmd_session_get_size,
+ .session_configure = scheduler_pmd_session_configure,
+ .session_clear = scheduler_pmd_session_clear,
+};
+
+struct rte_cryptodev_ops *rte_crypto_scheduler_pmd_ops = &scheduler_pmd_ops;
diff --git a/drivers/crypto/scheduler/scheduler_pmd_private.h b/drivers/crypto/scheduler/scheduler_pmd_private.h
new file mode 100644
index 0000000..550fdcc
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_pmd_private.h
@@ -0,0 +1,122 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015-2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _SCHEDULER_PMD_PRIVATE_H
+#define _SCHEDULER_PMD_PRIVATE_H
+
+#include <rte_hash.h>
+#include <rte_reorder.h>
+#include <rte_cryptodev_scheduler.h>
+#include <rte_cryptodev_scheduler_ioctls.h>
+
+/**< Maximum number of bonded devices per devices */
+#ifndef MAX_SLAVES_NUM
+#define MAX_SLAVES_NUM (8)
+#endif
+
+#define PER_SLAVE_BUFF_SIZE (256)
+
+#define CS_LOG_ERR(fmt, args...) \
+ RTE_LOG(ERR, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), \
+ __func__, __LINE__, ## args)
+
+#ifdef RTE_LIBRTE_CRYPTO_SCHEDULER_DEBUG
+#define CS_LOG_INFO(fmt, args...) \
+ RTE_LOG(INFO, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), \
+ __func__, __LINE__, ## args)
+
+#define CS_LOG_DBG(fmt, args...) \
+ RTE_LOG(DEBUG, CRYPTODEV, "[%s] %s() line %u: " fmt "\n", \
+ RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD), \
+ __func__, __LINE__, ## args)
+#else
+#define CS_LOG_INFO(fmt, args...)
+#define CS_LOG_DBG(fmt, args...)
+#endif
+
+struct scheduler_slave {
+ uint8_t dev_id;
+ uint16_t qp_id;
+ uint32_t nb_inflight_cops;
+
+ enum rte_cryptodev_type dev_type;
+};
+
+struct scheduler_ctx {
+ void *private_ctx;
+ /**< private scheduler context pointer */
+
+ struct rte_cryptodev_capabilities *capabilities;
+ unsigned nb_capabilities;
+
+ unsigned max_nb_queue_pairs;
+
+ struct scheduler_slave slaves[MAX_SLAVES_NUM];
+ unsigned nb_slaves;
+
+ enum rte_cryptodev_scheduler_mode mode;
+
+ uint32_t ioctl_count;
+ struct rte_cryptodev_scheduler_ioctl **ioctls;
+
+ uint32_t nb_options;
+ struct rte_cryptodev_scheduler_option **options;
+
+ struct rte_cryptodev_scheduler_ops ops;
+
+ uint8_t reordering_enabled;
+
+ char name[RTE_CRYPTODEV_SCHEDULER_NAME_MAX_LEN];
+ char description[RTE_CRYPTODEV_SCHEDULER_DESC_MAX_LEN];
+} __rte_cache_aligned;
+
+struct scheduler_qp_ctx {
+ void *private_qp_ctx;
+
+ rte_cryptodev_scheduler_burst_enqueue_t schedule_enqueue;
+ rte_cryptodev_scheduler_burst_dequeue_t schedule_dequeue;
+
+ struct rte_reorder_buffer *reorder_buf;
+ uint32_t seqn;
+} __rte_cache_aligned;
+
+struct scheduler_session {
+ struct rte_cryptodev_sym_session *sessions[MAX_SLAVES_NUM];
+};
+
+/** device specific operations function pointer structure */
+extern struct rte_cryptodev_ops *rte_crypto_scheduler_pmd_ops;
+
+#endif /* _SCHEDULER_PMD_PRIVATE_H */
diff --git a/drivers/crypto/scheduler/scheduler_roundrobin.c b/drivers/crypto/scheduler/scheduler_roundrobin.c
new file mode 100644
index 0000000..be0b7fd
--- /dev/null
+++ b/drivers/crypto/scheduler/scheduler_roundrobin.c
@@ -0,0 +1,419 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_cryptodev.h>
+#include <rte_malloc.h>
+#include <rte_cryptodev_scheduler_operations.h>
+
+#include "scheduler_pmd_private.h"
+
+struct roundround_scheduler_ctx {
+};
+
+struct rr_scheduler_qp_ctx {
+ struct scheduler_slave slaves[MAX_SLAVES_NUM];
+ unsigned nb_slaves;
+
+ unsigned last_enq_slave_idx;
+ unsigned last_deq_slave_idx;
+};
+
+static uint16_t
+schedule_enqueue(void *qp_ctx, struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ ((struct scheduler_qp_ctx *)qp_ctx)->private_qp_ctx;
+ uint32_t slave_idx = rr_qp_ctx->last_enq_slave_idx;
+ struct scheduler_slave *slave = &rr_qp_ctx->slaves[slave_idx];
+ uint16_t i, processed_ops;
+ struct scheduler_session *sess0, *sess1, *sess2, *sess3;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ for (i = 0; i < nb_ops && i < 4; i++)
+ rte_prefetch0(ops[i]->sym->session);
+
+ for (i = 0; i < nb_ops - 8; i += 4) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ sess1 = (struct scheduler_session *)
+ ops[i+1]->sym->session->_private;
+ sess2 = (struct scheduler_session *)
+ ops[i+2]->sym->session->_private;
+ sess3 = (struct scheduler_session *)
+ ops[i+3]->sym->session->_private;
+
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ ops[i + 1]->sym->session = sess1->sessions[slave_idx];
+ ops[i + 2]->sym->session = sess2->sessions[slave_idx];
+ ops[i + 3]->sym->session = sess3->sessions[slave_idx];
+
+ rte_prefetch0(ops[i + 4]->sym->session);
+ rte_prefetch0(ops[i + 5]->sym->session);
+ rte_prefetch0(ops[i + 6]->sym->session);
+ rte_prefetch0(ops[i + 7]->sym->session);
+ }
+
+ for (; i < nb_ops; i++) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ }
+
+ processed_ops = rte_cryptodev_enqueue_burst(slave->dev_id,
+ slave->qp_id, ops, nb_ops);
+
+ slave->nb_inflight_cops += processed_ops;
+
+ rr_qp_ctx->last_enq_slave_idx += 1;
+ if (unlikely(rr_qp_ctx->last_enq_slave_idx >= rr_qp_ctx->nb_slaves))
+ rr_qp_ctx->last_enq_slave_idx = 0;
+
+ return processed_ops;
+}
+
+static uint16_t
+schedule_enqueue_ordering(void *qp_ctx, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *gen_qp_ctx = qp_ctx;
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ gen_qp_ctx->private_qp_ctx;
+ uint32_t slave_idx = rr_qp_ctx->last_enq_slave_idx;
+ struct scheduler_slave *slave = &rr_qp_ctx->slaves[slave_idx];
+ uint16_t i, processed_ops;
+ struct scheduler_session *sess0, *sess1, *sess2, *sess3;
+
+ if (unlikely(nb_ops == 0))
+ return 0;
+
+ for (i = 0; i < nb_ops && i < 4; i++) {
+ rte_prefetch0(ops[i]->sym->session);
+ rte_prefetch0(ops[i]->sym->m_src);
+ }
+
+ for (i = 0; i < nb_ops - 8; i += 4) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ sess1 = (struct scheduler_session *)
+ ops[i+1]->sym->session->_private;
+ sess2 = (struct scheduler_session *)
+ ops[i+2]->sym->session->_private;
+ sess3 = (struct scheduler_session *)
+ ops[i+3]->sym->session->_private;
+
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ ops[i]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ ops[i + 1]->sym->session = sess1->sessions[slave_idx];
+ ops[i + 1]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ ops[i + 2]->sym->session = sess2->sessions[slave_idx];
+ ops[i + 2]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ ops[i + 3]->sym->session = sess3->sessions[slave_idx];
+ ops[i + 3]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+
+ rte_prefetch0(ops[i + 4]->sym->session);
+ rte_prefetch0(ops[i + 4]->sym->m_src);
+ rte_prefetch0(ops[i + 5]->sym->session);
+ rte_prefetch0(ops[i + 5]->sym->m_src);
+ rte_prefetch0(ops[i + 6]->sym->session);
+ rte_prefetch0(ops[i + 6]->sym->m_src);
+ rte_prefetch0(ops[i + 7]->sym->session);
+ rte_prefetch0(ops[i + 7]->sym->m_src);
+ }
+
+ for (; i < nb_ops; i++) {
+ sess0 = (struct scheduler_session *)
+ ops[i]->sym->session->_private;
+ ops[i]->sym->session = sess0->sessions[slave_idx];
+ ops[i]->sym->m_src->seqn = gen_qp_ctx->seqn++;
+ }
+
+ processed_ops = rte_cryptodev_enqueue_burst(slave->dev_id,
+ slave->qp_id, ops, nb_ops);
+
+ slave->nb_inflight_cops += processed_ops;
+
+ rr_qp_ctx->last_enq_slave_idx += 1;
+ if (unlikely(rr_qp_ctx->last_enq_slave_idx >= rr_qp_ctx->nb_slaves))
+ rr_qp_ctx->last_enq_slave_idx = 0;
+
+ return processed_ops;
+}
+
+
+static uint16_t
+schedule_dequeue(void *qp_ctx, struct rte_crypto_op **ops, uint16_t nb_ops)
+{
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ ((struct scheduler_qp_ctx *)qp_ctx)->private_qp_ctx;
+ struct scheduler_slave *slave;
+ uint32_t last_slave_idx = rr_qp_ctx->last_deq_slave_idx;
+ uint16_t nb_deq_ops;
+
+ if (unlikely(rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops == 0)) {
+ do {
+ last_slave_idx += 1;
+
+ if (unlikely(last_slave_idx >= rr_qp_ctx->nb_slaves))
+ last_slave_idx = 0;
+ /* looped back, means no inflight cops in the queue */
+ if (last_slave_idx == rr_qp_ctx->last_deq_slave_idx)
+ return 0;
+ } while (rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops
+ == 0);
+ }
+
+ slave = &rr_qp_ctx->slaves[last_slave_idx];
+
+ nb_deq_ops = rte_cryptodev_dequeue_burst(slave->dev_id,
+ slave->qp_id, ops, nb_ops);
+
+ last_slave_idx += 1;
+ if (unlikely(last_slave_idx >= rr_qp_ctx->nb_slaves))
+ last_slave_idx = 0;
+
+ rr_qp_ctx->last_deq_slave_idx = last_slave_idx;
+
+ slave->nb_inflight_cops -= nb_deq_ops;
+
+ return nb_deq_ops;
+}
+
+static uint16_t
+schedule_dequeue_ordering(void *qp_ctx, struct rte_crypto_op **ops,
+ uint16_t nb_ops)
+{
+ struct scheduler_qp_ctx *gen_qp_ctx = (struct scheduler_qp_ctx *)qp_ctx;
+ struct rr_scheduler_qp_ctx *rr_qp_ctx = (gen_qp_ctx->private_qp_ctx);
+ struct scheduler_slave *slave;
+ struct rte_reorder_buffer *reorder_buff = gen_qp_ctx->reorder_buf;
+ struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3;
+ uint16_t nb_deq_ops, nb_drained_mbufs;
+ const uint16_t nb_op_ops = nb_ops;
+ struct rte_crypto_op *op_ops[nb_op_ops];
+ struct rte_mbuf *reorder_mbufs[nb_op_ops];
+ uint32_t last_slave_idx = rr_qp_ctx->last_deq_slave_idx;
+ uint16_t i;
+
+ if (unlikely(rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops == 0)) {
+ do {
+ last_slave_idx += 1;
+
+ if (unlikely(last_slave_idx >= rr_qp_ctx->nb_slaves))
+ last_slave_idx = 0;
+ /* looped back, means no inflight cops in the queue */
+ if (last_slave_idx == rr_qp_ctx->last_deq_slave_idx)
+ return 0;
+ } while (rr_qp_ctx->slaves[last_slave_idx].nb_inflight_cops
+ == 0);
+ }
+
+ slave = &rr_qp_ctx->slaves[last_slave_idx];
+
+ nb_deq_ops = rte_cryptodev_dequeue_burst(slave->dev_id,
+ slave->qp_id, op_ops, nb_ops);
+
+ rr_qp_ctx->last_deq_slave_idx += 1;
+ if (unlikely(rr_qp_ctx->last_deq_slave_idx >= rr_qp_ctx->nb_slaves))
+ rr_qp_ctx->last_deq_slave_idx = 0;
+
+ slave->nb_inflight_cops -= nb_deq_ops;
+
+ for (i = 0; i < nb_deq_ops && i < 4; i++)
+ rte_prefetch0(op_ops[i]->sym->m_src);
+
+ for (i = 0; i < nb_deq_ops - 8; i += 4) {
+ mbuf0 = op_ops[i]->sym->m_src;
+ mbuf1 = op_ops[i + 1]->sym->m_src;
+ mbuf2 = op_ops[i + 2]->sym->m_src;
+ mbuf3 = op_ops[i + 3]->sym->m_src;
+
+ rte_memcpy(mbuf0->buf_addr, &op_ops[i], sizeof(op_ops[i]));
+ rte_memcpy(mbuf1->buf_addr, &op_ops[i+1], sizeof(op_ops[i+1]));
+ rte_memcpy(mbuf2->buf_addr, &op_ops[i+2], sizeof(op_ops[i+2]));
+ rte_memcpy(mbuf3->buf_addr, &op_ops[i+3], sizeof(op_ops[i+3]));
+
+ rte_reorder_insert(reorder_buff, mbuf0);
+ rte_reorder_insert(reorder_buff, mbuf1);
+ rte_reorder_insert(reorder_buff, mbuf2);
+ rte_reorder_insert(reorder_buff, mbuf3);
+
+ rte_prefetch0(op_ops[i + 4]->sym->m_src);
+ rte_prefetch0(op_ops[i + 5]->sym->m_src);
+ rte_prefetch0(op_ops[i + 6]->sym->m_src);
+ rte_prefetch0(op_ops[i + 7]->sym->m_src);
+ }
+
+ for (; i < nb_deq_ops; i++) {
+ mbuf0 = op_ops[i]->sym->m_src;
+ rte_memcpy(mbuf0->buf_addr, &op_ops[i], sizeof(op_ops[i]));
+ rte_reorder_insert(reorder_buff, mbuf0);
+ }
+
+ nb_drained_mbufs = rte_reorder_drain(reorder_buff, reorder_mbufs,
+ nb_ops);
+ for (i = 0; i < nb_drained_mbufs && i < 4; i++)
+ rte_prefetch0(reorder_mbufs[i]);
+
+ for (i = 0; i < nb_drained_mbufs - 8; i += 4) {
+ ops[i] = *(struct rte_crypto_op **)reorder_mbufs[i]->buf_addr;
+ ops[i + 1] = *(struct rte_crypto_op **)
+ reorder_mbufs[i + 1]->buf_addr;
+ ops[i + 2] = *(struct rte_crypto_op **)
+ reorder_mbufs[i + 2]->buf_addr;
+ ops[i + 3] = *(struct rte_crypto_op **)
+ reorder_mbufs[i + 3]->buf_addr;
+
+ *(struct rte_crypto_op **)reorder_mbufs[i]->buf_addr = NULL;
+ *(struct rte_crypto_op **)reorder_mbufs[i + 1]->buf_addr = NULL;
+ *(struct rte_crypto_op **)reorder_mbufs[i + 2]->buf_addr = NULL;
+ *(struct rte_crypto_op **)reorder_mbufs[i + 3]->buf_addr = NULL;
+
+ rte_prefetch0(reorder_mbufs[i + 4]);
+ rte_prefetch0(reorder_mbufs[i + 5]);
+ rte_prefetch0(reorder_mbufs[i + 6]);
+ rte_prefetch0(reorder_mbufs[i + 7]);
+ }
+
+ for (; i < nb_drained_mbufs; i++) {
+ ops[i] = *(struct rte_crypto_op **)
+ reorder_mbufs[i]->buf_addr;
+ *(struct rte_crypto_op **)reorder_mbufs[i]->buf_addr = NULL;
+ }
+
+ return nb_drained_mbufs;
+}
+
+static int
+slave_attach(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint8_t slave_id)
+{
+ return 0;
+}
+
+static int
+slave_detach(__rte_unused struct rte_cryptodev *dev,
+ __rte_unused uint8_t slave_id)
+{
+ return 0;
+}
+
+static int
+scheduler_start(struct rte_cryptodev *dev)
+{
+ struct scheduler_ctx *sched_ctx = dev->data->dev_private;
+
+ uint16_t i;
+
+ for (i = 0; i < dev->data->nb_queue_pairs; i++) {
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[i];
+ struct rr_scheduler_qp_ctx *rr_qp_ctx =
+ qp_ctx->private_qp_ctx;
+ uint32_t j;
+ uint16_t qp_id = rr_qp_ctx->slaves[0].qp_id;
+
+ memset(rr_qp_ctx->slaves, 0, MAX_SLAVES_NUM *
+ sizeof(struct scheduler_slave));
+ for (j = 0; j < sched_ctx->nb_slaves; j++) {
+ rr_qp_ctx->slaves[j].dev_id =
+ sched_ctx->slaves[i].dev_id;
+ rr_qp_ctx->slaves[j].qp_id = qp_id;
+ }
+
+ rr_qp_ctx->nb_slaves = sched_ctx->nb_slaves;
+
+ rr_qp_ctx->last_enq_slave_idx = 0;
+ rr_qp_ctx->last_deq_slave_idx = 0;
+
+ if (sched_ctx->reordering_enabled) {
+ qp_ctx->schedule_enqueue = &schedule_enqueue_ordering;
+ qp_ctx->schedule_dequeue = &schedule_dequeue_ordering;
+ } else {
+ qp_ctx->schedule_enqueue = &schedule_enqueue;
+ qp_ctx->schedule_dequeue = &schedule_dequeue;
+ }
+ }
+
+ return 0;
+}
+
+static int
+scheduler_stop(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+static int
+scheduler_config_qp(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct scheduler_qp_ctx *qp_ctx = dev->data->queue_pairs[qp_id];
+ struct rr_scheduler_qp_ctx *rr_qp_ctx;
+
+ rr_qp_ctx = rte_zmalloc_socket(NULL, sizeof(*rr_qp_ctx), 0,
+ rte_socket_id());
+ if (!rr_qp_ctx) {
+ CS_LOG_ERR("failed allocate memory for private queue pair");
+ return -ENOMEM;
+ }
+
+ qp_ctx->private_qp_ctx = (void *)rr_qp_ctx;
+
+ return 0;
+}
+
+static int
+scheduler_create_private_ctx(__rte_unused struct rte_cryptodev *dev)
+{
+ return 0;
+}
+
+struct rte_cryptodev_scheduler_ops ops = {
+ slave_attach,
+ slave_detach,
+ scheduler_start,
+ scheduler_stop,
+ scheduler_config_qp,
+ scheduler_create_private_ctx
+};
+
+struct rte_cryptodev_scheduler scheduler = {
+ .name = "roundrobin-scheduler",
+ .description = "scheduler which will round robin burst across "
+ "slave crypto devices",
+ .options = NULL,
+ .ops = &ops,
+ .ioctls = NULL
+};
+
+
+struct rte_cryptodev_scheduler *roundrobin_scheduler = &scheduler;
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 8f63e8f..61a3ce0 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -66,6 +66,7 @@ extern "C" {
/**< KASUMI PMD device name */
#define CRYPTODEV_NAME_ZUC_PMD crypto_zuc
/**< KASUMI PMD device name */
+#define CRYPTODEV_NAME_SCHEDULER_PMD crypto_scheduler
/** Crypto device type */
enum rte_cryptodev_type {
@@ -77,6 +78,9 @@ enum rte_cryptodev_type {
RTE_CRYPTODEV_KASUMI_PMD, /**< KASUMI PMD */
RTE_CRYPTODEV_ZUC_PMD, /**< ZUC PMD */
RTE_CRYPTODEV_OPENSSL_PMD, /**< OpenSSL PMD */
+ RTE_CRYPTODEV_SCHEDULER_PMD, /**< Crypto Scheduler PMD */
+
+ RTE_CRYPTODEV_TYPE_COUNT
};
extern const char **rte_cyptodev_names;
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index f75f0e2..ee34688 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -70,7 +70,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PORT) += -lrte_port
_LDLIBS-$(CONFIG_RTE_LIBRTE_PDUMP) += -lrte_pdump
_LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) += -lrte_distributor
-_LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder
_LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG) += -lrte_ip_frag
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lrte_meter
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched
@@ -98,6 +97,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring
_LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal
_LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline
_LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
+_LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += -lrte_pmd_bond
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += -lrte_pmd_xenvirt -lxenstore
@@ -145,6 +145,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += -lrte_pmd_zuc
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZUC) += -L$(LIBSSO_ZUC_PATH)/build -lsso_zuc
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER) += -lrte_pmd_crypto_scheduler
endif # CONFIG_RTE_LIBRTE_CRYPTODEV
endif # !CONFIG_RTE_BUILD_SHARED_LIBS
--
2.7.4
^ permalink raw reply related
* Re: [dpdk-stable] [PATCH] net/mlx5: fix RSS hash result for flows
From: Ferruh Yigit @ 2017-01-03 17:03 UTC (permalink / raw)
To: Nelio Laranjeiro, dev; +Cc: Adrien Mazarguil, stable
In-Reply-To: <3c12f3f7504908ddc1bc7119b4e69ba9581b8321.1482918974.git.nelio.laranjeiro@6wind.com>
On 12/28/2016 9:58 AM, Nelio Laranjeiro wrote:
> Flows redirected to a specific queue do not have a valid RSS hash result
> and the related mbuf flag must not be set.
>
> Fixes: ecf60761fc2a ("net/mlx5: return RSS hash result in mbuf")
>
> CC: stable@dpdk.org
> Signed-off-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>
> Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Applied to dpdk-next-net/master, thanks.
^ permalink raw reply
* [PATCH v4 2/2] net/vhost: emulate device start/stop behavior
From: Charles (Chas) Williams @ 2017-01-03 16:22 UTC (permalink / raw)
To: dev; +Cc: mtetsuyah, yuanhan.liu, Charles (Chas) Williams
In-Reply-To: <1483460563-5003-1-git-send-email-ciwillia@brocade.com>
.dev_start()/.dev_stop() roughly corresponds to the local device's port
being ready. This is different from the remote client being connected
which is roughly link up or down. Emulate the device start/stop behavior
by separately tracking the start/stop state to determine if we should
allow packets to be queued to/from the remote client.
Signed-off-by: Chas Williams <ciwillia@brocade.com>
---
drivers/net/vhost/rte_eth_vhost.c | 82 ++++++++++++++++++++++++---------------
1 file changed, 51 insertions(+), 31 deletions(-)
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index c669e79..0b5b80a 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -110,9 +110,11 @@ struct vhost_queue {
};
struct pmd_internal {
+ rte_atomic32_t dev_attached;
char *dev_name;
char *iface_name;
uint16_t max_queues;
+ rte_atomic32_t started;
};
struct internal_list {
@@ -490,6 +492,38 @@ find_internal_resource(char *ifname)
return list;
}
+static void
+update_queuing_status(struct rte_eth_dev *dev)
+{
+ struct pmd_internal *internal = dev->data->dev_private;
+ struct vhost_queue *vq;
+ unsigned int i;
+ int allow_queuing = 1;
+
+ if (rte_atomic32_read(&internal->started) == 0 ||
+ rte_atomic32_read(&internal->dev_attached) == 0)
+ allow_queuing = 0;
+
+ /* Wait until rx/tx_pkt_burst stops accessing vhost device */
+ for (i = 0; i < dev->data->nb_rx_queues; i++) {
+ vq = dev->data->rx_queues[i];
+ if (vq == NULL)
+ continue;
+ rte_atomic32_set(&vq->allow_queuing, allow_queuing);
+ while (rte_atomic32_read(&vq->while_queuing))
+ rte_pause();
+ }
+
+ for (i = 0; i < dev->data->nb_tx_queues; i++) {
+ vq = dev->data->tx_queues[i];
+ if (vq == NULL)
+ continue;
+ rte_atomic32_set(&vq->allow_queuing, allow_queuing);
+ while (rte_atomic32_read(&vq->while_queuing))
+ rte_pause();
+ }
+}
+
static int
new_device(int vid)
{
@@ -541,18 +575,8 @@ new_device(int vid)
eth_dev->data->dev_link.link_status = ETH_LINK_UP;
- for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
- vq = eth_dev->data->rx_queues[i];
- if (vq == NULL)
- continue;
- rte_atomic32_set(&vq->allow_queuing, 1);
- }
- for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
- vq = eth_dev->data->tx_queues[i];
- if (vq == NULL)
- continue;
- rte_atomic32_set(&vq->allow_queuing, 1);
- }
+ rte_atomic32_set(&internal->dev_attached, 1);
+ update_queuing_status(eth_dev);
RTE_LOG(INFO, PMD, "New connection established\n");
@@ -565,6 +589,7 @@ static void
destroy_device(int vid)
{
struct rte_eth_dev *eth_dev;
+ struct pmd_internal *internal;
struct vhost_queue *vq;
struct internal_list *list;
char ifname[PATH_MAX];
@@ -578,24 +603,10 @@ destroy_device(int vid)
return;
}
eth_dev = list->eth_dev;
+ internal = eth_dev->data->dev_private;
- /* Wait until rx/tx_pkt_burst stops accessing vhost device */
- for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
- vq = eth_dev->data->rx_queues[i];
- if (vq == NULL)
- continue;
- rte_atomic32_set(&vq->allow_queuing, 0);
- while (rte_atomic32_read(&vq->while_queuing))
- rte_pause();
- }
- for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
- vq = eth_dev->data->tx_queues[i];
- if (vq == NULL)
- continue;
- rte_atomic32_set(&vq->allow_queuing, 0);
- while (rte_atomic32_read(&vq->while_queuing))
- rte_pause();
- }
+ rte_atomic32_set(&internal->dev_attached, 0);
+ update_queuing_status(eth_dev);
eth_dev->data->dev_link.link_status = ETH_LINK_DOWN;
@@ -769,14 +780,23 @@ vhost_driver_session_stop(void)
}
static int
-eth_dev_start(struct rte_eth_dev *dev __rte_unused)
+eth_dev_start(struct rte_eth_dev *dev)
{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ rte_atomic32_set(&internal->started, 1);
+ update_queuing_status(dev);
+
return 0;
}
static void
-eth_dev_stop(struct rte_eth_dev *dev __rte_unused)
+eth_dev_stop(struct rte_eth_dev *dev)
{
+ struct pmd_internal *internal = dev->data->dev_private;
+
+ rte_atomic32_set(&internal->started, 0);
+ update_queuing_status(dev);
}
static int
--
2.1.4
^ permalink raw reply related
* [PATCH v4 1/2] net/vhost: create datagram sockets immediately
From: Charles (Chas) Williams @ 2017-01-03 16:22 UTC (permalink / raw)
To: dev; +Cc: mtetsuyah, yuanhan.liu, Charles (Chas) Williams
If you create a vhost server device, it doesn't create the actual datagram
socket until you call .dev_start(). If you call .dev_stop() is also
deletes those sockets. For QEMU clients, this is a problem since QEMU
doesn't know how to re-attach to datagram sockets that have gone away.
To fix this, register and unregister the datagram sockets during device
creation and removal.
Fixes: ee584e9710b9 ("vhost: add driver on top of the library")
Signed-off-by: Chas Williams <ciwillia@brocade.com>
---
drivers/net/vhost/rte_eth_vhost.c | 45 +++++++++++++++------------------------
1 file changed, 17 insertions(+), 28 deletions(-)
diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 60b0f51..c669e79 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -113,9 +113,6 @@ struct pmd_internal {
char *dev_name;
char *iface_name;
uint16_t max_queues;
- uint64_t flags;
-
- volatile uint16_t once;
};
struct internal_list {
@@ -772,35 +769,14 @@ vhost_driver_session_stop(void)
}
static int
-eth_dev_start(struct rte_eth_dev *dev)
+eth_dev_start(struct rte_eth_dev *dev __rte_unused)
{
- struct pmd_internal *internal = dev->data->dev_private;
- int ret = 0;
-
- if (rte_atomic16_cmpset(&internal->once, 0, 1)) {
- ret = rte_vhost_driver_register(internal->iface_name,
- internal->flags);
- if (ret)
- return ret;
- }
-
- /* We need only one message handling thread */
- if (rte_atomic16_add_return(&nb_started_ports, 1) == 1)
- ret = vhost_driver_session_start();
-
- return ret;
+ return 0;
}
static void
-eth_dev_stop(struct rte_eth_dev *dev)
+eth_dev_stop(struct rte_eth_dev *dev __rte_unused)
{
- struct pmd_internal *internal = dev->data->dev_private;
-
- if (rte_atomic16_cmpset(&internal->once, 1, 0))
- rte_vhost_driver_unregister(internal->iface_name);
-
- if (rte_atomic16_sub_return(&nb_started_ports, 1) == 0)
- vhost_driver_session_stop();
}
static int
@@ -1043,7 +1019,6 @@ eth_dev_vhost_create(const char *name, char *iface_name, int16_t queues,
internal->iface_name = strdup(iface_name);
if (internal->iface_name == NULL)
goto error;
- internal->flags = flags;
list->eth_dev = eth_dev;
pthread_mutex_lock(&internal_list_lock);
@@ -1078,6 +1053,15 @@ eth_dev_vhost_create(const char *name, char *iface_name, int16_t queues,
eth_dev->rx_pkt_burst = eth_vhost_rx;
eth_dev->tx_pkt_burst = eth_vhost_tx;
+ if (rte_vhost_driver_register(iface_name, flags))
+ goto error;
+
+ /* We need only one message handling thread */
+ if (rte_atomic16_add_return(&nb_started_ports, 1) == 1) {
+ if (vhost_driver_session_start())
+ goto error;
+ }
+
return data->port_id;
error:
@@ -1215,6 +1199,11 @@ rte_pmd_vhost_remove(const char *name)
eth_dev_stop(eth_dev);
+ rte_vhost_driver_unregister(internal->iface_name);
+
+ if (rte_atomic16_sub_return(&nb_started_ports, 1) == 0)
+ vhost_driver_session_stop();
+
rte_free(vring_states[eth_dev->data->port_id]);
vring_states[eth_dev->data->port_id] = NULL;
--
2.1.4
^ permalink raw reply related
* Re: [PATCH v5 0/6] net/mlx5: support flow API
From: Ferruh Yigit @ 2017-01-03 16:19 UTC (permalink / raw)
To: Nelio Laranjeiro, dev; +Cc: Adrien Mazarguil
In-Reply-To: <cover.1483022600.git.nelio.laranjeiro@6wind.com>
On 12/29/2016 3:15 PM, Nelio Laranjeiro wrote:
> Changes in v5:
>
> - Fix masking when only spec is present in item structure.
> - Fix first element of flow items array.
>
> Changes in v4:
>
> - Simplify flow parsing by using a graph.
> - Add VXLAN flow item.
> - Add mark flow action.
> - Extend IPv4 filter item (Type of service, Next Protocol ID).
>
> Changes in v3:
>
> - Fix Ethernet ether type issue.
>
> Changes in v2:
>
> - Fix several issues.
> - Support VLAN filtering.
>
> Nelio Laranjeiro (6):
> net/mlx5: add preliminary flow API support
> net/mlx5: support basic flow items and actions
> net/mlx5: support VLAN flow item
> net/mlx5: support VXLAN flow item
> net/mlx5: support mark flow action
> net/mlx5: extend IPv4 flow item
This patch is giving ICC warnings [1], but please check:
http://dpdk.org/dev/patchwork/patch/18808/
[1]
.../drivers/net/mlx5/mlx5_flow.c(550): error #188: enumerated type mixed
with another type
.type = flow->inner | IBV_EXP_FLOW_SPEC_ETH,
^
.../drivers/net/mlx5/mlx5_flow.c(626): error #188: enumerated type mixed
with another type
.type = flow->inner | IBV_EXP_FLOW_SPEC_IPV4_EXT,
^
.../drivers/net/mlx5/mlx5_flow.c(679): error #188: enumerated type mixed
with another type
.type = flow->inner | IBV_EXP_FLOW_SPEC_IPV6,
^
.../drivers/net/mlx5/mlx5_flow.c(727): error #188: enumerated type mixed
with another type
.type = flow->inner | IBV_EXP_FLOW_SPEC_UDP,
^
.../drivers/net/mlx5/mlx5_flow.c(769): error #188: enumerated type mixed
with another type
.type = flow->inner | IBV_EXP_FLOW_SPEC_TCP,
^
.../drivers/net/mlx5/mlx5_flow.c(816): error #188: enumerated type mixed
with another type
.type = flow->inner | IBV_EXP_FLOW_SPEC_VXLAN_TUNNEL,
^
>
> drivers/net/mlx5/Makefile | 1 +
> drivers/net/mlx5/mlx5.h | 19 +
> drivers/net/mlx5/mlx5_fdir.c | 15 +
> drivers/net/mlx5/mlx5_flow.c | 1248 +++++++++++++++++++++++++++++++++++++++
> drivers/net/mlx5/mlx5_prm.h | 70 ++-
> drivers/net/mlx5/mlx5_rxtx.c | 12 +-
> drivers/net/mlx5/mlx5_rxtx.h | 3 +-
> drivers/net/mlx5/mlx5_trigger.c | 2 +
> 8 files changed, 1367 insertions(+), 3 deletions(-)
> create mode 100644 drivers/net/mlx5/mlx5_flow.c
>
^ permalink raw reply
* [PATCH] mk: disable ICC warning 188
From: Ferruh Yigit @ 2017-01-03 16:15 UTC (permalink / raw)
To: dev; +Cc: Thomas Monjalon, Ferruh Yigit
error #188: enumerated type mixed with another type
This is get when an integer assigned to an enum variable.
Since this usage is common and causing many ICC compilation errors, and
other compilers accept this usage. Disabling the warning.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
mk/toolchain/icc/rte.vars.mk | 1 +
1 file changed, 1 insertion(+)
diff --git a/mk/toolchain/icc/rte.vars.mk b/mk/toolchain/icc/rte.vars.mk
index ba69f1f..86d9ef7 100644
--- a/mk/toolchain/icc/rte.vars.mk
+++ b/mk/toolchain/icc/rte.vars.mk
@@ -71,6 +71,7 @@ TOOLCHAIN_ASFLAGS =
# was declared "deprecated"
WERROR_FLAGS := -Wall -w2 -diag-disable 271 -diag-warning 1478
WERROR_FLAGS += -diag-disable 13368 -diag-disable 15527
+WERROR_FLAGS += -diag-disable 188
ifeq ($(RTE_DEVEL_BUILD),y)
WERROR_FLAGS += -Werror-all
--
2.9.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox