DPDK-dev Archive on lore.kernel.org
 help / color / mirror / Atom feed
* Re: Proposal for a new Committer model
From: Neil Horman @ 2016-11-23 13:48 UTC (permalink / raw)
  To: Ferruh Yigit; +Cc: Thomas Monjalon, dev, Mcnamara, John
In-Reply-To: <52ed2fa2-da41-1301-2d56-0fec05b79ce5@intel.com>

On Tue, Nov 22, 2016 at 08:56:23PM +0000, Ferruh Yigit wrote:
> On 11/22/2016 7:52 PM, Neil Horman wrote:
> > On Mon, Nov 21, 2016 at 09:52:41AM +0100, Thomas Monjalon wrote:
> >> 2016-11-18 13:09, Neil Horman:
> >>> A) Further promote subtree maintainership.  This was a conversation that I
> >>> proposed some time ago, but my proposed granularity was discarded in favor
> >>> of something that hasn't worked as well (in my opinion).  That is to say a
> >>> few driver pmds (i40e and fm10k come to mind) have their own tree that
> >>> send pull requests to Thomas.
> >>
> >> Yes we tried this fine granularity and stated that it was not working well.
> >> We are now using the bigger granularity that you describe below.
> >>
> > Ok, thats good, but that must be _very_ new.  Looking at your git tree, I see no
> > merge commits.  How are you pulling from those subtrees?
> 
> next-net tree is active for last three releases.
> 
What!?  What is the purpose of holding patches in a subtree for multiple
releases?  If a given changeset isn't ready for merge to Thomas tree the people
working on it should clone the subtree to some place they can all collaborate on
it.  Once it goes into a subtree there needs to be a defined workflow to get it
into the canonical tree that Thomas maintains on a regular, short time frame.
to do less is to confuse the process for everyone involved, and slow people
down, rather than accelerate their work.

> I guess following is the first commit to the sub-tree:
> http://dpdk.org/ml/archives/dev/2016-February/032580.html
> 
> sub-trees rebase on top of main tree regularly, that is why there is no
> merge commit.
> 
I'm not asking about merge commits in the sub-tree, I'm asking about merge
commits in thomas's tree.  There should be a merge commit every time he pulls
from a sub-tree (unless its a fast-forward I think, but with multiple subtrees
and commits going to thomas directly, that should never really happen).  I don't
see any Merge commits in the master branch of his tree, so I'm left wondering
what mechanic is being used to migrate patches from net-next or crypo-next to
his tree.  Thomas, can you comment here?

> > 
> > 
> >>> We should be sharding that at a much higher
> >>> granularity and using it much more consistently.  That is to say, that we
> >>> should have a maintainer for all the ethernet pmds, and another for the
> >>> crypto pmds, another for the core eal layer, another for misc libraries
> >>> that have low patch volumes, etc.
> >>
> >> Yes we could open a tree for EAL and another one for the core libraries.
> >>
> > That could be worthwhile.  Lets see how the net and crypto subtrees work out
> > (assuming again that these trees are newly founded)
> > 
> > 
> >>> Each of those subdivisions should have
> >>> their own list to communicate on, and each should have a tree that
> >>> integrates patches for their own subsystem, and they should on a regular
> >>> cycle send pull requests to Thomas.
> >>
> >> Yes I think it is now a good idea to split the mailing list traffic,
> >> at least for netdev and cryptodev.
> >>
> > Agreed, that serves two purposes, it lowers the volume for people with a
> > specific interest (i.e. its a rudimentary filter), and it avoids confusion
> > between you and the subtree maintainer (that is to say, you don't have to even
> > consider pulling patches that go to the crypo and net lists, you just have to
> > trust that they pull those patches in and send you appropriate pull requests).
> 
> I still find single mail list more useful.
Why?  If you have interest in all the subsystems of a project, then its a small
amount of overhead to subscribe to a set of mailing lists and dump them all to a
single mail folder.  If you only have interest in a subset, its much more
difficult to filter them out, given that we have a plethora of prefix tags for
patches to define subsystems that aren't always used consistently.  Given that
this thread is here because we've identified the patch volume as a problem, it
seems fragmenting the list is the better solution.

> Also with current process, after -rc2 release, patches directly merged
> into main tree instead of sub-trees...
> 
Thats fine, at that point, if everything works properly, Thomas should only be
getting low volume patch flow for stabilization/bug fixing.  If thats not the
case, then perhaps we need to consider doing extra merges from the subtrees
later in the cycle.

Neil

^ permalink raw reply

* Re: [PATCH 00/56] Solarflare libefx-based PMD
From: Thomas Monjalon @ 2016-11-23 13:34 UTC (permalink / raw)
  To: Mcnamara, John; +Cc: dev, Andrew Rybchenko, Yigit, Ferruh
In-Reply-To: <B27915DBBA3421428155699D51E4CFE2026649C6@IRSMSX103.ger.corp.intel.com>

2016-11-23 09:57, Mcnamara, John:
> From: Andrew Rybchenko
> > Yes, I have no ICC compilers. I'll try to fix these warnings, but I can't
> > be sure without checking it.
> > Also we cannot claim ICC supported without building and testing the
> > generated binary.
> 
> Hi,
> 
> You can get a copy of ICC with a 30 day evaluation license here: https://software.intel.com/en-us/try-buy-tools

I think we should stop having some ICC requirements.
That's OK to support it on best effort base but it cannot be
a responsibility of developers outside of Intel.

^ permalink raw reply

* Re: [PATCH] eal: postpone vdev initialization
From: Thomas Monjalon @ 2016-11-23 13:29 UTC (permalink / raw)
  To: Jerin Jacob, Ferruh Yigit; +Cc: dev, declan.doherty, david.marchand
In-Reply-To: <20161123000740.GA22068@svelivela-lt.caveonetworks.com>

2016-11-23 05:37, Jerin Jacob:
> On Mon, Nov 21, 2016 at 05:35:58PM +0000, Ferruh Yigit wrote:
> > On 11/21/2016 5:02 PM, Jerin Jacob wrote:
> > > On Mon, Nov 21, 2016 at 09:54:57AM +0000, Ferruh Yigit wrote:
> > >> This changes the port id assignments to the devices, right?
> > >>
> > >> Previously virtual devices get first available port ids (0..N1), later
> > >> physical devices (N1..N2). Now this becomes reverse.
> > >>
> > >> Can this change break some existing user applications?
> > > 
> > > I guess it may be effected only to ethdev bond pmd based application,
> > > which is broken anyway.
> > 
> > My concern is, this may effect the applications that use "--vdev" eal
> > parameter and has an assumption about port assignment.
> 
> Not sure. Application expectation on specific port assignment is bad anyway.
> But in any event, what we do with exiting ethdev bond pmd failure.
> 
> > 
> > And if this breaks any userspace application, does it require a
> > deprecation notice?
> 
> I am not sure. Thomas, Any input on this ?

Is the expectation thought by Ferruh, written somewhere?
If not, we can accept this change as is in this release.

^ permalink raw reply

* Re: [PATCH] maintainers: update testpmd maintainers
From: Thomas Monjalon @ 2016-11-23 13:09 UTC (permalink / raw)
  To: Wei Dai; +Cc: dev, pablo.de.lara.guarch, jingjing.wu, john.mcnamara,
	helin.zhang
In-Reply-To: <1479904483-2600-1-git-send-email-wei.dai@intel.com>

2016-11-23 20:34, Wei Dai:
> add Jingjing Wu and Wei Dai as new maintainers
> of test-pmd.

Thanks for proposing yourself.
I think it is a bit strange for you Wei Dai to become maintainer now,
as you are a newcomer and never sent a patch for testpmd yet.

^ permalink raw reply

* Re: [PATCH] maintainers: update testpmd maintainers
From: Zhang, Helin @ 2016-11-23 12:58 UTC (permalink / raw)
  To: Dai, Wei, dev@dpdk.org
  Cc: De Lara Guarch, Pablo, Wu, Jingjing, Mcnamara, John
In-Reply-To: <1479904483-2600-1-git-send-email-wei.dai@intel.com>



> -----Original Message-----
> From: Dai, Wei
> Sent: Wednesday, November 23, 2016 8:35 PM
> To: dev@dpdk.org
> Cc: De Lara Guarch, Pablo; Wu, Jingjing; Mcnamara, John; Zhang, Helin; Dai,
> Wei
> Subject: [PATCH] maintainers: update testpmd maintainers
> 
> add Jingjing Wu and Wei Dai as new maintainers of test-pmd.
> 
> Signed-off-by: Wei Dai <wei.dai@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>

> ---
>  MAINTAINERS | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index d6bb8f8..8070ed6 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -614,7 +614,8 @@ F: app/test/virtual_pmd.c
>  F: app/test/virtual_pmd.h
> 
>  Driver testing tool
> -M: Pablo de Lara <pablo.de.lara.guarch@intel.com>
> +M: Jingjing Wu <jingjing.wu@intel.com>
> +M: Wei Dai <wei.dai@intel.com>
>  F: app/test-pmd/
>  F: doc/guides/testpmd_app_ug/
> 
> --
> 2.7.4

^ permalink raw reply

* [PATCH] examples: fix ip_pipeline makefile typo
From: Ilya Matveychikov @ 2016-11-23 12:56 UTC (permalink / raw)
  To: dev

Signed-off-by: Ilya V. Matveychikov <matvejchikov@gmail.com>
---
 examples/ip_pipeline/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/ip_pipeline/Makefile b/examples/ip_pipeline/Makefile
index 5827117..6657237 100644
--- a/examples/ip_pipeline/Makefile
+++ b/examples/ip_pipeline/Makefile
@@ -36,7 +36,7 @@ endif
 # Default target, can be overridden by command line or environment
 RTE_TARGET ?= x86_64-native-linuxapp-gcc

-DIRS-(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline
+DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline

 include $(RTE_SDK)/mk/rte.vars.mk

--
2.5.5

^ permalink raw reply related

* [PATCH] maintainers: update testpmd maintainers
From: Wei Dai @ 2016-11-23 12:34 UTC (permalink / raw)
  To: dev; +Cc: pablo.de.lara.guarch, jingjing.wu, john.mcnamara, helin.zhang,
	Wei Dai

add Jingjing Wu and Wei Dai as new maintainers
of test-pmd.

Signed-off-by: Wei Dai <wei.dai@intel.com>
---
 MAINTAINERS | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index d6bb8f8..8070ed6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -614,7 +614,8 @@ F: app/test/virtual_pmd.c
 F: app/test/virtual_pmd.h
 
 Driver testing tool
-M: Pablo de Lara <pablo.de.lara.guarch@intel.com>
+M: Jingjing Wu <jingjing.wu@intel.com>
+M: Wei Dai <wei.dai@intel.com>
 F: app/test-pmd/
 F: doc/guides/testpmd_app_ug/
 
-- 
2.7.4

^ permalink raw reply related

* Re: [PATCH] app/testpmd: display port driver name
From: Mcnamara, John @ 2016-11-23 12:11 UTC (permalink / raw)
  To: David Marchand, De Lara Guarch, Pablo; +Cc: dev@dpdk.org
In-Reply-To: <1479898526-23489-1-git-send-email-david.marchand@6wind.com>

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David Marchand
> Sent: Wednesday, November 23, 2016 10:55 AM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH] app/testpmd: display port driver name
> 
> This makes it easier to check which driver name is reported through ethdev
> rte_eth_dev_info_get().
> 
> Example:
> 
> root@ubuntu1604:~/dpdk# ./build/app/testpmd -c 0x6 --vdev
> net_af_packet0,iface=mgmt0 -- -i  --total-num-mbufs 2049 [snip]
> testpmd> show port info all
> 
> ********************* Infos for port 0  ********************* MAC address:
> DE:AD:DE:01:02:03 Driver name: net_af_packet Connect to socket: 0 memory
> allocation on the socket: 0
> 
> Signed-off-by: David Marchand <david.marchand@6wind.com>

Good idea.

Acked-by: John McNamara <john.mcnamara@intel.com>

^ permalink raw reply

* [PATCH] app/testpmd: display port driver name
From: David Marchand @ 2016-11-23 10:55 UTC (permalink / raw)
  To: pablo.de.lara.guarch; +Cc: dev

This makes it easier to check which driver name is reported through ethdev
rte_eth_dev_info_get().

Example:

root@ubuntu1604:~/dpdk# ./build/app/testpmd -c 0x6
--vdev net_af_packet0,iface=mgmt0 -- -i  --total-num-mbufs 2049
[snip]
testpmd> show port info all

********************* Infos for port 0  *********************
MAC address: DE:AD:DE:01:02:03
Driver name: net_af_packet
Connect to socket: 0
memory allocation on the socket: 0

Signed-off-by: David Marchand <david.marchand@6wind.com>
---

This small patch is what I used to check 
http://dpdk.org/dev/patchwork/patch/17170/ and
http://dpdk.org/dev/patchwork/patch/17171/.

---
 app/test-pmd/config.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 36c47ab..8cf537d 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -454,10 +454,13 @@ port_infos_display(portid_t port_id)
 	}
 	port = &ports[port_id];
 	rte_eth_link_get_nowait(port_id, &link);
+	memset(&dev_info, 0, sizeof(dev_info));
+	rte_eth_dev_info_get(port_id, &dev_info);
 	printf("\n%s Infos for port %-2d %s\n",
 	       info_border, port_id, info_border);
 	rte_eth_macaddr_get(port_id, &mac_addr);
 	print_ethaddr("MAC address: ", &mac_addr);
+	printf("\nDriver name: %s", dev_info.driver_name);
 	printf("\nConnect to socket: %u", port->socket_id);
 
 	if (port_numa[port_id] != NUMA_NO_CONFIG) {
@@ -500,8 +503,6 @@ port_infos_display(portid_t port_id)
 			printf("  qinq(extend) off \n");
 	}
 
-	memset(&dev_info, 0, sizeof(dev_info));
-	rte_eth_dev_info_get(port_id, &dev_info);
 	if (dev_info.hash_key_size > 0)
 		printf("Hash key size in bytes: %u\n", dev_info.hash_key_size);
 	if (dev_info.reta_size > 0)
-- 
2.7.4

^ permalink raw reply related

* Re: [PATCH 2/4] eventdev: implement the northbound APIs
From: Bruce Richardson @ 2016-11-23  9:57 UTC (permalink / raw)
  To: Eads, Gage
  Cc: Jerin Jacob, dev@dpdk.org, Van Haaren, Harry,
	hemant.agrawal@nxp.com
In-Reply-To: <9184057F7FC11744A2107296B6B8EB1E01E31C40@FMSMSX108.amr.corp.intel.com>

Hi Gage,

just FYI, you can make it easier on your readers if you cut off the end
of the original email that you are not replying to. It saves us having
to scroll down to check for more comments. :-)

/Bruce

On Tue, Nov 22, 2016 at 03:15:52PM +0000, Eads, Gage wrote:
> 
> 
> >  -----Original Message-----
> >  From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> >  Sent: Monday, November 21, 2016 1:32 PM
> >  To: Eads, Gage <gage.eads@intel.com>
> >  Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Van
> >  Haaren, Harry <harry.van.haaren@intel.com>; hemant.agrawal@nxp.com
> >  Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs
> >  
> >  On Tue, Nov 22, 2016 at 12:43:58AM +0530, Jerin Jacob wrote:
> >  > On Mon, Nov 21, 2016 at 05:45:51PM +0000, Eads, Gage wrote:
> >  > > Hi Jerin,
> >  > >
> >  > > I did a quick review and overall this implementation looks good. I noticed
> >  just one issue in rte_event_queue_setup(): the check of
> >  nb_atomic_order_sequences is being applied to atomic-type queues, but that
> >  field applies to ordered-type queues.
> >  >
> >  > Thanks Gage. I will fix that in v2.
> >  >
> >  > >
> >  > > One open issue I noticed is the "typical workflow" description starting in
> >  rte_eventdev.h:204 conflicts with the centralized software PMD that Harry
> >  posted last week. Specifically, that PMD expects a single core to call the
> >  schedule function. We could extend the documentation to account for this
> >  alternative style of scheduler invocation, or discuss ways to make the software
> >  PMD work with the documented workflow. I prefer the former, but either way I
> >  think we ought to expose the scheduler's expected usage to the user -- perhaps
> >  through an RTE_EVENT_DEV_CAP flag?
> >  >
> >  > I prefer former too, you can propose the documentation change required for
> >  software PMD.
> 
> Sure, proposal follows. The "typical workflow" isn't the most optimal by having a conditional in the fast-path, of course, but it demonstrates the idea simply.
> 
> (line 204)
>  * An event driven based application has following typical workflow on fastpath:
>  * \code{.c}                                                                        
>  *      while (1) {                                                                 
>  *                                                                                  
>  *              if (dev_info.event_dev_cap &                                        
>  *                      RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED)                        
>  *                      rte_event_schedule(dev_id);                                 
>  *                                                                                  
>  *              rte_event_dequeue(...);                                             
>  *                                                                                  
>  *              (event processing)                                                  
>  *                                                                                  
>  *              rte_event_enqueue(...);                                             
>  *      }                                                                           
>  * \endcode                                                                         
>  *                                                                                  
>  * The *schedule* operation is intended to do event scheduling, and the             
>  * *dequeue* operation returns the scheduled events. An implementation              
>  * is free to define the semantics between *schedule* and *dequeue*. For            
>  * example, a system based on a hardware scheduler can define its                   
>  * rte_event_schedule() to be an NOOP, whereas a software scheduler can use         
>  * the *schedule* operation to schedule events. The                                 
>  * RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag indicates whether
>  * rte_event_schedule() should be called by all cores or by a single (typically 
>  * dedicated) core.
> 
> (line 308)
> #define RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED (1ULL < 2)                          
> /**< Event scheduling implementation is distributed and all cores must execute       
>  *  rte_event_schedule(). If unset, the implementation is centralized and     
>  *  a single core must execute the schedule operation.                        
>  *                                                                              
>  *  \see rte_event_schedule()                                                   
>  */
> 
> >  >
> >  > On same note, If software PMD based workflow need  a separate core(s) for
> >  > schedule function then, Can we hide that from API specification and pass an
> >  > argument to SW pmd to define the scheduling core(s)?
> >  >
> >  > Something like --vdev=eventsw0,schedule_cmask=0x2
> 
> An API for controlling the scheduler coremask instead of (or perhaps in addition to) the vdev argument would be good, to allow runtime control. I can imagine apps that scale the number of cores based on load, and in doing so may want to migrate the scheduler to a different core.
> 
> >  
> >  Just a thought,
> >  
> >  Perhaps, We could introduce generic "service" cores concept to DPDK to hide
> >  the
> >  requirement where the implementation needs dedicated core to do certain
> >  work. I guess it would useful for other NPU integration in DPDK.
> >  
> 
> That's an interesting idea. As you suggested in the other thread, this concept could be extended to the "producer" code in the example for configurations where the NIC requires software to feed into the eventdev. And to the other subsystems mentioned in your original PDF, crypto and timer.
> 
> >  >
> >  > >
> >  > > Thanks,
> >  > > Gage
> >  > >
> >  > > >  -----Original Message-----
> >  > > >  From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> >  > > >  Sent: Thursday, November 17, 2016 11:45 PM
> >  > > >  To: dev@dpdk.org
> >  > > >  Cc: Richardson, Bruce <bruce.richardson@intel.com>; Van Haaren, Harry
> >  > > >  <harry.van.haaren@intel.com>; hemant.agrawal@nxp.com; Eads, Gage
> >  > > >  <gage.eads@intel.com>; Jerin Jacob
> >  <jerin.jacob@caviumnetworks.com>
> >  > > >  Subject: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound
> >  APIs
> >  > > >
> >  > > >  This patch set defines the southbound driver interface
> >  > > >  and implements the common code required for northbound
> >  > > >  eventdev API interface.
> >  > > >
> >  > > >  Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> >  > > >  ---
> >  > > >   config/common_base                           |    6 +
> >  > > >   lib/Makefile                                 |    1 +
> >  > > >   lib/librte_eal/common/include/rte_log.h      |    1 +
> >  > > >   lib/librte_eventdev/Makefile                 |   57 ++
> >  > > >   lib/librte_eventdev/rte_eventdev.c           | 1211
> >  > > >  ++++++++++++++++++++++++++
> >  > > >   lib/librte_eventdev/rte_eventdev_pmd.h       |  504 +++++++++++
> >  > > >   lib/librte_eventdev/rte_eventdev_version.map |   39 +
> >  > > >   mk/rte.app.mk                                |    1 +
> >  > > >   8 files changed, 1820 insertions(+)
> >  > > >   create mode 100644 lib/librte_eventdev/Makefile
> >  > > >   create mode 100644 lib/librte_eventdev/rte_eventdev.c
> >  > > >   create mode 100644 lib/librte_eventdev/rte_eventdev_pmd.h
> >  > > >   create mode 100644 lib/librte_eventdev/rte_eventdev_version.map
> >  > > >
> >  > > >  diff --git a/config/common_base b/config/common_base
> >  > > >  index 4bff83a..7a8814e 100644
> >  > > >  --- a/config/common_base
> >  > > >  +++ b/config/common_base
> >  > > >  @@ -411,6 +411,12 @@ CONFIG_RTE_LIBRTE_PMD_ZUC_DEBUG=n
> >  > > >   CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y
> >  > > >
> >  > > >   #
> >  > > >  +# Compile generic event device library
> >  > > >  +#
> >  > > >  +CONFIG_RTE_LIBRTE_EVENTDEV=y
> >  > > >  +CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n
> >  > > >  +CONFIG_RTE_EVENT_MAX_DEVS=16
> >  > > >  +CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64
> >  > > >   # Compile librte_ring
> >  > > >   #
> >  > > >   CONFIG_RTE_LIBRTE_RING=y
> >  > > >  diff --git a/lib/Makefile b/lib/Makefile
> >  > > >  index 990f23a..1a067bf 100644
> >  > > >  --- a/lib/Makefile
> >  > > >  +++ b/lib/Makefile
> >  > > >  @@ -41,6 +41,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) +=
> >  librte_cfgfile
> >  > > >   DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline
> >  > > >   DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ether
> >  > > >   DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += librte_cryptodev
> >  > > >  +DIRS-$(CONFIG_RTE_LIBRTE_EVENTDEV) += librte_eventdev
> >  > > >   DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += librte_vhost
> >  > > >   DIRS-$(CONFIG_RTE_LIBRTE_HASH) += librte_hash
> >  > > >   DIRS-$(CONFIG_RTE_LIBRTE_LPM) += librte_lpm
> >  > > >  diff --git a/lib/librte_eal/common/include/rte_log.h
> >  > > >  b/lib/librte_eal/common/include/rte_log.h
> >  > > >  index 29f7d19..9a07d92 100644
> >  > > >  --- a/lib/librte_eal/common/include/rte_log.h
> >  > > >  +++ b/lib/librte_eal/common/include/rte_log.h
> >  > > >  @@ -79,6 +79,7 @@ extern struct rte_logs rte_logs;
> >  > > >   #define RTE_LOGTYPE_PIPELINE 0x00008000 /**< Log related to
> >  pipeline. */
> >  > > >   #define RTE_LOGTYPE_MBUF    0x00010000 /**< Log related to mbuf.
> >  */
> >  > > >   #define RTE_LOGTYPE_CRYPTODEV 0x00020000 /**< Log related to
> >  > > >  cryptodev. */
> >  > > >  +#define RTE_LOGTYPE_EVENTDEV 0x00040000 /**< Log related to
> >  eventdev.
> >  > > >  */
> >  > > >
> >  > > >   /* these log types can be used in an application */
> >  > > >   #define RTE_LOGTYPE_USER1   0x01000000 /**< User-defined log type
> >  1. */
> >  > > >  diff --git a/lib/librte_eventdev/Makefile b/lib/librte_eventdev/Makefile
> >  > > >  new file mode 100644
> >  > > >  index 0000000..dac0663
> >  > > >  --- /dev/null
> >  > > >  +++ b/lib/librte_eventdev/Makefile
> >  > > >  @@ -0,0 +1,57 @@
> >  > > >  +#   BSD LICENSE
> >  > > >  +#
> >  > > >  +#   Copyright(c) 2016 Cavium networks. All rights reserved.
> >  > > >  +#
> >  > > >  +#   Redistribution and use in source and binary forms, with or without
> >  > > >  +#   modification, are permitted provided that the following conditions
> >  > > >  +#   are met:
> >  > > >  +#
> >  > > >  +#     * Redistributions of source code must retain the above copyright
> >  > > >  +#       notice, this list of conditions and the following disclaimer.
> >  > > >  +#     * Redistributions in binary form must reproduce the above copyright
> >  > > >  +#       notice, this list of conditions and the following disclaimer in
> >  > > >  +#       the documentation and/or other materials provided with the
> >  > > >  +#       distribution.
> >  > > >  +#     * Neither the name of Cavium networks nor the names of its
> >  > > >  +#       contributors may be used to endorse or promote products derived
> >  > > >  +#       from this software without specific prior written permission.
> >  > > >  +#
> >  > > >  +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> >  > > >  CONTRIBUTORS
> >  > > >  +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,
> >  BUT
> >  > > >  NOT
> >  > > >  +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> >  > > >  FITNESS FOR
> >  > > >  +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> >  > > >  COPYRIGHT
> >  > > >  +#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> >  > > >  INCIDENTAL,
> >  > > >  +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
> >  BUT
> >  > > >  NOT
> >  > > >  +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
> >  LOSS
> >  > > >  OF USE,
> >  > > >  +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> >  CAUSED AND
> >  > > >  ON ANY
> >  > > >  +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> >  OR
> >  > > >  TORT
> >  > > >  +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> >  OUT OF
> >  > > >  THE USE
> >  > > >  +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> >  > > >  DAMAGE.
> >  > > >  +
> >  > > >  +include $(RTE_SDK)/mk/rte.vars.mk
> >  > > >  +
> >  > > >  +# library name
> >  > > >  +LIB = librte_eventdev.a
> >  > > >  +
> >  > > >  +# library version
> >  > > >  +LIBABIVER := 1
> >  > > >  +
> >  > > >  +# build flags
> >  > > >  +CFLAGS += -O3
> >  > > >  +CFLAGS += $(WERROR_FLAGS)
> >  > > >  +
> >  > > >  +# library source files
> >  > > >  +SRCS-y += rte_eventdev.c
> >  > > >  +
> >  > > >  +# export include files
> >  > > >  +SYMLINK-y-include += rte_eventdev.h
> >  > > >  +SYMLINK-y-include += rte_eventdev_pmd.h
> >  > > >  +
> >  > > >  +# versioning export map
> >  > > >  +EXPORT_MAP := rte_eventdev_version.map
> >  > > >  +
> >  > > >  +# library dependencies
> >  > > >  +DEPDIRS-y += lib/librte_eal
> >  > > >  +DEPDIRS-y += lib/librte_mbuf
> >  > > >  +
> >  > > >  +include $(RTE_SDK)/mk/rte.lib.mk
> >  > > >  diff --git a/lib/librte_eventdev/rte_eventdev.c
> >  > > >  b/lib/librte_eventdev/rte_eventdev.c
> >  > > >  new file mode 100644
> >  > > >  index 0000000..17ce5c3
> >  > > >  --- /dev/null
> >  > > >  +++ b/lib/librte_eventdev/rte_eventdev.c
> >  > > >  @@ -0,0 +1,1211 @@
> >  > > >  +/*
> >  > > >  + *   BSD LICENSE
> >  > > >  + *
> >  > > >  + *   Copyright(c) 2016 Cavium networks. All rights reserved.
> >  > > >  + *
> >  > > >  + *   Redistribution and use in source and binary forms, with or without
> >  > > >  + *   modification, are permitted provided that the following conditions
> >  > > >  + *   are met:
> >  > > >  + *
> >  > > >  + *     * Redistributions of source code must retain the above copyright
> >  > > >  + *       notice, this list of conditions and the following disclaimer.
> >  > > >  + *     * Redistributions in binary form must reproduce the above
> >  copyright
> >  > > >  + *       notice, this list of conditions and the following disclaimer in
> >  > > >  + *       the documentation and/or other materials provided with the
> >  > > >  + *       distribution.
> >  > > >  + *     * Neither the name of Cavium networks nor the names of its
> >  > > >  + *       contributors may be used to endorse or promote products derived
> >  > > >  + *       from this software without specific prior written permission.
> >  > > >  + *
> >  > > >  + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> >  > > >  CONTRIBUTORS
> >  > > >  + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,
> >  BUT
> >  > > >  NOT
> >  > > >  + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> >  > > >  FITNESS FOR
> >  > > >  + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> >  > > >  COPYRIGHT
> >  > > >  + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> >  > > >  INCIDENTAL,
> >  > > >  + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
> >  BUT
> >  > > >  NOT
> >  > > >  + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
> >  LOSS
> >  > > >  OF USE,
> >  > > >  + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> >  CAUSED
> >  > > >  AND ON ANY
> >  > > >  + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> >  OR
> >  > > >  TORT
> >  > > >  + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> >  OUT OF
> >  > > >  THE USE
> >  > > >  + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> >  > > >  DAMAGE.
> >  > > >  + */
> >  > > >  +
> >  > > >  +#include <ctype.h>
> >  > > >  +#include <stdio.h>
> >  > > >  +#include <stdlib.h>
> >  > > >  +#include <string.h>
> >  > > >  +#include <stdarg.h>
> >  > > >  +#include <errno.h>
> >  > > >  +#include <stdint.h>
> >  > > >  +#include <inttypes.h>
> >  > > >  +#include <sys/types.h>
> >  > > >  +#include <sys/queue.h>
> >  > > >  +
> >  > > >  +#include <rte_byteorder.h>
> >  > > >  +#include <rte_log.h>
> >  > > >  +#include <rte_debug.h>
> >  > > >  +#include <rte_dev.h>
> >  > > >  +#include <rte_pci.h>
> >  > > >  +#include <rte_memory.h>
> >  > > >  +#include <rte_memcpy.h>
> >  > > >  +#include <rte_memzone.h>
> >  > > >  +#include <rte_eal.h>
> >  > > >  +#include <rte_per_lcore.h>
> >  > > >  +#include <rte_lcore.h>
> >  > > >  +#include <rte_atomic.h>
> >  > > >  +#include <rte_branch_prediction.h>
> >  > > >  +#include <rte_common.h>
> >  > > >  +#include <rte_malloc.h>
> >  > > >  +#include <rte_errno.h>
> >  > > >  +
> >  > > >  +#include "rte_eventdev.h"
> >  > > >  +#include "rte_eventdev_pmd.h"
> >  > > >  +
> >  > > >  +struct rte_eventdev rte_event_devices[RTE_EVENT_MAX_DEVS];
> >  > > >  +
> >  > > >  +struct rte_eventdev *rte_eventdevs = &rte_event_devices[0];
> >  > > >  +
> >  > > >  +static struct rte_eventdev_global eventdev_globals = {
> >  > > >  +	.nb_devs		= 0
> >  > > >  +};
> >  > > >  +
> >  > > >  +struct rte_eventdev_global *rte_eventdev_globals =
> >  &eventdev_globals;
> >  > > >  +
> >  > > >  +/* Event dev north bound API implementation */
> >  > > >  +
> >  > > >  +uint8_t
> >  > > >  +rte_event_dev_count(void)
> >  > > >  +{
> >  > > >  +	return rte_eventdev_globals->nb_devs;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_get_dev_id(const char *name)
> >  > > >  +{
> >  > > >  +	int i;
> >  > > >  +
> >  > > >  +	if (!name)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	for (i = 0; i < rte_eventdev_globals->nb_devs; i++)
> >  > > >  +		if ((strcmp(rte_event_devices[i].data->name, name)
> >  > > >  +				== 0) &&
> >  > > >  +				(rte_event_devices[i].attached ==
> >  > > >  +						RTE_EVENTDEV_ATTACHED))
> >  > > >  +			return i;
> >  > > >  +	return -ENODEV;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_socket_id(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	return dev->data->socket_id;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_info_get(uint8_t dev_id, struct rte_event_dev_info
> >  *dev_info)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	if (dev_info == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	memset(dev_info, 0, sizeof(struct rte_event_dev_info));
> >  > > >  +
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -
> >  > > >  ENOTSUP);
> >  > > >  +	(*dev->dev_ops->dev_infos_get)(dev, dev_info);
> >  > > >  +
> >  > > >  +	dev_info->pci_dev = dev->pci_dev;
> >  > > >  +	if (dev->driver)
> >  > > >  +		dev_info->driver_name = dev->driver->pci_drv.driver.name;
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +static inline int
> >  > > >  +rte_event_dev_queue_config(struct rte_eventdev *dev, uint8_t
> >  nb_queues)
> >  > > >  +{
> >  > > >  +	uint8_t old_nb_queues = dev->data->nb_queues;
> >  > > >  +	void **queues;
> >  > > >  +	uint8_t *queues_prio;
> >  > > >  +	unsigned int i;
> >  > > >  +
> >  > > >  +	EDEV_LOG_DEBUG("Setup %d queues on device %u", nb_queues,
> >  > > >  +			 dev->data->dev_id);
> >  > > >  +
> >  > > >  +	/* First time configuration */
> >  > > >  +	if (dev->data->queues == NULL && nb_queues != 0) {
> >  > > >  +		dev->data->queues = rte_zmalloc_socket("eventdev->data-
> >  > > >  >queues",
> >  > > >  +				sizeof(dev->data->queues[0]) * nb_queues,
> >  > > >  +				RTE_CACHE_LINE_SIZE, dev->data-
> >  > > >  >socket_id);
> >  > > >  +		if (dev->data->queues == NULL) {
> >  > > >  +			dev->data->nb_queues = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to get memory for queue meta
> >  > > >  data,"
> >  > > >  +					"nb_queues %u", nb_queues);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +		/* Allocate memory to store queue priority */
> >  > > >  +		dev->data->queues_prio = rte_zmalloc_socket(
> >  > > >  +				"eventdev->data->queues_prio",
> >  > > >  +				sizeof(dev->data->queues_prio[0]) *
> >  > > >  nb_queues,
> >  > > >  +				RTE_CACHE_LINE_SIZE, dev->data-
> >  > > >  >socket_id);
> >  > > >  +		if (dev->data->queues_prio == NULL) {
> >  > > >  +			dev->data->nb_queues = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to get memory for queue
> >  > > >  priority,"
> >  > > >  +					"nb_queues %u", nb_queues);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +	} else if (dev->data->queues != NULL && nb_queues != 0) {/* re-config
> >  > > >  */
> >  > > >  +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops-
> >  > > >  >queue_release, -ENOTSUP);
> >  > > >  +
> >  > > >  +		queues = dev->data->queues;
> >  > > >  +		for (i = nb_queues; i < old_nb_queues; i++)
> >  > > >  +			(*dev->dev_ops->queue_release)(queues[i]);
> >  > > >  +
> >  > > >  +		queues = rte_realloc(queues, sizeof(queues[0]) * nb_queues,
> >  > > >  +				RTE_CACHE_LINE_SIZE);
> >  > > >  +		if (queues == NULL) {
> >  > > >  +			EDEV_LOG_ERR("failed to realloc queue meta data,"
> >  > > >  +						" nb_queues %u",
> >  > > >  nb_queues);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +		dev->data->queues = queues;
> >  > > >  +
> >  > > >  +		/* Re allocate memory to store queue priority */
> >  > > >  +		queues_prio = dev->data->queues_prio;
> >  > > >  +		queues_prio = rte_realloc(queues_prio,
> >  > > >  +				sizeof(queues_prio[0]) * nb_queues,
> >  > > >  +				RTE_CACHE_LINE_SIZE);
> >  > > >  +		if (queues_prio == NULL) {
> >  > > >  +			EDEV_LOG_ERR("failed to realloc queue priority,"
> >  > > >  +						" nb_queues %u",
> >  > > >  nb_queues);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +		dev->data->queues_prio = queues_prio;
> >  > > >  +
> >  > > >  +		if (nb_queues > old_nb_queues) {
> >  > > >  +			uint8_t new_qs = nb_queues - old_nb_queues;
> >  > > >  +
> >  > > >  +			memset(queues + old_nb_queues, 0,
> >  > > >  +				sizeof(queues[0]) * new_qs);
> >  > > >  +			memset(queues_prio + old_nb_queues, 0,
> >  > > >  +				sizeof(queues_prio[0]) * new_qs);
> >  > > >  +		}
> >  > > >  +	} else if (dev->data->queues != NULL && nb_queues == 0) {
> >  > > >  +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops-
> >  > > >  >queue_release, -ENOTSUP);
> >  > > >  +
> >  > > >  +		queues = dev->data->queues;
> >  > > >  +		for (i = nb_queues; i < old_nb_queues; i++)
> >  > > >  +			(*dev->dev_ops->queue_release)(queues[i]);
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev->data->nb_queues = nb_queues;
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +static inline int
> >  > > >  +rte_event_dev_port_config(struct rte_eventdev *dev, uint8_t nb_ports)
> >  > > >  +{
> >  > > >  +	uint8_t old_nb_ports = dev->data->nb_ports;
> >  > > >  +	void **ports;
> >  > > >  +	uint16_t *links_map;
> >  > > >  +	uint8_t *ports_dequeue_depth;
> >  > > >  +	uint8_t *ports_enqueue_depth;
> >  > > >  +	unsigned int i;
> >  > > >  +
> >  > > >  +	EDEV_LOG_DEBUG("Setup %d ports on device %u", nb_ports,
> >  > > >  +			 dev->data->dev_id);
> >  > > >  +
> >  > > >  +	/* First time configuration */
> >  > > >  +	if (dev->data->ports == NULL && nb_ports != 0) {
> >  > > >  +		dev->data->ports = rte_zmalloc_socket("eventdev->data-
> >  > > >  >ports",
> >  > > >  +				sizeof(dev->data->ports[0]) * nb_ports,
> >  > > >  +				RTE_CACHE_LINE_SIZE, dev->data-
> >  > > >  >socket_id);
> >  > > >  +		if (dev->data->ports == NULL) {
> >  > > >  +			dev->data->nb_ports = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to get memory for port meta
> >  > > >  data,"
> >  > > >  +					"nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		/* Allocate memory to store ports dequeue depth */
> >  > > >  +		dev->data->ports_dequeue_depth =
> >  > > >  +			rte_zmalloc_socket("eventdev-
> >  > > >  >ports_dequeue_depth",
> >  > > >  +			sizeof(dev->data->ports_dequeue_depth[0]) *
> >  > > >  nb_ports,
> >  > > >  +			RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> >  > > >  +		if (dev->data->ports_dequeue_depth == NULL) {
> >  > > >  +			dev->data->nb_ports = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to get memory for port deq
> >  > > >  meta,"
> >  > > >  +					"nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		/* Allocate memory to store ports enqueue depth */
> >  > > >  +		dev->data->ports_enqueue_depth =
> >  > > >  +			rte_zmalloc_socket("eventdev-
> >  > > >  >ports_enqueue_depth",
> >  > > >  +			sizeof(dev->data->ports_enqueue_depth[0]) *
> >  > > >  nb_ports,
> >  > > >  +			RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> >  > > >  +		if (dev->data->ports_enqueue_depth == NULL) {
> >  > > >  +			dev->data->nb_ports = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to get memory for port enq
> >  > > >  meta,"
> >  > > >  +					"nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		/* Allocate memory to store queue to port link connection */
> >  > > >  +		dev->data->links_map =
> >  > > >  +			rte_zmalloc_socket("eventdev->links_map",
> >  > > >  +			sizeof(dev->data->links_map[0]) * nb_ports *
> >  > > >  +			RTE_EVENT_MAX_QUEUES_PER_DEV,
> >  > > >  +			RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> >  > > >  +		if (dev->data->links_map == NULL) {
> >  > > >  +			dev->data->nb_ports = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to get memory for port_map
> >  > > >  area,"
> >  > > >  +					"nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +	} else if (dev->data->ports != NULL && nb_ports != 0) {/* re-config */
> >  > > >  +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_release,
> >  > > >  -ENOTSUP);
> >  > > >  +
> >  > > >  +		ports = dev->data->ports;
> >  > > >  +		ports_dequeue_depth = dev->data->ports_dequeue_depth;
> >  > > >  +		ports_enqueue_depth = dev->data->ports_enqueue_depth;
> >  > > >  +		links_map = dev->data->links_map;
> >  > > >  +
> >  > > >  +		for (i = nb_ports; i < old_nb_ports; i++)
> >  > > >  +			(*dev->dev_ops->port_release)(ports[i]);
> >  > > >  +
> >  > > >  +		/* Realloc memory for ports */
> >  > > >  +		ports = rte_realloc(ports, sizeof(ports[0]) * nb_ports,
> >  > > >  +				RTE_CACHE_LINE_SIZE);
> >  > > >  +		if (ports == NULL) {
> >  > > >  +			EDEV_LOG_ERR("failed to realloc port meta data,"
> >  > > >  +						" nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		/* Realloc memory for ports_dequeue_depth */
> >  > > >  +		ports_dequeue_depth = rte_realloc(ports_dequeue_depth,
> >  > > >  +			sizeof(ports_dequeue_depth[0]) * nb_ports,
> >  > > >  +			RTE_CACHE_LINE_SIZE);
> >  > > >  +		if (ports_dequeue_depth == NULL) {
> >  > > >  +			EDEV_LOG_ERR("failed to realloc port deqeue meta
> >  > > >  data,"
> >  > > >  +						" nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		/* Realloc memory for ports_enqueue_depth */
> >  > > >  +		ports_enqueue_depth = rte_realloc(ports_enqueue_depth,
> >  > > >  +			sizeof(ports_enqueue_depth[0]) * nb_ports,
> >  > > >  +			RTE_CACHE_LINE_SIZE);
> >  > > >  +		if (ports_enqueue_depth == NULL) {
> >  > > >  +			EDEV_LOG_ERR("failed to realloc port enqueue meta
> >  > > >  data,"
> >  > > >  +						" nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		/* Realloc memory to store queue to port link connection */
> >  > > >  +		links_map = rte_realloc(links_map,
> >  > > >  +			sizeof(dev->data->links_map[0]) * nb_ports *
> >  > > >  +			RTE_EVENT_MAX_QUEUES_PER_DEV,
> >  > > >  +			RTE_CACHE_LINE_SIZE);
> >  > > >  +		if (dev->data->links_map == NULL) {
> >  > > >  +			dev->data->nb_ports = 0;
> >  > > >  +			EDEV_LOG_ERR("failed to realloc mem for port_map
> >  > > >  area,"
> >  > > >  +					"nb_ports %u", nb_ports);
> >  > > >  +			return -(ENOMEM);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		if (nb_ports > old_nb_ports) {
> >  > > >  +			uint8_t new_ps = nb_ports - old_nb_ports;
> >  > > >  +
> >  > > >  +			memset(ports + old_nb_ports, 0,
> >  > > >  +				sizeof(ports[0]) * new_ps);
> >  > > >  +			memset(ports_dequeue_depth + old_nb_ports, 0,
> >  > > >  +				sizeof(ports_dequeue_depth[0]) * new_ps);
> >  > > >  +			memset(ports_enqueue_depth + old_nb_ports, 0,
> >  > > >  +				sizeof(ports_enqueue_depth[0]) * new_ps);
> >  > > >  +			memset(links_map +
> >  > > >  +				(old_nb_ports *
> >  > > >  RTE_EVENT_MAX_QUEUES_PER_DEV),
> >  > > >  +				0, sizeof(ports_enqueue_depth[0]) * new_ps);
> >  > > >  +		}
> >  > > >  +
> >  > > >  +		dev->data->ports = ports;
> >  > > >  +		dev->data->ports_dequeue_depth = ports_dequeue_depth;
> >  > > >  +		dev->data->ports_enqueue_depth = ports_enqueue_depth;
> >  > > >  +		dev->data->links_map = links_map;
> >  > > >  +	} else if (dev->data->ports != NULL && nb_ports == 0) {
> >  > > >  +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_release,
> >  > > >  -ENOTSUP);
> >  > > >  +
> >  > > >  +		ports = dev->data->ports;
> >  > > >  +		for (i = nb_ports; i < old_nb_ports; i++)
> >  > > >  +			(*dev->dev_ops->port_release)(ports[i]);
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev->data->nb_ports = nb_ports;
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_configure(uint8_t dev_id, struct rte_event_dev_config
> >  > > >  *dev_conf)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	struct rte_event_dev_info info;
> >  > > >  +	int diag;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -
> >  > > >  ENOTSUP);
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_configure, -
> >  > > >  ENOTSUP);
> >  > > >  +
> >  > > >  +	if (dev->data->dev_started) {
> >  > > >  +		EDEV_LOG_ERR(
> >  > > >  +		    "device %d must be stopped to allow configuration",
> >  > > >  dev_id);
> >  > > >  +		return -EBUSY;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	if (dev_conf == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	(*dev->dev_ops->dev_infos_get)(dev, &info);
> >  > > >  +
> >  > > >  +	/* Check dequeue_wait_ns value is in limit */
> >  > > >  +	if (!dev_conf->event_dev_cfg &
> >  > > >  RTE_EVENT_DEV_CFG_PER_DEQUEUE_WAIT) {
> >  > > >  +		if (dev_conf->dequeue_wait_ns < info.min_dequeue_wait_ns
> >  > > >  ||
> >  > > >  +			dev_conf->dequeue_wait_ns >
> >  > > >  info.max_dequeue_wait_ns) {
> >  > > >  +			EDEV_LOG_ERR("dev%d invalid dequeue_wait_ns=%d"
> >  > > >  +			" min_dequeue_wait_ns=%d
> >  > > >  max_dequeue_wait_ns=%d",
> >  > > >  +			dev_id, dev_conf->dequeue_wait_ns,
> >  > > >  +			info.min_dequeue_wait_ns,
> >  > > >  +			info.max_dequeue_wait_ns);
> >  > > >  +			return -EINVAL;
> >  > > >  +		}
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_events_limit is in limit */
> >  > > >  +	if (dev_conf->nb_events_limit > info.max_num_events) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_events_limit=%d >
> >  > > >  max_num_events=%d",
> >  > > >  +		dev_id, dev_conf->nb_events_limit, info.max_num_events);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_event_queues is in limit */
> >  > > >  +	if (!dev_conf->nb_event_queues) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_event_queues cannot be zero",
> >  > > >  dev_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +	if (dev_conf->nb_event_queues > info.max_event_queues) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_event_queues=%d >
> >  > > >  max_event_queues=%d",
> >  > > >  +		dev_id, dev_conf->nb_event_queues,
> >  > > >  info.max_event_queues);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_event_ports is in limit */
> >  > > >  +	if (!dev_conf->nb_event_ports) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_event_ports cannot be zero",
> >  > > >  dev_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +	if (dev_conf->nb_event_ports > info.max_event_ports) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_event_ports=%d >
> >  > > >  max_event_ports= %d",
> >  > > >  +		dev_id, dev_conf->nb_event_ports, info.max_event_ports);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_event_queue_flows is in limit */
> >  > > >  +	if (!dev_conf->nb_event_queue_flows) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_flows cannot be zero", dev_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +	if (dev_conf->nb_event_queue_flows > info.max_event_queue_flows)
> >  > > >  {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_flows=%x > max_flows=%x",
> >  > > >  +		dev_id, dev_conf->nb_event_queue_flows,
> >  > > >  +		info.max_event_queue_flows);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_event_port_dequeue_depth is in limit */
> >  > > >  +	if (!dev_conf->nb_event_port_dequeue_depth) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_dequeue_depth cannot be zero",
> >  > > >  dev_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +	if (dev_conf->nb_event_port_dequeue_depth >
> >  > > >  +			 info.max_event_port_dequeue_depth) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_dequeue_depth=%d >
> >  > > >  max_dequeue_depth=%d",
> >  > > >  +		dev_id, dev_conf->nb_event_port_dequeue_depth,
> >  > > >  +		info.max_event_port_dequeue_depth);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_event_port_enqueue_depth is in limit */
> >  > > >  +	if (!dev_conf->nb_event_port_enqueue_depth) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_enqueue_depth cannot be zero",
> >  > > >  dev_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +	if (dev_conf->nb_event_port_enqueue_depth >
> >  > > >  +			 info.max_event_port_enqueue_depth) {
> >  > > >  +		EDEV_LOG_ERR("dev%d nb_enqueue_depth=%d >
> >  > > >  max_enqueue_depth=%d",
> >  > > >  +		dev_id, dev_conf->nb_event_port_enqueue_depth,
> >  > > >  +		info.max_event_port_enqueue_depth);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Copy the dev_conf parameter into the dev structure */
> >  > > >  +	memcpy(&dev->data->dev_conf, dev_conf, sizeof(dev->data-
> >  > > >  >dev_conf));
> >  > > >  +
> >  > > >  +	/* Setup new number of queues and reconfigure device. */
> >  > > >  +	diag = rte_event_dev_queue_config(dev, dev_conf-
> >  > > >  >nb_event_queues);
> >  > > >  +	if (diag != 0) {
> >  > > >  +		EDEV_LOG_ERR("dev%d rte_event_dev_queue_config = %d",
> >  > > >  +				dev_id, diag);
> >  > > >  +		return diag;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Setup new number of ports and reconfigure device. */
> >  > > >  +	diag = rte_event_dev_port_config(dev, dev_conf->nb_event_ports);
> >  > > >  +	if (diag != 0) {
> >  > > >  +		rte_event_dev_queue_config(dev, 0);
> >  > > >  +		EDEV_LOG_ERR("dev%d rte_event_dev_port_config = %d",
> >  > > >  +				dev_id, diag);
> >  > > >  +		return diag;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Configure the device */
> >  > > >  +	diag = (*dev->dev_ops->dev_configure)(dev);
> >  > > >  +	if (diag != 0) {
> >  > > >  +		EDEV_LOG_ERR("dev%d dev_configure = %d", dev_id, diag);
> >  > > >  +		rte_event_dev_queue_config(dev, 0);
> >  > > >  +		rte_event_dev_port_config(dev, 0);
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev->data->event_dev_cap = info.event_dev_cap;
> >  > > >  +	return diag;
> >  > > >  +}
> >  > > >  +
> >  > > >  +static inline int
> >  > > >  +is_valid_queue(struct rte_eventdev *dev, uint8_t queue_id)
> >  > > >  +{
> >  > > >  +	if (queue_id < dev->data->nb_queues && queue_id <
> >  > > >  +				RTE_EVENT_MAX_QUEUES_PER_DEV)
> >  > > >  +		return 1;
> >  > > >  +	else
> >  > > >  +		return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_queue_default_conf_get(uint8_t dev_id, uint8_t queue_id,
> >  > > >  +				 struct rte_event_queue_conf *queue_conf)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	if (queue_conf == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	if (!is_valid_queue(dev, queue_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid queue_id=%" PRIu8, queue_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_def_conf, -
> >  > > >  ENOTSUP);
> >  > > >  +	memset(queue_conf, 0, sizeof(struct rte_event_queue_conf));
> >  > > >  +	(*dev->dev_ops->queue_def_conf)(dev, queue_id, queue_conf);
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +static inline int
> >  > > >  +is_valid_atomic_queue_conf(struct rte_event_queue_conf
> >  *queue_conf)
> >  > > >  +{
> >  > > >  +	if (queue_conf && (
> >  > > >  +		((queue_conf->event_queue_cfg &
> >  > > >  RTE_EVENT_QUEUE_CFG_TYPE_MASK)
> >  > > >  +			== RTE_EVENT_QUEUE_CFG_ALL_TYPES) ||
> >  > > >  +		((queue_conf->event_queue_cfg &
> >  > > >  RTE_EVENT_QUEUE_CFG_TYPE_MASK)
> >  > > >  +			== RTE_EVENT_QUEUE_CFG_ATOMIC_ONLY)
> >  > > >  +		))
> >  > > >  +		return 1;
> >  > > >  +	else
> >  > > >  +		return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_queue_setup(uint8_t dev_id, uint8_t queue_id,
> >  > > >  +		      struct rte_event_queue_conf *queue_conf)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	struct rte_event_queue_conf def_conf;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	if (!is_valid_queue(dev, queue_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid queue_id=%" PRIu8, queue_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_atomic_flows limit */
> >  > > >  +	if (is_valid_atomic_queue_conf(queue_conf)) {
> >  > > >  +		if (queue_conf->nb_atomic_flows == 0 ||
> >  > > >  +		    queue_conf->nb_atomic_flows >
> >  > > >  +			dev->data->dev_conf.nb_event_queue_flows) {
> >  > > >  +			EDEV_LOG_ERR(
> >  > > >  +		"dev%d queue%d Invalid nb_atomic_flows=%d
> >  > > >  max_flows=%d",
> >  > > >  +			dev_id, queue_id, queue_conf->nb_atomic_flows,
> >  > > >  +			dev->data->dev_conf.nb_event_queue_flows);
> >  > > >  +			return -EINVAL;
> >  > > >  +		}
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check nb_atomic_order_sequences limit */
> >  > > >  +	if (is_valid_atomic_queue_conf(queue_conf)) {
> >  > > >  +		if (queue_conf->nb_atomic_order_sequences == 0 ||
> >  > > >  +		    queue_conf->nb_atomic_order_sequences >
> >  > > >  +			dev->data->dev_conf.nb_event_queue_flows) {
> >  > > >  +			EDEV_LOG_ERR(
> >  > > >  +		"dev%d queue%d Invalid nb_atomic_order_seq=%d
> >  > > >  max_flows=%d",
> >  > > >  +			dev_id, queue_id, queue_conf-
> >  > > >  >nb_atomic_order_sequences,
> >  > > >  +			dev->data->dev_conf.nb_event_queue_flows);
> >  > > >  +			return -EINVAL;
> >  > > >  +		}
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	if (dev->data->dev_started) {
> >  > > >  +		EDEV_LOG_ERR(
> >  > > >  +		    "device %d must be stopped to allow queue setup", dev_id);
> >  > > >  +		return -EBUSY;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_setup, -
> >  > > >  ENOTSUP);
> >  > > >  +
> >  > > >  +	if (queue_conf == NULL) {
> >  > > >  +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops-
> >  > > >  >queue_def_conf,
> >  > > >  +					-ENOTSUP);
> >  > > >  +		(*dev->dev_ops->queue_def_conf)(dev, queue_id, &def_conf);
> >  > > >  +		def_conf.event_queue_cfg =
> >  > > >  RTE_EVENT_QUEUE_CFG_DEFAULT;
> >  > > >  +		queue_conf = &def_conf;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev->data->queues_prio[queue_id] = queue_conf->priority;
> >  > > >  +	return (*dev->dev_ops->queue_setup)(dev, queue_id, queue_conf);
> >  > > >  +}
> >  > > >  +
> >  > > >  +uint8_t
> >  > > >  +rte_event_queue_count(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	return dev->data->nb_queues;
> >  > > >  +}
> >  > > >  +
> >  > > >  +uint8_t
> >  > > >  +rte_event_queue_priority(uint8_t dev_id, uint8_t queue_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	if (dev->data->event_dev_cap & RTE_EVENT_DEV_CAP_QUEUE_QOS)
> >  > > >  +		return dev->data->queues_prio[queue_id];
> >  > > >  +	else
> >  > > >  +		return RTE_EVENT_QUEUE_PRIORITY_NORMAL;
> >  > > >  +}
> >  > > >  +
> >  > > >  +static inline int
> >  > > >  +is_valid_port(struct rte_eventdev *dev, uint8_t port_id)
> >  > > >  +{
> >  > > >  +	if (port_id < dev->data->nb_ports)
> >  > > >  +		return 1;
> >  > > >  +	else
> >  > > >  +		return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id,
> >  > > >  +				 struct rte_event_port_conf *port_conf)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	if (port_conf == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	if (!is_valid_port(dev, port_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_def_conf, -
> >  > > >  ENOTSUP);
> >  > > >  +	memset(port_conf, 0, sizeof(struct rte_event_port_conf));
> >  > > >  +	(*dev->dev_ops->port_def_conf)(dev, port_id, port_conf);
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_port_setup(uint8_t dev_id, uint8_t port_id,
> >  > > >  +		      struct rte_event_port_conf *port_conf)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	struct rte_event_port_conf def_conf;
> >  > > >  +	int diag;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	if (!is_valid_port(dev, port_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check new_event_threshold limit */
> >  > > >  +	if ((port_conf && !port_conf->new_event_threshold) ||
> >  > > >  +			(port_conf && port_conf->new_event_threshold >
> >  > > >  +				 dev->data->dev_conf.nb_events_limit)) {
> >  > > >  +		EDEV_LOG_ERR(
> >  > > >  +		   "dev%d port%d Invalid event_threshold=%d
> >  > > >  nb_events_limit=%d",
> >  > > >  +			dev_id, port_id, port_conf->new_event_threshold,
> >  > > >  +			dev->data->dev_conf.nb_events_limit);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check dequeue_depth limit */
> >  > > >  +	if ((port_conf && !port_conf->dequeue_depth) ||
> >  > > >  +			(port_conf && port_conf->dequeue_depth >
> >  > > >  +		dev->data->dev_conf.nb_event_port_dequeue_depth)) {
> >  > > >  +		EDEV_LOG_ERR(
> >  > > >  +		   "dev%d port%d Invalid dequeue depth=%d
> >  > > >  max_dequeue_depth=%d",
> >  > > >  +			dev_id, port_id, port_conf->dequeue_depth,
> >  > > >  +			dev->data-
> >  > > >  >dev_conf.nb_event_port_dequeue_depth);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Check enqueue_depth limit */
> >  > > >  +	if ((port_conf && !port_conf->enqueue_depth) ||
> >  > > >  +			(port_conf && port_conf->enqueue_depth >
> >  > > >  +		dev->data->dev_conf.nb_event_port_enqueue_depth)) {
> >  > > >  +		EDEV_LOG_ERR(
> >  > > >  +		   "dev%d port%d Invalid enqueue depth=%d
> >  > > >  max_enqueue_depth=%d",
> >  > > >  +			dev_id, port_id, port_conf->enqueue_depth,
> >  > > >  +			dev->data-
> >  > > >  >dev_conf.nb_event_port_enqueue_depth);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	if (dev->data->dev_started) {
> >  > > >  +		EDEV_LOG_ERR(
> >  > > >  +		    "device %d must be stopped to allow port setup", dev_id);
> >  > > >  +		return -EBUSY;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_setup, -
> >  > > >  ENOTSUP);
> >  > > >  +
> >  > > >  +	if (port_conf == NULL) {
> >  > > >  +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops-
> >  > > >  >port_def_conf,
> >  > > >  +					-ENOTSUP);
> >  > > >  +		(*dev->dev_ops->port_def_conf)(dev, port_id, &def_conf);
> >  > > >  +		port_conf = &def_conf;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev->data->ports_dequeue_depth[port_id] =
> >  > > >  +			port_conf->dequeue_depth;
> >  > > >  +	dev->data->ports_enqueue_depth[port_id] =
> >  > > >  +			port_conf->enqueue_depth;
> >  > > >  +
> >  > > >  +	diag = (*dev->dev_ops->port_setup)(dev, port_id, port_conf);
> >  > > >  +
> >  > > >  +	/* Unlink all the queues from this port(default state after setup) */
> >  > > >  +	if (!diag)
> >  > > >  +		diag = rte_event_port_unlink(dev_id, port_id, NULL, 0);
> >  > > >  +
> >  > > >  +	if (diag < 0)
> >  > > >  +		return diag;
> >  > > >  +
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +uint8_t
> >  > > >  +rte_event_port_dequeue_depth(uint8_t dev_id, uint8_t port_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	return dev->data->ports_dequeue_depth[port_id];
> >  > > >  +}
> >  > > >  +
> >  > > >  +uint8_t
> >  > > >  +rte_event_port_enqueue_depth(uint8_t dev_id, uint8_t port_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	return dev->data->ports_enqueue_depth[port_id];
> >  > > >  +}
> >  > > >  +
> >  > > >  +uint8_t
> >  > > >  +rte_event_port_count(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	return dev->data->nb_ports;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_port_link(uint8_t dev_id, uint8_t port_id,
> >  > > >  +		    struct rte_event_queue_link link[], uint16_t nb_links)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	struct rte_event_queue_link
> >  > > >  all_queues[RTE_EVENT_MAX_QUEUES_PER_DEV];
> >  > > >  +	uint16_t *links_map;
> >  > > >  +	int i, diag;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_link, -ENOTSUP);
> >  > > >  +
> >  > > >  +	if (!is_valid_port(dev, port_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	if (link == NULL) {
> >  > > >  +		for (i = 0; i < dev->data->nb_queues; i++) {
> >  > > >  +			all_queues[i].queue_id = i;
> >  > > >  +			all_queues[i].priority =
> >  > > >  +
> >  > > >  	RTE_EVENT_QUEUE_SERVICE_PRIORITY_NORMAL;
> >  > > >  +		}
> >  > > >  +		link = all_queues;
> >  > > >  +		nb_links = dev->data->nb_queues;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	for (i = 0; i < nb_links; i++)
> >  > > >  +		if (link[i].queue_id >= RTE_EVENT_MAX_QUEUES_PER_DEV)
> >  > > >  +			return -EINVAL;
> >  > > >  +
> >  > > >  +	diag = (*dev->dev_ops->port_link)(dev->data->ports[port_id], link,
> >  > > >  +						 nb_links);
> >  > > >  +	if (diag < 0)
> >  > > >  +		return diag;
> >  > > >  +
> >  > > >  +	links_map = dev->data->links_map;
> >  > > >  +	/* Point links_map to this port specific area */
> >  > > >  +	links_map += (port_id * RTE_EVENT_MAX_QUEUES_PER_DEV);
> >  > > >  +	for (i = 0; i < diag; i++)
> >  > > >  +		links_map[link[i].queue_id] = (uint8_t)link[i].priority;
> >  > > >  +
> >  > > >  +	return diag;
> >  > > >  +}
> >  > > >  +
> >  > > >  +#define EVENT_QUEUE_SERVICE_PRIORITY_INVALID (0xdead)
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_port_unlink(uint8_t dev_id, uint8_t port_id,
> >  > > >  +		      uint8_t queues[], uint16_t nb_unlinks)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	uint8_t all_queues[RTE_EVENT_MAX_QUEUES_PER_DEV];
> >  > > >  +	int i, diag;
> >  > > >  +	uint16_t *links_map;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_unlink, -
> >  > > >  ENOTSUP);
> >  > > >  +
> >  > > >  +	if (!is_valid_port(dev, port_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	if (queues == NULL) {
> >  > > >  +		for (i = 0; i < dev->data->nb_queues; i++)
> >  > > >  +			all_queues[i] = i;
> >  > > >  +		queues = all_queues;
> >  > > >  +		nb_unlinks = dev->data->nb_queues;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	for (i = 0; i < nb_unlinks; i++)
> >  > > >  +		if (queues[i] >= RTE_EVENT_MAX_QUEUES_PER_DEV)
> >  > > >  +			return -EINVAL;
> >  > > >  +
> >  > > >  +	diag = (*dev->dev_ops->port_unlink)(dev->data->ports[port_id],
> >  > > >  queues,
> >  > > >  +					nb_unlinks);
> >  > > >  +
> >  > > >  +	if (diag < 0)
> >  > > >  +		return diag;
> >  > > >  +
> >  > > >  +	links_map = dev->data->links_map;
> >  > > >  +	/* Point links_map to this port specific area */
> >  > > >  +	links_map += (port_id * RTE_EVENT_MAX_QUEUES_PER_DEV);
> >  > > >  +	for (i = 0; i < diag; i++)
> >  > > >  +		links_map[queues[i]] =
> >  > > >  EVENT_QUEUE_SERVICE_PRIORITY_INVALID;
> >  > > >  +
> >  > > >  +	return diag;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_port_links_get(uint8_t dev_id, uint8_t port_id,
> >  > > >  +			struct rte_event_queue_link link[])
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	uint16_t *links_map;
> >  > > >  +	int i, count = 0;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	if (!is_valid_port(dev, port_id)) {
> >  > > >  +		EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id);
> >  > > >  +		return -EINVAL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	links_map = dev->data->links_map;
> >  > > >  +	/* Point links_map to this port specific area */
> >  > > >  +	links_map += (port_id * RTE_EVENT_MAX_QUEUES_PER_DEV);
> >  > > >  +	for (i = 0; i < RTE_EVENT_MAX_QUEUES_PER_DEV; i++) {
> >  > > >  +		if (links_map[i] !=
> >  > > >  EVENT_QUEUE_SERVICE_PRIORITY_INVALID) {
> >  > > >  +			link[count].queue_id = i;
> >  > > >  +			link[count].priority = (uint8_t)links_map[i];
> >  > > >  +			++count;
> >  > > >  +		}
> >  > > >  +	}
> >  > > >  +	return count;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dequeue_wait_time(uint8_t dev_id, uint64_t ns, uint64_t
> >  > > >  *wait_ticks)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->wait_time, -
> >  > > >  ENOTSUP);
> >  > > >  +
> >  > > >  +	if (wait_ticks == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	(*dev->dev_ops->wait_time)(dev, ns, wait_ticks);
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_dump(uint8_t dev_id, FILE *f)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dump, -ENOTSUP);
> >  > > >  +
> >  > > >  +	(*dev->dev_ops->dump)(dev, f);
> >  > > >  +	return 0;
> >  > > >  +
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_start(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	int diag;
> >  > > >  +
> >  > > >  +	EDEV_LOG_DEBUG("Start dev_id=%" PRIu8, dev_id);
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_start, -ENOTSUP);
> >  > > >  +
> >  > > >  +	if (dev->data->dev_started != 0) {
> >  > > >  +		EDEV_LOG_ERR("Device with dev_id=%" PRIu8 " already
> >  > > >  started",
> >  > > >  +			dev_id);
> >  > > >  +		return 0;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	diag = (*dev->dev_ops->dev_start)(dev);
> >  > > >  +	if (diag == 0)
> >  > > >  +		dev->data->dev_started = 1;
> >  > > >  +	else
> >  > > >  +		return diag;
> >  > > >  +
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +void
> >  > > >  +rte_event_dev_stop(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	EDEV_LOG_DEBUG("Stop dev_id=%" PRIu8, dev_id);
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_RET(dev_id);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_RET(*dev->dev_ops->dev_stop);
> >  > > >  +
> >  > > >  +	if (dev->data->dev_started == 0) {
> >  > > >  +		EDEV_LOG_ERR("Device with dev_id=%" PRIu8 " already
> >  > > >  stopped",
> >  > > >  +			dev_id);
> >  > > >  +		return;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev->data->dev_started = 0;
> >  > > >  +	(*dev->dev_ops->dev_stop)(dev);
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_event_dev_close(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -
> >  > > >  ENOTSUP);
> >  > > >  +
> >  > > >  +	/* Device must be stopped before it can be closed */
> >  > > >  +	if (dev->data->dev_started == 1) {
> >  > > >  +		EDEV_LOG_ERR("Device %u must be stopped before closing",
> >  > > >  +				dev_id);
> >  > > >  +		return -EBUSY;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	return (*dev->dev_ops->dev_close)(dev);
> >  > > >  +}
> >  > > >  +
> >  > > >  +static inline int
> >  > > >  +rte_eventdev_data_alloc(uint8_t dev_id, struct rte_eventdev_data
> >  **data,
> >  > > >  +		int socket_id)
> >  > > >  +{
> >  > > >  +	char mz_name[RTE_EVENTDEV_NAME_MAX_LEN];
> >  > > >  +	const struct rte_memzone *mz;
> >  > > >  +	int n;
> >  > > >  +
> >  > > >  +	/* Generate memzone name */
> >  > > >  +	n = snprintf(mz_name, sizeof(mz_name), "rte_eventdev_data_%u",
> >  > > >  dev_id);
> >  > > >  +	if (n >= (int)sizeof(mz_name))
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> >  > > >  +		mz = rte_memzone_reserve(mz_name,
> >  > > >  +				sizeof(struct rte_eventdev_data),
> >  > > >  +				socket_id, 0);
> >  > > >  +	} else
> >  > > >  +		mz = rte_memzone_lookup(mz_name);
> >  > > >  +
> >  > > >  +	if (mz == NULL)
> >  > > >  +		return -ENOMEM;
> >  > > >  +
> >  > > >  +	*data = mz->addr;
> >  > > >  +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> >  > > >  +		memset(*data, 0, sizeof(struct rte_eventdev_data));
> >  > > >  +
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +static uint8_t
> >  > > >  +rte_eventdev_find_free_device_index(void)
> >  > > >  +{
> >  > > >  +	uint8_t dev_id;
> >  > > >  +
> >  > > >  +	for (dev_id = 0; dev_id < RTE_EVENT_MAX_DEVS; dev_id++) {
> >  > > >  +		if (rte_eventdevs[dev_id].attached ==
> >  > > >  +				RTE_EVENTDEV_DETACHED)
> >  > > >  +			return dev_id;
> >  > > >  +	}
> >  > > >  +	return RTE_EVENT_MAX_DEVS;
> >  > > >  +}
> >  > > >  +
> >  > > >  +struct rte_eventdev *
> >  > > >  +rte_eventdev_pmd_allocate(const char *name, int socket_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *eventdev;
> >  > > >  +	uint8_t dev_id;
> >  > > >  +
> >  > > >  +	if (rte_eventdev_pmd_get_named_dev(name) != NULL) {
> >  > > >  +		EDEV_LOG_ERR("Event device with name %s already "
> >  > > >  +				"allocated!", name);
> >  > > >  +		return NULL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	dev_id = rte_eventdev_find_free_device_index();
> >  > > >  +	if (dev_id == RTE_EVENT_MAX_DEVS) {
> >  > > >  +		EDEV_LOG_ERR("Reached maximum number of event
> >  > > >  devices");
> >  > > >  +		return NULL;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	eventdev = &rte_eventdevs[dev_id];
> >  > > >  +
> >  > > >  +	if (eventdev->data == NULL) {
> >  > > >  +		struct rte_eventdev_data *eventdev_data = NULL;
> >  > > >  +
> >  > > >  +		int retval = rte_eventdev_data_alloc(dev_id, &eventdev_data,
> >  > > >  +				socket_id);
> >  > > >  +
> >  > > >  +		if (retval < 0 || eventdev_data == NULL)
> >  > > >  +			return NULL;
> >  > > >  +
> >  > > >  +		eventdev->data = eventdev_data;
> >  > > >  +
> >  > > >  +		snprintf(eventdev->data->name,
> >  > > >  RTE_EVENTDEV_NAME_MAX_LEN,
> >  > > >  +				"%s", name);
> >  > > >  +
> >  > > >  +		eventdev->data->dev_id = dev_id;
> >  > > >  +		eventdev->data->socket_id = socket_id;
> >  > > >  +		eventdev->data->dev_started = 0;
> >  > > >  +
> >  > > >  +		eventdev->attached = RTE_EVENTDEV_ATTACHED;
> >  > > >  +
> >  > > >  +		eventdev_globals.nb_devs++;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	return eventdev;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_eventdev_pmd_release(struct rte_eventdev *eventdev)
> >  > > >  +{
> >  > > >  +	int ret;
> >  > > >  +
> >  > > >  +	if (eventdev == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	ret = rte_event_dev_close(eventdev->data->dev_id);
> >  > > >  +	if (ret < 0)
> >  > > >  +		return ret;
> >  > > >  +
> >  > > >  +	eventdev->attached = RTE_EVENTDEV_DETACHED;
> >  > > >  +	eventdev_globals.nb_devs--;
> >  > > >  +	eventdev->data = NULL;
> >  > > >  +
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  +
> >  > > >  +struct rte_eventdev *
> >  > > >  +rte_eventdev_pmd_vdev_init(const char *name, size_t
> >  dev_private_size,
> >  > > >  +		int socket_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *eventdev;
> >  > > >  +
> >  > > >  +	/* Allocate device structure */
> >  > > >  +	eventdev = rte_eventdev_pmd_allocate(name, socket_id);
> >  > > >  +	if (eventdev == NULL)
> >  > > >  +		return NULL;
> >  > > >  +
> >  > > >  +	/* Allocate private device structure */
> >  > > >  +	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> >  > > >  +		eventdev->data->dev_private =
> >  > > >  +				rte_zmalloc_socket("eventdev device private",
> >  > > >  +						dev_private_size,
> >  > > >  +						RTE_CACHE_LINE_SIZE,
> >  > > >  +						socket_id);
> >  > > >  +
> >  > > >  +		if (eventdev->data->dev_private == NULL)
> >  > > >  +			rte_panic("Cannot allocate memzone for private
> >  > > >  device"
> >  > > >  +					" data");
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	return eventdev;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_eventdev_pmd_pci_probe(struct rte_pci_driver *pci_drv,
> >  > > >  +			struct rte_pci_device *pci_dev)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev_driver *eventdrv;
> >  > > >  +	struct rte_eventdev *eventdev;
> >  > > >  +
> >  > > >  +	char eventdev_name[RTE_EVENTDEV_NAME_MAX_LEN];
> >  > > >  +
> >  > > >  +	int retval;
> >  > > >  +
> >  > > >  +	eventdrv = (struct rte_eventdev_driver *)pci_drv;
> >  > > >  +	if (eventdrv == NULL)
> >  > > >  +		return -ENODEV;
> >  > > >  +
> >  > > >  +	rte_eal_pci_device_name(&pci_dev->addr, eventdev_name,
> >  > > >  +			sizeof(eventdev_name));
> >  > > >  +
> >  > > >  +	eventdev = rte_eventdev_pmd_allocate(eventdev_name,
> >  > > >  +			 pci_dev->device.numa_node);
> >  > > >  +	if (eventdev == NULL)
> >  > > >  +		return -ENOMEM;
> >  > > >  +
> >  > > >  +	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> >  > > >  +		eventdev->data->dev_private =
> >  > > >  +				rte_zmalloc_socket(
> >  > > >  +						"eventdev private structure",
> >  > > >  +						eventdrv->dev_private_size,
> >  > > >  +						RTE_CACHE_LINE_SIZE,
> >  > > >  +						rte_socket_id());
> >  > > >  +
> >  > > >  +		if (eventdev->data->dev_private == NULL)
> >  > > >  +			rte_panic("Cannot allocate memzone for private "
> >  > > >  +					"device data");
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	eventdev->pci_dev = pci_dev;
> >  > > >  +	eventdev->driver = eventdrv;
> >  > > >  +
> >  > > >  +	/* Invoke PMD device initialization function */
> >  > > >  +	retval = (*eventdrv->eventdev_init)(eventdev);
> >  > > >  +	if (retval == 0)
> >  > > >  +		return 0;
> >  > > >  +
> >  > > >  +	EDEV_LOG_ERR("driver %s: event_dev_init(vendor_id=0x%x
> >  > > >  device_id=0x%x)"
> >  > > >  +			" failed", pci_drv->driver.name,
> >  > > >  +			(unsigned int) pci_dev->id.vendor_id,
> >  > > >  +			(unsigned int) pci_dev->id.device_id);
> >  > > >  +
> >  > > >  +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> >  > > >  +		rte_free(eventdev->data->dev_private);
> >  > > >  +
> >  > > >  +	eventdev->attached = RTE_EVENTDEV_DETACHED;
> >  > > >  +	eventdev_globals.nb_devs--;
> >  > > >  +
> >  > > >  +	return -ENXIO;
> >  > > >  +}
> >  > > >  +
> >  > > >  +int
> >  > > >  +rte_eventdev_pmd_pci_remove(struct rte_pci_device *pci_dev)
> >  > > >  +{
> >  > > >  +	const struct rte_eventdev_driver *eventdrv;
> >  > > >  +	struct rte_eventdev *eventdev;
> >  > > >  +	char eventdev_name[RTE_EVENTDEV_NAME_MAX_LEN];
> >  > > >  +	int ret;
> >  > > >  +
> >  > > >  +	if (pci_dev == NULL)
> >  > > >  +		return -EINVAL;
> >  > > >  +
> >  > > >  +	rte_eal_pci_device_name(&pci_dev->addr, eventdev_name,
> >  > > >  +			sizeof(eventdev_name));
> >  > > >  +
> >  > > >  +	eventdev = rte_eventdev_pmd_get_named_dev(eventdev_name);
> >  > > >  +	if (eventdev == NULL)
> >  > > >  +		return -ENODEV;
> >  > > >  +
> >  > > >  +	eventdrv = (const struct rte_eventdev_driver *)pci_dev->driver;
> >  > > >  +	if (eventdrv == NULL)
> >  > > >  +		return -ENODEV;
> >  > > >  +
> >  > > >  +	/* Invoke PMD device uninit function */
> >  > > >  +	if (*eventdrv->eventdev_uninit) {
> >  > > >  +		ret = (*eventdrv->eventdev_uninit)(eventdev);
> >  > > >  +		if (ret)
> >  > > >  +			return ret;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	/* Free event device */
> >  > > >  +	rte_eventdev_pmd_release(eventdev);
> >  > > >  +
> >  > > >  +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> >  > > >  +		rte_free(eventdev->data->dev_private);
> >  > > >  +
> >  > > >  +	eventdev->pci_dev = NULL;
> >  > > >  +	eventdev->driver = NULL;
> >  > > >  +
> >  > > >  +	return 0;
> >  > > >  +}
> >  > > >  diff --git a/lib/librte_eventdev/rte_eventdev_pmd.h
> >  > > >  b/lib/librte_eventdev/rte_eventdev_pmd.h
> >  > > >  new file mode 100644
> >  > > >  index 0000000..e9d9b83
> >  > > >  --- /dev/null
> >  > > >  +++ b/lib/librte_eventdev/rte_eventdev_pmd.h
> >  > > >  @@ -0,0 +1,504 @@
> >  > > >  +/*
> >  > > >  + *
> >  > > >  + *   Copyright(c) 2016 Cavium networks. All rights reserved.
> >  > > >  + *
> >  > > >  + *   Redistribution and use in source and binary forms, with or without
> >  > > >  + *   modification, are permitted provided that the following conditions
> >  > > >  + *   are met:
> >  > > >  + *
> >  > > >  + *     * Redistributions of source code must retain the above copyright
> >  > > >  + *       notice, this list of conditions and the following disclaimer.
> >  > > >  + *     * Redistributions in binary form must reproduce the above
> >  copyright
> >  > > >  + *       notice, this list of conditions and the following disclaimer in
> >  > > >  + *       the documentation and/or other materials provided with the
> >  > > >  + *       distribution.
> >  > > >  + *     * Neither the name of Cavium networks nor the names of its
> >  > > >  + *       contributors may be used to endorse or promote products derived
> >  > > >  + *       from this software without specific prior written permission.
> >  > > >  + *
> >  > > >  + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> >  > > >  CONTRIBUTORS
> >  > > >  + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,
> >  BUT
> >  > > >  NOT
> >  > > >  + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> >  > > >  FITNESS FOR
> >  > > >  + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> >  > > >  COPYRIGHT
> >  > > >  + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> >  > > >  INCIDENTAL,
> >  > > >  + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
> >  BUT
> >  > > >  NOT
> >  > > >  + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
> >  LOSS
> >  > > >  OF USE,
> >  > > >  + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> >  CAUSED
> >  > > >  AND ON ANY
> >  > > >  + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> >  OR
> >  > > >  TORT
> >  > > >  + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> >  OUT OF
> >  > > >  THE USE
> >  > > >  + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> >  > > >  DAMAGE.
> >  > > >  + */
> >  > > >  +
> >  > > >  +#ifndef _RTE_EVENTDEV_PMD_H_
> >  > > >  +#define _RTE_EVENTDEV_PMD_H_
> >  > > >  +
> >  > > >  +/** @file
> >  > > >  + * RTE Event PMD APIs
> >  > > >  + *
> >  > > >  + * @note
> >  > > >  + * These API are from event PMD only and user applications should not
> >  call
> >  > > >  + * them directly.
> >  > > >  + */
> >  > > >  +
> >  > > >  +#ifdef __cplusplus
> >  > > >  +extern "C" {
> >  > > >  +#endif
> >  > > >  +
> >  > > >  +#include <string.h>
> >  > > >  +
> >  > > >  +#include <rte_dev.h>
> >  > > >  +#include <rte_pci.h>
> >  > > >  +#include <rte_malloc.h>
> >  > > >  +#include <rte_log.h>
> >  > > >  +#include <rte_common.h>
> >  > > >  +
> >  > > >  +#include "rte_eventdev.h"
> >  > > >  +
> >  > > >  +#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
> >  > > >  +#define RTE_PMD_DEBUG_TRACE(...) \
> >  > > >  +	rte_pmd_debug_trace(__func__, __VA_ARGS__)
> >  > > >  +#else
> >  > > >  +#define RTE_PMD_DEBUG_TRACE(...)
> >  > > >  +#endif
> >  > > >  +
> >  > > >  +/* Logging Macros */
> >  > > >  +#define EDEV_LOG_ERR(fmt, args...) \
> >  > > >  +	RTE_LOG(ERR, EVENTDEV, "%s() line %u: " fmt "\n",  \
> >  > > >  +			__func__, __LINE__, ## args)
> >  > > >  +
> >  > > >  +#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
> >  > > >  +#define EDEV_LOG_DEBUG(fmt, args...) \
> >  > > >  +	RTE_LOG(DEBUG, EVENTDEV, "%s() line %u: " fmt "\n",  \
> >  > > >  +			__func__, __LINE__, ## args)
> >  > > >  +#else
> >  > > >  +#define EDEV_LOG_DEBUG(fmt, args...) (void)0
> >  > > >  +#endif
> >  > > >  +
> >  > > >  +/* Macros to check for valid device */
> >  > > >  +#define RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, retval) do
> >  { \
> >  > > >  +	if (!rte_eventdev_pmd_is_valid_dev((dev_id))) { \
> >  > > >  +		EDEV_LOG_ERR("Invalid dev_id=%d\n", dev_id); \
> >  > > >  +		return retval; \
> >  > > >  +	} \
> >  > > >  +} while (0)
> >  > > >  +
> >  > > >  +#define RTE_EVENTDEV_VALID_DEVID_OR_RET(dev_id) do { \
> >  > > >  +	if (!rte_eventdev_pmd_is_valid_dev((dev_id))) { \
> >  > > >  +		EDEV_LOG_ERR("Invalid dev_id=%d\n", dev_id); \
> >  > > >  +		return; \
> >  > > >  +	} \
> >  > > >  +} while (0)
> >  > > >  +
> >  > > >  +#define RTE_EVENTDEV_DETACHED  (0)
> >  > > >  +#define RTE_EVENTDEV_ATTACHED  (1)
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Initialisation function of a event driver invoked for each matching
> >  > > >  + * event PCI device detected during the PCI probing phase.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   The dev pointer is the address of the *rte_eventdev* structure
> >  associated
> >  > > >  + *   with the matching device and which has been [automatically]
> >  allocated in
> >  > > >  + *   the *rte_event_devices* array.
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   - 0: Success, the device is properly initialised by the driver.
> >  > > >  + *        In particular, the driver MUST have set up the *dev_ops* pointer
> >  > > >  + *        of the *dev* structure.
> >  > > >  + *   - <0: Error code of the device initialisation failure.
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_init_t)(struct rte_eventdev *dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Finalisation function of a driver invoked for each matching
> >  > > >  + * PCI device detected during the PCI closing phase.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   The dev pointer is the address of the *rte_eventdev* structure
> >  associated
> >  > > >  + *   with the matching device and which	has been [automatically]
> >  allocated in
> >  > > >  + *   the *rte_event_devices* array.
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   - 0: Success, the device is properly finalised by the driver.
> >  > > >  + *        In particular, the driver MUST free the *dev_ops* pointer
> >  > > >  + *        of the *dev* structure.
> >  > > >  + *   - <0: Error code of the device initialisation failure.
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_uninit_t)(struct rte_eventdev *dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * The structure associated with a PMD driver.
> >  > > >  + *
> >  > > >  + * Each driver acts as a PCI driver and is represented by a generic
> >  > > >  + * *event_driver* structure that holds:
> >  > > >  + *
> >  > > >  + * - An *rte_pci_driver* structure (which must be the first field).
> >  > > >  + *
> >  > > >  + * - The *eventdev_init* function invoked for each matching PCI device.
> >  > > >  + *
> >  > > >  + * - The size of the private data to allocate for each matching device.
> >  > > >  + */
> >  > > >  +struct rte_eventdev_driver {
> >  > > >  +	struct rte_pci_driver pci_drv;	/**< The PMD is also a PCI driver. */
> >  > > >  +	unsigned int dev_private_size;	/**< Size of device private data. */
> >  > > >  +
> >  > > >  +	eventdev_init_t eventdev_init;	/**< Device init function. */
> >  > > >  +	eventdev_uninit_t eventdev_uninit; /**< Device uninit function. */
> >  > > >  +};
> >  > > >  +
> >  > > >  +/** Global structure used for maintaining state of allocated event
> >  devices */
> >  > > >  +struct rte_eventdev_global {
> >  > > >  +	uint8_t nb_devs;	/**< Number of devices found */
> >  > > >  +	uint8_t max_devs;	/**< Max number of devices */
> >  > > >  +};
> >  > > >  +
> >  > > >  +extern struct rte_eventdev_global *rte_eventdev_globals;
> >  > > >  +/** Pointer to global event devices data structure. */
> >  > > >  +extern struct rte_eventdev *rte_eventdevs;
> >  > > >  +/** The pool of rte_eventdev structures. */
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Get the rte_eventdev structure device pointer for the named device.
> >  > > >  + *
> >  > > >  + * @param name
> >  > > >  + *   device name to select the device structure.
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   - The rte_eventdev structure pointer for the given device ID.
> >  > > >  + */
> >  > > >  +static inline struct rte_eventdev *
> >  > > >  +rte_eventdev_pmd_get_named_dev(const char *name)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +	unsigned int i;
> >  > > >  +
> >  > > >  +	if (name == NULL)
> >  > > >  +		return NULL;
> >  > > >  +
> >  > > >  +	for (i = 0, dev = &rte_eventdevs[i];
> >  > > >  +			i < rte_eventdev_globals->max_devs; i++) {
> >  > > >  +		if ((dev->attached == RTE_EVENTDEV_ATTACHED) &&
> >  > > >  +				(strcmp(dev->data->name, name) == 0))
> >  > > >  +			return dev;
> >  > > >  +	}
> >  > > >  +
> >  > > >  +	return NULL;
> >  > > >  +}
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Validate if the event device index is valid attached event device.
> >  > > >  + *
> >  > > >  + * @param dev_id
> >  > > >  + *   Event device index.
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   - If the device index is valid (1) or not (0).
> >  > > >  + */
> >  > > >  +static inline unsigned
> >  > > >  +rte_eventdev_pmd_is_valid_dev(uint8_t dev_id)
> >  > > >  +{
> >  > > >  +	struct rte_eventdev *dev;
> >  > > >  +
> >  > > >  +	if (dev_id >= rte_eventdev_globals->nb_devs)
> >  > > >  +		return 0;
> >  > > >  +
> >  > > >  +	dev = &rte_eventdevs[dev_id];
> >  > > >  +	if (dev->attached != RTE_EVENTDEV_ATTACHED)
> >  > > >  +		return 0;
> >  > > >  +	else
> >  > > >  +		return 1;
> >  > > >  +}
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Definitions of all functions exported by a driver through the
> >  > > >  + * the generic structure of type *event_dev_ops* supplied in the
> >  > > >  + * *rte_eventdev* structure associated with a device.
> >  > > >  + */
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Get device information of a device.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param dev_info
> >  > > >  + *   Event device information structure
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_info_get_t)(struct rte_eventdev *dev,
> >  > > >  +		struct rte_event_dev_info *dev_info);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Configure a device.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_configure_t)(struct rte_eventdev *dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Start a configured device.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_start_t)(struct rte_eventdev *dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Stop a configured device.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_stop_t)(struct rte_eventdev *dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Close a configured device.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + * - 0 on success
> >  > > >  + * - (-EAGAIN) if can't close as device is busy
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_close_t)(struct rte_eventdev *dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Retrieve the default event queue configuration.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param queue_id
> >  > > >  + *   Event queue index
> >  > > >  + * @param[out] queue_conf
> >  > > >  + *   Event queue configuration structure
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_queue_default_conf_get_t)(struct
> >  rte_eventdev
> >  > > >  *dev,
> >  > > >  +		uint8_t queue_id, struct rte_event_queue_conf *queue_conf);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Setup an event queue.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param queue_id
> >  > > >  + *   Event queue index
> >  > > >  + * @param queue_conf
> >  > > >  + *   Event queue configuration structure
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success.
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_queue_setup_t)(struct rte_eventdev *dev,
> >  > > >  +		uint8_t queue_id, struct rte_event_queue_conf *queue_conf);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Release memory resources allocated by given event queue.
> >  > > >  + *
> >  > > >  + * @param queue
> >  > > >  + *   Event queue pointer
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_queue_release_t)(void *queue);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Retrieve the default event port configuration.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param port_id
> >  > > >  + *   Event port index
> >  > > >  + * @param[out] port_conf
> >  > > >  + *   Event port configuration structure
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_port_default_conf_get_t)(struct rte_eventdev
> >  *dev,
> >  > > >  +		uint8_t port_id, struct rte_event_port_conf *port_conf);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Setup an event port.
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param port_id
> >  > > >  + *   Event port index
> >  > > >  + * @param port_conf
> >  > > >  + *   Event port configuration structure
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success.
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_port_setup_t)(struct rte_eventdev *dev,
> >  > > >  +		uint8_t port_id, struct rte_event_port_conf *port_conf);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Release memory resources allocated by given event port.
> >  > > >  + *
> >  > > >  + * @param port
> >  > > >  + *   Event port pointer
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_port_release_t)(void *port);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Link multiple source event queues to destination event port.
> >  > > >  + *
> >  > > >  + * @param port
> >  > > >  + *   Event port pointer
> >  > > >  + * @param link
> >  > > >  + *   An array of *nb_links* pointers to *rte_event_queue_link* structure
> >  > > >  + * @param nb_links
> >  > > >  + *   The number of links to establish
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success.
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_port_link_t)(void *port,
> >  > > >  +		struct rte_event_queue_link link[], uint16_t nb_links);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Unlink multiple source event queues from destination event port.
> >  > > >  + *
> >  > > >  + * @param port
> >  > > >  + *   Event port pointer
> >  > > >  + * @param queues
> >  > > >  + *   An array of *nb_unlinks* event queues to be unlinked from the event
> >  port.
> >  > > >  + * @param nb_unlinks
> >  > > >  + *   The number of unlinks to establish
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   Returns 0 on success.
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef int (*eventdev_port_unlink_t)(void *port,
> >  > > >  +		uint8_t queues[], uint16_t nb_unlinks);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Converts nanoseconds to *wait* value for rte_event_dequeue()
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param ns
> >  > > >  + *   Wait time in nanosecond
> >  > > >  + * @param[out] wait_ticks
> >  > > >  + *   Value for the *wait* parameter in rte_event_dequeue() function
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_dequeue_wait_time_t)(struct rte_eventdev
> >  *dev,
> >  > > >  +		uint64_t ns, uint64_t *wait_ticks);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Dump internal information
> >  > > >  + *
> >  > > >  + * @param dev
> >  > > >  + *   Event device pointer
> >  > > >  + * @param f
> >  > > >  + *   A pointer to a file for output
> >  > > >  + *
> >  > > >  + */
> >  > > >  +typedef void (*eventdev_dump_t)(struct rte_eventdev *dev, FILE *f);
> >  > > >  +
> >  > > >  +/** Event device operations function pointer table */
> >  > > >  +struct rte_eventdev_ops {
> >  > > >  +	eventdev_info_get_t dev_infos_get;	/**< Get device info. */
> >  > > >  +	eventdev_configure_t dev_configure;	/**< Configure device. */
> >  > > >  +	eventdev_start_t dev_start;		/**< Start device. */
> >  > > >  +	eventdev_stop_t dev_stop;		/**< Stop device. */
> >  > > >  +	eventdev_close_t dev_close;		/**< Close device. */
> >  > > >  +
> >  > > >  +	eventdev_queue_default_conf_get_t queue_def_conf;
> >  > > >  +	/**< Get default queue configuration. */
> >  > > >  +	eventdev_queue_setup_t queue_setup;
> >  > > >  +	/**< Set up an event queue. */
> >  > > >  +	eventdev_queue_release_t queue_release;
> >  > > >  +	/**< Release an event queue. */
> >  > > >  +
> >  > > >  +	eventdev_port_default_conf_get_t port_def_conf;
> >  > > >  +	/**< Get default port configuration. */
> >  > > >  +	eventdev_port_setup_t port_setup;
> >  > > >  +	/**< Set up an event port. */
> >  > > >  +	eventdev_port_release_t port_release;
> >  > > >  +	/**< Release an event port. */
> >  > > >  +
> >  > > >  +	eventdev_port_link_t port_link;
> >  > > >  +	/**< Link event queues to an event port. */
> >  > > >  +	eventdev_port_unlink_t port_unlink;
> >  > > >  +	/**< Unlink event queues from an event port. */
> >  > > >  +	eventdev_dequeue_wait_time_t wait_time;
> >  > > >  +	/**< Converts nanoseconds to *wait* value for rte_event_dequeue()
> >  > > >  */
> >  > > >  +	eventdev_dump_t dump;
> >  > > >  +	/* Dump internal information */
> >  > > >  +};
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Allocates a new eventdev slot for an event device and returns the
> >  pointer
> >  > > >  + * to that slot for the driver to use.
> >  > > >  + *
> >  > > >  + * @param name
> >  > > >  + *   Unique identifier name for each device
> >  > > >  + * @param socket_id
> >  > > >  + *   Socket to allocate resources on.
> >  > > >  + * @return
> >  > > >  + *   - Slot in the rte_dev_devices array for a new device;
> >  > > >  + */
> >  > > >  +struct rte_eventdev *
> >  > > >  +rte_eventdev_pmd_allocate(const char *name, int socket_id);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Release the specified eventdev device.
> >  > > >  + *
> >  > > >  + * @param eventdev
> >  > > >  + * The *eventdev* pointer is the address of the *rte_eventdev*
> >  structure.
> >  > > >  + * @return
> >  > > >  + *   - 0 on success, negative on error
> >  > > >  + */
> >  > > >  +int
> >  > > >  +rte_eventdev_pmd_release(struct rte_eventdev *eventdev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Creates a new virtual event device and returns the pointer to that
> >  device.
> >  > > >  + *
> >  > > >  + * @param name
> >  > > >  + *   PMD type name
> >  > > >  + * @param dev_private_size
> >  > > >  + *   Size of event PMDs private data
> >  > > >  + * @param socket_id
> >  > > >  + *   Socket to allocate resources on.
> >  > > >  + *
> >  > > >  + * @return
> >  > > >  + *   - Eventdev pointer if device is successfully created.
> >  > > >  + *   - NULL if device cannot be created.
> >  > > >  + */
> >  > > >  +struct rte_eventdev *
> >  > > >  +rte_eventdev_pmd_vdev_init(const char *name, size_t
> >  dev_private_size,
> >  > > >  +		int socket_id);
> >  > > >  +
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Wrapper for use by pci drivers as a .probe function to attach to a
> >  event
> >  > > >  + * interface.
> >  > > >  + */
> >  > > >  +int rte_eventdev_pmd_pci_probe(struct rte_pci_driver *pci_drv,
> >  > > >  +			    struct rte_pci_device *pci_dev);
> >  > > >  +
> >  > > >  +/**
> >  > > >  + * Wrapper for use by pci drivers as a .remove function to detach a
> >  event
> >  > > >  + * interface.
> >  > > >  + */
> >  > > >  +int rte_eventdev_pmd_pci_remove(struct rte_pci_device *pci_dev);
> >  > > >  +
> >  > > >  +#ifdef __cplusplus
> >  > > >  +}
> >  > > >  +#endif
> >  > > >  +
> >  > > >  +#endif /* _RTE_EVENTDEV_PMD_H_ */
> >  > > >  diff --git a/lib/librte_eventdev/rte_eventdev_version.map
> >  > > >  b/lib/librte_eventdev/rte_eventdev_version.map
> >  > > >  new file mode 100644
> >  > > >  index 0000000..ef40aae
> >  > > >  --- /dev/null
> >  > > >  +++ b/lib/librte_eventdev/rte_eventdev_version.map
> >  > > >  @@ -0,0 +1,39 @@
> >  > > >  +DPDK_17.02 {
> >  > > >  +	global:
> >  > > >  +
> >  > > >  +	rte_eventdevs;
> >  > > >  +
> >  > > >  +	rte_event_dev_count;
> >  > > >  +	rte_event_dev_get_dev_id;
> >  > > >  +	rte_event_dev_socket_id;
> >  > > >  +	rte_event_dev_info_get;
> >  > > >  +	rte_event_dev_configure;
> >  > > >  +	rte_event_dev_start;
> >  > > >  +	rte_event_dev_stop;
> >  > > >  +	rte_event_dev_close;
> >  > > >  +	rte_event_dev_dump;
> >  > > >  +
> >  > > >  +	rte_event_port_default_conf_get;
> >  > > >  +	rte_event_port_setup;
> >  > > >  +	rte_event_port_dequeue_depth;
> >  > > >  +	rte_event_port_enqueue_depth;
> >  > > >  +	rte_event_port_count;
> >  > > >  +	rte_event_port_link;
> >  > > >  +	rte_event_port_unlink;
> >  > > >  +	rte_event_port_links_get;
> >  > > >  +
> >  > > >  +	rte_event_queue_default_conf_get
> >  > > >  +	rte_event_queue_setup;
> >  > > >  +	rte_event_queue_count;
> >  > > >  +	rte_event_queue_priority;
> >  > > >  +
> >  > > >  +	rte_event_dequeue_wait_time;
> >  > > >  +
> >  > > >  +	rte_eventdev_pmd_allocate;
> >  > > >  +	rte_eventdev_pmd_release;
> >  > > >  +	rte_eventdev_pmd_vdev_init;
> >  > > >  +	rte_eventdev_pmd_pci_probe;
> >  > > >  +	rte_eventdev_pmd_pci_remove;
> >  > > >  +
> >  > > >  +	local: *;
> >  > > >  +};
> >  > > >  diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> >  > > >  index f75f0e2..716725a 100644
> >  > > >  --- a/mk/rte.app.mk
> >  > > >  +++ b/mk/rte.app.mk
> >  > > >  @@ -93,6 +93,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF)           += -
> >  > > >  lrte_mbuf
> >  > > >   _LDLIBS-$(CONFIG_RTE_LIBRTE_NET)            += -lrte_net
> >  > > >   _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER)          += -lrte_ethdev
> >  > > >   _LDLIBS-$(CONFIG_RTE_LIBRTE_CRYPTODEV)      += -lrte_cryptodev
> >  > > >  +_LDLIBS-$(CONFIG_RTE_LIBRTE_EVENTDEV)       += -lrte_eventdev
> >  > > >   _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMPOOL)        += -lrte_mempool
> >  > > >   _LDLIBS-$(CONFIG_RTE_LIBRTE_RING)           += -lrte_ring
> >  > > >   _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
> >  > > >  --
> >  > > >  2.5.5
> >  > >

^ permalink raw reply

* Re: [PATCH 00/56] Solarflare libefx-based PMD
From: Mcnamara, John @ 2016-11-23  9:57 UTC (permalink / raw)
  To: Andrew Rybchenko, Yigit, Ferruh, dev@dpdk.org
In-Reply-To: <5d041f12-ec8d-77da-47ac-671eb3a39a5b@solarflare.com>



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Andrew Rybchenko
> Sent: Wednesday, November 23, 2016 7:50 AM
> To: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 00/56] Solarflare libefx-based PMD
> 
> On 11/23/2016 03:02 AM, Ferruh Yigit wrote:
> > On 11/21/2016 3:00 PM, Andrew Rybchenko wrote:
> >> The patch series adds Solarflare libefx-based network PMD.
> >>
> >> This version of the driver supports Solarflare SFN7xxx and SFN8xxx
> >> families of 10/40 Gbps adapters.
> >>
> >> ...
> 
> > 2- Got following compile issues, not investigated, directly sharing
> here:
> >
> > b) for icc getting following warnings:
> > =======================================
> > icc: command line warning #10006: ignoring unknown option '-Wno-empty-
> body'
> > icc: command line warning #10006: ignoring unknown option
> > '-Waggregate-return'
> > icc: command line warning #10006: ignoring unknown option
> > '-Wbad-function-cast'
> > icc: command line warning #10006: ignoring unknown option '-Wnested-
> externs'
> >
> >
> > c) icc compiler errors:
> > =======================================
> > In file included from
> > .../x86_64-native-linuxapp-icc/include/rte_ethdev.h(185),
> >                   from .../drivers/net/sfc/efx/sfc.h(35),
> >                   from .../drivers/net/sfc/efx/sfc.c(37):
> > .../x86_64-native-linuxapp-icc/include/rte_ether.h(258): warning #2203:
> > cast discards qualifiers from target type
> >          uint16_t *from_words = (uint16_t *)(ea_from->addr_bytes);
> >                                 ^
> >
> > .../drivers/net/sfc/efx/base/efx_mcdi.c(1157): warning #3179:
> > deprecated conversion of string literal to char* (should be const
> > char*)
> > .../drivers/net/sfc/efx/base/ef10_filter.c(1276): warning #188:
> > enumerated type mixed with another type
> >                  : "unknown assertion";
> >                  ^
> >
> >                  filter_flags = 0;
> >                               ^
> >
> > .../drivers/net/sfc/efx/base/efx_mcdi.c(1426): warning #188:
> > enumerated type mixed with another type
> >          epp->ep_fixed_port_type =
> >                                  ^
> >
> > .../drivers/net/sfc/efx/base/efx_nic.c(556): warning #188: enumerated
> > type mixed with another type
> >          enp->en_family = 0;
> 
> Yes, I have no ICC compilers. I'll try to fix these warnings, but I can't
> be sure without checking it.
> Also we cannot claim ICC supported without building and testing the
> generated binary.

Hi,

You can get a copy of ICC with a 30 day evaluation license here: https://software.intel.com/en-us/try-buy-tools

John

^ permalink raw reply

* Re: [RFC PATCH 0/6] Restructure EAL device model for bus support
From: Shreyansh Jain @ 2016-11-23  9:45 UTC (permalink / raw)
  To: David Marchand; +Cc: dev@dpdk.org
In-Reply-To: <CALwxeUuDNdS67YJm3=Ga6vikh=0T8eyou+_jBNO3_HxtanTJxg@mail.gmail.com>

I should have replied to this earlier, apologies.

On Sunday 20 November 2016 09:00 PM, David Marchand wrote:
> On Thu, Nov 17, 2016 at 6:29 AM, Shreyansh Jain <shreyansh.jain@nxp.com> wrote:
>> DPDK has been inherently a PCI inclined framework. Because of this, the
>> design of device tree (or list) within DPDK is also PCI inclined. A non-PCI
>> device doesn't have a way of being expressed without using hooks started from
>> EAL to PMD.
>>
>> With this cover letter, some patches are presented which try to break this
>> strict linkage of EAL with PCI devices. Aim is to generalize the device
>> hierarchy on the lines of how Linux handles it:
>>
>>         device A1
>>           |
>>   +==.===='==============.============+ Bus A.
>>      |                    `--> driver A11     \
>>   device A2                `-> driver A12      \______
>>                                                 |CPU |
>>                                                 /`````
>>         device A1                              /
>>           |                                   /
>>   +==.===='==============.============+ Bus A`
>>      |                    `--> driver B11
>>   device A2                `-> driver B12
>>
>> Simply put:
>>  - a bus is connect to CPU (or core)
>>  - devices are conneted to Bus
>>  - drivers are running instances which manage one or more devices
>>  - bus is responsible for identifying devices (and interrupt propogation)
>>  - driver is responsible for initializing the device
>>
>> (*Reusing text from email [1])
>> In context of DPDK EAL:
>>  - a generic bus (not a driver, not a device). I don't know how to categorize
>>    a bus. It is certainly not a device, and then handler for a bus (physical)
>>    can be considered a 'bus driver'. So, just 'rte_bus'.
>>  - there is a bus for each physical implementation (or virtual). So, a rte_bus
>>    Object for PCI, VDEV, ABC, DEF and so on.
>>  - Buses are registered just like a PMD - RTE_PMD_BUS_REGISTER()
>>  - Each registered bus is part of a doubly list.
>>    -- Each device inherits rte_bus
>>    -- Each driver inherits rte_bus
>>    -- Device and Drivers lists are part of rte_bus
>>  - eth_driver is no more required - it was just a holder for PMDs to register
>>    themselves. It can be replaced with rte_xxx_driver and corresponding init/
>>    uninit moved to rte_driver
>>  - rte_eth_dev modified to disassociate itself from rte_pci_device and connect
>>    to generic rte_device
>>
>> Once again, improvising from [1]:
>>
>>                                   __ rte_bus_list
>>                                  /
>>                      +----------'---+
>>                      |rte_bus       |
>>                      | driver_list------> List of rte_bus specific
>>                      | device_list----    devices
>>                      | scan         | `-> List of rte_bus associated
>>                      | match        |     drivers
>>                      | dump         |
>>                      | ..some refcnt| (#)
>>                      +--|------|----+
>>               _________/        \_________
>>     +--------/----+                     +-\---------------+
>>     |rte_device   |                     |rte_driver       |
>>     | rte_bus     |                     | rte_bus         |
>>     | rte_driver  |(#)                  | init            |
>>     |             |                     | uninit          |
>>     |  devargs    |                     | dev_private_size|
>>     +---||--------+                     | drv_flags       |(#)
>>         ||                              | intr_handle(2*) |(#)
>>         | \                             +----------\\\----+
>>         |  \_____________                           \\\
>>         |                \                          |||
>>  +------|---------+ +----|----------+               |||
>>  |rte_pci_device  | |rte_xxx_device | (4*)          |||
>>  | PCI specific   | | xxx device    |               |||
>>  | info (mem,)    | | specific fns  |              / | \
>>  +----------------+ +---------------+             /  |  \
>>                             _____________________/  /    \
>>                            /                    ___/      \
>>             +-------------'--+    +------------'---+    +--'------------+
>>             |rte_pci_driver  |    |rte_vdev_driver |    |rte_xxx_driver |
>>             | PCI id table,  |    | <probably,     |    | ....          |
>>             | other driver   |    |  nothing>      |    +---------------+
>>             | data           |    +----------------+
>>             +----------------+
>>
>>     (1*) Problem is that probe functions have different arguments. So,
>>          generalizing them might be some rework in the respective device
>>          layers
>>     (2*) Interrupt handling for each driver type might be different. I am not
>>          sure how to generalize that either. This is grey area for me.
>>     (3*) Probably exposing a bitmask for device capabilities. Nothing similar
>>          exists now to relate it. Don't know if that is useful. Allowing
>>          applications to question a device about what it supports and what it
>>          doesn't - making it more flexible at application layer (but more code
>>          as well.)
>>     (4*) Even vdev would be an instantiated as a device. It is not being done
>>          currently.
>>     (#)  Items which are still pending
>>
>> With this cover letter, some patches have been posted. These are _only_ for
>> discussion purpose. They are not complete and neither compilable.
>> All the patches, except 0001, have sufficient context about what the changes
>> are and rationale for same. Obviously, code is best answer.
>
> As said by Jan B., I too think that splitting the patches into smaller
> patchsets is important.
>
> Looking at your datamodel, some quick comments :
> - Why init/uninit in rte_driver ? Why not probe/remove ?

No specific reason. At first I wasn't planning to replace driver->init 
with driver->probe (which rte_pci_driver) is doing. But, I will do this 
in v2.

> - I don't think dev_private_size makes sense in rte_driver. The bus is
> responsible for creating rte_device objects (embedded or not in
> rte_$bus_device objects). If a driver needs to allocate some priv (for
> whatever needs), the driver should do the allocation itself (or maybe
> a helper for current eth_driver), then reference the original
> rte_$bus_device object it got from the probe method.

first off, this came as a suggestion on the ML somewhere as far as I 
remember.
Secondly, your point makes sense. I will move it back.

>
>
> For a first patchset, I would see:
> - introduce the rte_bus object. In rte_eal_init, for each bus, we call
> the scan method. Then, for each bus, we find the appropriate
> rte_driver using the bus match method then call the probe method. If
> the probe succeeds, the rte_device points to the associated
> rte_driver,
> - migrate the pci scan code to a pci bus (scan looks at sysfs for
> linux / ioctl for bsd + devargs for blacklist / whitelist ?), match is
> the same at what is done in rte_eal_pci_probe_one_driver() at the
> moment,
> - migrate the vdev init code to a vdev bus (scan looks at devargs):
> this is new, we must create rte_device objects for vdev drivers to use
> later

Thanks for outlay - I agree with that.

>
> Then we can talk about the next steps once the bus is in place.

I will post the new set probably tomorrow or day-after.

>
>

-
Shreyansh

^ permalink raw reply

* Re: Proposal for a new Committer model
From: Mcnamara, John @ 2016-11-23  8:21 UTC (permalink / raw)
  To: Neil Horman, Thomas Monjalon; +Cc: dev@dpdk.org, Jerin Jacob, Stephen Hemminger
In-Reply-To: <20161122195215.GA4463@hmsreliant.think-freely.org>



> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Tuesday, November 22, 2016 7:52 PM
> To: Thomas Monjalon <thomas.monjalon@6wind.com>
> Cc: dev@dpdk.org; Mcnamara, John <john.mcnamara@intel.com>
> Subject: Re: [dpdk-dev] Proposal for a new Committer model
> 
> On Mon, Nov 21, 2016 at 09:52:41AM +0100, Thomas Monjalon wrote:
> > 2016-11-18 13:09, Neil Horman:
> > > A) Further promote subtree maintainership.  This was a conversation
> > > that I proposed some time ago, but my proposed granularity was
> > > discarded in favor of something that hasn't worked as well (in my
> > > opinion).  That is to say a few driver pmds (i40e and fm10k come to
> > > mind) have their own tree that send pull requests to Thomas.
> >
> > Yes we tried this fine granularity and stated that it was not working
> well.
> > We are now using the bigger granularity that you describe below.
> >
> Ok, thats good, but that must be _very_ new.  Looking at your git tree, I
> see no merge commits.  How are you pulling from those subtrees?
> 


Hi Neil,

It seems like the weight of consensus in around your proposal for further subtree maintainers and backups. If you don't mind I'll take your text and redraft it as a potential section on maintainership for a future Project Charter document. Or at least so that we have a documented maintainship process.

 
> > > We should be sharding that at a much higher granularity and using it
> > > much more consistently.  That is to say, that we should have a
> > > maintainer for all the ethernet pmds, and another for the crypto
> > > pmds, another for the core eal layer, another for misc libraries
> > > that have low patch volumes, etc.
> >
> > Yes we could open a tree for EAL and another one for the core libraries.
> >
> That could be worthwhile.  Lets see how the net and crypto subtrees work
> out (assuming again that these trees are newly founded)

Could we define some of the potential subtrees now and look to introduce them in the this release cycle? EAL and the Core libs, as suggested by Thomas, seem like 2 obvious ones.


> 
> > > Each of those subdivisions should have their own list to communicate
> > > on, and each should have a tree that integrates patches for their
> > > own subsystem, and they should on a regular cycle send pull requests
> > > to Thomas.
> >
> > Yes I think it is now a good idea to split the mailing list traffic,
> > at least for netdev and cryptodev.

I'd prefer not to have split dev lists, for now at least. We can reevaluate that again in a few months though.


> >
> 
> > > B) Designate alternates to serve as backups for the maintainer when
> > > they are unavailable.  This provides high-availablility, and sounds
> > > very much like your proposal, but in the interests of clarity, there
> > > is still a single maintainer at any one time, it just may change to
> > > ensure the continued merging of patches, if the primary maintainer
> isn't available.
> > > Ideally however, those backup alternates arent needed, because most
> > > of the primary maintainers work in merging pull requests, which are
> > > done based on the trust of the submaintainer, and done during a very
> > > limited window of time.  This also partially addreses multi-vendor
> > > fairness if your subtree maintainers come from multiple participating
> companies.


+1 on this apart from the limited merge window (for reasons similar to Thomas).

Should we have a call for volunteers for backup, on master and the sub-trees, followed by a simple +1 from community members to endorse them?


John

^ permalink raw reply

* Re: [PATCH 00/56] Solarflare libefx-based PMD
From: Andrew Rybchenko @ 2016-11-23  7:49 UTC (permalink / raw)
  To: Ferruh Yigit, dev
In-Reply-To: <9cdced57-ba4c-fd0e-c509-904d333aab50@intel.com>

On 11/23/2016 03:02 AM, Ferruh Yigit wrote:
> On 11/21/2016 3:00 PM, Andrew Rybchenko wrote:
>> The patch series adds Solarflare libefx-based network PMD.
>>
>> This version of the driver supports Solarflare SFN7xxx and SFN8xxx
>> families of 10/40 Gbps adapters.
>>
>> libefx is a platform-independent library to implement drivers for
>> Solarflare network adapters. It provides unified adapter family
>> independent interface (if possible). FreeBSD [1] and illumos [2]
>> drivers are built on top of the library.
>>
>> The patch series could be logically structured into 5 sub-series:
>>   1. (1) add the driver skeleton including documentation
>>   2. (2-30) import libefx and include it in build with the latest patch
>>   3. (31-43) implement minimal device level operations in steps
>>   4. (44-51) implement Rx subsystem
>>   5. (52-56) implement Tx subsystem
>>
>> Functional driver with multi-queue support capable to send and receive
>> traffic appears with the last patch in the series.
>>
>> The following design decisions are made during development:
>>
>>   1. Since libefx uses positive errno return codes, positive errno
>>      return codes are used inside the driver and coversion to negative
>>      is done on return from eth_dev_ops callbacks. We think that it
>>      is the less error-prone way.
>>
>>   2. Another Solarflare PMD with in-kernel part (for control operations)
>>      is considered and could be added in the future. Code for data path
>>      should be shared by these two drivers. libefx-based PMD is put into
>>      'efx' subdirectory to have a space for another PMD and shared code.
>>
>>   3. Own event queue (a way to deliver events from HW to host CPU) is
>>      used for house-keeping (e.g. link status notifications), each Tx
>>      and each Rx queue. No locks on datapath are requires in this case.
>>
>>   4. Alarm is used to periodically poll house-keeping event queue.
>>      The event queue is used to deliver link status change notifications,
>>      Rx/Tx queue flush events, SRAM events. It is not used on datapath.
>>      The event queue polling is protected using spin-lock since
>>      concurrent access from different contexts is possible (e.g. device
>>      stop when polling alarm is running).
>>
>> [1] https://svnweb.freebsd.org/base/head/sys/dev/sfxge/common/
>> [2] https://github.com/illumos/illumos-gate/tree/master/usr/src/uts/common/io/sfxge/common/
>>
>> ---
>>
>> Andrew Rybchenko (49):
>>    net/sfc: libefx-based PMD stub sufficient to build and init
>>    net/sfc: import libefx base
>>    net/sfc: import libefx register definitions
>>    net/sfc: import libefx filters support
>>    net/sfc: import libefx MCDI definition
>>    net/sfc: import libefx MCDI implementation
>>    net/sfc: import libefx MCDI logging support
>>    net/sfc: import libefx MCDI proxy authorization support
>>    net/sfc: import libefx 5xxx/6xxx family support
>>    net/sfc: import libefx SFN7xxx family support
>>    net/sfc: import libefx SFN8xxx family support
>>    net/sfc: import libefx diagnostics support
>>    net/sfc: import libefx built-in selftest support
>>    net/sfc: import libefx software per-queue statistics support
>>    net/sfc: import libefx PHY flags control support
>>    net/sfc: import libefx PHY statistics support
>>    net/sfc: import libefx PHY LEDs control support
>>    net/sfc: import libefx MAC statistics support
>>    net/sfc: import libefx event prefetch support
>>    net/sfc: import libefx Rx scatter support
>>    net/sfc: import libefx RSS support
>>    net/sfc: import libefx loopback control support
>>    net/sfc: import libefx monitors statistics support
>>    net/sfc: import libefx support to access monitors via MCDI
>>    net/sfc: import libefx support for Rx packed stream mode
>>    net/sfc: import libefx NVRAM support
>>    net/sfc: import libefx VPD support
>>    net/sfc: import libefx bootrom configuration support
>>    net/sfc: import libefx licensing support
>>    net/sfc: implement dummy callback to get device information
>>    net/sfc: implement driver operation to init device on attach
>>    net/sfc: add device configure and close stubs
>>    net/sfc: add device configuration checks
>>    net/sfc: implement device start and stop operations
>>    net/sfc: make available resources estimation and allocation
>>    net/sfc: interrupts support sufficient for event queue init
>>    net/sfc: implement event queue support
>>    net/sfc: implement EVQ dummy exception handling
>>    net/sfc: maintain management event queue
>>    net/sfc: periodic management EVQ polling using alarm
>>    net/sfc: minimum port control sufficient to receive traffic
>>    net/sfc: implement Rx subsystem stubs
>>    net/sfc: check configured rxmode
>>    net/sfc: implement Rx queue setup release operations
>>    net/sfc: calculate Rx buffer size which may be used
>>    net/sfc: validate Rx queue buffers setup
>>    net/sfc: implement Rx queue start and stop operations
>>    net/sfc: implement device callback to Rx burst of packets
>>    net/sfc: discard scattered packet on Rx correctly
>>
>> Artem Andreev (2):
>>    net/sfc: include libefx in build
>>    net/sfc: implement device operation to retrieve link info
>>
>> Ivan Malov (5):
>>    net/sfc: provide basic stubs for Tx subsystem
>>    net/sfc: add function to check configured Tx mode
>>    net/sfc: add callbacks to set up and release Tx queues
>>    net/sfc: implement transmit path start / stop
>>    net/sfc: add callback to send bursts of packets
>>
> Hi Andrew,
>
> Thank you for the patch, I have encounter with a few minor issues, can
> you please check them [1]?	
>
> Also folder structure is drivers/net/sfc/efx/<all_src_files>, why /sfc/
> layer is created?
> sfc is company name (solarflare communications), right? Other driver
> folders not structured based on company, what about using
> drivers/net/efx/* ?

I've tried to explain it above in item (2):

 >>>

  2. Another Solarflare PMD with in-kernel part (for control operations)
     is considered and could be added in the future. Code for data path
     should be shared by these two drivers. libefx-based PMD is put into
     'efx' subdirectory to have a space for another PMD and shared code.

<<<

So, main reason is to have location for the code shared by two Solarflare
network PMDs. May be it better to relocate when we really have it.
I'm open for other ideas/suggestions.

> [1]:
> 1- There are a few (non-base) checkpatch warnings, can you please check
> patch 36, 49, 50 and 55 please?

Thanks, I'll fix spelling in v2.
36, 49 and 55 also ask to check multiple assignments. IMHO, it is the 
right usage
of multiple assignment when logically bound variables must have the same 
value.

> 2- Got following compile issues, not investigated, directly sharing here:
>
> b) for icc getting following warnings:
> =======================================
> icc: command line warning #10006: ignoring unknown option '-Wno-empty-body'
> icc: command line warning #10006: ignoring unknown option
> '-Waggregate-return'
> icc: command line warning #10006: ignoring unknown option
> '-Wbad-function-cast'
> icc: command line warning #10006: ignoring unknown option '-Wnested-externs'
>
>
> c) icc compiler errors:
> =======================================
> In file included from
> .../x86_64-native-linuxapp-icc/include/rte_ethdev.h(185),
>                   from .../drivers/net/sfc/efx/sfc.h(35),
>                   from .../drivers/net/sfc/efx/sfc.c(37):
> .../x86_64-native-linuxapp-icc/include/rte_ether.h(258): warning #2203:
> cast discards qualifiers from target type
>          uint16_t *from_words = (uint16_t *)(ea_from->addr_bytes);
>                                 ^
>
> .../drivers/net/sfc/efx/base/efx_mcdi.c(1157): warning #3179: deprecated
> conversion of string literal to char* (should be const char*)
> .../drivers/net/sfc/efx/base/ef10_filter.c(1276): warning #188:
> enumerated type mixed with another type
>                  : "unknown assertion";
>                  ^
>
>                  filter_flags = 0;
>                               ^
>
> .../drivers/net/sfc/efx/base/efx_mcdi.c(1426): warning #188: enumerated
> type mixed with another type
>          epp->ep_fixed_port_type =
>                                  ^
>
> .../drivers/net/sfc/efx/base/efx_nic.c(556): warning #188: enumerated
> type mixed with another type
>          enp->en_family = 0;

Yes, I have no ICC compilers. I'll try to fix these warnings, but I 
can't be sure without checking it.
Also we cannot claim ICC supported without building and testing the 
generated binary.

Many thanks,
Andrew.

^ permalink raw reply

* Re: [PATCH 5/7] test/eventdev: unit and functional tests
From: Jerin Jacob @ 2016-11-23  3:32 UTC (permalink / raw)
  To: Harry van Haaren; +Cc: dev, Gage Eads, David Hunt
In-Reply-To: <1479319207-130646-6-git-send-email-harry.van.haaren@intel.com>

On Wed, Nov 16, 2016 at 06:00:05PM +0000, Harry van Haaren wrote:
> This commit adds basic unit and functional tests for the eventdev
> API. The test code is added in this commit, but not yet enabled until
> the next commit.
> 
> Signed-off-by: Gage Eads <gage.eads@intel.com>
> Signed-off-by: David Hunt <david.hunt@intel.com>
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---

A few comments on portability and usage perspective. See below,

> +#include <rte_eal.h>
> +#include <rte_per_lcore.h>
> +#include <rte_lcore.h>
> +#include <rte_debug.h>
> +#include <rte_ethdev.h>
> +#include <rte_cycles.h>
> +
> +#include <rte_eventdev.h>
> +#include "test.h"
> +
> +
> +static inline int
> +create_ports(struct test *t, int num_ports)
> +{
> +	int i;
> +	static const struct rte_event_port_conf conf = {
> +			.dequeue_queue_depth = 32,
> +			.enqueue_queue_depth = 64,
> +	};

Check the max supported through info get first.

> +
> +	for (i = 0; i < num_ports; i++) {
> +		if (rte_event_port_setup(t->ev, i, &conf) < 0) {
> +			printf("Error setting up port %d\n", i);
> +			return -1;
> +		}
> +		t->port[i] = i;
> +	}
> +
> +	return 0;
> +}
> +
> +
> +static int
> +run_prio_packet_test(struct test *t)

Run per event enqueue priority test if the platform supports
RTE_EVENT_DEV_CAP_EVENT_QOS


> +{
> +	int err;
> +	const uint32_t MAGIC_SEQN[] = {4711, 1234};
> +	const uint32_t PRIORITY[] = {3, 0};
> +	unsigned i;
> +	for(i = 0; i < RTE_DIM(MAGIC_SEQN); i++) {
> +		/* generate pkt and enqueue */
> +		struct rte_event ev;
> +		struct rte_mbuf *arp = rte_gen_arp(0, t->mbuf_pool);
> +		if (!arp) {
> +			printf("%d: gen of pkt failed\n", __LINE__);
> +			return -1;
> +		}
> +		arp->seqn = MAGIC_SEQN[i];

For me, it make sense to don't touch any field in mbuf to make eventdev
model works. use private field to store test specific data

> +
> +		ev = (struct rte_event){
> +			.priority = PRIORITY[i],
> +			.operation = RTE_EVENT_OP_NEW,
> +			.queue_id = t->qid[0],
> +			.mbuf = arp
> +		};
> +		err = rte_event_enqueue(t->ev, t->port[0], &ev, 0);
> +		if (err < 0) {
> +			printf("%d: error failed to enqueue\n", __LINE__);
> +			return -1;
> +		}
> +	}
> +
> +	rte_event_schedule(t->ev);
> +
> +	struct rte_event_dev_stats stats;
> +	err = rte_event_dev_stats_get(t->ev, &stats);
> +	if (err) {
> +		printf("%d: error failed to get stats\n", __LINE__);
> +		return -1;
> +	}
> +
> +	if (stats.port_rx_pkts[t->port[0]] != 2) {
> +		printf("%d: error stats incorrect for directed port\n", __LINE__);
> +		rte_event_dev_dump(stdout, t->ev);
> +		return -1;
> +	}

rely on stats for functional verification may not work in all the
implementation. makes sense to have more concrete functional
verification without stats

> +
> +	struct rte_event ev, ev2;
> +	uint32_t deq_pkts;
> +	deq_pkts = rte_event_dequeue(t->ev, t->port[0], &ev, 0);
> +	if (deq_pkts != 1) {
> +		printf("%d: error failed to deq\n", __LINE__);
> +		rte_event_dev_dump(stdout, t->ev);
> +		return -1;
> +	}
> +	if(ev.mbuf->seqn != MAGIC_SEQN[1]) {
> +		printf("%d: first packet out not highest priority\n", __LINE__);
> +		rte_event_dev_dump(stdout, t->ev);
> +		return -1;
> +	}
> +	rte_pktmbuf_free(ev.mbuf);
> +
> +
> +static int
> +test_overload_trip(struct test *t)

overload tests wont fail in ddr backed systems.(ddr backed system will
mimic infinite size queue to application). So testing against failure
may not work at all in some implementation

> +{
> +	int err;
> +
> +	/* Create instance with 3 directed QIDs going to 3 ports */
> +	if (init(t, 1, 1) < 0 ||
> +			create_ports(t, 1) < 0 ||
> +			create_atomic_qids(t, 1) < 0)
> +		return -1;
> +

^ permalink raw reply

* Re: [PATCH v2] examples/ethtool: fix bug in drvinfo callback
From: Remy Horton @ 2016-11-23  1:00 UTC (permalink / raw)
  To: Qiming Yang, dev
In-Reply-To: <1479778884-26993-1-git-send-email-qiming.yang@intel.com>

Not sure this is a problem in practice, as the same set if fields is 
updated each time...at least for now.

On 22/11/2016 09:41, Qiming Yang wrote:
> Function pcmd_drvinfo_callback uses struct info to get
> the ethtool information of each port. Struct info will
> store the information of previous port until this
> information be updated. This patch fixes this issue.
>
> Fixes: bda68ab9d1e7 ("examples/ethtool: add user-space ethtool sample application")
>
> Signed-off-by: Qiming Yang <qiming.yang@intel.com>

Acked-by: Remy Horton <remy.horton@intel.com>

^ permalink raw reply

* Re: [PATCH 7/7] examples/eventdev_pipeline: adding example
From: Jerin Jacob @ 2016-11-23  0:30 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: Van Haaren, Harry, dev@dpdk.org, Eads, Gage
In-Reply-To: <59AF69C657FD0841A61C55336867B5B035B4D827@IRSMSX103.ger.corp.intel.com>

On Tue, Nov 22, 2016 at 02:04:27PM +0000, Richardson, Bruce wrote:
> 
> 
> > -----Original Message-----
> > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> > Sent: Tuesday, November 22, 2016 6:02 AM
> > To: Van Haaren, Harry <harry.van.haaren@intel.com>
> > Cc: dev@dpdk.org; Eads, Gage <gage.eads@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Subject: Re: [dpdk-dev] [PATCH 7/7] examples/eventdev_pipeline: adding
> > example
> > 
> > On Wed, Nov 16, 2016 at 06:00:07PM +0000, Harry van Haaren wrote:
> > > This patch adds a sample app to the examples/ directory, which can be
> > > used as a reference application and for general testing. The
> > > application requires two ethdev ports and expects traffic to be
> > > flowing. The application must be run with the --vdev flags as follows
> > > to indicate to EAL that a virtual eventdev device called "evdev_sw0" is
> > available to be used:
> > >
> > >     ./build/eventdev_pipeline --vdev evdev_sw0
> > >
> > > The general flow of the traffic is as follows:
> > >
> > >     Rx core -> Atomic Queue => 4 worker cores => TX core
> > >
> > > A scheduler core is required to do the packet scheduling, making this
> > > configuration require 7 cores (Rx, Tx, Scheduler, and 4 workers).
> > > Finally a master core brings the core count to 8 for this
> > > configuration. The
> > 
> > Thanks for the example application.I will try to share my views on ethdev
> > integration and usability perspective. Hope we can converge.
> 
> Hi Jerin, 
> 
> thanks for the feedback. We'll take it on board for a subsequent version
> we produce. Additional comments and queries on your feedback inline below.

Thanks Bruce.

> 
> /Bruce
> 
> > 
> > Some of the high level details first before getting into exact details.
> > 
> > 1) From the HW and ethdev integration perspective, The integrated NIC
> > controllers does not need producer core(s) to push the event/packets to
> > event queue. So, I was thinking to use 6WIND rte_flow spec to create the
> > "ethdev port to event queue wiring" connection by extending the output
> > ACTION definition, which specifies event queue its need to enqueued to for
> > the given ethdev port (something your are doing in application).
> > 
> > I guess, the producer part of this example can be created as common code,
> > somewhere in rte_flow/ethdev to reuse. We would need this scheme also
> > where when we deal with external nics + HW event manager use case
> > 
> Yes. This is something to consider.
> 
> For the pure-software model, we also might want to look at the opposite
> approach, where we register an ethdev with the scheduler for it to "pull"
> new packets from. This would allow it to bypass the port logic for the new
> packets. 

Not sure,I understand this completely. How different its integrating
with rte_flow specification ?

> 
> An alternative for this is to extend the schedule API to allow a burst of
> packets to be passed in to be scheduled immediately as "NEW" packets. The end
> results should be the same, saving cycles by bypassing unneeded processing
> for the new packets.
> 
> > The complete event driven model can be verified and exercised without
> > integrating with eventdev subsystem. So I think, may be we need to focus
> > on functional applications without ethdev to verify the eventdev features
> > like(automatic multicore scaling,  dynamic load balancing, pipelining,
> > packet ingress order maintenance and synchronization services) and then
> > integrate with ethdev
> 
> Yes, comprehensive unit tests will be needed too. But an example app that
> pulls packets from an external NIC I also think is needed to get a feel
> for the performance of the scheduler with real traffic.

I agree, we need to have example to show case with real traffic.

Please check on ethdev integration aspects. Cavium has both server
(that's going to use SW event pmd) and NPU based platform(that's going to
use HW event pmd). So we would like to have common approach that makes
integration of both models with out changing the application.

I was thinking more with "service core" and "rte_flow" based
integration methodology to make that happen.

> 
> > 
> > > +	const unsigned cores_needed = num_workers +
> > > +			/*main*/1 +
> > > +			/*sched*/1 +
> > > +			/*TX*/1 +
> > > +			/*RX*/1;
> > > +
> > 
> > 2) One of the prime aims of the event driven model is to remove the fixed
> > function core mappings and enable automatic multicore scaling,  dynamic
> > load balancing etc.I will try to use an example in review section to show
> > the method for removing "consumer core" in this case.
> 
> Yes, I agree, but unfortunately, for some tasks, distributing those tasks
> across multiple cores can hurt performance overall do to resource contention.

May only in SW implementation.

>  
> > 
> > > application can be configured for various numbers of flows and worker
> > > cores. Run the application with -h for details.
> > >
> > 
> > Another way to deal wit out additional consumer core(it creates issue in
> > scaling and load balancing) is
> > 
> > in worker:
> > while(1) {
> > 
> > 	ev = dequeue(port);
> > 
> > 	// stage 1 app processing
> > 	if (ev.event_type == RTE_EVENT_TYPE_ETHDEV) {
> > 		// identify the Ethernet port and tx queue the packet needs to
> > go
> > 		// create the flow based on that
> > 		ev.flow_id = flow(port_id, tx_queue_id);
> > 		ev.sched_type = RTE_SCHED_TYPE_ATOMIC;
> > 		ev.operation = RTE_EVENT_OP_FORWARD;
> > 		ev.event_type = RTE_EVENT_TYPE_CORE;
> > 	} // stage 2 app processing
> > 	else if (ev.event_type == RTE_EVENT_TYPE_CORE) {
> > 		port_id = function_of(ev.flow_id) ;// look stage 1 processing
> > 		tx_queue_id = function_of(ev.flow_id) //look stage 1
> > processing
> > 		remaining ethdev based tx is same as yours
> > 	}
> > 	enqueue(ev);
> > }
> >
> Yes, but you still need some core to do the work of pushing the packets into
> the scheduler from the NIC, if you don't have a hardware path from NIC to 
> HW scheduler. [Features like RSS can obviously help here with distributing that
> work if needed]

Yes. make sense to have producer portion of code as common code.

> 
> In the case you do have a HW path - which I assume is the Cavium case - I assume
> that the EVENT_TYPE_ETHDEV branch above needs also to take care of desc to mbuf
> processing, as is normally done by the PMD?
>  
> > 
> > 
> > > +			ev->priority = 0;
> > > +			ev->sched_type = RTE_SCHED_TYPE_ATOMIC;
> > > +			ev->operation = RTE_EVENT_OP_FORWARD;
> > > +
> > > +			uint64_t now = rte_rdtsc();
> > > +			while(now + 750 > rte_rdtsc()) {}
> > 
> > Why delay ?
> 
> Simulate some work being done by the worker, which makes the app slightly more
> realistic and also helps the scheduler as there is not so much contention on the
> shared cache lines.

May not for performance test-cases.

> 

^ permalink raw reply

* Re: [PATCH 00/56] Solarflare libefx-based PMD
From: Ferruh Yigit @ 2016-11-23  0:10 UTC (permalink / raw)
  To: Andrew Rybchenko, dev
In-Reply-To: <9cdced57-ba4c-fd0e-c509-904d333aab50@intel.com>

On 11/23/2016 12:02 AM, Ferruh Yigit wrote:
> On 11/21/2016 3:00 PM, Andrew Rybchenko wrote:
>> The patch series adds Solarflare libefx-based network PMD.
>>
>> This version of the driver supports Solarflare SFN7xxx and SFN8xxx
>> families of 10/40 Gbps adapters.
>>
>> libefx is a platform-independent library to implement drivers for
>> Solarflare network adapters. It provides unified adapter family
>> independent interface (if possible). FreeBSD [1] and illumos [2]
>> drivers are built on top of the library.
>>
>> The patch series could be logically structured into 5 sub-series:
>>  1. (1) add the driver skeleton including documentation
>>  2. (2-30) import libefx and include it in build with the latest patch
>>  3. (31-43) implement minimal device level operations in steps
>>  4. (44-51) implement Rx subsystem
>>  5. (52-56) implement Tx subsystem
>>
>> Functional driver with multi-queue support capable to send and receive
>> traffic appears with the last patch in the series.
>>
>> The following design decisions are made during development:
>>
>>  1. Since libefx uses positive errno return codes, positive errno
>>     return codes are used inside the driver and coversion to negative
>>     is done on return from eth_dev_ops callbacks. We think that it
>>     is the less error-prone way.
>>
>>  2. Another Solarflare PMD with in-kernel part (for control operations)
>>     is considered and could be added in the future. Code for data path
>>     should be shared by these two drivers. libefx-based PMD is put into
>>     'efx' subdirectory to have a space for another PMD and shared code.
>>
>>  3. Own event queue (a way to deliver events from HW to host CPU) is
>>     used for house-keeping (e.g. link status notifications), each Tx
>>     and each Rx queue. No locks on datapath are requires in this case.
>>
>>  4. Alarm is used to periodically poll house-keeping event queue.
>>     The event queue is used to deliver link status change notifications,
>>     Rx/Tx queue flush events, SRAM events. It is not used on datapath.
>>     The event queue polling is protected using spin-lock since
>>     concurrent access from different contexts is possible (e.g. device
>>     stop when polling alarm is running).
>>
>> [1] https://svnweb.freebsd.org/base/head/sys/dev/sfxge/common/
>> [2] https://github.com/illumos/illumos-gate/tree/master/usr/src/uts/common/io/sfxge/common/
>>
>> ---
>>
>> Andrew Rybchenko (49):
>>   net/sfc: libefx-based PMD stub sufficient to build and init
>>   net/sfc: import libefx base
>>   net/sfc: import libefx register definitions
>>   net/sfc: import libefx filters support
>>   net/sfc: import libefx MCDI definition
>>   net/sfc: import libefx MCDI implementation
>>   net/sfc: import libefx MCDI logging support
>>   net/sfc: import libefx MCDI proxy authorization support
>>   net/sfc: import libefx 5xxx/6xxx family support
>>   net/sfc: import libefx SFN7xxx family support
>>   net/sfc: import libefx SFN8xxx family support
>>   net/sfc: import libefx diagnostics support
>>   net/sfc: import libefx built-in selftest support
>>   net/sfc: import libefx software per-queue statistics support
>>   net/sfc: import libefx PHY flags control support
>>   net/sfc: import libefx PHY statistics support
>>   net/sfc: import libefx PHY LEDs control support
>>   net/sfc: import libefx MAC statistics support
>>   net/sfc: import libefx event prefetch support
>>   net/sfc: import libefx Rx scatter support
>>   net/sfc: import libefx RSS support
>>   net/sfc: import libefx loopback control support
>>   net/sfc: import libefx monitors statistics support
>>   net/sfc: import libefx support to access monitors via MCDI
>>   net/sfc: import libefx support for Rx packed stream mode
>>   net/sfc: import libefx NVRAM support
>>   net/sfc: import libefx VPD support
>>   net/sfc: import libefx bootrom configuration support
>>   net/sfc: import libefx licensing support
>>   net/sfc: implement dummy callback to get device information
>>   net/sfc: implement driver operation to init device on attach
>>   net/sfc: add device configure and close stubs
>>   net/sfc: add device configuration checks
>>   net/sfc: implement device start and stop operations
>>   net/sfc: make available resources estimation and allocation
>>   net/sfc: interrupts support sufficient for event queue init
>>   net/sfc: implement event queue support
>>   net/sfc: implement EVQ dummy exception handling
>>   net/sfc: maintain management event queue
>>   net/sfc: periodic management EVQ polling using alarm
>>   net/sfc: minimum port control sufficient to receive traffic
>>   net/sfc: implement Rx subsystem stubs
>>   net/sfc: check configured rxmode
>>   net/sfc: implement Rx queue setup release operations
>>   net/sfc: calculate Rx buffer size which may be used
>>   net/sfc: validate Rx queue buffers setup
>>   net/sfc: implement Rx queue start and stop operations
>>   net/sfc: implement device callback to Rx burst of packets
>>   net/sfc: discard scattered packet on Rx correctly
>>
>> Artem Andreev (2):
>>   net/sfc: include libefx in build
>>   net/sfc: implement device operation to retrieve link info
>>
>> Ivan Malov (5):
>>   net/sfc: provide basic stubs for Tx subsystem
>>   net/sfc: add function to check configured Tx mode
>>   net/sfc: add callbacks to set up and release Tx queues
>>   net/sfc: implement transmit path start / stop
>>   net/sfc: add callback to send bursts of packets
>>
> 
> Hi Andrew,
> 
> Thank you for the patch, I have encounter with a few minor issues, can
> you please check them [1]?	
> 
> Also folder structure is drivers/net/sfc/efx/<all_src_files>, why /sfc/
> layer is created?
> sfc is company name (solarflare communications), right? Other driver
> folders not structured based on company, what about using
> drivers/net/efx/* ?
> 
> 
> [1]:
> 1- There are a few (non-base) checkpatch warnings, can you please check
> patch 36, 49, 50 and 55 please?
> 
> 2- Got following compile issues, not investigated, directly sharing here:
> 
> a) for i686-native-linuxapp-gcc target:
> =======================================

I just saw that driver doesn't support 32bit, please forget about this ones.

> In file included from .../i686-native-linuxapp-gcc/include/rte_debug.h:46:0,
>                  from .../drivers/net/sfc/efx/efsys.h:39,
>                  from .../drivers/net/sfc/efx/base/efx.h:34,
>                  from .../drivers/net/sfc/efx/sfc.c:35:
> .../drivers/net/sfc/efx/sfc.c: In function ‘sfc_dma_alloc’:
> .../drivers/net/sfc/efx/sfc.c:51:32: warning: format ‘%lu’ expects
> argument of type ‘long unsigned int’, but argument 12 has type ‘size_t
> {aka unsigned int}’ [-Wformat=]
>         name, id, len, socket_id);
>                                 ^
> .../i686-native-linuxapp-gcc/include/rte_log.h:268:25: note: in
> definition of macro ‘RTE_LOG’
>     RTE_LOGTYPE_ ## t, # t ": " __VA_ARGS__) : \
>                          ^
> .../drivers/net/sfc/efx/sfc_log.h:65:4: note: in expansion of macro
> ‘SFC_LOG’
>     SFC_LOG(sa, INFO,    \
>     ^~~~~~~
> .../drivers/net/sfc/efx/sfc.c:50:2: note: in expansion of macro
> ‘sfc_log_init’
>   sfc_log_init(sa, "name=%s id=%u len=%lu socket_id=%d",
>   ^~~~~~~~~~~~
> 
> b) for icc getting following warnings:
> =======================================
> icc: command line warning #10006: ignoring unknown option '-Wno-empty-body'
> icc: command line warning #10006: ignoring unknown option
> '-Waggregate-return'
> icc: command line warning #10006: ignoring unknown option
> '-Wbad-function-cast'
> icc: command line warning #10006: ignoring unknown option '-Wnested-externs'
> 
> 
> c) icc compiler errors:
> =======================================

And probably you don't have ICC compilers, I can help if that is the case.

> In file included from
> .../x86_64-native-linuxapp-icc/include/rte_ethdev.h(185),
>                  from .../drivers/net/sfc/efx/sfc.h(35),
>                  from .../drivers/net/sfc/efx/sfc.c(37):
> .../x86_64-native-linuxapp-icc/include/rte_ether.h(258): warning #2203:
> cast discards qualifiers from target type
>         uint16_t *from_words = (uint16_t *)(ea_from->addr_bytes);
>                                ^
> 
> .../drivers/net/sfc/efx/base/efx_mcdi.c(1157): warning #3179: deprecated
> conversion of string literal to char* (should be const char*)
> .../drivers/net/sfc/efx/base/ef10_filter.c(1276): warning #188:
> enumerated type mixed with another type
>                 : "unknown assertion";
>                 ^
> 
>                 filter_flags = 0;
>                              ^
> 
> .../drivers/net/sfc/efx/base/efx_mcdi.c(1426): warning #188: enumerated
> type mixed with another type
>         epp->ep_fixed_port_type =
>                                 ^
> 
> .../drivers/net/sfc/efx/base/efx_nic.c(556): warning #188: enumerated
> type mixed with another type
>         enp->en_family = 0;
>                        ^
> 
> Thanks,
> ferruh
> 

^ permalink raw reply

* Re: [PATCH] eal: postpone vdev initialization
From: Jerin Jacob @ 2016-11-23  0:07 UTC (permalink / raw)
  To: Ferruh Yigit; +Cc: dev, declan.doherty, david.marchand, thomas.monjalon
In-Reply-To: <fb64cdec-f086-896a-3463-6d23bdea440b@intel.com>

On Mon, Nov 21, 2016 at 05:35:58PM +0000, Ferruh Yigit wrote:
> On 11/21/2016 5:02 PM, Jerin Jacob wrote:
> > On Mon, Nov 21, 2016 at 09:54:57AM +0000, Ferruh Yigit wrote:
> >> On 11/20/2016 8:00 AM, Jerin Jacob wrote:
> >>> Some platform like octeontx may use pci and
> >>> vdev based combined device to represent a logical
> >>> dpdk functional device.In such case, postponing the
> >>> vdev initialization after pci device
> >>> initialization will provide the better view of
> >>> the pci device resources in the system in
> >>> vdev's probe function, and it allows better
> >>> functional subsystem registration in vdev probe
> >>> function.
> >>>
> >>> As a bonus, This patch fixes a bond device
> >>> initialization use case.
> >>>
> >>> example command to reproduce the issue:
> >>> ./testpmd -c 0x2  --vdev 'eth_bond0,mode=0,
> >>> slave=0000:02:00.0,slave=0000:03:00.0' --
> >>> --port-topology=chained
> >>>
> >>> root cause:
> >>> In existing case(vdev initialization and then pci
> >>> initialization), creates three Ethernet ports with
> >>> following port ids
> >>> 0 - Bond device
> >>> 1 - PCI device 0
> >>> 2 - PCI devive 1
> >>>
> >>> Since testpmd, calls the configure/start on all the ports on
> >>> start up,it will translate to following illegal setup sequence
> >>>
> >>> 1)bond device configure/start
> >>> 1.1) pci device0 stop/configure/start
> >>> 1.2) pci device1 stop/configure/start
> >>> 2)pci device 0 configure(illegal setup case,
> >>> as device in start state)
> >>>
> >>> The fix changes the initialization sequence and
> >>> allow initialization in following valid setup order
> >>> 1) pcie device 0 configure/start
> >>> 2) pcie device 1 configure/start
> >>> 3) bond device 2 configure/start
> >>> 3.1) pcie device 0/stop/configure/start
> >>> 3.2) pcie device 1/stop/configure/start
> >>>
> >>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> >>> ---
> >>
> >> This changes the port id assignments to the devices, right?
> >>
> >> Previously virtual devices get first available port ids (0..N1), later
> >> physical devices (N1..N2). Now this becomes reverse.
> >>
> >> Can this change break some existing user applications?
> > 
> > I guess it may be effected only to ethdev bond pmd based application,
> > which is broken anyway.
> 
> My concern is, this may effect the applications that use "--vdev" eal
> parameter and has an assumption about port assignment.

Not sure. Application expectation on specific port assignment is bad anyway.
But in any event, what we do with exiting ethdev bond pmd failure.

> 
> And if this breaks any userspace application, does it require a
> deprecation notice?

I am not sure. Thomas, Any input on this ?

> 
> > Let me know what it takes to make forward progress on this patch. I can
> > fix the same in v2.
> > 
> > Jerin
> > 
> 

^ permalink raw reply

* Re: [PATCH 00/56] Solarflare libefx-based PMD
From: Ferruh Yigit @ 2016-11-23  0:02 UTC (permalink / raw)
  To: Andrew Rybchenko, dev
In-Reply-To: <1479740470-6723-1-git-send-email-arybchenko@solarflare.com>

On 11/21/2016 3:00 PM, Andrew Rybchenko wrote:
> The patch series adds Solarflare libefx-based network PMD.
> 
> This version of the driver supports Solarflare SFN7xxx and SFN8xxx
> families of 10/40 Gbps adapters.
> 
> libefx is a platform-independent library to implement drivers for
> Solarflare network adapters. It provides unified adapter family
> independent interface (if possible). FreeBSD [1] and illumos [2]
> drivers are built on top of the library.
> 
> The patch series could be logically structured into 5 sub-series:
>  1. (1) add the driver skeleton including documentation
>  2. (2-30) import libefx and include it in build with the latest patch
>  3. (31-43) implement minimal device level operations in steps
>  4. (44-51) implement Rx subsystem
>  5. (52-56) implement Tx subsystem
> 
> Functional driver with multi-queue support capable to send and receive
> traffic appears with the last patch in the series.
> 
> The following design decisions are made during development:
> 
>  1. Since libefx uses positive errno return codes, positive errno
>     return codes are used inside the driver and coversion to negative
>     is done on return from eth_dev_ops callbacks. We think that it
>     is the less error-prone way.
> 
>  2. Another Solarflare PMD with in-kernel part (for control operations)
>     is considered and could be added in the future. Code for data path
>     should be shared by these two drivers. libefx-based PMD is put into
>     'efx' subdirectory to have a space for another PMD and shared code.
> 
>  3. Own event queue (a way to deliver events from HW to host CPU) is
>     used for house-keeping (e.g. link status notifications), each Tx
>     and each Rx queue. No locks on datapath are requires in this case.
> 
>  4. Alarm is used to periodically poll house-keeping event queue.
>     The event queue is used to deliver link status change notifications,
>     Rx/Tx queue flush events, SRAM events. It is not used on datapath.
>     The event queue polling is protected using spin-lock since
>     concurrent access from different contexts is possible (e.g. device
>     stop when polling alarm is running).
> 
> [1] https://svnweb.freebsd.org/base/head/sys/dev/sfxge/common/
> [2] https://github.com/illumos/illumos-gate/tree/master/usr/src/uts/common/io/sfxge/common/
> 
> ---
> 
> Andrew Rybchenko (49):
>   net/sfc: libefx-based PMD stub sufficient to build and init
>   net/sfc: import libefx base
>   net/sfc: import libefx register definitions
>   net/sfc: import libefx filters support
>   net/sfc: import libefx MCDI definition
>   net/sfc: import libefx MCDI implementation
>   net/sfc: import libefx MCDI logging support
>   net/sfc: import libefx MCDI proxy authorization support
>   net/sfc: import libefx 5xxx/6xxx family support
>   net/sfc: import libefx SFN7xxx family support
>   net/sfc: import libefx SFN8xxx family support
>   net/sfc: import libefx diagnostics support
>   net/sfc: import libefx built-in selftest support
>   net/sfc: import libefx software per-queue statistics support
>   net/sfc: import libefx PHY flags control support
>   net/sfc: import libefx PHY statistics support
>   net/sfc: import libefx PHY LEDs control support
>   net/sfc: import libefx MAC statistics support
>   net/sfc: import libefx event prefetch support
>   net/sfc: import libefx Rx scatter support
>   net/sfc: import libefx RSS support
>   net/sfc: import libefx loopback control support
>   net/sfc: import libefx monitors statistics support
>   net/sfc: import libefx support to access monitors via MCDI
>   net/sfc: import libefx support for Rx packed stream mode
>   net/sfc: import libefx NVRAM support
>   net/sfc: import libefx VPD support
>   net/sfc: import libefx bootrom configuration support
>   net/sfc: import libefx licensing support
>   net/sfc: implement dummy callback to get device information
>   net/sfc: implement driver operation to init device on attach
>   net/sfc: add device configure and close stubs
>   net/sfc: add device configuration checks
>   net/sfc: implement device start and stop operations
>   net/sfc: make available resources estimation and allocation
>   net/sfc: interrupts support sufficient for event queue init
>   net/sfc: implement event queue support
>   net/sfc: implement EVQ dummy exception handling
>   net/sfc: maintain management event queue
>   net/sfc: periodic management EVQ polling using alarm
>   net/sfc: minimum port control sufficient to receive traffic
>   net/sfc: implement Rx subsystem stubs
>   net/sfc: check configured rxmode
>   net/sfc: implement Rx queue setup release operations
>   net/sfc: calculate Rx buffer size which may be used
>   net/sfc: validate Rx queue buffers setup
>   net/sfc: implement Rx queue start and stop operations
>   net/sfc: implement device callback to Rx burst of packets
>   net/sfc: discard scattered packet on Rx correctly
> 
> Artem Andreev (2):
>   net/sfc: include libefx in build
>   net/sfc: implement device operation to retrieve link info
> 
> Ivan Malov (5):
>   net/sfc: provide basic stubs for Tx subsystem
>   net/sfc: add function to check configured Tx mode
>   net/sfc: add callbacks to set up and release Tx queues
>   net/sfc: implement transmit path start / stop
>   net/sfc: add callback to send bursts of packets
> 

Hi Andrew,

Thank you for the patch, I have encounter with a few minor issues, can
you please check them [1]?	

Also folder structure is drivers/net/sfc/efx/<all_src_files>, why /sfc/
layer is created?
sfc is company name (solarflare communications), right? Other driver
folders not structured based on company, what about using
drivers/net/efx/* ?


[1]:
1- There are a few (non-base) checkpatch warnings, can you please check
patch 36, 49, 50 and 55 please?

2- Got following compile issues, not investigated, directly sharing here:

a) for i686-native-linuxapp-gcc target:
=======================================
In file included from .../i686-native-linuxapp-gcc/include/rte_debug.h:46:0,
                 from .../drivers/net/sfc/efx/efsys.h:39,
                 from .../drivers/net/sfc/efx/base/efx.h:34,
                 from .../drivers/net/sfc/efx/sfc.c:35:
.../drivers/net/sfc/efx/sfc.c: In function ‘sfc_dma_alloc’:
.../drivers/net/sfc/efx/sfc.c:51:32: warning: format ‘%lu’ expects
argument of type ‘long unsigned int’, but argument 12 has type ‘size_t
{aka unsigned int}’ [-Wformat=]
        name, id, len, socket_id);
                                ^
.../i686-native-linuxapp-gcc/include/rte_log.h:268:25: note: in
definition of macro ‘RTE_LOG’
    RTE_LOGTYPE_ ## t, # t ": " __VA_ARGS__) : \
                         ^
.../drivers/net/sfc/efx/sfc_log.h:65:4: note: in expansion of macro
‘SFC_LOG’
    SFC_LOG(sa, INFO,    \
    ^~~~~~~
.../drivers/net/sfc/efx/sfc.c:50:2: note: in expansion of macro
‘sfc_log_init’
  sfc_log_init(sa, "name=%s id=%u len=%lu socket_id=%d",
  ^~~~~~~~~~~~

b) for icc getting following warnings:
=======================================
icc: command line warning #10006: ignoring unknown option '-Wno-empty-body'
icc: command line warning #10006: ignoring unknown option
'-Waggregate-return'
icc: command line warning #10006: ignoring unknown option
'-Wbad-function-cast'
icc: command line warning #10006: ignoring unknown option '-Wnested-externs'


c) icc compiler errors:
=======================================
In file included from
.../x86_64-native-linuxapp-icc/include/rte_ethdev.h(185),
                 from .../drivers/net/sfc/efx/sfc.h(35),
                 from .../drivers/net/sfc/efx/sfc.c(37):
.../x86_64-native-linuxapp-icc/include/rte_ether.h(258): warning #2203:
cast discards qualifiers from target type
        uint16_t *from_words = (uint16_t *)(ea_from->addr_bytes);
                               ^

.../drivers/net/sfc/efx/base/efx_mcdi.c(1157): warning #3179: deprecated
conversion of string literal to char* (should be const char*)
.../drivers/net/sfc/efx/base/ef10_filter.c(1276): warning #188:
enumerated type mixed with another type
                : "unknown assertion";
                ^

                filter_flags = 0;
                             ^

.../drivers/net/sfc/efx/base/efx_mcdi.c(1426): warning #188: enumerated
type mixed with another type
        epp->ep_fixed_port_type =
                                ^

.../drivers/net/sfc/efx/base/efx_nic.c(556): warning #188: enumerated
type mixed with another type
        enp->en_family = 0;
                       ^

Thanks,
ferruh

^ permalink raw reply

* Re: [PATCH 2/4] eventdev: implement the northbound APIs
From: Jerin Jacob @ 2016-11-22 23:43 UTC (permalink / raw)
  To: Eads, Gage
  Cc: dev@dpdk.org, Richardson, Bruce, Van Haaren, Harry,
	hemant.agrawal@nxp.com
In-Reply-To: <9184057F7FC11744A2107296B6B8EB1E01E331A3@FMSMSX108.amr.corp.intel.com>

On Tue, Nov 22, 2016 at 10:48:32PM +0000, Eads, Gage wrote:
> 
> 
> >  -----Original Message-----
> >  From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> >  Sent: Tuesday, November 22, 2016 2:00 PM
> >  To: Eads, Gage <gage.eads@intel.com>
> >  Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Van
> >  Haaren, Harry <harry.van.haaren@intel.com>; hemant.agrawal@nxp.com
> >  Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs
> >  
> >  On Tue, Nov 22, 2016 at 07:43:03PM +0000, Eads, Gage wrote:
> >  > >  > >  > > One open issue I noticed is the "typical workflow"
> >  > > description starting in  > >  rte_eventdev.h:204 conflicts with the
> >  > > centralized software PMD that Harry  > >  posted last week.
> >  > > Specifically, that PMD expects a single core to call the  > >
> >  > > schedule function. We could extend the documentation to account for
> >  > > this  > >  alternative style of scheduler invocation, or discuss
> >  > > ways to make the  software  > >  PMD work with the documented
> >  > > workflow. I prefer the former, but either  way I  > >  think we
> >  > > ought to expose the scheduler's expected usage to the user --
> >  > > perhaps  > >  through an RTE_EVENT_DEV_CAP flag?
> >  > >  > >  >
> >  > >  > >  > I prefer former too, you can propose the documentation
> >  > > change required  for  > >  software PMD.
> >  > >  >
> >  > >  > Sure, proposal follows. The "typical workflow" isn't the most
> >  > > optimal by  having a conditional in the fast-path, of course, but it
> >  > > demonstrates the idea  simply.
> >  > >  >
> >  > >  > (line 204)
> >  > >  >  * An event driven based application has following typical
> >  > > workflow on
> >  > >  fastpath:
> >  > >  >  * \code{.c}
> >  > >  >  *      while (1) {
> >  > >  >  *
> >  > >  >  *              if (dev_info.event_dev_cap &
> >  > >  >  *                      RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED)
> >  > >  >  *                      rte_event_schedule(dev_id);
> >  > >
> >  > >  Yes, I like the idea of RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED.
> >  > >  It  can be input to application/subsystem to  launch separate
> >  > > core(s) for schedule functions.
> >  > >  But, I think, the "dev_info.event_dev_cap &
> >  > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED"
> >  > >  check can be moved inside the implementation(to make the better
> >  > > decisions  and  avoiding consuming cycles on HW based schedulers.
> >  >
> >  > How would this check work? Wouldn't it prevent any core from running the
> >  software scheduler in the centralized case?
> >  
> >  I guess you may not need RTE_EVENT_DEV_CAP here, instead need flag for
> >  device configure here
> >  
> >  #define RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED (1ULL << 1)
> >  
> >  struct rte_event_dev_config config;
> >  config.event_dev_cfg = RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED;
> >  rte_event_dev_configure(.., &config);
> >  
> >  on the driver side on configure,
> >  if (config.event_dev_cfg & RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED)
> >  	eventdev->schedule = NULL;
> >  else // centralized case
> >  	eventdev->schedule = your_centrized_schedule_function;
> >  
> >  Does that work?
> 
> Hm, I fear the API would give users the impression that they can select the scheduling behavior of a given eventdev, when a software scheduler is more likely to be either distributed or centralized -- not both.

Even if it is capability flag then also it is per "device". Right ?
capability flag is more of read only too. Am i missing something here?

> 
> What if we use the capability flag, and define rte_event_schedule() as the scheduling function for centralized schedulers and rte_event_dequeue() as the scheduling function for distributed schedulers? That way, the datapath could be the simple dequeue -> process -> enqueue. Applications would check the capability flag at configuration time to decide whether or not to launch an lcore that calls rte_event_schedule().

I am all for simple "dequeue -> process -> enqueue".
rte_event_schedule() added for SW scheduler only,  now it may not make
sense to add one more check on top of "rte_event_schedule()" to see
it is really need or not in fastpath?

> 
> >  
> >  >
> >  > >
> >  > >  >  *
> >  > >  >  *              rte_event_dequeue(...);
> >  > >  >  *
> >  > >  >  *              (event processing)
> >  > >  >  *
> >  > >  >  *              rte_event_enqueue(...);
> >  > >  >  *      }
> >  > >  >  * \endcode
> >  > >  >  *
> >  > >  >  * The *schedule* operation is intended to do event scheduling,
> >  > > and the  >  * *dequeue* operation returns the scheduled events. An
> >  > > implementation  >  * is free to define the semantics between
> >  > > *schedule* and *dequeue*. For  >  * example, a system based on a
> >  > > hardware scheduler can define its  >  * rte_event_schedule() to be
> >  > > an NOOP, whereas a software scheduler can  use  >  * the *schedule*
> >  > > operation to schedule events. The  >  *
> >  > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag indicates
> >  > > whether  >  * rte_event_schedule() should be called by all cores or
> >  > > by a single (typically  >  * dedicated) core.
> >  > >  >
> >  > >  > (line 308)
> >  > >  > #define RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED (1ULL < 2)  > /**<
> >  > > Event scheduling implementation is distributed and all cores must
> >  > > execute  >  *  rte_event_schedule(). If unset, the implementation is
> >  > > centralized and  >  *  a single core must execute the schedule
> >  > > operation.
> >  > >  >  *
> >  > >  >  *  \see rte_event_schedule()
> >  > >  >  */
> >  > >  >
> >  > >  > >  >
> >  > >  > >  > On same note, If software PMD based workflow need  a
> >  > > separate core(s)  for  > >  > schedule function then, Can we hide
> >  > > that from API specification and pass  an  > >  > argument to SW pmd
> >  > > to define the scheduling core(s)?
> >  > >  > >  >
> >  > >  > >  > Something like --vdev=eventsw0,schedule_cmask=0x2
> >  > >  >
> >  > >  > An API for controlling the scheduler coremask instead of (or
> >  > > perhaps in  addition to) the vdev argument would be good, to allow
> >  > > runtime control. I can  imagine apps that scale the number of cores
> >  > > based on load, and in doing so  may want to migrate the scheduler to a
> >  different core.
> >  > >
> >  > >  Yes, an API for number of scheduler core looks OK. But if we are
> >  > > going to  have service core approach then we just need to specify at
> >  > > one place as  application will not creating the service functions.
> >  > >
> >  > >  >
> >  > >  > >
> >  > >  > >  Just a thought,
> >  > >  > >
> >  > >  > >  Perhaps, We could introduce generic "service" cores concept to
> >  > > DPDK to  hide  > >  the  > >  requirement where the implementation
> >  > > needs dedicated core to do certain  > >  work. I guess it would
> >  > > useful for other NPU integration in DPDK.
> >  > >  > >
> >  > >  >
> >  > >  > That's an interesting idea. As you suggested in the other thread,
> >  > > this concept  could be extended to the "producer" code in the
> >  > > example for configurations  where the NIC requires software to feed
> >  > > into the eventdev. And to the other  subsystems mentioned in your original
> >  PDF, crypto and timer.
> >  > >
> >  > >  Yes. Producers should come in service core category. I think, that
> >  > > enables us to have better NPU integration.(same application code for
> >  > > NPU vs non NPU)
> >  > >

^ permalink raw reply

* Re: [PATCH 2/4] eventdev: implement the northbound APIs
From: Eads, Gage @ 2016-11-22 22:48 UTC (permalink / raw)
  To: Jerin Jacob
  Cc: dev@dpdk.org, Richardson, Bruce, Van Haaren, Harry,
	hemant.agrawal@nxp.com
In-Reply-To: <20161122200022.GA12168@svelivela-lt.caveonetworks.com>



>  -----Original Message-----
>  From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
>  Sent: Tuesday, November 22, 2016 2:00 PM
>  To: Eads, Gage <gage.eads@intel.com>
>  Cc: dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Van
>  Haaren, Harry <harry.van.haaren@intel.com>; hemant.agrawal@nxp.com
>  Subject: Re: [dpdk-dev] [PATCH 2/4] eventdev: implement the northbound APIs
>  
>  On Tue, Nov 22, 2016 at 07:43:03PM +0000, Eads, Gage wrote:
>  > >  > >  > > One open issue I noticed is the "typical workflow"
>  > > description starting in  > >  rte_eventdev.h:204 conflicts with the
>  > > centralized software PMD that Harry  > >  posted last week.
>  > > Specifically, that PMD expects a single core to call the  > >
>  > > schedule function. We could extend the documentation to account for
>  > > this  > >  alternative style of scheduler invocation, or discuss
>  > > ways to make the  software  > >  PMD work with the documented
>  > > workflow. I prefer the former, but either  way I  > >  think we
>  > > ought to expose the scheduler's expected usage to the user --
>  > > perhaps  > >  through an RTE_EVENT_DEV_CAP flag?
>  > >  > >  >
>  > >  > >  > I prefer former too, you can propose the documentation
>  > > change required  for  > >  software PMD.
>  > >  >
>  > >  > Sure, proposal follows. The "typical workflow" isn't the most
>  > > optimal by  having a conditional in the fast-path, of course, but it
>  > > demonstrates the idea  simply.
>  > >  >
>  > >  > (line 204)
>  > >  >  * An event driven based application has following typical
>  > > workflow on
>  > >  fastpath:
>  > >  >  * \code{.c}
>  > >  >  *      while (1) {
>  > >  >  *
>  > >  >  *              if (dev_info.event_dev_cap &
>  > >  >  *                      RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED)
>  > >  >  *                      rte_event_schedule(dev_id);
>  > >
>  > >  Yes, I like the idea of RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED.
>  > >  It  can be input to application/subsystem to  launch separate
>  > > core(s) for schedule functions.
>  > >  But, I think, the "dev_info.event_dev_cap &
>  > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED"
>  > >  check can be moved inside the implementation(to make the better
>  > > decisions  and  avoiding consuming cycles on HW based schedulers.
>  >
>  > How would this check work? Wouldn't it prevent any core from running the
>  software scheduler in the centralized case?
>  
>  I guess you may not need RTE_EVENT_DEV_CAP here, instead need flag for
>  device configure here
>  
>  #define RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED (1ULL << 1)
>  
>  struct rte_event_dev_config config;
>  config.event_dev_cfg = RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED;
>  rte_event_dev_configure(.., &config);
>  
>  on the driver side on configure,
>  if (config.event_dev_cfg & RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED)
>  	eventdev->schedule = NULL;
>  else // centralized case
>  	eventdev->schedule = your_centrized_schedule_function;
>  
>  Does that work?

Hm, I fear the API would give users the impression that they can select the scheduling behavior of a given eventdev, when a software scheduler is more likely to be either distributed or centralized -- not both.

What if we use the capability flag, and define rte_event_schedule() as the scheduling function for centralized schedulers and rte_event_dequeue() as the scheduling function for distributed schedulers? That way, the datapath could be the simple dequeue -> process -> enqueue. Applications would check the capability flag at configuration time to decide whether or not to launch an lcore that calls rte_event_schedule().

>  
>  >
>  > >
>  > >  >  *
>  > >  >  *              rte_event_dequeue(...);
>  > >  >  *
>  > >  >  *              (event processing)
>  > >  >  *
>  > >  >  *              rte_event_enqueue(...);
>  > >  >  *      }
>  > >  >  * \endcode
>  > >  >  *
>  > >  >  * The *schedule* operation is intended to do event scheduling,
>  > > and the  >  * *dequeue* operation returns the scheduled events. An
>  > > implementation  >  * is free to define the semantics between
>  > > *schedule* and *dequeue*. For  >  * example, a system based on a
>  > > hardware scheduler can define its  >  * rte_event_schedule() to be
>  > > an NOOP, whereas a software scheduler can  use  >  * the *schedule*
>  > > operation to schedule events. The  >  *
>  > > RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag indicates
>  > > whether  >  * rte_event_schedule() should be called by all cores or
>  > > by a single (typically  >  * dedicated) core.
>  > >  >
>  > >  > (line 308)
>  > >  > #define RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED (1ULL < 2)  > /**<
>  > > Event scheduling implementation is distributed and all cores must
>  > > execute  >  *  rte_event_schedule(). If unset, the implementation is
>  > > centralized and  >  *  a single core must execute the schedule
>  > > operation.
>  > >  >  *
>  > >  >  *  \see rte_event_schedule()
>  > >  >  */
>  > >  >
>  > >  > >  >
>  > >  > >  > On same note, If software PMD based workflow need  a
>  > > separate core(s)  for  > >  > schedule function then, Can we hide
>  > > that from API specification and pass  an  > >  > argument to SW pmd
>  > > to define the scheduling core(s)?
>  > >  > >  >
>  > >  > >  > Something like --vdev=eventsw0,schedule_cmask=0x2
>  > >  >
>  > >  > An API for controlling the scheduler coremask instead of (or
>  > > perhaps in  addition to) the vdev argument would be good, to allow
>  > > runtime control. I can  imagine apps that scale the number of cores
>  > > based on load, and in doing so  may want to migrate the scheduler to a
>  different core.
>  > >
>  > >  Yes, an API for number of scheduler core looks OK. But if we are
>  > > going to  have service core approach then we just need to specify at
>  > > one place as  application will not creating the service functions.
>  > >
>  > >  >
>  > >  > >
>  > >  > >  Just a thought,
>  > >  > >
>  > >  > >  Perhaps, We could introduce generic "service" cores concept to
>  > > DPDK to  hide  > >  the  > >  requirement where the implementation
>  > > needs dedicated core to do certain  > >  work. I guess it would
>  > > useful for other NPU integration in DPDK.
>  > >  > >
>  > >  >
>  > >  > That's an interesting idea. As you suggested in the other thread,
>  > > this concept  could be extended to the "producer" code in the
>  > > example for configurations  where the NIC requires software to feed
>  > > into the eventdev. And to the other  subsystems mentioned in your original
>  PDF, crypto and timer.
>  > >
>  > >  Yes. Producers should come in service core category. I think, that
>  > > enables us to have better NPU integration.(same application code for
>  > > NPU vs non NPU)
>  > >

^ permalink raw reply

* Re: Proposal for a new Committer model
From: Ferruh Yigit @ 2016-11-22 20:56 UTC (permalink / raw)
  To: Neil Horman, Thomas Monjalon; +Cc: dev, Mcnamara, John
In-Reply-To: <20161122195215.GA4463@hmsreliant.think-freely.org>

On 11/22/2016 7:52 PM, Neil Horman wrote:
> On Mon, Nov 21, 2016 at 09:52:41AM +0100, Thomas Monjalon wrote:
>> 2016-11-18 13:09, Neil Horman:
>>> A) Further promote subtree maintainership.  This was a conversation that I
>>> proposed some time ago, but my proposed granularity was discarded in favor
>>> of something that hasn't worked as well (in my opinion).  That is to say a
>>> few driver pmds (i40e and fm10k come to mind) have their own tree that
>>> send pull requests to Thomas.
>>
>> Yes we tried this fine granularity and stated that it was not working well.
>> We are now using the bigger granularity that you describe below.
>>
> Ok, thats good, but that must be _very_ new.  Looking at your git tree, I see no
> merge commits.  How are you pulling from those subtrees?

next-net tree is active for last three releases.

I guess following is the first commit to the sub-tree:
http://dpdk.org/ml/archives/dev/2016-February/032580.html

sub-trees rebase on top of main tree regularly, that is why there is no
merge commit.

> 
> 
>>> We should be sharding that at a much higher
>>> granularity and using it much more consistently.  That is to say, that we
>>> should have a maintainer for all the ethernet pmds, and another for the
>>> crypto pmds, another for the core eal layer, another for misc libraries
>>> that have low patch volumes, etc.
>>
>> Yes we could open a tree for EAL and another one for the core libraries.
>>
> That could be worthwhile.  Lets see how the net and crypto subtrees work out
> (assuming again that these trees are newly founded)
> 
> 
>>> Each of those subdivisions should have
>>> their own list to communicate on, and each should have a tree that
>>> integrates patches for their own subsystem, and they should on a regular
>>> cycle send pull requests to Thomas.
>>
>> Yes I think it is now a good idea to split the mailing list traffic,
>> at least for netdev and cryptodev.
>>
> Agreed, that serves two purposes, it lowers the volume for people with a
> specific interest (i.e. its a rudimentary filter), and it avoids confusion
> between you and the subtree maintainer (that is to say, you don't have to even
> consider pulling patches that go to the crypo and net lists, you just have to
> trust that they pull those patches in and send you appropriate pull requests).

I still find single mail list more useful.
Also with current process, after -rc2 release, patches directly merged
into main tree instead of sub-trees...

> 
>>> Thomas in turn should by and large,
>>> only be integrating pull requests.  This should address our high-
>>> throughput issue, in that it will allow multiple maintainers to share the
>>> workload, and integration should be relatively easy.
>>
>> Yes in an ideal organization, the last committer does only a last check
>> that technical plan and fairness are respected.
>> So it gives more time to coordinate the plans :)
>>
> Correct.  Thats never 100% accurate of course, some things will still have to
> come to you directly, simply by virtue of the fact that they don't completely
> fit anywhere else, but thats ok, the goal is really just to get your total patch
> volume lower, and replace it with pull requests that you can either trivialy
> mere or figure out with the help of the subtree maintainer.
> 
>>> B) Designate alternates to serve as backups for the maintainer when they
>>> are unavailable.  This provides high-availablility, and sounds very much
>>> like your proposal, but in the interests of clarity, there is still a
>>> single maintainer at any one time, it just may change to ensure the
>>> continued merging of patches, if the primary maintainer isn't available.
>>> Ideally however, those backup alternates arent needed, because most of the
>>> primary maintainers work in merging pull requests, which are done based on
>>> the trust of the submaintainer, and done during a very limited window of
>>> time.  This also partially addreses multi-vendor fairness if your subtree
>>> maintainers come from multiple participating companies.
>>
>> About the merge window, I do not have a strong opinion about how it can be
>> improved. However, I know that closing the window too early makes developer
>> unhappy because it makes wait - between development start and its release -
>> longer.
> 
> This is a fair point, but I'm not talking about closing it early here, all
> I'm suggesting is that, if you do proper pull requests from subtrees, your tree
> Thomas will only need a reasonably small window of time to accept new features,
> because you'll just merge the subtrees, rather than integrating individual
> patches.  E.g. you won't be constantly merging patches over the course of a
> development cycle, your tree's HEAD will mostly consist of merge commits as
> subtree maintainers send you pull requests, and ideally they will send those
> near the start of a window.  How long you keep your merge window open after that
> is up to you.
> 
> Neil
> 
> 
>  
>>

^ permalink raw reply

* Re: Adding API to force freeing consumed buffers in TX ring
From: Wiles, Keith @ 2016-11-22 20:41 UTC (permalink / raw)
  To: Richardson, Bruce; +Cc: Olivier Matz, Damjan Marion (damarion), dev@dpdk.org
In-Reply-To: <20161121152544.GA47052@bricha3-MOBL3.ger.corp.intel.com>


> On Nov 21, 2016, at 9:25 AM, Richardson, Bruce <bruce.richardson@intel.com> wrote:
> 
> On Mon, Nov 21, 2016 at 04:06:32PM +0100, Olivier Matz wrote:
>> Hi,
>> 
>> On 11/21/2016 03:33 PM, Wiles, Keith wrote:
>>> 
>>>> On Nov 21, 2016, at 4:48 AM, Damjan Marion (damarion) <damarion@cisco.com> wrote:
>>>> 
>>>> 
>>>> Hi,
>>>> 
>>>> Currently in VPP we do memcpy of whole packet when we need to do 
>>>> replication as we cannot know if specific buffer is transmitted
>>>> from tx ring before we update it again (i.e. l2 header rewrite).
>>>> 
>>>> Unless there is already a way to address this issue in DPDK which I’m not aware
>>>> of my proposal is that we provide mechanism for polling TX ring 
>>>> for consumed buffers. This can be either completely new API or 
>>>> extension of rte_etx_tx_burst (i.e. special case when nb_pkts=0).
>>>> 
>>>> This will allows us to start polling tx ring when we expect some 
>>>> mbuf back, instead of waiting for next tx burst (which we don’t know
>>>> when it will happen) and hoping that we will reach free_threshold soon.
>>> 
>>> +1
>>> 
>>> In Pktgen I have the problem of not being able to reclaim all of the TX mbufs to update them for the next set of packets to send. I know this is not a common case, but I do see the case where the application needs its mbufs freed off the TX ring. Currently you need to have at least a TX ring size of mbufs on hand to make sure you can send to a TX ring. If you allocate too few you run into a deadlock case as the number of mbufs  on a TX ring does not hit the flush mark. If you are sending to multiple TX rings on the same numa node from the a single TX pool you have to understand the total number of mbufs you need to have allocated to hit the TX flush on each ring. Not a clean way to handle the problems as you may have limited memory or require some logic to add more mbufs for dynamic ports.
>>> 
>>> Anyway it would be great to require a way to clean up the TX done ring, using nb_pkts == 0 is the simplest way, but a new API is fine too.
>>>> 
>>>> Any thoughts?
>> 
>> Yes, it looks useful to have a such API.
>> 
>> I would prefer another function instead of diverting the meaning of
>> nb_pkts. Maybe this?
>> 
>>  void rte_eth_tx_free_bufs(uint8_t port_id, uint16_t queue_id);
>> 
> 
> Third parameter for a limit(hint) of the number of bufs to free? If the
> TX ring is big, we might not want to stall other work for a long time
> while we free a huge number of buffers.

In order to move this along some, if we create the following API:

int rte_eth_tx_done_cleanup(uint8_t port_id, uint16_t queue_id, uint32_t free_cnt);

Return the number of freed mbufs or -1 if not supported or invalid params.
free_cnt of zero means free all possible mbufs or just at most the number suggested.
The free_cnt could be a uint16_t, but I do not think it matters much.

The rte_eth_tx_done_cleanup() call will return -1 if the PMD does not support or port_id, queue_id are invalid.

The default in the eth_dev structure of function pointers would be NULL(not supported) to not require all of the drivers to be updated today. We can then add the support as we go along.

We could have a features request API for tx_done support and PCTYPE, plus others if we want to go down that path too.

> 
> 	/Bruce

Regards,
Keith


^ permalink raw reply

* Re: [PATCH 2/4] eventdev: implement the northbound APIs
From: Jerin Jacob @ 2016-11-22 20:00 UTC (permalink / raw)
  To: Eads, Gage
  Cc: dev@dpdk.org, Richardson, Bruce, Van Haaren, Harry,
	hemant.agrawal@nxp.com
In-Reply-To: <9184057F7FC11744A2107296B6B8EB1E01E32F3E@FMSMSX108.amr.corp.intel.com>

On Tue, Nov 22, 2016 at 07:43:03PM +0000, Eads, Gage wrote:
> >  > >  > > One open issue I noticed is the "typical workflow" description starting in
> >  > >  rte_eventdev.h:204 conflicts with the centralized software PMD that Harry
> >  > >  posted last week. Specifically, that PMD expects a single core to call the
> >  > >  schedule function. We could extend the documentation to account for this
> >  > >  alternative style of scheduler invocation, or discuss ways to make the
> >  software
> >  > >  PMD work with the documented workflow. I prefer the former, but either
> >  way I
> >  > >  think we ought to expose the scheduler's expected usage to the user --
> >  perhaps
> >  > >  through an RTE_EVENT_DEV_CAP flag?
> >  > >  >
> >  > >  > I prefer former too, you can propose the documentation change required
> >  for
> >  > >  software PMD.
> >  >
> >  > Sure, proposal follows. The "typical workflow" isn't the most optimal by
> >  having a conditional in the fast-path, of course, but it demonstrates the idea
> >  simply.
> >  >
> >  > (line 204)
> >  >  * An event driven based application has following typical workflow on
> >  fastpath:
> >  >  * \code{.c}
> >  >  *      while (1) {
> >  >  *
> >  >  *              if (dev_info.event_dev_cap &
> >  >  *                      RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED)
> >  >  *                      rte_event_schedule(dev_id);
> >  
> >  Yes, I like the idea of RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED.
> >  It  can be input to application/subsystem to
> >  launch separate core(s) for schedule functions.
> >  But, I think, the "dev_info.event_dev_cap &
> >  RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED"
> >  check can be moved inside the implementation(to make the better decisions
> >  and
> >  avoiding consuming cycles on HW based schedulers.
> 
> How would this check work? Wouldn't it prevent any core from running the software scheduler in the centralized case?

I guess you may not need RTE_EVENT_DEV_CAP here, instead need flag for device
configure here

#define RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED (1ULL << 1)

struct rte_event_dev_config config;
config.event_dev_cfg = RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED;
rte_event_dev_configure(.., &config);

on the driver side on configure,
if (config.event_dev_cfg & RTE_EVENT_DEV_CFG_DISTRIBUTED_SCHED)
	eventdev->schedule = NULL;
else // centralized case
	eventdev->schedule = your_centrized_schedule_function;

Does that work?

> 
> >  
> >  >  *
> >  >  *              rte_event_dequeue(...);
> >  >  *
> >  >  *              (event processing)
> >  >  *
> >  >  *              rte_event_enqueue(...);
> >  >  *      }
> >  >  * \endcode
> >  >  *
> >  >  * The *schedule* operation is intended to do event scheduling, and the
> >  >  * *dequeue* operation returns the scheduled events. An implementation
> >  >  * is free to define the semantics between *schedule* and *dequeue*. For
> >  >  * example, a system based on a hardware scheduler can define its
> >  >  * rte_event_schedule() to be an NOOP, whereas a software scheduler can
> >  use
> >  >  * the *schedule* operation to schedule events. The
> >  >  * RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag indicates
> >  whether
> >  >  * rte_event_schedule() should be called by all cores or by a single (typically
> >  >  * dedicated) core.
> >  >
> >  > (line 308)
> >  > #define RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED (1ULL < 2)
> >  > /**< Event scheduling implementation is distributed and all cores must
> >  execute
> >  >  *  rte_event_schedule(). If unset, the implementation is centralized and
> >  >  *  a single core must execute the schedule operation.
> >  >  *
> >  >  *  \see rte_event_schedule()
> >  >  */
> >  >
> >  > >  >
> >  > >  > On same note, If software PMD based workflow need  a separate core(s)
> >  for
> >  > >  > schedule function then, Can we hide that from API specification and pass
> >  an
> >  > >  > argument to SW pmd to define the scheduling core(s)?
> >  > >  >
> >  > >  > Something like --vdev=eventsw0,schedule_cmask=0x2
> >  >
> >  > An API for controlling the scheduler coremask instead of (or perhaps in
> >  addition to) the vdev argument would be good, to allow runtime control. I can
> >  imagine apps that scale the number of cores based on load, and in doing so
> >  may want to migrate the scheduler to a different core.
> >  
> >  Yes, an API for number of scheduler core looks OK. But if we are going to
> >  have service core approach then we just need to specify at one place as
> >  application will not creating the service functions.
> >  
> >  >
> >  > >
> >  > >  Just a thought,
> >  > >
> >  > >  Perhaps, We could introduce generic "service" cores concept to DPDK to
> >  hide
> >  > >  the
> >  > >  requirement where the implementation needs dedicated core to do certain
> >  > >  work. I guess it would useful for other NPU integration in DPDK.
> >  > >
> >  >
> >  > That's an interesting idea. As you suggested in the other thread, this concept
> >  could be extended to the "producer" code in the example for configurations
> >  where the NIC requires software to feed into the eventdev. And to the other
> >  subsystems mentioned in your original PDF, crypto and timer.
> >  
> >  Yes. Producers should come in service core category. I think, that
> >  enables us to have better NPU integration.(same application code for
> >  NPU vs non NPU)
> >  

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox