Re: [dpdk-dev] [RFC PATCH 0/5] graph: introduce graph subsystem

[Thomas Monjalon <thomas@monjalon.net>]

22/02/2020 10:05, Jerin Jacob:
> On Fri, Feb 21, 2020 at 9:44 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > 21/02/2020 16:56, Jerin Jacob:
> > > On Fri, Feb 21, 2020 at 4:40 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > 21/02/2020 11:30, Jerin Jacob:
> > > > > On Mon, Feb 17, 2020 at 4:28 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > > > > > On Mon, Feb 17, 2020 at 2:08 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > > > > If we add rte_graph to DPDK, we will have 2 similar libraries.
> > > > > > >
> > > > > > > I already proposed several times to move rte_pipeline in a separate
> > > > > > > repository for two reasons:
> > > > > > >         1/ it is acting at a higher API layer level
> > > > > >
> > > > > > We need to define what is the higher layer API. Is it processing beyond L2?
> > > >
> > > > My opinion is that any API which is implemented differently
> > > > for different hardware should be in DPDK.
> > >
> > > We need to define SIMD optimization(not HW specific to  but
> > > architecture-specific)
> > > treatment as well, as the graph and node library will have SIMD
> > > optimization as well.
> >
> > I think SIMD optimization is generic to any performance-related project,
> > not specific to DPDK.
> >
> >
> > > In general, by the above policy enforced, we need to split DPDK like below,
> > > dpdk.git
> > > ----------
> > > librte_compressdev
> > > librte_bbdev
> > > librte_eventdev
> > > librte_pci
> > > librte_rawdev
> > > librte_eal
> > > librte_security
> > > librte_mempool
> > > librte_mbuf
> > > librte_cryptodev
> > > librte_ethdev
> > >
> > > other repo(s).
> > > ----------------
> > > librte_cmdline
> > > librte_cfgfile
> > > librte_bitratestats
> > > librte_efd
> > > librte_latencystats
> > > librte_kvargs
> > > librte_jobstats
> > > librte_gso
> > > librte_gro
> > > librte_flow_classify
> > > librte_pipeline
> > > librte_net
> > > librte_metrics
> > > librte_meter
> > > librte_member
> > > librte_table
> > > librte_stack
> > > librte_sched
> > > librte_rib
> > > librte_reorder
> > > librte_rcu
> > > librte_power
> > > librte_distributor
> > > librte_bpf
> > > librte_ip_frag
> > > librte_hash
> > > librte_fib
> > > librte_timer
> > > librte_telemetry
> > > librte_port
> > > librte_pdump
> > > librte_kni
> > > librte_acl
> > > librte_vhost
> > > librte_ring
> > > librte_lpm
> > > librte_ipsec
> >
> > I think it is a fair conclusion of the scope I am arguing, yes.
> 
> OK. See below.
> 
> > > > What is expected to be maintained, tested, etc.
> > >
> > > We need to maintain and test other code in OTHER dpdk repo as well.
> >
> > Yes but the ones responsible are not the same.
> 
> I see your point. Can I interpret it as you would like to NOT take
> responsibility
> of  SW libraries(Items enumerated in the second list)?

It's not only about me. This is a community decision.


> I think, the main question would be, how it will deliver to distros
> and/or end-users
> and what will be part of the dpdk release?
> 
> I can think of two options. Maybe distro folks have better view on this.
> 
> options 1:
> - Split dpdk to dpdk-core.git, dpdk-algo.git etc based on the
> functionalities and maintainer's availability.
> - Follow existing release cadence and deliver single release tarball
> with content from the above repos.
> 
> options 2:
> - Introduce more subtrees(dpdk-next-algo.git etc) based on the
> functionalities and maintainer's availability.
> - Follow existing release cadence and have a pull request to main
> dpdk.git just like Linux kernel or existing scheme of things.
> 
> I am for option 2.
> 
> NOTE: This new graph and node library, I would like to make its new
> subtree in the existing scheme of
> things so that it will NOT be a burden for you to manage.

The option 2 is to make maintainers life easier.
Keeping all libraries in the same repository allows to have
an unique release and a central place for the apps and docs.

The option 1 may make contributors life easier if we consider
adding new libraries can make contributions harder in case of dependencies.
The option 1 makes also repositories smaller, so maybe easier to approach.
It makes easier to fully validate testing and quality of a repository.
Having separate packages makes easier to select what is distributed and supported.

After years thinking about the scope of DPDK repository,
I am still not sure which solution is best.
I really would like to see more opinions, thanks.