Re: Beyond DPDK 2.0

[Avi Kivity <avi-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org>]

On 05/07/2015 06:27 PM, Wiles, Keith wrote:
>
> On 5/7/15, 7:02 AM, "Avi Kivity" <avi-RmZWMc9puTNJc61us3aD9laTQe2KTcn/@public.gmane.org> wrote:
>
>> On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
>> <tim.o'driscoll-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> wrote:
>>
>>> Does anybody have any input or comments on this?
>>>
>>>
>>>> -----Original Message-----
>>>> From: O'Driscoll, Tim
>>>> Sent: Thursday, April 16, 2015 11:39 AM
>>>> To: dev-VfR2kkLFssw@public.gmane.org
>>>> Subject: Beyond DPDK 2.0
>>>>
>>>> Following the launch of DPDK by Intel as an internal development
>>>> project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK
>>> RPM
>>>> packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
>>>> prepare for future releases after DPDK 2.0 by starting a discussion on
>>>> its evolution. Anyone is welcome to join this initiative.
>>>>
>>>> Since then, the project has grown significantly:
>>>> -    The number of commits and mailing list posts has increased
>>>> steadily.
>>>> -    Support has been added for a wide range of new NICs (Mellanox
>>>> support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
>>>> -    DPDK is now supported on multiple architectures (IBM Power
>>> support
>>>> in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
>>>> applied).
>>>>
>>>> While this is great progress, we need to make sure that the project is
>>>> structured in a way that enables it to continue to grow. To achieve
>>>> this, 6WIND, Red Hat and Intel would like to start a discussion about
>>>> the future of the project, so that we can agree and establish
>>> processes
>>>> that satisfy the needs of the current and future DPDK community.
>>>>
>>>> We're very interested in hearing the views of everybody in the
>>>> community. In addition to debate on the mailing list, we'll also
>>>> schedule community calls to discuss this.
>>>>
>>>>
>>>> Project Goals
>>>> -------------
>>>>
>>>> Some topics to be considered for the DPDK project include:
>>>> -    Project Charter: The charter of the DPDK project should be
>>> clearly
>>>> defined, and should explain the limits of DPDK (what it does and does
>>>> not cover). This does not mean that we would be stuck with a singular
>>>> charter for all time, but the direction and intent of the project
>>> should
>>>> be well understood.
>>
>> One problem we've seen with dpdk is that it is a framework, not a library:
>> it wants to create threads, manage memory, and generally take over.  This
>> is a problem for us, as we are writing a framework (seastar, [1]) and need
>> to create threads, manage memory, and generally take over ourselves.
>>
>> Perhaps dpdk can be split into two layers, a library layer that only
>> provides mechanisms, and a framework layer that glues together those
>> mechanisms and applies a policy, trading in generality for ease of use.
> The DPDK system is somewhat divided now between the EAL, PMDS and utility
> functions like malloc/rings/Š
>
> The problem I see is the PMDs need a framework to be usable and the EAL
> plus the ethdev layers provide that support today. Setting up and
> initializing the DPDK system is pretty clean just call the EAL init
> routines along with the pool creates and the basic configs for the
> PMDs/hardware. Once the system is inited one can create new threads and
> not requiring anyone to use DPDK launch routines. Maybe I am not
> understanding your needs can you explain more?

An initialization routine that accepts argc/argv can hardly be called clean.

In seastar, we have our own malloc() (since seastar is sharded we can 
provide a faster thread-unsafe malloc implementation).  We also have our 
own threading, and since dpdk is an optional component in seastar, dpdk 
support requires code duplication.

I would like to launch my own threads, pin them where I like, and call 
PMD drivers to send and receive packets.  Practically everything else 
that dpdk does gets in my way, including mbuf pools.  I'd much prefer to 
allocate mbufs myself.


>> [1] http://seastar-project.org