From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?iso-8859-1?Q?Fran=E7ois-Fr=E9d=E9ric_Ozog?= Subject: Re: Decoupling DPDK from EAL Date: Wed, 4 Dec 2013 14:37:28 +0100 Message-ID: <03b101cef0f5$faa2cc70$efe86550$@com> References: <035f01cef0ca$a0d7a470$e286ed50$@com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Cc: dev-VfR2kkLFssw@public.gmane.org To: "'Jason Vassbender'" Return-path: In-Reply-To: Content-Language: fr List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces-VfR2kkLFssw@public.gmane.org Sender: "dev" Hi, I think I get the picture. DPDK is not really flexible at memory = allocation (nor the Linux kernel which requires boot parameters for 1GB huge = pages)... Let's assume that "static" memory configuration is acceptable.=20 Is the thread model integration issue related to the fact we set = affinity ATFER thread creation? (I agree this is complex, but you can still = create an adaptation layer in your thread API to accommodate DPDK thread model). Fran=E7ois-Fr=E9d=E9ric > -----Message d'origine----- > De=A0: Jason Vassbender [mailto:jason.vassbender-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org] > Envoy=E9=A0: mercredi 4 d=E9cembre 2013 10:25 > =C0=A0: Fran=E7ois-Fr=E9d=E9ric Ozog > Cc=A0: dev-VfR2kkLFssw@public.gmane.org > Objet=A0: Re: [dpdk-dev] Decoupling DPDK from EAL >=20 > Hey, >=20 > I guess the main hurdle is that we already have our own multi-threaded > architecture and ways to control thread startup/shutdown, priorities = and > affinities and they are all balanced very delicately (our application = is > latency sensitive, runs on rt_preempt, boots with isolcpus, etc). In > addition, we are already using the command line to initialize some of = our > things, and part of the configuration for the application does not = even > come from the command line, but from eg. > XML configuration file over the network. So ideally what I would have > preferred is that EAL initialization could be done by other means (for > example a simple initialization function with a dictionary as to be = more > flexible) and thread creation/shutdown could be left to the = application if > it so desires, provided it meets the execution conditions expected by DPDK. >=20 > Essentially, at its current state, DPDK offers a complete solution to = your > problem including the entire surrounding framework. But for most big > applications they already have their own frameworks in place and > integrating DPDK becomes harder than it should be. So if DPDK were to = be > decoupled from EAL, made more modular, and some of the functions optionally > left to applications to provide if they already have the facilities = for > them would make integration much easier and more flexible. >=20 > -Jason >=20 > On Wed, Dec 4, 2013 at 10:27 AM, Fran=E7ois-Fr=E9d=E9ric Ozog = > wrote: > > Hi, > > > > I just completed such a consulting mission for a customer. They were > > using libpcap as the network back end and the most challenging = hurdle > > was to transform a single threaded capture architecture to a > > multi-threaded one with DPDK. The other key take away, is that DPDK > > capture helps to get only 20% of the 20 times performance boost I > > managed to achieve: most of the latency is due to "application" and other > internal communication mechanisms. > > > > So I agree that DPDK is not light, but I think most of the power of > > DPDK comes from EAL thread management and "IPC"... > > > > Having said all that, I may have missed a critical point, so, what = is > > the specific major hurdle you see in the integration? > > > > Fran=E7ois-Fr=E9d=E9ric > > > > > >> -----Message d'origine----- > >> De : dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] De la part de Jason = Vassbender > >> Envoy=E9 : mardi 3 d=E9cembre 2013 22:51 =C0 : dev-VfR2kkLFssw@public.gmane.org Objet : > >> [dpdk-dev] Decoupling DPDK from EAL > >> > >> Hello, > >> > >> I am trying to integrate DPDK into an existing application in order > >> to improve packet processing latency, but it is proving rather > >> difficult because of DPDK's dependency on EAL's thread management = and > >> bootstrap mechanism. Our application already has its own framework > >> for managing threads and their affinities/priorities, IPC, timers = and > >> its own bootstrap mechanism (not necessarily via command line > >> arguments), we wish to integrate DPDK as an alternative network > >> back-end, but it wants to to take over our entire way of doing = things. > >> > >> Are there any plans to decouple DPDK's core functionality away from > >> EAL so that it can be more easily integrated into existing applications? > >> > >> -Jason > >