From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thomas Monjalon Subject: Re: DPDK and forked processes Date: Fri, 27 Jul 2018 17:03:48 +0200 Message-ID: <7441138.EijKcuSYIv@xps> References: <9184057F7FC11744A2107296B6B8EB1E446E9A79@FMSMSX108.amr.corp.intel.com> <9184057F7FC11744A2107296B6B8EB1E446F214F@FMSMSX108.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Cc: "Burakov, Anatoly" , "dev@dpdk.org" , "Richardson, Bruce" , "jerin.jacob@caviumnetworks.com" , "Yigit, Ferruh" , "hemant.agrawal@nxp.com" , "Ananyev, Konstantin" , Olivier Matz , "stephen@networkplumber.org" To: "Eads, Gage" Return-path: Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id D9DC325D9 for ; Fri, 27 Jul 2018 17:03:55 +0200 (CEST) In-Reply-To: <9184057F7FC11744A2107296B6B8EB1E446F214F@FMSMSX108.amr.corp.intel.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 27/07/2018 15:46, Eads, Gage: > As this discussion has broad implications for DPDK, is it a good candidat= e for a techboard meeting topic?=20 We can discuss it in techboard, but usually we prefer discussing topics whose resolution is not clear. In this case, I think everybody agree with Anatoly, isn't it? > > -----Original Message----- > > From: Burakov, Anatoly > > Sent: Monday, July 16, 2018 10:09 AM > > To: Eads, Gage ; dev@dpdk.org > > Subject: Re: DPDK and forked processes > >=20 > > On 16-Jul-18 4:00 PM, Eads, Gage wrote: > > > Hi all, > > > > > > Does DPDK support forking secondary processes after executing > > > rte_eal_init()? The l2fwd_fork example and at least one application > > > (OpenEM: https://sourceforge.net/projects/eventmachine/) use this > > > model, and they do so by fixing up the EAL internals (e.g. manually > > > changing process_type from primary to secondary) at the start of the > > > child process. This feels like a hack, and I can=E2=80=99t find any > > > documentation describing this model. > > > > > > Moreover, this approach doesn=E2=80=99t appear to be compatible with = recent > > > EAL changes. For instance, the multi-process communication creates a > > > couple handler threads (=E2=80=9Crte_mp_handle=E2=80=9D and =E2=80=9C= rte_mp_async=E2=80=9D) during EAL > > > initialization. The child processes won=E2=80=99t inherit these threa= ds, and > > > so won=E2=80=99t be able to participate in multi-process comms. This = means the > > > reworked memory subsystem and upcoming device hotplug support > > > (http://mails.dpdk.org/archives/dev/2018-July/107704.html) won=E2=80= =99t work > > > with this fork-after-init model. > > > > > > This is just one example =E2=80=93 there may be other features/subsys= tems that > > > won=E2=80=99t work. As far as I can tell there is no official stance = (though > > > the l2fwd_fork example implies it=E2=80=99s supported, IMO); I think = either > > > DPDK should either drop the example and not support this model, or > > > support it and either document its limitations or resolve them. This > > > model could be an interesting way to run multi-process DPDK on an > > > ASLR-enabled system, but supporting this wouldn=E2=80=99t be trivial. > > > > > > Thanks, > > > > > > Gage > > > > >=20 > > I think it's a very bad idea to use such a model in recent versions of = DPDK. As you > > have correctly pointed out, IPC will not work in such a scenario, and g= iven how > > our memory subsystem relies on IPC, this is a recipe for memory corrupt= ion and > > divergent memory maps (since technically both initial and forked proces= ses > > believe they are primary). > >=20 > > Even hacking rte_config to make DPDK think it's a secondary process wil= l not > > work, because the initialization has already completed, but all of the = threads > > (IPC, interrupt, etc.) are gone and correct IPC socket was not created,= which > > means the process becomes invisible to the primary for all intents and = purposes. > >=20 > > We _could_ introduce some kind of "official DPDK fork" function that wo= uld fork > > the process and then restart interrupt, IPC etc. stuff on an already ru= nning > > instance of DPDK, but that seems like a workaround for a problem that s= houldn't > > exist in the first place, because such usage is fundamentally incompati= ble with > > DPDK as it stands now. > >=20 > > -- > > Thanks, > > Anatoly >=20