From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Hanoch Haim (hhaim)" Subject: Re: [PATCH v2] mbuf: optimize rte_mbuf_refcnt_update Date: Mon, 4 Jan 2016 14:43:42 +0000 Message-ID: <7f5255b98dcb4f2396ada16d5eb43e5a@XCH-RTP-017.cisco.com> References: <568A7959.7030506@6wind.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Cc: "dev@dpdk.org" , "Ido Barnea \(ibarnea\)" , "Itay Marom \(imarom\)" To: Olivier MATZ , "bruce.richardson@intel.com" Return-path: Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by dpdk.org (Postfix) with ESMTP id 7CDD69192 for ; Mon, 4 Jan 2016 15:43:45 +0100 (CET) In-Reply-To: <568A7959.7030506@6wind.com> Content-Language: en-US List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Oliver,=20 Let's take your drawing as a reference and add my question The use case is sending a duplicate multicast packet by many threads. I can split it to x threads to do the job and with atomic-ref (my multicast= not mbuf) count it until it reaches zero. In my following example the two cores (0 and 1) sending the indirect m1/m2 = do alloc/attach/send=20 core0 | core1 --------------------------------- |----------------= ----------------------- m_const=3Drte_pktmbuf_alloc(mp) | | while true: | while True: m1 =3Drte_pktmbuf_alloc(mp_64) | m2 =3Drte_pktmbuf_alloc(m= p_64) rte_pktmbuf_attach(m1, m_const) | rte_pktmbuf_attach(m1, m_con= st) tx_burst(m1) | tx_burst(m2) Is this example is not valid?=20 BTW this is our workaround=20 =20 core0 | core1 --------------------------------- |-----------------------= ---------------- m_const=3Drte_pktmbuf_alloc(mp) | rte_mbuf_refcnt_update(m_const,1)| <<-- workaround=20 | while true: | while True: m1 =3Drte_pktmbuf_alloc(mp_64) | m2 =3Drte_pktmbuf_alloc(mp_64) rte_pktmbuf_attach(m1, m_const) | rte_pktmbuf_attach(m1, m_const) tx_burst(m1) | tx_burst(m2) thanks, Hanoh -----Original Message----- From: Olivier MATZ [mailto:olivier.matz@6wind.com]=20 Sent: Monday, January 04, 2016 3:53 PM To: Hanoch Haim (hhaim); bruce.richardson@intel.com Cc: dev@dpdk.org; Ido Barnea (ibarnea); Itay Marom (imarom) Subject: Re: [dpdk-dev] [PATCH v2] mbuf: optimize rte_mbuf_refcnt_update Hi Hanoch, Please find some comments below. On 12/27/2015 10:39 AM, Hanoch Haim (hhaim) wrote: > Hi Bruce, >=20 > I'm Hanoch from Cisco Systems works for the https://github.com/cisco-sys= tem-traffic-generator/trex-core traffic generator project. >=20 > While upgrading from DPDK 1.8 to 2.2 Ido found that the following=20 > commit creates a mbuf corruption and result in Tx hang >=20 >=20 >=20 > commit f20b50b946da9070d21e392e4dbc7d9f68bc983e >=20 > Author: Olivier Matz >=20 > Date: Mon Jun 8 16:57:22 2015 +0200 >=20 >=20 >=20 > Looking at the change it is clear why there is an issue, wanted to get yo= ur input. >=20 >=20 >=20 > Init >=20 > ----- >=20 > alloc const mbuf =3D=3D> mbuf-a (ref=3D1) >=20 >=20 >=20 > Simple case that works >=20 > --------------------- >=20 >=20 >=20 > thread 1 , tx: alloc-mbuf->attach(mbuf-a) (ref=3D2) inc- non atomic >=20 > thread 1 , tx: alloc-mbuf->attach(mbuf-a) (ref32) inc- atomic do you mean "(ref=3D3)" ? [hh] yes ref=3D3.=20 >=20 > thread 1 , drv : free() (ref=3D2) dec- atomic >=20 > thread 1 , drv : free() (ref=3D3) dec - non atomic do you mean "(ref=3D1)" ? >=20 > Simple case that does not work >=20 > --------------------- >=20 >=20 >=20 > Both do that in parallel >=20 >=20 >=20 > thread 2 tx : alloc-mbuf->attach(mbuf-a) (ref=3D2) inc- non atomic >=20 > thread 1 tx : alloc-mbuf->attach(mbuf-a) (ref=3D2) inc- non atomic It is not allowed to call a function from the mbuf API in parallel. Example: core0 | core1 --------------------------------|--------------------------------------- m =3D rte_pktmbuf_alloc(m); | enqueue(m); | |m =3D dequeue(); do_something(m); |do_something(m); do_something() is not allowed because it accesses the same mbuf structure. do_something() can be any function of mbuf API: rte_pktmbuf_prepend(), rte_= pktmbuf_attach(), ... This is allowed: core0 | core1 --------------------------------|--------------------------------------- m =3D rte_pktmbuf_alloc(m); | m2 =3D rte_pktmbuf_attach(m); | enqueue(m2); | |m2 =3D dequeue(); do_something(m); |do_something(m2); Regards, Olivier