From mboxrd@z Thu Jan  1 00:00:00 1970
From: Bruce Richardson <bruce.richardson@intel.com>
Subject: Re: dpdk multi process increase the number of mbufs,
 throughput gets dropped
Date: Fri, 18 Dec 2015 12:07:41 +0000
Message-ID: <20151218120740.GA11116@bricha3-MOBL3>
References: <44e6e2b3.65ad.151ae293aa7.Coremail.zhangwqh@126.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Cc: dev@dpdk.org
To: =?utf-8?B?5byg5Lyf?= <zhangwqh@126.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga02.intel.com (mga02.intel.com [134.134.136.20])
 by dpdk.org (Postfix) with ESMTP id 9AB123B5
 for <dev@dpdk.org>; Fri, 18 Dec 2015 13:07:45 +0100 (CET)
Content-Disposition: inline
In-Reply-To: <44e6e2b3.65ad.151ae293aa7.Coremail.zhangwqh@126.com>
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

On Thu, Dec 17, 2015 at 12:18:36PM +0800, =E5=BC=A0=E4=BC=9F wrote:
> Hi all,=20
>=20
>=20
> When running the multi process example, does anybody know that why incr=
easing the number of mbufs, the performance gets dropped.=20
>=20
>=20
> In multi process example, there are two macros which are related to the=
 number of mbufs
>=20
>=20
> #defineMBUFS_PER_CLIENT1536
> |
> | #defineMBUFS_PER_PORT1536 |
> | |
>=20
>=20
> If increasing these two numbers by 8 times, the performance drops about=
 10%. Does anybody know why?
>=20
> | constunsigned num_mbufs =3D (num_clients * MBUFS_PER_CLIENT) \ |
> | | + (ports->num_ports * MBUFS_PER_PORT); |
> | pktmbuf_pool =3D rte_mempool_create(PKTMBUF_POOL_NAME, num_mbufs, |
> | | MBUF_SIZE, MBUF_CACHE_SIZE, |
> | | sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, |
> | | NULL, rte_pktmbuf_init, NULL, rte_socket_id(), NO_FLAGS ); |

One possible explanation could be due to the memory footprint of the memo=
ry=20
pool. While the per-lcore mempool caches of buffers operate in a LIFO (i.=
e. stack)
manner, when mbufs are allocated on one core and freed on another, they p=
ass
through a FIFO (i.e. ring) inside the mempool. This means that you iterat=
e
through all buffers in the pool in this case, which can cause a slowdown =
if the
mempool size is bigger than your cache.

/Bruce