Re: [dpdk-dev] time taken for allocation of mempool.

[Olivier Matz <olivier.matz@6wind.com>]

Hi Venu,

On Wed, Nov 13, 2019 at 02:41:04PM +0530, Venumadhav Josyula wrote:
> Hi Oliver,
> 
> 
> 
> *> Could you give some more details about you use case? (hugepage size,
> number of objects, object size, additional mempool flags, ...)*
> 
> Ours in telecom product, we support multiple rats. Let us take example of
> 4G case where we act as an gtpu proxy.
> 
> ·        Hugepage size :- 2 Mb
> 
> ·        *rte_mempool_create in param*
> 
> o    { name=”gtpu-mem”,
> 
> o   n=1500000,
> 
> o   elt_size=224,
> 
> o   cache_size=0,
> 
> o   private_data_size=0,
> 
> o   mp_init=NULL,
> 
> o   mp_init_arg=NULL,
> 
> o   obj_init=NULL,
> 
> o   obj_init_arg=NULL,
> 
> o   socket_id=rte_socket_id(),
> 
> o   flags=MEMPOOL_F_SP_PUT }
> 

OK, that's quite big mempools (~300MB) but I don't think it should
take that much time.

I suspect that using 1G hugepages could help, in case it is related
to the memory allocator.

> *> Did you manage to reproduce it in a small test example? We could do some
> profiling to investigate.*
> 
> No I would love to try that ? Are there examples ?

The simplest way for me is to hack the unit tests. Add this code (not
tested) at the beginning of test_mempool.c:test_mempool():

	int i;
	for (i = 0; i < 100; i++) {
		struct rte_mempool *mp;

		mp = rte_mempool_create("test", 1500000,
					224, 0, 0, NULL, NULL,
					NULL, NULL, SOCKET_ID_ANY,
					MEMPOOL_F_SP_PUT);
		if (mp == NULL) {
			printf("rte_mempool_create() failed\n");
			return -1;
		}
		rte_mempool_free(mp);
	}
	return 0;

Then, you can launch the test application and run you test with
"mempool_autotest". I suggest to compile with EXTRA_CFLAGS="-g", so you
can run "perf top" (https://perf.wiki.kernel.org/index.php/Main_Page) to
see where you spend the time.  By using "perf record" / "perf report"
with options, you can also analyze the call stack.

Please share your results, especially comparison between 17.05 and 18.11.

Thanks,
Olivier


> 
> 
> 
> Thanks,
> 
> Regards,
> 
> Venu
> 
> On Wed, 13 Nov 2019 at 14:02, Olivier Matz <olivier.matz@6wind.com> wrote:
> 
> > Hi Venu,
> >
> > On Wed, Nov 13, 2019 at 10:42:07AM +0530, Venumadhav Josyula wrote:
> > > Hi,
> > >
> > > Few more points
> > >
> > > Operating system  : Centos 7.6
> > > Logging mechanism : syslog
> > >
> > > We have logged using syslog before the call and syslog after the call.
> > >
> > > Thanks & Regards
> > > Venu
> > >
> > > On Wed, 13 Nov 2019 at 10:37, Venumadhav Josyula <vjosyula@gmail.com>
> > wrote:
> > >
> > > > Hi ,
> > > > We are using 'rte_mempool_create' for allocation of flow memory. This
> > has
> > > > been there for a while. We just migrated to dpdk-18.11 from
> > dpdk-17.05. Now
> > > > here is problem statement
> > > >
> > > > Problem statement :
> > > > In new dpdk ( 18.11 ), the 'rte_mempool_create' take approximately ~4.4
> > > > sec for allocation compared to older dpdk (17.05). We have som 8-9
> > mempools
> > > > for our entire product. We do upfront allocation for all of them ( i.e.
> > > > when dpdk application is coming up). Our application is run to
> > completion
> > > > model.
> > > >
> > > > Questions:-
> > > > i)  is that acceptable / has anybody seen such a thing ?
> > > > ii) What has changed between two dpdk versions ( 18.11 v/s 17.05 ) from
> > > > memory perspective ?
> >
> > Could you give some more details about you use case? (hugepage size, number
> > of objects, object size, additional mempool flags, ...)
> >
> > Did you manage to reproduce it in a small test example? We could do some
> > profiling to investigate.
> >
> > Thanks for the feedback.
> > Olivier
> >