Huge ring allocation

Huge ring allocation

[Gregory Etelson]

Hello,

I have a DPDK process that creates pktmbuf pool with 2_000_000 objects
In CentOS 6.x x86-64, ring size of this pool is 16MB
8 2MB hugepages required to hold such ring.
In some cases, hugepages are too fragmented and there are no 8 contiguous hugepages for the ring.
As the result, the process has enough hugepages memory, but ring allocation fails.
I use a workaround for these cases and create a ring with standard OS allocation routines.
Is there another way I could use unclaimed dpdk hugepages for that allocation ?

Regards,
Gregory
\0

Re: Huge ring allocation

[Wiles, Keith]

Regards,
Keith

> On Aug 25, 2016, at 1:05 AM, Gregory Etelson <gregory@weka.io> wrote:
> 
> Hello,
> 
> I have a DPDK process that creates pktmbuf pool with 2_000_000 objects
> In CentOS 6.x x86-64, ring size of this pool is 16MB
> 8 2MB hugepages required to hold such ring.

Have you try to use 1G huge pages, is that an option?

> In some cases, hugepages are too fragmented and there are no 8 contiguous hugepages for the ring.
> As the result, the process has enough hugepages memory, but ring allocation fails.

Another issue sometimes is huge pages are allocated after the system has booted, which means you need to assign the number of huge pages very early in the boot process. This means adding a line in the sysctrl.conf file instead of poking the value later.

vm.nr_hugepages=XXX

> I use a workaround for these cases and create a ring with standard OS allocation routines.
> Is there another way I could use unclaimed dpdk hugepages for that allocation ?
> 
> Regards,
> Gregory

Re: Huge ring allocation

[anupam]

>>>>> [2016-08-25T17:52:26+0530]: "Keith Wiles" (keith-wiles):

,----[ keith-wiles ]
| Another issue sometimes is huge pages are allocated after the system
| has booted, which means you need to assign the number of huge pages very
| early in the boot process. This means adding a line in the sysctrl.conf
| file instead of poking the value later.
|
| vm.nr_hugepages=XXX
`----
you might also try passing 'hugepages' option as kernel command line
parameters e.g.

           '... hugepages=N '

from linux-kernel's Documentation/vm/hugetlbpage.txt we have the
following

,----[Documentation/vm/hugetlbpage.txt:69]
| The administrator can allocate persistent huge pages on the kernel boot
| command line by specifying the "hugepages=N" parameter, where 'N' = the
| number of huge pages requested.  This is the most reliable method of
| allocating huge pages as memory has not yet become fragmented.
`----


---
thanks
anupam