Re: Question on huge pages and running DPDK apps inside containers

[Anjali Kulkarni <anjali-3r7Miqu9kMnR7s880joybQ@public.gmane.org>]

Ignore the previous email, it was too long. Pls see this one only:

Thanks - I got a confirmation from some Intel folks earlier that Intel
DPDK apps do work inside containers. Also, the weird thing is, testpmd
application runs in the container using lxc-execute. Lxc-execute runs the
app inside container. It works, see below:
root@anjali:/dpdk/x86_64-default-linuxapp-gcc/app# lxc-execute -n foo --
./testpmd -c f -n 4 -- -i
lxc: call to cgmanager_move_pid_abs_sync(cpuset) failed: Escape request
from different namespace requires a proxy
rte_eal_init : 859
opt : 99
opt : 110
EAL: Using free_hugepages
EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
EAL: No of hugepages is 2048
rte_eal_init : 884
EAL: No of hugepages is 0
EAL: No of hugepages is 1
EAL: No of hugepages is 2
EAL: No of hugepages is 3
EAL: Setting up hugepage memory...
EAL: Ask a virtual area of 0x8388608 bytes
EAL: Virtual area found at 0x7f7c39c00000 (size = 0x800000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7c39800000 (size = 0x200000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7c39400000 (size = 0x200000)
EAL: Ask a virtual area of 0x293601280 bytes
EAL: Virtual area found at 0x7f7c27a00000 (size = 0x11800000)
EAL: Ask a virtual area of 0x1841299456 bytes
EAL: Virtual area found at 0x7f7bb9c00000 (size = 0x6dc00000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7bb9800000 (size = 0x200000)
EAL: Ask a virtual area of 0x2143289344 bytes
EAL: Virtual area found at 0x7f7b39a00000 (size = 0x7fc00000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7b39600000 (size = 0x200000)
EAL: Requesting 1024 pages of size 2MB from socket 0
EAL: Requesting 1024 pages of size 2MB from socket 1
EAL: TSC frequency is ~2593752 KHz
EAL: No of hugepages is 32902
EAL: No of hugepages is 3584
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
Š

Š

When I logged inside the container and ran the app, I saw this issue.


Anjali

On 7/3/14 7:53 AM, "Shaw, Jeffrey B" <jeffrey.b.shaw-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:

>Do you know if the host's hugepages are mapped into the container?
>Seeing as containers are meant to provide isolation, it seems to make
>sense that the host would not automatically share hugepages with a
>container, but I'm not sure.
>
>Jeff
>
>-----Original Message-----
>From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Anjali Kulkarni
>Sent: Thursday, July 03, 2014 7:41 AM
>To: dev-VfR2kkLFssw@public.gmane.org
>Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside
>containers
>
>Hi,
>
>I am encountering a problem running DPDK apps inside containers, and need
>your help:
>
>I create containers by doing:
>
>lxc-create -t ubuntu -n fpc0
>lxc-start -n fpc0
>
>(Reserve hugepages and follow all other steps to run the DPDK app on host)
>
>Then run a standard dpdk app, on host, it works fine, inside container,
>it runs into this problem:
>
>ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 --
>-i rte_eal_init : 857 opt : 99 opt : 110
>EAL: No free hugepages reported in hugepages-2048kB PANIC in
>rte_eal_init():
>Cannot get hugepage information
>6: [./testpmd() [0x402941]]
>5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)
>[0x7f7f14f50ec5]]
>4: [./testpmd() [0x40237e]]
>3: [./testpmd() [0x478edb]]
>2: [./testpmd() [0x4022ad]]
>1: [./testpmd() [0x47e3c3]]
>Aborted
>ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$
>
>However, running this same app on the host, or doing lxc-execute works.
>
>While trying to debug my own dpdk app, I see that reading the same file
>for no of huge pages (
>
>/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>
> ), on the host, its value is read 2048, while inside the container it is
>0. This is same file.
>Also, if you cat the contents of this file, it is actually 0!
>(nr_hugepages is 2048, but the DPDK code does not read this file.)
>
>cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>0
>
>
>In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are
>added by me in DPDL eal library.
>
>On host:
>rte_eal_init : 859
>opt : 99
>opt : 110
>EAL: Using free_hugepages
>EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>EAL: No of hugepages is 2048
>
>Inside container:
>pt : 99
>opt : 110
>EAL: Using free_hugepages
>EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>EAL: No of hugepages is 0
>
>
>Has anyone run into this problem and know what's the issue?
>
>Anjali