* Question on huge pages and running DPDK apps inside containers @ 2014-07-03 14:40 Anjali Kulkarni [not found] ` <CFDAB971.2ABC8%anjali-3r7Miqu9kMnR7s880joybQ@public.gmane.org> 0 siblings, 1 reply; 6+ messages in thread From: Anjali Kulkarni @ 2014-07-03 14:40 UTC (permalink / raw) To: dev-VfR2kkLFssw@public.gmane.org Hi, I am encountering a problem running DPDK apps inside containers, and need your help: I create containers by doing: lxc-create -t ubuntu -n fpc0 lxc-start -n fpc0 (Reserve hugepages and follow all other steps to run the DPDK app on host) Then run a standard dpdk app, on host, it works fine, inside container, it runs into this problem: ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- -i rte_eal_init : 857 opt : 99 opt : 110 EAL: No free hugepages reported in hugepages-2048kB PANIC in rte_eal_init(): Cannot get hugepage information 6: [./testpmd() [0x402941]] 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7f14f50ec5]] 4: [./testpmd() [0x40237e]] 3: [./testpmd() [0x478edb]] 2: [./testpmd() [0x4022ad]] 1: [./testpmd() [0x47e3c3]] Aborted ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ However, running this same app on the host, or doing lxc-execute works. While trying to debug my own dpdk app, I see that reading the same file for no of huge pages ( /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages ), on the host, its value is read 2048, while inside the container it is 0. This is same file. Also, if you cat the contents of this file, it is actually 0! (nr_hugepages is 2048, but the DPDK code does not read this file.) cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages 0 In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are added by me in DPDL eal library. On host: rte_eal_init : 859 opt : 99 opt : 110 EAL: Using free_hugepages EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages EAL: No of hugepages is 2048 Inside container: pt : 99 opt : 110 EAL: Using free_hugepages EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages EAL: No of hugepages is 0 Has anyone run into this problem and know what's the issue? Anjali ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CFDAB971.2ABC8%anjali-3r7Miqu9kMnR7s880joybQ@public.gmane.org>]
* Re: Question on huge pages and running DPDK apps inside containers [not found] ` <CFDAB971.2ABC8%anjali-3r7Miqu9kMnR7s880joybQ@public.gmane.org> @ 2014-07-03 14:53 ` Shaw, Jeffrey B [not found] ` <4032A54B6BB5F04B8C08B6CFF08C59285544C729-AtyAts71sc9Qxe9IK+vIArfspsVTdybXVpNB7YpNyf8@public.gmane.org> 2014-07-03 16:02 ` Richardson, Bruce 1 sibling, 1 reply; 6+ messages in thread From: Shaw, Jeffrey B @ 2014-07-03 14:53 UTC (permalink / raw) To: Anjali Kulkarni, dev-VfR2kkLFssw@public.gmane.org Do you know if the host's hugepages are mapped into the container? Seeing as containers are meant to provide isolation, it seems to make sense that the host would not automatically share hugepages with a container, but I'm not sure. Jeff -----Original Message----- From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Anjali Kulkarni Sent: Thursday, July 03, 2014 7:41 AM To: dev-VfR2kkLFssw@public.gmane.org Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside containers Hi, I am encountering a problem running DPDK apps inside containers, and need your help: I create containers by doing: lxc-create -t ubuntu -n fpc0 lxc-start -n fpc0 (Reserve hugepages and follow all other steps to run the DPDK app on host) Then run a standard dpdk app, on host, it works fine, inside container, it runs into this problem: ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- -i rte_eal_init : 857 opt : 99 opt : 110 EAL: No free hugepages reported in hugepages-2048kB PANIC in rte_eal_init(): Cannot get hugepage information 6: [./testpmd() [0x402941]] 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7f14f50ec5]] 4: [./testpmd() [0x40237e]] 3: [./testpmd() [0x478edb]] 2: [./testpmd() [0x4022ad]] 1: [./testpmd() [0x47e3c3]] Aborted ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ However, running this same app on the host, or doing lxc-execute works. While trying to debug my own dpdk app, I see that reading the same file for no of huge pages ( /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages ), on the host, its value is read 2048, while inside the container it is 0. This is same file. Also, if you cat the contents of this file, it is actually 0! (nr_hugepages is 2048, but the DPDK code does not read this file.) cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages 0 In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are added by me in DPDL eal library. On host: rte_eal_init : 859 opt : 99 opt : 110 EAL: Using free_hugepages EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages EAL: No of hugepages is 2048 Inside container: pt : 99 opt : 110 EAL: Using free_hugepages EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages EAL: No of hugepages is 0 Has anyone run into this problem and know what's the issue? Anjali ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <4032A54B6BB5F04B8C08B6CFF08C59285544C729-AtyAts71sc9Qxe9IK+vIArfspsVTdybXVpNB7YpNyf8@public.gmane.org>]
* Re: Question on huge pages and running DPDK apps inside containers [not found] ` <4032A54B6BB5F04B8C08B6CFF08C59285544C729-AtyAts71sc9Qxe9IK+vIArfspsVTdybXVpNB7YpNyf8@public.gmane.org> @ 2014-07-03 15:31 ` Anjali Kulkarni 2014-07-03 15:33 ` Anjali Kulkarni 1 sibling, 0 replies; 6+ messages in thread From: Anjali Kulkarni @ 2014-07-03 15:31 UTC (permalink / raw) To: Shaw, Jeffrey B, dev-VfR2kkLFssw@public.gmane.org Thanks - I got a confirmation from some Intel folks earlier that Intel DPDK apps do work inside containers. Also, the weird thing is, testpmd application runs in the container using lxc-execute. Lxc-execute runs the app inside container. It works, see below: root@anjali:/dpdk/x86_64-default-linuxapp-gcc/app# lxc-execute -n foo -- ./testpmd -c f -n 4 -- -i lxc: call to cgmanager_move_pid_abs_sync(cpuset) failed: Escape request from different namespace requires a proxy rte_eal_init : 859 opt : 99 opt : 110 EAL: Using free_hugepages EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages EAL: No of hugepages is 2048 rte_eal_init : 884 EAL: No of hugepages is 0 EAL: No of hugepages is 1 EAL: No of hugepages is 2 EAL: No of hugepages is 3 EAL: Setting up hugepage memory... EAL: Ask a virtual area of 0x8388608 bytes EAL: Virtual area found at 0x7f7c39c00000 (size = 0x800000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7c39800000 (size = 0x200000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7c39400000 (size = 0x200000) EAL: Ask a virtual area of 0x293601280 bytes EAL: Virtual area found at 0x7f7c27a00000 (size = 0x11800000) EAL: Ask a virtual area of 0x1841299456 bytes EAL: Virtual area found at 0x7f7bb9c00000 (size = 0x6dc00000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7bb9800000 (size = 0x200000) EAL: Ask a virtual area of 0x2143289344 bytes EAL: Virtual area found at 0x7f7b39a00000 (size = 0x7fc00000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7b39600000 (size = 0x200000) EAL: Requesting 1024 pages of size 2MB from socket 0 EAL: Requesting 1024 pages of size 2MB from socket 1 EAL: TSC frequency is ~2593752 KHz EAL: No of hugepages is 32902 EAL: No of hugepages is 3584 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3586 EAL: No of hugepages is 32902 EAL: No of hugepages is 3586 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3587 EAL: No of hugepages is 32902 EAL: No of hugepages is 3587 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3588 EAL: No of hugepages is 32902 EAL: No of hugepages is 3588 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3592 EAL: No of hugepages is 32902 EAL: No of hugepages is 3592 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3594 EAL: No of hugepages is 32902 EAL: No of hugepages is 3594 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3624 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3626 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3628 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7486 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7482 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7483 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7469 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7440 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7462 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 9294 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7489 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7426 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7458 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7460 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 5409 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 5409 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 4347 EAL: No of hugepages is 5421 EAL: No of hugepages is 35198 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 4347 EAL: No of hugepages is 5421 EAL: No of hugepages is 35198 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 7531 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 6659 EAL: No of hugepages is 4432 EAL: No of hugepages is 6659 EAL: No of hugepages is 4432 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3592 EAL: No of hugepages is 32902 EAL: No of hugepages is 3592 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3624 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3626 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3628 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 4347 EAL: No of hugepages is 32902 EAL: No of hugepages is 3 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 4347 EAL: No of hugepages is 32902 EAL: No of hugepages is 3 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3712 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3728 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3760 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3761 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3762 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3763 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3764 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3765 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3766 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3767 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3613 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3636 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3713 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3638 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3784 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3785 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3786 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3776 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3777 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3778 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3779 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3614 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3615 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3808 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3810 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3812 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3814 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3809 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3811 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3813 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3815 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3744 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3632 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3752 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3697 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3754 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3755 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3756 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 32902 EAL: No of hugepages is 3757 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 1 EAL: No of hugepages is 6659 EAL: No of hugepages is 8192 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3712 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3728 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3760 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3761 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3762 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3763 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3764 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3765 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3766 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3767 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3613 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3636 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3713 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3638 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3784 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3785 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3786 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3776 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3777 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3778 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3779 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3614 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3615 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3808 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3810 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3812 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3814 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3809 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3811 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3813 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3815 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3744 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3632 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3752 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3697 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3754 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3755 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3756 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 EAL: No of hugepages is 32902 EAL: No of hugepages is 3757 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 ... When I logged inside the container and ran the app, I saw this issue. Anjali On 7/3/14 7:53 AM, "Shaw, Jeffrey B" <jeffrey.b.shaw-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote: >Do you know if the host's hugepages are mapped into the container? >Seeing as containers are meant to provide isolation, it seems to make >sense that the host would not automatically share hugepages with a >container, but I'm not sure. > >Jeff > >-----Original Message----- >From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Anjali Kulkarni >Sent: Thursday, July 03, 2014 7:41 AM >To: dev-VfR2kkLFssw@public.gmane.org >Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside >containers > >Hi, > >I am encountering a problem running DPDK apps inside containers, and need >your help: > >I create containers by doing: > >lxc-create -t ubuntu -n fpc0 >lxc-start -n fpc0 > >(Reserve hugepages and follow all other steps to run the DPDK app on host) > >Then run a standard dpdk app, on host, it works fine, inside container, >it runs into this problem: > >ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- >-i rte_eal_init : 857 opt : 99 opt : 110 >EAL: No free hugepages reported in hugepages-2048kB PANIC in >rte_eal_init(): >Cannot get hugepage information >6: [./testpmd() [0x402941]] >5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >[0x7f7f14f50ec5]] >4: [./testpmd() [0x40237e]] >3: [./testpmd() [0x478edb]] >2: [./testpmd() [0x4022ad]] >1: [./testpmd() [0x47e3c3]] >Aborted >ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ > >However, running this same app on the host, or doing lxc-execute works. > >While trying to debug my own dpdk app, I see that reading the same file >for no of huge pages ( > >/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > > ), on the host, its value is read 2048, while inside the container it is >0. This is same file. >Also, if you cat the contents of this file, it is actually 0! >(nr_hugepages is 2048, but the DPDK code does not read this file.) > >cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >0 > > >In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are >added by me in DPDL eal library. > >On host: >rte_eal_init : 859 >opt : 99 >opt : 110 >EAL: Using free_hugepages >EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >EAL: No of hugepages is 2048 > >Inside container: >pt : 99 >opt : 110 >EAL: Using free_hugepages >EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >EAL: No of hugepages is 0 > > >Has anyone run into this problem and know what's the issue? > >Anjali ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question on huge pages and running DPDK apps inside containers [not found] ` <4032A54B6BB5F04B8C08B6CFF08C59285544C729-AtyAts71sc9Qxe9IK+vIArfspsVTdybXVpNB7YpNyf8@public.gmane.org> 2014-07-03 15:31 ` Anjali Kulkarni @ 2014-07-03 15:33 ` Anjali Kulkarni 1 sibling, 0 replies; 6+ messages in thread From: Anjali Kulkarni @ 2014-07-03 15:33 UTC (permalink / raw) To: Shaw, Jeffrey B, dev-VfR2kkLFssw@public.gmane.org Ignore the previous email, it was too long. Pls see this one only: Thanks - I got a confirmation from some Intel folks earlier that Intel DPDK apps do work inside containers. Also, the weird thing is, testpmd application runs in the container using lxc-execute. Lxc-execute runs the app inside container. It works, see below: root@anjali:/dpdk/x86_64-default-linuxapp-gcc/app# lxc-execute -n foo -- ./testpmd -c f -n 4 -- -i lxc: call to cgmanager_move_pid_abs_sync(cpuset) failed: Escape request from different namespace requires a proxy rte_eal_init : 859 opt : 99 opt : 110 EAL: Using free_hugepages EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages EAL: No of hugepages is 2048 rte_eal_init : 884 EAL: No of hugepages is 0 EAL: No of hugepages is 1 EAL: No of hugepages is 2 EAL: No of hugepages is 3 EAL: Setting up hugepage memory... EAL: Ask a virtual area of 0x8388608 bytes EAL: Virtual area found at 0x7f7c39c00000 (size = 0x800000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7c39800000 (size = 0x200000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7c39400000 (size = 0x200000) EAL: Ask a virtual area of 0x293601280 bytes EAL: Virtual area found at 0x7f7c27a00000 (size = 0x11800000) EAL: Ask a virtual area of 0x1841299456 bytes EAL: Virtual area found at 0x7f7bb9c00000 (size = 0x6dc00000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7bb9800000 (size = 0x200000) EAL: Ask a virtual area of 0x2143289344 bytes EAL: Virtual area found at 0x7f7b39a00000 (size = 0x7fc00000) EAL: Ask a virtual area of 0x2097152 bytes EAL: Virtual area found at 0x7f7b39600000 (size = 0x200000) EAL: Requesting 1024 pages of size 2MB from socket 0 EAL: Requesting 1024 pages of size 2MB from socket 1 EAL: TSC frequency is ~2593752 KHz EAL: No of hugepages is 32902 EAL: No of hugepages is 3584 EAL: No of hugepages is 5421 EAL: No of hugepages is 35207 EAL: No of hugepages is 0 Š Š When I logged inside the container and ran the app, I saw this issue. Anjali On 7/3/14 7:53 AM, "Shaw, Jeffrey B" <jeffrey.b.shaw-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote: >Do you know if the host's hugepages are mapped into the container? >Seeing as containers are meant to provide isolation, it seems to make >sense that the host would not automatically share hugepages with a >container, but I'm not sure. > >Jeff > >-----Original Message----- >From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Anjali Kulkarni >Sent: Thursday, July 03, 2014 7:41 AM >To: dev-VfR2kkLFssw@public.gmane.org >Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside >containers > >Hi, > >I am encountering a problem running DPDK apps inside containers, and need >your help: > >I create containers by doing: > >lxc-create -t ubuntu -n fpc0 >lxc-start -n fpc0 > >(Reserve hugepages and follow all other steps to run the DPDK app on host) > >Then run a standard dpdk app, on host, it works fine, inside container, >it runs into this problem: > >ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- >-i rte_eal_init : 857 opt : 99 opt : 110 >EAL: No free hugepages reported in hugepages-2048kB PANIC in >rte_eal_init(): >Cannot get hugepage information >6: [./testpmd() [0x402941]] >5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >[0x7f7f14f50ec5]] >4: [./testpmd() [0x40237e]] >3: [./testpmd() [0x478edb]] >2: [./testpmd() [0x4022ad]] >1: [./testpmd() [0x47e3c3]] >Aborted >ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ > >However, running this same app on the host, or doing lxc-execute works. > >While trying to debug my own dpdk app, I see that reading the same file >for no of huge pages ( > >/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > > ), on the host, its value is read 2048, while inside the container it is >0. This is same file. >Also, if you cat the contents of this file, it is actually 0! >(nr_hugepages is 2048, but the DPDK code does not read this file.) > >cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >0 > > >In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are >added by me in DPDL eal library. > >On host: >rte_eal_init : 859 >opt : 99 >opt : 110 >EAL: Using free_hugepages >EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >EAL: No of hugepages is 2048 > >Inside container: >pt : 99 >opt : 110 >EAL: Using free_hugepages >EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >EAL: No of hugepages is 0 > > >Has anyone run into this problem and know what's the issue? > >Anjali ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Question on huge pages and running DPDK apps inside containers [not found] ` <CFDAB971.2ABC8%anjali-3r7Miqu9kMnR7s880joybQ@public.gmane.org> 2014-07-03 14:53 ` Shaw, Jeffrey B @ 2014-07-03 16:02 ` Richardson, Bruce [not found] ` <59AF69C657FD0841A61C55336867B5B02CF13B78-kPTMFJFq+rELt2AQoY/u9bfspsVTdybXVpNB7YpNyf8@public.gmane.org> 1 sibling, 1 reply; 6+ messages in thread From: Richardson, Bruce @ 2014-07-03 16:02 UTC (permalink / raw) To: Anjali Kulkarni, dev-VfR2kkLFssw@public.gmane.org Can you try deleting the hugepage files on the host before trying to run the app in a container? > -----Original Message----- > From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Anjali Kulkarni > Sent: Thursday, July 03, 2014 7:41 AM > To: dev-VfR2kkLFssw@public.gmane.org > Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside > containers > > Hi, > > I am encountering a problem running DPDK apps inside containers, and need > your help: > > I create containers by doing: > > lxc-create -t ubuntu -n fpc0 > lxc-start -n fpc0 > > (Reserve hugepages and follow all other steps to run the DPDK app on host) > > Then run a standard dpdk app, on host, it works fine, inside container, it runs > into this problem: > > ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- -i > rte_eal_init : 857 > opt : 99 > opt : 110 > EAL: No free hugepages reported in hugepages-2048kB > PANIC in rte_eal_init(): > Cannot get hugepage information > 6: [./testpmd() [0x402941]] > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7f14f50ec5]] > 4: [./testpmd() [0x40237e]] > 3: [./testpmd() [0x478edb]] > 2: [./testpmd() [0x4022ad]] > 1: [./testpmd() [0x47e3c3]] > Aborted > ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ > > However, running this same app on the host, or doing lxc-execute works. > > While trying to debug my own dpdk app, I see that reading the same file for no > of huge pages ( > > /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > > ), on the host, its value is read 2048, while inside the container it is 0. This is > same file. > Also, if you cat the contents of this file, it is actually 0! (nr_hugepages is 2048, > but the DPDK code does not read this file.) > > cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > 0 > > > In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are added by me > in DPDL eal library. > > On host: > rte_eal_init : 859 > opt : 99 > opt : 110 > EAL: Using free_hugepages > EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > EAL: No of hugepages is 2048 > > Inside container: > pt : 99 > opt : 110 > EAL: Using free_hugepages > EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages > EAL: No of hugepages is 0 > > > Has anyone run into this problem and know what's the issue? > > Anjali ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <59AF69C657FD0841A61C55336867B5B02CF13B78-kPTMFJFq+rELt2AQoY/u9bfspsVTdybXVpNB7YpNyf8@public.gmane.org>]
* Re: Question on huge pages and running DPDK apps inside containers [not found] ` <59AF69C657FD0841A61C55336867B5B02CF13B78-kPTMFJFq+rELt2AQoY/u9bfspsVTdybXVpNB7YpNyf8@public.gmane.org> @ 2014-07-03 18:09 ` Anjali Kulkarni 0 siblings, 0 replies; 6+ messages in thread From: Anjali Kulkarni @ 2014-07-03 18:09 UTC (permalink / raw) To: Richardson, Bruce, dev-VfR2kkLFssw@public.gmane.org Richard, thanks. Looking at lxc-execute, I see this, which seems to be the key in running the applications created via lxc-create and those run lxc-execute: Note "lxc-execute is intended to run application containers that share the host's root file system, and not to run system containers that you create using lxc-create. Use lxc-start to run system containers." Anjali On 7/3/14 9:02 AM, "Richardson, Bruce" <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote: >Can you try deleting the hugepage files on the host before trying to run >the app in a container? > >> -----Original Message----- >> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Anjali Kulkarni >> Sent: Thursday, July 03, 2014 7:41 AM >> To: dev-VfR2kkLFssw@public.gmane.org >> Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside >> containers >> >> Hi, >> >> I am encountering a problem running DPDK apps inside containers, and >>need >> your help: >> >> I create containers by doing: >> >> lxc-create -t ubuntu -n fpc0 >> lxc-start -n fpc0 >> >> (Reserve hugepages and follow all other steps to run the DPDK app on >>host) >> >> Then run a standard dpdk app, on host, it works fine, inside container, >>it runs >> into this problem: >> >> ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 >>-- -i >> rte_eal_init : 857 >> opt : 99 >> opt : 110 >> EAL: No free hugepages reported in hugepages-2048kB >> PANIC in rte_eal_init(): >> Cannot get hugepage information >> 6: [./testpmd() [0x402941]] >> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>[0x7f7f14f50ec5]] >> 4: [./testpmd() [0x40237e]] >> 3: [./testpmd() [0x478edb]] >> 2: [./testpmd() [0x4022ad]] >> 1: [./testpmd() [0x47e3c3]] >> Aborted >> ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ >> >> However, running this same app on the host, or doing lxc-execute works. >> >> While trying to debug my own dpdk app, I see that reading the same file >>for no >> of huge pages ( >> >> /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >> >> ), on the host, its value is read 2048, while inside the container it >>is 0. This is >> same file. >> Also, if you cat the contents of this file, it is actually 0! >>(nr_hugepages is 2048, >> but the DPDK code does not read this file.) >> >> cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >> 0 >> >> >> In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are >>added by me >> in DPDL eal library. >> >> On host: >> rte_eal_init : 859 >> opt : 99 >> opt : 110 >> EAL: Using free_hugepages >> EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >> EAL: No of hugepages is 2048 >> >> Inside container: >> pt : 99 >> opt : 110 >> EAL: Using free_hugepages >> EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages >> EAL: No of hugepages is 0 >> >> >> Has anyone run into this problem and know what's the issue? >> >> Anjali ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-07-03 18:09 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-07-03 14:40 Question on huge pages and running DPDK apps inside containers Anjali Kulkarni [not found] ` <CFDAB971.2ABC8%anjali-3r7Miqu9kMnR7s880joybQ@public.gmane.org> 2014-07-03 14:53 ` Shaw, Jeffrey B [not found] ` <4032A54B6BB5F04B8C08B6CFF08C59285544C729-AtyAts71sc9Qxe9IK+vIArfspsVTdybXVpNB7YpNyf8@public.gmane.org> 2014-07-03 15:31 ` Anjali Kulkarni 2014-07-03 15:33 ` Anjali Kulkarni 2014-07-03 16:02 ` Richardson, Bruce [not found] ` <59AF69C657FD0841A61C55336867B5B02CF13B78-kPTMFJFq+rELt2AQoY/u9bfspsVTdybXVpNB7YpNyf8@public.gmane.org> 2014-07-03 18:09 ` Anjali Kulkarni
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).