* [Bug 95] eal/memory: testpmd failed to start with --socket-mem > 16384 with 2MB huge pages and with memory hotplug
@ 2018-10-16 16:38 bugzilla
0 siblings, 0 replies; only message in thread
From: bugzilla @ 2018-10-16 16:38 UTC (permalink / raw)
To: dev
https://bugs.dpdk.org/show_bug.cgi?id=95
Bug ID: 95
Summary: eal/memory: testpmd failed to start with --socket-mem
> 16384 with 2MB huge pages and with memory hotplug
Product: DPDK
Version: 18.11
Hardware: x86
OS: Linux
Status: CONFIRMED
Severity: normal
Priority: Normal
Component: core
Assignee: dev@dpdk.org
Reporter: maxime.leroy@6wind.com
Target Milestone: ---
With the lastest version on dpdk.org (i.e. commit id
94d31549c380ee8b9108c4621d6fad8d624a395e)
First configure 2Mb hugepages:
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 12598 >
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
Start test pmd with 25Gb memory:
root@ubuntu1604:~/dpdk# ./build/app/testpmd -c7 --socket-mem 25196 -- -i
--nb-cores=2 - --total-num-mbufs=2048
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
PANIC in main():
Cannot init EAL
5: [./build/app/testpmd(_start+0x29) [0x48b509]]
4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f28f9adb830]]
3: [./build/app/testpmd(main+0xb0e) [0x4825ae]]
2: [./build/app/testpmd(__rte_panic+0xc3) [0x475920]]
1: [./build/app/testpmd(rte_dump_stack+0x2b) [0x5ba22b]]
Aborted (core dumped)
It failed.
With --legacy-mem option, it's work fine:
./build/app/testpmd -c7 --socket-mem 25196 --legacy-mem -- -i --nb-cores=2 -
--total-num-mbufs=2048
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
..
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=2048, size=2176,
socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Done
testpmd>
testpmd> dump_physmem
Segment 0-0: IOVA:0x2400000, len:2097152, virt:0x7f3491a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:11
Segment 0-1: IOVA:0x2600000, len:2097152, virt:0x7f3491c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:12
Segment 0-2: IOVA:0x2800000, len:2097152, virt:0x7f3491e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:13
Segment 0-3: IOVA:0x2a00000, len:2097152, virt:0x7f3492000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:14
Segment 0-4: IOVA:0x2c00000, len:2097152, virt:0x7f3492200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:15
Segment 0-5: IOVA:0x2e00000, len:2097152, virt:0x7f3492400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:16
Segment 0-6: IOVA:0x3000000, len:2097152, virt:0x7f3492600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:17
Segment 0-7: IOVA:0x3200000, len:2097152, virt:0x7f3492800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:18
Segment 0-8: IOVA:0x3400000, len:2097152, virt:0x7f3492a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:19
Segment 0-9: IOVA:0x3600000, len:2097152, virt:0x7f3492c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:20
Segment 0-10: IOVA:0x3800000, len:2097152, virt:0x7f3492e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:21
Segment 0-11: IOVA:0x3a00000, len:2097152, virt:0x7f3493000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:22
Segment 0-12: IOVA:0x3c00000, len:2097152, virt:0x7f3493200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:23
Segment 0-13: IOVA:0x3e00000, len:2097152, virt:0x7f3493400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:24
Segment 0-14: IOVA:0x4000000, len:2097152, virt:0x7f3493600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:25
...
Segment 0-8188: IOVA:0x539200000, len:2097152, virt:0x7f3891200000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:7617
Segment 0-8189: IOVA:0x539400000, len:2097152, virt:0x7f3891400000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:7618
Segment 0-8190: IOVA:0x539600000, len:2097152, virt:0x7f3891600000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:7619
Segment 1-0: IOVA:0x514800000, len:2097152, virt:0x7f3091800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7335
Segment 1-1: IOVA:0x514a00000, len:2097152, virt:0x7f3091a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7336
Segment 1-2: IOVA:0x514c00000, len:2097152, virt:0x7f3091c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7337
Segment 1-3: IOVA:0x514e00000, len:2097152, virt:0x7f3091e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7338
Segment 1-4: IOVA:0x515000000, len:2097152, virt:0x7f3092000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:7339
...
Segment 1-5517: IOVA:0x7f3c00000, len:2097152, virt:0x7f3343200000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12602
Segment 1-5519: IOVA:0x7f4c00000, len:2097152, virt:0x7f3343600000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12603
Segment 1-5520: IOVA:0x7f4e00000, len:2097152, virt:0x7f3343800000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12604
Segment 1-5522: IOVA:0x7f7000000, len:2097152, virt:0x7f3343c00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12605
Segment 1-5524: IOVA:0x7f7a00000, len:2097152, virt:0x7f3344000000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12606
Segment 1-5526: IOVA:0x7fc600000, len:2097152, virt:0x7f3344400000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12607
Segment 1-5528: IOVA:0x800c00000, len:2097152, virt:0x7f3344800000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12608
We have 2 segments. The first one has 8192 pages and the second one 5529
segment.
If we increaze the MAX_MEMSEG_LIST:
--- a/config/common_base
+++ b/config/common_base
@@ -65,7 +65,7 @@ CONFIG_RTE_MAX_HEAPS=32
CONFIG_RTE_MAX_MEMSEG_LISTS=64
# each memseg list will be limited to either RTE_MAX_MEMSEG_PER_LIST pages
# or RTE_MAX_MEM_MB_PER_LIST megabytes worth of memory, whichever is smaller
-CONFIG_RTE_MAX_MEMSEG_PER_LIST=8192
+CONFIG_RTE_MAX_MEMSEG_PER_LIST=16384
./build/app/testpmd -c7 --socket-mem 25196 -- -i --nb-cores=2 -
--total-num-mbufs=2048
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=2048, size=2176,
socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Done
testpmd>
It's working fine.
testpmd> dump_physmem
Segment 0-0: IOVA:0x2400000, len:2097152, virt:0x7f3491a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:11
Segment 0-1: IOVA:0x2600000, len:2097152, virt:0x7f3491c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:12
Segment 0-2: IOVA:0x2800000, len:2097152, virt:0x7f3491e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:13
Segment 0-3: IOVA:0x2a00000, len:2097152, virt:0x7f3492000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:14
Segment 0-4: IOVA:0x2c00000, len:2097152, virt:0x7f3492200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:15
Segment 0-5: IOVA:0x2e00000, len:2097152, virt:0x7f3492400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:16
Segment 0-6: IOVA:0x3000000, len:2097152, virt:0x7f3492600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:17
Segment 0-7: IOVA:0x3200000, len:2097152, virt:0x7f3492800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:18
Segment 0-8: IOVA:0x3400000, len:2097152, virt:0x7f3492a00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:19
Segment 0-9: IOVA:0x3600000, len:2097152, virt:0x7f3492c00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:20
Segment 0-10: IOVA:0x3800000, len:2097152, virt:0x7f3492e00000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:21
Segment 0-11: IOVA:0x3a00000, len:2097152, virt:0x7f3493000000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:22
Segment 0-12: IOVA:0x3c00000, len:2097152, virt:0x7f3493200000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:23
Segment 0-13: IOVA:0x3e00000, len:2097152, virt:0x7f3493400000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:24
Segment 0-14: IOVA:0x4000000, len:2097152, virt:0x7f3493600000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:25
Segment 0-15: IOVA:0x4200000, len:2097152, virt:0x7f3493800000, socket_id:0,
hugepage_sz:2097152, nchannel:0, nrank:0 fd:26
..
Segment 0-12588: IOVA:0x4cd000000, len:2097152, virt:0x7fb892c00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12599
Segment 0-12589: IOVA:0x4d2e00000, len:2097152, virt:0x7fb892e00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12600
Segment 0-12590: IOVA:0x73ba00000, len:2097152, virt:0x7fb893000000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12601
Segment 0-12591: IOVA:0x4d4a00000, len:2097152, virt:0x7fb893200000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12602
Segment 0-12592: IOVA:0xc4600000, len:2097152, virt:0x7fb893400000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12603
Segment 0-12593: IOVA:0x514400000, len:2097152, virt:0x7fb893600000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12604
Segment 0-12594: IOVA:0x7e3000000, len:2097152, virt:0x7fb893800000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12605
Segment 0-12595: IOVA:0x4f9800000, len:2097152, virt:0x7fb893a00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12606
Segment 0-12596: IOVA:0x294400000, len:2097152, virt:0x7fb893c00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12607
Segment 0-12597: IOVA:0x4d5000000, len:2097152, virt:0x7fb893e00000,
socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:12608
But we have only one segment in this case.
Question
--------
1) It's normal that we don't support anymore to allocate more that 16GB with
hotplug memory and default config value ? It was working with previous dpdk
version.
2) Should we increase the default value of CONFIG_RTE_MAX_MEMSEG_PER_LIST ?
Note: This http://patches.dpdk.org/patch/46112/ patch doesn't fix the issue.
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2018-10-16 16:38 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-10-16 16:38 [Bug 95] eal/memory: testpmd failed to start with --socket-mem > 16384 with 2MB huge pages and with memory hotplug bugzilla
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.