From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bruce Richardson Subject: Re: problem: Cannot allocate memzone for ethernet port data Date: Wed, 22 Apr 2015 14:34:17 +0100 Message-ID: <20150422133416.GA8676@bricha3-MOBL3> References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "dev-VfR2kkLFssw@public.gmane.org" To: ChenXiaodong Return-path: Content-Disposition: inline In-Reply-To: List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces-VfR2kkLFssw@public.gmane.org Sender: "dev" On Wed, Apr 22, 2015 at 07:36:59PM +0800, ChenXiaodong wrote: > Thanks for you reply! There is actually only one NUMA node. here is wh= at the commands return: >=20 Hi, there seems to be a mismatch in terms of the numa node the memory is repo= rted as being on (node 0) and the numa node that the cores are detected as bei= ng on - as reported by the physical package id value. That's why the app is fai= ling as it's looking for the memory on the wrong numa node. Can you try applying the patch I've just posted to change how we detect the numa node of a core, and see if it helps in your case. thanks, /Bruce > [root@test dpdk-2.0.0]# tools/cpu_layout.py=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Core and Socket Information (as reported by '/proc/cpuinfo') > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > cores =3D [0, 1, 2, 3] > sockets =3D [1] >=20 > Socket 1 =20 > -------- =20 > Core 0 [0, 4] =20 >=20 > Core 1 [1, 5] =20 >=20 > Core 2 [2, 6] =20 >=20 > Core 3 [3, 7] =20 >=20 > [root@test dpdk-2.0.0]# cat /sys/devices/system/node/node0/hugepages/hu= gepages-2048kB/nr_hugepages=20 > 1024 >=20 > > From: pablo.de.lara.guarch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org > > To: ch.xd-25CfU72zatU@public.gmane.org; dev-VfR2kkLFssw@public.gmane.org > > Subject: RE: [dpdk-dev] problem: Cannot allocate memzone for ethernet= port data > > Date: Wed, 22 Apr 2015 11:01:08 +0000 > >=20 > > Hi , > >=20 > > > -----Original Message----- > > > From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of ChenXiaodong > > > Sent: Wednesday, April 22, 2015 11:33 AM > > > To: dev-VfR2kkLFssw@public.gmane.org > > > Subject: [dpdk-dev] problem: Cannot allocate memzone for ethernet p= ort > > > data > > >=20 > > > Hi, every one > > >=20 > > > Is there any one who can help me ? I'm new to DPDK, here is a prob= lem that > > > I don't know how to deal with. The program is running on an machine= with > > > two Intel 82576 NICs. One of the NICs is binded with the uio_pci_g= eneric > > > driver=E3=80=821024 2M hugepages have been setup. > > >=20 > > > Any suggestion about the cause of the problem or solution to it wil= l be > > > appreciated! > >=20 > > Could you tell us what these commands return? > >=20 > > ./tools/cpu_layout.py > > cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_huge= pages > > cat /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_huge= pages > >=20 > > (assuming you have 2 NUMA nodes) > >=20 > > Pablo > > >=20 > > > Here is the output of the program: > > >=20 > > > [root@test dpdk-2.0.0]# build/app/testpmd -c ff -n 4 --socket-mem=3D= 1024 > > > EAL: Cannot read numa node link for lcore 0 - using physical packag= e id > > > instead > > > EAL: Detected lcore 0 as core 0 on socket 1 > > > EAL: Cannot read numa node link for lcore 1 - using physical packag= e id > > > instead > > > EAL: Detected lcore 1 as core 1 on socket 1 > > > EAL: Cannot read numa node link for lcore 2 - using physical packag= e id > > > instead > > > EAL: Detected lcore 2 as core 2 on socket 1 > > > EAL: Cannot read numa node link for lcore 3 - using physical packag= e id > > > instead > > > EAL: Detected lcore 3 as core 3 on socket 1 > > > EAL: Cannot read numa node link for lcore 4 - using physical packag= e id > > > instead > > > EAL: Detected lcore 4 as core 0 on socket 1 > > > EAL: Cannot read numa node link for lcore 5 - using physical packag= e id > > > instead > > > EAL: Detected lcore 5 as core 1 on socket 1 > > > EAL: Cannot read numa node link for lcore 6 - using physical packag= e id > > > instead > > > EAL: Detected lcore 6 as core 2 on socket 1 > > > EAL: Cannot read numa node link for lcore 7 - using physical packag= e id > > > instead > > > EAL: Detected lcore 7 as core 3 on socket 1 > > > EAL: Support maximum 128 logical core(s) by configuration. > > > EAL: Detected 8 lcore(s) > > > EAL: Setting up memory... > > > EAL: Ask a virtual area of 0x8000000 bytes > > > EAL: Virtual area found at 0x7f83fae00000 (size =3D 0x8000000) > > > EAL: Ask a virtual area of 0x38000000 bytes > > > EAL: Virtual area found at 0x7f83c2c00000 (size =3D 0x38000000) > > > EAL: Requesting 512 pages of size 2MB from socket 0 > > > EAL: TSC frequency is ~2394002 KHz > > > EAL: WARNING: Master core has no memory on local socket! > > > EAL: Cannot read numa node link for lcore 0 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 0 - using physical packag= e id > > > instead > > > EAL: Master lcore 0 is ready (tid=3D319e880;cpuset=3D[0]) > > > PMD: ENICPMD trace: rte_enic_pmd_init > > > EAL: Cannot read numa node link for lcore 4 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 5 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 1 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 1 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 2 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 3 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 5 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 6 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 7 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 2 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 7 - using physical packag= e id > > > instead > > > EAL: lcore 1 is ready (tid=3Dc1ff8700;cpuset=3D[1]) > > > EAL: lcore 2 is ready (tid=3Dc15f7700;cpuset=3D[2]) > > > EAL: Cannot read numa node link for lcore 4 - using physical packag= e id > > > instead > > > EAL: Cannot read numa node link for lcore 3 - using physical packag= e id > > > instead > > > EAL: lcore 5 is ready (tid=3Dbf7f4700;cpuset=3D[5]) > > > EAL: Cannot read numa node link for lcore 6 - using physical packag= e id > > > instead > > > EAL: lcore 7 is ready (tid=3Dbe3f2700;cpuset=3D[7]) > > > EAL: lcore 3 is ready (tid=3Dc0bf6700;cpuset=3D[3]) > > > EAL: lcore 6 is ready (tid=3Dbedf3700;cpuset=3D[6]) > > > EAL: lcore 4 is ready (tid=3Dc01f5700;cpuset=3D[4]) > > > EAL: PCI device 0000:07:00.0 on NUMA socket -1 > > > EAL: probe driver: 8086:10c9 rte_igb_pmd > > > EAL: PCI memory mapped at 0x7f8402e00000 > > > EAL: PCI memory mapped at 0x7f83bd5f2000 > > > EAL: PCI memory mapped at 0x7f84031ba000 > > > PANIC in rte_eth_dev_data_alloc(): > > > Cannot allocate memzone for ethernet port data > > > 8: [build/app/testpmd() [0x4285d9]] > > > 7: [/lib64/libc.so.6(__libc_start_main+0xfd) [0x3d18e1ecdd]] > > > 6: [build/app/testpmd(main+0x18) [0x42dce8]] > > > 5: [build/app/testpmd(rte_eal_init+0xab8) [0x4982c8]] > > > 4: [build/app/testpmd(rte_eal_pci_probe+0xe2) [0x4a27e2]] > > > 3: [build/app/testpmd() [0x491ea7]] > > > 2: [build/app/testpmd(__rte_panic+0xc0) [0x428520]] > > > 1: [build/app/testpmd(rte_dump_stack+0x1e) [0x49e91e]] > > >=20 > > >=20 > =20