From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Tan, Jianfeng" <jianfeng.tan@intel.com>
Subject: Re: Fwd: EAL: map_all_hugepages(): mmap failed: Cannot
 allocate memory
Date: Fri, 18 Mar 2016 10:51:21 +0800
Message-ID: <56EB6D29.9020907@intel.com>
References: <CAGaeUppJkrWXxcx5tMyeeJiW2ivGmPVAimYW1zNA4A=pSV2Z_g@mail.gmail.com>
 <CAGaeUpr6KSp2ToRwYG_xx8BQP=8ZLQVr4gfKcWceJ5Bb_ndTKw@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
To: John Wei <johntwei@gmail.com>, dev@dpdk.org
Return-path: <dev-bounces@dpdk.org>
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
 by dpdk.org (Postfix) with ESMTP id E407B56A2
 for <dev@dpdk.org>; Fri, 18 Mar 2016 03:51:24 +0100 (CET)
In-Reply-To: <CAGaeUpr6KSp2ToRwYG_xx8BQP=8ZLQVr4gfKcWceJ5Bb_ndTKw@mail.gmail.com>
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>



On 3/18/2016 6:41 AM, John Wei wrote:
> I am setting up OVS inside a Linux container. This OVS is built using DPDK
> library.
> During the startup of ovs-vswitchd, it core dumped due to fail to mmap.
>        in eal_memory.c
>         virtaddr = mmap(vma_addr, hugepage_sz, PROT_READ | PROT_WRITE,
>                  MAP_SHARED, fd, 0);
>
> This call is made inside a for loop that loops through all the pages and
> mmap them.
> My server has two cores, and I allocated 8192 2MB pages.
> The mmap for the first 4096 pages were successful. It failed when trying to
> map 4096th page.
>
> Can someone help me understand when the mmap for the first 4096 pages were
> successful and it failed on 4096th page?

In my limited experience, there are some scenario that may lead to such 
failure: a. specified an option size when do mount hugetlbfs; b. cgroup 
limitation, /sys/fs/cgroup/hugetlb/<cgroup 
name>/hugetlb.2MB.limit_in_bytes; c. open files by ulimit...

Workaround: as only "--socket-mem 128,128" is needed, you can reduce the 
total number of 2M hugepages from 8192 to 512 (or something else).
In addition: this is a case why I sent a patchset: 
http://dpdk.org/dev/patchwork/patch/11194/

Thanks,
Jianfeng

>
>
> John
>
>
>
> ovs-vswitchd --dpdk -c 0x1 -n 4 -l 1 --file-prefix ct0000- --socket-mem
> 128,128 -- unix:$DB_SOCK --pidfile --detach --log-file=ct.log
>
>
> EAL: Detected lcore 23 as core 5 on socket 1
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 24 lcore(s)
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: VFIO modules not all loaded, skip VFIO support...
> EAL: Setting up physically contiguous memory...
> EAL: map_all_hugepages(): mmap failed: Cannot allocate memory
> EAL: Failed to mmap 2 MB hugepages
> PANIC in rte_eal_init():
> Cannot init memory
> 7: [ovs-vswitchd() [0x411f15]]
> 6: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff5f6133b15]]
> 5: [ovs-vswitchd() [0x4106f9]]
> 4: [ovs-vswitchd() [0x66917d]]
> 3: [ovs-vswitchd() [0x42b6f5]]
> 2: [ovs-vswitchd() [0x40dd8c]]
> 1: [ovs-vswitchd() [0x56b3ba]]
> Aborted (core dumped)