From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sergio Gonzalez Monroy Subject: Re: [PATCH v9 1/2] mem: balanced allocation of hugepages Date: Wed, 28 Jun 2017 11:30:31 +0100 Message-ID: References: <1498553186-24541-1-git-send-email-i.maximets@samsung.com> <1498559080-27331-1-git-send-email-i.maximets@samsung.com> <1498559080-27331-2-git-send-email-i.maximets@samsung.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Heetae Ahn , Yuanhan Liu , Jianfeng Tan , Neil Horman , Yulong Pei , Bruce Richardson , Jerin Jacob To: Ilya Maximets , dev@dpdk.org, David Marchand , Thomas Monjalon Return-path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 8E4072BE1 for ; Wed, 28 Jun 2017 12:30:36 +0200 (CEST) In-Reply-To: <1498559080-27331-2-git-send-email-i.maximets@samsung.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 27/06/2017 11:24, Ilya Maximets wrote: > Currently EAL allocates hugepages one by one not paying attention > from which NUMA node allocation was done. > > Such behaviour leads to allocation failure if number of available > hugepages for application limited by cgroups or hugetlbfs and > memory requested not only from the first socket. > > Example: > # 90 x 1GB hugepages availavle in a system > > cgcreate -g hugetlb:/test > # Limit to 32GB of hugepages > cgset -r hugetlb.1GB.limit_in_bytes=34359738368 test > # Request 4GB from each of 2 sockets > cgexec -g hugetlb:test testpmd --socket-mem=4096,4096 ... > > EAL: SIGBUS: Cannot mmap more hugepages of size 1024 MB > EAL: 32 not 90 hugepages of size 1024 MB allocated > EAL: Not enough memory available on socket 1! > Requested: 4096MB, available: 0MB > PANIC in rte_eal_init(): > Cannot init memory > > This happens beacause all allocated pages are > on socket 0. > > Fix this issue by setting mempolicy MPOL_PREFERRED for each hugepage > to one of requested nodes using following schema: > > 1) Allocate essential hugepages: > 1.1) Allocate as many hugepages from numa N to > only fit requested memory for this numa. > 1.2) repeat 1.1 for all numa nodes. > 2) Try to map all remaining free hugepages in a round-robin > fashion. > 3) Sort pages and choose the most suitable. > > In this case all essential memory will be allocated and all remaining > pages will be fairly distributed between all requested nodes. > > New config option RTE_EAL_NUMA_AWARE_HUGEPAGES introduced and > enabled by default for linuxapp except armv7 and dpaa2. > Enabling of this option adds libnuma as a dependency for EAL. > > Fixes: 77988fc08dc5 ("mem: fix allocating all free hugepages") > > Signed-off-by: Ilya Maximets > --- > config/common_base | 1 + > config/common_linuxapp | 1 + > config/defconfig_arm-armv7a-linuxapp-gcc | 3 + > config/defconfig_arm64-dpaa2-linuxapp-gcc | 3 + > lib/librte_eal/linuxapp/eal/Makefile | 3 + > lib/librte_eal/linuxapp/eal/eal_memory.c | 120 ++++++++++++++++++++++++++++-- > mk/rte.app.mk | 3 + > 7 files changed, 126 insertions(+), 8 deletions(-) Good stuff Ilya! Hemant, Jerin, could you also ack the patch if you are happy with it? Thanks. Acked-by: Sergio Gonzalez Monroy