From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============8991251582919623046==" MIME-Version: 1.0 From: Sudhe (Sudheendra) Sampath Subject: [SPDK] Question about open files Date: Tue, 20 Aug 2019 22:13:54 +0000 Message-ID: In-Reply-To: CGME20190820221355uscas1p205e413b1361e490aa93c5aa4bf665073@uscas1p2.samsung.com List-ID: To: spdk@lists.01.org --===============8991251582919623046== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable We ran into an interesting problem yesterday where SPDK ended up running ou= t of open files. We determined that when we set IOUnitSize and MaxIOSize t= o 1 MB and use some number of cores (around 20+), we hit the open files iss= ue below. EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files When using fewer core, we did not run into this issue. We worked around this by upping the limit of open files (ulimit -r 32768), = but we want to understand better going forward. We are assuming that SPDK = is grabbing more huge page memory based off the IOUnitSize and MaxIOSize * = number of cores. What is the correlation between open files and number of = cores so that we can calculate or needs in the future? We are using 2 MB huge pages, so we are assuming that if we use 1 GB huge p= ages, this will not be an issue. Do you have any other insights in order t= o avoid this issue when scaling up? Thanks, -Sudhe --===============8991251582919623046==--