[SPDK] Question about open files

[Sudhe (Sudheendra) Sampath <sudhe.s at samsung.com>]

[-- Attachment #1: Type: text/plain, Size: 1178 bytes --]

We ran into an interesting problem yesterday where SPDK ended up running out of open files.  We determined that when we set IOUnitSize and MaxIOSize to 1 MB and use some number of cores (around 20+), we hit the open files issue below.

EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open files

When using fewer core, we did not run into this issue.


We worked around this by upping the limit of open files (ulimit -r 32768), but we want to understand better going forward.  We are assuming that SPDK is grabbing more huge page memory based off the IOUnitSize and MaxIOSize * number of cores.  What is the correlation between open files and number of cores so that we can calculate or needs in the future?

We are using 2 MB huge pages, so we are assuming that if we use 1 GB huge pages, this will not be an issue.  Do you have any other insights in order to avoid this issue when scaling up?

Thanks,

-Sudhe