From mboxrd@z Thu Jan  1 00:00:00 1970
From: Iain Barker <iain.barker@oracle.com>
Subject: Re: Question about DPDK hugepage fd change
Date: Wed, 27 Feb 2019 05:57:07 -0800 (PST)
Message-ID: <eaa8311f-f5d2-417f-8a6f-a423be7aa799@default>
References: <da423cbe-2ff7-4d3f-8479-9ec0a501b689@default>
 <ACD02F93-6C71-4DEB-95B8-001F9AAD9E21@intel.com>
 <631579E3-02F5-4E12-8BE6-DDAC0AE2E4A7@oracle.com>
 <549A6EB0-6E19-460D-9BE5-52AA40003AF0@intel.com>
 <345EDE69-C416-405F-B88C-04EE8384D20F@oracle.com>
 <896AF59A-4CCF-42FE-B2D7-160C69427DD2@intel.com>
 <aa530fd1-1292-4305-8ad3-093748725202@default>
 <2b3d84de-f0a5-4b38-f670-6318725821ab@intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Cc: dev@dpdk.org, Edwin Leung <edwin.leung@oracle.com>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>, "Wiles, Keith"
 <keith.wiles@intel.com>
Return-path: <dev-bounces@dpdk.org>
Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86])
 by dpdk.org (Postfix) with ESMTP id 570825A6A
 for <dev@dpdk.org>; Wed, 27 Feb 2019 14:57:16 +0100 (CET)
In-Reply-To: <2b3d84de-f0a5-4b38-f670-6318725821ab@intel.com>
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Original Message from: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]=
=20

>I just realized that, unless you're using --legacy-mem switch, one other=
=20
>way to alleviate the issue would be to use --single-file-segments=20
>option. This will still store the fd's, however it will only do so per=20
>memseg list, not per page. So, instead of 1000's of fd's with 2MB pages,=
=20
>you'd end up with under 10. Hope this helps!

Hi Anatoly,

Thanks for the update and suggestion. We did try using --single-file-segmen=
ts previously. Although it lowers the amount of fd's allocated for tracking=
 the segments as you noted, there is still a problem.

It seems that a .lock file is created for each huge page, not for each segm=
ent. So with 2MB pages the glibc limit of 1024 fd's is still exhausted quic=
kly if there is ~2GB of 2MB huge pages.

Edwin can provide more details from his testing. In our case much sooner, a=
s we already use >500 fd's for the application, just 1GB of 2MB huge pages =
is enough to hit the fd limit due to the .lock files.

Thanks.