From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: [PATCH v3] tools/xenconsoled: Increase file descriptor limit Date: Thu, 19 Feb 2015 17:56:07 +0000 Message-ID: <54E623B7.6070004@citrix.com> References: <20150217164811.GI2159@zion.uk.xensource.com> <1424195752-10842-1-git-send-email-andrew.cooper3@citrix.com> <20150219110457.GQ2159@zion.uk.xensource.com> <21734.4009.740481.828551@mariner.uk.xensource.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <21734.4009.740481.828551@mariner.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Jackson , Wei Liu Cc: Ian Campbell , Xen-devel List-Id: xen-devel@lists.xenproject.org On 19/02/15 16:30, Ian Jackson wrote: > Wei Liu writes ("Re: [PATCH v3] tools/xenconsoled: Increase file descriptor limit"): >> On Tue, Feb 17, 2015 at 05:55:52PM +0000, Andrew Cooper wrote: >>> XenServer's VM density testing uncovered a regression when moving from >>> sysvinit to systemd where the file descriptor limit dropped from 4096 to >>> 1024. (XenServer had previously inserted a ulimit statement into its >>> initscripts.) > I think that putting something like this in the initscripts is a good > idea. > > I don't like the idea of this kind of resource limit mangling (quite > vigorous, too!) being buried in the C code. > >>> One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the >>> lost ulimit, but that is only a stopgap solution. > Why is this only a stopgap solution ? It is yet another place with an arbitrary limit, which is one more moving part to go wrong. > >>> As Xenconsoled genuinely needs a large number of file descriptors >>> if a large number of domains are running, and is well behaved with >>> its descriptors, attempt to up the limit to the system maximum. > Perhaps you mean it should be "unlimited" but that doesn't seem right > either. Where do I perhaps mean "unlimited"? Attempting to set NOFILE with RLIM_INFINITY is an unconditional failure, and there is unfortunately no RLIM_SYSTEM_MAX. > The value should be some multiple of the maximum number of > domains. The maximum number of domains (limited by the domid abi) is 32751 However, a brief grep through the code shows that the fd situation is far worse than I first thought. There are 4 FDs held open per domain[1], and some overhead for hypervisor logging, gntdev and privcmd. That would put the number of fds needed at 131004 + overhead (20 perhaps?). This is substantially larger than the default on Linux, but within the default system max of 2^20. ~Andrew [1] along with a todo note suggesting that opening evtchn for each domain is inefficient.