From mboxrd@z Thu Jan 1 00:00:00 1970 From: annie li Subject: Re: netfront/netback multiqueue exhausting grants Date: Wed, 20 Jan 2016 11:18:48 -0500 Message-ID: <569FB368.6050306@oracle.com> References: <1453292623.26343.95.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1453292623.26343.95.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell , xen-devel Cc: Boris Ostrovsky , Wei Liu , David Vrabel List-Id: xen-devel@lists.xenproject.org On 2016/1/20 7:23, Ian Campbell wrote: > There have been a few reports recently[0] which relate to a failure of > netfront to allocate sufficient grant refs for all the queues: > > [ 0.533589] xen_netfront: can't alloc rx grant refs > [ 0.533612] net eth0: only created 31 queues > > Which can be worked around by increasing the number of grants on the > hypervisor command line or by limiting the number of queues permitted by > either back or front using a module param (which was broken but is now > fixed on both sides, but I'm not sure it has been backported everywhere > such that it is a reliable thing to always tell users as a workaround). Following are the patches to fix module param, they exist since v4.3. xen-netfront: respect user provided max_queues https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=32a844056fd43dda647e1c3c6b9983bdfa04d17d xen-netback: respect user provided max_queues https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4c82ac3c37363e8c4ded6a5fe1ec5fa756b34df3 > > Is there any plan to do anything about the default/out of the box > experience? Either limiting the number of queues or making both ends cope > more gracefully with failure to create some queues (or both) might be > sufficient? We run into similar issue recently, and guess it is better to suggest user to set netback module parameter with the default value as 8? see this link, http://wiki.xenproject.org/wiki/Xen-netback_and_xen-netfront_multi-queue_performance_testing Probably more test are needed to get the default number of best experience. > > I think the crash after the above in the first link at [0] is fixed? I > think that was the purpose of ca88ea1247df "xen-netfront: update num_queues > to real created" which was in 4.3. Correct. Thanks Annie > > Ian. > > [0] http://lists.xen.org/archives/html/xen-users/2016-01/msg00100.html > http://lists.xen.org/archives/html/xen-users/2016-01/msg00072.html > some before hte xmas break too IIRC > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel