From mboxrd@z Thu Jan 1 00:00:00 1970 From: annie li Subject: Re: Xen-unstable Linux 3.14-rc3 and 3.13 Network troubles Date: Fri, 21 Feb 2014 14:32:08 +0800 Message-ID: <5306F2E8.5090509@oracle.com> References: <1772884781.20140218222513@eikelenboom.it> <5305CFC6.3080502@oracle.com> <587238484.20140220121842@eikelenboom.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <587238484.20140220121842@eikelenboom.it> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Sander Eikelenboom Cc: Paul Durrant , Wei Liu , Zoltan Kiss , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 2014/2/20 19:18, Sander Eikelenboom wrote: > Thursday, February 20, 2014, 10:49:58 AM, you wrote: > > >> On 2014/2/19 5:25, Sander Eikelenboom wrote: >>> Hi All, >>> >>> I'm currently having some network troubles with Xen and recent linux kernels. >>> >>> - When running with a 3.14-rc3 kernel in dom0 and a 3.13 kernel in domU >>> I get what seems to be described in this thread: http://www.spinics.net/lists/netdev/msg242953.html >>> >>> In the guest: >>> [57539.859584] net eth0: rx->offset: 0, size: 4294967295 >>> [57539.859599] net eth0: rx->offset: 0, size: 4294967295 >>> [57539.859605] net eth0: rx->offset: 0, size: 4294967295 >>> [57539.859610] net eth0: Need more slots >>> [58157.675939] net eth0: Need more slots >>> [58725.344712] net eth0: Need more slots >>> [61815.849180] net eth0: rx->offset: 0, size: 4294967295 >>> [61815.849205] net eth0: rx->offset: 0, size: 4294967295 >>> [61815.849216] net eth0: rx->offset: 0, size: 4294967295 >>> [61815.849225] net eth0: Need more slots >> This issue is familiar... and I thought it get fixed. >> From original analysis for similar issue I hit before, the root cause >> is netback still creates response when the ring is full. I remember >> larger MTU can trigger this issue before, what is the MTU size? > In dom0 both for the physical nics and the guest vif's MTU=1500 > In domU the eth0 also has MTU=1500. > > So it's not jumbo frames .. just everywhere the same plain defaults .. > > With the patch from Wei that solves the other issue, i'm still seeing the Need more slots issue on 3.14-rc3+wei's patch now. > I have extended the "need more slots warn" to also print the cons, slots, max, rx->offset, size, hope that gives some more insight. > But it indeed is the VM were i had similar issues before, the primary thing this VM does is 2 simultaneous rsync's (one push one pull) with some gigabytes of data. > > This time it was also acompanied by a "grant_table.c:1857:d0 Bad grant reference " as seen below, don't know if it's a cause or a effect though. The log "grant_table.c:1857:d0 Bad grant reference " was also seen before. Probably the response overlaps the request and grantcopy return error when using wrong grant reference, Netback returns resp->status with ||XEN_NETIF_RSP_ERROR(-1) which is 4294967295 printed above from frontend. Would it be possible to print log in xenvif_rx_action of netback to see whether something wrong with max slots and used slots? Thanks Annie > > Will keep you posted when it triggers again with the extra info in the warn. > > -- > Sander > > > >> Thanks >> Annie >>> Xen reports: >>> (XEN) [2014-02-18 03:22:47] grant_table.c:1857:d0 Bad grant reference 19791875 >>> (XEN) [2014-02-18 03:42:33] grant_table.c:1857:d0 Bad grant reference 268435460 >>> (XEN) [2014-02-18 04:15:23] grant_table.c:289:d0 Increased maptrack size to 14 frames >>> (XEN) [2014-02-18 04:15:27] grant_table.c:289:d0 Increased maptrack size to 15 frames >>> (XEN) [2014-02-18 04:15:48] grant_table.c:289:d0 Increased maptrack size to 16 frames >>> (XEN) [2014-02-18 04:15:50] grant_table.c:289:d0 Increased maptrack size to 17 frames >>> (XEN) [2014-02-18 04:15:55] grant_table.c:289:d0 Increased maptrack size to 18 frames >>> (XEN) [2014-02-18 04:15:55] grant_table.c:289:d0 Increased maptrack size to 19 frames >>> (XEN) [2014-02-18 04:15:56] grant_table.c:289:d0 Increased maptrack size to 20 frames >>> (XEN) [2014-02-18 04:15:56] grant_table.c:289:d0 Increased maptrack size to 21 frames >>> (XEN) [2014-02-18 04:15:59] grant_table.c:289:d0 Increased maptrack size to 22 frames >>> (XEN) [2014-02-18 04:15:59] grant_table.c:289:d0 Increased maptrack size to 23 frames >>> (XEN) [2014-02-18 04:16:00] grant_table.c:289:d0 Increased maptrack size to 24 frames >>> (XEN) [2014-02-18 04:16:05] grant_table.c:289:d0 Increased maptrack size to 25 frames >>> (XEN) [2014-02-18 04:16:05] grant_table.c:289:d0 Increased maptrack size to 26 frames >>> (XEN) [2014-02-18 04:16:06] grant_table.c:289:d0 Increased maptrack size to 27 frames >>> (XEN) [2014-02-18 04:16:12] grant_table.c:289:d0 Increased maptrack size to 28 frames >>> (XEN) [2014-02-18 04:16:18] grant_table.c:289:d0 Increased maptrack size to 29 frames >>> (XEN) [2014-02-18 04:17:00] grant_table.c:1857:d0 Bad grant reference 268435460 >>> (XEN) [2014-02-18 04:17:00] grant_table.c:1857:d0 Bad grant reference 268435460 >>> (XEN) [2014-02-18 04:34:03] grant_table.c:1857:d0 Bad grant reference 4325377 >>> >>> >>> >>> Another issue with networking is when running both dom0 and domU's with a 3.14-rc3 kernel: >>> - i can ping the guests from dom0 >>> - i can ping dom0 from the guests >>> - But i can't ssh or access things by http >>> - I don't see any relevant error messages ... >>> - This is with the same system and kernel config as with the 3.14 and 3.13 combination above >>> (that previously worked fine) >>> >>> -- >>> >>> Sander >>> >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xen.org >>> http://lists.xen.org/xen-devel > >