From mboxrd@z Thu Jan 1 00:00:00 1970 From: Joseph Gasparakis Subject: Re: vxlan/veth performance issues on net.git + latest kernels Date: Fri, 6 Dec 2013 02:30:54 -0800 (PST) Message-ID: References: <529DF340.70602@mellanox.com> <1386084620.30495.28.camel@edumazet-glaptop2.roam.corp.google.com> <52A197DF.5010806@mellanox.com> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Joseph Gasparakis , Pravin B Shelar , Or Gerlitz , Eric Dumazet , Jerry Chu , Eric Dumazet , Alexei Starovoitov , David Miller , netdev , jeffrey.t.kirsher@intel.com, John Fastabend To: Or Gerlitz Return-path: Received: from mga02.intel.com ([134.134.136.20]:36753 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754029Ab3LFKNY (ORCPT ); Fri, 6 Dec 2013 05:13:24 -0500 In-Reply-To: <52A197DF.5010806@mellanox.com> Sender: netdev-owner@vger.kernel.org List-ID: On Fri, 6 Dec 2013, Or Gerlitz wrote: > On 04/12/2013 11:41, Or Gerlitz wrote: > > On Wed, Dec 4, 2013 at 11:24 AM, Joseph > > Gasparakis wrote: > > > >And just for the record, > > > >you are seeing (SKB_UDP_TUNNEL | SKB_GSO_TCPV4) as 0x201 while I was > > > >seeing it as 0x81 because commit 61c1db7fae "ipv6: sit: add GSO/TSO > > > >support" pushed the SKB_UDP_TUNNEL two bits left, and I had done my tests > > > before it. > > indeed, also, on what kernel did you conducted your tests which you managed > > to WA the problem with unsetting that bit? > > > Hi Joseph, > > Really need your response here -- I'm sorry Or, I managed to miss your original request... > > 1. on which kernel did you manage to get along fine vxlan performance wise > with this hack? > I was running 3.10.6. > 2. did the hack helped for both veth host traffic or only on PV VM traffic? > No, just VM. I haven't tried veth. If you leave the DODGY bit, does your traffic get droped on Tx, after it leaves vxlan and before it hits your driver, which is what I had seen. Is that right? If you unset it, do you recover? What is the output of your ethtool -k on the interface you are transmitting from? > Currently it doesn't converge with 3.12.x or net.git, with veth/vxlan the > DODGE bit isn't set when looking on the skb in the vxlan xmit time, so there's > nothing for me to hack there. For VMs without unsetting the bit things don't > really work, but unsetting it for itself so far didn't get me far performance > wise. > > BTW guys, I saw the issues with both bridge/openvswitch configuration - seems > that we might have here somehow large breakage of the system w.r.t vxlan > traffic for rates that go over few Gbs -- so would love to get feedback of any > kind from the people that were involved with vxlan over the last months/year. > > Or. > > net.git]# grep -rn SKB_GSO_DODGY drivers/net/ net/ipv4 net/core > drivers/net/macvtap.c:585: skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY; > drivers/net/tun.c:1135: skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY; > drivers/net/virtio_net.c:497: skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY; > drivers/net/xen-netback/netback.c:1146: skb_shinfo(skb)->gso_type |= > SKB_GSO_DODGY; > drivers/net/xen-netfront.c:823: skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY; > net/ipv4/af_inet.c:1264: SKB_GSO_DODGY | > net/ipv4/tcp_offload.c:56: SKB_GSO_DODGY | > net/ipv4/gre_offload.c:40: SKB_GSO_DODGY | > net/ipv4/udp_offload.c:53: if (unlikely(type & ~(SKB_GSO_UDP | > SKB_GSO_DODGY | > net/core/dev.c:2694: if (shinfo->gso_type & SKB_GSO_DODGY) > > >