netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* sit tunnel slow since added GSO/TSO support
@ 2013-12-13 16:13 Michal Schmidt
  2013-12-13 16:28 ` Eric Dumazet
  2013-12-16 14:31 ` [PATCH] ipv6: sit: update mtu check to take care of gso packets Eric Dumazet
  0 siblings, 2 replies; 11+ messages in thread
From: Michal Schmidt @ 2013-12-13 16:13 UTC (permalink / raw)
  To: netdev@vger.kernel.org; +Cc: Eric Dumazet

Hello,

I've been experimenting with IPIP and SIT tunnels. Both of them
had the support for GSO/TSO added recently, by commits:
  cb32f511a7 "ipip: add GSO/TSO support"
  61c1db7fae "ipv6: sit: add GSO/TSO support"

IPIP works fine for me.

I am seeing very low TCP performance with SIT if TSO is enabled on the
SIT interface. It does not matter if the underlying hardware supports
NETIF_F_GSO_SIT or not (I have tested with bnx2x and tg3).

This is for example with TSO on a 1Gb interface:

  $ netperf -H 2001:0DB8:1234::000e
  MIGRATED TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 2001:0DB8:1234::000e () port 0 AF_INET6
  Recv   Send    Send                          
  Socket Socket  Message  Elapsed              
  Size   Size    Size     Time     Throughput  
  bytes  bytes   bytes    secs.    10^6bits/sec  
  
   87380  16384  16384    10.00      35.92

After "ethtool -K sittun tso off":

  $ netperf -H 2001:0DB8:1234::000e
  MIGRATED TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 2001:0DB8:1234::000e () port 0 AF_INET6
  Recv   Send    Send                          
  Socket Socket  Message  Elapsed              
  Size   Size    Size     Time     Throughput  
  bytes  bytes   bytes    secs.    10^6bits/sec  
  
   87380  16384  16384    10.01     706.86

After looking into what was happening I discovered that in the
TSO-enabled case big skbs enters into ipip6_tunnel_xmit() where they are
dropped by this branch:

        if (df) {
                /* ... */
                if (skb->len > mtu) {
                        icmpv6_send(skb, ICMPV6_PKT_TOOBIG, 0, mtu);
                        ip_rt_put(rt);
                        goto tx_error;
                }
        }

After having looked at the meaning of "df" I found that I can use
the "nopmtudisc" option when creating the sit tunnel to work around
the problem:
  ip tun add sittun mode sit remote $REMOTE local $LOCAL dev $IFACE nopmtudisc

With "nopmtudisc" and TSO enabled:

  $ netperf -H 2001:0DB8:1234::000e
  MIGRATED TCP STREAM TEST from ::0 (::) port 0 AF_INET6 to 2001:0DB8:1234::000e () port 0 AF_INET6
  Recv   Send    Send                          
  Socket Socket  Message  Elapsed              
  Size   Size    Size     Time     Throughput  
  bytes  bytes   bytes    secs.    10^6bits/sec  
  
   87380  16384  16384    10.01     716.85 

Regards,
Michal

^ permalink raw reply	[flat|nested] 11+ messages in thread
end of thread, other threads:[~2013-12-18 22:57 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-12-13 16:13 sit tunnel slow since added GSO/TSO support Michal Schmidt
2013-12-13 16:28 ` Eric Dumazet
2013-12-13 22:54   ` Eric Dumazet
2013-12-14 21:25     ` Michal Schmidt
2013-12-15  1:23       ` Eric Dumazet
2013-12-16 14:31 ` [PATCH] ipv6: sit: update mtu check to take care of gso packets Eric Dumazet
2013-12-17 14:22   ` Michal Schmidt
2013-12-17 14:48     ` Eric Dumazet
2013-12-17 20:19   ` Hannes Frederic Sowa
2013-12-17 21:34     ` Eric Dumazet
2013-12-18 22:57   ` David Miller

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).