netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Florian Westphal <fw@strlen.de>
To: netdev@vger.kernel.org
Cc: amirv@mellanox.com, ogerlitz@mellanox.com
Subject: mlx4+vxlan offload breaks gre tunnels
Date: Wed, 5 Nov 2014 16:04:48 +0100	[thread overview]
Message-ID: <20141105150448.GA20776@breakpoint.cc> (raw)

tl,dr: all tcp packets sent via gre tunnel have broken tcp csum if vxlan offload
is enabled with mlx4 driver.

Given following config on tx-side:
dev=enp3s0
ip addr add dev $dev 192.168.23.1/24
ip link set $dev up
ip link add mygre type gretap remote 192.168.23.2 local 192.168.23.1
ip addr add dev mygre 192.168.42.1/24
ip link set gre0 up
ip link set mygre up

and

options mlx4_core log_num_mgm_entry_size=-1 debug_level=1
port_type_array=2,2

in
/etc/modprobe.d/mlx4.conf

all tcp packets sent to destinations over the gre tunnel have bogus tcp
checksums (and are tossed on rx side when stack validates tcp checksum).

net-next head is commit 30349bdbc4da5ecf0efa25556e3caff9c9b8c5f7 .

What makes things work for me:
either

options mlx4_core 1 debug_level=1 port_type_array=2,2

(ie. no MLX4_TUNNEL_OFFLOAD_MODE_VXLAN)

or not setting NETIF_F_IP_CSUM in enc_features:

--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -2579,10 +2579,12 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
                dev->priv_flags |= IFF_UNICAST_FLT;
 
        if (mdev->dev->caps.tunnel_offload_mode == MLX4_TUNNEL_OFFLOAD_MODE_VXLAN) {
-               dev->hw_enc_features |= NETIF_F_IP_CSUM | NETIF_F_RXCSUM |
+               dev->hw_enc_features |= NETIF_F_RXCSUM |
                                        NETIF_F_TSO | NETIF_F_GSO_UDP_TUNNEL;

I am not sure if its right fix, but to my eyes this basically looks like
mlx4 is telling stack that it can handle tcp checksum offload within
tunnels, and that doesn't seem to be the case for all types (e.g. gre).

Could someone who understand the enc_features specifics better confirm that
above patch is correct (or provide a better/proper fix)?

Thanks,
Florian

             reply	other threads:[~2014-11-05 15:04 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-11-05 15:04 Florian Westphal [this message]
2014-11-05 16:17 ` mlx4+vxlan offload breaks gre tunnels Or Gerlitz
2014-11-05 16:53   ` Florian Westphal
2014-11-06 16:30     ` Or Gerlitz
2014-11-05 21:59   ` Tom Herbert
2014-11-06  7:03     ` Or Gerlitz
2014-11-06  7:21   ` Sathya Perla
2014-11-06  7:53     ` Or Gerlitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141105150448.GA20776@breakpoint.cc \
    --to=fw@strlen.de \
    --cc=amirv@mellanox.com \
    --cc=netdev@vger.kernel.org \
    --cc=ogerlitz@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).