[pull request][net-next 00/13] Mellanox, mlx5 updates 2018-07-27 (Vxlan updates)

[pull request][net-next 00/13] Mellanox, mlx5 updates 2018-07-27 (Vxlan updates)

[Saeed Mahameed]

Hi Dave,

This series from Gal and Saeed provides updates to mlx5 vxlan implementation.

For more information please see tag log below.

Please pull and let me know if there's any problem.

Thanks,
Saeed.

---

The following changes since commit 1f3ed383fb9a073ae2e408cd7a0717b04c7c3a21:

  net: sched: don't dump chains only held by actions (2018-07-27 09:38:46 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5e-updates-2018-07-27

for you to fetch changes up to 50920f3f268405af2b047ae9277efdbf4ed7510f:

  net/mlx5e: Issue direct lookup on vxlan ports by vport representors (2018-07-27 13:57:55 -0700)

----------------------------------------------------------------
mlx5e-updates-2018-07-27 (Vxlan updates)

This series from Gal and Saeed provides updates to mlx5 vxlan implementation.

Gal, started with three cleanups to reflect the actual hardware vxlan state
- reflect 4789 UDP port default addition to software database
- check maximum number of vxlan  UDP ports
- cleanup an unused member in vxlan work

Then Gal provides two performance optimizations by replacing the
vxlan radix tree with a hash table, and replacing the vxlan table
spin lock with a read-write lock.

Measuring mlx5e_vxlan_lookup_port execution time:

                      Radix Tree   Hash Table
     --------------- ------------ ------------
      Single Stream   161 ns       79  ns (51% improvement)
      Multi Stream    259 ns       136 ns (47% improvement)

    Measuring UDP stream packet rate, single fully utilized TX core:
    Radix Tree: 498,300 PPS
    Hash Table: 555,468 PPS (11% improvement)

Next, from Saeed, vxlan refactoring to allow sharing the vxlan table
between different mlx5 netdevice instances like PF and VF representors,
this is done by making mlx5 vxlan interface more generic and decoupling
it from PF netdevice structures and logic, then moving it into mlx5 core
as a low level interface so it can be used by VF representors, which is
illustrated in the last patch of the serious.

-Saeed.

----------------------------------------------------------------
Gal Pressman (5):
      net/mlx5e: Vxlan, reflect 4789 UDP port default addition to software database
      net/mlx5e: Vxlan, check maximum number of UDP ports
      net/mlx5e: Vxlan, replace ports radix-tree with hash table
      net/mlx5e: Vxlan, replace spinlock with read-write lock
      net/mlx5e: Vxlan, cleanup an unused member in vxlan work

Saeed Mahameed (8):
      net/mlx5e: Vxlan, add direct delete function
      net/mlx5e: Vxlan, move netdev only logic to en_main.c
      net/mlx5e: Vxlan, rename struct mlx5e_vxlan to mlx5_vxlan_port
      net/mlx5e: Vxlan, rename from mlx5e to mlx5
      net/mlx5e: Vxlan, return values for add/del port
      net/mlx5e: Vxlan, add sync lock for add/del vxlan port
      net/mlx5e: Vxlan, move vxlan logic to core driver
      net/mlx5e: Issue direct lookup on vxlan ports by vport representors

 drivers/net/ethernet/mellanox/mlx5/core/Makefile   |   4 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h       |   6 -
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c  |  71 ++++++-
 drivers/net/ethernet/mellanox/mlx5/core/en_tc.c    |  14 +-
 .../net/ethernet/mellanox/mlx5/core/lib/vxlan.c    | 230 +++++++++++++++++++++
 .../ethernet/mellanox/mlx5/core/{ => lib}/vxlan.h  |  39 ++--
 drivers/net/ethernet/mellanox/mlx5/core/main.c     |   5 +
 drivers/net/ethernet/mellanox/mlx5/core/vxlan.c    | 190 -----------------
 include/linux/mlx5/driver.h                        |   2 +
 include/linux/mlx5/mlx5_ifc.h                      |   4 +-
 10 files changed, 325 insertions(+), 240 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.c
 rename drivers/net/ethernet/mellanox/mlx5/core/{ => lib}/vxlan.h (66%)
 delete mode 100644 drivers/net/ethernet/mellanox/mlx5/core/vxlan.c

[net-next 01/13] net/mlx5e: Vxlan, reflect 4789 UDP port default addition to software database

[Saeed Mahameed]

From: Gal Pressman <galp@mellanox.com>

The hardware offloads 4789 UDP port (default VXLAN port) automatically.
Add it to the software database as well in order to reflect the hardware
state appropriately.

Signed-off-by: Gal Pressman <galp@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/vxlan.c   | 40 +++++++++++++------
 1 file changed, 27 insertions(+), 13 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 2f74953e4561..2f699998d13e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -36,12 +36,20 @@
 #include "mlx5_core.h"
 #include "vxlan.h"
 
+static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port);
+
 void mlx5e_vxlan_init(struct mlx5e_priv *priv)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 
 	spin_lock_init(&vxlan_db->lock);
 	INIT_RADIX_TREE(&vxlan_db->tree, GFP_ATOMIC);
+
+	if (mlx5e_vxlan_allowed(priv->mdev))
+		/* Hardware adds 4789 by default.
+		 * Lockless since we are the only hash table consumers, wq and TX are disabled.
+		 */
+		mlx5e_vxlan_add_port(priv, 4789);
 }
 
 static int mlx5e_vxlan_core_add_port_cmd(struct mlx5_core_dev *mdev, u16 port)
@@ -78,25 +86,20 @@ struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
 	return vxlan;
 }
 
-static void mlx5e_vxlan_add_port(struct work_struct *work)
+static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 {
-	struct mlx5e_vxlan_work *vxlan_work =
-		container_of(work, struct mlx5e_vxlan_work, work);
-	struct mlx5e_priv *priv = vxlan_work->priv;
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	u16 port = vxlan_work->port;
 	struct mlx5e_vxlan *vxlan;
 	int err;
 
-	mutex_lock(&priv->state_lock);
 	vxlan = mlx5e_vxlan_lookup_port(priv, port);
 	if (vxlan) {
 		atomic_inc(&vxlan->refcount);
-		goto free_work;
+		return;
 	}
 
 	if (mlx5e_vxlan_core_add_port_cmd(priv->mdev, port))
-		goto free_work;
+		return;
 
 	vxlan = kzalloc(sizeof(*vxlan), GFP_KERNEL);
 	if (!vxlan)
@@ -111,18 +114,29 @@ static void mlx5e_vxlan_add_port(struct work_struct *work)
 	if (err)
 		goto err_free;
 
-	goto free_work;
+	return;
 
 err_free:
 	kfree(vxlan);
 err_delete_port:
 	mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
-free_work:
+}
+
+static void mlx5e_vxlan_add_work(struct work_struct *work)
+{
+	struct mlx5e_vxlan_work *vxlan_work =
+		container_of(work, struct mlx5e_vxlan_work, work);
+	struct mlx5e_priv *priv = vxlan_work->priv;
+	u16 port = vxlan_work->port;
+
+	mutex_lock(&priv->state_lock);
+	mlx5e_vxlan_add_port(priv, port);
 	mutex_unlock(&priv->state_lock);
+
 	kfree(vxlan_work);
 }
 
-static void mlx5e_vxlan_del_port(struct work_struct *work)
+static void mlx5e_vxlan_del_work(struct work_struct *work)
 {
 	struct mlx5e_vxlan_work *vxlan_work =
 		container_of(work, struct mlx5e_vxlan_work, work);
@@ -164,9 +178,9 @@ void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, sa_family_t sa_family,
 		return;
 
 	if (add)
-		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_add_port);
+		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_add_work);
 	else
-		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_del_port);
+		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_del_work);
 
 	vxlan_work->priv = priv;
 	vxlan_work->port = port;
-- 
2.17.0

[net-next 02/13] net/mlx5e: Vxlan, check maximum number of UDP ports

[Saeed Mahameed]

From: Gal Pressman <galp@mellanox.com>

The NIC has a limited number of offloaded VXLAN UDP ports (usually 4).
Instead of letting the firmware fail when trying to add more ports than
it can handle, let the driver check it on its own.

Signed-off-by: Gal Pressman <galp@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h    |  1 +
 drivers/net/ethernet/mellanox/mlx5/core/vxlan.c | 14 ++++++++++++++
 include/linux/mlx5/mlx5_ifc.h                   |  4 +++-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index c41cfc2a4b70..c4d4db8722f5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -657,6 +657,7 @@ enum {
 struct mlx5e_vxlan_db {
 	spinlock_t			lock; /* protect vxlan table */
 	struct radix_tree_root		tree;
+	int				num_ports;
 };
 
 struct mlx5e_l2_rule {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 2f699998d13e..e3af2efe18ce 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -52,6 +52,11 @@ void mlx5e_vxlan_init(struct mlx5e_priv *priv)
 		mlx5e_vxlan_add_port(priv, 4789);
 }
 
+static inline u8 mlx5e_vxlan_max_udp_ports(struct mlx5_core_dev *mdev)
+{
+	return MLX5_CAP_ETH(mdev, max_vxlan_udp_ports) ?: 4;
+}
+
 static int mlx5e_vxlan_core_add_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 {
 	u32 in[MLX5_ST_SZ_DW(add_vxlan_udp_dport_in)]   = {0};
@@ -98,6 +103,13 @@ static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 		return;
 	}
 
+	if (vxlan_db->num_ports >= mlx5e_vxlan_max_udp_ports(priv->mdev)) {
+		netdev_info(priv->netdev,
+			    "UDP port (%d) not offloaded, max number of UDP ports (%d) are already offloaded\n",
+			    port, mlx5e_vxlan_max_udp_ports(priv->mdev));
+		return;
+	}
+
 	if (mlx5e_vxlan_core_add_port_cmd(priv->mdev, port))
 		return;
 
@@ -114,6 +126,7 @@ static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 	if (err)
 		goto err_free;
 
+	vxlan_db->num_ports++;
 	return;
 
 err_free:
@@ -163,6 +176,7 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 	if (remove) {
 		mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
 		kfree(vxlan);
+		vxlan_db->num_ports--;
 	}
 	mutex_unlock(&priv->state_lock);
 	kfree(vxlan_work);
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 22f54bedfaae..60c2308fe062 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -668,7 +668,9 @@ struct mlx5_ifc_per_protocol_networking_offload_caps_bits {
 	u8         swp[0x1];
 	u8         swp_csum[0x1];
 	u8         swp_lso[0x1];
-	u8         reserved_at_23[0x1b];
+	u8         reserved_at_23[0xd];
+	u8         max_vxlan_udp_ports[0x8];
+	u8         reserved_at_38[0x6];
 	u8         max_geneve_opt_len[0x1];
 	u8         tunnel_stateless_geneve_rx[0x1];
 
-- 
2.17.0

[net-next 03/13] net/mlx5e: Vxlan, replace ports radix-tree with hash table

[Saeed Mahameed]

From: Gal Pressman <galp@mellanox.com>

The VXLAN database is accessed in the data path for each VXLAN TX skb in
order to check whether the UDP port is being offloaded or not.
The number of elements in the database is relatively small, we can
simplify the radix-tree to a hash table and speedup the lookup process.

Measuring mlx5e_vxlan_lookup_port execution time:

                  Radix Tree   Hash Table
 --------------- ------------ ------------
  Single Stream   161 ns       79  ns (51% improvement)
  Multi Stream    259 ns       136 ns (47% improvement)

Measuring UDP stream packet rate, single fully utilized TX core:
Radix Tree: 498,300 PPS
Hash Table: 555,468 PPS (11% improvement)

Signed-off-by: Gal Pressman <galp@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  3 +-
 .../net/ethernet/mellanox/mlx5/core/vxlan.c   | 41 +++++++++++--------
 .../net/ethernet/mellanox/mlx5/core/vxlan.h   |  1 +
 3 files changed, 28 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index c4d4db8722f5..6878925c3abf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -656,7 +656,8 @@ enum {
 
 struct mlx5e_vxlan_db {
 	spinlock_t			lock; /* protect vxlan table */
-	struct radix_tree_root		tree;
+	/* max_num_ports is usuallly 4, 16 buckets is more than enough */
+	DECLARE_HASHTABLE(htable, 4);
 	int				num_ports;
 };
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index e3af2efe18ce..3c0ea9bc20e3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -43,7 +43,7 @@ void mlx5e_vxlan_init(struct mlx5e_priv *priv)
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 
 	spin_lock_init(&vxlan_db->lock);
-	INIT_RADIX_TREE(&vxlan_db->tree, GFP_ATOMIC);
+	hash_init(vxlan_db->htable);
 
 	if (mlx5e_vxlan_allowed(priv->mdev))
 		/* Hardware adds 4789 by default.
@@ -79,13 +79,27 @@ static int mlx5e_vxlan_core_del_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 	return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
 }
 
+static struct mlx5e_vxlan *mlx5e_vxlan_lookup_port_locked(struct mlx5e_priv *priv,
+							  u16 port)
+{
+	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
+	struct mlx5e_vxlan    *vxlan;
+
+	hash_for_each_possible(vxlan_db->htable, vxlan, hlist, port) {
+		if (vxlan->udp_port == port)
+			return vxlan;
+	}
+
+	return NULL;
+}
+
 struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5e_vxlan *vxlan;
 
 	spin_lock_bh(&vxlan_db->lock);
-	vxlan = radix_tree_lookup(&vxlan_db->tree, port);
+	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
 	spin_unlock_bh(&vxlan_db->lock);
 
 	return vxlan;
@@ -95,7 +109,6 @@ static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5e_vxlan *vxlan;
-	int err;
 
 	vxlan = mlx5e_vxlan_lookup_port(priv, port);
 	if (vxlan) {
@@ -121,16 +134,12 @@ static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 	atomic_set(&vxlan->refcount, 1);
 
 	spin_lock_bh(&vxlan_db->lock);
-	err = radix_tree_insert(&vxlan_db->tree, vxlan->udp_port, vxlan);
+	hash_add(vxlan_db->htable, &vxlan->hlist, port);
 	spin_unlock_bh(&vxlan_db->lock);
-	if (err)
-		goto err_free;
 
 	vxlan_db->num_ports++;
 	return;
 
-err_free:
-	kfree(vxlan);
 err_delete_port:
 	mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
 }
@@ -161,12 +170,12 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 
 	mutex_lock(&priv->state_lock);
 	spin_lock_bh(&vxlan_db->lock);
-	vxlan = radix_tree_lookup(&vxlan_db->tree, port);
+	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
 	if (!vxlan)
 		goto out_unlock;
 
 	if (atomic_dec_and_test(&vxlan->refcount)) {
-		radix_tree_delete(&vxlan_db->tree, port);
+		hash_del(&vxlan->hlist);
 		remove = true;
 	}
 
@@ -206,13 +215,13 @@ void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5e_vxlan *vxlan;
-	unsigned int port = 0;
+	struct hlist_node *tmp;
+	int bkt;
 
-	/* Lockless since we are the only radix-tree consumers, wq is disabled */
-	while (radix_tree_gang_lookup(&vxlan_db->tree, (void **)&vxlan, port, 1)) {
-		port = vxlan->udp_port;
-		radix_tree_delete(&vxlan_db->tree, port);
-		mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
+	/* Lockless since we are the only hash table consumers, wq and TX are disabled */
+	hash_for_each_safe(vxlan_db->htable, bkt, tmp, vxlan, hlist) {
+		hash_del(&vxlan->hlist);
+		mlx5e_vxlan_core_del_port_cmd(priv->mdev, vxlan->udp_port);
 		kfree(vxlan);
 	}
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
index 5ef6ae7d568a..52c41c22235d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
@@ -36,6 +36,7 @@
 #include "en.h"
 
 struct mlx5e_vxlan {
+	struct hlist_node hlist;
 	atomic_t refcount;
 	u16 udp_port;
 };
-- 
2.17.0

[net-next 04/13] net/mlx5e: Vxlan, replace spinlock with read-write lock

[Saeed Mahameed]

From: Gal Pressman <galp@mellanox.com>

The VXLAN database is mainly used by readers in data path, and rarely
used by control path writers.
Multiple readers (threads) should not block each other and cause an
unnecessary contention on the lock.

Replacing the spinlock with rwlock optimizes the common use case where
adding ports to the table (adding VXLAN interfaces) is quite rare, but
the table is accessed for each VXLAN TX skb.

Signed-off-by: Gal Pressman <galp@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h    |  2 +-
 drivers/net/ethernet/mellanox/mlx5/core/vxlan.c | 14 +++++++-------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 6878925c3abf..870ac617550c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -655,7 +655,7 @@ enum {
 };
 
 struct mlx5e_vxlan_db {
-	spinlock_t			lock; /* protect vxlan table */
+	rwlock_t			lock; /* protect vxlan table */
 	/* max_num_ports is usuallly 4, 16 buckets is more than enough */
 	DECLARE_HASHTABLE(htable, 4);
 	int				num_ports;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 3c0ea9bc20e3..2733ca63e46b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -42,7 +42,7 @@ void mlx5e_vxlan_init(struct mlx5e_priv *priv)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 
-	spin_lock_init(&vxlan_db->lock);
+	rwlock_init(&vxlan_db->lock);
 	hash_init(vxlan_db->htable);
 
 	if (mlx5e_vxlan_allowed(priv->mdev))
@@ -98,9 +98,9 @@ struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5e_vxlan *vxlan;
 
-	spin_lock_bh(&vxlan_db->lock);
+	read_lock_bh(&vxlan_db->lock);
 	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
-	spin_unlock_bh(&vxlan_db->lock);
+	read_unlock_bh(&vxlan_db->lock);
 
 	return vxlan;
 }
@@ -133,9 +133,9 @@ static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 	vxlan->udp_port = port;
 	atomic_set(&vxlan->refcount, 1);
 
-	spin_lock_bh(&vxlan_db->lock);
+	write_lock_bh(&vxlan_db->lock);
 	hash_add(vxlan_db->htable, &vxlan->hlist, port);
-	spin_unlock_bh(&vxlan_db->lock);
+	write_unlock_bh(&vxlan_db->lock);
 
 	vxlan_db->num_ports++;
 	return;
@@ -169,7 +169,7 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 	bool remove = false;
 
 	mutex_lock(&priv->state_lock);
-	spin_lock_bh(&vxlan_db->lock);
+	write_lock_bh(&vxlan_db->lock);
 	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
 	if (!vxlan)
 		goto out_unlock;
@@ -180,7 +180,7 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 	}
 
 out_unlock:
-	spin_unlock_bh(&vxlan_db->lock);
+	write_unlock_bh(&vxlan_db->lock);
 
 	if (remove) {
 		mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
-- 
2.17.0

Re: [net-next 04/13] net/mlx5e: Vxlan, replace spinlock with read-write lock

[Stephen Hemminger]

On Fri, 27 Jul 2018 14:15:09 -0700
Saeed Mahameed <saeedm@mellanox.com> wrote:

> From: Gal Pressman <galp@mellanox.com>
> 
> The VXLAN database is mainly used by readers in data path, and rarely
> used by control path writers.
> Multiple readers (threads) should not block each other and cause an
> unnecessary contention on the lock.
> 
> Replacing the spinlock with rwlock optimizes the common use case where
> adding ports to the table (adding VXLAN interfaces) is quite rare, but
> the table is accessed for each VXLAN TX skb.
> 
> Signed-off-by: Gal Pressman <galp@mellanox.com>
> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>

Did you know that for small sections a spinlock is significantly faster than
a reader-writer lock. It turns out that reader-writer locks the reader
creates a cache line bounce.

https://www.kernel.org/doc/Documentation/locking/spinlocks.txt


Lesson 2: reader-writer spinlocks.

If your data accesses have a very natural pattern where you usually tend
to mostly read from the shared variables, the reader-writer locks
(rw_lock) versions of the spinlocks are sometimes useful. They allow multiple
readers to be in the same critical region at once, but if somebody wants
to change the variables it has to get an exclusive write lock.

   NOTE! reader-writer locks require more atomic memory operations than
   simple spinlocks.  Unless the reader critical section is long, you
   are better off just using spinlocks.

Re: [net-next 04/13] net/mlx5e: Vxlan, replace spinlock with read-write lock

[Saeed Mahameed]

On Fri, Jul 27, 2018 at 2:48 PM, Stephen Hemminger
<stephen@networkplumber.org> wrote:
> On Fri, 27 Jul 2018 14:15:09 -0700
> Saeed Mahameed <saeedm@mellanox.com> wrote:
>
>> From: Gal Pressman <galp@mellanox.com>
>>
>> The VXLAN database is mainly used by readers in data path, and rarely
>> used by control path writers.
>> Multiple readers (threads) should not block each other and cause an
>> unnecessary contention on the lock.
>>
>> Replacing the spinlock with rwlock optimizes the common use case where
>> adding ports to the table (adding VXLAN interfaces) is quite rare, but
>> the table is accessed for each VXLAN TX skb.
>>
>> Signed-off-by: Gal Pressman <galp@mellanox.com>
>> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
>
> Did you know that for small sections a spinlock is significantly faster than
> a reader-writer lock. It turns out that reader-writer locks the reader
> creates a cache line bounce.
>
> https://www.kernel.org/doc/Documentation/locking/spinlocks.txt
>
>
> Lesson 2: reader-writer spinlocks.
>
> If your data accesses have a very natural pattern where you usually tend
> to mostly read from the shared variables, the reader-writer locks
> (rw_lock) versions of the spinlocks are sometimes useful. They allow multiple
> readers to be in the same critical region at once, but if somebody wants
> to change the variables it has to get an exclusive write lock.
>
>    NOTE! reader-writer locks require more atomic memory operations than
>    simple spinlocks.  Unless the reader critical section is long, you
>    are better off just using spinlocks.

Thanks Stephen, very usefull information !
I will drop this patch for now, I will consider  using rcu lock
instead in a future patch.

[net-next 05/13] net/mlx5e: Vxlan, cleanup an unused member in vxlan work

[Saeed Mahameed]

From: Gal Pressman <galp@mellanox.com>

Cleanup the sa_family member of the vxlan work, it is unused/needed
anywhere in the code.

Signed-off-by: Gal Pressman <galp@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 ++--
 drivers/net/ethernet/mellanox/mlx5/core/vxlan.c   | 4 +---
 drivers/net/ethernet/mellanox/mlx5/core/vxlan.h   | 4 +---
 3 files changed, 4 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index fad947079a43..14a201cbb0a4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -3980,7 +3980,7 @@ static void mlx5e_add_vxlan_port(struct net_device *netdev,
 	if (!mlx5e_vxlan_allowed(priv->mdev))
 		return;
 
-	mlx5e_vxlan_queue_work(priv, ti->sa_family, be16_to_cpu(ti->port), 1);
+	mlx5e_vxlan_queue_work(priv, be16_to_cpu(ti->port), 1);
 }
 
 static void mlx5e_del_vxlan_port(struct net_device *netdev,
@@ -3994,7 +3994,7 @@ static void mlx5e_del_vxlan_port(struct net_device *netdev,
 	if (!mlx5e_vxlan_allowed(priv->mdev))
 		return;
 
-	mlx5e_vxlan_queue_work(priv, ti->sa_family, be16_to_cpu(ti->port), 0);
+	mlx5e_vxlan_queue_work(priv, be16_to_cpu(ti->port), 0);
 }
 
 static netdev_features_t mlx5e_tunnel_features_check(struct mlx5e_priv *priv,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 2733ca63e46b..4a86d8132fc1 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -191,8 +191,7 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 	kfree(vxlan_work);
 }
 
-void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, sa_family_t sa_family,
-			    u16 port, int add)
+void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, u16 port, int add)
 {
 	struct mlx5e_vxlan_work *vxlan_work;
 
@@ -207,7 +206,6 @@ void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, sa_family_t sa_family,
 
 	vxlan_work->priv = priv;
 	vxlan_work->port = port;
-	vxlan_work->sa_family = sa_family;
 	queue_work(priv->wq, &vxlan_work->work);
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
index 52c41c22235d..51f19e3e5784 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
@@ -44,7 +44,6 @@ struct mlx5e_vxlan {
 struct mlx5e_vxlan_work {
 	struct work_struct	work;
 	struct mlx5e_priv	*priv;
-	sa_family_t		sa_family;
 	u16			port;
 };
 
@@ -57,8 +56,7 @@ static inline bool mlx5e_vxlan_allowed(struct mlx5_core_dev *mdev)
 void mlx5e_vxlan_init(struct mlx5e_priv *priv);
 void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv);
 
-void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, sa_family_t sa_family,
-			    u16 port, int add);
+void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, u16 port, int add);
 struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port);
 
 #endif /* __MLX5_VXLAN_H__ */
-- 
2.17.0

[net-next 06/13] net/mlx5e: Vxlan, add direct delete function

[Saeed Mahameed]

Add direct vxlan delete function to be called from vxlan_delete_work.
Needed in downstream patch.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/vxlan.c    | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 4a86d8132fc1..64883ca451aa 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -158,17 +158,12 @@ static void mlx5e_vxlan_add_work(struct work_struct *work)
 	kfree(vxlan_work);
 }
 
-static void mlx5e_vxlan_del_work(struct work_struct *work)
+static void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 {
-	struct mlx5e_vxlan_work *vxlan_work =
-		container_of(work, struct mlx5e_vxlan_work, work);
-	struct mlx5e_priv *priv         = vxlan_work->priv;
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	u16 port = vxlan_work->port;
 	struct mlx5e_vxlan *vxlan;
 	bool remove = false;
 
-	mutex_lock(&priv->state_lock);
 	write_lock_bh(&vxlan_db->lock);
 	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
 	if (!vxlan)
@@ -187,6 +182,17 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 		kfree(vxlan);
 		vxlan_db->num_ports--;
 	}
+}
+
+static void mlx5e_vxlan_del_work(struct work_struct *work)
+{
+	struct mlx5e_vxlan_work *vxlan_work =
+		container_of(work, struct mlx5e_vxlan_work, work);
+	struct mlx5e_priv *priv         = vxlan_work->priv;
+	u16 port = vxlan_work->port;
+
+	mutex_lock(&priv->state_lock);
+	mlx5e_vxlan_del_port(priv, port);
 	mutex_unlock(&priv->state_lock);
 	kfree(vxlan_work);
 }
-- 
2.17.0

[net-next 07/13] net/mlx5e: Vxlan, move netdev only logic to en_main.c

[Saeed Mahameed]

Create a direct vxlan API to add and delete vxlan ports from HW.
+void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port);
+void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port);

And move vxlan_add/del_work to en_main.c since they are netdev only
logic.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 51 +++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/vxlan.c   | 55 +++----------------
 .../net/ethernet/mellanox/mlx5/core/vxlan.h   | 16 +-----
 3 files changed, 61 insertions(+), 61 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 14a201cbb0a4..7a6b78e3b5f7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -3969,6 +3969,57 @@ static int mlx5e_get_vf_stats(struct net_device *dev,
 }
 #endif
 
+struct mlx5e_vxlan_work {
+	struct work_struct	work;
+	struct mlx5e_priv	*priv;
+	u16			port;
+};
+
+static void mlx5e_vxlan_add_work(struct work_struct *work)
+{
+	struct mlx5e_vxlan_work *vxlan_work =
+		container_of(work, struct mlx5e_vxlan_work, work);
+	struct mlx5e_priv *priv = vxlan_work->priv;
+	u16 port = vxlan_work->port;
+
+	mutex_lock(&priv->state_lock);
+	mlx5e_vxlan_add_port(priv, port);
+	mutex_unlock(&priv->state_lock);
+
+	kfree(vxlan_work);
+}
+
+static void mlx5e_vxlan_del_work(struct work_struct *work)
+{
+	struct mlx5e_vxlan_work *vxlan_work =
+		container_of(work, struct mlx5e_vxlan_work, work);
+	struct mlx5e_priv *priv         = vxlan_work->priv;
+	u16 port = vxlan_work->port;
+
+	mutex_lock(&priv->state_lock);
+	mlx5e_vxlan_del_port(priv, port);
+	mutex_unlock(&priv->state_lock);
+	kfree(vxlan_work);
+}
+
+static void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, u16 port, int add)
+{
+	struct mlx5e_vxlan_work *vxlan_work;
+
+	vxlan_work = kmalloc(sizeof(*vxlan_work), GFP_ATOMIC);
+	if (!vxlan_work)
+		return;
+
+	if (add)
+		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_add_work);
+	else
+		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_del_work);
+
+	vxlan_work->priv = priv;
+	vxlan_work->port = port;
+	queue_work(priv->wq, &vxlan_work->work);
+}
+
 static void mlx5e_add_vxlan_port(struct net_device *netdev,
 				 struct udp_tunnel_info *ti)
 {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 64883ca451aa..2a25c2dd6e6f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -36,7 +36,11 @@
 #include "mlx5_core.h"
 #include "vxlan.h"
 
-static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port);
+struct mlx5e_vxlan {
+	struct hlist_node hlist;
+	atomic_t refcount;
+	u16 udp_port;
+};
 
 void mlx5e_vxlan_init(struct mlx5e_priv *priv)
 {
@@ -105,7 +109,7 @@ struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
 	return vxlan;
 }
 
-static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
+void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5e_vxlan *vxlan;
@@ -144,21 +148,7 @@ static void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 	mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
 }
 
-static void mlx5e_vxlan_add_work(struct work_struct *work)
-{
-	struct mlx5e_vxlan_work *vxlan_work =
-		container_of(work, struct mlx5e_vxlan_work, work);
-	struct mlx5e_priv *priv = vxlan_work->priv;
-	u16 port = vxlan_work->port;
-
-	mutex_lock(&priv->state_lock);
-	mlx5e_vxlan_add_port(priv, port);
-	mutex_unlock(&priv->state_lock);
-
-	kfree(vxlan_work);
-}
-
-static void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
+void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5e_vxlan *vxlan;
@@ -184,37 +174,6 @@ static void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 	}
 }
 
-static void mlx5e_vxlan_del_work(struct work_struct *work)
-{
-	struct mlx5e_vxlan_work *vxlan_work =
-		container_of(work, struct mlx5e_vxlan_work, work);
-	struct mlx5e_priv *priv         = vxlan_work->priv;
-	u16 port = vxlan_work->port;
-
-	mutex_lock(&priv->state_lock);
-	mlx5e_vxlan_del_port(priv, port);
-	mutex_unlock(&priv->state_lock);
-	kfree(vxlan_work);
-}
-
-void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, u16 port, int add)
-{
-	struct mlx5e_vxlan_work *vxlan_work;
-
-	vxlan_work = kmalloc(sizeof(*vxlan_work), GFP_ATOMIC);
-	if (!vxlan_work)
-		return;
-
-	if (add)
-		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_add_work);
-	else
-		INIT_WORK(&vxlan_work->work, mlx5e_vxlan_del_work);
-
-	vxlan_work->priv = priv;
-	vxlan_work->port = port;
-	queue_work(priv->wq, &vxlan_work->work);
-}
-
 void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
index 51f19e3e5784..1a02f5b38009 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
@@ -35,17 +35,7 @@
 #include <linux/mlx5/driver.h>
 #include "en.h"
 
-struct mlx5e_vxlan {
-	struct hlist_node hlist;
-	atomic_t refcount;
-	u16 udp_port;
-};
-
-struct mlx5e_vxlan_work {
-	struct work_struct	work;
-	struct mlx5e_priv	*priv;
-	u16			port;
-};
+struct mlx5e_vxlan;
 
 static inline bool mlx5e_vxlan_allowed(struct mlx5_core_dev *mdev)
 {
@@ -55,8 +45,8 @@ static inline bool mlx5e_vxlan_allowed(struct mlx5_core_dev *mdev)
 
 void mlx5e_vxlan_init(struct mlx5e_priv *priv);
 void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv);
-
-void mlx5e_vxlan_queue_work(struct mlx5e_priv *priv, u16 port, int add);
+void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port);
+void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port);
 struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port);
 
 #endif /* __MLX5_VXLAN_H__ */
-- 
2.17.0

[net-next 08/13] net/mlx5e: Vxlan, rename struct mlx5e_vxlan to mlx5_vxlan_port

[Saeed Mahameed]

The name mlx5e_vxlan will be used in downstream patch to describe
mlx5 vxlan structure that will replace mlx5e_vxlan_db.

Hence we rename struct mlx5e_vxlan to mlx5_vxlan_port which describes a
mlx5 vxlan port.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/vxlan.c   | 62 +++++++++----------
 .../net/ethernet/mellanox/mlx5/core/vxlan.h   |  4 +-
 2 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 2a25c2dd6e6f..95e3140da65c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -36,7 +36,7 @@
 #include "mlx5_core.h"
 #include "vxlan.h"
 
-struct mlx5e_vxlan {
+struct mlx5_vxlan_port {
 	struct hlist_node hlist;
 	atomic_t refcount;
 	u16 udp_port;
@@ -83,40 +83,40 @@ static int mlx5e_vxlan_core_del_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 	return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
 }
 
-static struct mlx5e_vxlan *mlx5e_vxlan_lookup_port_locked(struct mlx5e_priv *priv,
-							  u16 port)
+static struct mlx5_vxlan_port*
+mlx5e_vxlan_lookup_port_locked(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	struct mlx5e_vxlan    *vxlan;
+	struct mlx5_vxlan_port *vxlanp;
 
-	hash_for_each_possible(vxlan_db->htable, vxlan, hlist, port) {
-		if (vxlan->udp_port == port)
-			return vxlan;
+	hash_for_each_possible(vxlan_db->htable, vxlanp, hlist, port) {
+		if (vxlanp->udp_port == port)
+			return vxlanp;
 	}
 
 	return NULL;
 }
 
-struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
+struct mlx5_vxlan_port *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	struct mlx5e_vxlan *vxlan;
+	struct mlx5_vxlan_port *vxlanp;
 
 	read_lock_bh(&vxlan_db->lock);
-	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
+	vxlanp = mlx5e_vxlan_lookup_port_locked(priv, port);
 	read_unlock_bh(&vxlan_db->lock);
 
-	return vxlan;
+	return vxlanp;
 }
 
 void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	struct mlx5e_vxlan *vxlan;
+	struct mlx5_vxlan_port *vxlanp;
 
-	vxlan = mlx5e_vxlan_lookup_port(priv, port);
-	if (vxlan) {
-		atomic_inc(&vxlan->refcount);
+	vxlanp = mlx5e_vxlan_lookup_port(priv, port);
+	if (vxlanp) {
+		atomic_inc(&vxlanp->refcount);
 		return;
 	}
 
@@ -130,15 +130,15 @@ void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 	if (mlx5e_vxlan_core_add_port_cmd(priv->mdev, port))
 		return;
 
-	vxlan = kzalloc(sizeof(*vxlan), GFP_KERNEL);
-	if (!vxlan)
+	vxlanp = kzalloc(sizeof(*vxlanp), GFP_KERNEL);
+	if (!vxlanp)
 		goto err_delete_port;
 
-	vxlan->udp_port = port;
-	atomic_set(&vxlan->refcount, 1);
+	vxlanp->udp_port = port;
+	atomic_set(&vxlanp->refcount, 1);
 
 	write_lock_bh(&vxlan_db->lock);
-	hash_add(vxlan_db->htable, &vxlan->hlist, port);
+	hash_add(vxlan_db->htable, &vxlanp->hlist, port);
 	write_unlock_bh(&vxlan_db->lock);
 
 	vxlan_db->num_ports++;
@@ -151,16 +151,16 @@ void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	struct mlx5e_vxlan *vxlan;
+	struct mlx5_vxlan_port *vxlanp;
 	bool remove = false;
 
 	write_lock_bh(&vxlan_db->lock);
-	vxlan = mlx5e_vxlan_lookup_port_locked(priv, port);
-	if (!vxlan)
+	vxlanp = mlx5e_vxlan_lookup_port_locked(priv, port);
+	if (!vxlanp)
 		goto out_unlock;
 
-	if (atomic_dec_and_test(&vxlan->refcount)) {
-		hash_del(&vxlan->hlist);
+	if (atomic_dec_and_test(&vxlanp->refcount)) {
+		hash_del(&vxlanp->hlist);
 		remove = true;
 	}
 
@@ -169,7 +169,7 @@ void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 
 	if (remove) {
 		mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
-		kfree(vxlan);
+		kfree(vxlanp);
 		vxlan_db->num_ports--;
 	}
 }
@@ -177,14 +177,14 @@ void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv)
 {
 	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-	struct mlx5e_vxlan *vxlan;
+	struct mlx5_vxlan_port *vxlanp;
 	struct hlist_node *tmp;
 	int bkt;
 
 	/* Lockless since we are the only hash table consumers, wq and TX are disabled */
-	hash_for_each_safe(vxlan_db->htable, bkt, tmp, vxlan, hlist) {
-		hash_del(&vxlan->hlist);
-		mlx5e_vxlan_core_del_port_cmd(priv->mdev, vxlan->udp_port);
-		kfree(vxlan);
+	hash_for_each_safe(vxlan_db->htable, bkt, tmp, vxlanp, hlist) {
+		hash_del(&vxlanp->hlist);
+		mlx5e_vxlan_core_del_port_cmd(priv->mdev, vxlanp->udp_port);
+		kfree(vxlanp);
 	}
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
index 1a02f5b38009..6b38b6fbd030 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
@@ -35,7 +35,7 @@
 #include <linux/mlx5/driver.h>
 #include "en.h"
 
-struct mlx5e_vxlan;
+struct mlx5_vxlan_port;
 
 static inline bool mlx5e_vxlan_allowed(struct mlx5_core_dev *mdev)
 {
@@ -47,6 +47,6 @@ void mlx5e_vxlan_init(struct mlx5e_priv *priv);
 void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv);
 void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port);
 void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port);
-struct mlx5e_vxlan *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port);
+struct mlx5_vxlan_port *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port);
 
 #endif /* __MLX5_VXLAN_H__ */
-- 
2.17.0

[net-next 09/13] net/mlx5e: Vxlan, rename from mlx5e to mlx5

[Saeed Mahameed]

Rename vxlan functions from mlx5e_vxlan_* to mlx5_vxlan_*.
Rename mlx5e_vxlan_db to mlx5_vxlan and move it from en.h to vxlan.c
since it is not related to mlx5e anymore.

Allocate mlx5_vxlan structure dynamically in order to make it easier to
move later to core driver and to make it private in vxlan.c.

This is in preparation to move vxlan API to mlx5 core.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  10 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  21 ++--
 .../net/ethernet/mellanox/mlx5/core/en_tc.c   |   4 +-
 .../net/ethernet/mellanox/mlx5/core/vxlan.c   | 118 ++++++++++--------
 .../net/ethernet/mellanox/mlx5/core/vxlan.h   |  30 +++--
 5 files changed, 104 insertions(+), 79 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 870ac617550c..1bd4536b9061 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -52,6 +52,7 @@
 #include "wq.h"
 #include "mlx5_core.h"
 #include "en_stats.h"
+#include "vxlan.h"
 
 struct page_pool;
 
@@ -654,13 +655,6 @@ enum {
 	MLX5E_STATE_DESTROYING,
 };
 
-struct mlx5e_vxlan_db {
-	rwlock_t			lock; /* protect vxlan table */
-	/* max_num_ports is usuallly 4, 16 buckets is more than enough */
-	DECLARE_HASHTABLE(htable, 4);
-	int				num_ports;
-};
-
 struct mlx5e_l2_rule {
 	u8  addr[ETH_ALEN + 2];
 	struct mlx5_flow_handle *rule;
@@ -818,7 +812,7 @@ struct mlx5e_priv {
 	u32                        tx_rates[MLX5E_MAX_NUM_SQS];
 
 	struct mlx5e_flow_steering fs;
-	struct mlx5e_vxlan_db      vxlan;
+	struct mlx5_vxlan          *vxlan;
 
 	struct workqueue_struct    *wq;
 	struct work_struct         update_carrier_work;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 7a6b78e3b5f7..ef4b2f0c427c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -2974,7 +2974,7 @@ int mlx5e_open(struct net_device *netdev)
 		mlx5_set_port_admin_status(priv->mdev, MLX5_PORT_UP);
 	mutex_unlock(&priv->state_lock);
 
-	if (mlx5e_vxlan_allowed(priv->mdev))
+	if (mlx5_vxlan_allowed(priv->vxlan))
 		udp_tunnel_get_rx_info(netdev);
 
 	return err;
@@ -3983,7 +3983,7 @@ static void mlx5e_vxlan_add_work(struct work_struct *work)
 	u16 port = vxlan_work->port;
 
 	mutex_lock(&priv->state_lock);
-	mlx5e_vxlan_add_port(priv, port);
+	mlx5_vxlan_add_port(priv->vxlan, port);
 	mutex_unlock(&priv->state_lock);
 
 	kfree(vxlan_work);
@@ -3997,7 +3997,7 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 	u16 port = vxlan_work->port;
 
 	mutex_lock(&priv->state_lock);
-	mlx5e_vxlan_del_port(priv, port);
+	mlx5_vxlan_del_port(priv->vxlan, port);
 	mutex_unlock(&priv->state_lock);
 	kfree(vxlan_work);
 }
@@ -4028,7 +4028,7 @@ static void mlx5e_add_vxlan_port(struct net_device *netdev,
 	if (ti->type != UDP_TUNNEL_TYPE_VXLAN)
 		return;
 
-	if (!mlx5e_vxlan_allowed(priv->mdev))
+	if (!mlx5_vxlan_allowed(priv->vxlan))
 		return;
 
 	mlx5e_vxlan_queue_work(priv, be16_to_cpu(ti->port), 1);
@@ -4042,7 +4042,7 @@ static void mlx5e_del_vxlan_port(struct net_device *netdev,
 	if (ti->type != UDP_TUNNEL_TYPE_VXLAN)
 		return;
 
-	if (!mlx5e_vxlan_allowed(priv->mdev))
+	if (!mlx5_vxlan_allowed(priv->vxlan))
 		return;
 
 	mlx5e_vxlan_queue_work(priv, be16_to_cpu(ti->port), 0);
@@ -4076,7 +4076,7 @@ static netdev_features_t mlx5e_tunnel_features_check(struct mlx5e_priv *priv,
 		port = be16_to_cpu(udph->dest);
 
 		/* Verify if UDP port is being offloaded by HW */
-		if (mlx5e_vxlan_lookup_port(priv, port))
+		if (mlx5_vxlan_lookup_port(priv->vxlan, port))
 			return features;
 	}
 
@@ -4648,7 +4648,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 	netdev->hw_features      |= NETIF_F_HW_VLAN_CTAG_FILTER;
 	netdev->hw_features      |= NETIF_F_HW_VLAN_STAG_TX;
 
-	if (mlx5e_vxlan_allowed(mdev) || MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
+	if (mlx5_vxlan_allowed(priv->vxlan) || MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
 		netdev->hw_enc_features |= NETIF_F_IP_CSUM;
 		netdev->hw_enc_features |= NETIF_F_IPV6_CSUM;
 		netdev->hw_enc_features |= NETIF_F_TSO;
@@ -4656,7 +4656,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 		netdev->hw_enc_features |= NETIF_F_GSO_PARTIAL;
 	}
 
-	if (mlx5e_vxlan_allowed(mdev)) {
+	if (mlx5_vxlan_allowed(priv->vxlan)) {
 		netdev->hw_features     |= NETIF_F_GSO_UDP_TUNNEL |
 					   NETIF_F_GSO_UDP_TUNNEL_CSUM;
 		netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL |
@@ -4758,6 +4758,8 @@ static void mlx5e_nic_init(struct mlx5_core_dev *mdev,
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	int err;
 
+	priv->vxlan = mlx5_vxlan_create(mdev);
+
 	mlx5e_build_nic_netdev_priv(mdev, netdev, profile, ppriv);
 	err = mlx5e_ipsec_init(priv);
 	if (err)
@@ -4767,14 +4769,13 @@ static void mlx5e_nic_init(struct mlx5_core_dev *mdev,
 		mlx5_core_err(mdev, "TLS initialization failed, %d\n", err);
 	mlx5e_build_nic_netdev(netdev);
 	mlx5e_build_tc2txq_maps(priv);
-	mlx5e_vxlan_init(priv);
 }
 
 static void mlx5e_nic_cleanup(struct mlx5e_priv *priv)
 {
+	mlx5_vxlan_destroy(priv->vxlan);
 	mlx5e_tls_cleanup(priv);
 	mlx5e_ipsec_cleanup(priv);
-	mlx5e_vxlan_cleanup(priv);
 }
 
 static int mlx5e_init_nic_rx(struct mlx5e_priv *priv)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index 1010ee983ab7..1b4931b62094 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -1133,7 +1133,7 @@ static int parse_tunnel_attr(struct mlx5e_priv *priv,
 		if (memchr_inv(&mask->dst, 0xff, sizeof(mask->dst)))
 			goto vxlan_match_offload_err;
 
-		if (mlx5e_vxlan_lookup_port(up_priv, be16_to_cpu(key->dst)) &&
+		if (mlx5_vxlan_lookup_port(up_priv->vxlan, be16_to_cpu(key->dst)) &&
 		    MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap))
 			parse_vxlan_attr(spec, f);
 		else {
@@ -2557,7 +2557,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
 		return -EOPNOTSUPP;
 	}
 
-	if (mlx5e_vxlan_lookup_port(up_priv, be16_to_cpu(key->tp_dst)) &&
+	if (mlx5_vxlan_lookup_port(up_priv->vxlan, be16_to_cpu(key->tp_dst)) &&
 	    MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap)) {
 		tunnel_type = MLX5_HEADER_TYPE_VXLAN;
 	} else {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 95e3140da65c..502af3bdf088 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -36,32 +36,26 @@
 #include "mlx5_core.h"
 #include "vxlan.h"
 
+struct mlx5_vxlan {
+	struct mlx5_core_dev		*mdev;
+	rwlock_t			lock; /* protect vxlan table */
+	int				num_ports;
+	/* max_num_ports is usuallly 4, 16 buckets is more than enough */
+	DECLARE_HASHTABLE(htable, 4);
+};
+
 struct mlx5_vxlan_port {
 	struct hlist_node hlist;
 	atomic_t refcount;
 	u16 udp_port;
 };
 
-void mlx5e_vxlan_init(struct mlx5e_priv *priv)
-{
-	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
-
-	rwlock_init(&vxlan_db->lock);
-	hash_init(vxlan_db->htable);
-
-	if (mlx5e_vxlan_allowed(priv->mdev))
-		/* Hardware adds 4789 by default.
-		 * Lockless since we are the only hash table consumers, wq and TX are disabled.
-		 */
-		mlx5e_vxlan_add_port(priv, 4789);
-}
-
-static inline u8 mlx5e_vxlan_max_udp_ports(struct mlx5_core_dev *mdev)
+static inline u8 mlx5_vxlan_max_udp_ports(struct mlx5_core_dev *mdev)
 {
 	return MLX5_CAP_ETH(mdev, max_vxlan_udp_ports) ?: 4;
 }
 
-static int mlx5e_vxlan_core_add_port_cmd(struct mlx5_core_dev *mdev, u16 port)
+static int mlx5_vxlan_core_add_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 {
 	u32 in[MLX5_ST_SZ_DW(add_vxlan_udp_dport_in)]   = {0};
 	u32 out[MLX5_ST_SZ_DW(add_vxlan_udp_dport_out)] = {0};
@@ -72,7 +66,7 @@ static int mlx5e_vxlan_core_add_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 	return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
 }
 
-static int mlx5e_vxlan_core_del_port_cmd(struct mlx5_core_dev *mdev, u16 port)
+static int mlx5_vxlan_core_del_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 {
 	u32 in[MLX5_ST_SZ_DW(delete_vxlan_udp_dport_in)]   = {0};
 	u32 out[MLX5_ST_SZ_DW(delete_vxlan_udp_dport_out)] = {0};
@@ -84,12 +78,11 @@ static int mlx5e_vxlan_core_del_port_cmd(struct mlx5_core_dev *mdev, u16 port)
 }
 
 static struct mlx5_vxlan_port*
-mlx5e_vxlan_lookup_port_locked(struct mlx5e_priv *priv, u16 port)
+mlx5_vxlan_lookup_port_locked(struct mlx5_vxlan *vxlan, u16 port)
 {
-	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5_vxlan_port *vxlanp;
 
-	hash_for_each_possible(vxlan_db->htable, vxlanp, hlist, port) {
+	hash_for_each_possible(vxlan->htable, vxlanp, hlist, port) {
 		if (vxlanp->udp_port == port)
 			return vxlanp;
 	}
@@ -97,37 +90,38 @@ mlx5e_vxlan_lookup_port_locked(struct mlx5e_priv *priv, u16 port)
 	return NULL;
 }
 
-struct mlx5_vxlan_port *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port)
+struct mlx5_vxlan_port *mlx5_vxlan_lookup_port(struct mlx5_vxlan *vxlan, u16 port)
 {
-	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5_vxlan_port *vxlanp;
 
-	read_lock_bh(&vxlan_db->lock);
-	vxlanp = mlx5e_vxlan_lookup_port_locked(priv, port);
-	read_unlock_bh(&vxlan_db->lock);
+	if (!mlx5_vxlan_allowed(vxlan))
+		return NULL;
+
+	read_lock_bh(&vxlan->lock);
+	vxlanp = mlx5_vxlan_lookup_port_locked(vxlan, port);
+	read_unlock_bh(&vxlan->lock);
 
 	return vxlanp;
 }
 
-void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
+void mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port)
 {
-	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5_vxlan_port *vxlanp;
 
-	vxlanp = mlx5e_vxlan_lookup_port(priv, port);
+	vxlanp = mlx5_vxlan_lookup_port(vxlan, port);
 	if (vxlanp) {
 		atomic_inc(&vxlanp->refcount);
 		return;
 	}
 
-	if (vxlan_db->num_ports >= mlx5e_vxlan_max_udp_ports(priv->mdev)) {
-		netdev_info(priv->netdev,
-			    "UDP port (%d) not offloaded, max number of UDP ports (%d) are already offloaded\n",
-			    port, mlx5e_vxlan_max_udp_ports(priv->mdev));
+	if (vxlan->num_ports >= mlx5_vxlan_max_udp_ports(vxlan->mdev)) {
+		mlx5_core_info(vxlan->mdev,
+			       "UDP port (%d) not offloaded, max number of UDP ports (%d) are already offloaded\n",
+			       port, mlx5_vxlan_max_udp_ports(vxlan->mdev));
 		return;
 	}
 
-	if (mlx5e_vxlan_core_add_port_cmd(priv->mdev, port))
+	if (mlx5_vxlan_core_add_port_cmd(vxlan->mdev, port))
 		return;
 
 	vxlanp = kzalloc(sizeof(*vxlanp), GFP_KERNEL);
@@ -137,25 +131,24 @@ void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port)
 	vxlanp->udp_port = port;
 	atomic_set(&vxlanp->refcount, 1);
 
-	write_lock_bh(&vxlan_db->lock);
-	hash_add(vxlan_db->htable, &vxlanp->hlist, port);
-	write_unlock_bh(&vxlan_db->lock);
+	write_lock_bh(&vxlan->lock);
+	hash_add(vxlan->htable, &vxlanp->hlist, port);
+	write_unlock_bh(&vxlan->lock);
 
-	vxlan_db->num_ports++;
+	vxlan->num_ports++;
 	return;
 
 err_delete_port:
-	mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
+	mlx5_vxlan_core_del_port_cmd(vxlan->mdev, port);
 }
 
-void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
+void mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port)
 {
-	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5_vxlan_port *vxlanp;
 	bool remove = false;
 
-	write_lock_bh(&vxlan_db->lock);
-	vxlanp = mlx5e_vxlan_lookup_port_locked(priv, port);
+	write_lock_bh(&vxlan->lock);
+	vxlanp = mlx5_vxlan_lookup_port_locked(vxlan, port);
 	if (!vxlanp)
 		goto out_unlock;
 
@@ -165,26 +158,51 @@ void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port)
 	}
 
 out_unlock:
-	write_unlock_bh(&vxlan_db->lock);
+	write_unlock_bh(&vxlan->lock);
 
 	if (remove) {
-		mlx5e_vxlan_core_del_port_cmd(priv->mdev, port);
+		mlx5_vxlan_core_del_port_cmd(vxlan->mdev, port);
 		kfree(vxlanp);
-		vxlan_db->num_ports--;
+		vxlan->num_ports--;
 	}
 }
 
-void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv)
+struct mlx5_vxlan *mlx5_vxlan_create(struct mlx5_core_dev *mdev)
+{
+	struct mlx5_vxlan *vxlan;
+
+	if (!MLX5_CAP_ETH(mdev, tunnel_stateless_vxlan) || !mlx5_core_is_pf(mdev))
+		return ERR_PTR(-ENOTSUPP);
+
+	vxlan = kzalloc(sizeof(*vxlan), GFP_KERNEL);
+	if (!vxlan)
+		return ERR_PTR(-ENOMEM);
+
+	vxlan->mdev = mdev;
+	rwlock_init(&vxlan->lock);
+	hash_init(vxlan->htable);
+
+	/* Hardware adds 4789 by default */
+	mlx5_vxlan_add_port(vxlan, 4789);
+
+	return vxlan;
+}
+
+void mlx5_vxlan_destroy(struct mlx5_vxlan *vxlan)
 {
-	struct mlx5e_vxlan_db *vxlan_db = &priv->vxlan;
 	struct mlx5_vxlan_port *vxlanp;
 	struct hlist_node *tmp;
 	int bkt;
 
-	/* Lockless since we are the only hash table consumers, wq and TX are disabled */
-	hash_for_each_safe(vxlan_db->htable, bkt, tmp, vxlanp, hlist) {
+	if (!mlx5_vxlan_allowed(vxlan))
+		return;
+
+	/* Lockless since we are the only hash table consumers*/
+	hash_for_each_safe(vxlan->htable, bkt, tmp, vxlanp, hlist) {
 		hash_del(&vxlanp->hlist);
-		mlx5e_vxlan_core_del_port_cmd(priv->mdev, vxlanp->udp_port);
+		mlx5_vxlan_core_del_port_cmd(vxlan->mdev, vxlanp->udp_port);
 		kfree(vxlanp);
 	}
+
+	kfree(vxlan);
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
index 6b38b6fbd030..9d6327321814 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
@@ -33,20 +33,32 @@
 #define __MLX5_VXLAN_H__
 
 #include <linux/mlx5/driver.h>
-#include "en.h"
 
+struct mlx5_vxlan;
 struct mlx5_vxlan_port;
 
-static inline bool mlx5e_vxlan_allowed(struct mlx5_core_dev *mdev)
+#ifdef CONFIG_MLX5_CORE_EN
+
+static inline bool mlx5_vxlan_allowed(struct mlx5_vxlan *vxlan)
 {
-	return (MLX5_CAP_ETH(mdev, tunnel_stateless_vxlan) &&
-		mlx5_core_is_pf(mdev));
+	/* not allowed reason is encoded in vxlan pointer as error,
+	 * on mlx5_vxlan_create
+	 */
+	return !IS_ERR_OR_NULL(vxlan);
 }
 
-void mlx5e_vxlan_init(struct mlx5e_priv *priv);
-void mlx5e_vxlan_cleanup(struct mlx5e_priv *priv);
-void mlx5e_vxlan_add_port(struct mlx5e_priv *priv, u16 port);
-void mlx5e_vxlan_del_port(struct mlx5e_priv *priv, u16 port);
-struct mlx5_vxlan_port *mlx5e_vxlan_lookup_port(struct mlx5e_priv *priv, u16 port);
+struct mlx5_vxlan *mlx5_vxlan_create(struct mlx5_core_dev *mdev);
+void mlx5_vxlan_destroy(struct mlx5_vxlan *vxlan);
+void mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port);
+void mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port);
+struct mlx5_vxlan_port *mlx5_vxlan_lookup_port(struct mlx5_vxlan *vxlan, u16 port);
+
+#else
+
+static inline struct mlx5_vxlan*
+mlx5_vxlan_create(struct mlx5_core_dev *mdev) { return ERR_PTR(-ENOTSUPP); }
+static inline void mlx5_vxlan_destroy(struct mlx5_vxlan *vxlan) { return; }
+
+#endif
 
 #endif /* __MLX5_VXLAN_H__ */
-- 
2.17.0

[net-next 10/13] net/mlx5e: Vxlan, return values for add/del port

[Saeed Mahameed]

For a better API mlx5_vxlan_{add/del}_port can fail, make them return
error values.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/vxlan.c   | 28 +++++++++++++------
 .../net/ethernet/mellanox/mlx5/core/vxlan.h   |  4 +--
 2 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index 502af3bdf088..f5353134542d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -104,29 +104,34 @@ struct mlx5_vxlan_port *mlx5_vxlan_lookup_port(struct mlx5_vxlan *vxlan, u16 por
 	return vxlanp;
 }
 
-void mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port)
+int mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port)
 {
 	struct mlx5_vxlan_port *vxlanp;
+	int ret = -ENOSPC;
 
 	vxlanp = mlx5_vxlan_lookup_port(vxlan, port);
 	if (vxlanp) {
 		atomic_inc(&vxlanp->refcount);
-		return;
+		return 0;
 	}
 
 	if (vxlan->num_ports >= mlx5_vxlan_max_udp_ports(vxlan->mdev)) {
 		mlx5_core_info(vxlan->mdev,
 			       "UDP port (%d) not offloaded, max number of UDP ports (%d) are already offloaded\n",
 			       port, mlx5_vxlan_max_udp_ports(vxlan->mdev));
-		return;
+		ret = -ENOSPC;
+		return ret;
 	}
 
-	if (mlx5_vxlan_core_add_port_cmd(vxlan->mdev, port))
-		return;
+	ret = mlx5_vxlan_core_add_port_cmd(vxlan->mdev, port);
+	if (ret)
+		return ret;
 
 	vxlanp = kzalloc(sizeof(*vxlanp), GFP_KERNEL);
-	if (!vxlanp)
+	if (!vxlanp) {
+		ret = -ENOMEM;
 		goto err_delete_port;
+	}
 
 	vxlanp->udp_port = port;
 	atomic_set(&vxlanp->refcount, 1);
@@ -136,21 +141,25 @@ void mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port)
 	write_unlock_bh(&vxlan->lock);
 
 	vxlan->num_ports++;
-	return;
+	return 0;
 
 err_delete_port:
 	mlx5_vxlan_core_del_port_cmd(vxlan->mdev, port);
+	return ret;
 }
 
-void mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port)
+int mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port)
 {
 	struct mlx5_vxlan_port *vxlanp;
 	bool remove = false;
+	int ret = 0;
 
 	write_lock_bh(&vxlan->lock);
 	vxlanp = mlx5_vxlan_lookup_port_locked(vxlan, port);
-	if (!vxlanp)
+	if (!vxlanp) {
+		ret = -ENOENT;
 		goto out_unlock;
+	}
 
 	if (atomic_dec_and_test(&vxlanp->refcount)) {
 		hash_del(&vxlanp->hlist);
@@ -165,6 +174,7 @@ void mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port)
 		kfree(vxlanp);
 		vxlan->num_ports--;
 	}
+	return ret;
 }
 
 struct mlx5_vxlan *mlx5_vxlan_create(struct mlx5_core_dev *mdev)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
index 9d6327321814..fd874a30c4d0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
@@ -49,8 +49,8 @@ static inline bool mlx5_vxlan_allowed(struct mlx5_vxlan *vxlan)
 
 struct mlx5_vxlan *mlx5_vxlan_create(struct mlx5_core_dev *mdev);
 void mlx5_vxlan_destroy(struct mlx5_vxlan *vxlan);
-void mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port);
-void mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port);
+int mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port);
+int mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port);
 struct mlx5_vxlan_port *mlx5_vxlan_lookup_port(struct mlx5_vxlan *vxlan, u16 port);
 
 #else
-- 
2.17.0

[net-next 11/13] net/mlx5e: Vxlan, add sync lock for add/del vxlan port

[Saeed Mahameed]

Vxlan API can and will be called from different mlx5 modules, we should
not count on mlx5e private state lock only, hence we introduce a vxlan
private mutex to sync between add/del vxlan port operations.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/vxlan.c    | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
index f5353134542d..c126a790234d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
@@ -38,8 +38,9 @@
 
 struct mlx5_vxlan {
 	struct mlx5_core_dev		*mdev;
-	rwlock_t			lock; /* protect vxlan table */
 	int				num_ports;
+	struct mutex                    sync_lock; /* sync add/del port HW operations */
+	rwlock_t                        lock; /* sync vxlan table with data path access */
 	/* max_num_ports is usuallly 4, 16 buckets is more than enough */
 	DECLARE_HASHTABLE(htable, 4);
 };
@@ -115,17 +116,18 @@ int mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port)
 		return 0;
 	}
 
+	mutex_lock(&vxlan->sync_lock);
 	if (vxlan->num_ports >= mlx5_vxlan_max_udp_ports(vxlan->mdev)) {
 		mlx5_core_info(vxlan->mdev,
 			       "UDP port (%d) not offloaded, max number of UDP ports (%d) are already offloaded\n",
 			       port, mlx5_vxlan_max_udp_ports(vxlan->mdev));
 		ret = -ENOSPC;
-		return ret;
+		goto unlock;
 	}
 
 	ret = mlx5_vxlan_core_add_port_cmd(vxlan->mdev, port);
 	if (ret)
-		return ret;
+		goto unlock;
 
 	vxlanp = kzalloc(sizeof(*vxlanp), GFP_KERNEL);
 	if (!vxlanp) {
@@ -141,10 +143,14 @@ int mlx5_vxlan_add_port(struct mlx5_vxlan *vxlan, u16 port)
 	write_unlock_bh(&vxlan->lock);
 
 	vxlan->num_ports++;
+	mutex_unlock(&vxlan->sync_lock);
 	return 0;
 
 err_delete_port:
 	mlx5_vxlan_core_del_port_cmd(vxlan->mdev, port);
+
+unlock:
+	mutex_unlock(&vxlan->sync_lock);
 	return ret;
 }
 
@@ -154,6 +160,8 @@ int mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port)
 	bool remove = false;
 	int ret = 0;
 
+	mutex_lock(&vxlan->sync_lock);
+
 	write_lock_bh(&vxlan->lock);
 	vxlanp = mlx5_vxlan_lookup_port_locked(vxlan, port);
 	if (!vxlanp) {
@@ -174,6 +182,9 @@ int mlx5_vxlan_del_port(struct mlx5_vxlan *vxlan, u16 port)
 		kfree(vxlanp);
 		vxlan->num_ports--;
 	}
+
+	mutex_unlock(&vxlan->sync_lock);
+
 	return ret;
 }
 
@@ -189,6 +200,7 @@ struct mlx5_vxlan *mlx5_vxlan_create(struct mlx5_core_dev *mdev)
 		return ERR_PTR(-ENOMEM);
 
 	vxlan->mdev = mdev;
+	mutex_init(&vxlan->sync_lock);
 	rwlock_init(&vxlan->lock);
 	hash_init(vxlan->htable);
 
-- 
2.17.0

[net-next 12/13] net/mlx5e: Vxlan, move vxlan logic to core driver

[Saeed Mahameed]

Move vxlan logic and objects to mlx5 core dirver.
Since it going to be used from different mlx5 interfaces.
e.g. mlx5e PF NIC netdev and mlx5e E-Switch representors.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |  4 ++--
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  2 --
 .../net/ethernet/mellanox/mlx5/core/en_main.c | 21 ++++++++-----------
 .../net/ethernet/mellanox/mlx5/core/en_tc.c   |  6 +++---
 .../mellanox/mlx5/core/{ => lib}/vxlan.c      |  0
 .../mellanox/mlx5/core/{ => lib}/vxlan.h      |  0
 .../net/ethernet/mellanox/mlx5/core/main.c    |  5 +++++
 include/linux/mlx5/driver.h                   |  2 ++
 8 files changed, 21 insertions(+), 19 deletions(-)
 rename drivers/net/ethernet/mellanox/mlx5/core/{ => lib}/vxlan.c (100%)
 rename drivers/net/ethernet/mellanox/mlx5/core/{ => lib}/vxlan.h (100%)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index ae2bdcb1647c..f20fda1ced4f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -14,8 +14,8 @@ mlx5_core-$(CONFIG_MLX5_FPGA) += fpga/cmd.o fpga/core.o fpga/conn.o fpga/sdk.o \
 		fpga/ipsec.o fpga/tls.o
 
 mlx5_core-$(CONFIG_MLX5_CORE_EN) += en_main.o en_common.o en_fs.o en_ethtool.o \
-		en_tx.o en_rx.o en_dim.o en_txrx.o en/xdp.o en_stats.o vxlan.o \
-		en_arfs.o en_fs_ethtool.o en_selftest.o en/port.o
+		en_tx.o en_rx.o en_dim.o en_txrx.o en/xdp.o en_stats.o \
+		en_arfs.o en_fs_ethtool.o en_selftest.o en/port.o lib/vxlan.o
 
 mlx5_core-$(CONFIG_MLX5_MPFS) += lib/mpfs.o
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 1bd4536b9061..c7ed3d20fd54 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -52,7 +52,6 @@
 #include "wq.h"
 #include "mlx5_core.h"
 #include "en_stats.h"
-#include "vxlan.h"
 
 struct page_pool;
 
@@ -812,7 +811,6 @@ struct mlx5e_priv {
 	u32                        tx_rates[MLX5E_MAX_NUM_SQS];
 
 	struct mlx5e_flow_steering fs;
-	struct mlx5_vxlan          *vxlan;
 
 	struct workqueue_struct    *wq;
 	struct work_struct         update_carrier_work;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index ef4b2f0c427c..fde35021a257 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -45,7 +45,7 @@
 #include "en_accel/tls.h"
 #include "accel/ipsec.h"
 #include "accel/tls.h"
-#include "vxlan.h"
+#include "lib/vxlan.h"
 #include "en/port.h"
 #include "en/xdp.h"
 
@@ -2974,7 +2974,7 @@ int mlx5e_open(struct net_device *netdev)
 		mlx5_set_port_admin_status(priv->mdev, MLX5_PORT_UP);
 	mutex_unlock(&priv->state_lock);
 
-	if (mlx5_vxlan_allowed(priv->vxlan))
+	if (mlx5_vxlan_allowed(priv->mdev->vxlan))
 		udp_tunnel_get_rx_info(netdev);
 
 	return err;
@@ -3983,7 +3983,7 @@ static void mlx5e_vxlan_add_work(struct work_struct *work)
 	u16 port = vxlan_work->port;
 
 	mutex_lock(&priv->state_lock);
-	mlx5_vxlan_add_port(priv->vxlan, port);
+	mlx5_vxlan_add_port(priv->mdev->vxlan, port);
 	mutex_unlock(&priv->state_lock);
 
 	kfree(vxlan_work);
@@ -3997,7 +3997,7 @@ static void mlx5e_vxlan_del_work(struct work_struct *work)
 	u16 port = vxlan_work->port;
 
 	mutex_lock(&priv->state_lock);
-	mlx5_vxlan_del_port(priv->vxlan, port);
+	mlx5_vxlan_del_port(priv->mdev->vxlan, port);
 	mutex_unlock(&priv->state_lock);
 	kfree(vxlan_work);
 }
@@ -4028,7 +4028,7 @@ static void mlx5e_add_vxlan_port(struct net_device *netdev,
 	if (ti->type != UDP_TUNNEL_TYPE_VXLAN)
 		return;
 
-	if (!mlx5_vxlan_allowed(priv->vxlan))
+	if (!mlx5_vxlan_allowed(priv->mdev->vxlan))
 		return;
 
 	mlx5e_vxlan_queue_work(priv, be16_to_cpu(ti->port), 1);
@@ -4042,7 +4042,7 @@ static void mlx5e_del_vxlan_port(struct net_device *netdev,
 	if (ti->type != UDP_TUNNEL_TYPE_VXLAN)
 		return;
 
-	if (!mlx5_vxlan_allowed(priv->vxlan))
+	if (!mlx5_vxlan_allowed(priv->mdev->vxlan))
 		return;
 
 	mlx5e_vxlan_queue_work(priv, be16_to_cpu(ti->port), 0);
@@ -4076,7 +4076,7 @@ static netdev_features_t mlx5e_tunnel_features_check(struct mlx5e_priv *priv,
 		port = be16_to_cpu(udph->dest);
 
 		/* Verify if UDP port is being offloaded by HW */
-		if (mlx5_vxlan_lookup_port(priv->vxlan, port))
+		if (mlx5_vxlan_lookup_port(priv->mdev->vxlan, port))
 			return features;
 	}
 
@@ -4648,7 +4648,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 	netdev->hw_features      |= NETIF_F_HW_VLAN_CTAG_FILTER;
 	netdev->hw_features      |= NETIF_F_HW_VLAN_STAG_TX;
 
-	if (mlx5_vxlan_allowed(priv->vxlan) || MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
+	if (mlx5_vxlan_allowed(mdev->vxlan) || MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
 		netdev->hw_enc_features |= NETIF_F_IP_CSUM;
 		netdev->hw_enc_features |= NETIF_F_IPV6_CSUM;
 		netdev->hw_enc_features |= NETIF_F_TSO;
@@ -4656,7 +4656,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
 		netdev->hw_enc_features |= NETIF_F_GSO_PARTIAL;
 	}
 
-	if (mlx5_vxlan_allowed(priv->vxlan)) {
+	if (mlx5_vxlan_allowed(mdev->vxlan)) {
 		netdev->hw_features     |= NETIF_F_GSO_UDP_TUNNEL |
 					   NETIF_F_GSO_UDP_TUNNEL_CSUM;
 		netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL |
@@ -4758,8 +4758,6 @@ static void mlx5e_nic_init(struct mlx5_core_dev *mdev,
 	struct mlx5e_priv *priv = netdev_priv(netdev);
 	int err;
 
-	priv->vxlan = mlx5_vxlan_create(mdev);
-
 	mlx5e_build_nic_netdev_priv(mdev, netdev, profile, ppriv);
 	err = mlx5e_ipsec_init(priv);
 	if (err)
@@ -4773,7 +4771,6 @@ static void mlx5e_nic_init(struct mlx5_core_dev *mdev,
 
 static void mlx5e_nic_cleanup(struct mlx5e_priv *priv)
 {
-	mlx5_vxlan_destroy(priv->vxlan);
 	mlx5e_tls_cleanup(priv);
 	mlx5e_ipsec_cleanup(priv);
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index 1b4931b62094..288a57f76e84 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -50,7 +50,7 @@
 #include "en_rep.h"
 #include "en_tc.h"
 #include "eswitch.h"
-#include "vxlan.h"
+#include "lib/vxlan.h"
 #include "fs_core.h"
 #include "en/port.h"
 
@@ -1133,7 +1133,7 @@ static int parse_tunnel_attr(struct mlx5e_priv *priv,
 		if (memchr_inv(&mask->dst, 0xff, sizeof(mask->dst)))
 			goto vxlan_match_offload_err;
 
-		if (mlx5_vxlan_lookup_port(up_priv->vxlan, be16_to_cpu(key->dst)) &&
+		if (mlx5_vxlan_lookup_port(up_priv->mdev->vxlan, be16_to_cpu(key->dst)) &&
 		    MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap))
 			parse_vxlan_attr(spec, f);
 		else {
@@ -2557,7 +2557,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
 		return -EOPNOTSUPP;
 	}
 
-	if (mlx5_vxlan_lookup_port(up_priv->vxlan, be16_to_cpu(key->tp_dst)) &&
+	if (mlx5_vxlan_lookup_port(up_priv->mdev->vxlan, be16_to_cpu(key->tp_dst)) &&
 	    MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap)) {
 		tunnel_type = MLX5_HEADER_TYPE_VXLAN;
 	} else {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.c
similarity index 100%
rename from drivers/net/ethernet/mellanox/mlx5/core/vxlan.c
rename to drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.c
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vxlan.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.h
similarity index 100%
rename from drivers/net/ethernet/mellanox/mlx5/core/vxlan.h
rename to drivers/net/ethernet/mellanox/mlx5/core/lib/vxlan.h
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 6ddbb70e95de..03b9c6733eed 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -62,6 +62,7 @@
 #include "accel/ipsec.h"
 #include "accel/tls.h"
 #include "lib/clock.h"
+#include "lib/vxlan.h"
 #include "diag/fw_tracer.h"
 
 MODULE_AUTHOR("Eli Cohen <eli@mellanox.com>");
@@ -961,6 +962,8 @@ static int mlx5_init_once(struct mlx5_core_dev *dev, struct mlx5_priv *priv)
 
 	mlx5_init_clock(dev);
 
+	dev->vxlan = mlx5_vxlan_create(dev);
+
 	err = mlx5_init_rl_table(dev);
 	if (err) {
 		dev_err(&pdev->dev, "Failed to init rate limiting\n");
@@ -1004,6 +1007,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev, struct mlx5_priv *priv)
 err_rl_cleanup:
 	mlx5_cleanup_rl_table(dev);
 err_tables_cleanup:
+	mlx5_vxlan_destroy(dev->vxlan);
 	mlx5_cleanup_mkey_table(dev);
 	mlx5_cleanup_srq_table(dev);
 	mlx5_cleanup_qp_table(dev);
@@ -1024,6 +1028,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev)
 	mlx5_eswitch_cleanup(dev->priv.eswitch);
 	mlx5_mpfs_cleanup(dev);
 	mlx5_cleanup_rl_table(dev);
+	mlx5_vxlan_destroy(dev->vxlan);
 	mlx5_cleanup_clock(dev);
 	mlx5_cleanup_reserved_gids(dev);
 	mlx5_cleanup_mkey_table(dev);
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index fd0aaa5568fe..54f385cc8811 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -818,6 +818,7 @@ struct mlx5_clock {
 };
 
 struct mlx5_fw_tracer;
+struct mlx5_vxlan;
 
 struct mlx5_core_dev {
 	struct pci_dev	       *pdev;
@@ -850,6 +851,7 @@ struct mlx5_core_dev {
 	atomic_t		num_qps;
 	u32			issi;
 	struct mlx5e_resources  mlx5e_res;
+	struct mlx5_vxlan       *vxlan;
 	struct {
 		struct mlx5_rsvd_gids	reserved_gids;
 		u32			roce_en;
-- 
2.17.0

[net-next 13/13] net/mlx5e: Issue direct lookup on vxlan ports by vport representors

[Saeed Mahameed]

Remove uplink representor netdevice private structure lookup, and use
mlx5 core handle directly from representor private structure to lookup
vxlan ports.

Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Reviewed-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_tc.c | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index 288a57f76e84..c28fe469b04a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -1124,16 +1124,12 @@ static int parse_tunnel_attr(struct mlx5e_priv *priv,
 			skb_flow_dissector_target(f->dissector,
 						  FLOW_DISSECTOR_KEY_ENC_PORTS,
 						  f->mask);
-		struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
-		struct mlx5e_rep_priv *uplink_rpriv = mlx5_eswitch_get_uplink_priv(esw, REP_ETH);
-		struct net_device *up_dev = uplink_rpriv->netdev;
-		struct mlx5e_priv *up_priv = netdev_priv(up_dev);
 
 		/* Full udp dst port must be given */
 		if (memchr_inv(&mask->dst, 0xff, sizeof(mask->dst)))
 			goto vxlan_match_offload_err;
 
-		if (mlx5_vxlan_lookup_port(up_priv->mdev->vxlan, be16_to_cpu(key->dst)) &&
+		if (mlx5_vxlan_lookup_port(priv->mdev->vxlan, be16_to_cpu(key->dst)) &&
 		    MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap))
 			parse_vxlan_attr(spec, f);
 		else {
@@ -2533,11 +2529,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
 			      struct mlx5e_tc_flow *flow)
 {
 	struct mlx5_eswitch *esw = priv->mdev->priv.eswitch;
-	struct mlx5e_rep_priv *uplink_rpriv = mlx5_eswitch_get_uplink_priv(esw,
-									   REP_ETH);
-	struct net_device *up_dev = uplink_rpriv->netdev;
 	unsigned short family = ip_tunnel_info_af(tun_info);
-	struct mlx5e_priv *up_priv = netdev_priv(up_dev);
 	struct mlx5_esw_flow_attr *attr = flow->esw_attr;
 	struct ip_tunnel_key *key = &tun_info->key;
 	struct mlx5e_encap_entry *e;
@@ -2557,7 +2549,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
 		return -EOPNOTSUPP;
 	}
 
-	if (mlx5_vxlan_lookup_port(up_priv->mdev->vxlan, be16_to_cpu(key->tp_dst)) &&
+	if (mlx5_vxlan_lookup_port(priv->mdev->vxlan, be16_to_cpu(key->tp_dst)) &&
 	    MLX5_CAP_ESW(priv->mdev, vxlan_encap_decap)) {
 		tunnel_type = MLX5_HEADER_TYPE_VXLAN;
 	} else {
-- 
2.17.0