Netdev List
 help / color / mirror / Atom feed
From: Tariq Toukan <tariqt@nvidia.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	Jakub Kicinski <kuba@kernel.org>, <netdev@vger.kernel.org>,
	Paolo Abeni <pabeni@redhat.com>
Cc: Akiva Goldberger <agoldberger@nvidia.com>,
	Gal Pressman <gal@nvidia.com>, Jason Gunthorpe <jgg@ziepe.ca>,
	Leon Romanovsky <leon@kernel.org>, <linux-rdma@vger.kernel.org>,
	Mark Bloch <mbloch@nvidia.com>, Moshe Shemesh <moshe@nvidia.com>,
	Or Har-Toov <ohartoov@nvidia.com>,
	Saeed Mahameed <saeedm@nvidia.com>, Shay Drory <shayd@nvidia.com>,
	Tariq Toukan <tariqt@nvidia.com>
Subject: [PATCH net-next V2 5/5] net/mlx5: LAG, enable LAG over functions that do not represent a physical port
Date: Fri, 20 Nov 2026 09:59:58 +0200	[thread overview]
Message-ID: <20261120075959.2533702-6-tariqt@nvidia.com> (raw)
In-Reply-To: <20261120075959.2533702-1-tariqt@nvidia.com>

From: Shay Drory <shayd@nvidia.com>

LAG membership assumes every member represents a physical port.
A nested LAG of VFs/SFs does not fit that assumption; its member
functions sit behind a multi_uplink_eswitch and each reports
num_lag_ports < 2, so they do not represent a physical port.

Detect this with mlx5_virt_lag_is_supported(): let such a function join
the LAG in mlx5_lag_add_mdev(), and skip the MLX5_LAG_XA_MARK_PORT mark
and the PORT_CHANGE event notifier in mlx5_ldev_add_mdev(), since it is
not a physical port.

These function only support MPESW, and netdevs addition to LAG is only
needed for linux bond. Hence, don't allow such a lag to add netdevs to
itself. For the same reason, don't register the LAG netdev notifier for
it. Multipath LAG is not supported either, so skip mlx5_lag_mp_init()
and do not register the FIB notifier for it.

Such a function may report num_lag_ports == 0, which ldev->ports keeps
as is. Report at least one port from mlx5_lag_get_num_ports(), so user
would avoid divide by 0.

Such a function selects the uplink through port_select_eswitch, not
through a TX port affinity. Yet once its LAG is active, RDMA sets
lag_active, and mlx5_ib_lag_should_assign_affinity() then assigns a TX
port affinity to every QP, bypassing the num_lag_ports > 1 and
lag_tx_port_affinity capability checks. Make
mlx5_ib_lag_should_assign_affinity() return false for such a function.

Signed-off-by: Shay Drory <shayd@nvidia.com>
Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  3 ++
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 32 +++++++++++++------
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   |  2 +-
 3 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4a6ee14ffd7d..494b7036321b 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1612,6 +1612,9 @@ static inline void mlx5r_deref_wait_odp_mkey(struct mlx5_ib_mkey *mmkey)
 
 static inline bool mlx5_ib_lag_should_assign_affinity(struct mlx5_ib_dev *dev)
 {
+	if (dev->lag_active && mlx5_virt_lag_is_supported(dev->mdev))
+		return 0;
+
 	/*
 	 * If the driver is in hash mode and the port_select_flow_table_bypass cap
 	 * is supported, it means that the driver no longer needs to assign the port
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
index 193fbe47fef1..2cd2319f9318 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
@@ -366,6 +366,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 
 	ldev->ports = MLX5_CAP_GEN(dev, num_lag_ports);
 	ldev->buckets = 1;
+	ldev->virt_lag = mlx5_virt_lag_is_supported(dev);
 	ldev->v2p_map = kcalloc(ldev->ports * MLX5_LAG_MAX_HASH_BUCKETS,
 				sizeof(*ldev->v2p_map), GFP_KERNEL);
 	if (!ldev->v2p_map)
@@ -381,7 +382,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 	INIT_DELAYED_WORK(&ldev->bond_work, mlx5_do_bond_work);
 	INIT_WORK(&ldev->speed_update_work, mlx5_mpesw_speed_update_work);
 
-	if (!mlx5_sd_is_supported(dev)) {
+	if (!mlx5_sd_is_supported(dev) && !ldev->virt_lag) {
 		ldev->nb.notifier_call = mlx5_lag_netdev_event;
 		write_pnet(&ldev->net, mlx5_core_net(dev));
 		if (register_netdevice_notifier_net(read_pnet(&ldev->net),
@@ -392,10 +393,13 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 	}
 	ldev->mode = MLX5_LAG_MODE_NONE;
 
-	err = mlx5_lag_mp_init(ldev);
-	if (err)
-		mlx5_core_err(dev, "Failed to init multipath lag err=%d\n",
-			      err);
+	if (!ldev->virt_lag) {
+		err = mlx5_lag_mp_init(ldev);
+		if (err)
+			mlx5_core_err(dev,
+				      "Failed to init multipath lag err=%d\n",
+				      err);
+	}
 
 	ldev->max_fns = ldev->ports;
 	return ldev;
@@ -468,7 +472,9 @@ int mlx5_lag_get_dev_index_by_seq(struct mlx5_lag *ldev, int seq)
 /* Return the appropriate iterator filter for a device in LAG:
  * - SD shared FDB active: iterate only the device's SD group
  * - SD group exists but shared FDB not active: iterate all devices
- * - No SD: iterate ports only
+ * - Virt LAG: iterate all devices, since its members are not marked as
+ *   ports, as they do not represent a physical port
+ * - Otherwise: iterate ports only
  */
 static u32 mlx5_lag_get_filter(struct mlx5_lag *ldev, struct mlx5_core_dev *dev)
 {
@@ -476,7 +482,7 @@ static u32 mlx5_lag_get_filter(struct mlx5_lag *ldev, struct mlx5_core_dev *dev)
 
 	if (fn && fn->sd_fdb_active)
 		return fn->group_id;
-	if (fn && fn->group_id)
+	if (fn && (fn->group_id || ldev->virt_lag))
 		return MLX5_LAG_FILTER_ALL;
 	return MLX5_LAG_FILTER_PORTS;
 }
@@ -2398,7 +2404,7 @@ int mlx5_ldev_add_mdev(struct mlx5_lag *ldev,
 	fn->group_id = group_id;
 	dev->priv.lag = ldev;
 
-	if (group_id)
+	if (group_id || ldev->virt_lag)
 		return 0;
 
 	xa_set_mark(&ldev->fns, idx, MLX5_LAG_XA_MARK_PORT);
@@ -2547,7 +2553,7 @@ void mlx5_lag_add_mdev(struct mlx5_core_dev *dev)
 {
 	int err;
 
-	if (!mlx5_lag_is_supported(dev))
+	if (!mlx5_lag_is_supported(dev) && !mlx5_virt_lag_is_supported(dev))
 		return;
 
 	if (mlx5_lag_register_hca_devcom_comp(dev))
@@ -2582,6 +2588,9 @@ void mlx5_lag_remove_netdev(struct mlx5_core_dev *dev,
 	if (!ldev)
 		return;
 
+	if (ldev->virt_lag)
+		return;
+
 	mutex_lock(&ldev->lock);
 	mlx5_ldev_remove_netdev(ldev, netdev);
 	clear_bit(MLX5_LAG_FLAG_NDEVS_READY, &ldev->state_flags);
@@ -2603,6 +2612,9 @@ void mlx5_lag_add_netdev(struct mlx5_core_dev *dev,
 	if (!ldev)
 		return;
 
+	if (ldev->virt_lag)
+		return;
+
 	mutex_lock(&ldev->lock);
 	mlx5_ldev_add_netdev(ldev, dev, netdev);
 	num = mlx5_lag_num_netdevs(ldev);
@@ -2886,7 +2898,7 @@ u8 mlx5_lag_get_num_ports(struct mlx5_core_dev *dev)
 	if (!ldev)
 		return 0;
 
-	return ldev->ports;
+	return max_t(u8, ldev->ports, 1);
 }
 EXPORT_SYMBOL(mlx5_lag_get_num_ports);
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
index 2d3e61541d8a..0ef2d13d0096 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
@@ -336,7 +336,7 @@ bool mlx5_lag_is_mpesw(struct mlx5_core_dev *dev)
 	struct mlx5_lag *ldev = mlx5_lag_dev(dev);
 
 	return ldev && ldev->mode == MLX5_LAG_MODE_MPESW &&
-	       __mlx5_lag_dev_is_port(ldev, dev);
+	       (ldev->virt_lag || __mlx5_lag_dev_is_port(ldev, dev));
 }
 EXPORT_SYMBOL(mlx5_lag_is_mpesw);
 
-- 
2.44.0


      parent reply	other threads:[~2026-11-20  8:01 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-20  7:59 [PATCH net-next V2 0/5] net/mlx5: LAG over VFs/SFs (virtual LAG) Tariq Toukan
2026-11-20  7:59 ` [PATCH net-next V2 1/5] net/mlx5: E-switch, add match-metadata allocator for nested VF/SF eswitches Tariq Toukan
2026-11-20  7:59 ` [PATCH net-next V2 2/5] net/mlx5: E-switch, match master egress rule by vport metadata Tariq Toukan
2026-11-20  7:59 ` [PATCH net-next V2 3/5] net/mlx5: E-switch, drop master egress rule under a SW vport LAG Tariq Toukan
2026-11-20  7:59 ` [PATCH net-next V2 4/5] net/mlx5e: Register eswitch-manager VF/SF self-port with switch-id Tariq Toukan
2026-11-20  7:59 ` Tariq Toukan [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120075959.2533702-6-tariqt@nvidia.com \
    --to=tariqt@nvidia.com \
    --cc=agoldberger@nvidia.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=gal@nvidia.com \
    --cc=jgg@ziepe.ca \
    --cc=kuba@kernel.org \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mbloch@nvidia.com \
    --cc=moshe@nvidia.com \
    --cc=netdev@vger.kernel.org \
    --cc=ohartoov@nvidia.com \
    --cc=pabeni@redhat.com \
    --cc=saeedm@nvidia.com \
    --cc=shayd@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox