From: Tariq Toukan <tariqt@nvidia.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>, <netdev@vger.kernel.org>,
Paolo Abeni <pabeni@redhat.com>
Cc: Akiva Goldberger <agoldberger@nvidia.com>,
Gal Pressman <gal@nvidia.com>, Jason Gunthorpe <jgg@ziepe.ca>,
Leon Romanovsky <leon@kernel.org>, <linux-rdma@vger.kernel.org>,
Mark Bloch <mbloch@nvidia.com>, Moshe Shemesh <moshe@nvidia.com>,
Or Har-Toov <ohartoov@nvidia.com>,
Saeed Mahameed <saeedm@nvidia.com>, Shay Drory <shayd@nvidia.com>,
Tariq Toukan <tariqt@nvidia.com>
Subject: [PATCH net-next V2 5/5] net/mlx5: LAG, enable LAG over functions that do not represent a physical port
Date: Fri, 20 Nov 2026 09:59:58 +0200 [thread overview]
Message-ID: <20261120075959.2533702-6-tariqt@nvidia.com> (raw)
In-Reply-To: <20261120075959.2533702-1-tariqt@nvidia.com>
From: Shay Drory <shayd@nvidia.com>
LAG membership assumes every member represents a physical port.
A nested LAG of VFs/SFs does not fit that assumption; its member
functions sit behind a multi_uplink_eswitch and each reports
num_lag_ports < 2, so they do not represent a physical port.
Detect this with mlx5_virt_lag_is_supported(): let such a function join
the LAG in mlx5_lag_add_mdev(), and skip the MLX5_LAG_XA_MARK_PORT mark
and the PORT_CHANGE event notifier in mlx5_ldev_add_mdev(), since it is
not a physical port.
These function only support MPESW, and netdevs addition to LAG is only
needed for linux bond. Hence, don't allow such a lag to add netdevs to
itself. For the same reason, don't register the LAG netdev notifier for
it. Multipath LAG is not supported either, so skip mlx5_lag_mp_init()
and do not register the FIB notifier for it.
Such a function may report num_lag_ports == 0, which ldev->ports keeps
as is. Report at least one port from mlx5_lag_get_num_ports(), so user
would avoid divide by 0.
Such a function selects the uplink through port_select_eswitch, not
through a TX port affinity. Yet once its LAG is active, RDMA sets
lag_active, and mlx5_ib_lag_should_assign_affinity() then assigns a TX
port affinity to every QP, bypassing the num_lag_ports > 1 and
lag_tx_port_affinity capability checks. Make
mlx5_ib_lag_should_assign_affinity() return false for such a function.
Signed-off-by: Shay Drory <shayd@nvidia.com>
Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 ++
.../net/ethernet/mellanox/mlx5/core/lag/lag.c | 32 +++++++++++++------
.../ethernet/mellanox/mlx5/core/lag/mpesw.c | 2 +-
3 files changed, 26 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4a6ee14ffd7d..494b7036321b 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1612,6 +1612,9 @@ static inline void mlx5r_deref_wait_odp_mkey(struct mlx5_ib_mkey *mmkey)
static inline bool mlx5_ib_lag_should_assign_affinity(struct mlx5_ib_dev *dev)
{
+ if (dev->lag_active && mlx5_virt_lag_is_supported(dev->mdev))
+ return 0;
+
/*
* If the driver is in hash mode and the port_select_flow_table_bypass cap
* is supported, it means that the driver no longer needs to assign the port
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
index 193fbe47fef1..2cd2319f9318 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
@@ -366,6 +366,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
ldev->ports = MLX5_CAP_GEN(dev, num_lag_ports);
ldev->buckets = 1;
+ ldev->virt_lag = mlx5_virt_lag_is_supported(dev);
ldev->v2p_map = kcalloc(ldev->ports * MLX5_LAG_MAX_HASH_BUCKETS,
sizeof(*ldev->v2p_map), GFP_KERNEL);
if (!ldev->v2p_map)
@@ -381,7 +382,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
INIT_DELAYED_WORK(&ldev->bond_work, mlx5_do_bond_work);
INIT_WORK(&ldev->speed_update_work, mlx5_mpesw_speed_update_work);
- if (!mlx5_sd_is_supported(dev)) {
+ if (!mlx5_sd_is_supported(dev) && !ldev->virt_lag) {
ldev->nb.notifier_call = mlx5_lag_netdev_event;
write_pnet(&ldev->net, mlx5_core_net(dev));
if (register_netdevice_notifier_net(read_pnet(&ldev->net),
@@ -392,10 +393,13 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
}
ldev->mode = MLX5_LAG_MODE_NONE;
- err = mlx5_lag_mp_init(ldev);
- if (err)
- mlx5_core_err(dev, "Failed to init multipath lag err=%d\n",
- err);
+ if (!ldev->virt_lag) {
+ err = mlx5_lag_mp_init(ldev);
+ if (err)
+ mlx5_core_err(dev,
+ "Failed to init multipath lag err=%d\n",
+ err);
+ }
ldev->max_fns = ldev->ports;
return ldev;
@@ -468,7 +472,9 @@ int mlx5_lag_get_dev_index_by_seq(struct mlx5_lag *ldev, int seq)
/* Return the appropriate iterator filter for a device in LAG:
* - SD shared FDB active: iterate only the device's SD group
* - SD group exists but shared FDB not active: iterate all devices
- * - No SD: iterate ports only
+ * - Virt LAG: iterate all devices, since its members are not marked as
+ * ports, as they do not represent a physical port
+ * - Otherwise: iterate ports only
*/
static u32 mlx5_lag_get_filter(struct mlx5_lag *ldev, struct mlx5_core_dev *dev)
{
@@ -476,7 +482,7 @@ static u32 mlx5_lag_get_filter(struct mlx5_lag *ldev, struct mlx5_core_dev *dev)
if (fn && fn->sd_fdb_active)
return fn->group_id;
- if (fn && fn->group_id)
+ if (fn && (fn->group_id || ldev->virt_lag))
return MLX5_LAG_FILTER_ALL;
return MLX5_LAG_FILTER_PORTS;
}
@@ -2398,7 +2404,7 @@ int mlx5_ldev_add_mdev(struct mlx5_lag *ldev,
fn->group_id = group_id;
dev->priv.lag = ldev;
- if (group_id)
+ if (group_id || ldev->virt_lag)
return 0;
xa_set_mark(&ldev->fns, idx, MLX5_LAG_XA_MARK_PORT);
@@ -2547,7 +2553,7 @@ void mlx5_lag_add_mdev(struct mlx5_core_dev *dev)
{
int err;
- if (!mlx5_lag_is_supported(dev))
+ if (!mlx5_lag_is_supported(dev) && !mlx5_virt_lag_is_supported(dev))
return;
if (mlx5_lag_register_hca_devcom_comp(dev))
@@ -2582,6 +2588,9 @@ void mlx5_lag_remove_netdev(struct mlx5_core_dev *dev,
if (!ldev)
return;
+ if (ldev->virt_lag)
+ return;
+
mutex_lock(&ldev->lock);
mlx5_ldev_remove_netdev(ldev, netdev);
clear_bit(MLX5_LAG_FLAG_NDEVS_READY, &ldev->state_flags);
@@ -2603,6 +2612,9 @@ void mlx5_lag_add_netdev(struct mlx5_core_dev *dev,
if (!ldev)
return;
+ if (ldev->virt_lag)
+ return;
+
mutex_lock(&ldev->lock);
mlx5_ldev_add_netdev(ldev, dev, netdev);
num = mlx5_lag_num_netdevs(ldev);
@@ -2886,7 +2898,7 @@ u8 mlx5_lag_get_num_ports(struct mlx5_core_dev *dev)
if (!ldev)
return 0;
- return ldev->ports;
+ return max_t(u8, ldev->ports, 1);
}
EXPORT_SYMBOL(mlx5_lag_get_num_ports);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
index 2d3e61541d8a..0ef2d13d0096 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
@@ -336,7 +336,7 @@ bool mlx5_lag_is_mpesw(struct mlx5_core_dev *dev)
struct mlx5_lag *ldev = mlx5_lag_dev(dev);
return ldev && ldev->mode == MLX5_LAG_MODE_MPESW &&
- __mlx5_lag_dev_is_port(ldev, dev);
+ (ldev->virt_lag || __mlx5_lag_dev_is_port(ldev, dev));
}
EXPORT_SYMBOL(mlx5_lag_is_mpesw);
--
2.44.0
prev parent reply other threads:[~2026-11-20 8:01 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 7:59 [PATCH net-next V2 0/5] net/mlx5: LAG over VFs/SFs (virtual LAG) Tariq Toukan
2026-11-20 7:59 ` [PATCH net-next V2 1/5] net/mlx5: E-switch, add match-metadata allocator for nested VF/SF eswitches Tariq Toukan
2026-11-20 7:59 ` [PATCH net-next V2 2/5] net/mlx5: E-switch, match master egress rule by vport metadata Tariq Toukan
2026-11-20 7:59 ` [PATCH net-next V2 3/5] net/mlx5: E-switch, drop master egress rule under a SW vport LAG Tariq Toukan
2026-11-20 7:59 ` [PATCH net-next V2 4/5] net/mlx5e: Register eswitch-manager VF/SF self-port with switch-id Tariq Toukan
2026-11-20 7:59 ` Tariq Toukan [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120075959.2533702-6-tariqt@nvidia.com \
--to=tariqt@nvidia.com \
--cc=agoldberger@nvidia.com \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=gal@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=moshe@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=ohartoov@nvidia.com \
--cc=pabeni@redhat.com \
--cc=saeedm@nvidia.com \
--cc=shayd@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox