[PATCH net-next V2 0/7] net/mlx5: Improve representor lifecycle and allow switchdev by default

Tariq Toukan
2026-05-01T04:17:40Z

Hi,

Find detailed description by Mark below.

Regards,
Tariq

This series addresses three problems that have been present for years, and
fixes one representor reload error-unwind case exposed while making the
reload path reusable.

First, there is no coordination between E-Switch reconfiguration and
representor registration. The E-Switch can be mid-way through a mode
change or VF count update while mlx5_ib walks in and registers or
unregisters representors. Nothing stops them. The race window is small
and there is no field report, but it is clearly wrong.

Second, loading mlx5_ib while the device is already in switchdev mode
does not bring up the IB representors. mlx5_eswitch_register_vport_reps()
only stores callbacks; nobody triggers the actual load after registration.

Third, deployments that always use switchdev mode still need a manual
devlink mode change after probe. That makes automated provisioning more
fragile than needed.

The series fixes the registration race with a per-E-Switch representor
mutex. The lock is introduced first, then LAG shared-FDB and multiport
E-Switch transitions are adjusted so auxiliary device rescans and IB
representor reloads do not hold ldev->lock while taking the representor
lock. This keeps the intermediate commits bisectable before the stricter
E-Switch serialization and lock assertions are enabled.

After the LAG ordering is fixed, all E-Switch reconfiguration paths that
create, destroy, load, or unload representors take the representor mutex.
esw_mode_change() deliberately drops the mutex around
mlx5_rescan_drivers_locked(), because auxiliary probe and remove paths
re-enter mlx5_eswitch_register_vport_reps() and
mlx5_eswitch_unregister_vport_reps() on the same thread.

The shared-FDB peer IB registration path can hold one E-Switch
representor mutex and then register peer representor ops on another
E-Switch. The series annotates that case as nested locking so lockdep can
distinguish it from recursive locking on the same E-Switch.

For the missing IB representors, mlx5_eswitch_register_vport_reps() queues
a work item that acquires the devlink lock and loads all relevant
representors. This is the change that actually fixes the long-standing
bug.

The reload path also learns to track which representor types were loaded by
the current attempt, so an error does not unload representors that were
already active before the retry.

Patch 1 is cleanup. LAG and MPESW had the same representor reload
sequence duplicated in several places and the copies had started to
drift. This consolidates them into one helper.

Patch 2 adds the per-E-Switch representor lifecycle lock and helper APIs.

Patch 3 adjusts the LAG shared-FDB and multiport E-Switch transitions so
auxiliary device rescans and IB representor reloads run without
ldev->lock held while taking the representor lock.

Patch 4 protects the E-Switch reconfiguration, representor registration
and peer IB representor paths with the representor lock.

Patch 5 fixes representor load error unwind so only representor types
loaded by the current attempt are unloaded on failure.

Patch 6 moves the representor load triggered by
mlx5_eswitch_register_vport_reps() onto the work queue. This is the patch
that fixes IB representors not coming up when mlx5_ib is loaded while the
device is already in switchdev mode.

Patch 7 adds a driver profile that auto-enables switchdev at device init,
for deployments that always operate in switchdev mode and want to avoid a
manual devlink command after every probe.

V2:

Split v1 into two parts: the E-Switch workqueue deadlock fix and the
representor lifecycle changes. This is the second part; the first part
has already been accepted [1].

Patch 1: Add a cont_on_fail flag so callers can decide whether reload
should continue after a failure.

Patches 2, 3, 4: Replace the atomic-variable based scheme with a mutex,
per Jakub's feedback.

Patch 5: New patch that fixes the unwind on representor load failure.

Patch 7: Switch from profile 4 to profile 8. Since the profile mainly
targets E-Switch handling, keep it separate from the NIC profiles.

[1] https://lore.kernel.org/all/20260428051018.219093-1-tariqt@nvidia.com/

Link to V1:
https://lore.kernel.org/all/20260409115550.156419-1-tariqt@nvidia.com/

Mark Bloch (7):
net/mlx5: Lag: refactor representor reload handling
net/mlx5: E-Switch, add representor lifecycle lock
net/mlx5: Lag, avoid LAG and representor lock cycles
net/mlx5: E-Switch, serialize representor lifecycle
net/mlx5: E-Switch, unwind only newly loaded representor types
net/mlx5: E-switch, load reps via work queue after registration
net/mlx5: Add profile to auto-enable switchdev mode at device init

drivers/infiniband/hw/mlx5/ib_rep.c | 6 +-
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 10 +
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 12 ++
.../mellanox/mlx5/core/eswitch_offloads.c | 186 ++++++++++++++++--
.../net/ethernet/mellanox/mlx5/core/lag/lag.c | 171 ++++++++++++----
.../net/ethernet/mellanox/mlx5/core/lag/lag.h | 5 +
.../ethernet/mellanox/mlx5/core/lag/mpesw.c | 18 +-
.../ethernet/mellanox/mlx5/core/lib/devcom.c | 8 +
.../ethernet/mellanox/mlx5/core/lib/devcom.h | 1 +
.../net/ethernet/mellanox/mlx5/core/main.c | 43 +++-
.../ethernet/mellanox/mlx5/core/sf/devlink.c | 5 +
include/linux/mlx5/driver.h | 2 +
include/linux/mlx5/eswitch.h | 6 +
13 files changed, 405 insertions(+), 68 deletions(-)

base-commit: 4e37987362bcac8909f2d4b4458f3aa645e41641
--
2.44.0