From: Chenguang Zhao <chenguang.zhao@linux.dev>
To: Tariq Toukan <tariqt@nvidia.com>
Cc: Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>,
netdev@vger.kernel.org, Paolo Abeni <pabeni@redhat.com>,
Cosmin Ratiu <cratiu@nvidia.com>,
Dragos Tatulea <dtatulea@nvidia.com>,
Gal Pressman <gal@nvidia.com>, Jason Gunthorpe <jgg@ziepe.ca>,
Jianbo Liu <jianbol@nvidia.com>, Jiri Pirko <jiri@resnulli.us>,
Leon Romanovsky <leon@kernel.org>,
open list <linux-kernel@vger.kernel.org>,
linux-rdma@vger.kernel.org, Mark Bloch <mbloch@nvidia.com>,
Saeed Mahameed <saeedm@nvidia.com>, Shay Drori <shayd@nvidia.com>
Subject: Re: [PATCH net] net/mlx5: Attach late uplink netdev to loaded representors
Date: Fri, 16 Oct 2026 10:36:09 +0800 [thread overview]
Message-ID: <20261016023609.GA318264@pc> (raw)
In-Reply-To: <20261006110219.257714-1-tariqt@nvidia.com>
On Tue, Oct 06, 2026 at 02:02:19PM +0300, Tariq Toukan wrote:
> From: Jianbo Liu <jianbol@nvidia.com>
>
> mlx5e_vport_uplink_rep_load() returns success when the uplink netdev is
> missing, so the ETH rep is marked loaded with a NULL netdev, the IB rep
> copies that NULL association, and nothing repairs it later. Unbind
> mlx5_core.eth.<N> in legacy mode, enter switchdev, bind it again, and
> rdma link show still reports no netdev on the uplink port. The netdev
> also keeps the NIC profile instead of the uplink rep profile.
>
> Add an attach_uplink_netdev rep op, run from the existing reload-reps
> work once mlx5_core_uplink_netdev_set() reports a netdev. ETH re-runs
> the uplink load, switching profile and setting rpriv->netdev; IB redoes
> ib_device_set_netdev(). Deferring to the work keeps the devlink lock
> out of probe context.
>
> Fixes: 6b4be64fd9fe ("net/mlx5e: Harden uplink netdev access against device unbind")
> Signed-off-by: Jianbo Liu <jianbol@nvidia.com>
> Reviewed-by: Shay Drori <shayd@nvidia.com>
> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
> ---
> drivers/infiniband/hw/mlx5/ib_rep.c | 24 +++++++++++++
> .../net/ethernet/mellanox/mlx5/core/en_rep.c | 15 ++++++++
> .../net/ethernet/mellanox/mlx5/core/eswitch.h | 3 ++
> .../mellanox/mlx5/core/eswitch_offloads.c | 35 +++++++++++++++++++
> .../net/ethernet/mellanox/mlx5/core/main.c | 3 ++
> include/linux/mlx5/eswitch.h | 5 +++
> 6 files changed, 85 insertions(+)
>
> diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c
> index 65d8767d1830..f9931a94157a 100644
> --- a/drivers/infiniband/hw/mlx5/ib_rep.c
> +++ b/drivers/infiniband/hw/mlx5/ib_rep.c
> @@ -273,10 +273,34 @@ mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep)
> }
> }
>
> +static int
> +mlx5_ib_vport_uplink_rep_attach_netdev(struct mlx5_core_dev *mdev,
> + struct mlx5_eswitch_rep *rep)
> +{
> + struct mlx5_ib_dev *dev = mlx5_ib_rep_to_dev(rep);
> + struct net_device *ndev;
> + int i;
> +
> + /* Shared FDB slave uplinks share the master's IB device. */
> + if (!dev)
> + return 0;
> +
> + ndev = mlx5_ib_get_rep_netdev(rep->esw, rep->vport);
> + if (!ndev)
> + return -ENODEV;
> +
> + for (i = 0; i < dev->num_ports; i++)
> + if (dev->port[i].rep == rep)
> + return ib_device_set_netdev(&dev->ib_dev, ndev, i + 1);
> +
> + return 0;
> +}
> +
> static const struct mlx5_eswitch_rep_ops rep_ops = {
> .load = mlx5_ib_vport_rep_load,
> .unload = mlx5_ib_vport_rep_unload,
> .get_proto_dev = mlx5_ib_rep_to_dev,
> + .attach_uplink_netdev = mlx5_ib_vport_uplink_rep_attach_netdev,
> };
>
> static void mlx5_ib_register_peer_vport_reps(struct mlx5_core_dev *mdev)
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
> index 88a170e40bd9..19812bf6a830 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
> @@ -1490,10 +1490,24 @@ mlx5e_vport_uplink_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *
> rpriv->netdev = netdev;
> err = mlx5e_netdev_change_profile(netdev, dev,
> &mlx5e_uplink_rep_profile, rpriv);
> + if (err)
> + rpriv->netdev = NULL;
> mlx5_uplink_netdev_put(dev, netdev);
> return err;
> }
>
> +static int
> +mlx5e_vport_uplink_rep_attach_netdev(struct mlx5_core_dev *dev,
> + struct mlx5_eswitch_rep *rep)
> +{
> + struct mlx5e_rep_priv *rpriv = mlx5e_rep_to_rep_priv(rep);
> +
> + if (rpriv->netdev)
> + return 0;
> +
> + return mlx5e_vport_uplink_rep_load(dev, rep);
> +}
> +
> static void
> mlx5e_vport_uplink_rep_unload(struct mlx5e_rep_priv *rpriv)
> {
> @@ -1741,6 +1755,7 @@ static const struct mlx5_eswitch_rep_ops rep_ops = {
> .unload = mlx5e_vport_rep_unload,
> .get_proto_dev = mlx5e_vport_rep_get_proto_dev,
> .event = mlx5e_vport_rep_event,
> + .attach_uplink_netdev = mlx5e_vport_uplink_rep_attach_netdev,
> };
>
> static int mlx5e_rep_probe(struct auxiliary_device *adev,
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
> index 8b1f93b13ea9..3dd6931f13cd 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
> @@ -1012,6 +1012,7 @@ mlx5_esw_lag_demux_rule_create(struct mlx5_eswitch *esw, u16 vport_num,
> struct mlx5_flow_table *lag_ft);
> void mlx5_esw_reps_block(struct mlx5_eswitch *esw);
> void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw);
> +void mlx5_esw_offloads_uplink_netdev_attach(struct mlx5_core_dev *dev);
> #else /* CONFIG_MLX5_ESWITCH */
> /* eswitch API stubs */
> static inline int mlx5_eswitch_init(struct mlx5_core_dev *dev) { return 0; }
> @@ -1098,6 +1099,8 @@ mlx5_esw_host_functions_enabled(const struct mlx5_core_dev *dev)
>
> static inline void mlx5_esw_reps_block(struct mlx5_eswitch *esw) {}
> static inline void mlx5_esw_reps_unblock(struct mlx5_eswitch *esw) {}
> +static inline void
> +mlx5_esw_offloads_uplink_netdev_attach(struct mlx5_core_dev *dev) {}
>
> static inline bool
> mlx5_esw_vport_vhca_id(struct mlx5_eswitch *esw, u16 vportn, u16 *vhca_id)
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
> index eb74b6260168..cf61bd76889c 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
> @@ -4842,6 +4842,29 @@ mlx5_eswitch_register_vport_reps_blocked(struct mlx5_eswitch *esw,
> }
> }
>
> +static void mlx5_eswitch_attach_uplink_netdev(struct mlx5_eswitch *esw,
> + struct mlx5_eswitch_rep *uplink)
> +{
> + const struct mlx5_eswitch_rep_ops *ops;
> + int type;
> + int err;
> +
> + for (type = 0; type < NUM_REP_TYPES; type++) {
> + if (atomic_read(&uplink->rep_data[type].state) != REP_LOADED)
> + continue;
> +
> + ops = esw->offloads.rep_ops[type];
> + if (!ops || !ops->attach_uplink_netdev)
> + continue;
> +
> + err = ops->attach_uplink_netdev(esw->dev, uplink);
> + if (err)
> + esw_warn(esw->dev,
> + "Failed to attach uplink netdev to rep type %d, err(%d)\n",
> + type, err);
> + }
> +}
> +
> static void mlx5_eswitch_reload_reps_blocked(struct mlx5_eswitch *esw)
> {
> struct mlx5_eswitch_rep *uplink;
> @@ -4862,6 +4885,8 @@ static void mlx5_eswitch_reload_reps_blocked(struct mlx5_eswitch *esw)
> return;
> }
>
> + mlx5_eswitch_attach_uplink_netdev(esw, uplink);
> +
> if (mlx5_get_sd(esw->dev) && !mlx5_lag_is_active(esw->dev))
> return;
>
> @@ -4886,6 +4911,16 @@ static void mlx5_eswitch_reload_reps(struct mlx5_eswitch *esw)
> mlx5_esw_reps_unblock(esw);
> }
>
> +void mlx5_esw_offloads_uplink_netdev_attach(struct mlx5_core_dev *dev)
> +{
> + struct mlx5_eswitch *esw = dev->priv.eswitch;
> +
> + if (!mlx5_esw_allowed(esw))
> + return;
> +
> + mlx5_esw_add_work(esw, mlx5_eswitch_reload_reps, GFP_KERNEL);
> +}
> +
> static void
> mlx5_eswitch_register_vport_reps_locked(struct mlx5_eswitch *esw,
> const struct mlx5_eswitch_rep_ops *ops,
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> index 5f28d906c35b..ab93c807b58c 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
> @@ -278,6 +278,9 @@ void mlx5_core_uplink_netdev_set(struct mlx5_core_dev *dev, struct net_device *n
> mlx5_blocking_notifier_call_chain(dev, MLX5_DRIVER_EVENT_UPLINK_NETDEV,
> netdev);
> mutex_unlock(&dev->mlx5e_res.uplink_netdev_lock);
> +
> + if (netdev)
> + mlx5_esw_offloads_uplink_netdev_attach(dev);
> }
>
> void mlx5_core_uplink_netdev_event_replay(struct mlx5_core_dev *dev)
> diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h
> index a0dd162baa78..e823fbe7ff40 100644
> --- a/include/linux/mlx5/eswitch.h
> +++ b/include/linux/mlx5/eswitch.h
> @@ -43,6 +43,11 @@ struct mlx5_eswitch_rep_ops {
> struct mlx5_eswitch_rep *rep,
> enum mlx5_switchdev_event event,
> void *data);
> + /* Attach an uplink netdev that showed up only after the uplink
> + * representor was already loaded.
> + */
> + int (*attach_uplink_netdev)(struct mlx5_core_dev *dev,
> + struct mlx5_eswitch_rep *rep);
> };
>
> struct mlx5_eswitch_rep_data {
>
> base-commit: d5a007b9b457c915ab1a53227e8939e4018aa97a
> --
> 2.44.0
>
Hi all,
I’m wondering if we could add a check during mode configuration to
disallow mode switching when the device is unbound.
Maybe we can call `mlx5_uplink_netdev_get()` within `mlx5_devlink_eswitch_mode_set()`
to retrieve the netdev, and return `-ENODEV` if it comes back NULL.
What do you think about this approach?
Thanks
Chenguang
next prev parent reply other threads:[~2026-10-16 2:35 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-06 11:02 [PATCH net] net/mlx5: Attach late uplink netdev to loaded representors Tariq Toukan
2026-10-08 11:02 ` netdev-bot+sashiko
2026-10-13 11:23 ` Paolo Abeni
2026-10-14 7:26 ` Jianbo Liu
2026-10-16 2:36 ` Chenguang Zhao [this message]
2026-10-16 8:44 ` Jianbo Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261016023609.GA318264@pc \
--to=chenguang.zhao@linux.dev \
--cc=andrew+netdev@lunn.ch \
--cc=cratiu@nvidia.com \
--cc=davem@davemloft.net \
--cc=dtatulea@nvidia.com \
--cc=edumazet@kernel.org \
--cc=gal@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=jianbol@nvidia.com \
--cc=jiri@resnulli.us \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=saeedm@nvidia.com \
--cc=shayd@nvidia.com \
--cc=tariqt@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox