Netdev List
 help / color / mirror / Atom feed
From: Shay Drory <shayd@nvidia.com>
To: <netdev-bot+sashiko@kernel.org>
Cc: <tariqt@nvidia.com>, <andrew+netdev@lunn.ch>,
	<davem@davemloft.net>, <edumazet@kernel.org>, <kuba@kernel.org>,
	<netdev@vger.kernel.org>, <pabeni@redhat.com>,
	<cjubran@nvidia.com>, <cratiu@nvidia.com>, <gal@nvidia.com>,
	<leon@kernel.org>, <linux-kernel@vger.kernel.org>,
	<linux-rdma@vger.kernel.org>, <mbloch@nvidia.com>,
	<moshe@nvidia.com>, <ohartoov@nvidia.com>, <phaddad@nvidia.com>,
	<saeedm@nvidia.com>, <udar@nvidia.com>
Subject: Re: [PATCH net-next V2 09/10] net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan
Date: Thu, 19 Nov 2026 10:40:42 +0200	[thread overview]
Message-ID: <20261119084042.1589375-1-shayd@nvidia.com> (raw)
In-Reply-To: <179499791924.3136.16901930891335328702@kernel.org>

On Wed, 18 Nov 2026 10:31:59 +0000, netdev-bot+sashiko@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential
> issue(s) to consider.
>
> Critical: 0 · High: 1 · Medium: 0 · Low: 0
>
> If you believe none of the issues are legitimate, please reply to each
> issue of Medium severity or higher and explain why the AI is wrong.
> If any of them are legitimate and you plan to respin, please let patchwork
> know by including "pw-bot: cr" as a separate line at the end of your reply
> (one such reply per series is enough).
>
> - [High] In mlx5_lag_rescan_devices_locked_filter()
>   (drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c), this patch…
>
> --
>
> net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan
>
> This commit removes the devs[MLX5_MAX_PORTS] array from
> mlx5_lag_rescan_devices_locked_filter(). The post-unlock rescan loop now
> walks the LAG members again with mlx5_lag_for_each(). The commit message
> describes this as having no functional change.
>
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
> > index a7b12353b7225..3e88b3f51612e 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
>
> [ ... ]
>
> > @@ -1356,12 +1354,16 @@ static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev,
> >  			fn->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
> >  		else
> >  			fn->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
> > -		devs[num_devs++] = fn->dev;
> >  	}
> >
> >  	mlx5_lag_drop_lock_for_reps(ldev, filter);
> > -	for (i = 0; i < num_devs; i++)
> > -		mlx5_rescan_drivers_locked(devs[i]);
> > +	mlx5_lag_for_each(i, 0, ldev, filter) {
> > +		fn = mlx5_lag_fn(ldev, i);
> > +		if (fn->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
> > +			continue;
> > +
> > +		mlx5_rescan_drivers_locked(fn->dev);
> > +	}
>
> [Severity: High]
> This loop now walks ldev->fns after mlx5_lag_drop_lock_for_reps() has
> released ldev->lock. Can a concurrent SD member removal free a lag_fn
> while the walk is in progress?

Valid. The race between SD join/leave and a LAG transition that drops
ldev->lock is pre-existing; SD join/leave doesn't observe
mode_changes_in_progress. It will be fixed in a follow-up series.

>
> On each step, mlx5_get_next_lag_func() reads xarray entries and fn->dev.
> mlx5_lag_fn() then does an xa_load(), and the loop dereferences
> fn->dev->priv.flags. The only thing guarding membership while the lock is
> dropped is ldev->mode_changes_in_progress. The comment in
> mlx5_lag_drop_lock_for_reps() says:
>
> 	/* Keep PF membership stable while ldev->lock is dropped. Device add
> 	 * and remove paths observe mode_changes_in_progress and retry.
> 	 */
>
> The SD paths in lib/sd.c don't appear to check that counter, though.
> sd_lag_cleanup() takes the lock directly:
>
> 	mutex_lock(&ldev->lock);
> 	mlx5_sd_for_each_secondary(i, primary, pos)
> 		mlx5_ldev_remove_mdev(ldev, pos);
>
> mlx5_ldev_remove_mdev() then frees the entry right away, with no RCU
> deferral:
>
> 	fn->dev = NULL;
> 	dev->priv.lag = NULL;
> 	xa_erase(&ldev->fns, fn->idx);
> 	kfree(fn);
>
> sd_lag_cleanup() is serialized only by sd->devcom. For
> MLX5_LAG_FILTER_ALL, this function runs under the HCA devcom instead.
> mlx5_do_bond_work() takes only the HCA devcom trylock plus ldev->lock, and
> not mlx5_mpesw_sd_devcoms_lock(). It reaches this function as follows:
>
> mlx5_do_bond_work()
>   mlx5_do_bond()
>     mlx5_lag_shared_fdb_create(ldev, ..., MLX5_LAG_MODE_SRIOV, 0)
>       mlx5_lag_remove_devices_filter(ldev, MLX5_LAG_FILTER_ALL)
>         mlx5_lag_rescan_devices_locked_filter()
>
> mlx5_disable_lag() -> mlx5_lag_shared_fdb_destroy(ldev, 0) reaches it the
> same way. Group id 0 maps to MLX5_LAG_FILTER_ALL, which matches every
> member, SD secondaries included.
>
> At the same time, removing an SD member's eth aux driver can run:
>
> mlx5e_remove() / mlx5e_suspend()
>   mlx5_sd_cleanup()
>     sd_lag_cleanup()
>       mlx5_ldev_remove_mdev()
>
> This removal can come from a sysfs unbind of mlx5_core.eth.N. It can also
> come from a reload of an SD secondary whose hca_devcom_comp is NULL or in a
> different component. Neither case holds the HCA devcom that the bond work
> holds.
>
> If that happens, mlx5_get_next_lag_func() could read fn->dev from a freed
> lag_fn. mlx5_lag_fn() could also return NULL after xa_erase(), and the
> loop would then dereference it.
>
> The old loop touched only the devs[] core_dev pointers once the lock was
> dropped. Those pointers stay valid across an aux unbind, so lag_fn and
> xarray state was never accessed without the lock.
>
> A similar question applies to sd_lag_init(), which also takes ldev->lock
> without checking mode_changes_in_progress and calls mlx5_ldev_add_mdev().
> Could the second loop then rescan a newly added member whose
> MLX5_PRIV_FLAGS_DISABLE_IB_ADEV flag the first loop never updated?
>
> Given the above, is the "no functional change" statement in the commit
> message accurate?
>
> Would it be safer to keep a snapshot taken under the lock? For example, a
> dynamically sized array, or a reference on each core_dev collected before
> mlx5_lag_drop_lock_for_reps().
>
> As far as I can tell, the last patch in the series doesn't touch this
> function or the SD paths, so this would still be present at the end of the
> series.
> >  	mlx5_lag_retake_lock_after_reps(ldev);
> >  }
> >
>
> --
> Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com

  reply	other threads:[~2026-11-19  8:41 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 10:30 [PATCH net-next V2 00/10] net/mlx5: LAG, preparations for LAG over VFs/SFs Tariq Toukan
2026-11-16 10:30 ` [PATCH net-next V2 01/10] net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping Tariq Toukan
2026-11-18 10:31   ` netdev-bot+sashiko
2026-11-19  7:40     ` Shay Drory
2026-11-16 10:30 ` [PATCH net-next V2 02/10] net/mlx5: LAG, select the MPESW master from the live member snapshot Tariq Toukan
2026-11-16 10:30 ` [PATCH net-next V2 03/10] net/mlx5: LAG, skip phys LAG prerequisites for a virt LAG Tariq Toukan
2026-11-16 10:30 ` [PATCH net-next V2 04/10] net/mlx5: LAG, treat virt LAG as a software managed LAG Tariq Toukan
2026-11-16 10:30 ` [PATCH net-next V2 05/10] net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS Tariq Toukan
2026-11-18 10:31   ` netdev-bot+sashiko
2026-11-19  8:10     ` Shay Drory
2026-11-16 10:30 ` [PATCH net-next V2 06/10] net/mlx5: LAG, rename PF naming to fn Tariq Toukan
2026-11-18 10:31   ` netdev-bot+sashiko
2026-11-16 10:30 ` [PATCH net-next V2 07/10] net/mlx5: LAG, resolve affinity indices only in queue-affinity mode Tariq Toukan
2026-11-16 10:30 ` [PATCH net-next V2 08/10] net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation Tariq Toukan
2026-11-18 10:31   ` netdev-bot+sashiko
2026-11-19  8:20     ` Shay Drory
2026-11-16 10:30 ` [PATCH net-next V2 09/10] net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan Tariq Toukan
2026-11-18 10:31   ` netdev-bot+sashiko
2026-11-19  8:40     ` Shay Drory [this message]
2026-11-16 10:30 ` [PATCH net-next V2 10/10] net/mlx5: LAG, do not add a device to an already-active virt LAG Tariq Toukan
2026-11-18 10:32   ` netdev-bot+sashiko
2026-11-19  8:58     ` Shay Drory

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119084042.1589375-1-shayd@nvidia.com \
    --to=shayd@nvidia.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=cjubran@nvidia.com \
    --cc=cratiu@nvidia.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=gal@nvidia.com \
    --cc=kuba@kernel.org \
    --cc=leon@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mbloch@nvidia.com \
    --cc=moshe@nvidia.com \
    --cc=netdev-bot+sashiko@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=ohartoov@nvidia.com \
    --cc=pabeni@redhat.com \
    --cc=phaddad@nvidia.com \
    --cc=saeedm@nvidia.com \
    --cc=tariqt@nvidia.com \
    --cc=udar@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox