Re: [PATCH rdma-rc RESEND v1] RDMA/mlx5: Fix devlink deadlock on net namespace deletion

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Oct 26, 2020 at 03:43:59PM +0200, Parav Pandit wrote:
> When a mlx5 core devlink instance is reloaded in different net
> namespace, its associated IB device is deleted and recreated.
> 
> Example sequence is:
> $ ip netns add foo
> $ devlink dev reload pci/0000:00:08.0 netns foo
> $ ip netns del foo
> 
> mlx5 IB device needs to attach and detach the netdevice to it
> through the netdev notifier chain during load and unload sequence.
> A below call graph of the unload flow.
> 
> cleanup_net()
>    down_read(&pernet_ops_rwsem); <- first sem acquired
>      ops_pre_exit_list()
>        pre_exit()
>          devlink_pernet_pre_exit()
>            devlink_reload()
>              mlx5_devlink_reload_down()
>                mlx5_unload_one()
>                [...]
>                  mlx5_ib_remove()
>                    mlx5_ib_unbind_slave_port()
>                      mlx5_remove_netdev_notifier()
>                        unregister_netdevice_notifier()
>                          down_write(&pernet_ops_rwsem);<- recurrsive lock
> 
> Hence, when net namespace is deleted, mlx5 reload results in deadlock.
> 
> When deadlock occurs, devlink mutex is also held. This not only deadlocks
> the mlx5 device under reload, but all the processes which attempt to access
> unrelated devlink devices are deadlocked.
> 
> Hence, fix this by mlx5 ib driver to register for per net netdev
> notifier instead of global one, which operats on the net namespace
> without holding the pernet_ops_rwsem.
> 
> Fixes: 4383cfcc65e7 ("net/mlx5: Add devlink reload")
> Signed-off-by: Parav Pandit <parav@nvidia.com>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> Changelog:
> v0->v1:
>  - updated comment for mlx5_core_net API to be used by multiple mlx5
>    drivers
> ---
>  drivers/infiniband/hw/mlx5/main.c              |  6 ++++--
>  .../net/ethernet/mellanox/mlx5/core/lib/mlx5.h |  5 -----
>  include/linux/mlx5/driver.h                    | 18 ++++++++++++++++++
>  3 files changed, 22 insertions(+), 7 deletions(-)

Applied to for-rc, thanks

Jason