Re: [PATCH 0/3] RDMA net namespace

[yanjun.zhu@linux.dev]

October 24, 2022 9:10 AM, "Dust Li" <dust.li@linux.alibaba.com> wrote:

> On Sun, Oct 23, 2022 at 06:04:47PM -0400, Zhu Yanjun wrote:
> 
>> From: Zhu Yanjun <yanjun.zhu@linux.dev>
>> 
>> There are shared and exclusive modes in RDMA net namespace. After
>> discussion with Leon, the above modes are compatible with legacy IB
>> device.
>> 
>> To the RoCE and iWARP devices, the ib devices should be in the same net
>> namespace with the related net devices regardless of in shared or
>> exclusive mode.
> 
> Does this mean that shared mode is no longer supported for RoCE and iWarp
> devices ?

From the discussion,  a RoCE and iWarp device should make ib devices and net devices in the same net. So a RoCE and iWarp device has no shared/exclusive modes.

Shared/exclusive modes are for legacy ib devices, such as ipoib. 

In this patch series, shared/exclusive modes are left for legacy ib devices.
To a RoCE and iWarp device, we just keep net devices and ib devices in the same net.



> 
>> In the first commit, when the net devices are moved to a new net
>> namespace, the related ib devices are also moved to the same net
>> namespace.
>> 
>> In the second commit, the shared/exclusive modes still work with legacy
>> ib devices. To the RoCE and iWARP devices, these modes will not be
>> considered.
>> 
>> Because MLX4/5 do not call the function ib_device_set_netdev to map ib
>> devices and the related net devices, the function ib_device_get_by_netdev
>> can not get ib devices from net devices. In the third commit, all the
>> registered ib devices are parsed to get the net devices, then compared
>> with the given net devices.
>> 
>> The steps to make tests:
>> 1) Create a new net namespace net0
>> 
>> ip netns add net0
>> 
>> 2) Show the rdma links in init_net
>> 
>> rdma link
>> 
>> "
>> link mlx5_0/1 state DOWN physical_state DISABLED netdev enp7s0np1
>> "
>> 
>> 3) Move the net device to net namespace net0
>> 
>> ip link set enp7s0np1 netns net0
>> 
>> 4) Show the rdma links in init_net again
>> 
>> rdma link
>> 
>> There is no rdma links
>> 
>> 5) Show the rdma links in net0
>> 
>> ip netns exec net0 rdma link
>> 
>> "
>> link mlx5_0/1 state DOWN physical_state DISABLED netdev enp7s0np1
>> "
>> 
>> We can confirm that rdma links are moved to the same net namespace with
>> the net devices.
>> 
>> Zhu Yanjun (3):
>> RDMA/core: Move ib device to the same net namespace with net device
>> RDMA/core: The legacy IB devices still work with shared/exclusive mode
>> RDMA/core: Get all the ib devices from net devices
>> 
>> drivers/infiniband/core/device.c | 107 ++++++++++++++++++++++++++++++-
>> 1 file changed, 105 insertions(+), 2 deletions(-)
>> 
>> --
>> 2.27.0