From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB0BCC433F5 for ; Fri, 30 Sep 2022 07:25:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229882AbiI3HZP (ORCPT ); Fri, 30 Sep 2022 03:25:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229875AbiI3HZO (ORCPT ); Fri, 30 Sep 2022 03:25:14 -0400 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EF0F7AC30 for ; Fri, 30 Sep 2022 00:25:09 -0700 (PDT) Message-ID: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1664522706; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8Tb044iaRQPpSp6fY2q2A7CQyC3tjNI+YMFaJovHq14=; b=id/Vk3vB/LevRUeV++wDD0YZHGCf6QmKWKAJBdYqj79cD3w8Y7tdjSSWmIy3Cs3wjMzlhw CT4y5Mx2smXtcSWKGE0uszov30ob/vaY+jEp7NeFaKEhwuRyZ+6ODPucskrPHBS6Yk/MvO ozXHscB7vyiIXZSOL6DR12feDWIn3xQ= Date: Fri, 30 Sep 2022 15:25:00 +0800 MIME-Version: 1.0 Subject: Re: [PATCH] rdma: not display the rdma link in other net namespace To: Leon Romanovsky , Yanjun Zhu Cc: Leon Romanovsky , linux-rdma@vger.kernel.org, jgg@nvidia.com References: <20220926024033.284341-1-yanjun.zhu@linux.dev> <4e5d49fe-38a3-4891-3755-3decf8ffebda@linux.dev> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Yanjun Zhu In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org 在 2022/9/28 14:04, Leon Romanovsky 写道: > On Tue, Sep 27, 2022 at 06:58:50PM +0800, Yanjun Zhu wrote: >> >> 在 2022/9/27 18:34, Leon Romanovsky 写道: >>> On Sun, Sep 25, 2022 at 10:40:33PM -0400, yanjun.zhu@linux.dev wrote: >>>> From: Zhu Yanjun >>>> >>>> When the net devices are moved to another net namespace, the command >>>> "rdma link" should not dispaly the rdma link about this net device. >>>> >>>> For example, when the net device eno12399 is moved to net namespace net0 >>>> from init_net, the rdma link of eno12399 should not display in init_net. >>>> >>>> Before this change: >>>> >>>> Init_net: >>>> >>>> link roceo12399/1 state DOWN physical_state DISABLED <---should not display >>>> link roceo12409/1 state DOWN physical_state DISABLED netdev eno12409 >>>> link rocep202s0f0/1 state DOWN physical_state DISABLED netdev ens7f0 >>>> link rocep202s0f1/1 state ACTIVE physical_state LINK_UP netdev ens7f1 >>>> >>>> net0: >>>> >>>> link roceo12399/1 state DOWN physical_state DISABLED netdev eno12399 >>>> link roceo12409/1 state DOWN physical_state DISABLED <---should not display >>>> link rocep202s0f0/1 state DOWN physical_state DISABLED <---should not display >>>> link rocep202s0f1/1 state ACTIVE physical_state LINK_UP <---should not display >>>> >>>> After this change >>>> >>>> Init_net: >>>> >>>> link roceo12409/1 state DOWN physical_state DISABLED netdev eno12409 >>>> link rocep202s0f0/1 state DOWN physical_state DISABLED netdev ens7f0 >>>> link rocep202s0f1/1 state ACTIVE physical_state LINK_UP netdev ens7f1 >>>> >>>> net0: >>>> >>>> link roceo12399/1 state DOWN physical_state DISABLED netdev eno12399 >>>> >>>> Fixes: da990ab40a92 ("rdma: Add link object") >>>> Signed-off-by: Zhu Yanjun >>>> --- >>>> rdma/link.c | 3 +++ >>>> 1 file changed, 3 insertions(+) >>>> >>>> diff --git a/rdma/link.c b/rdma/link.c >>>> index bf24b849..449a7636 100644 >>>> --- a/rdma/link.c >>>> +++ b/rdma/link.c >>>> @@ -238,6 +238,9 @@ static int link_parse_cb(const struct nlmsghdr *nlh, void *data) >>>> return MNL_CB_ERROR; >>>> } >>>> + if (!tb[RDMA_NLDEV_ATTR_NDEV_NAME] || !tb[RDMA_NLDEV_ATTR_NDEV_INDEX]) >>>> + return MNL_CB_OK; >>>> + >>> Regarding your question where it should go in addition to RDMA, the answer >>> is netdev ML. The rdmatool is part of iproute2 and the relevant maintainers >>> should be CCed. >> Thanks. I will also send it to netdev ML and CC the maintainers. >>> >>> Regarding the change, I don't think that it is right. User space tool is >>> a simple viewer of data returned from the kernel. It is not a mistake to >>> return device without netdev. >> >> Normally a rdma link based on RoCEv2 should be with a NIC. This NIC device >> >> will send/recv udp packets. With mellanox/intel NIC device, this net device >> also >> >> do more work than sending/receiving packets. >> >> From this perspective, a rdma link is dependent on a net device. >> >> In this problem, net device is moved to another net namespace. So it can not >> be >> >> obtained.  And this rdma link can also not work in this net namespace. >> >> So this rdma link should not appear in this net namespace. Or else, it would >> confuse >> >> the user. >> >> In fact, net namespace is a concept in tcp/ip stack. And it does not exist >> in rdma stack. > > RDMA has two different net namespace mode: shared and exclusive. This is different from net namespace in network. > > In shared mode, the IB devices are shared across all net namespaces and > "moving" net device into different namespace just "hides" it, but don't > disconnect. In exclusive mode, the net device also hide. It is the same with shared mode. # rdma system netns exclusive copy-on-fork on # rdma link link roceo12399/1 state DOWN physical_state DISABLED link roceo12409/1 state DOWN physical_state DISABLED netdev eno12409 link rocep202s0f0/1 state DOWN physical_state DISABLED netdev ens7f0 link rocep202s0f1/1 state ACTIVE physical_state LINK_UP netdev ens7f1 Is it better to append "exclusive" or "shared" in the end of the line? For example, Exclusive mode: # rdma system netns exclusive copy-on-fork on # rdma link link roceo12399/1 state DOWN physical_state DISABLED exclusive Shared mode: # rdma system netns shared copy-on-fork on # rdma link link roceo12399/1 state DOWN physical_state DISABLED shared Thanks and Regards Zhu Yanjun > > See comments around various usages of ib_devices_shared_netns variable. > > Thanks