From: Kamal Heib <kamalheib1@gmail.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: linux-rdma@vger.kernel.org, Doug Ledford <dledford@redhat.com>
Subject: Re: [PATCH for-rc] RDMA/ipoib: Set rtnl_link_ops for ipoib interfaces
Date: Tue, 29 Sep 2020 09:04:38 +0300 [thread overview]
Message-ID: <20200929060438.GA73375@kheib-workstation> (raw)
In-Reply-To: <20200928223602.GS9916@ziepe.ca>
On Mon, Sep 28, 2020 at 07:36:02PM -0300, Jason Gunthorpe wrote:
> On Mon, Sep 28, 2020 at 11:26:31PM +0300, Kamal Heib wrote:
> > Before this patch, the rtnl_link_ops are set only for ipoib network
> > devices that are created via the rtnl_link_ops->newlink() callback, this
> > patch fixes that by setting the rtnl_link_ops for all ipoib network
> > devices. Also, implement the dellink() callback to block users from
> > trying to remove the base ipoib network device while allowing it only
> > for child interfaces.
>
> Why?
>
This is needed to avoid the inconsistent user experience for PKeys that
is created via netlink VS PKeys that is created via sysfs and the based
ipoib interface, as you can see below the ipoib attributes are reported
only for PKeys that is created via netlink in the 'ip -d link show'
output:
PKey created via netlink (pkey, mode, and umcast attributes are present):
$ ip link add link mlx5_ib0 name mlx5_ib0.8001 type ipoib pkey 0x8001
$ ip -d link show dev mlx5_ib0.8001
28: mlx5_ib0.8001@mlx5_ib0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN mode DEFAULT group default qlen 256
link/infiniband 00:00:1f:e3:fe:80:00:00:00:00:00:00:24:8a:07:03:00:a3:19:64 brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65520
ipoib pkey 0x8001 mode datagram umcast 0000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
While:
PKey created via sysfs (the attributes are not present):
$ ip -d link show dev mlx5_ib0.8002
20: mlx5_ib0.8002@mlx5_ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2044 qdisc mq state UP mode DEFAULT group default qlen 256
link/infiniband 00:00:11:7b:fe:80:00:00:00:00:00:00:24:8a:07:03:00:a3:19:64 brd 00:ff:ff:ff:ff:12:40:1b:80:02:00:00:00:00:00:00:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65520 addrgenmode none numtxqueues 256 numrxqueues 32 gso_max_size 65536 gso_max_segs 65535
Same for the base interface:
$ ip -d link show dev mlx5_ib0
19: mlx5_ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2044 qdisc mq state UP mode DEFAULT group default qlen 256
link/infiniband 00:00:11:79:fe:80:00:00:00:00:00:00:24:8a:07:03:00:a3:19:64 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65520 addrgenmode none numtxqueues 256 numrxqueues 32 gso_max_size 65536 gso_max_segs 65535
altname ibp7s0f0
After applying this patch:
$ ip link add link mlx5_ib0 name mlx5_ib0.8001 type ipoib pkey 0x8001
$ ip -d link show dev mlx5_ib0.8001
38: mlx5_ib0.8001@mlx5_ib0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN mode DEFAULT group default qlen 256
link/infiniband 00:00:2e:4e:fe:80:00:00:00:00:00:00:24:8a:07:03:00:a3:19:64 brd 00:ff:ff:ff:ff:12:40:1b:80:01:00:00:00:00:00:00:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65520
ipoib pkey 0x8001 mode datagram umcast 0000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
$ ip -d link show dev mlx5_ib0.8002
30: mlx5_ib0.8002@mlx5_ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2044 qdisc mq state UP mode DEFAULT group default qlen 256
link/infiniband 00:00:1f:e6:fe:80:00:00:00:00:00:00:24:8a:07:03:00:a3:19:64 brd 00:ff:ff:ff:ff:12:40:1b:80:02:00:00:00:00:00:00:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65520
ipoib pkey 0x8002 mode datagram umcast 0000 addrgenmode none numtxqueues 256 numrxqueues 32 gso_max_size 65536 gso_max_segs 65535
$ ip -d link show dev mlx5_ib0
29: mlx5_ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2044 qdisc mq state UP mode DEFAULT group default qlen 256
link/infiniband 00:00:1f:e4:fe:80:00:00:00:00:00:00:24:8a:07:03:00:a3:19:64 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 65520
ipoib pkey 0xffff mode datagram umcast 0000 addrgenmode none numtxqueues 256 numrxqueues 32 gso_max_size 65536 gso_max_segs 65535
Also modifying the the ipoib attributes will work only for PKeys that is
created via netlink (for example, setting the mode).
Thanks,
Kamal
> Jason
next prev parent reply other threads:[~2020-09-29 6:04 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-28 20:26 [PATCH for-rc] RDMA/ipoib: Set rtnl_link_ops for ipoib interfaces Kamal Heib
2020-09-28 22:36 ` Jason Gunthorpe
2020-09-29 6:04 ` Kamal Heib [this message]
2020-09-30 0:22 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200929060438.GA73375@kheib-workstation \
--to=kamalheib1@gmail.com \
--cc=dledford@redhat.com \
--cc=jgg@ziepe.ca \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.