[PATCH v3 iproute2-next 0/4] Dynamic rdma link creation

[PATCH v3 iproute2-next 0/4] Dynamic rdma link creation

[Steve Wise]

This series adds rdmatool support for creating/deleting rdma links.
This will be used, mainly, by soft rdma drivers to allow adding/deleting
rdma links over netdev interfaces.  It provides the user side for
the following kernel changes merged in linux-5.1.

Changes since v2:

- move checks for required parameters in the parameter handlers
- move final 'link add' processing to link_add_netdev()
- added reviewed-by tags

Changes since v1:

- move error receive checking from rd_sendrecv_msg() to rd_recv_msg().
- Add rd->suppress_errors to allow control over whether errors when
  reading a response should be ignored.  Namely: resource queries can
  get errors like "none found" when querying for resources, and this
  error should not be displayed.  So on a rd object basis, error
  suppression can be controlled.
- Rebased on rdma/for-next UABI (no need to sync rdma_netlink.h now)
- use chains of struct rd_cmd and rd_exec_cmd vs open coding the parsing
  for the 'link add' command.
- minor nit resolution
- added .mailmap file.  If this is not desired for iproute2, then please
  drop the patch.

Changes since RFC:

- add rd_sendrecv_msg() and make use of it in dev_set as well
  as the new link commands.
- fixed problems with the man pages
- changed the command line to use "netdev" as the keyword
  for the network device, do avoid confused with the ib_device
  name.
- got rid of the "type" parameter for link delete.  Also pass
  down the device index instead of the name, using the common
  rd services for validating the device name and fetching the
  index.

Thanks,

Steve.

Steve Wise (4):
  Add .mailmap file
  rdma: add helper rd_sendrecv_msg()
  rdma: add 'link add/delete' commands
  rdma: man page update for link add/delete

 .mailmap             |  8 +++++
 man/man8/rdma-link.8 | 47 ++++++++++++++++++++++++++
 rdma/dev.c           |  2 +-
 rdma/link.c          | 78 ++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.h          |  4 +++
 rdma/res.h           |  1 +
 rdma/utils.c         | 18 ++++++++++
 7 files changed, 157 insertions(+), 1 deletion(-)
 create mode 100644 .mailmap

-- 
2.20.1

[PATCH v3 iproute2-next 1/4] Add .mailmap file

[Steve Wise]

.mailmap allows tracking multiple email addresses to the proper user name.

Signed-off-by: Steve Wise <larrystevenwise@gmail.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
 .mailmap | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 .mailmap

diff --git a/.mailmap b/.mailmap
new file mode 100644
index 00000000..c012d3d0
--- /dev/null
+++ b/.mailmap
@@ -0,0 +1,8 @@
+#
+# This list is used by git-shortlog to fix a few botched name translations
+# in the git archive, either because the author's full name was messed up
+# and/or not always written the same way, making contributions from the
+# same person appearing not to be so or badly displayed.
+#
+Steve Wise <larrystevenwise@gmail.com> <swise@opengridcomputing.com>
+Steve Wise <larrystevenwise@gmail.com> <swise@chelsio.com>
-- 
2.20.1

[PATCH v3 iproute2-next 2/4] rdma: add helper rd_sendrecv_msg()

[Steve Wise]

This function sends the constructed netlink message and then
receives the response.

Change rd_recv_msg() to display any error messages.

Change 'rdma dev set' to use rd_sendrecv_msg().

Signed-off-by: Steve Wise <larrystevenwise@gmail.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/dev.c   |  2 +-
 rdma/rdma.h  |  2 ++
 rdma/res.h   |  1 +
 rdma/utils.c | 18 ++++++++++++++++++
 4 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/rdma/dev.c b/rdma/dev.c
index 954e0015..33962520 100644
--- a/rdma/dev.c
+++ b/rdma/dev.c
@@ -268,7 +268,7 @@ static int dev_set_name(struct rd *rd)
 	mnl_attr_put_u32(rd->nlh, RDMA_NLDEV_ATTR_DEV_INDEX, rd->dev_idx);
 	mnl_attr_put_strz(rd->nlh, RDMA_NLDEV_ATTR_DEV_NAME, rd_argv(rd));
 
-	return rd_send_msg(rd);
+	return rd_sendrecv_msg(rd, seq);
 }
 
 static int dev_one_set(struct rd *rd)
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 1022e9a2..6c7f7d15 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -68,6 +68,7 @@ struct rd {
 	json_writer_t *jw;
 	bool json_output;
 	bool pretty_output;
+	bool suppress_errors;
 	struct list_head filter_list;
 };
 
@@ -119,6 +120,7 @@ bool rd_is_string_filtered_attr(struct rd *rd, const char *key, const char *val,
  */
 int rd_send_msg(struct rd *rd);
 int rd_recv_msg(struct rd *rd, mnl_cb_t callback, void *data, uint32_t seq);
+int rd_sendrecv_msg(struct rd *rd, unsigned int seq);
 void rd_prepare_msg(struct rd *rd, uint32_t cmd, uint32_t *seq, uint16_t flags);
 int rd_dev_init_cb(const struct nlmsghdr *nlh, void *data);
 int rd_attr_cb(const struct nlattr *attr, void *data);
diff --git a/rdma/res.h b/rdma/res.h
index b4a7e552..525171fc 100644
--- a/rdma/res.h
+++ b/rdma/res.h
@@ -31,6 +31,7 @@ int res_qp_idx_parse_cb(const struct nlmsghdr *nlh, void *data);
 		if (id) {                                                              \
 			ret = rd_doit_index(rd, &idx);                                 \
 			if (ret) {                                                     \
+				rd->suppress_errors = true;                            \
 				ret = _res_send_idx_msg(rd, command,                   \
 							name##_idx_parse_cb,           \
 							idx, id);                      \
diff --git a/rdma/utils.c b/rdma/utils.c
index 1f6bf330..11ed8a73 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -693,10 +693,28 @@ int rd_recv_msg(struct rd *rd, mnl_cb_t callback, void *data, unsigned int seq)
 		ret = mnl_cb_run(buf, ret, seq, portid, callback, data);
 	} while (ret > 0);
 
+	if (ret < 0 && !rd->suppress_errors)
+		perror("error");
+
 	mnl_socket_close(rd->nl);
 	return ret;
 }
 
+static int null_cb(const struct nlmsghdr *nlh, void *data)
+{
+	return MNL_CB_OK;
+}
+
+int rd_sendrecv_msg(struct rd *rd, unsigned int seq)
+{
+	int ret;
+
+	ret = rd_send_msg(rd);
+	if (!ret)
+		ret = rd_recv_msg(rd, null_cb, rd, seq);
+	return ret;
+}
+
 static struct dev_map *_dev_map_lookup(struct rd *rd, const char *dev_name)
 {
 	struct dev_map *dev_map;
-- 
2.20.1

[PATCH v3 iproute2-next 3/4] rdma: add 'link add/delete' commands

[Steve Wise]

Add new 'link' subcommand 'add' and 'delete' to allow binding a soft-rdma
device to a netdev interface.

EG:

rdma link add rxe_eth0 type rxe netdev eth0
rdma link delete rxe_eth0

Signed-off-by: Steve Wise <larrystevenwise@gmail.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/link.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.h |  2 ++
 2 files changed, 80 insertions(+)

diff --git a/rdma/link.c b/rdma/link.c
index 89e81b84..10b2e513 100644
--- a/rdma/link.c
+++ b/rdma/link.c
@@ -9,6 +9,9 @@
 static int link_help(struct rd *rd)
 {
 	pr_out("Usage: %s link show [DEV/PORT_INDEX]\n", rd->filename);
+	pr_out("Usage: %s link add NAME type TYPE netdev NETDEV\n",
+	       rd->filename);
+	pr_out("Usage: %s link delete NAME\n", rd->filename);
 	return 0;
 }
 
@@ -336,10 +339,85 @@ static int link_show(struct rd *rd)
 	return rd_exec_link(rd, link_one_show, true);
 }
 
+static int link_add_netdev(struct rd *rd)
+{
+	char *link_netdev;
+	uint32_t seq;
+
+	if (rd_no_arg(rd)) {
+		pr_err("Please provide a net device name.\n");
+		return -EINVAL;
+	}
+
+	link_netdev = rd_argv(rd);
+	rd_prepare_msg(rd, RDMA_NLDEV_CMD_NEWLINK, &seq,
+		       (NLM_F_REQUEST | NLM_F_ACK));
+	mnl_attr_put_strz(rd->nlh, RDMA_NLDEV_ATTR_DEV_NAME, rd->link_name);
+	mnl_attr_put_strz(rd->nlh, RDMA_NLDEV_ATTR_LINK_TYPE, rd->link_type);
+	mnl_attr_put_strz(rd->nlh, RDMA_NLDEV_ATTR_NDEV_NAME, link_netdev);
+	return rd_sendrecv_msg(rd, seq);
+}
+
+static int link_add_type(struct rd *rd)
+{
+	const struct rd_cmd cmds[] = {
+		{ NULL,		link_help},
+		{ "netdev",	link_add_netdev},
+		{ 0 }
+	};
+
+	if (rd_no_arg(rd)) {
+		pr_err("Please provide a link type name.\n");
+		return -EINVAL;
+	}
+	rd->link_type = rd_argv(rd);
+	rd_arg_inc(rd);
+	return rd_exec_cmd(rd, cmds, "parameter");
+}
+
+static int link_add(struct rd *rd)
+{
+	const struct rd_cmd cmds[] = {
+		{ NULL,		link_help},
+		{ "type",	link_add_type},
+		{ 0 }
+	};
+
+	if (rd_no_arg(rd)) {
+		pr_err("Please provide a link name to add.\n");
+		return -EINVAL;
+	}
+	rd->link_name = rd_argv(rd);
+	rd_arg_inc(rd);
+
+	return rd_exec_cmd(rd, cmds, "parameter");
+}
+
+static int _link_del(struct rd *rd)
+{
+	uint32_t seq;
+
+	if (!rd_no_arg(rd)) {
+		pr_err("Unknown parameter %s\n", rd_argv(rd));
+		return -EINVAL;
+	}
+	rd_prepare_msg(rd, RDMA_NLDEV_CMD_DELLINK, &seq,
+		       (NLM_F_REQUEST | NLM_F_ACK));
+	mnl_attr_put_u32(rd->nlh, RDMA_NLDEV_ATTR_DEV_INDEX, rd->dev_idx);
+	return rd_sendrecv_msg(rd, seq);
+}
+
+static int link_del(struct rd *rd)
+{
+	return rd_exec_require_dev(rd, _link_del);
+}
+
 int cmd_link(struct rd *rd)
 {
 	const struct rd_cmd cmds[] = {
 		{ NULL,		link_show },
+		{ "add",	link_add },
+		{ "delete",	link_del },
 		{ "show",	link_show },
 		{ "list",	link_show },
 		{ "help",	link_help },
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 6c7f7d15..9ed9e045 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -70,6 +70,8 @@ struct rd {
 	bool pretty_output;
 	bool suppress_errors;
 	struct list_head filter_list;
+	char *link_name;
+	char *link_type;
 };
 
 struct rd_cmd {
-- 
2.20.1

[PATCH v3 iproute2-next 4/4] rdma: man page update for link add/delete

[Steve Wise]

Update the 'rdma link' man page with 'link add/delete' info.

Signed-off-by: Steve Wise <larrystevenwise@gmail.com>
---
 man/man8/rdma-link.8 | 47 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/man/man8/rdma-link.8 b/man/man8/rdma-link.8
index bddf3474..b3b40de7 100644
--- a/man/man8/rdma-link.8
+++ b/man/man8/rdma-link.8
@@ -22,6 +22,18 @@ rdma-link \- rdma link configuration
 .B rdma link show
 .RI "[ " DEV/PORT_INDEX " ]"
 
+.ti -8
+.B rdma link add
+.BR NAME
+.BR type
+.BR TYPE
+.BR netdev
+.BR NETDEV
+
+.ti -8
+.B rdma link delete
+.RI NAME
+
 .ti -8
 .B rdma link help
 
@@ -33,6 +45,31 @@ rdma-link \- rdma link configuration
 - specifies the RDMA link to show.
 If this argument is omitted all links are listed.
 
+.SS rdma link add NAME type TYPE netdev NETDEV - add an rdma link for the specified type to the network device
+.sp
+.BR NAME
+- specifies the new name of the rdma link to add
+
+.BR TYPE
+- specifies which rdma type to use.  Link types:
+.sp
+.in +8
+.B rxe
+- Soft RoCE driver
+.sp
+.B siw
+- Soft iWARP driver
+.in -8
+
+.BR NETDEV
+- specifies the network device to which the link is bound
+
+.SS rdma link delete NAME - delete an rdma link
+.PP
+.BR NAME
+- specifies the name of the rdma link to delete
+.PP
+
 .SH "EXAMPLES"
 .PP
 rdma link show
@@ -45,6 +82,16 @@ rdma link show mlx5_2/1
 Shows the state of specified rdma link.
 .RE
 .PP
+rdma link add rxe_eth0 type rxe netdev eth0
+.RS 4
+Adds a RXE link named rxe_eth0 to network device eth0
+.RE
+.PP
+rdma link del rxe_eth0
+.RS 4
+Removes RXE link rxe_eth0
+.RE
+.PP
 
 .SH SEE ALSO
 .BR rdma (8),
-- 
2.20.1

Re: [PATCH v3 iproute2-next 4/4] rdma: man page update for link add/delete

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 359 bytes --]

On Wed, Apr 03, 2019 at 12:10:32PM -0500, Steve Wise wrote:
> Update the 'rdma link' man page with 'link add/delete' info.
>
> Signed-off-by: Steve Wise <larrystevenwise@gmail.com>
> ---
>  man/man8/rdma-link.8 | 47 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>

Thanks,
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH v3 iproute2-next 0/4] Dynamic rdma link creation

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

On Wed, Apr 03, 2019 at 12:10:28PM -0500, Steve Wise wrote:
> This series adds rdmatool support for creating/deleting rdma links.
> This will be used, mainly, by soft rdma drivers to allow adding/deleting
> rdma links over netdev interfaces.  It provides the user side for
> the following kernel changes merged in linux-5.1.
>

Thanks a lot Steve,
Looks very good.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH v3 iproute2-next 0/4] Dynamic rdma link creation

[David Ahern]

On 4/3/19 11:10 AM, Steve Wise wrote:
> This series adds rdmatool support for creating/deleting rdma links.
> This will be used, mainly, by soft rdma drivers to allow adding/deleting
> rdma links over netdev interfaces.  It provides the user side for
> the following kernel changes merged in linux-5.1.
> 

applied to iproute2-next. Thanks