[PATCH iproute2 0/5] RDMAtool

[PATCH iproute2 0/5] RDMAtool

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Hi,

This series is implementing the RDAMtool -  the tool to configure RDMA devices.
The initial proposal was sent as RFC [1] and was based on sysfs entries as POC.

The current series was rewritten completely to work with RDMA netlinks as
a source of user<->kernel communications. In order to achieve that, the
RDMA netlinks were extensively refactored and modernized [2, 3 and 4].

The changes from RFC:
1. Removed everything that is not implemented yet.
2. Abandoned sysfs interfaces in favor of netlink.

Available in the "topic/rdmatool-netlink" topic branch of this git repo:
git://git.kernel.org/pub/scm/linux/kernel/git/leon/iproute2.git

Or for browsing:
https://git.kernel.org/cgit/linux/kernel/git/leon/iproute2.git/log/?h=topic/rdmatool-netlink

Thanks

[1] https://www.spinics.net/lists/linux-rdma/msg49575.html
[2] https://patchwork.kernel.org/patch/9752865/
[3] https://www.spinics.net/lists/linux-rdma/msg50827.html
[4] https://www.spinics.net/lists/linux-rdma/msg51210.html

Cc: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Ariel Almog <ariela-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Cc: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Cc: Linux RDMA <linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Cc: Linux Netdev <netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>

Leon Romanovsky (5):
  rdma: Add basic infrastructure for RDMA tool
  rdma: Add dev object
  rdma: Add device capability parsing
  rdma: Add link option and parsing
  rdma: Add initial manual for the tool

 Makefile          |   2 +-
 man/man8/Makefile |   3 +-
 man/man8/rdma.8   |  82 +++++++++++++++
 rdma/.gitignore   |   1 +
 rdma/Makefile     |  22 ++++
 rdma/dev.c        | 140 +++++++++++++++++++++++++
 rdma/link.c       | 201 +++++++++++++++++++++++++++++++++++
 rdma/rdma.c       | 112 ++++++++++++++++++++
 rdma/rdma.h       |  89 ++++++++++++++++
 rdma/utils.c      | 307 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 10 files changed, 957 insertions(+), 2 deletions(-)
 create mode 100644 man/man8/rdma.8
 create mode 100644 rdma/.gitignore
 create mode 100644 rdma/Makefile
 create mode 100644 rdma/dev.c
 create mode 100644 rdma/link.c
 create mode 100644 rdma/rdma.c
 create mode 100644 rdma/rdma.h
 create mode 100644 rdma/utils.c

--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH iproute2 1/5] rdma: Add basic infrastructure for RDMA tool

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

RDMA devices are cross-functional devices from one side,
but very tailored for the specific markets from another.

Such diversity caused to spread of RDMA related configuration
across various tools, e.g. devlink, ip, ethtool, ib specific and
vendor specific solutions.

This patch adds ability to fill device and port information
by reading RDMA netlink.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 Makefile          |   2 +-
 man/man8/Makefile |   3 +-
 rdma/.gitignore   |   1 +
 rdma/Makefile     |  22 +++++
 rdma/rdma.c       | 110 ++++++++++++++++++++++
 rdma/rdma.h       |  76 +++++++++++++++
 rdma/utils.c      | 270 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 482 insertions(+), 2 deletions(-)
 create mode 100644 rdma/.gitignore
 create mode 100644 rdma/Makefile
 create mode 100644 rdma/rdma.c
 create mode 100644 rdma/rdma.h
 create mode 100644 rdma/utils.c

diff --git a/Makefile b/Makefile
index 18de7dcb..c255063b 100644
--- a/Makefile
+++ b/Makefile
@@ -52,7 +52,7 @@ WFLAGS += -Wmissing-declarations -Wold-style-definition -Wformat=2
 CFLAGS := $(WFLAGS) $(CCOPTS) -I../include $(DEFINES) $(CFLAGS)
 YACCFLAGS = -d -t -v

-SUBDIRS=lib ip tc bridge misc netem genl tipc devlink man
+SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma man

 LIBNETLINK=../lib/libnetlink.a ../lib/libutil.a
 LDLIBS += $(LIBNETLINK)
diff --git a/man/man8/Makefile b/man/man8/Makefile
index f3318644..81979a07 100644
--- a/man/man8/Makefile
+++ b/man/man8/Makefile
@@ -19,7 +19,8 @@ MAN8PAGES = $(TARGETS) ip.8 arpd.8 lnstat.8 routel.8 rtacct.8 rtmon.8 rtpr.8 ss.
 	tc-simple.8 tc-skbedit.8 tc-vlan.8 tc-xt.8 tc-skbmod.8 tc-ife.8 \
 	tc-tunnel_key.8 tc-sample.8 \
 	devlink.8 devlink-dev.8 devlink-monitor.8 devlink-port.8 devlink-sb.8 \
-	ifstat.8
+	ifstat.8 \
+	rdma.8

 all: $(TARGETS)

diff --git a/rdma/.gitignore b/rdma/.gitignore
new file mode 100644
index 00000000..51fb172b
--- /dev/null
+++ b/rdma/.gitignore
@@ -0,0 +1 @@
+rdma
diff --git a/rdma/Makefile b/rdma/Makefile
new file mode 100644
index 00000000..64da2142
--- /dev/null
+++ b/rdma/Makefile
@@ -0,0 +1,22 @@
+include ../Config
+
+ifeq ($(HAVE_MNL),y)
+
+RDMA_OBJ = rdma.o utils.o
+
+TARGETS=rdma
+CFLAGS += $(shell $(PKG_CONFIG) libmnl --cflags)
+LDLIBS += $(shell $(PKG_CONFIG) libmnl --libs)
+
+endif
+
+all:	$(TARGETS) $(LIBS)
+
+rdma:	$(RDMA_OBJ) $(LIBS)
+	$(QUIET_LINK)$(CC) $^ $(LDFLAGS) $(LDLIBS) -o $@
+
+install: all
+	install -m 0755 $(TARGETS) $(DESTDIR)$(SBINDIR)
+
+clean:
+	rm -f $(RDMA_OBJ) $(TARGETS)
diff --git a/rdma/rdma.c b/rdma/rdma.c
new file mode 100644
index 00000000..9c754da3
--- /dev/null
+++ b/rdma/rdma.c
@@ -0,0 +1,110 @@
+/*
+ * rdma.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+
+#include <limits.h>
+#include <rdma/rdma_netlink.h>
+
+#include "rdma.h"
+#include "SNAPSHOT.h"
+
+static void help(char *name)
+{
+	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
+	       "where  OBJECT := { help }\n"
+	       "       OPTIONS := { -V[ersion] }\n", name);
+}
+
+static int cmd_help(struct rdma *rd)
+{
+	help(rd->filename);
+	return 0;
+}
+
+static int rd_cmd(struct rdma *rd)
+{
+	const struct rdma_cmd cmds[] = {
+		{ NULL,		cmd_help },
+		{ "help",	cmd_help },
+		{ 0 }
+	};
+
+	return rdma_exec_cmd(rd, cmds, "object");
+}
+
+static int rd_init(struct rdma *rd, int argc, char **argv, char *filename)
+{
+	uint32_t seq;
+	int ret;
+
+	rd->filename = filename;
+	rd->argc = argc;
+	rd->argv = argv;
+	INIT_LIST_HEAD(&rd->dev_map_list);
+	rd->buff = malloc(MNL_SOCKET_BUFFER_SIZE);
+	if (!rd->buff)
+		return -ENOMEM;
+
+	rdma_prepare_msg(rd, RDMA_NLDEV_CMD_GET, &seq, (NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP));
+	if ((ret = rdma_send_msg(rd)))
+		return ret;
+
+	return rdma_recv_msg(rd, rd_dev_init_cb, rd, seq);
+}
+
+static void rd_free(struct rdma *rd)
+{
+	free(rd->buff);
+	rdma_free_devmap(rd);
+}
+int main(int argc, char **argv)
+{
+	char *filename;
+	static const struct option long_options[] = {
+		{ "version",		no_argument,		NULL, 'V' },
+		{ "help",		no_argument,		NULL, 'h' },
+		{ NULL, 0, NULL, 0 }
+	};
+	struct rdma rd;
+	int opt;
+	int err;
+
+	filename = basename(argv[0]);
+
+	while ((opt = getopt_long(argc, argv, "Vh",
+				  long_options, NULL)) >= 0) {
+
+		switch (opt) {
+		case 'V':
+			printf("%s utility, iproute2-ss%s\n", filename, SNAPSHOT);
+			return EXIT_SUCCESS;
+		case 'h':
+			help(filename);
+			return EXIT_SUCCESS;
+		default:
+			pr_err("Unknown option.\n");
+			help(filename);
+			return EXIT_FAILURE;
+		}
+	}
+
+	argc -= optind;
+	argv += optind;
+
+	err = rd_init(&rd, argc, argv, filename);
+	if (err)
+		goto out;
+
+	err = rd_cmd(&rd);
+out:
+	/* Always cleanup */
+	rd_free(&rd);
+	return (err) ? EXIT_FAILURE:EXIT_SUCCESS;
+}
diff --git a/rdma/rdma.h b/rdma/rdma.h
new file mode 100644
index 00000000..9841aebf
--- /dev/null
+++ b/rdma/rdma.h
@@ -0,0 +1,76 @@
+/*
+ * rdma.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+#ifndef _RDMA_TOOL_H_
+#define _RDMA_TOOL_H_
+
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <getopt.h>
+#include <libmnl/libmnl.h>
+
+#include <rdma/rdma_netlink.h>
+#include "list.h"
+
+#define pr_err(args...) fprintf(stderr, ##args)
+#define pr_out(args...) fprintf(stdout, ##args)
+
+struct port_map {
+	struct list_head list;
+	uint32_t idx;
+};
+
+struct dev_map {
+	struct list_head list;
+	char *dev_name;
+	uint32_t num_ports;
+	struct list_head port_map_list;
+	uint32_t idx;
+};
+
+struct rdma {
+	int argc;
+	char **argv;
+	char *filename;
+	struct list_head dev_map_list;
+	struct mnl_socket *nl;
+	struct nlmsghdr *nlh;
+	char *buff;
+};
+
+struct rdma_cmd {
+	const char *cmd;
+	int (*func)(struct rdma *rd);
+};
+
+/*
+ * Parser interface
+ */
+bool rd_no_arg(struct rdma *rd);
+bool rd_argv_match(struct rdma *rd, const char *pattern);
+void rd_arg_inc(struct rdma *rd);
+
+int rdma_exec_cmd(struct rdma *rd, const struct rdma_cmd *c, const char *str);
+
+/*
+ * Device manipulation
+ */
+void rdma_free_devmap(struct rdma *rd);
+
+/*
+ * Netlink
+ */
+int rdma_send_msg(struct rdma *rd);
+int rdma_recv_msg(struct rdma *rd, mnl_cb_t callback, void *data, uint32_t seq);
+int rdma_prepare_msg(struct rdma *rd, uint32_t cmd, uint32_t *seq, uint16_t flags);
+int rd_dev_init_cb(const struct nlmsghdr *nlh, void *data);
+int rd_attr_cb(const struct nlattr *attr, void *data);
+#endif /* _RDMA_TOOL_H_ */
diff --git a/rdma/utils.c b/rdma/utils.c
new file mode 100644
index 00000000..96278a4c
--- /dev/null
+++ b/rdma/utils.c
@@ -0,0 +1,270 @@
+/*
+ * utils.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+
+#include <sys/types.h>
+#include <dirent.h>
+#include <time.h>
+
+#include "rdma.h"
+#include <rdma/rdma_netlink.h>
+
+/*
+ * This macro will be moved to generic layer,
+ * after code will be accepted.
+ * it is placed here to avoid rebases with upstream code.
+ */
+#define MAX(a, b)	((a) > (b) ? (a) : (b))
+
+static int rd_argc(struct rdma *rd)
+{
+	return rd->argc;
+}
+
+static char *rd_argv(struct rdma *rd)
+{
+	if (!rd_argc(rd))
+		return NULL;
+	return *rd->argv;
+}
+
+static int strcmpx(const char *str1, const char *str2)
+{
+	if (strlen(str1) > strlen(str2))
+			return -1;
+	return strncmp(str1, str2, strlen(str1));
+}
+
+bool rd_argv_match(struct rdma *rd, const char *pattern)
+{
+	if (!rd_argc(rd))
+		return false;
+	return strcmpx(rd_argv(rd), pattern) == 0;
+}
+
+void rd_arg_inc(struct rdma *rd)
+{
+	if (!rd_argc(rd))
+		return;
+	rd->argc--;
+	rd->argv++;
+}
+
+bool rd_no_arg(struct rdma *rd)
+{
+	return rd_argc(rd) == 0;
+}
+
+static struct dev_map *dev_map_alloc(const char *dev_name)
+{
+	struct dev_map *dev_map;
+
+	dev_map = calloc(1, sizeof(*dev_map));
+	if (!dev_map)
+		return NULL;
+	dev_map->dev_name = strdup(dev_name);
+	INIT_LIST_HEAD(&dev_map->port_map_list);
+
+	return dev_map;
+}
+
+static void port_map_free(struct port_map *port_map)
+{
+	if(!port_map)
+		return;
+	free(port_map);
+}
+
+static void dev_map_free(struct dev_map *dev_map)
+{
+	struct port_map *port_map, *tmp;
+
+	if(!dev_map)
+		return;
+
+	list_for_each_entry_safe(port_map, tmp,
+				 &dev_map->port_map_list, list) {
+		list_del(&port_map->list);
+		port_map_free(port_map);
+	}
+
+	free(dev_map->dev_name);
+	free(dev_map);
+}
+
+static void dev_map_cleanup(struct rdma *rd)
+{
+	struct dev_map *dev_map, *tmp;
+
+	list_for_each_entry_safe(dev_map, tmp,
+				 &rd->dev_map_list, list) {
+		list_del(&dev_map->list);
+		dev_map_free(dev_map);
+	}
+}
+
+static int port_map_alloc(struct dev_map *dev_map, uint32_t num_ports)
+{
+	struct port_map *port_map;
+	int idx = 1;
+
+	while (idx <= num_ports) {
+		port_map = calloc(1, sizeof(*port_map));
+		if (!port_map)
+			return -ENOMEM;
+
+		list_add_tail(&port_map->list, &dev_map->port_map_list);
+		port_map->idx = idx++;
+	}
+
+	return 0;
+}
+
+static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
+	[RDMA_NLDEV_ATTR_DEV_NAME] = MNL_TYPE_NUL_STRING,
+	[RDMA_NLDEV_ATTR_PORT_INDEX] = MNL_TYPE_U32,
+};
+
+int rd_attr_cb(const struct nlattr *attr, void *data)
+{
+	const struct nlattr **tb = data;
+	int type;
+
+	if (mnl_attr_type_valid(attr, RDMA_NLDEV_ATTR_MAX) < 0)
+		return MNL_CB_ERROR;
+
+	type = mnl_attr_get_type(attr);
+
+	if (mnl_attr_validate(attr, nldev_policy[type]) < 0)
+		return MNL_CB_ERROR;
+
+	tb[type] = attr;
+	return MNL_CB_OK;
+}
+
+int rd_dev_init_cb(const struct nlmsghdr *nlh, void *data)
+{
+	struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+	struct dev_map *dev_map;
+	struct rdma *rd = data;
+	const char *dev_name;
+	uint32_t num_ports;
+	static int i = 1;
+
+	mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+	if (!tb[RDMA_NLDEV_ATTR_DEV_NAME])
+		return MNL_CB_ERROR;
+	if (!tb[RDMA_NLDEV_ATTR_PORT_INDEX]) {
+		pr_err("This tool doesn't support switches yet\n");
+		return MNL_CB_ERROR;
+	}
+
+	dev_name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+	num_ports = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_PORT_INDEX]);
+
+	dev_map = dev_map_alloc(dev_name);
+	if (!dev_map)
+		/* The main function will cleanup the allocations */
+		return MNL_CB_ERROR;
+	list_add_tail(&dev_map->list, &rd->dev_map_list);
+	dev_map->idx = i++;
+
+	if (port_map_alloc(dev_map, num_ports))
+		return MNL_CB_ERROR;
+	dev_map->num_ports = num_ports;
+
+	return MNL_CB_OK;
+}
+
+void rdma_free_devmap(struct rdma *rd)
+{
+	if(!rd)
+		return;
+	dev_map_cleanup(rd);
+	return;
+}
+
+int rdma_exec_cmd(struct rdma *rd, const struct rdma_cmd *cmds, const char *str)
+{
+	const struct rdma_cmd *c;
+
+	/* First argument in objs table is default variant */
+	if (rd_no_arg(rd))
+		return cmds->func(rd);
+
+	for (c = cmds + 1; c->cmd; ++c) {
+		if (rd_argv_match(rd, c->cmd)) {
+			/* Move to next argument */
+			rd_arg_inc(rd);
+			return c->func(rd);
+		}
+	}
+
+	pr_err("Unknown %s '%s'.\n", str, rd_argv(rd));
+	return 0;
+}
+
+int rdma_prepare_msg(struct rdma *rd, uint32_t cmd, uint32_t *seq, uint16_t flags)
+{
+	*seq = time(NULL);
+
+	rd->nlh = mnl_nlmsg_put_header(rd->buff);
+	rd->nlh->nlmsg_type = RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, cmd);
+	rd->nlh->nlmsg_seq = *seq;
+	rd->nlh->nlmsg_flags = flags;
+
+	return 0;
+}
+
+int rdma_send_msg(struct rdma *rd)
+{
+	int ret;
+
+	rd->nl = mnl_socket_open(NETLINK_RDMA);
+	if (!rd->nl) {
+		pr_err("Failed to open NETLINK_RDMA socket\n");
+		return -ENODEV;
+	}
+
+	ret = mnl_socket_bind(rd->nl, 0, MNL_SOCKET_AUTOPID);
+	if (ret < 0) {
+		pr_err("Failed to bind socket with err %d\n", ret);
+		goto err;
+	}
+
+	ret = mnl_socket_sendto(rd->nl, rd->nlh, rd->nlh->nlmsg_len);
+	if (ret < 0) {
+		pr_err("Failed to send to socket with err %d\n", ret);
+		goto err;
+	}
+	return 0;
+
+err:	mnl_socket_close(rd->nl);
+	return ret;
+}
+
+int rdma_recv_msg(struct rdma *rd, mnl_cb_t callback, void *data, unsigned int seq)
+{
+	int ret;
+	unsigned int portid;
+	char buf[MNL_SOCKET_BUFFER_SIZE];
+
+	portid = mnl_socket_get_portid(rd->nl);
+	do {
+		ret = mnl_socket_recvfrom(rd->nl, buf, sizeof(buf));
+		if (ret <= 0)
+			break;
+
+		ret = mnl_cb_run(buf, ret, seq, portid, callback, data);
+	} while (ret > 0);
+
+	mnl_socket_close(rd->nl);
+	return ret;
+}
--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add parsing interface for the device capability flags

$ rdma dev show
1: mlx5_0: caps 0x1257e1c26
2: mlx5_1: caps 0x1257e1c26
3: mlx5_2: caps 0x1257e1c26
4: mlx5_3: caps 0x1257e1c26
5: mlx5_4: caps 0x1257e1c26

$ rdma dev show mlx5_4
5: mlx5_4: caps 0x1257e1c26

$ rdma dev show mlx5_4 caps
5: mlx5_4: caps 0x1257e1c26
Bit	Description
------------------------------------------
 01	DEVICE_BAD_PKEY_CNTR
 02	DEVICE_BAD_QKEY_CNTR
 05	DEVICE_CHANGE_PHY_PORT
 10	DEVICE_PORT_ACTIVE_EVENT
 11	DEVICE_SYS_IMAGE_GUID
 12	DEVICE_RC_RNR_NAK_GEN
 17	DEVICE_MEM_WINDOW
 18	DEVICE_UD_IP_CSUM
 19	DEVICE_UD_TSO
 20	DEVICE_XRC
 21	DEVICE_MEM_MGT_EXTENSIONS
 22	DEVICE_BLOCK_MULTICAST_LOOPBACK
 24	DEVICE_MEM_WINDOW_TYPE_2B
 26	DEVICE_RAW_IP_CSUM
 29	DEVICE_SIGNATURE_HANDOVER
 32	DEVICE_VIRTUAL_FUNCTION

$ rdma dev show mlx5_4 cap_flags
Unknown parameter 'caps_flags'.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 rdma/dev.c   | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 rdma/rdma.h  |  3 ++
 rdma/utils.c |  3 +-
 3 files changed, 93 insertions(+), 9 deletions(-)

diff --git a/rdma/dev.c b/rdma/dev.c
index 5a3ee126..31c235e8 100644
--- a/rdma/dev.c
+++ b/rdma/dev.c
@@ -14,6 +14,7 @@
 static int dev_help(struct rdma *rd)
 {
 	pr_out("Usage: %s dev show [DEV]\n", rd->filename);
+	pr_out("       %s dev show DEV caps\n", rd->filename);

 	/*
 	 * Example of set command:
@@ -22,28 +23,107 @@ static int dev_help(struct rdma *rd)
 	return 0;
 }

-static void dev_one_show(const struct dev_map *dev_map)
+static const char *dev_caps[64] = {
+	"DEVICE_RESIZE_MAX_WR",
+	"DEVICE_BAD_PKEY_CNTR",
+	"DEVICE_BAD_QKEY_CNTR",
+	"DEVICE_RAW_MULTI",
+	"DEVICE_AUTO_PATH_MIG",
+	"DEVICE_CHANGE_PHY_PORT",
+	"DEVICE_UD_AV_PORT_ENFORCE",
+	"DEVICE_CURR_QP_STATE_MOD",
+	"DEVICE_SHUTDOWN_PORT",
+	"DEVICE_INIT_TYPE",
+	"DEVICE_PORT_ACTIVE_EVENT",
+	"DEVICE_SYS_IMAGE_GUID",
+	"DEVICE_RC_RNR_NAK_GEN",
+	"DEVICE_SRQ_RESIZE",
+	"DEVICE_N_NOTIFY_CQ",
+	"DEVICE_LOCAL_DMA_LKEY",
+	"DEVICE_RESERVED",
+	"DEVICE_MEM_WINDOW",
+	"DEVICE_UD_IP_CSUM",
+	"DEVICE_UD_TSO",
+	"DEVICE_XRC",
+	"DEVICE_MEM_MGT_EXTENSIONS",
+	"DEVICE_BLOCK_MULTICAST_LOOPBACK",
+	"DEVICE_MEM_WINDOW_TYPE_2A",
+	"DEVICE_MEM_WINDOW_TYPE_2B",
+	"DEVICE_RC_IP_CSUM",
+	"DEVICE_RAW_IP_CSUM",
+	"DEVICE_CROSS_CHANNEL",
+	"DEVICE_MANAGED_FLOW_STEERING",
+	"DEVICE_SIGNATURE_HANDOVER",
+	"DEVICE_ON_DEMAND_PAGING",
+	"DEVICE_SG_GAPS_REG",
+	"DEVICE_VIRTUAL_FUNCTION",
+	"DEVICE_RAW_SCATTER_FCS",
+	"DEVICE_RDMA_NETDEV_OPA_VNIC",
+};
+
+static int dev_print_caps(struct rdma *rd)
+{
+	struct dev_map *dev_map = rd->dev_map_curr;
+	uint64_t caps = dev_map->caps;
+	uint32_t idx;
+
+	pr_out("%u: %s: ", dev_map->idx, dev_map->dev_name);
+	pr_out("caps 0x%" PRIx64 "\n", dev_map->caps);
+	pr_out("Bit\tDescription\n");
+	pr_out("------------------------------------------\n");
+	for (idx = 0; idx < 64; idx++) {
+		if (caps & 0x1)
+			pr_out(" %02u\t%s\n", idx, dev_caps[idx]?dev_caps[idx]:"UNKNONW");
+		caps >>= 0x1;
+	}
+	return 0;
+}
+
+static int dev_no_args(struct rdma *rd)
+{
+	struct dev_map *dev_map = rd->dev_map_curr;
+
+	pr_out("%u: %s: ", dev_map->idx, dev_map->dev_name);
+	pr_out("caps 0x%" PRIx64 "\n", dev_map->caps);
+	return 0;
+}
+
+static int dev_one_show(struct rdma *rd)
 {
-	pr_out("%u: %s:\n", dev_map->idx, dev_map->dev_name);
+	const struct rdma_cmd cmds[] = {
+		{ NULL,		dev_no_args},
+		{ "caps",	dev_print_caps},
+		{ 0 }
+	};
+
+	return rdma_exec_cmd(rd, cmds, "parameter");
+
 }

 static int dev_show(struct rdma *rd)
 {
 	struct dev_map *dev_map;
+	int ret = 0;

 	if (rd_no_arg(rd)) {
-		list_for_each_entry(dev_map, &rd->dev_map_list, list)
-			dev_one_show(dev_map);
+		list_for_each_entry(dev_map, &rd->dev_map_list, list) {
+			rd->dev_map_curr = dev_map;
+			ret = dev_one_show(rd);
+			if (ret)
+				return ret;
+		}
+
 	}
 	else {
-		dev_map = dev_map_lookup(rd, false);
-		if (!dev_map) {
+		rd->dev_map_curr = dev_map_lookup(rd, false);
+		if (!rd->dev_map_curr) {
 			pr_err("Wrong device name\n");
 			return -ENOENT;
 		}
-		dev_one_show(dev_map);
+		rd_arg_inc(rd);
+		ret = dev_one_show(rd);
 	}
-	return 0;
+	return ret;
 }

 int cmd_dev(struct rdma *rd)
diff --git a/rdma/rdma.h b/rdma/rdma.h
index f5e104ec..8cca0f28 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -34,6 +34,7 @@ struct dev_map {
 	uint32_t num_ports;
 	struct list_head port_map_list;
 	uint32_t idx;
+	uint64_t caps;
 };

 struct rdma {
@@ -41,6 +42,7 @@ struct rdma {
 	char **argv;
 	char *filename;
 	struct list_head dev_map_list;
+	struct dev_map *dev_map_curr;
 	struct mnl_socket *nl;
 	struct nlmsghdr *nlh;
 	char *buff;
@@ -57,6 +59,7 @@ struct rdma_cmd {
 bool rd_no_arg(struct rdma *rd);
 bool rd_argv_match(struct rdma *rd, const char *pattern);
 void rd_arg_inc(struct rdma *rd);
+char *rd_argv(struct rdma *rd);

 /*
  * Commands interface
diff --git a/rdma/utils.c b/rdma/utils.c
index e5c3dd6c..e7f257e3 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -28,7 +28,7 @@ static int rd_argc(struct rdma *rd)
 	return rd->argc;
 }

-static char *rd_argv(struct rdma *rd)
+char *rd_argv(struct rdma *rd)
 {
 	if (!rd_argc(rd))
 		return NULL;
@@ -128,6 +128,7 @@ static int port_map_alloc(struct dev_map *dev_map, uint32_t num_ports)
 }

 static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
+	[RDMA_NLDEV_ATTR_DEV_INDEX] = MNL_TYPE_U32,
 	[RDMA_NLDEV_ATTR_DEV_NAME] = MNL_TYPE_NUL_STRING,
 	[RDMA_NLDEV_ATTR_PORT_INDEX] = MNL_TYPE_U32,
 };
--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Jason Gunthorpe]

On Mon, Jun 26, 2017 at 09:21:26PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> Add parsing interface for the device capability flags
> 
> $ rdma dev show
> 1: mlx5_0: caps 0x1257e1c26

This seems very un ip-like. I wouldn't show an undecoded hex value
like that, it isn't really useful.

> $ rdma dev show mlx5_4 caps
> 5: mlx5_4: caps 0x1257e1c26
> Bit	Description
>  01	DEVICE_BAD_PKEY_CNTR
>  02	DEVICE_BAD_QKEY_CNTR

This table also seems un ip-like, the usual format is a list of words,
I think.

> $ rdma dev show mlx5_4 cap_flags
> Unknown parameter 'caps_flags'.

?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]

On Mon, Jun 26, 2017 at 12:29:24PM -0600, Jason Gunthorpe wrote:
> On Mon, Jun 26, 2017 at 09:21:26PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >
> > Add parsing interface for the device capability flags
> >
> > $ rdma dev show
> > 1: mlx5_0: caps 0x1257e1c26
>
> This seems very un ip-like. I wouldn't show an undecoded hex value
> like that, it isn't really useful.

It is first supported field, after new fields will be added, we will
have very similar to ip interface.

1: mlx5_0: caps 0x1257e1c2 key_1 val_1 key_2 val_2 ....

The value are presented as is can be usable as an input for different scripts.

>
> > $ rdma dev show mlx5_4 caps
> > 5: mlx5_4: caps 0x1257e1c26
> > Bit	Description
> >  01	DEVICE_BAD_PKEY_CNTR
> >  02	DEVICE_BAD_QKEY_CNTR
>
> This table also seems un ip-like, the usual format is a list of words,
> I think.

It is true for key<->value data, but it is less obvious for bit parsing.
Internally, I tried to present them as list and it was ugly like hell
without any chance (without extra parsing) to actual see if specific
capability is present or no.

The question is: is such presentation usable and does it make sense?

>
> > $ rdma dev show mlx5_4 cap_flags
> > Unknown parameter 'caps_flags'.
>
> ?

An example of "wrong" parameter. There is no such caps_flags.

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Jason Gunthorpe]

On Mon, Jun 26, 2017 at 10:21:03PM +0300, Leon Romanovsky wrote:
> On Mon, Jun 26, 2017 at 12:29:24PM -0600, Jason Gunthorpe wrote:
> > On Mon, Jun 26, 2017 at 09:21:26PM +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > Add parsing interface for the device capability flags
> > >
> > > $ rdma dev show
> > > 1: mlx5_0: caps 0x1257e1c26
> >
> > This seems very un ip-like. I wouldn't show an undecoded hex value
> > like that, it isn't really useful.
> 
> It is first supported field, after new fields will be added, we will
> have very similar to ip interface.
> 
> 1: mlx5_0: caps 0x1257e1c2 key_1 val_1 key_2 val_2 ....
> 
> The value are presented as is can be usable as an input for different scripts.

I still wouldn't show an undecoded hex value.. It isn't useful.

> > > $ rdma dev show mlx5_4 caps
> > > 5: mlx5_4: caps 0x1257e1c26
> > > Bit	Description
> > >  01	DEVICE_BAD_PKEY_CNTR
> > >  02	DEVICE_BAD_QKEY_CNTR
> >
> > This table also seems un ip-like, the usual format is a list of words,
> > I think.
> 
> It is true for key<->value data, but it is less obvious for bit
> parsing.

Several of the word decodes are from bit fields..

> Internally, I tried to present them as list and it was ugly like hell
> without any chance (without extra parsing) to actual see if specific
> capability is present or no.

lspci seems to have no problem being readable while doing this..

Jason

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1909 bytes --]

On Mon, Jun 26, 2017 at 02:36:10PM -0600, Jason Gunthorpe wrote:
> On Mon, Jun 26, 2017 at 10:21:03PM +0300, Leon Romanovsky wrote:
> > On Mon, Jun 26, 2017 at 12:29:24PM -0600, Jason Gunthorpe wrote:
> > > On Mon, Jun 26, 2017 at 09:21:26PM +0300, Leon Romanovsky wrote:
> > > > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > > >
> > > > Add parsing interface for the device capability flags
> > > >
> > > > $ rdma dev show
> > > > 1: mlx5_0: caps 0x1257e1c26
> > >
> > > This seems very un ip-like. I wouldn't show an undecoded hex value
> > > like that, it isn't really useful.
> >
> > It is first supported field, after new fields will be added, we will
> > have very similar to ip interface.
> >
> > 1: mlx5_0: caps 0x1257e1c2 key_1 val_1 key_2 val_2 ....
> >
> > The values are presented as is can be usable as an input for different scripts.
>
> I still wouldn't show an undecoded hex value.. It isn't useful.
>
> > > > $ rdma dev show mlx5_4 caps
> > > > 5: mlx5_4: caps 0x1257e1c26
> > > > Bit	Description
> > > >  01	DEVICE_BAD_PKEY_CNTR
> > > >  02	DEVICE_BAD_QKEY_CNTR
> > >
> > > This table also seems un ip-like, the usual format is a list of words,
> > > I think.
> >
> > It is true for key<->value data, but it is less obvious for bit
> > parsing.
>
> Several of the word decodes are from bit fields..
>
> > Internally, I tried to present them as list and it was ugly like hell
> > without any chance (without extra parsing) to actual see if specific
> > capability is present or no.
>
> lspci seems to have no problem being readable while doing this..

No problem,
If i understand you correctly, you are suggesting to drop parsing of
"caps" as a separate command and embed it into general show <devname>.

Can you help me and give an example of how would you present those caps?
What will be the output of such command?
 $ rdma dev show mlx5_4

 Thanks

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2474 bytes --]

On Tue, Jun 27, 2017 at 07:06:04AM +0300, Leon Romanovsky wrote:
> On Mon, Jun 26, 2017 at 02:36:10PM -0600, Jason Gunthorpe wrote:
> > On Mon, Jun 26, 2017 at 10:21:03PM +0300, Leon Romanovsky wrote:
> > > On Mon, Jun 26, 2017 at 12:29:24PM -0600, Jason Gunthorpe wrote:
> > > > On Mon, Jun 26, 2017 at 09:21:26PM +0300, Leon Romanovsky wrote:
> > > > > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > > > >
> > > > > Add parsing interface for the device capability flags
> > > > >
> > > > > $ rdma dev show
> > > > > 1: mlx5_0: caps 0x1257e1c26
> > > >
> > > > This seems very un ip-like. I wouldn't show an undecoded hex value
> > > > like that, it isn't really useful.
> > >
> > > It is first supported field, after new fields will be added, we will
> > > have very similar to ip interface.
> > >
> > > 1: mlx5_0: caps 0x1257e1c2 key_1 val_1 key_2 val_2 ....
> > >
> > > The values are presented as is can be usable as an input for different scripts.
> >
> > I still wouldn't show an undecoded hex value.. It isn't useful.
> >
> > > > > $ rdma dev show mlx5_4 caps
> > > > > 5: mlx5_4: caps 0x1257e1c26
> > > > > Bit	Description
> > > > >  01	DEVICE_BAD_PKEY_CNTR
> > > > >  02	DEVICE_BAD_QKEY_CNTR
> > > >
> > > > This table also seems un ip-like, the usual format is a list of words,
> > > > I think.
> > >
> > > It is true for key<->value data, but it is less obvious for bit
> > > parsing.
> >
> > Several of the word decodes are from bit fields..
> >
> > > Internally, I tried to present them as list and it was ugly like hell
> > > without any chance (without extra parsing) to actual see if specific
> > > capability is present or no.
> >
> > lspci seems to have no problem being readable while doing this..
>
> No problem,
> If i understand you correctly, you are suggesting to drop parsing of
> "caps" as a separate command and embed it into general show <devname>.
>
> Can you help me and give an example of how would you present those caps?
> What will be the output of such command?
>  $ rdma dev show mlx5_4

ip-like style:

$ rdma dev show mlx5_4
5: mlx5_4:
    caps: <BAD_PKEY_CNTR, BAD_QKEY_CNTR, CHANGE_PHY_PORT, PORT_ACTIVE_EVENT, SYS_IMAGE_GUID, RC_RNR_NAK_GEN, MEM_WINDOW, UD_IP_CSUM, UD_TSO, XRC, MEM_MGT_EXTENSIONS, BLOCK_MULTICAST_LOOPBACK, MEM_WINDOW_TYPE_2B, RAW_IP_CSUM, SIGNATURE_HANDOVER, VIRTUAL_FUNCTION>
$ rdma link show mlx5_3
4/1: mlx5_3/1:
    caps: <CM, IP_BASED_GIDS>

Thanks

>
>  Thanks
>
> >
> > Jason



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Jason Gunthorpe]

On Tue, Jun 27, 2017 at 12:21:29PM +0300, Leon Romanovsky wrote:
> > What will be the output of such command?
> >  $ rdma dev show mlx5_4
> 
> ip-like style:
> 
> $ rdma dev show mlx5_4
> 5: mlx5_4:
>     caps: <BAD_PKEY_CNTR, BAD_QKEY_CNTR, CHANGE_PHY_PORT, PORT_ACTIVE_EVENT, SYS_IMAGE_GUID, RC_RNR_NAK_GEN, MEM_WINDOW, UD_IP_CSUM, UD_TSO, XRC, MEM_MGT_EXTENSIONS, BLOCK_MULTICAST_LOOPBACK, MEM_WINDOW_TYPE_2B, RAW_IP_CSUM, SIGNATURE_HANDOVER, VIRTUAL_FUNCTION>
> $ rdma link show mlx5_3
> 4/1: mlx5_3/1:
>     caps: <CM, IP_BASED_GIDS>

I think that is better, maybe it should only show under some kind of
verbose mode, I don't know, it depends what other stuff ends up being
displayed..

Are you going to dump the gid table and pkey table too in one of these commands?

Jason

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1290 bytes --]

On Tue, Jun 27, 2017 at 10:41:50AM -0600, Jason Gunthorpe wrote:
> On Tue, Jun 27, 2017 at 12:21:29PM +0300, Leon Romanovsky wrote:
> > > What will be the output of such command?
> > >  $ rdma dev show mlx5_4
> >
> > ip-like style:
> >
> > $ rdma dev show mlx5_4
> > 5: mlx5_4:
> >     caps: <BAD_PKEY_CNTR, BAD_QKEY_CNTR, CHANGE_PHY_PORT, PORT_ACTIVE_EVENT, SYS_IMAGE_GUID, RC_RNR_NAK_GEN, MEM_WINDOW, UD_IP_CSUM, UD_TSO, XRC, MEM_MGT_EXTENSIONS, BLOCK_MULTICAST_LOOPBACK, MEM_WINDOW_TYPE_2B, RAW_IP_CSUM, SIGNATURE_HANDOVER, VIRTUAL_FUNCTION>
> > $ rdma link show mlx5_3
> > 4/1: mlx5_3/1:
> >     caps: <CM, IP_BASED_GIDS>
>
> I think that is better, maybe it should only show under some kind of
> verbose mode, I don't know, it depends what other stuff ends up being
> displayed..
>
> Are you going to dump the gid table and pkey table too in one of these commands?

My initial plan was to put all parsers under their respective names, in
the similar way as I did for caps: $ rdma dev show mlx5_4 caps

So for large dumps, I'm going to use that technique again and maybe print summary as a default.
For example, for gids, we can print utilization as a summary while whole
table if someone really wants it: $ rdma link show mlx5_4/1 gids <index>

Something like that.

Thanks

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Jason Gunthorpe]

On Tue, Jun 27, 2017 at 08:33:01PM +0300, Leon Romanovsky wrote:

> My initial plan was to put all parsers under their respective names, in
> the similar way as I did for caps: $ rdma dev show mlx5_4 caps

I think you should have a useful summary display similar to 'ip a' and
other commands.

guid(s), subnet prefix or default gid for IB, lid/lmc, link state,
speed, mtu, pkeys protocol(s)

A show gid table command makes sense for rocee where it can show the
gid and the IP binding for it, rocee mode, etc..

Jason

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]

On Tue, Jun 27, 2017 at 11:37:35AM -0600, Jason Gunthorpe wrote:
> On Tue, Jun 27, 2017 at 08:33:01PM +0300, Leon Romanovsky wrote:
>
> > My initial plan was to put all parsers under their respective names, in
> > the similar way as I did for caps: $ rdma dev show mlx5_4 caps
>
> I think you should have a useful summary display similar to 'ip a' and
> other commands.
>
> guid(s), subnet prefix or default gid for IB, lid/lmc, link state,
> speed, mtu, pkeys protocol(s)

It will, but before I would like to see this tool be a part of
iproute2, so other people will be able to extend it in addition
to me.

Are you fine with the proposed code?

>
> A show gid table command makes sense for rocee where it can show the
> gid and the IP binding for it, rocee mode, etc..
>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Stephen Hemminger]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Tue, 27 Jun 2017 20:46:15 +0300
Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:

> On Tue, Jun 27, 2017 at 11:37:35AM -0600, Jason Gunthorpe wrote:
> > On Tue, Jun 27, 2017 at 08:33:01PM +0300, Leon Romanovsky wrote:
> >  
> > > My initial plan was to put all parsers under their respective names, in
> > > the similar way as I did for caps: $ rdma dev show mlx5_4 caps  
> >
> > I think you should have a useful summary display similar to 'ip a' and
> > other commands.
> >
> > guid(s), subnet prefix or default gid for IB, lid/lmc, link state,
> > speed, mtu, pkeys protocol(s)  
> 
> It will, but before I would like to see this tool be a part of
> iproute2, so other people will be able to extend it in addition
> to me.
> 
> Are you fine with the proposed code?
> 

Output formats need to be nailed down. The output of iproute2 commands is almost
like an ABI. Users build scripts to parse it (whether that is a great idea or not
is debateable, it mostly shows the weakness in programatic API's). Therefore fully
changing output formats in later revisions is likely to get users upset.

The first version doesn't have to be perfect, just close to the overall goal
of what is planned.  

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

On Tue, Jun 27, 2017 at 03:18:59PM -0700, Stephen Hemminger wrote:
> On Tue, 27 Jun 2017 20:46:15 +0300
> Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
>
> > On Tue, Jun 27, 2017 at 11:37:35AM -0600, Jason Gunthorpe wrote:
> > > On Tue, Jun 27, 2017 at 08:33:01PM +0300, Leon Romanovsky wrote:
> > >
> > > > My initial plan was to put all parsers under their respective names, in
> > > > the similar way as I did for caps: $ rdma dev show mlx5_4 caps
> > >
> > > I think you should have a useful summary display similar to 'ip a' and
> > > other commands.
> > >
> > > guid(s), subnet prefix or default gid for IB, lid/lmc, link state,
> > > speed, mtu, pkeys protocol(s)
> >
> > It will, but before I would like to see this tool be a part of
> > iproute2, so other people will be able to extend it in addition
> > to me.
> >
> > Are you fine with the proposed code?
> >
>
> Output formats need to be nailed down. The output of iproute2 commands is almost
> like an ABI. Users build scripts to parse it (whether that is a great idea or not
> is debateable, it mostly shows the weakness in programatic API's). Therefore fully
> changing output formats in later revisions is likely to get users upset.
>
> The first version doesn't have to be perfect, just close to the overall goal
> of what is planned.

In this version, I'm going to use arrays without indexes, because I prefer
to expose the bare minimum from the kernel, which is RDMA netlink. After
everything else is settled, I'll move those defines to UAPI and reuse them
in rdmatool.

I'll send new version with -d/--details flag and enrich minimal summary.
It won't include anything related to tables gids, pkey yet.

Thanks


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Jason Gunthorpe]

On Tue, Jun 27, 2017 at 03:18:59PM -0700, Stephen Hemminger wrote:
> On Tue, 27 Jun 2017 20:46:15 +0300
> Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> 
> > On Tue, Jun 27, 2017 at 11:37:35AM -0600, Jason Gunthorpe wrote:
> > > On Tue, Jun 27, 2017 at 08:33:01PM +0300, Leon Romanovsky wrote:
> > >  
> > > > My initial plan was to put all parsers under their respective names, in
> > > > the similar way as I did for caps: $ rdma dev show mlx5_4 caps  
> > >
> > > I think you should have a useful summary display similar to 'ip a' and
> > > other commands.
> > >
> > > guid(s), subnet prefix or default gid for IB, lid/lmc, link state,
> > > speed, mtu, pkeys protocol(s)  
> > 
> > It will, but before I would like to see this tool be a part of
> > iproute2, so other people will be able to extend it in addition
> > to me.
> > 
> > Are you fine with the proposed code?
> > 
> 
> Output formats need to be nailed down. The output of iproute2 commands is almost
> like an ABI. Users build scripts to parse it (whether that is a great idea or not
> is debateable, it mostly shows the weakness in programatic API's). Therefore fully
> changing output formats in later revisions is likely to get users upset.

It would be nice to see an example of what the completed command
should output to make judgements on the format.. Going bit by bit
doesn't really give a full picture, IHO.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1631 bytes --]

On Wed, Jun 28, 2017 at 10:11:12AM -0600, Jason Gunthorpe wrote:
> On Tue, Jun 27, 2017 at 03:18:59PM -0700, Stephen Hemminger wrote:
> > On Tue, 27 Jun 2017 20:46:15 +0300
> > Leon Romanovsky <leon@kernel.org> wrote:
> >
> > > On Tue, Jun 27, 2017 at 11:37:35AM -0600, Jason Gunthorpe wrote:
> > > > On Tue, Jun 27, 2017 at 08:33:01PM +0300, Leon Romanovsky wrote:
> > > >
> > > > > My initial plan was to put all parsers under their respective names, in
> > > > > the similar way as I did for caps: $ rdma dev show mlx5_4 caps
> > > >
> > > > I think you should have a useful summary display similar to 'ip a' and
> > > > other commands.
> > > >
> > > > guid(s), subnet prefix or default gid for IB, lid/lmc, link state,
> > > > speed, mtu, pkeys protocol(s)
> > >
> > > It will, but before I would like to see this tool be a part of
> > > iproute2, so other people will be able to extend it in addition
> > > to me.
> > >
> > > Are you fine with the proposed code?
> > >
> >
> > Output formats need to be nailed down. The output of iproute2 commands is almost
> > like an ABI. Users build scripts to parse it (whether that is a great idea or not
> > is debateable, it mostly shows the weakness in programatic API's). Therefore fully
> > changing output formats in later revisions is likely to get users upset.
>
> It would be nice to see an example of what the completed command
> should output to make judgements on the format.. Going bit by bit
> doesn't really give a full picture, IHO.

Bit by bit expansion allow easily control of what is needed. Mostly,
those full examples have nothing close to real use case.

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH iproute2 3/5] rdma: Add device capability parsing

[Stephen Hemminger]

[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]

On Tue, 27 Jun 2017 20:33:01 +0300
Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:

> On Tue, Jun 27, 2017 at 10:41:50AM -0600, Jason Gunthorpe wrote:
> > On Tue, Jun 27, 2017 at 12:21:29PM +0300, Leon Romanovsky wrote:  
> > > > What will be the output of such command?
> > > >  $ rdma dev show mlx5_4  
> > >
> > > ip-like style:
> > >
> > > $ rdma dev show mlx5_4
> > > 5: mlx5_4:
> > >     caps: <BAD_PKEY_CNTR, BAD_QKEY_CNTR, CHANGE_PHY_PORT, PORT_ACTIVE_EVENT, SYS_IMAGE_GUID, RC_RNR_NAK_GEN, MEM_WINDOW, UD_IP_CSUM, UD_TSO, XRC, MEM_MGT_EXTENSIONS, BLOCK_MULTICAST_LOOPBACK, MEM_WINDOW_TYPE_2B, RAW_IP_CSUM, SIGNATURE_HANDOVER, VIRTUAL_FUNCTION>
> > > $ rdma link show mlx5_3
> > > 4/1: mlx5_3/1:
> > >     caps: <CM, IP_BASED_GIDS>  
> >
> > I think that is better, maybe it should only show under some kind of
> > verbose mode, I don't know, it depends what other stuff ends up being
> > displayed..
> >
> > Are you going to dump the gid table and pkey table too in one of these commands?  
> 
> My initial plan was to put all parsers under their respective names, in
> the similar way as I did for caps: $ rdma dev show mlx5_4 caps
> 
> So for large dumps, I'm going to use that technique again and maybe print summary as a default.
> For example, for gids, we can print utilization as a summary while whole
> table if someone really wants it: $ rdma link show mlx5_4/1 gids <index>
> 
> Something like that.
> 
> Thanks
> 
> >
> > Jason  

Agree with discussion so far.

For iproute2 style commands, the show and set commands need to have similar arguments.
Ideally, everything after the colon in the show would be parameters to set command.

Please consider having a concise form for normal users and a detail form (with -d) for 
configuration and setup cases. The caps should not need to be displayed in normal show
output.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH iproute2 2/5] rdma: Add dev object

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Device (dev) object represents struct ib_device to user space.

The supported commands are show, set and help, but it doesn't print
anything except device name at this stage. The downstream patches will
fill this object with subcommands.

Print all devices:
 # rdma dev
1: mlx5_0:
2: mlx5_1:
3: mlx5_2:

Print specific device:
 # rdma dev show mlx5_1
2: mlx5_1:

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/Makefile |  2 +-
 rdma/dev.c    | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |  3 ++-
 rdma/rdma.h   |  6 ++++++
 rdma/utils.c  | 37 +++++++++++++++++++++++++++++++-----
 5 files changed, 101 insertions(+), 7 deletions(-)
 create mode 100644 rdma/dev.c

diff --git a/rdma/Makefile b/rdma/Makefile
index 64da2142..123d7ac5 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -2,7 +2,7 @@ include ../Config

 ifeq ($(HAVE_MNL),y)

-RDMA_OBJ = rdma.o utils.o
+RDMA_OBJ = rdma.o utils.o dev.o

 TARGETS=rdma
 CFLAGS += $(shell $(PKG_CONFIG) libmnl --cflags)
diff --git a/rdma/dev.c b/rdma/dev.c
new file mode 100644
index 00000000..5a3ee126
--- /dev/null
+++ b/rdma/dev.c
@@ -0,0 +1,60 @@
+/*
+ * dev.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+static int dev_help(struct rdma *rd)
+{
+	pr_out("Usage: %s dev show [DEV]\n", rd->filename);
+
+	/*
+	 * Example of set command:
+	 * pr_out("       %s dev set DEV [ node_desc { DESCRIPTION } ]\n", rd->filename);
+	 */
+	return 0;
+}
+
+static void dev_one_show(const struct dev_map *dev_map)
+{
+	pr_out("%u: %s:\n", dev_map->idx, dev_map->dev_name);
+}
+
+static int dev_show(struct rdma *rd)
+{
+	struct dev_map *dev_map;
+
+	if (rd_no_arg(rd)) {
+		list_for_each_entry(dev_map, &rd->dev_map_list, list)
+			dev_one_show(dev_map);
+	}
+	else {
+		dev_map = dev_map_lookup(rd, false);
+		if (!dev_map) {
+			pr_err("Wrong device name\n");
+			return -ENOENT;
+		}
+		dev_one_show(dev_map);
+	}
+	return 0;
+}
+
+int cmd_dev(struct rdma *rd)
+{
+	const struct rdma_cmd cmds[] = {
+		{ NULL,		dev_show },
+		{ "show",	dev_show },
+		{ "list",	dev_show },
+		{ "help",	dev_help },
+		{ 0 }
+	};
+
+	return rdma_exec_cmd(rd, cmds, "dev command");
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index 9c754da3..f904532c 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -18,7 +18,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { help }\n"
+	       "where  OBJECT := { dev | help }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }

@@ -33,6 +33,7 @@ static int rd_cmd(struct rdma *rd)
 	const struct rdma_cmd cmds[] = {
 		{ NULL,		cmd_help },
 		{ "help",	cmd_help },
+		{ "dev",	cmd_dev },
 		{ 0 }
 	};

diff --git a/rdma/rdma.h b/rdma/rdma.h
index 9841aebf..f5e104ec 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -58,12 +58,18 @@ bool rd_no_arg(struct rdma *rd);
 bool rd_argv_match(struct rdma *rd, const char *pattern);
 void rd_arg_inc(struct rdma *rd);

+/*
+ * Commands interface
+ */
+int cmd_dev(struct rdma *rd);
 int rdma_exec_cmd(struct rdma *rd, const struct rdma_cmd *c, const char *str);

 /*
  * Device manipulation
  */
 void rdma_free_devmap(struct rdma *rd);
+struct dev_map *dev_map_lookup(struct rdma *rd, bool allow_port_index);
+struct dev_map *_dev_map_lookup(struct rdma *rd, const char *dev_name);

 /*
  * Netlink
diff --git a/rdma/utils.c b/rdma/utils.c
index 96278a4c..e5c3dd6c 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -156,10 +156,9 @@ int rd_dev_init_cb(const struct nlmsghdr *nlh, void *data)
 	struct rdma *rd = data;
 	const char *dev_name;
 	uint32_t num_ports;
-	static int i = 1;

 	mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
-	if (!tb[RDMA_NLDEV_ATTR_DEV_NAME])
+	if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] || !tb[RDMA_NLDEV_ATTR_DEV_NAME])
 		return MNL_CB_ERROR;
 	if (!tb[RDMA_NLDEV_ATTR_PORT_INDEX]) {
 		pr_err("This tool doesn't support switches yet\n");
@@ -167,18 +166,18 @@ int rd_dev_init_cb(const struct nlmsghdr *nlh, void *data)
 	}

 	dev_name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
-	num_ports = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_PORT_INDEX]);
-
 	dev_map = dev_map_alloc(dev_name);
 	if (!dev_map)
 		/* The main function will cleanup the allocations */
 		return MNL_CB_ERROR;
 	list_add_tail(&dev_map->list, &rd->dev_map_list);
-	dev_map->idx = i++;

+	num_ports = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_PORT_INDEX]);
 	if (port_map_alloc(dev_map, num_ports))
 		return MNL_CB_ERROR;
 	dev_map->num_ports = num_ports;
+	dev_map->idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+	dev_map->caps = mnl_attr_get_u64(tb[RDMA_NLDEV_ATTR_CAP_FLAGS]);

 	return MNL_CB_OK;
 }
@@ -268,3 +267,31 @@ int rdma_recv_msg(struct rdma *rd, mnl_cb_t callback, void *data, unsigned int s
 	mnl_socket_close(rd->nl);
 	return ret;
 }
+
+struct dev_map *_dev_map_lookup(struct rdma *rd, const char *dev_name)
+{
+	struct dev_map *dev_map;
+
+	list_for_each_entry(dev_map, &rd->dev_map_list, list)
+		if (strcmp(dev_name, dev_map->dev_name) == 0)
+			return dev_map;
+
+	return NULL;
+}
+struct dev_map *dev_map_lookup(struct rdma *rd, bool allow_port_index)
+{
+	struct dev_map *dev_map;
+	char *dev_name;
+	char *slash;
+
+	dev_name = strdup(rd_argv(rd));
+	if (allow_port_index) {
+		slash = strrchr(dev_name, '/');
+		if (slash)
+			*slash = '\0';
+	}
+
+	dev_map = _dev_map_lookup(rd, dev_name);
+	free(dev_name);
+	return dev_map;
+}

[PATCH iproute2 4/5] rdma: Add link option and parsing

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Add link object interface together with port capability parsing command

$ rdma link
1/1: mlx5_0/1: caps 0x20
2/1: mlx5_1/1: caps 0x20
3/1: mlx5_2/1: caps 0x20
4/1: mlx5_3/1: caps 0x4010000
5/1: mlx5_4/1: caps 0x20

$ rdma link show mlx5_4
5/1: mlx5_4/1: caps 0x20

$ rdma link show mlx5_3 caps
4/1: mlx5_3/1: caps 0x4010000
Bit	Description
------------------------------------------
 16	PORT_CM
 26	PORT_IP_BASED_GIDS

$ rdma link show mlx5_4/1 caps
5/1: mlx5_4/1: caps 0x20
Bit	Description
------------------------------------------
 05	PORT_AUTO_MIGR

$ rdma link show mlx5_4/1 caps_flags
Unknown parameter 'caps_flags'

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/Makefile |   2 +-
 rdma/link.c   | 201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |   1 +
 rdma/rdma.h   |   4 ++
 rdma/utils.c  |   9 +++
 5 files changed, 216 insertions(+), 1 deletion(-)
 create mode 100644 rdma/link.c

diff --git a/rdma/Makefile b/rdma/Makefile
index 123d7ac5..1a9e4b1a 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -2,7 +2,7 @@ include ../Config

 ifeq ($(HAVE_MNL),y)

-RDMA_OBJ = rdma.o utils.o dev.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o

 TARGETS=rdma
 CFLAGS += $(shell $(PKG_CONFIG) libmnl --cflags)
diff --git a/rdma/link.c b/rdma/link.c
new file mode 100644
index 00000000..3d697205
--- /dev/null
+++ b/rdma/link.c
@@ -0,0 +1,201 @@
+/*
+ * link.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+static int link_help(struct rdma *rd)
+{
+	pr_out("Usage: %s link show [DEV/PORT_INDEX]\n", rd->filename);
+	pr_out("       %s link show DEV/PORT_INDEX caps\n", rd->filename);
+	return 0;
+}
+
+static const char *link_caps[64] = {
+	"UNKNOWN",
+	"PORT_SM",
+	"PORT_NOTICE",
+	"PORT_TRAP",
+	"PORT_OPT_IPD",
+	"PORT_AUTO_MIGR",
+	"PORT_SL_MAP",
+	"PORT_MKEY_NVRAM",
+	"PORT_PKEY_NVRAM",
+	"PORT_LED_INFO",
+	"PORT_SM_DISABLED",
+	"PORT_SYS_IMAGE_GUID",
+	"PORT_PKEY_SW_EXT_PORT_TRAP",
+	"UNKNOWN",
+	"PORT_EXTENDED_SPEEDS",
+	"UNKNOWN",
+	"PORT_CM",
+	"PORT_SNMP_TUNNEL",
+	"PORT_REINIT",
+	"PORT_DEVICE_MGMT",
+	"PORT_VENDOR_CLASS",
+	"PORT_DR_NOTICE",
+	"PORT_CAP_MASK_NOTICE",
+	"PORT_BOOT_MGMT",
+	"PORT_LINK_LATENCY",
+	"PORT_CLIENT_REG",
+	"PORT_IP_BASED_GIDS",
+};
+
+static int link_print_caps(struct rdma *rd)
+{
+	struct port_map *port_map = rd->port_map_curr;
+	struct dev_map *dev_map = rd->dev_map_curr;
+	uint64_t caps = port_map->caps;
+	uint32_t idx;
+
+	pr_out("%u/%u: %s/%u: ", dev_map->idx, port_map->idx, dev_map->dev_name, port_map->idx);
+	pr_out("caps 0x%" PRIx64 "\n", port_map->caps);
+	pr_out("Bit\tDescription\n");
+	pr_out("------------------------------------------\n");
+	for (idx = 0; idx < 64; idx++) {
+		if (caps & 0x1)
+			pr_out(" %02u\t%s\n", idx, link_caps[idx]?link_caps[idx]:"UNKNONW");
+		caps >>= 0x1;
+	}
+	return 0;
+}
+
+static int link_no_args(struct rdma *rd)
+{
+	struct port_map *port_map = rd->port_map_curr;
+	struct dev_map *dev_map = rd->dev_map_curr;
+
+	pr_out("%u/%u: %s/%u: ", dev_map->idx, port_map->idx, dev_map->dev_name, port_map->idx);
+	pr_out("caps 0x%" PRIx64 "\n", port_map->caps);
+	return 0;
+}
+
+static int link_one_show(struct rdma *rd)
+{
+	const struct rdma_cmd cmds[] = {
+		{ NULL,		link_no_args},
+		{ "caps",	link_print_caps},
+		{ 0 }
+	};
+
+	return rdma_exec_cmd(rd, cmds, "parameter");
+
+}
+
+static int port_init_cb(const struct nlmsghdr *nlh, void *data)
+{
+	struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+	struct port_map *port_map;
+	struct dev_map *dev_map;
+	struct rdma *rd = data;
+	uint32_t port_idx;
+	uint32_t caps;
+
+	mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+	if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] || !tb[RDMA_NLDEV_ATTR_DEV_NAME])
+		return MNL_CB_ERROR;
+	if (!tb[RDMA_NLDEV_ATTR_PORT_INDEX]) {
+		pr_err("This tool doesn't support switches yet\n");
+		return MNL_CB_ERROR;
+	}
+
+	dev_map = rd->dev_map_curr;
+
+	port_idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_PORT_INDEX]);
+	caps = mnl_attr_get_u64(tb[RDMA_NLDEV_ATTR_CAP_FLAGS]);
+
+	list_for_each_entry(port_map, &dev_map->port_map_list, list) {
+		if (port_map->idx != port_idx)
+			continue;
+
+		port_map->caps = caps;
+	}
+
+	return MNL_CB_OK;
+}
+
+
+static int fill_port_map(struct rdma *rd)
+{
+	uint32_t seq;
+	int ret;
+
+	rdma_prepare_msg(rd, RDMA_NLDEV_CMD_PORT_GET, &seq, (NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP));
+	mnl_attr_put_u32(rd->nlh, RDMA_NLDEV_ATTR_DEV_INDEX, rd->dev_map_curr->idx);
+	if ((ret = rdma_send_msg(rd)))
+		return ret;
+
+	return rdma_recv_msg(rd, port_init_cb, rd, seq);
+}
+
+static int link_show(struct rdma *rd)
+{
+	struct port_map *port_map;
+	struct dev_map *dev_map;
+	int ret = 0;
+
+	if (rd_no_arg(rd)) {
+		list_for_each_entry(dev_map, &rd->dev_map_list, list) {
+			rd->dev_map_curr = dev_map;
+			ret = fill_port_map(rd);
+			if (ret)
+				return ret;
+
+			list_for_each_entry(port_map, &dev_map->port_map_list, list) {
+				rd->port_map_curr = port_map;
+				ret = link_one_show(rd);
+				if (ret)
+					return ret;
+			}
+		}
+
+	}
+	else {
+		uint32_t port;
+		rd->dev_map_curr = dev_map_lookup(rd, true);
+		port = get_port_from_argv(rd);
+		if (!rd->dev_map_curr || port > rd->dev_map_curr->num_ports) {
+			pr_err("Wrong device name\n");
+			return -ENOENT;
+		}
+		rd_arg_inc(rd);
+
+		ret = fill_port_map(rd);
+		if (ret)
+			return ret;
+
+		list_for_each_entry(port_map, &rd->dev_map_curr->port_map_list, list) {
+			rd->port_map_curr = port_map;
+			if (port && port_map->idx != port)
+				continue;
+
+			if (!port || port_map->idx == port) {
+				ret = link_one_show(rd);
+				if (ret)
+					return ret;
+			}
+		}
+
+	}
+	return ret;
+}
+
+int cmd_link(struct rdma *rd)
+{
+	const struct rdma_cmd cmds[] = {
+		{ NULL,		link_show },
+		{ "show",	link_show },
+		{ "list",	link_show },
+		{ "help",	link_help },
+		{ 0 }
+	};
+
+	return rdma_exec_cmd(rd, cmds, "link command");
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index f904532c..ec6c2edb 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -34,6 +34,7 @@ static int rd_cmd(struct rdma *rd)
 		{ NULL,		cmd_help },
 		{ "help",	cmd_help },
 		{ "dev",	cmd_dev },
+		{ "link",	cmd_link },
 		{ 0 }
 	};

diff --git a/rdma/rdma.h b/rdma/rdma.h
index 8cca0f28..553a4fc2 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -26,6 +26,7 @@
 struct port_map {
 	struct list_head list;
 	uint32_t idx;
+	uint64_t caps;
 };

 struct dev_map {
@@ -43,6 +44,7 @@ struct rdma {
 	char *filename;
 	struct list_head dev_map_list;
 	struct dev_map *dev_map_curr;
+	struct port_map *port_map_curr;
 	struct mnl_socket *nl;
 	struct nlmsghdr *nlh;
 	char *buff;
@@ -60,11 +62,13 @@ bool rd_no_arg(struct rdma *rd);
 bool rd_argv_match(struct rdma *rd, const char *pattern);
 void rd_arg_inc(struct rdma *rd);
 char *rd_argv(struct rdma *rd);
+uint32_t get_port_from_argv(struct rdma *rd);

 /*
  * Commands interface
  */
 int cmd_dev(struct rdma *rd);
+int cmd_link(struct rdma *rd);
 int rdma_exec_cmd(struct rdma *rd, const struct rdma_cmd *c, const char *str);

 /*
diff --git a/rdma/utils.c b/rdma/utils.c
index e7f257e3..204d4c4e 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -62,6 +62,15 @@ bool rd_no_arg(struct rdma *rd)
 	return rd_argc(rd) == 0;
 }

+uint32_t get_port_from_argv(struct rdma *rd)
+{
+        char *slash;
+
+        slash = strchr(rd_argv(rd), '/');
+        /* if no port found, return 0 */
+        return (slash) ? (atoi(slash + 1)):0;
+}
+
 static struct dev_map *dev_map_alloc(const char *dev_name)
 {
 	struct dev_map *dev_map;

[PATCH iproute2 5/5] rdma: Add initial manual for the tool

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 man/man8/rdma.8 | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)
 create mode 100644 man/man8/rdma.8

diff --git a/man/man8/rdma.8 b/man/man8/rdma.8
new file mode 100644
index 00000000..7578c15e
--- /dev/null
+++ b/man/man8/rdma.8
@@ -0,0 +1,82 @@
+.TH RDMA 8 "28 Mar 2017" "iproute2" "Linux"
+.SH NAME
+rdma \- RDMA tool
+.SH SYNOPSIS
+.sp
+.ad l
+.in +8
+.ti -8
+.B rdma
+.RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | "
+.BR help " }"
+.sp
+
+.ti -8
+.IR OBJECT " := { "
+.BR dev " | " link " }"
+.sp
+
+.ti -8
+.IR OPTIONS " := { "
+\fB\-V\fR[\fIersion\fR] }
+
+.SH OPTIONS
+
+.TP
+.BR "\-V" , " -Version"
+Print the version of the
+.B rdma
+tool and exit.
+
+.SS
+.I OBJECT
+
+.TP
+.B dev
+- RDMA device.
+
+.TP
+.B link
+- RDMA port related.
+
+.PP
+The names of all objects may be written in full or
+abbreviated form, for example
+.B stats
+can be abbreviated as
+.B stat
+or just
+.B s.
+
+.SS
+.I COMMAND
+
+Specifies the action to perform on the object.
+The set of possible actions depends on the object type.
+As a rule, it is possible to
+.B show
+(or
+.B list
+) objects, but some objects do not allow all of these operations
+or have some additional commands. The
+.B help
+command is available for all objects. It prints
+out a list of available commands and argument syntax conventions.
+.sp
+If no command is given, some default command is assumed.
+Usually it is
+.B list
+or, if the objects of this class cannot be listed,
+.BR "help" .
+
+.SH EXIT STATUS
+Exit status is 0 if command was successful or a positive integer upon failure.
+
+.SH REPORTING BUGS
+Report any bugs to the Linux RDMA mailing list
+.B <linux-rdma@vger.kernel.org>
+where the development and maintenance is primarily done.
+You do not have to be subscribed to the list to send a message there.
+
+.SH AUTHOR
+Leon Romanovsky <leonro@mellanox.com>