[PATCH for-3.3 0/6] IB: set of fixes and additions

[PATCH for-3.3 0/6] IB: set of fixes and additions

[Or Gerlitz]

The series is made of four elements

This subset (patches 1-3) fixes two bugs port info related bugs, where
the first patch is just making following code related to speed possible...

[PATCH 1/6] IB: use central enum for speed instead of hard-coded values
[PATCH 2/6] IB/core: fix wrong display of rate in sysfs
[PATCH 3/6] IB/mlx4: fix wrong info returned when querying IBoE ports

This patch comes to address the long pending patch for 4k mtu
along the lines of what was requested over the last review

[PATCH 4/6] net/mlx4: allow for dynamic mtu configuration for IB ports

This patch is simple and comes to unify the approach for checksum
indication with possible / hopefully soon changes in libibverbs

[PATCH 5/6] IB: change CQ TCP/IP checksum offload mark to use bit flag indication instead of integer

This patch extends the IPoIB child scheme, and will serve as
a building block when using IPoIB in the Virtualization space.

[PATCH 6/6] IB/ipoib: add support for clones / multiple childs on the same partition

Roland, Yep, this submission comes a bit into the 3.3 merge cycle, but
with its fairly limited scope, I hope it can make it there... as of the recent
changes in mlx4_core, patches 3 & 4 need you to pull Linus tree, but this should
be done now anyway for any future patches to the IB stack through out the 3.3 cycle
and elsewhere


Or Gerlitz (6):
  IB: use central enum for speed instead of hard-coded values
  IB/core: fix wrong display of rate in sysfs
  IB/mlx4: fix wrong info returned when querying IBoE ports
  net/mlx4: allow for dynamic mtu configuration for IB ports
  IB: change CQ TCP/IP checksum offload mark to use bit flag indication instead of integer
  IB/ipoib: add support for clones / multiple childs on the same partition

 Documentation/infiniband/ipoib.txt           |   24 +++++
 drivers/infiniband/core/sysfs.c              |   25 ++++--
 drivers/infiniband/hw/amso1100/c2_provider.c |    2 +-
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |    2 +-
 drivers/infiniband/hw/cxgb4/provider.c       |    2 +-
 drivers/infiniband/hw/ehca/ehca_hca.c        |    2 +-
 drivers/infiniband/hw/mlx4/cq.c              |    3 +-
 drivers/infiniband/hw/mlx4/main.c            |  107 +++++++++++++-----------
 drivers/infiniband/hw/mthca/mthca_cq.c       |    3 +-
 drivers/infiniband/hw/nes/nes_verbs.c        |    2 +-
 drivers/infiniband/hw/qib/qib_rc.c           |    1 -
 drivers/infiniband/hw/qib/qib_uc.c           |    1 -
 drivers/infiniband/ulp/ipoib/ipoib.h         |    9 ++-
 drivers/infiniband/ulp/ipoib/ipoib_ib.c      |    3 +-
 drivers/infiniband/ulp/ipoib/ipoib_main.c    |   48 ++++++++---
 drivers/infiniband/ulp/ipoib/ipoib_vlan.c    |   40 ++++++---
 drivers/net/ethernet/mellanox/mlx4/main.c    |  118 +++++++++++++++++++++++++-
 drivers/net/ethernet/mellanox/mlx4/mlx4.h    |    2 +
 drivers/net/ethernet/mellanox/mlx4/port.c    |   25 +++++-
 include/linux/mlx4/device.h                  |    1 +
 include/rdma/ib_verbs.h                      |   11 ++-
 21 files changed, 330 insertions(+), 101 deletions(-)

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/6] IB: use central enum for speed instead of hard-coded values

[Or Gerlitz]

The kernel IB stack uses one enumeration for IB speed, which wasn't
explicitly specified in the verbs header file. Add that enum, and
use it all over the code. Note that the IB speed/width notation is
also used by iWARP and IBoE hw drivers who apply the convention of
rate = speed X width, to advertize their port link rate.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/sysfs.c              |   15 +++++++++------
 drivers/infiniband/hw/amso1100/c2_provider.c |    2 +-
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |    2 +-
 drivers/infiniband/hw/cxgb4/provider.c       |    2 +-
 drivers/infiniband/hw/ehca/ehca_hca.c        |    2 +-
 drivers/infiniband/hw/mlx4/main.c            |   10 +++++-----
 drivers/infiniband/hw/nes/nes_verbs.c        |    2 +-
 include/rdma/ib_verbs.h                      |    9 +++++++++
 8 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index c61bca3..9ce70ca 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -189,21 +189,24 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 	rate = (25 * attr.active_speed) / 10;

 	switch (attr.active_speed) {
-	case 2:
+	case IB_SPEED_SDR:
+		speed = " SDR";
+		break;
+	case IB_SPEED_DDR:
 		speed = " DDR";
 		break;
-	case 4:
+	case IB_SPEED_QDR:
 		speed = " QDR";
 		break;
-	case 8:
+	case IB_SPEED_FDR10:
 		speed = " FDR10";
 		rate = 10;
 		break;
-	case 16:
+	case IB_SPEED_FDR:
 		speed = " FDR";
 		rate = 14;
 		break;
-	case 32:
+	case IB_SPEED_EDR:
 		speed = " EDR";
 		rate = 25;
 		break;
@@ -214,7 +217,7 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 		return -EINVAL;

 	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
-		       rate, (attr.active_speed == 1) ? ".5" : "",
+		       rate, (attr.active_speed == IB_SPEED_SDR) ? ".5" : "",
 		       ib_width_enum_to_int(attr.active_width), speed);
 }

diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c
index 12f923d..07eb3a8 100644
--- a/drivers/infiniband/hw/amso1100/c2_provider.c
+++ b/drivers/infiniband/hw/amso1100/c2_provider.c
@@ -94,7 +94,7 @@ static int c2_query_port(struct ib_device *ibdev,
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = 1;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;

 	return 0;
 }
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 37c224f..0bdf09a 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -1227,7 +1227,7 @@ static int iwch_query_port(struct ib_device *ibdev,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index 247fe70..be1c18f 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -329,7 +329,7 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index 73edc36..9ed4d25 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -233,7 +233,7 @@ int ehca_query_port(struct ib_device *ibdev,
 		props->phys_state      = 5;
 		props->state           = rblock->state;
 		props->active_width    = IB_WIDTH_12X;
-		props->active_speed    = 0x1;
+		props->active_speed    = IB_SPEED_SDR;
 	}

 query_port1:
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 7b445df..6ff6bdf 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -215,16 +215,16 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 		switch (ext_active_speed) {
 		case 1:
-			props->active_speed = 16; /* FDR */
+			props->active_speed = IB_SPEED_FDR;
 			break;
 		case 2:
-			props->active_speed = 32; /* EDR */
+			props->active_speed = IB_SPEED_EDR;
 			break;
 		}
 	}

 	/* If reported active speed is QDR, check if is FDR-10 */
-	if (props->active_speed == 4) {
+	if (props->active_speed == IB_SPEED_QDR) {
 		if (to_mdev(ibdev)->dev->caps.ext_port_cap[port] &
 		    MLX_EXT_PORT_CAP_FLAG_EXTENDED_PORT_INFO) {
 			init_query_mad(in_mad);
@@ -238,7 +238,7 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 			/* Checking LinkSpeedActive for FDR-10 */
 			if (out_mad->data[15] & 0x1)
-				props->active_speed = 8;
+				props->active_speed = IB_SPEED_FDR10;
 		}
 	}

@@ -259,7 +259,7 @@ static int eth_link_query_port(struct ib_device *ibdev, u8 port,
 	enum ib_mtu tmp;

 	props->active_width	= IB_WIDTH_1X;
-	props->active_speed	= 4;
+	props->active_speed	= IB_SPEED_QDR;
 	props->port_cap_flags	= IB_PORT_CM_SUP;
 	props->gid_tbl_len	= to_mdev(ibdev)->dev->caps.gid_table_len[port];
 	props->max_msg_sz	= to_mdev(ibdev)->dev->caps.max_msg_sz;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index 5095bc4..b7b9c85 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -597,7 +597,7 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = IB_WIDTH_4X;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;
 	props->max_msg_sz = 0x80000000;

 	return 0;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index bf5daaf..01179b0 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -239,6 +239,15 @@ static inline int ib_width_enum_to_int(enum ib_port_width width)
 	}
 }

+enum ib_port_seed {
+	IB_SPEED_SDR	= 1,
+	IB_SPEED_DDR	= 2,
+	IB_SPEED_QDR	= 4,
+	IB_SPEED_FDR10	= 8,
+	IB_SPEED_FDR	= 16,
+	IB_SPEED_EDR	= 32
+};
+
 struct ib_protocol_stats {
 	/* TBD... */
 };
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 1/6] IB: use central enum for speed instead of hard-coded values

[Hefty, Sean]

> +enum ib_port_seed {

typo on enum name (seed -> speed)

The rest of the patch looks good to me.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V1 1/6] IB: use central enum for speed instead of hard-coded values

[Or Gerlitz]

The kernel IB stack uses one enumeration for IB speed, which wasn't
explicitly specified in the verbs header file. Add that enum, and
use it all over the code. Note that the IB speed/width notation is
also used by iWARP and IBoE hw drivers who apply the convention of
rate = speed X width, to advertize their port link rate.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---

changes from v0:
 fixed typo in the enum type name (was ib_port_seed instead of ib_port_speed)

 drivers/infiniband/core/sysfs.c              |   15 +++++++-----
 drivers/infiniband/core/uverbs_cmd.c         |    3 ++
 drivers/infiniband/core/verbs.c              |    1 +
 drivers/infiniband/hw/amso1100/c2_provider.c |    2 +-
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |    2 +-
 drivers/infiniband/hw/cxgb4/provider.c       |    2 +-
 drivers/infiniband/hw/ehca/ehca_hca.c        |    2 +-
 drivers/infiniband/hw/mlx4/main.c            |   10 ++++----
 drivers/infiniband/hw/mlx4/qp.c              |   31 +++++++++++++++++++++----
 drivers/infiniband/hw/nes/nes_verbs.c        |    2 +-
 include/rdma/ib_verbs.h                      |   11 ++++++++-
 11 files changed, 59 insertions(+), 22 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index c61bca3..9ce70ca 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -189,21 +189,24 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 	rate = (25 * attr.active_speed) / 10;

 	switch (attr.active_speed) {
-	case 2:
+	case IB_SPEED_SDR:
+		speed = " SDR";
+		break;
+	case IB_SPEED_DDR:
 		speed = " DDR";
 		break;
-	case 4:
+	case IB_SPEED_QDR:
 		speed = " QDR";
 		break;
-	case 8:
+	case IB_SPEED_FDR10:
 		speed = " FDR10";
 		rate = 10;
 		break;
-	case 16:
+	case IB_SPEED_FDR:
 		speed = " FDR";
 		rate = 14;
 		break;
-	case 32:
+	case IB_SPEED_EDR:
 		speed = " EDR";
 		rate = 25;
 		break;
@@ -214,7 +217,7 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 		return -EINVAL;

 	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
-		       rate, (attr.active_speed == 1) ? ".5" : "",
+		       rate, (attr.active_speed == IB_SPEED_SDR) ? ".5" : "",
 		       ib_width_enum_to_int(attr.active_width), speed);
 }

diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index b930da4..8722e96 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1399,6 +1399,9 @@ ssize_t ib_uverbs_create_qp(struct ib_uverbs_file *file,
 	if (copy_from_user(&cmd, buf, sizeof cmd))
 		return -EFAULT;

+	if (cmd.qp_type == IB_QPT_RAW_PACKET && !capable(CAP_NET_RAW))
+		return -EPERM;
+
 	INIT_UDATA(&udata, buf + sizeof cmd,
 		   (unsigned long) cmd.response + sizeof resp,
 		   in_len - sizeof cmd, out_len - sizeof resp);
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 602b1bd..f73e15b 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -479,6 +479,7 @@ static const struct {
 				[IB_QPT_UD]  = (IB_QP_PKEY_INDEX		|
 						IB_QP_PORT			|
 						IB_QP_QKEY),
+				[IB_QPT_RAW_PACKET] = IB_QP_PORT,
 				[IB_QPT_UC]  = (IB_QP_PKEY_INDEX		|
 						IB_QP_PORT			|
 						IB_QP_ACCESS_FLAGS),
diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c
index 12f923d..07eb3a8 100644
--- a/drivers/infiniband/hw/amso1100/c2_provider.c
+++ b/drivers/infiniband/hw/amso1100/c2_provider.c
@@ -94,7 +94,7 @@ static int c2_query_port(struct ib_device *ibdev,
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = 1;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;

 	return 0;
 }
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 37c224f..0bdf09a 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -1227,7 +1227,7 @@ static int iwch_query_port(struct ib_device *ibdev,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index 247fe70..be1c18f 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -329,7 +329,7 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index 73edc36..9ed4d25 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -233,7 +233,7 @@ int ehca_query_port(struct ib_device *ibdev,
 		props->phys_state      = 5;
 		props->state           = rblock->state;
 		props->active_width    = IB_WIDTH_12X;
-		props->active_speed    = 0x1;
+		props->active_speed    = IB_SPEED_SDR;
 	}

 query_port1:
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 7b445df..6ff6bdf 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -215,16 +215,16 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 		switch (ext_active_speed) {
 		case 1:
-			props->active_speed = 16; /* FDR */
+			props->active_speed = IB_SPEED_FDR;
 			break;
 		case 2:
-			props->active_speed = 32; /* EDR */
+			props->active_speed = IB_SPEED_EDR;
 			break;
 		}
 	}

 	/* If reported active speed is QDR, check if is FDR-10 */
-	if (props->active_speed == 4) {
+	if (props->active_speed == IB_SPEED_QDR) {
 		if (to_mdev(ibdev)->dev->caps.ext_port_cap[port] &
 		    MLX_EXT_PORT_CAP_FLAG_EXTENDED_PORT_INFO) {
 			init_query_mad(in_mad);
@@ -238,7 +238,7 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 			/* Checking LinkSpeedActive for FDR-10 */
 			if (out_mad->data[15] & 0x1)
-				props->active_speed = 8;
+				props->active_speed = IB_SPEED_FDR10;
 		}
 	}

@@ -259,7 +259,7 @@ static int eth_link_query_port(struct ib_device *ibdev, u8 port,
 	enum ib_mtu tmp;

 	props->active_width	= IB_WIDTH_1X;
-	props->active_speed	= 4;
+	props->active_speed	= IB_SPEED_QDR;
 	props->port_cap_flags	= IB_PORT_CM_SUP;
 	props->gid_tbl_len	= to_mdev(ibdev)->dev->caps.gid_table_len[port];
 	props->max_msg_sz	= to_mdev(ibdev)->dev->caps.max_msg_sz;
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index aa2aefa..3b434f8 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -84,6 +84,10 @@ enum {
 	MLX4_IB_CACHE_LINE_SIZE	= 64,
 };

+enum {
+	IB_MTU_RAW_QP = 7,
+};
+
 static const __be32 mlx4_ib_opcode[] = {
 	[IB_WR_SEND]				= cpu_to_be32(MLX4_OPCODE_SEND),
 	[IB_WR_LSO]				= cpu_to_be32(MLX4_OPCODE_LSO),
@@ -573,7 +577,12 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd,
 	if (sqpn) {
 		qpn = sqpn;
 	} else {
-		err = mlx4_qp_reserve_range(dev->dev, 1, 1, &qpn);
+		/* Raw packet QPNs must be aligned to 8 bits.
+		   If not, the blueflame setup code runs over the vlan bits */
+		if (init_attr->qp_type == IB_QPT_RAW_PACKET)
+			err = mlx4_qp_reserve_range(dev->dev, 1, 1 << 8, &qpn);
+		else
+			err = mlx4_qp_reserve_range(dev->dev, 1, 1, &qpn);
 		if (err)
 			goto err_wrid;
 	}
@@ -791,6 +800,7 @@ struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd,
 	case IB_QPT_RC:
 	case IB_QPT_UC:
 	case IB_QPT_UD:
+	case IB_QPT_RAW_PACKET:
 	{
 		qp = kzalloc(sizeof *qp, GFP_KERNEL);
 		if (!qp)
@@ -872,7 +882,8 @@ static int to_mlx4_st(enum ib_qp_type type)
 	case IB_QPT_XRC_INI:
 	case IB_QPT_XRC_TGT:	return MLX4_QP_ST_XRC;
 	case IB_QPT_SMI:
-	case IB_QPT_GSI:	return MLX4_QP_ST_MLX;
+	case IB_QPT_GSI:
+	case IB_QPT_RAW_PACKET:	return MLX4_QP_ST_MLX;
 	default:		return -1;
 	}
 }
@@ -1039,8 +1050,9 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,
 			break;
 		}
 	}
-
-	if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI)
+	if (ibqp->qp_type == IB_QPT_RAW_PACKET)
+		context->mtu_msgmax = (IB_MTU_RAW_QP << 5) | 31;
+	else if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI)
 		context->mtu_msgmax = (IB_MTU_4096 << 5) | 11;
 	else if (ibqp->qp_type == IB_QPT_UD) {
 		if (qp->flags & MLX4_IB_QP_LSO)
@@ -1200,7 +1212,8 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,
 	if (cur_state == IB_QPS_INIT &&
 	    new_state == IB_QPS_RTR  &&
 	    (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI ||
-	     ibqp->qp_type == IB_QPT_UD)) {
+	     ibqp->qp_type == IB_QPT_UD ||
+	     ibqp->qp_type == IB_QPT_RAW_PACKET)) {
 		context->pri_path.sched_queue = (qp->port - 1) << 6;
 		if (is_qp0(dev, qp))
 			context->pri_path.sched_queue |= MLX4_IB_DEFAULT_QP0_SCHED_QUEUE;
@@ -1319,6 +1332,14 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
 		goto out;
 	}

+	if ((attr_mask & IB_QP_PORT) && (ibqp->qp_type == IB_QPT_RAW_PACKET) &&
+		(rdma_port_get_link_layer(&dev->ib_dev, attr->port_num)
+				!= IB_LINK_LAYER_ETHERNET)) {
+		printk(KERN_WARNING "mlx4_ib: RAW PACKET QPs can "
+			"only be used over Ethernet ports\n");
+		goto out;
+	}
+
 	if (attr_mask & IB_QP_PKEY_INDEX) {
 		int p = attr_mask & IB_QP_PORT ? attr->port_num : qp->port;
 		if (attr->pkey_index >= dev->dev->caps.pkey_table_len[p])
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index 5095bc4..b7b9c85 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -597,7 +597,7 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = IB_WIDTH_4X;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;
 	props->max_msg_sz = 0x80000000;

 	return 0;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index bf5daaf..857dbd5 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -239,6 +239,15 @@ static inline int ib_width_enum_to_int(enum ib_port_width width)
 	}
 }

+enum ib_port_speed {
+	IB_SPEED_SDR	= 1,
+	IB_SPEED_DDR	= 2,
+	IB_SPEED_QDR	= 4,
+	IB_SPEED_FDR10	= 8,
+	IB_SPEED_FDR	= 16,
+	IB_SPEED_EDR	= 32
+};
+
 struct ib_protocol_stats {
 	/* TBD... */
 };
@@ -596,7 +605,7 @@ enum ib_qp_type {
 	IB_QPT_UD,
 	IB_QPT_RAW_IPV6,
 	IB_QPT_RAW_ETHERTYPE,
-	/* Save 8 for RAW_PACKET */
+ 	IB_QPT_RAW_PACKET = 8,
 	IB_QPT_XRC_INI = 9,
 	IB_QPT_XRC_TGT,
 	IB_QPT_MAX
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V1 1/6] IB: use central enum for speed instead of hard-coded values

[Roland Dreier]

Seems to have the raw packet QP stuff mixed in now?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V1 1/6] IB: use central enum for speed instead of hard-coded values

[Or Gerlitz]

On Thu, Jan 12, 2012 at 9:30 PM, Roland Dreier <roland-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> Seems to have the raw packet QP stuff mixed in now?

sorry, my bad, will fix and resend

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V1 1/6] IB: use central enum for speed instead of hard-coded values

[Or Gerlitz]

On 1/12/2012 9:30 PM, Roland Dreier wrote:
> Seems to have the raw packet QP stuff mixed in now?
Hi Roland,

Will you be looking on this series for 3.3?

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 1/6] IB: use central enum for speed instead of hard-coded values

[Or Gerlitz]

The kernel IB stack uses one enumeration for IB speed, which wasn't
explicitly specified in the verbs header file. Add that enum, and
use it all over the code. Note that the IB speed/width notation is
also used by iWARP and IBoE hw drivers who apply the convention of
rate = speed X width, to advertize their port link rate.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---

changes from v0:
 fixed typo in the enum type name (was ib_port_seed instead of ib_port_speed)

changes from v1:
 removed raw qp code which went in by mistake

 drivers/infiniband/core/sysfs.c              |   15 +++++++++------
 drivers/infiniband/hw/amso1100/c2_provider.c |    2 +-
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |    2 +-
 drivers/infiniband/hw/cxgb4/provider.c       |    2 +-
 drivers/infiniband/hw/ehca/ehca_hca.c        |    2 +-
 drivers/infiniband/hw/mlx4/main.c            |   10 +++++-----
 drivers/infiniband/hw/nes/nes_verbs.c        |    2 +-
 include/rdma/ib_verbs.h                      |    9 +++++++++
 8 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index c61bca3..9ce70ca 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -189,21 +189,24 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 	rate = (25 * attr.active_speed) / 10;

 	switch (attr.active_speed) {
-	case 2:
+	case IB_SPEED_SDR:
+		speed = " SDR";
+		break;
+	case IB_SPEED_DDR:
 		speed = " DDR";
 		break;
-	case 4:
+	case IB_SPEED_QDR:
 		speed = " QDR";
 		break;
-	case 8:
+	case IB_SPEED_FDR10:
 		speed = " FDR10";
 		rate = 10;
 		break;
-	case 16:
+	case IB_SPEED_FDR:
 		speed = " FDR";
 		rate = 14;
 		break;
-	case 32:
+	case IB_SPEED_EDR:
 		speed = " EDR";
 		rate = 25;
 		break;
@@ -214,7 +217,7 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 		return -EINVAL;

 	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
-		       rate, (attr.active_speed == 1) ? ".5" : "",
+		       rate, (attr.active_speed == IB_SPEED_SDR) ? ".5" : "",
 		       ib_width_enum_to_int(attr.active_width), speed);
 }

diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c
index 12f923d..07eb3a8 100644
--- a/drivers/infiniband/hw/amso1100/c2_provider.c
+++ b/drivers/infiniband/hw/amso1100/c2_provider.c
@@ -94,7 +94,7 @@ static int c2_query_port(struct ib_device *ibdev,
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = 1;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;

 	return 0;
 }
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 37c224f..0bdf09a 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -1227,7 +1227,7 @@ static int iwch_query_port(struct ib_device *ibdev,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index 247fe70..be1c18f 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -329,7 +329,7 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index 73edc36..9ed4d25 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -233,7 +233,7 @@ int ehca_query_port(struct ib_device *ibdev,
 		props->phys_state      = 5;
 		props->state           = rblock->state;
 		props->active_width    = IB_WIDTH_12X;
-		props->active_speed    = 0x1;
+		props->active_speed    = IB_SPEED_SDR;
 	}

 query_port1:
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 7b445df..6ff6bdf 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -215,16 +215,16 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 		switch (ext_active_speed) {
 		case 1:
-			props->active_speed = 16; /* FDR */
+			props->active_speed = IB_SPEED_FDR;
 			break;
 		case 2:
-			props->active_speed = 32; /* EDR */
+			props->active_speed = IB_SPEED_EDR;
 			break;
 		}
 	}

 	/* If reported active speed is QDR, check if is FDR-10 */
-	if (props->active_speed == 4) {
+	if (props->active_speed == IB_SPEED_QDR) {
 		if (to_mdev(ibdev)->dev->caps.ext_port_cap[port] &
 		    MLX_EXT_PORT_CAP_FLAG_EXTENDED_PORT_INFO) {
 			init_query_mad(in_mad);
@@ -238,7 +238,7 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 			/* Checking LinkSpeedActive for FDR-10 */
 			if (out_mad->data[15] & 0x1)
-				props->active_speed = 8;
+				props->active_speed = IB_SPEED_FDR10;
 		}
 	}

@@ -259,7 +259,7 @@ static int eth_link_query_port(struct ib_device *ibdev, u8 port,
 	enum ib_mtu tmp;

 	props->active_width	= IB_WIDTH_1X;
-	props->active_speed	= 4;
+	props->active_speed	= IB_SPEED_QDR;
 	props->port_cap_flags	= IB_PORT_CM_SUP;
 	props->gid_tbl_len	= to_mdev(ibdev)->dev->caps.gid_table_len[port];
 	props->max_msg_sz	= to_mdev(ibdev)->dev->caps.max_msg_sz;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index 5095bc4..b7b9c85 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -597,7 +597,7 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = IB_WIDTH_4X;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;
 	props->max_msg_sz = 0x80000000;

 	return 0;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index bf5daaf..ff22a73 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -239,6 +239,15 @@ static inline int ib_width_enum_to_int(enum ib_port_width width)
 	}
 }

+enum ib_port_speed {
+	IB_SPEED_SDR	= 1,
+	IB_SPEED_DDR	= 2,
+	IB_SPEED_QDR	= 4,
+	IB_SPEED_FDR10	= 8,
+	IB_SPEED_FDR	= 16,
+	IB_SPEED_EDR	= 32
+};
+
 struct ib_protocol_stats {
 	/* TBD... */
 };
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V3 1/6] IB: use central enum for speed instead of hard-coded values

[Or Gerlitz]

The kernel IB stack uses one enumeration for IB speed, which wasn't
explicitly specified in the verbs header file. Add that enum, and
use it all over the code. Note that the IB speed/width notation is
also used by iWARP and IBoE hw drivers who apply the convention of
rate = speed X width, to advertize their port link rate.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---

changes from v0:
 fixed typo in the enum type name (was ib_port_seed instead of
ib_port_speed)

changes from v1:
 removed raw qp code which went in by mistake

changes from v2:
 changed the patch such that it applies (on
 drivers/infiniband/core/sysfs.c) **after**
 patch #2 which was modified by Roland

This twist doesn't have effect patches 3-6 of this series

 drivers/infiniband/core/sysfs.c              |   12 ++++++------
 drivers/infiniband/hw/amso1100/c2_provider.c |    2 +-
 drivers/infiniband/hw/cxgb3/iwch_provider.c  |    2 +-
 drivers/infiniband/hw/cxgb4/provider.c       |    2 +-
 drivers/infiniband/hw/ehca/ehca_hca.c        |    2 +-
 drivers/infiniband/hw/mlx4/main.c            |   10 +++++-----
 drivers/infiniband/hw/nes/nes_verbs.c        |    2 +-
 include/rdma/ib_verbs.h                      |    9 +++++++++
 8 files changed, 25 insertions(+), 16 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index acb4d7a..1611569 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -187,27 +187,27 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 		return ret;

 	switch (attr.active_speed) {
-	case 1:
+	case IB_SPEED_SDR:
 		speed = " SDR";
 		rate = 25;
 		break;
-	case 2:
+	case IB_SPEED_DDR:
 		speed = " DDR";
 		rate = 50;
 		break;
-	case 4:
+	case IB_SPEED_QDR:
 		speed = " QDR";
 		rate = 100;
 		break;
-	case 8:
+	case IB_SPEED_FDR10:
 		speed = " FDR10";
 		rate = 100;
 		break;
-	case 16:
+	case IB_SPEED_FDR:
 		speed = " FDR";
 		rate = 140;
 		break;
-	case 32:
+	case IB_SPEED_EDR:
 		speed = " EDR";
 		rate = 250;
 		break;
diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c
index 12f923d..07eb3a8 100644
--- a/drivers/infiniband/hw/amso1100/c2_provider.c
+++ b/drivers/infiniband/hw/amso1100/c2_provider.c
@@ -94,7 +94,7 @@ static int c2_query_port(struct ib_device *ibdev,
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = 1;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;

 	return 0;
 }
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 37c224f..0bdf09a 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -1227,7 +1227,7 @@ static int iwch_query_port(struct ib_device *ibdev,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index 247fe70..be1c18f 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -329,7 +329,7 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,
 	props->gid_tbl_len = 1;
 	props->pkey_tbl_len = 1;
 	props->active_width = 2;
-	props->active_speed = 2;
+	props->active_speed = IB_SPEED_DDR;
 	props->max_msg_sz = -1;

 	return 0;
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index 73edc36..9ed4d25 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -233,7 +233,7 @@ int ehca_query_port(struct ib_device *ibdev,
 		props->phys_state      = 5;
 		props->state           = rblock->state;
 		props->active_width    = IB_WIDTH_12X;
-		props->active_speed    = 0x1;
+		props->active_speed    = IB_SPEED_SDR;
 	}

 query_port1:
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 7b445df..6ff6bdf 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -215,16 +215,16 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 		switch (ext_active_speed) {
 		case 1:
-			props->active_speed = 16; /* FDR */
+			props->active_speed = IB_SPEED_FDR;
 			break;
 		case 2:
-			props->active_speed = 32; /* EDR */
+			props->active_speed = IB_SPEED_EDR;
 			break;
 		}
 	}

 	/* If reported active speed is QDR, check if is FDR-10 */
-	if (props->active_speed == 4) {
+	if (props->active_speed == IB_SPEED_QDR) {
 		if (to_mdev(ibdev)->dev->caps.ext_port_cap[port] &
 		    MLX_EXT_PORT_CAP_FLAG_EXTENDED_PORT_INFO) {
 			init_query_mad(in_mad);
@@ -238,7 +238,7 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,

 			/* Checking LinkSpeedActive for FDR-10 */
 			if (out_mad->data[15] & 0x1)
-				props->active_speed = 8;
+				props->active_speed = IB_SPEED_FDR10;
 		}
 	}

@@ -259,7 +259,7 @@ static int eth_link_query_port(struct ib_device *ibdev, u8 port,
 	enum ib_mtu tmp;

 	props->active_width	= IB_WIDTH_1X;
-	props->active_speed	= 4;
+	props->active_speed	= IB_SPEED_QDR;
 	props->port_cap_flags	= IB_PORT_CM_SUP;
 	props->gid_tbl_len	= to_mdev(ibdev)->dev->caps.gid_table_len[port];
 	props->max_msg_sz	= to_mdev(ibdev)->dev->caps.max_msg_sz;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index 0927b5c..8b8812d 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -597,7 +597,7 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr
 	props->pkey_tbl_len = 1;
 	props->qkey_viol_cntr = 0;
 	props->active_width = IB_WIDTH_4X;
-	props->active_speed = 1;
+	props->active_speed = IB_SPEED_SDR;
 	props->max_msg_sz = 0x80000000;

 	return 0;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index bf5daaf..ff22a73 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -239,6 +239,15 @@ static inline int ib_width_enum_to_int(enum ib_port_width width)
 	}
 }

+enum ib_port_speed {
+	IB_SPEED_SDR	= 1,
+	IB_SPEED_DDR	= 2,
+	IB_SPEED_QDR	= 4,
+	IB_SPEED_FDR10	= 8,
+	IB_SPEED_FDR	= 16,
+	IB_SPEED_EDR	= 32
+};
+
 struct ib_protocol_stats {
 	/* TBD... */
 };
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Or Gerlitz]

commit 71eeba16 "IB: Add new InfiniBand link speeds" introduced a bug
under which the rate for IB SDR/4X links was displayed as 8.5Gbs
instead of 10Gbs, fix that.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/sysfs.c |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index 9ce70ca..0b3edc0 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -213,12 +213,18 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 	}

 	rate *= ib_width_enum_to_int(attr.active_width);
+
 	if (rate < 0)
 		return -EINVAL;

-	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
-		       rate, (attr.active_speed == IB_SPEED_SDR) ? ".5" : "",
-		       ib_width_enum_to_int(attr.active_width), speed);
+	if (attr.active_speed == IB_SPEED_SDR) {
+		rate = (25 * ib_width_enum_to_int(attr.active_width)) / 10;
+		return sprintf(buf, "%d%s Gb/sec (%dX%s)\n", rate,
+			       (attr.active_width == IB_WIDTH_1X) ? ".5" : "",
+				ib_width_enum_to_int(attr.active_width), speed);
+	} else
+		return sprintf(buf, "%d Gb/sec (%dX%s)\n",
+		       rate, ib_width_enum_to_int(attr.active_width), speed);
 }

 static ssize_t phys_state_show(struct ib_port *p, struct port_attribute *unused,
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Hefty, Sean]

> @@ -213,12 +213,18 @@ static ssize_t rate_show(struct ib_port *p, struct
> port_attribute *unused,
>  	}
> 
>  	rate *= ib_width_enum_to_int(attr.active_width);
> +
>  	if (rate < 0)
>  		return -EINVAL;
> 
> -	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
> -		       rate, (attr.active_speed == IB_SPEED_SDR) ? ".5" : "",
> -		       ib_width_enum_to_int(attr.active_width), speed);
> +	if (attr.active_speed == IB_SPEED_SDR) {
> +		rate = (25 * ib_width_enum_to_int(attr.active_width)) / 10;
> +		return sprintf(buf, "%d%s Gb/sec (%dX%s)\n", rate,
> +			       (attr.active_width == IB_WIDTH_1X) ? ".5" : "",
> +				ib_width_enum_to_int(attr.active_width), speed);
> +	} else
> +		return sprintf(buf, "%d Gb/sec (%dX%s)\n",
> +		       rate, ib_width_enum_to_int(attr.active_width), speed);
>  }

This function ends up like this, with my comments inline:

	rate = (25 * attr.active_speed) / 10;
We set rate here, but...

	switch (attr.active_speed) {
	case 2:
		speed = " DDR";
		break;
	case 4:
		speed = " QDR";
		break;
	case 8:
		speed = " FDR10";
		rate = 10;
		break;
	case 16:
		speed = " FDR";
		rate = 14;
		break;
	case 32:
		speed = " EDR";
		rate = 25;
		break;
	}
The above switch statement changes the value for 3 of the 5 cases, with 1 case not covered.

	rate *= ib_width_enum_to_int(attr.active_width);
It is modified again here

	if (rate < 0)
		return -EINVAL;

	if (attr.active_speed == IB_SPEED_SDR) {
		rate = (25 * ib_width_enum_to_int(attr.active_width)) / 10;
And we override it in this case, which is the missing switch case.

		return sprintf(buf, "%d%s Gb/sec (%dX%s)\n", rate,
			       (attr.active_width == IB_WIDTH_1X) ? ".5" : "",
				ib_width_enum_to_int(attr.active_width), speed);
	} else
		return sprintf(buf, "%d Gb/sec (%dX%s)\n",
		       rate, ib_width_enum_to_int(attr.active_width), speed);

How about just setting rate (possibly as a string value) one time in each switch case?

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Or Gerlitz]

On Wed, Jan 11, 2012 at 9:30 PM, Hefty, Sean <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> This function ends up like this, with my comments inline:
>        rate = (25 * attr.active_speed) / 10;
> We set rate here, but [...] The above switch statement
>  changes the value for 3 of the 5 cases, with 1 case not covered.
>        rate *= ib_width_enum_to_int(attr.active_width);
> It is modified again here

Yep, this function is surely not programming state of the art..
however, except for SDR, the above line doing rate = rate *
ib_width_enum_to_int(attr.active_width) would hold for all the speeds
and all the widths (1X, 4X, 12X), if we just set rate (possibly as a
string value) one time in each switch case, we would need to repeat
that multiplication there... I'm okay with that, would you recommend
to go that way? Roland any preference?

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Or Gerlitz]

On 1/11/2012 9:45 PM, Or Gerlitz wrote:
> Yep, this function is surely not programming state of the art.. 
> however, except for SDR, the above line doing rate = rate * 
> ib_width_enum_to_int(attr.active_width) would hold for all the speeds 
> and all the widths (1X, 4X, 12X), if we just set rate (possibly as a 
> string value) one time in each switch case, we would need to repeat 
> that multiplication there... I'm okay with that, would you recommend 
> to go that way? Roland any preference? Or. 

Hi Roland,

Could you please let me know if you want a change in this patch, or this 
fix can go in using the current form for 3.4?

thanks,

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Roland Dreier]

> Could you please let me know if you want a change in this patch, or this fix
> can go in using the current form for 3.4?

It doesn't really work if everything needs me to micromanage every step...
the original patch obviously made some ugly code uglier, Sean pointed out an
obvious way to make things cleaner.

If no one has a problem, I'll just go ahead with the following...

--

    IB/core: Fix SDR rates in sysfs

    Commit 71eeba16 ("IB: Add new InfiniBand link speeds") introduced a bug
    where eg the rate for IB 4X SDR links iss displayed as "8.5 Gb/sec"
    instead of "10 Gb/sec" as it used to be.  Fix that.

    Reported-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
    Signed-off-by: Roland Dreier <roland-BHEL68pLQRGGvPXPguhicg@public.gmane.org>

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index c61bca3..29cd9d5 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -179,33 +179,36 @@ static ssize_t rate_show(struct ib_port *p,
struct port_attribute *unused,
 {
 	struct ib_port_attr attr;
 	char *speed = "";
-	int rate;
+	int rate = -1;		/* in deci-Gb/sec */
 	ssize_t ret;

 	ret = ib_query_port(p->ibdev, p->port_num, &attr);
 	if (ret)
 		return ret;

-	rate = (25 * attr.active_speed) / 10;
-
 	switch (attr.active_speed) {
+	case 1:
+		/* SDR */
+		rate = 25;
 	case 2:
 		speed = " DDR";
+		rate = 50;
 		break;
 	case 4:
 		speed = " QDR";
+		rate = 100;
 		break;
 	case 8:
 		speed = " FDR10";
-		rate = 10;
+		rate = 100;
 		break;
 	case 16:
 		speed = " FDR";
-		rate = 14;
+		rate = 140;
 		break;
 	case 32:
 		speed = " EDR";
-		rate = 25;
+		rate = 250;
 		break;
 	}

@@ -214,7 +217,7 @@ static ssize_t rate_show(struct ib_port *p, struct
port_attribute *unused,
 		return -EINVAL;

 	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
-		       rate, (attr.active_speed == 1) ? ".5" : "",
+		       rate / 10, rate % 10 ? ".5" : "",
 		       ib_width_enum_to_int(attr.active_width), speed);
 }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Or Gerlitz]

On 2/27/2012 7:23 PM, Roland Dreier wrote:
> It doesn't really work if everything needs me to micromanage every 
> step... the original patch obviously made some ugly code uglier, Sean 
> pointed out an obvious way to make things cleaner. If no one has a 
> problem, I'll just go ahead with the following...

okay, got the message, I could go implementing Sean's suggestion myself, 
next time... also on the other direction, when Sean provide ack/review 
on patches, I was thinking they can be accepted without too deep 
inspection which consumes many cycles from you? I'll test this patch, 
provide ack and adjust the 1st patch on the series such that it applies 
on top of this one.

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Or Gerlitz]

On 2/27/2012 7:23 PM, Roland Dreier wrote:

> If no one has a problem, I'll just go ahead with the following...
it has one little bug, see below
>
> --
>
>   	switch (attr.active_speed) {
> +	case 1:
> +		/* SDR */
> +		rate = 25;

missing break here, will send you now fixed version

Or.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] IB/core: fix wrong display of rate in sysfs

[Or Gerlitz]

> If no one has a problem, I'll just go ahead with the following...

Roland, here's your patch fixed to work, this function is something...
you've missed a break in the SDR switch case.. I added it and tested

Or.

--

    IB/core: Fix SDR rates in sysfs

    Commit 71eeba16 ("IB: Add new InfiniBand link speeds") introduced a bug
    where eg the rate for IB 4X SDR links iss displayed as "8.5 Gb/sec"
    instead of "10 Gb/sec" as it used to be.  Fix that.

    Reported-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
    Signed-off-by: Roland Dreier <roland-BHEL68pLQRGGvPXPguhicg@public.gmane.org>

Index: linux-2.6/drivers/infiniband/core/sysfs.c
===================================================================
--- linux-2.6.orig/drivers/infiniband/core/sysfs.c
+++ linux-2.6/drivers/infiniband/core/sysfs.c
@@ -179,33 +179,37 @@ static ssize_t rate_show(struct ib_port
 {
 	struct ib_port_attr attr;
 	char *speed = "";
-	int rate;
+	int rate = -1;		/* in deci-Gb/sec */
 	ssize_t ret;

 	ret = ib_query_port(p->ibdev, p->port_num, &attr);
 	if (ret)
 		return ret;

-	rate = (25 * attr.active_speed) / 10;
-
 	switch (attr.active_speed) {
+	case 1:
+		speed = " SDR";
+		rate = 25;
+		break;
 	case 2:
 		speed = " DDR";
+		rate = 50;
 		break;
 	case 4:
 		speed = " QDR";
+		rate = 100;
 		break;
 	case 8:
 		speed = " FDR10";
-		rate = 10;
+		rate = 100;
 		break;
 	case 16:
 		speed = " FDR";
-		rate = 14;
+		rate = 140;
 		break;
 	case 32:
 		speed = " EDR";
-		rate = 25;
+		rate = 250;
 		break;
 	}

@@ -214,7 +218,7 @@ static ssize_t rate_show(struct ib_port
 		return -EINVAL;

 	return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
-		       rate, (attr.active_speed == 1) ? ".5" : "",
+		       rate / 10, rate % 10 ? ".5" : "",
 		       ib_width_enum_to_int(attr.active_width), speed);
 }


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/6] IB/mlx4: fix wrong info returned when querying IBoE ports

[Or Gerlitz]

To issue port query, use the QUERY_(Ethernet)_PORT command instead of
MAD_IFC command as the latter attempts to query the firmware IB SMA agent,
which is irrelevant for IBoE ports. This allows to support both 10Gb/s and
40Gb/s rates (e.g in sysfs), using QDR speed (10Gb/s) and width of 1X or 4X.

Signed-off-by: Dotan Barak <dotanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx4/main.c |   97 ++++++++++++++++++++-----------------
 1 files changed, 53 insertions(+), 44 deletions(-)

diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index 6ff6bdf..85f6048 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -182,12 +182,27 @@ mlx4_ib_port_link_layer(struct ib_device *device, u8 port_num)
 }

 static int ib_link_query_port(struct ib_device *ibdev, u8 port,
-			      struct ib_port_attr *props,
-			      struct ib_smp *in_mad,
-			      struct ib_smp *out_mad)
+			      struct ib_port_attr *props)
 {
+	struct ib_smp *in_mad  = NULL;
+	struct ib_smp *out_mad = NULL;
 	int ext_active_speed;
-	int err;
+	int err = -ENOMEM;
+
+	in_mad  = kzalloc(sizeof *in_mad, GFP_KERNEL);
+	out_mad = kmalloc(sizeof *out_mad, GFP_KERNEL);
+	if (!in_mad || !out_mad)
+		goto out;
+
+	init_query_mad(in_mad);
+	in_mad->attr_id  = IB_SMP_ATTR_PORT_INFO;
+	in_mad->attr_mod = cpu_to_be32(port);
+
+	err = mlx4_MAD_IFC(to_mdev(ibdev), 1, 1, port, NULL, NULL,
+				in_mad, out_mad);
+	if (err)
+		goto out;
+

 	props->lid		= be16_to_cpup((__be16 *) (out_mad->data + 16));
 	props->lmc		= out_mad->data[34] & 0x7;
@@ -234,15 +249,17 @@ static int ib_link_query_port(struct ib_device *ibdev, u8 port,
 			err = mlx4_MAD_IFC(to_mdev(ibdev), 1, 1, port,
 					   NULL, NULL, in_mad, out_mad);
 			if (err)
-				return err;
+				goto out;

 			/* Checking LinkSpeedActive for FDR-10 */
 			if (out_mad->data[15] & 0x1)
 				props->active_speed = IB_SPEED_FDR10;
 		}
 	}
-
-	return 0;
+out:
+	kfree(in_mad);
+	kfree(out_mad);
+	return err;
 }

 static u8 state_to_phys_state(enum ib_port_state state)
@@ -251,32 +268,42 @@ static u8 state_to_phys_state(enum ib_port_state state)
 }

 static int eth_link_query_port(struct ib_device *ibdev, u8 port,
-			       struct ib_port_attr *props,
-			       struct ib_smp *out_mad)
+			       struct ib_port_attr *props)
 {
-	struct mlx4_ib_iboe *iboe = &to_mdev(ibdev)->iboe;
+
+	struct mlx4_ib_dev *mdev = to_mdev(ibdev);
+	struct mlx4_ib_iboe *iboe = &mdev->iboe;
 	struct net_device *ndev;
 	enum ib_mtu tmp;
+	struct mlx4_cmd_mailbox *mailbox;
+	int err = 0;
+
+	mailbox = mlx4_alloc_cmd_mailbox(mdev->dev);
+	if (IS_ERR(mailbox))
+		return PTR_ERR(mailbox);
+
+	err = mlx4_cmd_box(mdev->dev, 0, mailbox->dma, port, 0,
+			   MLX4_CMD_QUERY_PORT, MLX4_CMD_TIME_CLASS_B,
+			   MLX4_CMD_WRAPPED);
+	if (err)
+		goto out;

-	props->active_width	= IB_WIDTH_1X;
+	props->active_width	=  (((u8 *)mailbox->buf)[5] == 0x40) ?
+						IB_WIDTH_4X : IB_WIDTH_1X;
 	props->active_speed	= IB_SPEED_QDR;
 	props->port_cap_flags	= IB_PORT_CM_SUP;
-	props->gid_tbl_len	= to_mdev(ibdev)->dev->caps.gid_table_len[port];
-	props->max_msg_sz	= to_mdev(ibdev)->dev->caps.max_msg_sz;
+	props->gid_tbl_len	= mdev->dev->caps.gid_table_len[port];
+	props->max_msg_sz	= mdev->dev->caps.max_msg_sz;
 	props->pkey_tbl_len	= 1;
-	props->bad_pkey_cntr	= be16_to_cpup((__be16 *) (out_mad->data + 46));
-	props->qkey_viol_cntr	= be16_to_cpup((__be16 *) (out_mad->data + 48));
 	props->max_mtu		= IB_MTU_4096;
-	props->subnet_timeout	= 0;
-	props->max_vl_num	= out_mad->data[37] >> 4;
-	props->init_type_reply	= 0;
+	props->max_vl_num	= 2;
 	props->state		= IB_PORT_DOWN;
 	props->phys_state	= state_to_phys_state(props->state);
 	props->active_mtu	= IB_MTU_256;
 	spin_lock(&iboe->lock);
 	ndev = iboe->netdevs[port - 1];
 	if (!ndev)
-		goto out;
+		goto out_unlock;

 	tmp = iboe_get_mtu(ndev->mtu);
 	props->active_mtu = tmp ? min(props->max_mtu, tmp) : IB_MTU_256;
@@ -284,41 +311,23 @@ static int eth_link_query_port(struct ib_device *ibdev, u8 port,
 	props->state		= (netif_running(ndev) && netif_carrier_ok(ndev)) ?
 					IB_PORT_ACTIVE : IB_PORT_DOWN;
 	props->phys_state	= state_to_phys_state(props->state);
-
-out:
+out_unlock:
 	spin_unlock(&iboe->lock);
-	return 0;
+out:
+	mlx4_free_cmd_mailbox(mdev->dev, mailbox);
+	return err;
 }

 static int mlx4_ib_query_port(struct ib_device *ibdev, u8 port,
 			      struct ib_port_attr *props)
 {
-	struct ib_smp *in_mad  = NULL;
-	struct ib_smp *out_mad = NULL;
-	int err = -ENOMEM;
-
-	in_mad  = kzalloc(sizeof *in_mad, GFP_KERNEL);
-	out_mad = kmalloc(sizeof *out_mad, GFP_KERNEL);
-	if (!in_mad || !out_mad)
-		goto out;
+	int err;

 	memset(props, 0, sizeof *props);

-	init_query_mad(in_mad);
-	in_mad->attr_id  = IB_SMP_ATTR_PORT_INFO;
-	in_mad->attr_mod = cpu_to_be32(port);
-
-	err = mlx4_MAD_IFC(to_mdev(ibdev), 1, 1, port, NULL, NULL, in_mad, out_mad);
-	if (err)
-		goto out;
-
 	err = mlx4_ib_port_link_layer(ibdev, port) == IB_LINK_LAYER_INFINIBAND ?
-		ib_link_query_port(ibdev, port, props, in_mad, out_mad) :
-		eth_link_query_port(ibdev, port, props, out_mad);
-
-out:
-	kfree(in_mad);
-	kfree(out_mad);
+		ib_link_query_port(ibdev, port, props) :
+				eth_link_query_port(ibdev, port, props);

 	return err;
 }
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/6] net/mlx4: allow for dynamic mtu configuration for IB ports

[Or Gerlitz]

Set the mtu for IB ports by the driver instead of using the firmware
2KB default, with the default being 4KB. Allow for dynamic mtu
configuration through a new, per port sysfs entry.

As there's a dependency between the port mtu to the maximal number
of HW VLs the port can support, apply a mim/max approach, using a
loop that goes down from the highest possible number of VLs to
the lowest. Use the firmware return status as an indication for
the requested number of VLs being impossible with that mtu.

For now, and as done with the dynamic link type change / VPI support,
the sysfs entry to change the mtu is exposed only when running in
NON SRIOV mode. To allow changing the mtu for the master in SRIOV
mode, PF initiated FLR (Function Level Reset) has to be implemented.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

---

Roland, on v1 - http://marc.info/?l=linux-rdma&m=130636143927387&w=2 you made a comment
saying "I mean set the MTU port-by-port with the module loaded, the same way we are
supposed to be able to do for the port type.  Rather than having one global module
parameter", see thread http://marc.info/?l=linux-rdma&m=130632548008337&w=2, this
posting addresses that comment

changes from v1
  - added ib_port_mtu field to mlx4 device ports and use it on SET_PORT
  - removed module param
  - added sysfs entries to set and show the port IB mtu
  - rebased the patch against Linus tree which has the SRIOV patches

Also, some code (enum ibta_mtu and ibta_mtu_to_int) could be eliminated
if we allow for the mlx4_core driver to "#include <rdma/ib_verbs.h>"
basically I don't see a concrete problem arising from such inclusion.

 drivers/net/ethernet/mellanox/mlx4/main.c |  118 ++++++++++++++++++++++++++++-
 drivers/net/ethernet/mellanox/mlx4/mlx4.h |    2 +
 drivers/net/ethernet/mellanox/mlx4/port.c |   25 ++++++-
 include/linux/mlx4/device.h               |    1 +
 4 files changed, 141 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index 6bb62c5..7a58613 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -575,7 +575,6 @@ static ssize_t show_port_type(struct device *dev,

 	return strlen(buf);
 }
-
 static ssize_t set_port_type(struct device *dev,
 			     struct device_attribute *attr,
 			     const char *buf, size_t count)
@@ -648,6 +647,99 @@ out:
 	return err ? err : count;
 }

+enum ibta_mtu {
+	IB_MTU_256  = 1,
+	IB_MTU_512  = 2,
+	IB_MTU_1024 = 3,
+	IB_MTU_2048 = 4,
+	IB_MTU_4096 = 5
+};
+
+static inline int int_to_ibta_mtu(int mtu)
+{
+	switch (mtu) {
+	case 256:  return IB_MTU_256;
+	case 512:  return IB_MTU_512;
+	case 1024: return IB_MTU_1024;
+	case 2048: return IB_MTU_2048;
+	case 4096: return IB_MTU_4096;
+	default: return -1;
+	}
+}
+
+static inline int ibta_mtu_to_int(enum ibta_mtu mtu)
+{
+	switch (mtu) {
+	case IB_MTU_256:  return  256;
+	case IB_MTU_512:  return  512;
+	case IB_MTU_1024: return 1024;
+	case IB_MTU_2048: return 2048;
+	case IB_MTU_4096: return 4096;
+	default: return -1;
+	}
+}
+
+static ssize_t show_port_ib_mtu(struct device *dev,
+			     struct device_attribute *attr,
+			     char *buf)
+{
+	struct mlx4_port_info *info = container_of(attr, struct mlx4_port_info,
+						   port_mtu_attr);
+	struct mlx4_dev *mdev = info->dev;
+
+	if (mdev->caps.port_type[info->port] == MLX4_PORT_TYPE_ETH)
+		mlx4_warn(mdev, "port level mtu is only used for IB ports\n");
+
+	sprintf(buf, "%d\n",
+			ibta_mtu_to_int(mdev->caps.port_ib_mtu[info->port]));
+	return strlen(buf);
+}
+
+static ssize_t set_port_ib_mtu(struct device *dev,
+			     struct device_attribute *attr,
+			     const char *buf, size_t count)
+{
+	struct mlx4_port_info *info = container_of(attr, struct mlx4_port_info,
+						   port_mtu_attr);
+	struct mlx4_dev *mdev = info->dev;
+	struct mlx4_priv *priv = mlx4_priv(mdev);
+	int err, port, mtu, ibta_mtu = -1;
+
+	if (mdev->caps.port_type[info->port] == MLX4_PORT_TYPE_ETH) {
+		mlx4_warn(mdev, "port level mtu is only used for IB ports\n");
+		return -EINVAL;
+	}
+
+	err = sscanf(buf, "%d", &mtu);
+	if (err > 0)
+		ibta_mtu = int_to_ibta_mtu(mtu);
+
+	if (err <= 0 || ibta_mtu < 0) {
+		mlx4_err(mdev, "%s is invalid IBTA mtu\n", buf);
+		return -EINVAL;
+	}
+
+	mdev->caps.port_ib_mtu[info->port] = ibta_mtu;
+
+	mlx4_stop_sense(mdev);
+	mutex_lock(&priv->port_mutex);
+	mlx4_unregister_device(mdev);
+	for (port = 1; port <= mdev->caps.num_ports; port++) {
+		mlx4_CLOSE_PORT(mdev, port);
+		err = mlx4_SET_PORT(mdev, port);
+		if (err) {
+			mlx4_err(mdev, "Failed to set port %d, "
+				      "aborting\n", port);
+			goto err_set_port;
+		}
+	}
+	err = mlx4_register_device(mdev);
+err_set_port:
+	mutex_unlock(&priv->port_mutex);
+	mlx4_start_sense(mdev);
+	return err ? err : count;
+}
+
 static int mlx4_load_fw(struct mlx4_dev *dev)
 {
 	struct mlx4_priv *priv = mlx4_priv(dev);
@@ -1361,7 +1453,10 @@ static int mlx4_setup_hca(struct mlx4_dev *dev)
 					  "ib capabilities (%d). Continuing "
 					  "with caps = 0\n", port, err);
 			dev->caps.ib_port_def_cap[port] = ib_port_default_caps;
-
+			if (mlx4_is_mfunc(dev))
+				dev->caps.port_ib_mtu[port] = IB_MTU_2048;
+			else
+				dev->caps.port_ib_mtu[port] = IB_MTU_4096;
 			err = mlx4_check_ext_port_caps(dev, port);
 			if (err)
 				mlx4_warn(dev, "failed to get port %d extended "
@@ -1523,6 +1618,24 @@ static int mlx4_init_port_info(struct mlx4_dev *dev, int port)
 		info->port = -1;
 	}

+	sprintf(info->dev_mtu_name, "mlx4_port%d_mtu", port);
+	info->port_mtu_attr.attr.name = info->dev_mtu_name;
+	if (mlx4_is_mfunc(dev))
+		info->port_mtu_attr.attr.mode = S_IRUGO;
+	else {
+		info->port_mtu_attr.attr.mode = S_IRUGO | S_IWUSR;
+		info->port_mtu_attr.store     = set_port_ib_mtu;
+	}
+	info->port_mtu_attr.show      = show_port_ib_mtu;
+	sysfs_attr_init(&info->port_mtu_attr.attr);
+
+	err = device_create_file(&dev->pdev->dev, &info->port_mtu_attr);
+	if (err) {
+		mlx4_err(dev, "Failed to create mtu file for port %d\n", port);
+		device_remove_file(&info->dev->pdev->dev, &info->port_attr);
+		info->port = -1;
+	}
+
 	return err;
 }

@@ -1532,6 +1645,7 @@ static void mlx4_cleanup_port_info(struct mlx4_port_info *info)
 		return;

 	device_remove_file(&info->dev->pdev->dev, &info->port_attr);
+	device_remove_file(&info->dev->pdev->dev, &info->port_mtu_attr);
 }

 static int mlx4_init_steering(struct mlx4_dev *dev)
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4.h b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
index a80121a..e83887d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
@@ -680,6 +680,8 @@ struct mlx4_port_info {
 	char			dev_name[16];
 	struct device_attribute port_attr;
 	enum mlx4_port_type	tmp_type;
+	char			dev_mtu_name[16];
+	struct device_attribute port_mtu_attr;
 	struct mlx4_mac_table	mac_table;
 	struct radix_tree_root	mac_tree;
 	struct mlx4_vlan_table	vlan_table;
diff --git a/drivers/net/ethernet/mellanox/mlx4/port.c b/drivers/net/ethernet/mellanox/mlx4/port.c
index 88b52e5..d1ac422 100644
--- a/drivers/net/ethernet/mellanox/mlx4/port.c
+++ b/drivers/net/ethernet/mellanox/mlx4/port.c
@@ -761,10 +761,18 @@ int mlx4_SET_PORT_wrapper(struct mlx4_dev *dev, int slave,
 				    vhcr->op_modifier, inbox);
 }

+/* bit locations for set port command with zero op modifier */
+enum {
+	MLX4_SET_PORT_VL_CAP	 = 4, /* bits 7:4 */
+	MLX4_SET_PORT_MTU_CAP	 = 12, /* bits 15:12 */
+	MLX4_CHANGE_PORT_VL_CAP	 = 21,
+	MLX4_CHANGE_PORT_MTU_CAP = 22,
+};
+
 int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port)
 {
 	struct mlx4_cmd_mailbox *mailbox;
-	int err;
+	int err, vl_cap;

 	if (dev->caps.port_type[port] == MLX4_PORT_TYPE_ETH)
 		return 0;
@@ -776,8 +784,19 @@ int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port)
 	memset(mailbox->buf, 0, 256);

 	((__be32 *) mailbox->buf)[1] = dev->caps.ib_port_def_cap[port];
-	err = mlx4_cmd(dev, mailbox->dma, port, 0, MLX4_CMD_SET_PORT,
-		       MLX4_CMD_TIME_CLASS_B, MLX4_CMD_WRAPPED);
+
+	/* IB VL CAP enum isn't used by the firmware, just numerical values */
+	for (vl_cap = 8; vl_cap >= 1; vl_cap >>= 1) {
+		((__be32 *) mailbox->buf)[0] = cpu_to_be32(
+			(1 << MLX4_CHANGE_PORT_MTU_CAP) |
+			(1 << MLX4_CHANGE_PORT_VL_CAP)  |
+			(dev->caps.port_ib_mtu[port] << MLX4_SET_PORT_MTU_CAP) |
+			(vl_cap << MLX4_SET_PORT_VL_CAP));
+		err = mlx4_cmd(dev, mailbox->dma, port, 0, MLX4_CMD_SET_PORT,
+				MLX4_CMD_TIME_CLASS_B, MLX4_CMD_WRAPPED);
+		if (err != -ENOMEM)
+			break;
+	}

 	mlx4_free_cmd_mailbox(dev, mailbox);
 	return err;
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index 5c4fe8e..f241445 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -309,6 +309,7 @@ struct mlx4_caps {
 	enum mlx4_port_type	possible_type[MLX4_MAX_PORTS + 1];
 	u32			max_counters;
 	u8			ext_port_cap[MLX4_MAX_PORTS + 1];
+	u8			port_ib_mtu[MLX4_MAX_PORTS + 1];
 };

 struct mlx4_buf_list {
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/6] IB: change CQ TCP/IP checksum offload mark to use bit flag indication instead of integer

[Or Gerlitz]

Use a bit in wc_flags rather then a whole integer, this change for itself
doesn't reduce the size of struct ib_wc on 64bit machines, so it stays on 56
bytes as of padding. However, it will allow to add more fields in the future w.o
enlarging that structure. Also, it will serve to have unified approach with
future libibverbs checksum offload reporting towards applications, where
adding a bit flag doesn't break the library ABI. This patch was during
conversation with Liran Liss <liranl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx4/cq.c         |    3 ++-
 drivers/infiniband/hw/mthca/mthca_cq.c  |    3 ++-
 drivers/infiniband/hw/qib/qib_rc.c      |    1 -
 drivers/infiniband/hw/qib/qib_uc.c      |    1 -
 drivers/infiniband/ulp/ipoib/ipoib_ib.c |    3 ++-
 include/rdma/ib_verbs.h                 |    2 +-
 6 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 5ecf38d..09eb1d1 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -720,7 +720,8 @@ repoll:
 		wc->dlid_path_bits = (g_mlpath_rqpn >> 24) & 0x7f;
 		wc->wc_flags	  |= g_mlpath_rqpn & 0x80000000 ? IB_WC_GRH : 0;
 		wc->pkey_index     = be32_to_cpu(cqe->immed_rss_invalid) & 0x7f;
-		wc->csum_ok	   = mlx4_ib_ipoib_csum_ok(cqe->status, cqe->checksum);
+		wc->wc_flags	  |= mlx4_ib_ipoib_csum_ok(cqe->status,
+					cqe->checksum) ? IB_WC_IP_CSUM_OK : 0;
 		if (rdma_port_get_link_layer(wc->qp->device,
 				(*cur_qp)->port) == IB_LINK_LAYER_ETHERNET)
 			wc->sl  = be16_to_cpu(cqe->sl_vid) >> 13;
diff --git a/drivers/infiniband/hw/mthca/mthca_cq.c b/drivers/infiniband/hw/mthca/mthca_cq.c
index 53157b8..40ba833 100644
--- a/drivers/infiniband/hw/mthca/mthca_cq.c
+++ b/drivers/infiniband/hw/mthca/mthca_cq.c
@@ -643,7 +643,8 @@ static inline int mthca_poll_one(struct mthca_dev *dev,
 		entry->wc_flags   |= cqe->g_mlpath & 0x80 ? IB_WC_GRH : 0;
 		checksum = (be32_to_cpu(cqe->rqpn) >> 24) |
 				((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00);
-		entry->csum_ok = (cqe->sl_ipok & 1 && checksum == 0xffff);
+		entry->wc_flags	  |=  (cqe->sl_ipok & 1 && checksum == 0xffff) ?
+							IB_WC_IP_CSUM_OK : 0;
 	}

 	entry->status = IB_WC_SUCCESS;
diff --git a/drivers/infiniband/hw/qib/qib_rc.c b/drivers/infiniband/hw/qib/qib_rc.c
index 894afac..765b4cb 100644
--- a/drivers/infiniband/hw/qib/qib_rc.c
+++ b/drivers/infiniband/hw/qib/qib_rc.c
@@ -2048,7 +2048,6 @@ send_last:
 		wc.pkey_index = 0;
 		wc.dlid_path_bits = 0;
 		wc.port_num = 0;
-		wc.csum_ok = 0;
 		/* Signal completion event if the solicited bit is set. */
 		qib_cq_enter(to_icq(qp->ibqp.recv_cq), &wc,
 			     (ohdr->bth[0] &
diff --git a/drivers/infiniband/hw/qib/qib_uc.c b/drivers/infiniband/hw/qib/qib_uc.c
index 847e7af..7ce2ac2 100644
--- a/drivers/infiniband/hw/qib/qib_uc.c
+++ b/drivers/infiniband/hw/qib/qib_uc.c
@@ -422,7 +422,6 @@ last_imm:
 		wc.pkey_index = 0;
 		wc.dlid_path_bits = 0;
 		wc.port_num = 0;
-		wc.csum_ok = 0;
 		/* Signal completion event if the solicited bit is set. */
 		qib_cq_enter(to_icq(qp->ibqp.recv_cq), &wc,
 			     (ohdr->bth[0] &
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
index 4115be5..5c1bc99 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -296,7 +296,8 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
 	dev->stats.rx_bytes += skb->len;

 	skb->dev = dev;
-	if ((dev->features & NETIF_F_RXCSUM) && likely(wc->csum_ok))
+	if ((dev->features & NETIF_F_RXCSUM) &&
+			likely(wc->wc_flags & IB_WC_IP_CSUM_OK))
 		skb->ip_summed = CHECKSUM_UNNECESSARY;

 	napi_gro_receive(&priv->napi, skb);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 01179b0..31996e3 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -518,6 +518,7 @@ enum ib_wc_flags {
 	IB_WC_GRH		= 1,
 	IB_WC_WITH_IMM		= (1<<1),
 	IB_WC_WITH_INVALIDATE	= (1<<2),
+	IB_WC_IP_CSUM_OK	= (1<<3),
 };

 struct ib_wc {
@@ -538,7 +539,6 @@ struct ib_wc {
 	u8			sl;
 	u8			dlid_path_bits;
 	u8			port_num;	/* valid only for DR SMPs on switches */
-	int			csum_ok;
 };

 enum ib_cq_notify_flags {
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 5/6] IB: change CQ TCP/IP checksum offload mark to use bit flag indication instead of integer

[Hefty, Sean]

> Use a bit in wc_flags rather then a whole integer, this change for itself
> doesn't reduce the size of struct ib_wc on 64bit machines, so it stays on 56
> bytes as of padding. However, it will allow to add more fields in the future
> w.o
> enlarging that structure. Also, it will serve to have unified approach with
> future libibverbs checksum offload reporting towards applications, where
> adding a bit flag doesn't break the library ABI. This patch was during
> conversation with Liran Liss <liranl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Reviewed-by: Sean Hefty <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

> ---
>  drivers/infiniband/hw/mlx4/cq.c         |    3 ++-
>  drivers/infiniband/hw/mthca/mthca_cq.c  |    3 ++-
>  drivers/infiniband/hw/qib/qib_rc.c      |    1 -
>  drivers/infiniband/hw/qib/qib_uc.c      |    1 -
>  drivers/infiniband/ulp/ipoib/ipoib_ib.c |    3 ++-
>  include/rdma/ib_verbs.h                 |    2 +-
>  6 files changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
> index 5ecf38d..09eb1d1 100644
> --- a/drivers/infiniband/hw/mlx4/cq.c
> +++ b/drivers/infiniband/hw/mlx4/cq.c
> @@ -720,7 +720,8 @@ repoll:
>  		wc->dlid_path_bits = (g_mlpath_rqpn >> 24) & 0x7f;
>  		wc->wc_flags	  |= g_mlpath_rqpn & 0x80000000 ? IB_WC_GRH : 0;
>  		wc->pkey_index     = be32_to_cpu(cqe->immed_rss_invalid) & 0x7f;
> -		wc->csum_ok	   = mlx4_ib_ipoib_csum_ok(cqe->status, cqe-
> >checksum);
> +		wc->wc_flags	  |= mlx4_ib_ipoib_csum_ok(cqe->status,
> +					cqe->checksum) ? IB_WC_IP_CSUM_OK : 0;
>  		if (rdma_port_get_link_layer(wc->qp->device,
>  				(*cur_qp)->port) == IB_LINK_LAYER_ETHERNET)
>  			wc->sl  = be16_to_cpu(cqe->sl_vid) >> 13;
> diff --git a/drivers/infiniband/hw/mthca/mthca_cq.c
> b/drivers/infiniband/hw/mthca/mthca_cq.c
> index 53157b8..40ba833 100644
> --- a/drivers/infiniband/hw/mthca/mthca_cq.c
> +++ b/drivers/infiniband/hw/mthca/mthca_cq.c
> @@ -643,7 +643,8 @@ static inline int mthca_poll_one(struct mthca_dev *dev,
>  		entry->wc_flags   |= cqe->g_mlpath & 0x80 ? IB_WC_GRH : 0;
>  		checksum = (be32_to_cpu(cqe->rqpn) >> 24) |
>  				((be32_to_cpu(cqe->my_ee) >> 16) & 0xff00);
> -		entry->csum_ok = (cqe->sl_ipok & 1 && checksum == 0xffff);
> +		entry->wc_flags	  |=  (cqe->sl_ipok & 1 && checksum == 0xffff) ?
> +							IB_WC_IP_CSUM_OK : 0;
>  	}
> 
>  	entry->status = IB_WC_SUCCESS;
> diff --git a/drivers/infiniband/hw/qib/qib_rc.c
> b/drivers/infiniband/hw/qib/qib_rc.c
> index 894afac..765b4cb 100644
> --- a/drivers/infiniband/hw/qib/qib_rc.c
> +++ b/drivers/infiniband/hw/qib/qib_rc.c
> @@ -2048,7 +2048,6 @@ send_last:
>  		wc.pkey_index = 0;
>  		wc.dlid_path_bits = 0;
>  		wc.port_num = 0;
> -		wc.csum_ok = 0;
>  		/* Signal completion event if the solicited bit is set. */
>  		qib_cq_enter(to_icq(qp->ibqp.recv_cq), &wc,
>  			     (ohdr->bth[0] &
> diff --git a/drivers/infiniband/hw/qib/qib_uc.c
> b/drivers/infiniband/hw/qib/qib_uc.c
> index 847e7af..7ce2ac2 100644
> --- a/drivers/infiniband/hw/qib/qib_uc.c
> +++ b/drivers/infiniband/hw/qib/qib_uc.c
> @@ -422,7 +422,6 @@ last_imm:
>  		wc.pkey_index = 0;
>  		wc.dlid_path_bits = 0;
>  		wc.port_num = 0;
> -		wc.csum_ok = 0;
>  		/* Signal completion event if the solicited bit is set. */
>  		qib_cq_enter(to_icq(qp->ibqp.recv_cq), &wc,
>  			     (ohdr->bth[0] &
> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> index 4115be5..5c1bc99 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> @@ -296,7 +296,8 @@ static void ipoib_ib_handle_rx_wc(struct net_device *dev,
> struct ib_wc *wc)
>  	dev->stats.rx_bytes += skb->len;
> 
>  	skb->dev = dev;
> -	if ((dev->features & NETIF_F_RXCSUM) && likely(wc->csum_ok))
> +	if ((dev->features & NETIF_F_RXCSUM) &&
> +			likely(wc->wc_flags & IB_WC_IP_CSUM_OK))
>  		skb->ip_summed = CHECKSUM_UNNECESSARY;
> 
>  	napi_gro_receive(&priv->napi, skb);
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index 01179b0..31996e3 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -518,6 +518,7 @@ enum ib_wc_flags {
>  	IB_WC_GRH		= 1,
>  	IB_WC_WITH_IMM		= (1<<1),
>  	IB_WC_WITH_INVALIDATE	= (1<<2),
> +	IB_WC_IP_CSUM_OK	= (1<<3),
>  };
> 
>  struct ib_wc {
> @@ -538,7 +539,6 @@ struct ib_wc {
>  	u8			sl;
>  	u8			dlid_path_bits;
>  	u8			port_num;	/* valid only for DR SMPs on switches */
> -	int			csum_ok;
>  };
> 
>  enum ib_cq_notify_flags {
> --
> 1.7.1
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/6] IB/ipoib: add support for clones / multiple childs on the same partition

[Or Gerlitz]

Allow creating "clone" child interfaces which further partition an IPoIB
interface to sub interfaces which either use the same pkey as their parent
or use the same pkey as an already created child interface.

Each child now has a child index, which together with the pkey is
used as the identifier of the created network device.

Child interfaces can still be created/deleted using only the pkey,
and they are referred to as legacy childs.

Non-legacy, clone childs of IPoIB device named ibN are named ibN.pkey.index
and those using the same pkey as the parent ibN device are named ibN.index,
where (0x8001 <= pkey <= 0xffff) as before this change, and (0 < index <= 255),
example device names are:

legacy - ib0, ib1, ib0.8001, ib0.8002

non-legacy (clones) - ib0.1, ib0.2, ib0.8001.1, ib0.8001.2

All sorts of childs are still created/deleted through sysfs, in a similar
manner to the way legacy child interfaces are.

A major use case for clone childs is for virtualization purposes, e.g under
schemes where a per VM NIC / HW queue is desired at the hypervisor level.

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---

for the case of non-legacy and same pkey childs I wanted to use
a notation of ibN.pkey:index and ibN:index but this is problematic with
tools (e.g ifconfig) who treat devices with colon in their names as aliases
which are restriced, e.g w.r.t counters, etc, any ideas?

 Documentation/infiniband/ipoib.txt        |   24 ++++++++++++++
 drivers/infiniband/ulp/ipoib/ipoib.h      |    9 ++++-
 drivers/infiniband/ulp/ipoib/ipoib_main.c |   48 +++++++++++++++++++++-------
 drivers/infiniband/ulp/ipoib/ipoib_vlan.c |   40 +++++++++++++++--------
 4 files changed, 93 insertions(+), 28 deletions(-)

diff --git a/Documentation/infiniband/ipoib.txt b/Documentation/infiniband/ipoib.txt
index 64eeb55..2c958f8 100644
--- a/Documentation/infiniband/ipoib.txt
+++ b/Documentation/infiniband/ipoib.txt
@@ -24,6 +24,30 @@ Partitions and P_Keys
   The P_Key for any interface is given by the "pkey" file, and the
   main interface for a subinterface is in "parent."

+Clones
+
+  It is possible to further partition an IPoIB interfaces, and create
+  "clone" child interfaces which either use the same pkey as their
+  parent, or as an already created child interface. Each child now has
+  a child index, which together with the pkey is used as the identifier
+  of the created network device.
+
+ All sorts of childs are still created/deleted through sysfs, in a
+ similar manner to the way conventional child interfaces are, for example:
+
+    echo 0x8001.1 > /sys/class/net/ib0/create_child
+
+  will create an interface named ib0.8001.1 with P_Key 0x8001 and index 1
+
+    echo .1 > /sys/class/net/ib0/create_child
+
+  will create an interface named ib0.1 with same P_Key as ib0 and index 1
+
+  To remove a such subinterface, use the "delete_child" file:
+
+    echo 0x8001.1 > /sys/class/net/ib0/create_child
+    echo .1  > /sys/class/net/ib0/create_child
+
 Datagram vs Connected modes

   The IPoIB driver supports two modes of operation: datagram and
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h
index b3cc1e0..53e021d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -101,6 +101,8 @@ enum {

 	MAX_SEND_CQE		  = 16,
 	IPOIB_CM_COPYBREAK	  = 256,
+
+	MAX_CHILDS_PER_PKEY	  = 255,
 };

 #define	IPOIB_OP_RECV   (1ul << 31)
@@ -330,6 +332,7 @@ struct ipoib_dev_priv {
 	struct net_device *parent;
 	struct list_head child_intfs;
 	struct list_head list;
+	int child_index;

 #ifdef CONFIG_INFINIBAND_IPOIB_CM
 	struct ipoib_cm_dev_priv cm;
@@ -488,8 +491,10 @@ void ipoib_transport_dev_cleanup(struct net_device *dev);
 void ipoib_event(struct ib_event_handler *handler,
 		 struct ib_event *record);

-int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey);
-int ipoib_vlan_delete(struct net_device *pdev, unsigned short pkey);
+int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey,
+						unsigned char clone_index);
+int ipoib_vlan_delete(struct net_device *pdev, unsigned short pkey,
+						unsigned char clone_index);

 void ipoib_pkey_poll(struct work_struct *work);
 int ipoib_pkey_dev_delay_open(struct net_device *dev);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 3514ca0..3a6848d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -1110,17 +1110,44 @@ int ipoib_add_umcast_attr(struct net_device *dev)
 	return device_create_file(&dev->dev, &dev_attr_umcast);
 }

+static int parse_child(struct device *dev, const char *buf, int *pkey,
+		int *child_index)
+{
+	int ret;
+	struct ipoib_dev_priv *priv = netdev_priv(to_net_dev(dev));
+
+	*pkey = *child_index = -1;
+
+	/* 'pkey' or 'pkey.child_index' or '.child_index' are allowed */
+	ret = sscanf(buf, "%i.%i", pkey, child_index);
+	if (ret == 1)  /* just pkey, implicit child index is 0 */
+		*child_index = 0;
+	else  if (ret != 2) { /* pkey same as parent, specified child index */
+		*pkey = priv->pkey;
+		ret  = sscanf(buf, ".%i", child_index);
+		if (ret != 1 || *child_index == 0)
+			return -EINVAL;
+	}
+
+	if (*child_index < 0 || *child_index > MAX_CHILDS_PER_PKEY)
+		return -EINVAL;
+
+	if (*pkey < 0 || *pkey > 0xffff)
+		return -EINVAL;
+
+	ipoib_dbg(priv, "parse_child inp %s out pkey %04x index %d\n",
+		buf, *pkey, *child_index);
+	return 0;
+}
+
 static ssize_t create_child(struct device *dev,
 			    struct device_attribute *attr,
 			    const char *buf, size_t count)
 {
-	int pkey;
+	int pkey, child_index;
 	int ret;

-	if (sscanf(buf, "%i", &pkey) != 1)
-		return -EINVAL;
-
-	if (pkey < 0 || pkey > 0xffff)
+	if (parse_child(dev, buf, &pkey, &child_index))
 		return -EINVAL;

 	/*
@@ -1129,7 +1156,7 @@ static ssize_t create_child(struct device *dev,
 	 */
 	pkey |= 0x8000;

-	ret = ipoib_vlan_add(to_net_dev(dev), pkey);
+	ret = ipoib_vlan_add(to_net_dev(dev), pkey, child_index);

 	return ret ? ret : count;
 }
@@ -1139,16 +1166,13 @@ static ssize_t delete_child(struct device *dev,
 			    struct device_attribute *attr,
 			    const char *buf, size_t count)
 {
-	int pkey;
+	int pkey, child_index;
 	int ret;

-	if (sscanf(buf, "%i", &pkey) != 1)
-		return -EINVAL;
-
-	if (pkey < 0 || pkey > 0xffff)
+	if (parse_child(dev, buf, &pkey, &child_index))
 		return -EINVAL;

-	ret = ipoib_vlan_delete(to_net_dev(dev), pkey);
+	ret = ipoib_vlan_delete(to_net_dev(dev), pkey, child_index);

 	return ret ? ret : count;

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_vlan.c b/drivers/infiniband/ulp/ipoib/ipoib_vlan.c
index d7e9740..3ea888d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_vlan.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_vlan.c
@@ -49,7 +49,8 @@ static ssize_t show_parent(struct device *d, struct device_attribute *attr,
 }
 static DEVICE_ATTR(parent, S_IRUGO, show_parent, NULL);

-int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey)
+int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey,
+		unsigned char child_index)
 {
 	struct ipoib_dev_priv *ppriv, *priv;
 	char intf_name[IFNAMSIZ];
@@ -65,25 +66,34 @@ int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey)
 	mutex_lock(&ppriv->vlan_mutex);

 	/*
-	 * First ensure this isn't a duplicate. We check the parent device and
-	 * then all of the child interfaces to make sure the Pkey doesn't match.
+	 * First ensure this isn't a duplicate. We check all of the child
+	 * interfaces to make sure the Pkey AND the child index
+	 * don't match.
 	 */
-	if (ppriv->pkey == pkey) {
-		result = -ENOTUNIQ;
-		priv = NULL;
-		goto err;
-	}
-
 	list_for_each_entry(priv, &ppriv->child_intfs, list) {
-		if (priv->pkey == pkey) {
+		if (priv->pkey == pkey && priv->child_index == child_index) {
 			result = -ENOTUNIQ;
 			priv = NULL;
 			goto err;
 		}
 	}

-	snprintf(intf_name, sizeof intf_name, "%s.%04x",
-		 ppriv->dev->name, pkey);
+	if (ppriv->pkey != pkey && child_index == 0) /* legacy child */
+		snprintf(intf_name, sizeof intf_name, "%s.%04x",
+			 ppriv->dev->name, pkey);
+	else if (ppriv->pkey != pkey && child_index != 0) /* non-legacy child */
+		snprintf(intf_name, sizeof intf_name, "%s.%04x.%d",
+			 ppriv->dev->name, pkey, child_index);
+	else if (ppriv->pkey == pkey && child_index != 0) /* same pkey child */
+		snprintf(intf_name, sizeof intf_name, "%s.%d",
+			 ppriv->dev->name, child_index);
+	else  {
+		printk(KERN_ERR "wrong pkey/child_index pairing %04x %d\n",
+				pkey, child_index);
+		result = -EINVAL;
+		goto err;
+	}
+
 	priv = ipoib_intf_alloc(intf_name);
 	if (!priv) {
 		result = -ENOMEM;
@@ -101,6 +111,7 @@ int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey)
 		goto err;

 	priv->pkey = pkey;
+	priv->child_index = child_index;

 	memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr, INFINIBAND_ALEN);
 	priv->dev->broadcast[8] = pkey >> 8;
@@ -157,7 +168,8 @@ err:
 	return result;
 }

-int ipoib_vlan_delete(struct net_device *pdev, unsigned short pkey)
+int ipoib_vlan_delete(struct net_device *pdev, unsigned short pkey,
+		unsigned char child_index)
 {
 	struct ipoib_dev_priv *ppriv, *priv, *tpriv;
 	struct net_device *dev = NULL;
@@ -171,7 +183,7 @@ int ipoib_vlan_delete(struct net_device *pdev, unsigned short pkey)
 		return restart_syscall();
 	mutex_lock(&ppriv->vlan_mutex);
 	list_for_each_entry_safe(priv, tpriv, &ppriv->child_intfs, list) {
-		if (priv->pkey == pkey) {
+		if (priv->pkey == pkey && priv->child_index == child_index) {
 			unregister_netdevice(priv->dev);
 			ipoib_dev_cleanup(priv->dev);
 			list_del(&priv->list);
-- 
1.7.1


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] IB/ipoib: eliminate nested locking of the vlan mutex

[Or Gerlitz]

do nested flushing only if the device isn't a child

Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

----

setting CONFIG_DEBUG_MUTEXES I see the below warning, however,
for some reason, I didn't manage to trigger it without my other
patch that adds the clones, I don't see how that patch could
be the reason for the warning, as the code always goes nested,
I've instrumented the flush code to dump its caller/stack and
indeed, you can see that the flushing code is called recursively
and should have that warning, but it doesn't...

ib0.8001: downing ib_dev
ib0: downing ib_dev
ib0: ipoib_ib_dev_flush_light called
ib0: __ipoib_ib_dev_flush pid 29251
Pid: 29251, comm: kworker/u:1 Not tainted 3.2.0-06106-g75f0703-dirty #16
Call Trace:
 [<ffffffffa02b5e1e>] __ipoib_ib_dev_flush+0x57/0x204 [ib_ipoib]
 [<ffffffffa02b6057>] ? ipoib_ib_dev_flush_normal+0x46/0x46 [ib_ipoib]
 [<ffffffffa02b6096>] ipoib_ib_dev_flush_light+0x3f/0x43 [ib_ipoib]
 [<ffffffff81041ee6>] process_one_work+0x2bd/0x4a6
 [<ffffffff81041e39>] ? process_one_work+0x210/0x4a6
 [<ffffffff810424e6>] worker_thread+0x1d6/0x350
 [<ffffffff81042310>] ? rescuer_thread+0x241/0x241
 [<ffffffff81045d5a>] kthread+0x84/0x8c
 [<ffffffff81366ee4>] kernel_thread_helper+0x4/0x10
 [<ffffffff810514d1>] ? finish_task_switch+0x154/0x156
 [<ffffffff8135f243>] ? _raw_spin_unlock_irq+0x2b/0x40
 [<ffffffff8135f59d>] ? retint_restore_args+0xe/0xe
 [<ffffffff81045cd6>] ? __init_kthread_worker+0x56/0x56
 [<ffffffff81366ee0>] ? gs_change+0xb/0xb
ib0.8001: __ipoib_ib_dev_flush pid 29251
Pid: 29251, comm: kworker/u:1 Not tainted 3.2.0-06106-g75f0703-dirty #16
Call Trace:
 [<ffffffffa02b5e1e>] __ipoib_ib_dev_flush+0x57/0x204 [ib_ipoib]
 [<ffffffffa02b5e4e>] __ipoib_ib_dev_flush+0x87/0x204 [ib_ipoib]
 [<ffffffffa02b6057>] ? ipoib_ib_dev_flush_normal+0x46/0x46 [ib_ipoib]
 [<ffffffffa02b6096>] ipoib_ib_dev_flush_light+0x3f/0x43 [ib_ipoib]
 [<ffffffff81041ee6>] process_one_work+0x2bd/0x4a6
 [<ffffffff81041e39>] ? process_one_work+0x210/0x4a6
 [<ffffffff810424e6>] worker_thread+0x1d6/0x350
 [<ffffffff81042310>] ? rescuer_thread+0x241/0x241
 [<ffffffff81045d5a>] kthread+0x84/0x8c
 [<ffffffff81366ee4>] kernel_thread_helper+0x4/0x10
 [<ffffffff810514d1>] ? finish_task_switch+0x154/0x156
 [<ffffffff8135f243>] ? _raw_spin_unlock_irq+0x2b/0x40
 [<ffffffff8135f59d>] ? retint_restore_args+0xe/0xe
 [<ffffffff81045cd6>] ? __init_kthread_worker+0x56/0x56
 [<ffffffff81366ee0>] ? gs_change+0xb/0xb

---

=============================================
[ INFO: possible recursive locking detected ]
3.2.0-06106-g75f0703-dirty #16 Not tainted
---------------------------------------------
kworker/u:2/1578 is trying to acquire lock:
 (&priv->vlan_mutex){+.+.+.}, at: [<ffffffffa021ae9f>] __ipoib_ib_dev_flush+0x2c/0x1cf [ib_ipoib]

but task is already holding lock:
 (&priv->vlan_mutex){+.+.+.}, at: [<ffffffffa021ae9f>] __ipoib_ib_dev_flush+0x2c/0x1cf [ib_ipoib]

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&priv->vlan_mutex);
  lock(&priv->vlan_mutex);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

3 locks held by kworker/u:2/1578:
 #0:  (ipoib){.+.+.+}, at: [<ffffffff81041e39>] process_one_work+0x210/0x4a6
 #1:  ((&priv->flush_heavy)){+.+...}, at: [<ffffffff81041e39>] process_one_work+0x210/0x4a6
 #2:  (&priv->vlan_mutex){+.+.+.}, at: [<ffffffffa021ae9f>] __ipoib_ib_dev_flush+0x2c/0x1cf [ib_ipoib]

stack backtrace:
Pid: 1578, comm: kworker/u:2 Not tainted 3.2.0-06106-g75f0703-dirty #16
Call Trace:
 [<ffffffff81029a02>] ? console_unlock+0x10c/0x207
 [<ffffffff810668a6>] __lock_acquire+0x16b5/0x174e
 [<ffffffff8100ca22>] ? save_stack_trace+0x2a/0x47
 [<ffffffff81066a2f>] lock_acquire+0xf0/0x116
 [<ffffffffa021ae9f>] ? __ipoib_ib_dev_flush+0x2c/0x1cf [ib_ipoib]
 [<ffffffff8135cbb9>] mutex_lock_nested+0x64/0x2e6
 [<ffffffffa021ae9f>] ? __ipoib_ib_dev_flush+0x2c/0x1cf [ib_ipoib]
 [<ffffffff81063bad>] ? trace_hardirqs_on_caller+0x11e/0x155
 [<ffffffffa021ae9f>] __ipoib_ib_dev_flush+0x2c/0x1cf [ib_ipoib]
 [<ffffffffa021aec5>] __ipoib_ib_dev_flush+0x52/0x1cf [ib_ipoib]
 [<ffffffff81063bad>] ? trace_hardirqs_on_caller+0x11e/0x155
 [<ffffffffa021b042>] ? __ipoib_ib_dev_flush+0x1cf/0x1cf [ib_ipoib]
 [<ffffffffa021b057>] ipoib_ib_dev_flush_heavy+0x15/0x17 [ib_ipoib]
 [<ffffffff81041ee6>] process_one_work+0x2bd/0x4a6
 [<ffffffff81041e39>] ? process_one_work+0x210/0x4a6
 [<ffffffff8135f243>] ? _raw_spin_unlock_irq+0x2b/0x40
 [<ffffffff810424e6>] worker_thread+0x1d6/0x350
 [<ffffffff81042310>] ? rescuer_thread+0x241/0x241
 [<ffffffff81045d5a>] kthread+0x84/0x8c
 [<ffffffff81366ee4>] kernel_thread_helper+0x4/0x10
 [<ffffffff8135f59d>] ? retint_restore_args+0xe/0xe
 [<ffffffff81045cd6>] ? __init_kthread_worker+0x56/0x56
 [<ffffffff81366ee0>] ? gs_change+0xb/0xb
ADDRCONF(NETDEV_CHANGE): ib0.8001: link becomes ready
ADDRCONF(NETDEV_CHANGE): ib0: link becomes ready

 drivers/infiniband/ulp/ipoib/ipoib_ib.c |   18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
index 5c1bc99..cac2b71 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -934,16 +934,18 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv,
 	struct net_device *dev = priv->dev;
 	u16 new_index;

-	mutex_lock(&priv->vlan_mutex);
+	if (!priv->parent) {
+		mutex_lock(&priv->vlan_mutex);

-	/*
-	 * Flush any child interfaces too -- they might be up even if
-	 * the parent is down.
-	 */
-	list_for_each_entry(cpriv, &priv->child_intfs, list)
-		__ipoib_ib_dev_flush(cpriv, level);
+		/*
+		 * Flush any child interfaces too -- they might be up even if
+		 * the parent is down.
+		 */
+		list_for_each_entry(cpriv, &priv->child_intfs, list)
+			__ipoib_ib_dev_flush(cpriv, level);

-	mutex_unlock(&priv->vlan_mutex);
+		mutex_unlock(&priv->vlan_mutex);
+	}

 	if (!test_bit(IPOIB_FLAG_INITIALIZED, &priv->flags)) {
 		ipoib_dbg(priv, "Not flushing - IPOIB_FLAG_INITIALIZED not set.\n");

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH for-3.3 0/6] IB: set of fixes and additions

[Or Gerlitz]

On Wed, Jan 11, 2012 at 6:52 PM, Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org> wrote:

> The series is made of four elements

Hi Roland,

Can you provide feedback / comment on these patches? I've got Sean's
okay on patches #1 saying I need one fix, and also I addressed your
comment and posted V2 @
http://marc.info/?l=linux-rdma&m=132639997703455&w=2 also for #5 Sean
provided reviewed-by @
http://marc.info/?l=linux-rdma&m=132630851219457&w=2

Or.



> This subset (patches 1-3) fixes two bugs port info related bugs, where
> the first patch is just making following code related to speed possible...

> [PATCH 1/6] IB: use central enum for speed instead of hard-coded values
> [PATCH 2/6] IB/core: fix wrong display of rate in sysfs
> [PATCH 3/6] IB/mlx4: fix wrong info returned when querying IBoE ports
>
> This patch comes to address the long pending patch for 4k mtu
> along the lines of what was requested over the last review
> [PATCH 4/6] net/mlx4: allow for dynamic mtu configuration for IB ports
>
> This patch is simple and comes to unify the approach for checksum
> indication with possible / hopefully soon changes in libibverbs
>
> [PATCH 5/6] IB: change CQ TCP/IP checksum offload mark to use bit flag indication instead of integer
>
> This patch extends the IPoIB child scheme, and will serve as
> a building block when using IPoIB in the Virtualization space.
>
> [PATCH 6/6] IB/ipoib: add support for clones / multiple childs on the same partition
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH for-3.3 0/6] IB: set of fixes and additions

[Or Gerlitz]

On 1/11/2012 6:52 PM, Or Gerlitz wrote:
> This patch comes to address the long pending patch for 4k mtu
> along the lines of what was requested over the last review
> [PATCH 4/6] net/mlx4: allow for dynamic mtu configuration for IB ports
>
> This patch is simple and comes to unify the approach for checksum
> indication with possible / hopefully soon changes in libibverbs
> [PATCH 5/6] IB: change CQ TCP/IP checksum offload mark to use bit flag indication instead of integer
>
> This patch extends the IPoIB child scheme, and will serve as
> a building block when using IPoIB in the Virtualization space.
> [PATCH 6/6] IB/ipoib: add support for clones / multiple childs on the same partition
>

Hi Roland,

I see that you've picked patches 1-3 to for-next, thanks! as for the 
rest of the patches, well, I submitted this series on Nov 1st, 2011 and 
since then haven't heard from you. So 3.3 is on rc7 now which means we 
don't have much time, for this to be reviewed towards 3.4 - so I'd like 
to get feedback and fix if needed.

Specifically, the 4k mtu thing for mlx4 is pending over this list for 
about two years now, not that if I were you I would accepted V0... but I 
have changed it along the lines of what you asked and now adopted that 
to SR-IOV, so... awaiting your feedback. If you somehow lost track of 
patches 4-6 they are @

4/6 4k mtu http://marc.info/?l=linux-rdma&m=132630178816757&w=2
5/6 csum flag bit http://marc.info/?l=linux-rdma&m=132630196616819&w=2
6/6 ipoib clones http://marc.info/?l=linux-rdma&m=132630202116857&w=2

thanks,


Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH for-3.3 0/6] IB: set of fixes and additions

[Or Gerlitz]

On 3/12/2012 6:29 PM, Or Gerlitz wrote:
>> This patch is simple and comes to unify the approach for checksum
>> indication with possible / hopefully soon changes in libibverbs
>> [PATCH 5/6] IB: change CQ TCP/IP checksum offload mark to use bit 
>> flag indication instead of integer

for #5 Sean provided reviewed-by @ 
http://marc.info/?l=linux-rdma&m=132630851219457&w=2

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html