[PATCH rdma-next 0/3] Add support for active width and speed for RoCE

[PATCH rdma-next 0/3] Add support for active width and speed for RoCE

[Leon Romanovsky]

Hi Doug,

This series from Noa replaces the current hard-coded active speed
and width reported for RoCE in the mlx5 driver.

Two preliminary commits were needed for this change:
1. As a part of proper translation of the ib_port_spped enum,
   the IB_SPEED_HDR enum entry was added.
2. Set mlx5_query_roce_port's return value to void, since in case of
   an error, the properties reported back to the user are zeroed.

It is based on k.o/for-4.12 branch.

Thanks,
  Noa and Leon.

Noa Osherovich (3):
  IB/core: Add HDR speed enum
  IB/mlx5: Set mlx5_query_roce_port's return value to void
  IB/mlx5: Add support for active_width and active_speed in RoCE

 drivers/infiniband/core/sysfs.c   |  4 ++
 drivers/infiniband/hw/mlx5/main.c | 89 +++++++++++++++++++++++++++++++++------
 include/rdma/ib_verbs.h           |  3 +-
 3 files changed, 82 insertions(+), 14 deletions(-)

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 1/3] IB/core: Add HDR speed enum

[Leon Romanovsky]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add high data rate speed to the ib_port_speed enumeration.

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/core/sysfs.c | 4 ++++
 include/rdma/ib_verbs.h         | 3 ++-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index daadf3130c9f..b6317f220f7d 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -253,6 +253,10 @@ static ssize_t rate_show(struct ib_port *p, struct port_attribute *unused,
 		speed = " EDR";
 		rate = 250;
 		break;
+	case IB_SPEED_HDR:
+		speed = " HDR";
+		rate = 500;
+		break;
 	case IB_SPEED_SDR:
 	default:		/* default to SDR for invalid rates */
 		rate = 25;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 3a8e05894e9b..796a38609e1a 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -431,7 +431,8 @@ enum ib_port_speed {
 	IB_SPEED_QDR	= 4,
 	IB_SPEED_FDR10	= 8,
 	IB_SPEED_FDR	= 16,
-	IB_SPEED_EDR	= 32
+	IB_SPEED_EDR	= 32,
+	IB_SPEED_HDR	= 64
 };
 
 /**
-- 
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 2/3] IB/mlx5: Set mlx5_query_roce_port's return value to void

[Leon Romanovsky]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

In case of an error, the properties reported to user
are zeroed out, so no need for a return value.

Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/main.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 4dc0a8785fe0..6ba8bbc5c74a 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -165,15 +165,20 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device,
 	return ndev;
 }
 
-static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
-				struct ib_port_attr *props)
+static void mlx5_query_port_roce(struct ib_device *device, u8 port_num,
+				 struct ib_port_attr *props)
 {
 	struct mlx5_ib_dev *dev = to_mdev(device);
 	struct net_device *ndev, *upper;
 	enum ib_mtu ndev_ib_mtu;
 	u16 qkey_viol_cntr;
 
-	/* props being zeroed by the caller, avoid zeroing it here */
+	/* Getting netdev before filling out props so in case of an error it
+	 * will still be zeroed out.
+	 */
+	ndev = mlx5_ib_get_netdev(device, port_num);
+	if (!ndev)
+		return;
 
 	props->port_cap_flags  |= IB_PORT_CM_SUP;
 	props->port_cap_flags  |= IB_PORT_IP_BASED_GIDS;
@@ -189,10 +194,6 @@ static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
 	mlx5_query_nic_vport_qkey_viol_cntr(dev->mdev, &qkey_viol_cntr);
 	props->qkey_viol_cntr = qkey_viol_cntr;
 
-	ndev = mlx5_ib_get_netdev(device, port_num);
-	if (!ndev)
-		return 0;
-
 	if (mlx5_lag_is_active(dev->mdev)) {
 		rcu_read_lock();
 		upper = netdev_master_upper_dev_get_rcu(ndev);
@@ -217,8 +218,6 @@ static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
 
 	props->active_width	= IB_WIDTH_4X;  /* TODO */
 	props->active_speed	= IB_SPEED_QDR; /* TODO */
-
-	return 0;
 }
 
 static void ib_gid_to_mlx5_roce_addr(const union ib_gid *gid,
@@ -924,7 +923,8 @@ int mlx5_ib_query_port(struct ib_device *ibdev, u8 port,
 		return mlx5_query_hca_port(ibdev, port, props);
 
 	case MLX5_VPORT_ACCESS_METHOD_NIC:
-		return mlx5_query_port_roce(ibdev, port, props);
+		mlx5_query_port_roce(ibdev, port, props);
+		return 0;
 
 	default:
 		return -EINVAL;
-- 
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 3/3] IB/mlx5: Add support for active_width and active_speed in RoCE

[Leon Romanovsky]

From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add missing calculation and translation of active_width and
active_speed for RoCE.

Fixes: 3f89a643eb295 ('IB/mlx5: Extend query_device/port to ...')
Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/main.c | 73 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 68 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 6ba8bbc5c74a..0eaf769a34f5 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -165,21 +165,87 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device,
 	return ndev;
 }
 
+static int translate_eth_proto_oper(u32 eth_proto_oper, u8 *active_speed,
+				    u8 *active_width)
+{
+	switch (eth_proto_oper) {
+	case MLX5E_PROT_MASK(MLX5E_1000BASE_CX_SGMII):
+	case MLX5E_PROT_MASK(MLX5E_1000BASE_KX):
+	case MLX5E_PROT_MASK(MLX5E_100BASE_TX):
+	case MLX5E_PROT_MASK(MLX5E_1000BASE_T):
+		*active_width = IB_WIDTH_1X;
+		*active_speed = IB_SPEED_SDR;
+		break;
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_T):
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_CX4):
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_KX4):
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_KR):
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_CR):
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_SR):
+	case MLX5E_PROT_MASK(MLX5E_10GBASE_ER):
+		*active_width = IB_WIDTH_1X;
+		*active_speed = IB_SPEED_QDR;
+		break;
+	case MLX5E_PROT_MASK(MLX5E_25GBASE_CR):
+	case MLX5E_PROT_MASK(MLX5E_25GBASE_KR):
+	case MLX5E_PROT_MASK(MLX5E_25GBASE_SR):
+		*active_width = IB_WIDTH_1X;
+		*active_speed = IB_SPEED_EDR;
+		break;
+	case MLX5E_PROT_MASK(MLX5E_40GBASE_CR4):
+	case MLX5E_PROT_MASK(MLX5E_40GBASE_KR4):
+	case MLX5E_PROT_MASK(MLX5E_40GBASE_SR4):
+	case MLX5E_PROT_MASK(MLX5E_40GBASE_LR4):
+		*active_width = IB_WIDTH_4X;
+		*active_speed = IB_SPEED_QDR;
+		break;
+	case MLX5E_PROT_MASK(MLX5E_50GBASE_CR2):
+	case MLX5E_PROT_MASK(MLX5E_50GBASE_KR2):
+	case MLX5E_PROT_MASK(MLX5E_50GBASE_SR2):
+		*active_width = IB_WIDTH_1X;
+		*active_speed = IB_SPEED_HDR;
+		break;
+	case MLX5E_PROT_MASK(MLX5E_56GBASE_R4):
+		*active_width = IB_WIDTH_4X;
+		*active_speed = IB_SPEED_FDR;
+		break;
+	case MLX5E_PROT_MASK(MLX5E_100GBASE_CR4):
+	case MLX5E_PROT_MASK(MLX5E_100GBASE_SR4):
+	case MLX5E_PROT_MASK(MLX5E_100GBASE_KR4):
+	case MLX5E_PROT_MASK(MLX5E_100GBASE_LR4):
+		*active_width = IB_WIDTH_4X;
+		*active_speed = IB_SPEED_EDR;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static void mlx5_query_port_roce(struct ib_device *device, u8 port_num,
 				 struct ib_port_attr *props)
 {
 	struct mlx5_ib_dev *dev = to_mdev(device);
+	struct mlx5_core_dev *mdev = dev->mdev;
 	struct net_device *ndev, *upper;
 	enum ib_mtu ndev_ib_mtu;
 	u16 qkey_viol_cntr;
+	u32 eth_prot_oper;
 
-	/* Getting netdev before filling out props so in case of an error it
-	 * will still be zeroed out.
+	/* Possible bad flows are checked before filling out props so in case
+	 * of an error it will still be zeroed out.
 	 */
 	ndev = mlx5_ib_get_netdev(device, port_num);
 	if (!ndev)
 		return;
 
+	if (mlx5_query_port_eth_proto_oper(mdev, &eth_prot_oper, port_num))
+		return;
+
+	translate_eth_proto_oper(eth_prot_oper, &props->active_speed,
+				 &props->active_width);
+
 	props->port_cap_flags  |= IB_PORT_CM_SUP;
 	props->port_cap_flags  |= IB_PORT_IP_BASED_GIDS;
 
@@ -215,9 +281,6 @@ static void mlx5_query_port_roce(struct ib_device *device, u8 port_num,
 	dev_put(ndev);
 
 	props->active_mtu	= min(props->max_mtu, ndev_ib_mtu);
-
-	props->active_width	= IB_WIDTH_4X;  /* TODO */
-	props->active_speed	= IB_SPEED_QDR; /* TODO */
 }
 
 static void ib_gid_to_mlx5_roce_addr(const union ib_gid *gid,
-- 
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/3] IB/mlx5: Set mlx5_query_roce_port's return value to void

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2953 bytes --]

On Thu, Apr 20, 2017 at 08:53:32PM +0300, Leon Romanovsky wrote:
> From: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>
> In case of an error, the properties reported to user
> are zeroed out, so no need for a return value.
>
> Signed-off-by: Noa Osherovich <noaos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---
>  drivers/infiniband/hw/mlx5/main.c | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
> index 4dc0a8785fe0..6ba8bbc5c74a 100644
> --- a/drivers/infiniband/hw/mlx5/main.c
> +++ b/drivers/infiniband/hw/mlx5/main.c
> @@ -165,15 +165,20 @@ static struct net_device *mlx5_ib_get_netdev(struct ib_device *device,
>  	return ndev;
>  }
>
> -static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
> -				struct ib_port_attr *props)
> +static void mlx5_query_port_roce(struct ib_device *device, u8 port_num,
> +				 struct ib_port_attr *props)
>  {
>  	struct mlx5_ib_dev *dev = to_mdev(device);
>  	struct net_device *ndev, *upper;
>  	enum ib_mtu ndev_ib_mtu;
>  	u16 qkey_viol_cntr;
>
> -	/* props being zeroed by the caller, avoid zeroing it here */
> +	/* Getting netdev before filling out props so in case of an error it
> +	 * will still be zeroed out.
> +	 */
> +	ndev = mlx5_ib_get_netdev(device, port_num);
> +	if (!ndev)
> +		return;
>

It breaks ib_write_bw when netdevice is not bounded to ib device. Need
to get rid of this chunk. I'll resend.

>  	props->port_cap_flags  |= IB_PORT_CM_SUP;
>  	props->port_cap_flags  |= IB_PORT_IP_BASED_GIDS;
> @@ -189,10 +194,6 @@ static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
>  	mlx5_query_nic_vport_qkey_viol_cntr(dev->mdev, &qkey_viol_cntr);
>  	props->qkey_viol_cntr = qkey_viol_cntr;
>
> -	ndev = mlx5_ib_get_netdev(device, port_num);
> -	if (!ndev)
> -		return 0;
> -
>  	if (mlx5_lag_is_active(dev->mdev)) {
>  		rcu_read_lock();
>  		upper = netdev_master_upper_dev_get_rcu(ndev);
> @@ -217,8 +218,6 @@ static int mlx5_query_port_roce(struct ib_device *device, u8 port_num,
>
>  	props->active_width	= IB_WIDTH_4X;  /* TODO */
>  	props->active_speed	= IB_SPEED_QDR; /* TODO */
> -
> -	return 0;
>  }
>
>  static void ib_gid_to_mlx5_roce_addr(const union ib_gid *gid,
> @@ -924,7 +923,8 @@ int mlx5_ib_query_port(struct ib_device *ibdev, u8 port,
>  		return mlx5_query_hca_port(ibdev, port, props);
>
>  	case MLX5_VPORT_ACCESS_METHOD_NIC:
> -		return mlx5_query_port_roce(ibdev, port, props);
> +		mlx5_query_port_roce(ibdev, port, props);
> +		return 0;
>
>  	default:
>  		return -EINVAL;
> --
> 2.12.2
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-next 0/3] Add support for active width and speed for RoCE

[Doug Ledford]

On Thu, 2017-04-20 at 20:53 +0300, Leon Romanovsky wrote:
> Hi Doug,
> 
> This series from Noa replaces the current hard-coded active speed
> and width reported for RoCE in the mlx5 driver.
> 
> Two preliminary commits were needed for this change:
> 1. As a part of proper translation of the ib_port_spped enum,
>    the IB_SPEED_HDR enum entry was added.
> 2. Set mlx5_query_roce_port's return value to void, since in case of
>    an error, the properties reported back to the user are zeroed.
> 
> It is based on k.o/for-4.12 branch.
> 
> Thanks,
>   Noa and Leon.
> 
> Noa Osherovich (3):
>   IB/core: Add HDR speed enum
>   IB/mlx5: Set mlx5_query_roce_port's return value to void
>   IB/mlx5: Add support for active_width and active_speed in RoCE
> 
>  drivers/infiniband/core/sysfs.c   |  4 ++
>  drivers/infiniband/hw/mlx5/main.c | 89
> +++++++++++++++++++++++++++++++++------
>  include/rdma/ib_verbs.h           |  3 +-
>  3 files changed, 82 insertions(+), 14 deletions(-)

I fixed up the second patch so it wouldn't break things any more, then
series applied.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html