[PATCH net 0/4] Batch of mlx4 fixes for 3.15

[PATCH net 0/4] Batch of mlx4 fixes for 3.15

[Or Gerlitz]

Hi Dave, 

This series contains fixes for 3.15-rc, mostly around SRIOV. The patches by Jack, 
Matan and myself 
fix few issues related to mlx4 SRIOV support for RoCE and single port VFs, and the
patch from Eyal elimninates checking PCI caps for VFs which is misleading.

Patches done against the net tree, commit 014f1b2 "net: bonding: Fix format string 
mismatch in bond_sysfs.c"

We'd be happy to get Eyal's patch queued in your -stable list for 3.14.y

Or.

Eyal Perry (1):
  net/mlx4_core: Don't issue PCIe speed/width checks for VFs

Jack Morgenstein (1):
  net/mlx4_core: Adjust port number in qp_attach wrapper when detaching

Matan Barak (1):
  net/mlx4_core: Fix slave id computation for single port VF

Or Gerlitz (1):
  net/mlx4_core: Load the Eth driver first

 drivers/net/ethernet/mellanox/mlx4/main.c          |    7 ++--
 drivers/net/ethernet/mellanox/mlx4/port.c          |   35 +++++++++++--------
 .../net/ethernet/mellanox/mlx4/resource_tracker.c  |   23 +++++++++++++
 3 files changed, 47 insertions(+), 18 deletions(-)

[PATCH net 1/4] net/mlx4_core: Adjust port number in qp_attach wrapper when detaching

[Or Gerlitz]

From: Jack Morgenstein <jackm@dev.mellanox.co.il>

When using single ported VFs and the VF is using port 2, we need
to adjust the port accordingly (change it from 1 to 2).

Fixes: 449fc48 ('net/mlx4: Adapt code for N-Port VF')
Signed-off-by: Matan Barak <matanb@mellanox.com>
Signed-off-by: Jack Morgenstein <jackm@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 .../net/ethernet/mellanox/mlx4/resource_tracker.c  |   23 ++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
index 3b5f53e..3f27cba 100644
--- a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
+++ b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
@@ -3733,6 +3733,25 @@ static int qp_detach(struct mlx4_dev *dev, struct mlx4_qp *qp,
 	}
 }
 
+static int mlx4_adjust_port(struct mlx4_dev *dev, int slave,
+			    u8 *gid, enum mlx4_protocol prot)
+{
+	int real_port;
+
+	if (prot != MLX4_PROT_ETH)
+		return 0;
+
+	if (dev->caps.steering_mode == MLX4_STEERING_MODE_B0 ||
+	    (dev->caps.steering_mode == MLX4_STEERING_MODE_DEVICE_MANAGED)) {
+		real_port = mlx4_slave_convert_port(dev, slave, gid[5]);
+		if (real_port < 0)
+			return -EINVAL;
+		gid[5] = real_port;
+	}
+
+	return 0;
+}
+
 int mlx4_QP_ATTACH_wrapper(struct mlx4_dev *dev, int slave,
 			       struct mlx4_vhcr *vhcr,
 			       struct mlx4_cmd_mailbox *inbox,
@@ -3768,6 +3787,10 @@ int mlx4_QP_ATTACH_wrapper(struct mlx4_dev *dev, int slave,
 		if (err)
 			goto ex_detach;
 	} else {
+		err = mlx4_adjust_port(dev, slave, gid, prot);
+		if (err)
+			goto ex_put;
+
 		err = rem_mcg_res(dev, slave, rqp, gid, prot, type, &reg_id);
 		if (err)
 			goto ex_put;
-- 
1.7.1

Re: [PATCH net 1/4] net/mlx4_core: Adjust port number in qp_attach wrapper when detaching

[Sergei Shtylyov]

Hello.

On 05/01/2014 04:51 PM, Or Gerlitz wrote:

> From: Jack Morgenstein <jackm@dev.mellanox.co.il>

> When using single ported VFs and the VF is using port 2, we need
> to adjust the port accordingly (change it from 1 to 2).

> Fixes: 449fc48 ('net/mlx4: Adapt code for N-Port VF')
> Signed-off-by: Matan Barak <matanb@mellanox.com>
> Signed-off-by: Jack Morgenstein <jackm@mellanox.com>
> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
> ---
>   .../net/ethernet/mellanox/mlx4/resource_tracker.c  |   23 ++++++++++++++++++++
>   1 files changed, 23 insertions(+), 0 deletions(-)

> diff --git a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
> index 3b5f53e..3f27cba 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
> @@ -3733,6 +3733,25 @@ static int qp_detach(struct mlx4_dev *dev, struct mlx4_qp *qp,
>   	}
>   }
>
> +static int mlx4_adjust_port(struct mlx4_dev *dev, int slave,
> +			    u8 *gid, enum mlx4_protocol prot)
> +{
> +	int real_port;
> +
> +	if (prot != MLX4_PROT_ETH)
> +		return 0;
> +
> +	if (dev->caps.steering_mode == MLX4_STEERING_MODE_B0 ||
> +	    (dev->caps.steering_mode == MLX4_STEERING_MODE_DEVICE_MANAGED)) {

    It looks a bit strange when you enclose in parens the second == but not 
the first. In fact, they're not at all necessary.

WBR, Sergei

Re: [PATCH net 1/4] net/mlx4_core: Adjust port number in qp_attach wrapper when detaching

[Or Gerlitz]

On Thu, May 1, 2014 at 10:54 PM, Sergei Shtylyov
<sergei.shtylyov@cogentembedded.com> wrote:
> On 05/01/2014 04:51 PM, Or Gerlitz wrote:

>> +static int mlx4_adjust_port(struct mlx4_dev *dev, int slave,
>> +                           u8 *gid, enum mlx4_protocol prot)
>> +{
>> +       int real_port;
>> +
>> +       if (prot != MLX4_PROT_ETH)
>> +               return 0;
>> +
>> +       if (dev->caps.steering_mode == MLX4_STEERING_MODE_B0 ||
>> +           (dev->caps.steering_mode == MLX4_STEERING_MODE_DEVICE_MANAGED)) {
>
>
>    It looks a bit strange when you enclose in parens the second == but not the first. In fact, they're not at all necessary.

Well... so we'll fix it and resubmit

[PATCH net 2/4] net/mlx4_core: Fix slave id computation for single port VF

[Or Gerlitz]

From: Matan Barak <matanb@mellanox.com>

The code that deals with computing the slave id based on a given GID
gave wrong results when the number of single port VFs wasn't the
same for port 1 vs. port 2 and the relevant VF is single ported on
port 2. As a result, incoming CM MADs were dispatched to the wrong VF.
Fixed that and added documentation to clarify the computation steps.

Fixes: 449fc48 ('net/mlx4: Adapt code for N-Port VF')
Signed-off-by: Matan Barak <matanb@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/port.c |   35 ++++++++++++++++------------
 1 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/port.c b/drivers/net/ethernet/mellanox/mlx4/port.c
index cfcad26..b5b3549 100644
--- a/drivers/net/ethernet/mellanox/mlx4/port.c
+++ b/drivers/net/ethernet/mellanox/mlx4/port.c
@@ -1106,6 +1106,9 @@ int mlx4_get_slave_from_roce_gid(struct mlx4_dev *dev, int port, u8 *gid,
 	}
 
 	if (found_ix >= 0) {
+		/* Calculate a slave_gid which is the slave number in the gid
+		 * table and not a globally unique slave number.
+		 */
 		if (found_ix < MLX4_ROCE_PF_GIDS)
 			slave_gid = 0;
 		else if (found_ix < MLX4_ROCE_PF_GIDS + (vf_gids % num_vfs) *
@@ -1118,41 +1121,43 @@ int mlx4_get_slave_from_roce_gid(struct mlx4_dev *dev, int port, u8 *gid,
 			  ((vf_gids % num_vfs) * ((vf_gids / num_vfs + 1)))) /
 			 (vf_gids / num_vfs)) + vf_gids % num_vfs + 1;
 
+		/* Calculate the globally unique slave id */
 		if (slave_gid) {
 			struct mlx4_active_ports exclusive_ports;
 			struct mlx4_active_ports actv_ports;
 			struct mlx4_slaves_pport slaves_pport_actv;
 			unsigned max_port_p_one;
-			int num_slaves_before = 1;
+			int num_vfs_before = 0;
+			int candidate_slave_gid;
 
+			/* Calculate how many VFs are on the previous port, if exists */
 			for (i = 1; i < port; i++) {
 				bitmap_zero(exclusive_ports.ports, dev->caps.num_ports);
-				set_bit(i, exclusive_ports.ports);
+				set_bit(i - 1, exclusive_ports.ports);
 				slaves_pport_actv =
 					mlx4_phys_to_slaves_pport_actv(
 							dev, &exclusive_ports);
-				num_slaves_before += bitmap_weight(
+				num_vfs_before += bitmap_weight(
 						slaves_pport_actv.slaves,
 						dev->num_vfs + 1);
 			}
 
-			if (slave_gid < num_slaves_before) {
-				bitmap_zero(exclusive_ports.ports, dev->caps.num_ports);
-				set_bit(port - 1, exclusive_ports.ports);
-				slaves_pport_actv =
-					mlx4_phys_to_slaves_pport_actv(
-							dev, &exclusive_ports);
-				slave_gid += bitmap_weight(
-						slaves_pport_actv.slaves,
-						dev->num_vfs + 1) -
-						num_slaves_before;
-			}
-			actv_ports = mlx4_get_active_ports(dev, slave_gid);
+			/* candidate_slave_gid isn't necessarily the correct slave, but
+			 * it has the same number of ports and is assigned to the same
+			 * ports as the real slave we're looking for. On dual port VF,
+			 * slave_gid = [single port VFs on port <port>] +
+			 * [offset of the current slave from the first dual port VF] +
+			 * 1 (for the PF).
+			 */
+			candidate_slave_gid = slave_gid + num_vfs_before;
+
+			actv_ports = mlx4_get_active_ports(dev, candidate_slave_gid);
 			max_port_p_one = find_first_bit(
 				actv_ports.ports, dev->caps.num_ports) +
 				bitmap_weight(actv_ports.ports,
 					      dev->caps.num_ports) + 1;
 
+			/* Calculate the real slave number */
 			for (i = 1; i < max_port_p_one; i++) {
 				if (i == port)
 					continue;
-- 
1.7.1

[PATCH net 3/4] net/mlx4_core: Load the Eth driver first

[Or Gerlitz]

When running in SRIOV mode, VM that is assigned with a non-provisioned
Ethernet VFs get themselves a random mac when the Eth driver starts. In
this case, if the IB driver startup code that deals with RoCE runs first,
it will use a zero mac as the source mac for the Para-Virtual CM MADs
which is buggy. To handle that, we change the order of loading.

Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/main.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index cef267e..e98c15a 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -754,10 +754,10 @@ static void mlx4_request_modules(struct mlx4_dev *dev)
 			has_eth_port = true;
 	}
 
-	if (has_ib_port || (dev->caps.flags & MLX4_DEV_CAP_FLAG_IBOE))
-		request_module_nowait(IB_DRV_NAME);
 	if (has_eth_port)
 		request_module_nowait(EN_DRV_NAME);
+	if (has_ib_port || (dev->caps.flags & MLX4_DEV_CAP_FLAG_IBOE))
+		request_module_nowait(IB_DRV_NAME);
 }
 
 /*
-- 
1.7.1

[PATCH net 4/4] net/mlx4_core: Don't issue PCIe speed/width checks for VFs

[Or Gerlitz]

From: Eyal Perry <eyalpe@mellanox.com>

Carrying out PCI speed/width checks through pcie_get_minimum_link()
on VFs yield wrong results, so remove them.

Fixes: b912b2f ('net/mlx4_core: Warn if device doesn't have enough PCI bandwidth')
Signed-off-by: Eyal Perry <eyalpe@mellanox.com>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/main.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index e98c15a..7cf9dad 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -2440,7 +2440,8 @@ slave_start:
 	 * No return code for this call, just warn the user in case of PCI
 	 * express device capabilities are under-satisfied by the bus.
 	 */
-	mlx4_check_pcie_caps(dev);
+	if (!mlx4_is_slave(dev))
+		mlx4_check_pcie_caps(dev);
 
 	/* In master functions, the communication channel must be initialized
 	 * after obtaining its address from fw */
-- 
1.7.1