[PATCH net-next V2 0/2] Mellanox 100G mlx5 minimum inline header mode

[PATCH net-next V2 0/2] Mellanox 100G mlx5 minimum inline header mode

[Saeed Mahameed]

Hi Dave,

This small series from Hadar adds the support for minimum inline header mode query
in mlx5e NIC driver.

Today on TX the driver copies to the HW descriptor only up to L2 header which is the default
required mode and sufficient for today's needs.

The header in the HW descriptor is used for HW loopback steering decision, without it packets 
will go directly to the wire with no questions asked.

For TX loopback steering according to L2/L3/L4 headers, ConnectX-4 requires to copy the
corresponding headers into the send queue(SQ) WQE HW descriptor so it can decide whether to loop it back
or to forward to wire.

For legacy E-Switch mode only L2 headers copy is required.
For advanced steering (E-Switch offloads) more header layers may be required to be copied,
the required mode will be advertised by FW to each VF and PF according to the corresponding
E-Switch configuration.

Changes V2:
 - Allocate query_nic_vport_context_out on the stack

Thanks,
Saeed.

Hadar Hen Zion (2):
  net/mlx5e: Check the minimum inline header mode before xmit
  net/mlx5e: Query minimum required header copy during xmit

 drivers/net/ethernet/mellanox/mlx5/core/en.h      |  8 ++++
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 24 +++++++++++
 drivers/net/ethernet/mellanox/mlx5/core/en_tx.c   | 49 +++++++++++++++++++++--
 drivers/net/ethernet/mellanox/mlx5/core/vport.c   | 12 ++++++
 include/linux/mlx5/device.h                       |  7 ++++
 include/linux/mlx5/mlx5_ifc.h                     | 10 +++--
 include/linux/mlx5/vport.h                        |  2 +
 7 files changed, 105 insertions(+), 7 deletions(-)

-- 
2.8.0

[PATCH net-next V2 1/2] net/mlx5e: Check the minimum inline header mode before xmit

[Saeed Mahameed]

From: Hadar Hen Zion <hadarh@mellanox.com>

Each send queue (SQ) has inline mode that defines the minimal required
inline headers in the SQ WQE.
Before sending each packet check that the minimum required headers
on the WQE are copied.

Signed-off-by: Hadar Hen Zion <hadarh@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h    |  1 +
 drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 49 +++++++++++++++++++++++--
 include/linux/mlx5/device.h                     |  7 ++++
 3 files changed, 53 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 4cbd452..2c20c7b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -398,6 +398,7 @@ struct mlx5e_sq {
 	u32                        sqn;
 	u16                        bf_buf_size;
 	u16                        max_inline;
+	u8                         min_inline_mode;
 	u16                        edge;
 	struct device             *pdev;
 	struct mlx5e_tstamp       *tstamp;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
index 5740b46..e073bf59 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
@@ -128,6 +128,50 @@ u16 mlx5e_select_queue(struct net_device *dev, struct sk_buff *skb,
 	return priv->channeltc_to_txq_map[channel_ix][up];
 }
 
+static inline int mlx5e_skb_l2_header_offset(struct sk_buff *skb)
+{
+#define MLX5E_MIN_INLINE (ETH_HLEN + VLAN_HLEN)
+
+	return max(skb_network_offset(skb), MLX5E_MIN_INLINE);
+}
+
+static inline int mlx5e_skb_l3_header_offset(struct sk_buff *skb)
+{
+	struct flow_keys keys;
+
+	if (skb_transport_header_was_set(skb))
+		return skb_transport_offset(skb);
+	else if (skb_flow_dissect_flow_keys(skb, &keys, 0))
+		return keys.control.thoff;
+	else
+		return mlx5e_skb_l2_header_offset(skb);
+}
+
+static inline unsigned int mlx5e_calc_min_inline(enum mlx5_inline_modes mode,
+						 struct sk_buff *skb)
+{
+	int hlen;
+
+	switch (mode) {
+	case MLX5_INLINE_MODE_TCP_UDP:
+		hlen = eth_get_headlen(skb->data, skb_headlen(skb));
+		if (hlen == ETH_HLEN && !skb_vlan_tag_present(skb))
+			hlen += VLAN_HLEN;
+		return hlen;
+	case MLX5_INLINE_MODE_IP:
+		/* When transport header is set to zero, it means no transport
+		 * header. When transport header is set to 0xff's, it means
+		 * transport header wasn't set.
+		 */
+		if (skb_transport_offset(skb))
+			return mlx5e_skb_l3_header_offset(skb);
+		/* fall through */
+	case MLX5_INLINE_MODE_L2:
+	default:
+		return mlx5e_skb_l2_header_offset(skb);
+	}
+}
+
 static inline u16 mlx5e_get_inline_hdr_size(struct mlx5e_sq *sq,
 					    struct sk_buff *skb, bool bf)
 {
@@ -135,8 +179,6 @@ static inline u16 mlx5e_get_inline_hdr_size(struct mlx5e_sq *sq,
 	 * headers and occur before the data gather.
 	 * Therefore these headers must be copied into the WQE
 	 */
-#define MLX5E_MIN_INLINE (ETH_HLEN + VLAN_HLEN)
-
 	if (bf) {
 		u16 ihs = skb_headlen(skb);
 
@@ -146,8 +188,7 @@ static inline u16 mlx5e_get_inline_hdr_size(struct mlx5e_sq *sq,
 		if (ihs <= sq->max_inline)
 			return skb_headlen(skb);
 	}
-
-	return max(skb_network_offset(skb), MLX5E_MIN_INLINE);
+	return mlx5e_calc_min_inline(sq->min_inline_mode, skb);
 }
 
 static inline void mlx5e_tx_skb_pull_inline(unsigned char **skb_data,
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index e0a3ed7..0b6d15c 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -129,6 +129,13 @@ __mlx5_mask(typ, fld))
 		tmp;							  \
 		})
 
+enum mlx5_inline_modes {
+	MLX5_INLINE_MODE_NONE,
+	MLX5_INLINE_MODE_L2,
+	MLX5_INLINE_MODE_IP,
+	MLX5_INLINE_MODE_TCP_UDP,
+};
+
 enum {
 	MLX5_MAX_COMMANDS		= 32,
 	MLX5_CMD_DATA_BLOCK_SIZE	= 512,
-- 
2.8.0

[PATCH net-next V2 2/2] net/mlx5e: Query minimum required header copy during xmit

[Saeed Mahameed]

From: Hadar Hen Zion <hadarh@mellanox.com>

Add support for query the minimum inline mode from the Firmware.
It is required for correct TX steering according to L3/L4 packet
headers.

Each send queue (SQ) has inline mode that defines the minimal required
headers that needs to be copied into the SQ WQE.
The driver asks the Firmware for the wqe_inline_mode device capability
value.  In case the device capability defined as "vport context" the
driver must check the reported min inline mode from the vport context
before creating its SQs.

Signed-off-by: Hadar Hen Zion <hadarh@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h      |  7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 24 +++++++++++++++++++++++
 drivers/net/ethernet/mellanox/mlx5/core/vport.c   | 12 ++++++++++++
 include/linux/mlx5/mlx5_ifc.h                     | 10 +++++++---
 include/linux/mlx5/vport.h                        |  2 ++
 5 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 2c20c7b..1b495ef 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -129,6 +129,12 @@ static inline int mlx5_max_log_rq_size(int wq_type)
 	}
 }
 
+enum {
+	MLX5E_INLINE_MODE_L2,
+	MLX5E_INLINE_MODE_VPORT_CONTEXT,
+	MLX5_INLINE_MODE_NOT_REQUIRED,
+};
+
 struct mlx5e_tx_wqe {
 	struct mlx5_wqe_ctrl_seg ctrl;
 	struct mlx5_wqe_eth_seg  eth;
@@ -188,6 +194,7 @@ struct mlx5e_params {
 	bool lro_en;
 	u32 lro_wqe_sz;
 	u16 tx_max_inline;
+	u8  tx_min_inline_mode;
 	u8  rss_hfunc;
 	u8  toeplitz_hash_key[40];
 	u32 indirection_rqt[MLX5E_INDIR_RQT_SIZE];
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 611ab55..ca7b1e3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -56,6 +56,7 @@ struct mlx5e_sq_param {
 	u32                        sqc[MLX5_ST_SZ_DW(sqc)];
 	struct mlx5_wq_param       wq;
 	u16                        max_inline;
+	u8                         min_inline_mode;
 	bool                       icosq;
 };
 
@@ -649,6 +650,9 @@ static int mlx5e_create_sq(struct mlx5e_channel *c,
 	}
 	sq->bf_buf_size = (1 << MLX5_CAP_GEN(mdev, log_bf_reg_size)) / 2;
 	sq->max_inline  = param->max_inline;
+	sq->min_inline_mode =
+		MLX5_CAP_ETH(mdev, wqe_inline_mode) == MLX5E_INLINE_MODE_VPORT_CONTEXT ?
+		param->min_inline_mode : 0;
 
 	err = mlx5e_alloc_sq_db(sq, cpu_to_node(c->cpu));
 	if (err)
@@ -731,6 +735,7 @@ static int mlx5e_enable_sq(struct mlx5e_sq *sq, struct mlx5e_sq_param *param)
 
 	MLX5_SET(sqc,  sqc, tis_num_0, param->icosq ? 0 : priv->tisn[sq->tc]);
 	MLX5_SET(sqc,  sqc, cqn,		sq->cq.mcq.cqn);
+	MLX5_SET(sqc,  sqc, min_wqe_inline_mode, sq->min_inline_mode);
 	MLX5_SET(sqc,  sqc, state,		MLX5_SQC_STATE_RST);
 	MLX5_SET(sqc,  sqc, tis_lst_sz,		param->icosq ? 0 : 1);
 	MLX5_SET(sqc,  sqc, flush_in_error_en,	1);
@@ -1343,6 +1348,7 @@ static void mlx5e_build_sq_param(struct mlx5e_priv *priv,
 	MLX5_SET(wq, wq, log_wq_sz,     priv->params.log_sq_size);
 
 	param->max_inline = priv->params.tx_max_inline;
+	param->min_inline_mode = priv->params.tx_min_inline_mode;
 }
 
 static void mlx5e_build_common_cq_param(struct mlx5e_priv *priv,
@@ -2967,6 +2973,23 @@ void mlx5e_set_rx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode)
 			MLX5E_PARAMS_DEFAULT_RX_CQ_MODERATION_USEC_FROM_CQE;
 }
 
+static void mlx5e_query_min_inline(struct mlx5_core_dev *mdev,
+				   u8 *min_inline_mode)
+{
+	switch (MLX5_CAP_ETH(mdev, wqe_inline_mode)) {
+	case MLX5E_INLINE_MODE_L2:
+		*min_inline_mode = MLX5_INLINE_MODE_L2;
+		break;
+	case MLX5E_INLINE_MODE_VPORT_CONTEXT:
+		mlx5_query_nic_vport_min_inline(mdev,
+						min_inline_mode);
+		break;
+	case MLX5_INLINE_MODE_NOT_REQUIRED:
+		*min_inline_mode = MLX5_INLINE_MODE_NONE;
+		break;
+	}
+}
+
 static void mlx5e_build_nic_netdev_priv(struct mlx5_core_dev *mdev,
 					struct net_device *netdev,
 					const struct mlx5e_profile *profile,
@@ -3032,6 +3055,7 @@ static void mlx5e_build_nic_netdev_priv(struct mlx5_core_dev *mdev,
 	priv->params.tx_cq_moderation.pkts =
 		MLX5E_PARAMS_DEFAULT_TX_CQ_MODERATION_PKTS;
 	priv->params.tx_max_inline         = mlx5e_get_max_inline_cap(mdev);
+	mlx5e_query_min_inline(mdev, &priv->params.tx_min_inline_mode);
 	priv->params.num_tc                = 1;
 	priv->params.rss_hfunc             = ETH_RSS_HASH_XOR;
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c
index 91846df..21365d0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c
@@ -135,6 +135,18 @@ static int mlx5_modify_nic_vport_context(struct mlx5_core_dev *mdev, void *in,
 	return mlx5_cmd_exec_check_status(mdev, in, inlen, out, sizeof(out));
 }
 
+void mlx5_query_nic_vport_min_inline(struct mlx5_core_dev *mdev,
+				     u8 *min_inline_mode)
+{
+	u32 out[MLX5_ST_SZ_DW(query_nic_vport_context_out)] = {0};
+
+	mlx5_query_nic_vport_context(mdev, 0, out, sizeof(out));
+
+	*min_inline_mode = MLX5_GET(query_nic_vport_context_out, out,
+				    nic_vport_context.min_wqe_inline_mode);
+}
+EXPORT_SYMBOL_GPL(mlx5_query_nic_vport_min_inline);
+
 int mlx5_query_nic_vport_mac_address(struct mlx5_core_dev *mdev,
 				     u16 vport, u8 *addr)
 {
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index d671e4e..21bc455 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -536,7 +536,8 @@ struct mlx5_ifc_per_protocol_networking_offload_caps_bits {
 	u8         self_lb_en_modifiable[0x1];
 	u8         reserved_at_9[0x2];
 	u8         max_lso_cap[0x5];
-	u8         reserved_at_10[0x4];
+	u8         reserved_at_10[0x2];
+	u8	   wqe_inline_mode[0x2];
 	u8         rss_ind_tbl_cap[0x4];
 	u8         reg_umr_sq[0x1];
 	u8         scatter_fcs[0x1];
@@ -2270,7 +2271,8 @@ struct mlx5_ifc_sqc_bits {
 	u8         cd_master[0x1];
 	u8         fre[0x1];
 	u8         flush_in_error_en[0x1];
-	u8         reserved_at_4[0x4];
+	u8         reserved_at_4[0x1];
+	u8	   min_wqe_inline_mode[0x3];
 	u8         state[0x4];
 	u8         reg_umr[0x1];
 	u8         reserved_at_d[0x13];
@@ -2367,7 +2369,9 @@ struct mlx5_ifc_rmpc_bits {
 };
 
 struct mlx5_ifc_nic_vport_context_bits {
-	u8         reserved_at_0[0x1f];
+	u8         reserved_at_0[0x5];
+	u8         min_wqe_inline_mode[0x3];
+	u8         reserved_at_8[0x17];
 	u8         roce_en[0x1];
 
 	u8         arm_change_event[0x1];
diff --git a/include/linux/mlx5/vport.h b/include/linux/mlx5/vport.h
index 6c16c19..e087b7d 100644
--- a/include/linux/mlx5/vport.h
+++ b/include/linux/mlx5/vport.h
@@ -43,6 +43,8 @@ int mlx5_modify_vport_admin_state(struct mlx5_core_dev *mdev, u8 opmod,
 				  u16 vport, u8 state);
 int mlx5_query_nic_vport_mac_address(struct mlx5_core_dev *mdev,
 				     u16 vport, u8 *addr);
+void mlx5_query_nic_vport_min_inline(struct mlx5_core_dev *mdev,
+				     u8 *min_inline);
 int mlx5_modify_nic_vport_mac_address(struct mlx5_core_dev *dev,
 				      u16 vport, u8 *addr);
 int mlx5_query_nic_vport_mtu(struct mlx5_core_dev *mdev, u16 *mtu);
-- 
2.8.0

Re: [PATCH net-next V2 0/2] Mellanox 100G mlx5 minimum inline header mode

[David Miller]

From: Saeed Mahameed <saeedm@mellanox.com>
Date: Sun, 24 Jul 2016 16:12:38 +0300

> This small series from Hadar adds the support for minimum inline
> header mode query in mlx5e NIC driver.
> 
> Today on TX the driver copies to the HW descriptor only up to L2
> header which is the default required mode and sufficient for today's
> needs.
> 
> The header in the HW descriptor is used for HW loopback steering
> decision, without it packets will go directly to the wire with no
> questions asked.
> 
> For TX loopback steering according to L2/L3/L4 headers, ConnectX-4
> requires to copy the corresponding headers into the send queue(SQ)
> WQE HW descriptor so it can decide whether to loop it back or to
> forward to wire.
> 
> For legacy E-Switch mode only L2 headers copy is required.  For
> advanced steering (E-Switch offloads) more header layers may be
> required to be copied, the required mode will be advertised by FW to
> each VF and PF according to the corresponding E-Switch
> configuration.
> 
> Changes V2:
>  - Allocate query_nic_vport_context_out on the stack

Applied, but even doing an eth_get_headlen() every transmitted packet
it really too expensive.

You shouldn't be touching networking headers so much when forwarding
frames.

Re: [PATCH net-next V2 0/2] Mellanox 100G mlx5 minimum inline header mode

[Hadar Hen Zion]

In the default case eth_get_headlen() won't be called, it will happen
only if PF administrator changes the mode from default to L4.

In L4 mode, we need to copy all the packet headers including L4, do
you know of a better/cheaper way for doing that?

Thanks,
Hadar

On Tue, Jul 26, 2016 at 3:54 AM, David Miller <davem@davemloft.net> wrote:
> From: Saeed Mahameed <saeedm@mellanox.com>
> Date: Sun, 24 Jul 2016 16:12:38 +0300
>
>> This small series from Hadar adds the support for minimum inline
>> header mode query in mlx5e NIC driver.
>>
>> Today on TX the driver copies to the HW descriptor only up to L2
>> header which is the default required mode and sufficient for today's
>> needs.
>>
>> The header in the HW descriptor is used for HW loopback steering
>> decision, without it packets will go directly to the wire with no
>> questions asked.
>>
>> For TX loopback steering according to L2/L3/L4 headers, ConnectX-4
>> requires to copy the corresponding headers into the send queue(SQ)
>> WQE HW descriptor so it can decide whether to loop it back or to
>> forward to wire.
>>
>> For legacy E-Switch mode only L2 headers copy is required.  For
>> advanced steering (E-Switch offloads) more header layers may be
>> required to be copied, the required mode will be advertised by FW to
>> each VF and PF according to the corresponding E-Switch
>> configuration.
>>
>> Changes V2:
>>  - Allocate query_nic_vport_context_out on the stack
>
> Applied, but even doing an eth_get_headlen() every transmitted packet
> it really too expensive.
>
> You shouldn't be touching networking headers so much when forwarding
> frames.

Re: [PATCH net-next V2 0/2] Mellanox 100G mlx5 minimum inline header mode

[Hadar Hen Zion]

On Tue, Jul 26, 2016 at 3:54 AM, David Miller <davem@davemloft.net> wrote:
> From: Saeed Mahameed <saeedm@mellanox.com>
> Date: Sun, 24 Jul 2016 16:12:38 +0300
>
>> This small series from Hadar adds the support for minimum inline
>> header mode query in mlx5e NIC driver.
>>
>> Today on TX the driver copies to the HW descriptor only up to L2
>> header which is the default required mode and sufficient for today's
>> needs.
>>
>> The header in the HW descriptor is used for HW loopback steering
>> decision, without it packets will go directly to the wire with no
>> questions asked.
>>
>> For TX loopback steering according to L2/L3/L4 headers, ConnectX-4
>> requires to copy the corresponding headers into the send queue(SQ)
>> WQE HW descriptor so it can decide whether to loop it back or to
>> forward to wire.
>>
>> For legacy E-Switch mode only L2 headers copy is required.  For
>> advanced steering (E-Switch offloads) more header layers may be
>> required to be copied, the required mode will be advertised by FW to
>> each VF and PF according to the corresponding E-Switch
>> configuration.
>>
>> Changes V2:
>>  - Allocate query_nic_vport_context_out on the stack
>
> Applied, but even doing an eth_get_headlen() every transmitted packet
> it really too expensive.
>
> You shouldn't be touching networking headers so much when forwarding
> frames.

Sorry for re-sending, i had a problem before.

In the default case eth_get_headlen() won't be called, it will happen
only if PF administrator changes the mode from default to L4.

In L4 mode, we need to copy all the packet headers including L4, do
you know of a better/cheaper way for doing that?

Thanks,
Hadar

Re: [PATCH net-next V2 0/2] Mellanox 100G mlx5 minimum inline header mode

[David Miller]

From: Hadar Hen Zion <hadarh@dev.mellanox.co.il>
Date: Tue, 26 Jul 2016 13:30:46 +0300

> In the default case eth_get_headlen() won't be called, it will happen
> only if PF administrator changes the mode from default to L4.
> 
> In L4 mode, we need to copy all the packet headers including L4, do
> you know of a better/cheaper way for doing that?

You can just look straight at the ethernet header in this context.

eth_get_headlen() is expensive and has a complete header parser.  It
is meant to be used when none of the SKB context state has been setup
yet (early RX processing before SKB is allocated, for example).