[pull request][net 0/4] Mellanox, mlx5 fixes 2019-01-18

[pull request][net 0/4] Mellanox, mlx5 fixes 2019-01-18

[Saeed Mahameed]

Hi Dave,

This series introduces some fixes to mlx5 driver.

Please pull and let me know if there is any problem.

For -stable v4.18
('net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames')

The patch doesn't apply cleanly to 4.18.y, but it is very simple to
resolve, what should be the procedure here ?


Thanks,
Saeed.

---
The following changes since commit bfeffd155283772bbe78c6a05dec7c0128ee500c:

  Linux 5.0-rc1 (2019-01-06 17:08:20 -0800)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-fixes-2019-01-18

for you to fetch changes up to 25f2d0e7791e71cc89c58a7c2231768ad200764b:

  net/mlx5e: Fix cb_ident duplicate in indirect block register (2019-01-18 16:15:31 -0800)

----------------------------------------------------------------
mlx5-fixes-2019-01-18

----------------------------------------------------------------
Cong Wang (1):
      net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames

Eli Britstein (1):
      net/mlx5e: Fix cb_ident duplicate in indirect block register

Shay Agroskin (1):
      net/mlx5e: Fix wrong error code return on FEC query failure

Tariq Toukan (1):
      net/mlx5e: Fix wrong (zero) TX drop counter indication for representor

 .../net/ethernet/mellanox/mlx5/core/en_ethtool.c   |  5 +++-
 drivers/net/ethernet/mellanox/mlx5/core/en_rep.c   | 30 ++++++++++++----------
 drivers/net/ethernet/mellanox/mlx5/core/en_rx.c    | 13 ++++++++++
 3 files changed, 34 insertions(+), 14 deletions(-)

[net 1/4] net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames

[Saeed Mahameed]

From: Cong Wang <xiyou.wangcong@gmail.com>

When an ethernet frame is padded to meet the minimum ethernet frame
size, the padding octets are not covered by the hardware checksum.
Fortunately the padding octets are usually zero's, which don't affect
checksum. However, we have a switch which pads non-zero octets, this
causes kernel hardware checksum fault repeatedly.

Prior to:
commit '88078d98d1bb ("net: pskb_trim_rcsum() and CHECKSUM_COMPLETE ...")'
skb checksum was forced to be CHECKSUM_NONE when padding is detected.
After it, we need to keep skb->csum updated, like what we do for RXFCS.
However, fixing up CHECKSUM_COMPLETE requires to verify and parse IP
headers, it is not worthy the effort as the packets are so small that
CHECKSUM_COMPLETE can't save anything.

Fixes: 88078d98d1bb ("net: pskb_trim_rcsum() and CHECKSUM_COMPLETE are friends"),
Cc: Eric Dumazet <edumazet@google.com>
Cc: Tariq Toukan <tariqt@mellanox.com>
Cc: Nikola Ciprich <nikola.ciprich@linuxbox.cz>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 1d0bb5ff8c26..f86e4804e83e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -732,6 +732,8 @@ static u8 get_ip_proto(struct sk_buff *skb, int network_depth, __be16 proto)
 					    ((struct ipv6hdr *)ip_p)->nexthdr;
 }
 
+#define short_frame(size) ((size) <= ETH_ZLEN + ETH_FCS_LEN)
+
 static inline void mlx5e_handle_csum(struct net_device *netdev,
 				     struct mlx5_cqe64 *cqe,
 				     struct mlx5e_rq *rq,
@@ -754,6 +756,17 @@ static inline void mlx5e_handle_csum(struct net_device *netdev,
 	if (unlikely(test_bit(MLX5E_RQ_STATE_NO_CSUM_COMPLETE, &rq->state)))
 		goto csum_unnecessary;
 
+	/* CQE csum doesn't cover padding octets in short ethernet
+	 * frames. And the pad field is appended prior to calculating
+	 * and appending the FCS field.
+	 *
+	 * Detecting these padded frames requires to verify and parse
+	 * IP headers, so we simply force all those small frames to be
+	 * CHECKSUM_UNNECESSARY even if they are not padded.
+	 */
+	if (short_frame(skb->len))
+		goto csum_unnecessary;
+
 	if (likely(is_last_ethertype_ip(skb, &network_depth, &proto))) {
 		if (unlikely(get_ip_proto(skb, network_depth, proto) == IPPROTO_SCTP))
 			goto csum_unnecessary;
-- 
2.20.1

[net 2/4] net/mlx5e: Fix wrong error code return on FEC query failure

[Saeed Mahameed]

From: Shay Agroskin <shayag@mellanox.com>

Advertised and configured FEC query failure resulted in printing
wrong error code.

Fixes: 6cfa94605091 ("net/mlx5e: Ethtool driver callback for query/set FEC policy")
Signed-off-by: Shay Agroskin <shayag@mellanox.com>
Reported-by: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index c9df08133718..3bbccead2f63 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -844,9 +844,12 @@ int mlx5e_ethtool_get_link_ksettings(struct mlx5e_priv *priv,
 	ethtool_link_ksettings_add_link_mode(link_ksettings, supported,
 					     Autoneg);
 
-	if (get_fec_supported_advertised(mdev, link_ksettings))
+	err = get_fec_supported_advertised(mdev, link_ksettings);
+	if (err) {
 		netdev_dbg(priv->netdev, "%s: FEC caps query failed: %d\n",
 			   __func__, err);
+		err = 0; /* don't fail caps query because of FEC error */
+	}
 
 	if (!an_disable_admin)
 		ethtool_link_ksettings_add_link_mode(link_ksettings,
-- 
2.20.1

[net 3/4] net/mlx5e: Fix wrong (zero) TX drop counter indication for representor

[Saeed Mahameed]

From: Tariq Toukan <tariqt@mellanox.com>

For representors, the TX dropped counter is not folded from the
per-ring counters. Fix it.

Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index 96cc0c6a4014..c9a7081a5329 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -179,6 +179,7 @@ static void mlx5e_rep_update_sw_counters(struct mlx5e_priv *priv)
 
 			s->tx_packets		+= sq_stats->packets;
 			s->tx_bytes		+= sq_stats->bytes;
+			s->tx_queue_dropped	+= sq_stats->dropped;
 		}
 	}
 }
-- 
2.20.1

[net 4/4] net/mlx5e: Fix cb_ident duplicate in indirect block register

[Saeed Mahameed]

From: Eli Britstein <elibr@mellanox.com>

Previously the identifier used for indirect block callback registry
and for block rule cb registry (when done via indirect blocks) was the
pointer to the tunnel netdev we were interested in receiving updates on.
This worked fine if a single PF existed that registered one callback for
the tunnel netdev of interest. However, if multiple PFs are in place then
the 2nd PF tries to register with the same tunnel netdev identifier. This
leads to EEXIST errors and/or incorrect cb deletions.

Prevent this conflict by using the rpriv pointer as the identifier for
netdev indirect block cb registry, allowing each PF to register a unique
callback per tunnel netdev. For block cb registry, the same PF may
register multiple cbs to the same block if using TC shared blocks.
Instead of the rpriv, use the pointer to the allocated indr_priv data as
the identifier here. This means that there can be a unique block callback
for each PF/tunnel netdev combo.

Fixes: f5bc2c5de101 ("net/mlx5e: Support TC indirect block notifications
for eswitch uplink reprs")
Signed-off-by: Eli Britstein <elibr@mellanox.com>
Reviewed-by: Oz Shlomo <ozsh@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 .../net/ethernet/mellanox/mlx5/core/en_rep.c  | 29 ++++++++++---------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index c9a7081a5329..04736212a21c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -58,7 +58,8 @@ struct mlx5e_rep_indr_block_priv {
 	struct list_head list;
 };
 
-static void mlx5e_rep_indr_unregister_block(struct net_device *netdev);
+static void mlx5e_rep_indr_unregister_block(struct mlx5e_rep_priv *rpriv,
+					    struct net_device *netdev);
 
 static void mlx5e_rep_get_drvinfo(struct net_device *dev,
 				  struct ethtool_drvinfo *drvinfo)
@@ -664,7 +665,7 @@ static void mlx5e_rep_indr_clean_block_privs(struct mlx5e_rep_priv *rpriv)
 	struct list_head *head = &rpriv->uplink_priv.tc_indr_block_priv_list;
 
 	list_for_each_entry_safe(cb_priv, temp, head, list) {
-		mlx5e_rep_indr_unregister_block(cb_priv->netdev);
+		mlx5e_rep_indr_unregister_block(rpriv, cb_priv->netdev);
 		kfree(cb_priv);
 	}
 }
@@ -736,7 +737,7 @@ mlx5e_rep_indr_setup_tc_block(struct net_device *netdev,
 
 		err = tcf_block_cb_register(f->block,
 					    mlx5e_rep_indr_setup_block_cb,
-					    netdev, indr_priv, f->extack);
+					    indr_priv, indr_priv, f->extack);
 		if (err) {
 			list_del(&indr_priv->list);
 			kfree(indr_priv);
@@ -744,14 +745,15 @@ mlx5e_rep_indr_setup_tc_block(struct net_device *netdev,
 
 		return err;
 	case TC_BLOCK_UNBIND:
+		indr_priv = mlx5e_rep_indr_block_priv_lookup(rpriv, netdev);
+		if (!indr_priv)
+			return -ENOENT;
+
 		tcf_block_cb_unregister(f->block,
 					mlx5e_rep_indr_setup_block_cb,
-					netdev);
-		indr_priv = mlx5e_rep_indr_block_priv_lookup(rpriv, netdev);
-		if (indr_priv) {
-			list_del(&indr_priv->list);
-			kfree(indr_priv);
-		}
+					indr_priv);
+		list_del(&indr_priv->list);
+		kfree(indr_priv);
 
 		return 0;
 	default:
@@ -780,7 +782,7 @@ static int mlx5e_rep_indr_register_block(struct mlx5e_rep_priv *rpriv,
 
 	err = __tc_indr_block_cb_register(netdev, rpriv,
 					  mlx5e_rep_indr_setup_tc_cb,
-					  netdev);
+					  rpriv);
 	if (err) {
 		struct mlx5e_priv *priv = netdev_priv(rpriv->netdev);
 
@@ -790,10 +792,11 @@ static int mlx5e_rep_indr_register_block(struct mlx5e_rep_priv *rpriv,
 	return err;
 }
 
-static void mlx5e_rep_indr_unregister_block(struct net_device *netdev)
+static void mlx5e_rep_indr_unregister_block(struct mlx5e_rep_priv *rpriv,
+					    struct net_device *netdev)
 {
 	__tc_indr_block_cb_unregister(netdev, mlx5e_rep_indr_setup_tc_cb,
-				      netdev);
+				      rpriv);
 }
 
 static int mlx5e_nic_rep_netdevice_event(struct notifier_block *nb,
@@ -812,7 +815,7 @@ static int mlx5e_nic_rep_netdevice_event(struct notifier_block *nb,
 		mlx5e_rep_indr_register_block(rpriv, netdev);
 		break;
 	case NETDEV_UNREGISTER:
-		mlx5e_rep_indr_unregister_block(netdev);
+		mlx5e_rep_indr_unregister_block(rpriv, netdev);
 		break;
 	}
 	return NOTIFY_OK;
-- 
2.20.1

Re: [pull request][net 0/4] Mellanox, mlx5 fixes 2019-01-18

[David Miller]

From: Saeed Mahameed <saeedm@mellanox.com>
Date: Fri, 18 Jan 2019 16:25:19 -0800

> This series introduces some fixes to mlx5 driver.
> 
> Please pull and let me know if there is any problem.

I'll pull this, thanks!

> For -stable v4.18
> ('net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames')
> 
> The patch doesn't apply cleanly to 4.18.y, but it is very simple to
> resolve, what should be the procedure here ?

I'll take care of it.  If I need some help, I will let you know.

Re: [net 1/4] net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames

[Cong Wang]

On Fri, Jan 18, 2019 at 4:25 PM Saeed Mahameed <saeedm@mellanox.com> wrote:
>
> From: Cong Wang <xiyou.wangcong@gmail.com>

I don't know why you want to make me as the author here, but I never
agree on _your_ updates on my previous patch.

Please see below.


>  drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> index 1d0bb5ff8c26..f86e4804e83e 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> @@ -732,6 +732,8 @@ static u8 get_ip_proto(struct sk_buff *skb, int network_depth, __be16 proto)
>                                             ((struct ipv6hdr *)ip_p)->nexthdr;
>  }
>
> +#define short_frame(size) ((size) <= ETH_ZLEN + ETH_FCS_LEN)
> +


I don't agree on unconditionally comparing with ETH_ZLEN + ETH_FCS_LEN.


>  static inline void mlx5e_handle_csum(struct net_device *netdev,
>                                      struct mlx5_cqe64 *cqe,
>                                      struct mlx5e_rq *rq,
> @@ -754,6 +756,17 @@ static inline void mlx5e_handle_csum(struct net_device *netdev,
>         if (unlikely(test_bit(MLX5E_RQ_STATE_NO_CSUM_COMPLETE, &rq->state)))
>                 goto csum_unnecessary;
>
> +       /* CQE csum doesn't cover padding octets in short ethernet
> +        * frames. And the pad field is appended prior to calculating
> +        * and appending the FCS field.
> +        *
> +        * Detecting these padded frames requires to verify and parse
> +        * IP headers, so we simply force all those small frames to be
> +        * CHECKSUM_UNNECESSARY even if they are not padded.

This is inaccurate and misleading, it is unnecessary only if the packet
passes the if check right below the goto label 'csum_unnecessary',
otherwise still a CHECKSUM_NONE. IOW, you are not forcing anything
here.

> +        */
> +       if (short_frame(skb->len))

Missed an "unlikely()". Short frames are rare, comparing to non-short
ones.

I respect your judgement on CHECKSUM_UNNECESSARY, even when
I still disagree with you. Please respect me by not forcing me to accept
any updates from you, IOW, kindly removing my name from anything
in this commit, SoB and authorship.

Thank you for your understanding!

Re: [net 1/4] net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames

[Saeed Mahameed]

On Tue, 2019-01-22 at 13:30 -0800, Cong Wang wrote:
> On Fri, Jan 18, 2019 at 4:25 PM Saeed Mahameed <saeedm@mellanox.com>
> wrote:
> > From: Cong Wang <xiyou.wangcong@gmail.com>
> 
> I don't know why you want to make me as the author here, but I never
> agree on _your_ updates on my previous patch.
> 
> Please see below.
> 

sorry, i just took your patch and worked on top of it, i thought you
would like to get the credit for this.


> 
> >  drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 13 +++++++++++++
> >  1 file changed, 13 insertions(+)
> > 
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > index 1d0bb5ff8c26..f86e4804e83e 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > @@ -732,6 +732,8 @@ static u8 get_ip_proto(struct sk_buff *skb, int
> > network_depth, __be16 proto)
> >                                             ((struct ipv6hdr
> > *)ip_p)->nexthdr;
> >  }
> > 
> > +#define short_frame(size) ((size) <= ETH_ZLEN + ETH_FCS_LEN)
> > +
> 
> I don't agree on unconditionally comparing with ETH_ZLEN +
> ETH_FCS_LEN.
> 

This is more relaxed and it covers both cases unconditionally. 

> 
> >  static inline void mlx5e_handle_csum(struct net_device *netdev,
> >                                      struct mlx5_cqe64 *cqe,
> >                                      struct mlx5e_rq *rq,
> > @@ -754,6 +756,17 @@ static inline void mlx5e_handle_csum(struct
> > net_device *netdev,
> >         if (unlikely(test_bit(MLX5E_RQ_STATE_NO_CSUM_COMPLETE, &rq-
> > >state)))
> >                 goto csum_unnecessary;
> > 
> > +       /* CQE csum doesn't cover padding octets in short ethernet
> > +        * frames. And the pad field is appended prior to
> > calculating
> > +        * and appending the FCS field.
> > +        *
> > +        * Detecting these padded frames requires to verify and
> > parse
> > +        * IP headers, so we simply force all those small frames to
> > be
> > +        * CHECKSUM_UNNECESSARY even if they are not padded.
> 
> This is inaccurate and misleading, it is unnecessary only if the
> packet
> passes the if check right below the goto label 'csum_unnecessary',
> otherwise still a CHECKSUM_NONE. IOW, you are not forcing anything
> here.
> 

yes, the comment is not 100% accurate, but it delivers the message.

> > +        */
> > +       if (short_frame(skb->len))
> 
> Missed an "unlikely()". Short frames are rare, comparing to non-short
> ones.
> 
> I respect your judgement on CHECKSUM_UNNECESSARY, even when
> I still disagree with you. Please respect me by not forcing me to
> accept
> any updates from you, IOW, kindly removing my name from anything
> in this commit, SoB and authorship.
> 
> Thank you for your understanding!

Again sorry about this, will be more careful in the future.

Thanks for your support and great work.

Re: [net 1/4] net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames

[Eric Dumazet]

On 01/25/2019 10:22 AM, Saeed Mahameed wrote:
> On Tue, 2019-01-22 at 13:30 -0800, Cong Wang wrote:
>> On Fri, Jan 18, 2019 at 4:25 PM Saeed Mahameed <saeedm@mellanox.com>
>> wrote:
>>> From: Cong Wang <xiyou.wangcong@gmail.com>
>>
>> I don't know why you want to make me as the author here, but I never
>> agree on _your_ updates on my previous patch.
>>
>> Please see below.
>>
> 
> sorry, i just took your patch and worked on top of it, i thought you
> would like to get the credit for this.
>

I thought the issue was that the hardware csum provided by both mlx4 and mlx5 only
 covered the bytes included in the IP (v4 or v6) frame.

Meaning that any non zero padding bytes are not checksummed.

If this can not be fixed by a firmware change, then the fix has nothing to do with a frame being
smaller than ETH_ZLEN + ETH_FCS_LEN

Alternative would be for the driver to trim the frame (pretend the skb->len is exactly the expected one),
but one could argue that tcpdump should be able to see padding bytes.

Re: [net 1/4] net/mlx5e: Force CHECKSUM_UNNECESSARY for short ethernet frames

[Saeed Mahameed]

On Fri, 2019-01-25 at 10:31 -0800, Eric Dumazet wrote:
> 
> On 01/25/2019 10:22 AM, Saeed Mahameed wrote:
> > On Tue, 2019-01-22 at 13:30 -0800, Cong Wang wrote:
> > > On Fri, Jan 18, 2019 at 4:25 PM Saeed Mahameed <
> > > saeedm@mellanox.com>
> > > wrote:
> > > > From: Cong Wang <xiyou.wangcong@gmail.com>
> > > 
> > > I don't know why you want to make me as the author here, but I
> > > never
> > > agree on _your_ updates on my previous patch.
> > > 
> > > Please see below.
> > > 
> > 
> > sorry, i just took your patch and worked on top of it, i thought
> > you
> > would like to get the credit for this.
> > 
> 
> I thought the issue was that the hardware csum provided by both mlx4
> and mlx5 only
>  covered the bytes included in the IP (v4 or v6) frame.
> 
> Meaning that any non zero padding bytes are not checksummed.

in case of non IP, mlx5 will provide csum complete on the whole frame.

> If this can not be fixed by a firmware change, then the fix has
> nothing to do with a frame being
> smaller than ETH_ZLEN + ETH_FCS_LEN
> 

Again, most of the switches/routers that have non-zero padding bug are
padding only small frames.

> Alternative would be for the driver to trim the frame (pretend the
> skb->len is exactly the expected one),
> but one could argue that tcpdump should be able to see padding bytes.
> 

That requires parsing the IP headers in the driver, we are trying to
avoid that, this patch is not perfect but eliminates many of the csum
warnings seen on mlx5.

>