[PATCH v4 1/2] tg3: Move the [rt]x_dropped counters to tg3_napi

[PATCH v4 1/2] tg3: Move the [rt]x_dropped counters to tg3_napi

[alexey.pakhunov]

From: Alex Pakhunov <alexey.pakhunov@spacex.com>

This change moves [rt]x_dropped counters to tg3_napi so that they can be
updated by a single writer, race-free.

Signed-off-by: Alex Pakhunov <alexey.pakhunov@spacex.com>
Signed-off-by: Vincent Wong <vincent.wong2@spacex.com>

---
v4: Restore the correct commit message.
v3: Fix a checkpatch.pl error
    https://lore.kernel.org/netdev/20231113181325.26203-1-alexey.pakhunov@spacex.com/
v2: Save [rt]x_dropped across chip resets
    https://lore.kernel.org/netdev/20231110002340.3612515-1-alexey.pakhunov@spacex.com/
v1: https://lore.kernel.org/netdev/20231108010129.2009947-1-alexey.pakhunov@spacex.com/
---
 drivers/net/ethernet/broadcom/tg3.c | 38 +++++++++++++++++++++++++----
 drivers/net/ethernet/broadcom/tg3.h |  4 +--
 2 files changed, 35 insertions(+), 7 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 14b311196b8f..dcaf97198ac2 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -6845,7 +6845,7 @@ static int tg3_rx(struct tg3_napi *tnapi, int budget)
 				       desc_idx, *post_ptr);
 		drop_it_no_recycle:
 			/* Other statistics kept track of by card. */
-			tp->rx_dropped++;
+			tnapi->rx_dropped++;
 			goto next_pkt;
 		}
 
@@ -8146,7 +8146,7 @@ static netdev_tx_t tg3_start_xmit(struct sk_buff *skb, struct net_device *dev)
 drop:
 	dev_kfree_skb_any(skb);
 drop_nofree:
-	tp->tx_dropped++;
+	tnapi->tx_dropped++;
 	return NETDEV_TX_OK;
 }
 
@@ -9325,7 +9325,7 @@ static void __tg3_set_rx_mode(struct net_device *);
 /* tp->lock is held. */
 static int tg3_halt(struct tg3 *tp, int kind, bool silent)
 {
-	int err;
+	int err, i;
 
 	tg3_stop_fw(tp);
 
@@ -9346,6 +9346,13 @@ static int tg3_halt(struct tg3 *tp, int kind, bool silent)
 
 		/* And make sure the next sample is new data */
 		memset(tp->hw_stats, 0, sizeof(struct tg3_hw_stats));
+
+		for (i = 0; i < TG3_IRQ_MAX_VECS; ++i) {
+			struct tg3_napi *tnapi = &tp->napi[i];
+
+			tnapi->rx_dropped = 0;
+			tnapi->tx_dropped = 0;
+		}
 	}
 
 	return err;
@@ -11895,6 +11902,9 @@ static void tg3_get_nstats(struct tg3 *tp, struct rtnl_link_stats64 *stats)
 {
 	struct rtnl_link_stats64 *old_stats = &tp->net_stats_prev;
 	struct tg3_hw_stats *hw_stats = tp->hw_stats;
+	unsigned long rx_dropped;
+	unsigned long tx_dropped;
+	int i;
 
 	stats->rx_packets = old_stats->rx_packets +
 		get_stat64(&hw_stats->rx_ucast_packets) +
@@ -11941,8 +11951,26 @@ static void tg3_get_nstats(struct tg3 *tp, struct rtnl_link_stats64 *stats)
 	stats->rx_missed_errors = old_stats->rx_missed_errors +
 		get_stat64(&hw_stats->rx_discards);
 
-	stats->rx_dropped = tp->rx_dropped;
-	stats->tx_dropped = tp->tx_dropped;
+	/* Aggregate per-queue counters. The per-queue counters are updated
+	 * by a single writer, race-free. The result computed by this loop
+	 * might not be 100% accurate (counters can be updated in the middle of
+	 * the loop) but the next tg3_get_nstats() will recompute the current
+	 * value so it is acceptable.
+	 *
+	 * Note that these counters wrap around at 4G on 32bit machines.
+	 */
+	rx_dropped = (unsigned long)(old_stats->rx_dropped);
+	tx_dropped = (unsigned long)(old_stats->tx_dropped);
+
+	for (i = 0; i < tp->irq_cnt; i++) {
+		struct tg3_napi *tnapi = &tp->napi[i];
+
+		rx_dropped += tnapi->rx_dropped;
+		tx_dropped += tnapi->tx_dropped;
+	}
+
+	stats->rx_dropped = rx_dropped;
+	stats->tx_dropped = tx_dropped;
 }
 
 static int tg3_get_regs_len(struct net_device *dev)
diff --git a/drivers/net/ethernet/broadcom/tg3.h b/drivers/net/ethernet/broadcom/tg3.h
index 1000c894064f..8d753f8c5b06 100644
--- a/drivers/net/ethernet/broadcom/tg3.h
+++ b/drivers/net/ethernet/broadcom/tg3.h
@@ -3018,6 +3018,7 @@ struct tg3_napi {
 	u16				*rx_rcb_prod_idx;
 	struct tg3_rx_prodring_set	prodring;
 	struct tg3_rx_buffer_desc	*rx_rcb;
+	unsigned long			rx_dropped;
 
 	u32				tx_prod	____cacheline_aligned;
 	u32				tx_cons;
@@ -3026,6 +3027,7 @@ struct tg3_napi {
 	u32				prodmbox;
 	struct tg3_tx_buffer_desc	*tx_ring;
 	struct tg3_tx_ring_info		*tx_buffers;
+	unsigned long			tx_dropped;
 
 	dma_addr_t			status_mapping;
 	dma_addr_t			rx_rcb_mapping;
@@ -3219,8 +3221,6 @@ struct tg3 {
 
 
 	/* begin "everything else" cacheline(s) section */
-	unsigned long			rx_dropped;
-	unsigned long			tx_dropped;
 	struct rtnl_link_stats64	net_stats_prev;
 	struct tg3_ethtool_stats	estats_prev;
 

base-commit: ffc253263a1375a65fa6c9f62a893e9767fbebfa
-- 
2.39.3

[PATCH v4 2/2] tg3: Increment tx_dropped in tg3_tso_bug()

[alexey.pakhunov]

From: Alex Pakhunov <alexey.pakhunov@spacex.com>

tg3_tso_bug() drops a packet if it cannot be segmented for any reason.
The number of discarded frames should be incremented accordingly.

Signed-off-by: Alex Pakhunov <alexey.pakhunov@spacex.com>
Signed-off-by: Vincent Wong <vincent.wong2@spacex.com>
---
 drivers/net/ethernet/broadcom/tg3.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index dcaf97198ac2..8f22b4cdc31f 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -7874,8 +7874,10 @@ static int tg3_tso_bug(struct tg3 *tp, struct tg3_napi *tnapi,
 
 	segs = skb_gso_segment(skb, tp->dev->features &
 				    ~(NETIF_F_TSO | NETIF_F_TSO6));
-	if (IS_ERR(segs) || !segs)
+	if (IS_ERR(segs) || !segs) {
+		tnapi->tx_dropped++;
 		goto tg3_tso_bug_end;
+	}
 
 	skb_list_walk_safe(segs, seg, next) {
 		skb_mark_not_on_list(seg);
-- 
2.39.3

Re: [PATCH v4 1/2] tg3: Move the [rt]x_dropped counters to tg3_napi

[Jakub Kicinski]

On Mon, 13 Nov 2023 10:23:49 -0800 alexey.pakhunov@spacex.com wrote:
> This change moves [rt]x_dropped counters to tg3_napi so that they can be
> updated by a single writer, race-free.

Michael, do you have a preference on the using u64_stats_inc() ?
Since we're only doing inc here the conversion should be pretty
trivial. The semantics of local64 are a bit murky but looks like
other drivers think that it's okay to use inc without
u64_stats_update_begin() / end().

Re: [PATCH v4 1/2] tg3: Move the [rt]x_dropped counters to tg3_napi

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 892 bytes --]

On Mon, Nov 13, 2023 at 2:38 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Mon, 13 Nov 2023 10:23:49 -0800 alexey.pakhunov@spacex.com wrote:
> > This change moves [rt]x_dropped counters to tg3_napi so that they can be
> > updated by a single writer, race-free.
>
> Michael, do you have a preference on the using u64_stats_inc() ?
> Since we're only doing inc here the conversion should be pretty
> trivial. The semantics of local64 are a bit murky but looks like
> other drivers think that it's okay to use inc without
> u64_stats_update_begin() / end().

The existing driver has been using unsigned long for these 2 software
counters, so I think it is ok to just keep them unchanged.  I agree
with Alexey that these 2 counters should be relatively rare.  We can
change to use u64_stats later if needed.

Thanks,
Reviewed-by: Michael Chan <michael.chan@broadcom.com>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH v4 2/2] tg3: Increment tx_dropped in tg3_tso_bug()

[Pavan Chebbi]

[-- Attachment #1: Type: text/plain, Size: 1311 bytes --]

On Mon, Nov 13, 2023 at 11:54 PM <alexey.pakhunov@spacex.com> wrote:
>
> From: Alex Pakhunov <alexey.pakhunov@spacex.com>
>
> tg3_tso_bug() drops a packet if it cannot be segmented for any reason.
> The number of discarded frames should be incremented accordingly.
>
> Signed-off-by: Alex Pakhunov <alexey.pakhunov@spacex.com>
> Signed-off-by: Vincent Wong <vincent.wong2@spacex.com>
> ---
>  drivers/net/ethernet/broadcom/tg3.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
> index dcaf97198ac2..8f22b4cdc31f 100644
> --- a/drivers/net/ethernet/broadcom/tg3.c
> +++ b/drivers/net/ethernet/broadcom/tg3.c
> @@ -7874,8 +7874,10 @@ static int tg3_tso_bug(struct tg3 *tp, struct tg3_napi *tnapi,
>
>         segs = skb_gso_segment(skb, tp->dev->features &
>                                     ~(NETIF_F_TSO | NETIF_F_TSO6));
> -       if (IS_ERR(segs) || !segs)
> +       if (IS_ERR(segs) || !segs) {
> +               tnapi->tx_dropped++;
>                 goto tg3_tso_bug_end;
> +       }
>
>         skb_list_walk_safe(segs, seg, next) {
>                 skb_mark_not_on_list(seg);
> --
> 2.39.3
>
>

Reviewed-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
Thank you.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH v4 1/2] tg3: Move the [rt]x_dropped counters to tg3_napi

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Mon, 13 Nov 2023 10:23:49 -0800 you wrote:
> From: Alex Pakhunov <alexey.pakhunov@spacex.com>
> 
> This change moves [rt]x_dropped counters to tg3_napi so that they can be
> updated by a single writer, race-free.
> 
> Signed-off-by: Alex Pakhunov <alexey.pakhunov@spacex.com>
> Signed-off-by: Vincent Wong <vincent.wong2@spacex.com>
> 
> [...]

Here is the summary with links:
  - [v4,1/2] tg3: Move the [rt]x_dropped counters to tg3_napi
    https://git.kernel.org/netdev/net/c/907d1bdb8b2c
  - [v4,2/2] tg3: Increment tx_dropped in tg3_tso_bug()
    https://git.kernel.org/netdev/net/c/17dd5efe5f36

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html