* [PATCH iwl-next v3 1/9] ice: initialize ring_stats->syncp
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 2/9] ice: use cacheline groups for ice_rx_ring structure Jacob Keller
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The u64_stats_sync structure is empty on 64-bit systems. However, on 32-bit
systems it contains a seqcount_t which needs to be initialized. While the
memory is zero-initialized, a lack of u64_stats_init means that lockdep
won't get initialized properly. Fix this by adding u64_stats_init() calls
to the rings just after allocation.
Fixes: 2b245cb29421 ("ice: Implement transmit and NAPI support")
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lib.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index e366d089bef9..46cd8f33c38f 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -400,7 +400,10 @@ static int ice_vsi_alloc_ring_stats(struct ice_vsi *vsi)
if (!ring_stats)
goto err_out;
+ u64_stats_init(&ring_stats->syncp);
+
WRITE_ONCE(tx_ring_stats[i], ring_stats);
+
}
ring->ring_stats = ring_stats;
@@ -419,6 +422,8 @@ static int ice_vsi_alloc_ring_stats(struct ice_vsi *vsi)
if (!ring_stats)
goto err_out;
+ u64_stats_init(&ring_stats->syncp);
+
WRITE_ONCE(rx_ring_stats[i], ring_stats);
}
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 2/9] ice: use cacheline groups for ice_rx_ring structure
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 1/9] ice: initialize ring_stats->syncp Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 3/9] ice: use cacheline groups for ice_tx_ring structure Jacob Keller
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The ice ring structure was reorganized back by commit 65124bbf980c ("ice:
Reorganize tx_buf and ring structs"), and later split into a separate
ice_rx_ring structure by commit e72bba21355d ("ice: split ice_ring onto
Tx/Rx separate structs")
The ice_rx_ring structure has comments left over from this prior
reorganization indicating which fields belong to which cachelines.
Unfortunately, these comments are not all accurate. The intended layout is
for x86_64 systems with a 64-byte cache.
* Cacheline 1 spans from the start of the struct to the end of the rx_fqes
and xdp_buf union. The comments correctly match this.
* Cacheline 2 spans from hdr_fqes to the end of hdr_truesize, but the
comment indicates it should end xdp and xsk union.
* Cacheline 3 spans from the truesize field to the xsk_pool, but the
comment wants this to be from the pkt_ctx down to the rcu head field.
* Cacheline 4 spans from the rx_hdr_len down to the flags field, but the
comment indicates that it starts back at the ice_channel structure
pointer.
* Cacheline 5 is indicated to cover the xdp_rxq. Because this field is
aligned to 64 bytes, this is actually true. However, there is a large 45
byte gap at the end of cacheline 4.
The use of comments to indicate cachelines is poor design. In practice,
comments like this quickly become outdated as developers add or remove
fields, or as other sub-structures change layout and size unexpectedly.
The ice_rx_ring structure *is* 5 cachelines (320 bytes), but ends up having
quite a lot of empty space at the end just before xdp_rxq.
Replace the comments with __cacheline_group_(begin|end)_aligned()
annotations. These macros enforce alignment to the start of cachelines, and
enforce padding between groups, thus guaranteeing that a following group
cannot be part of the same cacheline.
Doing this changes the layout by effectively spreading the padding at the
tail of cacheline 4 between groups to ensure that the relevant fields are
kept as separate cachelines on x86_64 systems. For systems with the
expected cache line size of 64 bytes, the structure size does not change.
For systems with a larger SMB_CACHE_BYTES size, the structure *will*
increase in size a fair bit, however we'll now guarantee that each group is
in a separate cacheline. This has an advantage that updates to fields in
one group won't trigger cacheline eviction of the other groups. This comes
at the expense of extra memory footprint for the rings.
If fields get re-arranged, added, or removed, the alignment and padding
ensure the relevant fields are kept on separate cache lines. This could
result in unexpected changes in the structure size due to padding to keep
cachelines separate.
To catch such changes during early development, add build time compiler
assertions that check the size of each group to ensure it doesn't exceed
the size of one cache line. This will produce a compile failure in the
event that new fields get added or re-arranged such that any of the cache
groups exceed one cacheline.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.h | 26 +++++++++++++++++++++-----
drivers/net/ethernet/intel/ice/ice_main.c | 2 ++
2 files changed, 23 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index e440c55d9e9f..8b9247535fcb 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -236,7 +236,7 @@ struct ice_tstamp_ring {
} ____cacheline_internodealigned_in_smp;
struct ice_rx_ring {
- /* CL1 - 1st cacheline starts here */
+ __cacheline_group_begin_aligned(cl1);
void *desc; /* Descriptor ring memory */
struct page_pool *pp;
struct net_device *netdev; /* netdev ring maps to */
@@ -253,8 +253,9 @@ struct ice_rx_ring {
struct libeth_fqe *rx_fqes;
struct xdp_buff **xdp_buf;
};
+ __cacheline_group_end_aligned(cl1);
- /* CL2 - 2nd cacheline starts here */
+ __cacheline_group_begin_aligned(cl2);
struct libeth_fqe *hdr_fqes;
struct page_pool *hdr_pp;
@@ -262,8 +263,9 @@ struct ice_rx_ring {
struct libeth_xdp_buff_stash xdp;
struct libeth_xdp_buff *xsk;
};
+ __cacheline_group_end_aligned(cl2);
- /* CL3 - 3rd cacheline starts here */
+ __cacheline_group_begin_aligned(cl3);
union {
struct ice_pkt_ctx pkt_ctx;
struct {
@@ -284,7 +286,9 @@ struct ice_rx_ring {
struct ice_ring_stats *ring_stats;
struct rcu_head rcu; /* to avoid race on free */
- /* CL4 - 4th cacheline starts here */
+ __cacheline_group_end_aligned(cl3);
+
+ __cacheline_group_begin_aligned(cl4);
struct ice_channel *ch;
struct ice_tx_ring *xdp_ring;
struct ice_rx_ring *next; /* pointer to next ring in q_vector */
@@ -298,10 +302,22 @@ struct ice_rx_ring {
#define ICE_RX_FLAGS_MULTIDEV BIT(3)
#define ICE_RX_FLAGS_RING_GCS BIT(4)
u8 flags;
- /* CL5 - 5th cacheline starts here */
+ __cacheline_group_end_aligned(cl4);
+
+ __cacheline_group_begin_aligned(cl5);
struct xdp_rxq_info xdp_rxq;
+ __cacheline_group_end_aligned(cl5);
} ____cacheline_internodealigned_in_smp;
+static inline void ice_rx_ring_struct_check(void)
+{
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl1, SMP_CACHE_BYTES);
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl2, SMP_CACHE_BYTES);
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl3, SMP_CACHE_BYTES);
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl4, SMP_CACHE_BYTES);
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_rx_ring, cl5, SMP_CACHE_BYTES);
+}
+
struct ice_tx_ring {
/* CL1 - 1st cacheline starts here */
struct ice_tx_ring *next; /* pointer to next ring in q_vector */
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index b16ede1f139d..4731dbaca9de 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -5916,6 +5916,8 @@ static int __init ice_module_init(void)
{
int status = -ENOMEM;
+ ice_rx_ring_struct_check();
+
pr_info("%s\n", ice_driver_string);
pr_info("%s\n", ice_copyright);
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 3/9] ice: use cacheline groups for ice_tx_ring structure
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 1/9] ice: initialize ring_stats->syncp Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 2/9] ice: use cacheline groups for ice_rx_ring structure Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 4/9] ice: move prev_pkt from ice_txq_stats to ice_tx_ring Jacob Keller
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller
The ice ring structure was reorganized by commit 65124bbf980c ("ice:
Reorganize tx_buf and ring structs"), and later split into a separate
ice_tx_ring structure by commit e72bba21355d ("ice: split ice_ring onto
Tx/Rx separate structs").
The ice_tx_ring structure has comments left over from this reorganization
and split indicating which fields are supposed to belong to which
cachelines. Unfortunately, these comments are almost completely incorrect.
* Cacheline 1 spans from the start of the structure to the vsi pointer.
This cacheline is correct, and appears to be the only one that is.
* Cacheline 2 spans from the DMA address down to the xps_state field. The
comments indicate it should end at the rcu head field.
* Cacheline 3 spans from the ice_channel pointer to the end of the struct,
and completely contains what is marked as a separate 4th cacheline.
The use of such comments to indicate cache lines is error prone. It is
extremely likely that the comments will become out of date with future
refactors. Instead, use __cacheline_group_(begin|end)_aligned() which is
more explicit. It guarantees that members between the cacheline groups will
be in distinct cache lines through the use of padding. It additionally
enables compile time assertions to help prevent new fields from drastically
re-arranging the cache lines.
There are two main issues if we just replace the existing comments with
cache line group markers. First, the spinlock_t tx_lock field is 24 bytes
on most kernels, but is 64 bytes on CONFIG_DEBUG_LOCK_ALLOC kernels.
Ideally we want to place this field at the start of a cacheline so that
other fields in the group don't get split across such a debug kernel. While
optimizing such a debug kernel is not a priority, doing this makes the
assertions around the cacheline a bit easier to understand.
Remove the out-of-date cacheline comments, and add __cacheline_group
annotations. These are set to match the existing layout instead of matching
the original comments. The only change to layout is to re-order the tx_lock
to be the start of cacheline 3, and move txq_teid to avoid a 4-byte gap in
the layout.
Ideally, we should profile the Tx hot path and figure out which fields go
together and re-arrange the cacheline groups, possibly along "read_mostly",
"readwrite" and "cold" groupings similar to the idpf driver. This has been
left as an exercise for a later improvement.
Finally, add annotations which check the cacheline sizes. For cacheline 3,
we enforce that tx_lock is in this cacheline group, and only check the size
if CONFIG_DEBUG_LOCK_ALLOC is disabled. These are primarily intended to
produce a compiler failure if developers add or re-arrange fields in a way
that accidentally causes a group to exceed one cacheline.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.h | 32 +++++++++++++++++++++++++------
drivers/net/ethernet/intel/ice/ice_main.c | 1 +
2 files changed, 27 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index 8b9247535fcb..78fed538ea0f 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -319,7 +319,7 @@ static inline void ice_rx_ring_struct_check(void)
}
struct ice_tx_ring {
- /* CL1 - 1st cacheline starts here */
+ __cacheline_group_begin_aligned(cl1);
struct ice_tx_ring *next; /* pointer to next ring in q_vector */
void *desc; /* Descriptor ring memory */
struct device *dev; /* Used for DMA mapping */
@@ -328,7 +328,9 @@ struct ice_tx_ring {
struct ice_q_vector *q_vector; /* Backreference to associated vector */
struct net_device *netdev; /* netdev ring maps to */
struct ice_vsi *vsi; /* Backreference to associated VSI */
- /* CL2 - 2nd cacheline starts here */
+ __cacheline_group_end_aligned(cl1);
+
+ __cacheline_group_begin_aligned(cl2);
dma_addr_t dma; /* physical address of ring */
struct xsk_buff_pool *xsk_pool;
u16 next_to_use;
@@ -340,15 +342,18 @@ struct ice_tx_ring {
u16 xdp_tx_active;
/* stats structs */
struct ice_ring_stats *ring_stats;
- /* CL3 - 3rd cacheline starts here */
struct rcu_head rcu; /* to avoid race on free */
DECLARE_BITMAP(xps_state, ICE_TX_NBITS); /* XPS Config State */
+ __cacheline_group_end_aligned(cl2);
+
+ __cacheline_group_begin_aligned(cl3);
+ spinlock_t tx_lock;
struct ice_channel *ch;
struct ice_ptp_tx *tx_tstamps;
- spinlock_t tx_lock;
- u32 txq_teid; /* Added Tx queue TEID */
- /* CL4 - 4th cacheline starts here */
struct ice_tstamp_ring *tstamp_ring;
+
+ u32 txq_teid; /* Added Tx queue TEID */
+
#define ICE_TX_FLAGS_RING_XDP BIT(0)
#define ICE_TX_FLAGS_RING_VLAN_L2TAG1 BIT(1)
#define ICE_TX_FLAGS_RING_VLAN_L2TAG2 BIT(2)
@@ -356,8 +361,23 @@ struct ice_tx_ring {
u8 flags;
u8 dcb_tc; /* Traffic class of ring */
u16 quanta_prof_id;
+ __cacheline_group_end_aligned(cl3);
} ____cacheline_internodealigned_in_smp;
+static inline void ice_tx_ring_struct_check(void)
+{
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_tx_ring, cl1, SMP_CACHE_BYTES);
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_tx_ring, cl2, SMP_CACHE_BYTES);
+
+ CACHELINE_ASSERT_GROUP_MEMBER(struct ice_tx_ring, cl3, tx_lock);
+#ifndef CONFIG_DEBUG_LOCK_ALLOC
+ /* spinlock_t is larger on DEBUG_LOCK_ALLOC kernels, so do not check
+ * the size in that case.
+ */
+ CACHELINE_ASSERT_GROUP_SIZE(struct ice_tx_ring, cl3, SMP_CACHE_BYTES);
+#endif
+}
+
static inline bool ice_ring_ch_enabled(struct ice_tx_ring *ring)
{
return !!ring->ch;
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 4731dbaca9de..645a2113e8aa 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -5917,6 +5917,7 @@ static int __init ice_module_init(void)
int status = -ENOMEM;
ice_rx_ring_struct_check();
+ ice_tx_ring_struct_check();
pr_info("%s\n", ice_driver_string);
pr_info("%s\n", ice_copyright);
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 4/9] ice: move prev_pkt from ice_txq_stats to ice_tx_ring
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
` (2 preceding siblings ...)
2025-11-07 23:31 ` [PATCH iwl-next v3 3/9] ice: use cacheline groups for ice_tx_ring structure Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 5/9] ice: pass pointer to ice_fetch_u64_stats_per_ring Jacob Keller
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The prev_pkt field in ice_txq_stats is used by ice_check_for_hung_subtask
as a way to detect potential Tx hangs due to missed interrupts.
The value is based on the packet count, but its an int and not really a
"statistic". The value is signed so that we can use -1 as a "no work
pending" value. A following change is going to refactor the stats to all
use the u64_stat_t type and accessor functions. Leaving prev_pkt as the
lone int feels a bit strange.
Instead, move it out of ice_txq_stats and place it in the ice_tx_ring. We
have 8 bytes still available in the 3rd cacheline, so this move saves a
small amount of memory. It also shouldn't impact the Tx path heavily since
its only accessed during initialization and the hang subtask.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.h | 3 ++-
drivers/net/ethernet/intel/ice/ice_main.c | 6 +++---
drivers/net/ethernet/intel/ice/ice_txrx.c | 2 +-
3 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index 78fed538ea0f..aa0b74e45bba 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -138,7 +138,6 @@ struct ice_txq_stats {
u64 restart_q;
u64 tx_busy;
u64 tx_linearize;
- int prev_pkt; /* negative if no pending Tx descriptors */
};
struct ice_rxq_stats {
@@ -354,6 +353,8 @@ struct ice_tx_ring {
u32 txq_teid; /* Added Tx queue TEID */
+ int prev_pkt; /* negative if no pending Tx descriptors */
+
#define ICE_TX_FLAGS_RING_XDP BIT(0)
#define ICE_TX_FLAGS_RING_VLAN_L2TAG1 BIT(1)
#define ICE_TX_FLAGS_RING_VLAN_L2TAG2 BIT(2)
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 645a2113e8aa..df5da7b4ec62 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -160,7 +160,7 @@ static void ice_check_for_hang_subtask(struct ice_pf *pf)
* pending work.
*/
packets = ring_stats->stats.pkts & INT_MAX;
- if (ring_stats->tx_stats.prev_pkt == packets) {
+ if (tx_ring->prev_pkt == packets) {
/* Trigger sw interrupt to revive the queue */
ice_trigger_sw_intr(hw, tx_ring->q_vector);
continue;
@@ -170,8 +170,8 @@ static void ice_check_for_hang_subtask(struct ice_pf *pf)
* to ice_get_tx_pending()
*/
smp_rmb();
- ring_stats->tx_stats.prev_pkt =
- ice_get_tx_pending(tx_ring) ? packets : -1;
+ tx_ring->prev_pkt =
+ ice_get_tx_pending(tx_ring) ? packets : -1;
}
}
}
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index ad76768a4232..30073ed9ca99 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -499,7 +499,7 @@ int ice_setup_tx_ring(struct ice_tx_ring *tx_ring)
tx_ring->next_to_use = 0;
tx_ring->next_to_clean = 0;
- tx_ring->ring_stats->tx_stats.prev_pkt = -1;
+ tx_ring->prev_pkt = -1;
return 0;
err:
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 5/9] ice: pass pointer to ice_fetch_u64_stats_per_ring
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
` (3 preceding siblings ...)
2025-11-07 23:31 ` [PATCH iwl-next v3 4/9] ice: move prev_pkt from ice_txq_stats to ice_tx_ring Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 6/9] ice: remove ice_q_stats struct and use struct_group Jacob Keller
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The ice_fetch_u64_stats_per_ring function takes a pointer to the syncp from
the ring stats to synchronize reading of the packet stats. It also takes a
*copy* of the ice_q_stats fields instead of a pointer to the stats. This
completely defeats the point of using the u64_stats API. We pass the stats
by value, so they are static at the point of reading within the
u64_stats_fetch_retry loop.
Simplify the function to take a pointer to the ice_ring_stats instead of
two separate parameters. Additionally, since we never call this outside of
ice_main.c, make it a static function.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 3 ---
drivers/net/ethernet/intel/ice/ice_main.c | 24 +++++++++---------------
2 files changed, 9 insertions(+), 18 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 147aaee192a7..5c01e886e83e 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -957,9 +957,6 @@ u16 ice_get_avail_rxq_count(struct ice_pf *pf);
int ice_vsi_recfg_qs(struct ice_vsi *vsi, int new_rx, int new_tx, bool locked);
void ice_update_vsi_stats(struct ice_vsi *vsi);
void ice_update_pf_stats(struct ice_pf *pf);
-void
-ice_fetch_u64_stats_per_ring(struct u64_stats_sync *syncp,
- struct ice_q_stats stats, u64 *pkts, u64 *bytes);
int ice_up(struct ice_vsi *vsi);
int ice_down(struct ice_vsi *vsi);
int ice_down_up(struct ice_vsi *vsi);
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index df5da7b4ec62..5a3bcbb5f63c 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -6826,25 +6826,23 @@ int ice_up(struct ice_vsi *vsi)
/**
* ice_fetch_u64_stats_per_ring - get packets and bytes stats per ring
- * @syncp: pointer to u64_stats_sync
- * @stats: stats that pkts and bytes count will be taken from
+ * @stats: pointer to ring stats structure
* @pkts: packets stats counter
* @bytes: bytes stats counter
*
* This function fetches stats from the ring considering the atomic operations
* that needs to be performed to read u64 values in 32 bit machine.
*/
-void
-ice_fetch_u64_stats_per_ring(struct u64_stats_sync *syncp,
- struct ice_q_stats stats, u64 *pkts, u64 *bytes)
+static void ice_fetch_u64_stats_per_ring(struct ice_ring_stats *stats,
+ u64 *pkts, u64 *bytes)
{
unsigned int start;
do {
- start = u64_stats_fetch_begin(syncp);
- *pkts = stats.pkts;
- *bytes = stats.bytes;
- } while (u64_stats_fetch_retry(syncp, start));
+ start = u64_stats_fetch_begin(&stats->syncp);
+ *pkts = stats->stats.pkts;
+ *bytes = stats->stats.bytes;
+ } while (u64_stats_fetch_retry(&stats->syncp, start));
}
/**
@@ -6868,9 +6866,7 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,
ring = READ_ONCE(rings[i]);
if (!ring || !ring->ring_stats)
continue;
- ice_fetch_u64_stats_per_ring(&ring->ring_stats->syncp,
- ring->ring_stats->stats, &pkts,
- &bytes);
+ ice_fetch_u64_stats_per_ring(ring->ring_stats, &pkts, &bytes);
vsi_stats->tx_packets += pkts;
vsi_stats->tx_bytes += bytes;
vsi->tx_restart += ring->ring_stats->tx_stats.restart_q;
@@ -6914,9 +6910,7 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)
struct ice_ring_stats *ring_stats;
ring_stats = ring->ring_stats;
- ice_fetch_u64_stats_per_ring(&ring_stats->syncp,
- ring_stats->stats, &pkts,
- &bytes);
+ ice_fetch_u64_stats_per_ring(ring_stats, &pkts, &bytes);
vsi_stats->rx_packets += pkts;
vsi_stats->rx_bytes += bytes;
vsi->rx_buf_failed += ring_stats->rx_stats.alloc_buf_failed;
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 6/9] ice: remove ice_q_stats struct and use struct_group
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
` (4 preceding siblings ...)
2025-11-07 23:31 ` [PATCH iwl-next v3 5/9] ice: pass pointer to ice_fetch_u64_stats_per_ring Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 7/9] ice: use u64_stats API to access pkts/bytes in dim sample Jacob Keller
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The ice_qp_reset_stats function resets the stats for all rings on a VSI. It
currently behaves differently for Tx and Rx rings. For Rx rings, it only
clears the rx_stats which do not include the pkt and byte counts. For Tx
rings and XDP rings, it clears only the pkt and byte counts.
We could add extra memset calls to cover both the stats and relevant
tx/rx stats fields. Instead, lets convert stats into a struct_group which
contains both the pkts and bytes fields as well as the Tx or Rx stats, and
remove the ice_q_stats structure entirely.
The only remaining user of ice_q_stats is the ice_q_stats_len function in
ice_ethtool.c, which just counts the number of fields. Replace this with a
simple multiplication by 2. I find this to be simpler to reason about than
relying on knowing the layout of the ice_q_stats structure.
Now that the stats field of the ice_ring_stats covers all of the statistic
values, the ice_qp_reset_stats function will properly zero out all of the
fields.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.h | 18 ++++++++----------
drivers/net/ethernet/intel/ice/ice_base.c | 4 ++--
drivers/net/ethernet/intel/ice/ice_ethtool.c | 4 ++--
drivers/net/ethernet/intel/ice/ice_lib.c | 7 ++++---
4 files changed, 16 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index aa0b74e45bba..e0ace99ad876 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -129,11 +129,6 @@ struct ice_tx_offload_params {
u8 header_len;
};
-struct ice_q_stats {
- u64 pkts;
- u64 bytes;
-};
-
struct ice_txq_stats {
u64 restart_q;
u64 tx_busy;
@@ -148,12 +143,15 @@ struct ice_rxq_stats {
struct ice_ring_stats {
struct rcu_head rcu; /* to avoid race on free */
- struct ice_q_stats stats;
struct u64_stats_sync syncp;
- union {
- struct ice_txq_stats tx_stats;
- struct ice_rxq_stats rx_stats;
- };
+ struct_group(stats,
+ u64 pkts;
+ u64 bytes;
+ union {
+ struct ice_txq_stats tx_stats;
+ struct ice_rxq_stats rx_stats;
+ };
+ );
};
enum ice_ring_state_t {
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index eadb1e3d12b3..afbff8aa9ceb 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -1414,8 +1414,8 @@ static void ice_qp_reset_stats(struct ice_vsi *vsi, u16 q_idx)
if (!vsi_stat)
return;
- memset(&vsi_stat->rx_ring_stats[q_idx]->rx_stats, 0,
- sizeof(vsi_stat->rx_ring_stats[q_idx]->rx_stats));
+ memset(&vsi_stat->rx_ring_stats[q_idx]->stats, 0,
+ sizeof(vsi_stat->rx_ring_stats[q_idx]->stats));
memset(&vsi_stat->tx_ring_stats[q_idx]->stats, 0,
sizeof(vsi_stat->tx_ring_stats[q_idx]->stats));
if (vsi->xdp_rings)
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index a1d9abee97e5..0bc6f31a2b06 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -33,8 +33,8 @@ static int ice_q_stats_len(struct net_device *netdev)
{
struct ice_netdev_priv *np = netdev_priv(netdev);
- return ((np->vsi->alloc_txq + np->vsi->alloc_rxq) *
- (sizeof(struct ice_q_stats) / sizeof(u64)));
+ /* One packets and one bytes count per queue */
+ return ((np->vsi->alloc_txq + np->vsi->alloc_rxq) * 2);
}
#define ICE_PF_STATS_LEN ARRAY_SIZE(ice_gstrings_pf_stats)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 46cd8f33c38f..26d17813f426 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -3441,7 +3441,8 @@ int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc)
*
* This function assumes that caller has acquired a u64_stats_sync lock.
*/
-static void ice_update_ring_stats(struct ice_q_stats *stats, u64 pkts, u64 bytes)
+static void ice_update_ring_stats(struct ice_ring_stats *stats,
+ u64 pkts, u64 bytes)
{
stats->bytes += bytes;
stats->pkts += pkts;
@@ -3456,7 +3457,7 @@ static void ice_update_ring_stats(struct ice_q_stats *stats, u64 pkts, u64 bytes
void ice_update_tx_ring_stats(struct ice_tx_ring *tx_ring, u64 pkts, u64 bytes)
{
u64_stats_update_begin(&tx_ring->ring_stats->syncp);
- ice_update_ring_stats(&tx_ring->ring_stats->stats, pkts, bytes);
+ ice_update_ring_stats(tx_ring->ring_stats, pkts, bytes);
u64_stats_update_end(&tx_ring->ring_stats->syncp);
}
@@ -3469,7 +3470,7 @@ void ice_update_tx_ring_stats(struct ice_tx_ring *tx_ring, u64 pkts, u64 bytes)
void ice_update_rx_ring_stats(struct ice_rx_ring *rx_ring, u64 pkts, u64 bytes)
{
u64_stats_update_begin(&rx_ring->ring_stats->syncp);
- ice_update_ring_stats(&rx_ring->ring_stats->stats, pkts, bytes);
+ ice_update_ring_stats(rx_ring->ring_stats, pkts, bytes);
u64_stats_update_end(&rx_ring->ring_stats->syncp);
}
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 7/9] ice: use u64_stats API to access pkts/bytes in dim sample
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
` (5 preceding siblings ...)
2025-11-07 23:31 ` [PATCH iwl-next v3 6/9] ice: remove ice_q_stats struct and use struct_group Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 8/9] ice: shorten ring stat names and add accessors Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 9/9] ice: convert all ring stats to u64_stats_t Jacob Keller
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The __ice_update_sample and __ice_get_ethtool_stats functions directly
accesses the pkts and bytes counters from the ring stats. A following
change is going to update the fields to be u64_stats_t type, and will need
to be accessed appropriately. This will ensure that the accesses do not
cause load/store tearing.
Add helper functions similar to the ones used for updating the stats
values, and use them. This ensures use of the syncp pointer on 32-bit
architectures. Once the fields are updated to u64_stats_t, it will then
properly avoid tears on all architectures.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lib.h | 6 +++++
drivers/net/ethernet/intel/ice/ice_ethtool.c | 26 +++++++++++++-------
drivers/net/ethernet/intel/ice/ice_lib.c | 36 ++++++++++++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_txrx.c | 29 +++++++++++-----------
4 files changed, 75 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 2cb1eb98b9da..49454d98dcfe 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -92,6 +92,12 @@ void ice_update_tx_ring_stats(struct ice_tx_ring *ring, u64 pkts, u64 bytes);
void ice_update_rx_ring_stats(struct ice_rx_ring *ring, u64 pkts, u64 bytes);
+void ice_fetch_tx_ring_stats(const struct ice_tx_ring *ring,
+ u64 *pkts, u64 *bytes);
+
+void ice_fetch_rx_ring_stats(const struct ice_rx_ring *ring,
+ u64 *pkts, u64 *bytes);
+
void ice_write_intrl(struct ice_q_vector *q_vector, u8 intrl);
void ice_write_itr(struct ice_ring_container *rc, u16 itr);
void ice_set_q_vector_intrl(struct ice_q_vector *q_vector);
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index 0bc6f31a2b06..6c93e0e91ef5 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -1942,25 +1942,35 @@ __ice_get_ethtool_stats(struct net_device *netdev,
rcu_read_lock();
ice_for_each_alloc_txq(vsi, j) {
+ u64 pkts, bytes;
+
tx_ring = READ_ONCE(vsi->tx_rings[j]);
- if (tx_ring && tx_ring->ring_stats) {
- data[i++] = tx_ring->ring_stats->stats.pkts;
- data[i++] = tx_ring->ring_stats->stats.bytes;
- } else {
+ if (!tx_ring || !tx_ring->ring_stats) {
data[i++] = 0;
data[i++] = 0;
+ continue;
}
+
+ ice_fetch_tx_ring_stats(tx_ring, &pkts, &bytes);
+
+ data[i++] = pkts;
+ data[i++] = bytes;
}
ice_for_each_alloc_rxq(vsi, j) {
+ u64 pkts, bytes;
+
rx_ring = READ_ONCE(vsi->rx_rings[j]);
- if (rx_ring && rx_ring->ring_stats) {
- data[i++] = rx_ring->ring_stats->stats.pkts;
- data[i++] = rx_ring->ring_stats->stats.bytes;
- } else {
+ if (!rx_ring || !rx_ring->ring_stats) {
data[i++] = 0;
data[i++] = 0;
+ continue;
}
+
+ ice_fetch_rx_ring_stats(rx_ring, &pkts, &bytes);
+
+ data[i++] = pkts;
+ data[i++] = bytes;
}
rcu_read_unlock();
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 26d17813f426..897df9362638 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -3474,6 +3474,42 @@ void ice_update_rx_ring_stats(struct ice_rx_ring *rx_ring, u64 pkts, u64 bytes)
u64_stats_update_end(&rx_ring->ring_stats->syncp);
}
+/**
+ * ice_fetch_tx_ring_stats - Fetch Tx ring packet and byte counters
+ * @ring: ring to update
+ * @pkts: number of processed packets
+ * @bytes: number of processed bytes
+ */
+void ice_fetch_tx_ring_stats(const struct ice_tx_ring *ring,
+ u64 *pkts, u64 *bytes)
+{
+ unsigned int start;
+
+ do {
+ start = u64_stats_fetch_begin(&ring->ring_stats->syncp);
+ *pkts = ring->ring_stats->pkts;
+ *bytes = ring->ring_stats->bytes;
+ } while (u64_stats_fetch_retry(&ring->ring_stats->syncp, start));
+}
+
+/**
+ * ice_fetch_rx_ring_stats - Fetch Rx ring packet and byte counters
+ * @ring: ring to read
+ * @pkts: number of processed packets
+ * @bytes: number of processed bytes
+ */
+void ice_fetch_rx_ring_stats(const struct ice_rx_ring *ring,
+ u64 *pkts, u64 *bytes)
+{
+ unsigned int start;
+
+ do {
+ start = u64_stats_fetch_begin(&ring->ring_stats->syncp);
+ *pkts = ring->ring_stats->pkts;
+ *bytes = ring->ring_stats->bytes;
+ } while (u64_stats_fetch_retry(&ring->ring_stats->syncp, start));
+}
+
/**
* ice_is_dflt_vsi_in_use - check if the default forwarding VSI is being used
* @pi: port info of the switch with default VSI
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index 30073ed9ca99..f0f5133c389f 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -1087,35 +1087,36 @@ static void __ice_update_sample(struct ice_q_vector *q_vector,
struct dim_sample *sample,
bool is_tx)
{
- u64 packets = 0, bytes = 0;
+ u64 total_packets = 0, total_bytes = 0, pkts, bytes;
if (is_tx) {
struct ice_tx_ring *tx_ring;
ice_for_each_tx_ring(tx_ring, *rc) {
- struct ice_ring_stats *ring_stats;
-
- ring_stats = tx_ring->ring_stats;
- if (!ring_stats)
+ if (!tx_ring->ring_stats)
continue;
- packets += ring_stats->stats.pkts;
- bytes += ring_stats->stats.bytes;
+
+ ice_fetch_tx_ring_stats(tx_ring, &pkts, &bytes);
+
+ total_packets += pkts;
+ total_bytes += bytes;
}
} else {
struct ice_rx_ring *rx_ring;
ice_for_each_rx_ring(rx_ring, *rc) {
- struct ice_ring_stats *ring_stats;
-
- ring_stats = rx_ring->ring_stats;
- if (!ring_stats)
+ if (!rx_ring->ring_stats)
continue;
- packets += ring_stats->stats.pkts;
- bytes += ring_stats->stats.bytes;
+
+ ice_fetch_rx_ring_stats(rx_ring, &pkts, &bytes);
+
+ total_packets += pkts;
+ total_bytes += bytes;
}
}
- dim_update_sample(q_vector->total_events, packets, bytes, sample);
+ dim_update_sample(q_vector->total_events,
+ total_packets, total_bytes, sample);
sample->comp_ctr = 0;
/* if dim settings get stale, like when not updated for 1
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 8/9] ice: shorten ring stat names and add accessors
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
` (6 preceding siblings ...)
2025-11-07 23:31 ` [PATCH iwl-next v3 7/9] ice: use u64_stats API to access pkts/bytes in dim sample Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
2025-11-07 23:31 ` [PATCH iwl-next v3 9/9] ice: convert all ring stats to u64_stats_t Jacob Keller
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
The ice Tx/Rx hotpath has a few statistics counters for tracking unexpected
events. These values are stored as u64 but are not accumulated using the
u64_stats API. This could result in load/tear stores on some architectures.
Even some 64-bit architectures could have issues since the fields are not
read or written using ACCESS_ONCE or READ_ONCE.
A following change is going to refactor the stats accumulator code to use
the u64_stats API for all of these stats, and to use u64_stats_read and
u64_stats_inc properly to prevent load/store tears on all architectures.
Using u64_stats_inc and the syncp pointer is slightly verbose and would be
duplicated in a number of places in the Tx and Rx hot path. Add accessor
macros for the cases where only a single stat value is touched at once. To
keep lines short, also shorten the stats names and convert ice_txq_stats
and ice_rxq_stats to struct_group.
This will ease the transition to properly using the u64_stats API in the
following change.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.h | 52 +++++++++++++++++++--------
drivers/net/ethernet/intel/ice/ice_txrx_lib.h | 2 +-
drivers/net/ethernet/intel/ice/ice_main.c | 12 +++----
drivers/net/ethernet/intel/ice/ice_txrx.c | 14 ++++----
drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 2 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 4 +--
6 files changed, 55 insertions(+), 31 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index e0ace99ad876..227b75c941fc 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -129,18 +129,6 @@ struct ice_tx_offload_params {
u8 header_len;
};
-struct ice_txq_stats {
- u64 restart_q;
- u64 tx_busy;
- u64 tx_linearize;
-};
-
-struct ice_rxq_stats {
- u64 non_eop_descs;
- u64 alloc_page_failed;
- u64 alloc_buf_failed;
-};
-
struct ice_ring_stats {
struct rcu_head rcu; /* to avoid race on free */
struct u64_stats_sync syncp;
@@ -148,12 +136,48 @@ struct ice_ring_stats {
u64 pkts;
u64 bytes;
union {
- struct ice_txq_stats tx_stats;
- struct ice_rxq_stats rx_stats;
+ struct_group(tx,
+ u64 tx_restart_q;
+ u64 tx_busy;
+ u64 tx_linearize;
+ );
+ struct_group(rx,
+ u64 rx_non_eop_descs;
+ u64 rx_page_failed;
+ u64 rx_buf_failed;
+ );
};
);
};
+/**
+ * ice_stats_read - Read a single ring stat value
+ * @stats: pointer to ring_stats structure for a queue
+ * @member: the ice_ring_stats member to read
+ *
+ * Shorthand for reading a single 64-bit stat value from struct
+ * ice_ring_stats.
+ *
+ * Return: the value of the requested stat.
+ */
+#define ice_stats_read(stats, member) ({ \
+ struct ice_ring_stats *__stats = (stats); \
+ __stats->member; \
+})
+
+/**
+ * ice_stats_inc - Increment a single ring stat value
+ * @stats: pointer to the ring_stats structure for a queue
+ * @member: the ice_ring_stats member to increment
+ *
+ * Shorthand for incrementing a single 64-bit stat value in struct
+ * ice_ring_stats.
+ */
+#define ice_stats_inc(stats, member) do { \
+ struct ice_ring_stats *__stats = (stats); \
+ __stats->member++; \
+} while (0)
+
enum ice_ring_state_t {
ICE_TX_XPS_INIT_DONE,
ICE_TX_NBITS,
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.h b/drivers/net/ethernet/intel/ice/ice_txrx_lib.h
index 6a3f10f7a53f..f17990b68b62 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.h
@@ -38,7 +38,7 @@ ice_is_non_eop(const struct ice_rx_ring *rx_ring,
if (likely(ice_test_staterr(rx_desc->wb.status_error0, ICE_RXD_EOF)))
return false;
- rx_ring->ring_stats->rx_stats.non_eop_descs++;
+ ice_stats_inc(rx_ring->ring_stats, rx_non_eop_descs);
return true;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 5a3bcbb5f63c..885e85f478d8 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -159,7 +159,7 @@ static void ice_check_for_hang_subtask(struct ice_pf *pf)
* prev_pkt would be negative if there was no
* pending work.
*/
- packets = ring_stats->stats.pkts & INT_MAX;
+ packets = ice_stats_read(ring_stats, pkts) & INT_MAX;
if (tx_ring->prev_pkt == packets) {
/* Trigger sw interrupt to revive the queue */
ice_trigger_sw_intr(hw, tx_ring->q_vector);
@@ -6869,9 +6869,9 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,
ice_fetch_u64_stats_per_ring(ring->ring_stats, &pkts, &bytes);
vsi_stats->tx_packets += pkts;
vsi_stats->tx_bytes += bytes;
- vsi->tx_restart += ring->ring_stats->tx_stats.restart_q;
- vsi->tx_busy += ring->ring_stats->tx_stats.tx_busy;
- vsi->tx_linearize += ring->ring_stats->tx_stats.tx_linearize;
+ vsi->tx_restart += ring->ring_stats->tx_restart_q;
+ vsi->tx_busy += ring->ring_stats->tx_busy;
+ vsi->tx_linearize += ring->ring_stats->tx_linearize;
}
}
@@ -6913,8 +6913,8 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)
ice_fetch_u64_stats_per_ring(ring_stats, &pkts, &bytes);
vsi_stats->rx_packets += pkts;
vsi_stats->rx_bytes += bytes;
- vsi->rx_buf_failed += ring_stats->rx_stats.alloc_buf_failed;
- vsi->rx_page_failed += ring_stats->rx_stats.alloc_page_failed;
+ vsi->rx_buf_failed += ring_stats->rx_buf_failed;
+ vsi->rx_page_failed += ring_stats->rx_page_failed;
}
/* update XDP Tx rings counters */
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index f0f5133c389f..ddd40c87772a 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -379,7 +379,7 @@ static bool ice_clean_tx_irq(struct ice_tx_ring *tx_ring, int napi_budget)
if (netif_tx_queue_stopped(txring_txq(tx_ring)) &&
!test_bit(ICE_VSI_DOWN, vsi->state)) {
netif_tx_wake_queue(txring_txq(tx_ring));
- ++tx_ring->ring_stats->tx_stats.restart_q;
+ ice_stats_inc(tx_ring->ring_stats, tx_restart_q);
}
}
@@ -849,7 +849,7 @@ bool ice_alloc_rx_bufs(struct ice_rx_ring *rx_ring, unsigned int cleaned_count)
addr = libeth_rx_alloc(&fq, ntu);
if (addr == DMA_MAPPING_ERROR) {
- rx_ring->ring_stats->rx_stats.alloc_page_failed++;
+ ice_stats_inc(rx_ring->ring_stats, rx_page_failed);
break;
}
@@ -863,7 +863,7 @@ bool ice_alloc_rx_bufs(struct ice_rx_ring *rx_ring, unsigned int cleaned_count)
addr = libeth_rx_alloc(&hdr_fq, ntu);
if (addr == DMA_MAPPING_ERROR) {
- rx_ring->ring_stats->rx_stats.alloc_page_failed++;
+ ice_stats_inc(rx_ring->ring_stats, rx_page_failed);
libeth_rx_recycle_slow(fq.fqes[ntu].netmem);
break;
@@ -1045,7 +1045,7 @@ static int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget)
/* exit if we failed to retrieve a buffer */
if (!skb) {
libeth_xdp_return_buff_slow(xdp);
- rx_ring->ring_stats->rx_stats.alloc_buf_failed++;
+ ice_stats_inc(rx_ring->ring_stats, rx_buf_failed);
continue;
}
@@ -1363,7 +1363,7 @@ static int __ice_maybe_stop_tx(struct ice_tx_ring *tx_ring, unsigned int size)
/* A reprieve! - use start_queue because it doesn't call schedule */
netif_tx_start_queue(txring_txq(tx_ring));
- ++tx_ring->ring_stats->tx_stats.restart_q;
+ ice_stats_inc(tx_ring->ring_stats, tx_restart_q);
return 0;
}
@@ -2165,7 +2165,7 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)
if (__skb_linearize(skb))
goto out_drop;
count = ice_txd_use_count(skb->len);
- tx_ring->ring_stats->tx_stats.tx_linearize++;
+ ice_stats_inc(tx_ring->ring_stats, tx_linearize);
}
/* need: 1 descriptor per page * PAGE_SIZE/ICE_MAX_DATA_PER_TXD,
@@ -2176,7 +2176,7 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)
*/
if (ice_maybe_stop_tx(tx_ring, count + ICE_DESCS_PER_CACHE_LINE +
ICE_DESCS_FOR_CTX_DESC)) {
- tx_ring->ring_stats->tx_stats.tx_busy++;
+ ice_stats_inc(tx_ring->ring_stats, tx_busy);
return NETDEV_TX_BUSY;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
index 956da38d63b0..e68f3e5d35b4 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c
@@ -480,7 +480,7 @@ int __ice_xmit_xdp_ring(struct xdp_buff *xdp, struct ice_tx_ring *xdp_ring,
return ICE_XDP_CONSUMED;
busy:
- xdp_ring->ring_stats->tx_stats.tx_busy++;
+ ice_stats_inc(xdp_ring->ring_stats, tx_busy);
return ICE_XDP_CONSUMED;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 989ff1fd9110..953e68ed0f9a 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -497,7 +497,7 @@ static int ice_xmit_xdp_tx_zc(struct xdp_buff *xdp,
return ICE_XDP_TX;
busy:
- xdp_ring->ring_stats->tx_stats.tx_busy++;
+ ice_stats_inc(xdp_ring->ring_stats, tx_busy);
return ICE_XDP_CONSUMED;
}
@@ -659,7 +659,7 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring,
xsk_buff_free(first);
first = NULL;
- rx_ring->ring_stats->rx_stats.alloc_buf_failed++;
+ ice_stats_inc(rx_ring->ring_stats, rx_buf_failed);
continue;
}
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH iwl-next v3 9/9] ice: convert all ring stats to u64_stats_t
2025-11-07 23:31 [PATCH iwl-next v3 0/9] ice: properly use u64_stats API for all ring stats Jacob Keller
` (7 preceding siblings ...)
2025-11-07 23:31 ` [PATCH iwl-next v3 8/9] ice: shorten ring stat names and add accessors Jacob Keller
@ 2025-11-07 23:31 ` Jacob Keller
8 siblings, 0 replies; 10+ messages in thread
From: Jacob Keller @ 2025-11-07 23:31 UTC (permalink / raw)
To: Aleksandr Loktionov, Alexander Lobakin, Tony Nguyen,
Przemek Kitszel
Cc: Simon Horman, intel-wired-lan, netdev, Jacob Keller,
Aleksandr Loktionov
After several cleanups, the ice driver is now finally ready to convert all
Tx and Rx ring stats to the u64_stats_t and proper use of the u64 stats
APIs.
The final remaining part to cleanup is the VSI stats accumulation logic in
ice_update_vsi_ring_stats().
Refactor the function and its helpers so that all stat values (and not
just pkts and bytes) use the u64_stats APIs. The
ice_fetch_u64_(tx|rx)_stats functions read the stat values using
u64_stats_read and then copy them into local ice_vsi_(tx|rx)_stats
structures. This does require making a new struct with the stat fields as
u64.
The ice_update_vsi_(tx|rx)_ring_stats functions call the fetch functions
per ring and accumulate the result into one copy of the struct. This
accumulated total is then used to update the relevant VSI fields.
Since these are relatively small, the contents are all stored on the stack
rather than allocating and freeing memory.
Once the accumulator side is updated, the helper ice_stats_read and
ice_stats_inc and other related helper functions all easily translate to
use of u64_stats_read and u64_stats_inc. This completes the refactor and
ensures that all stats accesses now make proper use of the API.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
drivers/net/ethernet/intel/ice/ice_txrx.h | 28 +++--
drivers/net/ethernet/intel/ice/ice_lib.c | 29 ++---
drivers/net/ethernet/intel/ice/ice_main.c | 180 ++++++++++++++++++++----------
3 files changed, 147 insertions(+), 90 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index 227b75c941fc..1dcfc77a2ca7 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -133,18 +133,18 @@ struct ice_ring_stats {
struct rcu_head rcu; /* to avoid race on free */
struct u64_stats_sync syncp;
struct_group(stats,
- u64 pkts;
- u64 bytes;
+ u64_stats_t pkts;
+ u64_stats_t bytes;
union {
struct_group(tx,
- u64 tx_restart_q;
- u64 tx_busy;
- u64 tx_linearize;
+ u64_stats_t tx_restart_q;
+ u64_stats_t tx_busy;
+ u64_stats_t tx_linearize;
);
struct_group(rx,
- u64 rx_non_eop_descs;
- u64 rx_page_failed;
- u64 rx_buf_failed;
+ u64_stats_t rx_non_eop_descs;
+ u64_stats_t rx_page_failed;
+ u64_stats_t rx_buf_failed;
);
};
);
@@ -162,7 +162,13 @@ struct ice_ring_stats {
*/
#define ice_stats_read(stats, member) ({ \
struct ice_ring_stats *__stats = (stats); \
- __stats->member; \
+ unsigned int start; \
+ u64 val; \
+ do { \
+ start = u64_stats_fetch_begin(&__stats->syncp); \
+ val = u64_stats_read(&__stats->member); \
+ } while (u64_stats_fetch_retry(&__stats->syncp, start)); \
+ val; \
})
/**
@@ -175,7 +181,9 @@ struct ice_ring_stats {
*/
#define ice_stats_inc(stats, member) do { \
struct ice_ring_stats *__stats = (stats); \
- __stats->member++; \
+ u64_stats_update_begin(&__stats->syncp); \
+ u64_stats_inc(&__stats->member); \
+ u64_stats_update_end(&__stats->syncp); \
} while (0)
enum ice_ring_state_t {
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 897df9362638..6923fafdbcbf 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -3433,21 +3433,6 @@ int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc)
return ret;
}
-/**
- * ice_update_ring_stats - Update ring statistics
- * @stats: stats to be updated
- * @pkts: number of processed packets
- * @bytes: number of processed bytes
- *
- * This function assumes that caller has acquired a u64_stats_sync lock.
- */
-static void ice_update_ring_stats(struct ice_ring_stats *stats,
- u64 pkts, u64 bytes)
-{
- stats->bytes += bytes;
- stats->pkts += pkts;
-}
-
/**
* ice_update_tx_ring_stats - Update Tx ring specific counters
* @tx_ring: ring to update
@@ -3457,7 +3442,8 @@ static void ice_update_ring_stats(struct ice_ring_stats *stats,
void ice_update_tx_ring_stats(struct ice_tx_ring *tx_ring, u64 pkts, u64 bytes)
{
u64_stats_update_begin(&tx_ring->ring_stats->syncp);
- ice_update_ring_stats(tx_ring->ring_stats, pkts, bytes);
+ u64_stats_add(&tx_ring->ring_stats->pkts, pkts);
+ u64_stats_add(&tx_ring->ring_stats->bytes, bytes);
u64_stats_update_end(&tx_ring->ring_stats->syncp);
}
@@ -3470,7 +3456,8 @@ void ice_update_tx_ring_stats(struct ice_tx_ring *tx_ring, u64 pkts, u64 bytes)
void ice_update_rx_ring_stats(struct ice_rx_ring *rx_ring, u64 pkts, u64 bytes)
{
u64_stats_update_begin(&rx_ring->ring_stats->syncp);
- ice_update_ring_stats(rx_ring->ring_stats, pkts, bytes);
+ u64_stats_add(&rx_ring->ring_stats->pkts, pkts);
+ u64_stats_add(&rx_ring->ring_stats->bytes, bytes);
u64_stats_update_end(&rx_ring->ring_stats->syncp);
}
@@ -3487,8 +3474,8 @@ void ice_fetch_tx_ring_stats(const struct ice_tx_ring *ring,
do {
start = u64_stats_fetch_begin(&ring->ring_stats->syncp);
- *pkts = ring->ring_stats->pkts;
- *bytes = ring->ring_stats->bytes;
+ *pkts = u64_stats_read(&ring->ring_stats->pkts);
+ *bytes = u64_stats_read(&ring->ring_stats->bytes);
} while (u64_stats_fetch_retry(&ring->ring_stats->syncp, start));
}
@@ -3505,8 +3492,8 @@ void ice_fetch_rx_ring_stats(const struct ice_rx_ring *ring,
do {
start = u64_stats_fetch_begin(&ring->ring_stats->syncp);
- *pkts = ring->ring_stats->pkts;
- *bytes = ring->ring_stats->bytes;
+ *pkts = u64_stats_read(&ring->ring_stats->pkts);
+ *bytes = u64_stats_read(&ring->ring_stats->bytes);
} while (u64_stats_fetch_retry(&ring->ring_stats->syncp, start));
}
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 885e85f478d8..5f63799de75e 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -6824,54 +6824,132 @@ int ice_up(struct ice_vsi *vsi)
return err;
}
+struct ice_vsi_tx_stats {
+ u64 pkts;
+ u64 bytes;
+ u64 tx_restart_q;
+ u64 tx_busy;
+ u64 tx_linearize;
+};
+
+struct ice_vsi_rx_stats {
+ u64 pkts;
+ u64 bytes;
+ u64 rx_non_eop_descs;
+ u64 rx_page_failed;
+ u64 rx_buf_failed;
+};
+
/**
- * ice_fetch_u64_stats_per_ring - get packets and bytes stats per ring
- * @stats: pointer to ring stats structure
- * @pkts: packets stats counter
- * @bytes: bytes stats counter
+ * ice_fetch_u64_tx_stats - get Tx stats from a ring
+ * @ring: the Tx ring to copy stats from
+ * @copy: temporary storage for the ring statistics
*
- * This function fetches stats from the ring considering the atomic operations
- * that needs to be performed to read u64 values in 32 bit machine.
+ * Fetch the u64 stats from the ring using u64_stats_fetch. This ensures each
+ * stat value is self-consistent, though not necessarily consistent w.r.t
+ * other stats.
*/
-static void ice_fetch_u64_stats_per_ring(struct ice_ring_stats *stats,
- u64 *pkts, u64 *bytes)
+static void ice_fetch_u64_tx_stats(struct ice_tx_ring *ring,
+ struct ice_vsi_tx_stats *copy)
{
+ struct ice_ring_stats *stats = ring->ring_stats;
unsigned int start;
do {
start = u64_stats_fetch_begin(&stats->syncp);
- *pkts = stats->stats.pkts;
- *bytes = stats->stats.bytes;
+ copy->pkts = u64_stats_read(&stats->pkts);
+ copy->bytes = u64_stats_read(&stats->bytes);
+ copy->tx_restart_q = u64_stats_read(&stats->tx_restart_q);
+ copy->tx_busy = u64_stats_read(&stats->tx_busy);
+ copy->tx_linearize = u64_stats_read(&stats->tx_linearize);
+ } while (u64_stats_fetch_retry(&stats->syncp, start));
+}
+
+/**
+ * ice_fetch_u64_rx_stats - get Rx stats from a ring
+ * @ring: the Rx ring to copy stats from
+ * @copy: temporary storage for the ring statistics
+ *
+ * Fetch the u64 stats from the ring using u64_stats_fetch. This ensures each
+ * stat value is self-consistent, though not necessarily consistent w.r.t
+ * other stats.
+ */
+static void ice_fetch_u64_rx_stats(struct ice_rx_ring *ring,
+ struct ice_vsi_rx_stats *copy)
+{
+ struct ice_ring_stats *stats = ring->ring_stats;
+ unsigned int start;
+
+ do {
+ start = u64_stats_fetch_begin(&stats->syncp);
+ copy->pkts = u64_stats_read(&stats->pkts);
+ copy->bytes = u64_stats_read(&stats->bytes);
+ copy->rx_non_eop_descs =
+ u64_stats_read(&stats->rx_non_eop_descs);
+ copy->rx_page_failed = u64_stats_read(&stats->rx_page_failed);
+ copy->rx_buf_failed = u64_stats_read(&stats->rx_buf_failed);
} while (u64_stats_fetch_retry(&stats->syncp, start));
}
/**
* ice_update_vsi_tx_ring_stats - Update VSI Tx ring stats counters
* @vsi: the VSI to be updated
- * @vsi_stats: the stats struct to be updated
+ * @vsi_stats: accumulated stats for this VSI
* @rings: rings to work on
* @count: number of rings
*/
-static void
-ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,
- struct rtnl_link_stats64 *vsi_stats,
- struct ice_tx_ring **rings, u16 count)
+static void ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,
+ struct ice_vsi_tx_stats *vsi_stats,
+ struct ice_tx_ring **rings, u16 count)
{
+ struct ice_vsi_tx_stats copy = {};
u16 i;
for (i = 0; i < count; i++) {
struct ice_tx_ring *ring;
- u64 pkts = 0, bytes = 0;
ring = READ_ONCE(rings[i]);
if (!ring || !ring->ring_stats)
continue;
- ice_fetch_u64_stats_per_ring(ring->ring_stats, &pkts, &bytes);
- vsi_stats->tx_packets += pkts;
- vsi_stats->tx_bytes += bytes;
- vsi->tx_restart += ring->ring_stats->tx_restart_q;
- vsi->tx_busy += ring->ring_stats->tx_busy;
- vsi->tx_linearize += ring->ring_stats->tx_linearize;
+
+ ice_fetch_u64_tx_stats(ring, ©);
+
+ vsi_stats->pkts += copy.pkts;
+ vsi_stats->bytes += copy.bytes;
+ vsi_stats->tx_restart_q += copy.tx_restart_q;
+ vsi_stats->tx_busy += copy.tx_busy;
+ vsi_stats->tx_linearize += copy.tx_linearize;
+ }
+}
+
+/**
+ * ice_update_vsi_rx_ring_stats - Update VSI Rx ring stats counters
+ * @vsi: the VSI to be updated
+ * @vsi_stats: accumulated stats for this VSI
+ * @rings: rings to work on
+ * @count: number of rings
+ */
+static void ice_update_vsi_rx_ring_stats(struct ice_vsi *vsi,
+ struct ice_vsi_rx_stats *vsi_stats,
+ struct ice_rx_ring **rings, u16 count)
+{
+ struct ice_vsi_rx_stats copy = {};
+ u16 i;
+
+ for (i = 0; i < count; i++) {
+ struct ice_rx_ring *ring;
+
+ ring = READ_ONCE(rings[i]);
+ if (!ring || !ring->ring_stats)
+ continue;
+
+ ice_fetch_u64_rx_stats(ring, ©);
+
+ vsi_stats->pkts += copy.pkts;
+ vsi_stats->bytes += copy.bytes;
+ vsi_stats->rx_non_eop_descs += copy.rx_non_eop_descs;
+ vsi_stats->rx_page_failed += copy.rx_page_failed;
+ vsi_stats->rx_buf_failed += copy.rx_buf_failed;
}
}
@@ -6882,48 +6960,34 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,
static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)
{
struct rtnl_link_stats64 *net_stats, *stats_prev;
- struct rtnl_link_stats64 *vsi_stats;
+ struct ice_vsi_tx_stats tx_stats = {};
+ struct ice_vsi_rx_stats rx_stats = {};
struct ice_pf *pf = vsi->back;
- u64 pkts, bytes;
- int i;
-
- vsi_stats = kzalloc(sizeof(*vsi_stats), GFP_ATOMIC);
- if (!vsi_stats)
- return;
-
- /* reset non-netdev (extended) stats */
- vsi->tx_restart = 0;
- vsi->tx_busy = 0;
- vsi->tx_linearize = 0;
- vsi->rx_buf_failed = 0;
- vsi->rx_page_failed = 0;
rcu_read_lock();
/* update Tx rings counters */
- ice_update_vsi_tx_ring_stats(vsi, vsi_stats, vsi->tx_rings,
+ ice_update_vsi_tx_ring_stats(vsi, &tx_stats, vsi->tx_rings,
vsi->num_txq);
/* update Rx rings counters */
- ice_for_each_rxq(vsi, i) {
- struct ice_rx_ring *ring = READ_ONCE(vsi->rx_rings[i]);
- struct ice_ring_stats *ring_stats;
-
- ring_stats = ring->ring_stats;
- ice_fetch_u64_stats_per_ring(ring_stats, &pkts, &bytes);
- vsi_stats->rx_packets += pkts;
- vsi_stats->rx_bytes += bytes;
- vsi->rx_buf_failed += ring_stats->rx_buf_failed;
- vsi->rx_page_failed += ring_stats->rx_page_failed;
- }
+ ice_update_vsi_rx_ring_stats(vsi, &rx_stats, vsi->rx_rings,
+ vsi->num_rxq);
/* update XDP Tx rings counters */
if (ice_is_xdp_ena_vsi(vsi))
- ice_update_vsi_tx_ring_stats(vsi, vsi_stats, vsi->xdp_rings,
+ ice_update_vsi_tx_ring_stats(vsi, &tx_stats, vsi->xdp_rings,
vsi->num_xdp_txq);
rcu_read_unlock();
+ /* Save non-netdev (extended) stats */
+ vsi->tx_restart = tx_stats.tx_restart_q;
+ vsi->tx_busy = tx_stats.tx_busy;
+ vsi->tx_linearize = tx_stats.tx_linearize;
+ vsi->rx_buf_failed = rx_stats.rx_buf_failed;
+ vsi->rx_page_failed = rx_stats.rx_page_failed;
+
net_stats = &vsi->net_stats;
stats_prev = &vsi->net_stats_prev;
@@ -6933,18 +6997,16 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)
* let's skip this round.
*/
if (likely(pf->stat_prev_loaded)) {
- net_stats->tx_packets += vsi_stats->tx_packets - stats_prev->tx_packets;
- net_stats->tx_bytes += vsi_stats->tx_bytes - stats_prev->tx_bytes;
- net_stats->rx_packets += vsi_stats->rx_packets - stats_prev->rx_packets;
- net_stats->rx_bytes += vsi_stats->rx_bytes - stats_prev->rx_bytes;
+ net_stats->tx_packets += tx_stats.pkts - stats_prev->tx_packets;
+ net_stats->tx_bytes += tx_stats.bytes - stats_prev->tx_bytes;
+ net_stats->rx_packets += rx_stats.pkts - stats_prev->rx_packets;
+ net_stats->rx_bytes += rx_stats.bytes - stats_prev->rx_bytes;
}
- stats_prev->tx_packets = vsi_stats->tx_packets;
- stats_prev->tx_bytes = vsi_stats->tx_bytes;
- stats_prev->rx_packets = vsi_stats->rx_packets;
- stats_prev->rx_bytes = vsi_stats->rx_bytes;
-
- kfree(vsi_stats);
+ stats_prev->tx_packets = tx_stats.pkts;
+ stats_prev->tx_bytes = tx_stats.bytes;
+ stats_prev->rx_packets = rx_stats.pkts;
+ stats_prev->rx_bytes = rx_stats.bytes;
}
/**
--
2.51.0.rc1.197.g6d975e95c9d7
^ permalink raw reply related [flat|nested] 10+ messages in thread