From: Saeed Mahameed <saeed@kernel.org>
To: "David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Eric Dumazet <edumazet@google.com>
Cc: Saeed Mahameed <saeedm@nvidia.com>,
netdev@vger.kernel.org, Tariq Toukan <tariqt@nvidia.com>,
Adham Faris <afaris@nvidia.com>, Shay Drory <shayd@nvidia.com>
Subject: [net-next 03/15] net/mlx5e: aRFS, Introduce ethtool stats
Date: Wed, 16 Aug 2023 14:00:37 -0700 [thread overview]
Message-ID: <20230816210049.54733-4-saeed@kernel.org> (raw)
In-Reply-To: <20230816210049.54733-1-saeed@kernel.org>
From: Adham Faris <afaris@nvidia.com>
Improve aRFS observability by adding new set of counters. Each Rx
ring will have this set of counters listed below.
These counters are exposed through ethtool -S.
1) arfs_add: number of times a new rule has been created.
2) arfs_request_in: number of times a rule was requested to move from
its current Rx ring to a new Rx ring (incremented on the destination
Rx ring).
3) arfs_request_out: number of times a rule was requested to move out
from its current Rx ring (incremented on source/current Rx ring).
4) arfs_expired: number of times a rule has been expired by the
kernel and removed from HW.
5) arfs_err: number of times a rule creation or modification has
failed.
This patch removes rx[i]_xsk_arfs_err counter and its documentation in
mlx5/counters.rst since aRFS activity does not occur in XSK RQ's.
Signed-off-by: Adham Faris <afaris@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
Reviewed-by: Shay Drory <shayd@nvidia.com>
---
.../ethernet/mellanox/mlx5/counters.rst | 23 +++++++++++++++----
.../net/ethernet/mellanox/mlx5/core/en_arfs.c | 17 +++++++++++---
.../ethernet/mellanox/mlx5/core/en_stats.c | 22 ++++++++++++++----
.../ethernet/mellanox/mlx5/core/en_stats.h | 13 ++++++++++-
4 files changed, 62 insertions(+), 13 deletions(-)
diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst
index 008e560e12b5..f69ee1ebee01 100644
--- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst
+++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst
@@ -346,6 +346,24 @@ the software port.
- The number of receive packets with CQE compression on ring i [#accel]_.
- Acceleration
+ * - `rx[i]_arfs_add`
+ - The number of aRFS flow rules added to the device for direct RQ steering
+ on ring i [#accel]_.
+ - Acceleration
+
+ * - `rx[i]_arfs_request_in`
+ - Number of flow rules that have been requested to move into ring i for
+ direct RQ steering [#accel]_.
+ - Acceleration
+
+ * - `rx[i]_arfs_request_out`
+ - Number of flow rules that have been requested to move out of ring i [#accel]_.
+ - Acceleration
+
+ * - `rx[i]_arfs_expired`
+ - Number of flow rules that have been expired and removed [#accel]_.
+ - Acceleration
+
* - `rx[i]_arfs_err`
- Number of flow rules that failed to be added to the flow table.
- Error
@@ -445,11 +463,6 @@ the software port.
context.
- Error
- * - `rx[i]_xsk_arfs_err`
- - aRFS (accelerated Receive Flow Steering) does not occur in the XSK RQ
- context, so this counter should never increment.
- - Error
-
* - `rx[i]_xdp_tx_xmit`
- The number of packets forwarded back to the port due to XDP program
`XDP_TX` action (bouncing). these packets are not counted by other
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
index e8b0acf7d454..bb7f86c993e5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c
@@ -432,8 +432,10 @@ static void arfs_may_expire_flow(struct mlx5e_priv *priv)
}
spin_unlock_bh(&arfs->arfs_lock);
hlist_for_each_entry_safe(arfs_rule, htmp, &del_list, hlist) {
- if (arfs_rule->rule)
+ if (arfs_rule->rule) {
mlx5_del_flow_rules(arfs_rule->rule);
+ priv->channel_stats[arfs_rule->rxq]->rq.arfs_expired++;
+ }
hlist_del(&arfs_rule->hlist);
kfree(arfs_rule);
}
@@ -509,6 +511,7 @@ static struct mlx5_flow_handle *arfs_add_rule(struct mlx5e_priv *priv,
spec = kvzalloc(sizeof(*spec), GFP_KERNEL);
if (!spec) {
+ priv->channel_stats[arfs_rule->rxq]->rq.arfs_err++;
err = -ENOMEM;
goto out;
}
@@ -602,9 +605,11 @@ static void arfs_modify_rule_rq(struct mlx5e_priv *priv,
dst.type = MLX5_FLOW_DESTINATION_TYPE_TIR;
dst.tir_num = mlx5e_rx_res_get_tirn_direct(priv->rx_res, rxq);
err = mlx5_modify_rule_destination(rule, &dst, NULL);
- if (err)
+ if (err) {
+ priv->channel_stats[rxq]->rq.arfs_err++;
netdev_warn(priv->netdev,
"Failed to modify aRFS rule destination to rq=%d\n", rxq);
+ }
}
static void arfs_handle_work(struct work_struct *work)
@@ -634,6 +639,7 @@ static void arfs_handle_work(struct work_struct *work)
if (IS_ERR(rule))
goto out;
arfs_rule->rule = rule;
+ priv->channel_stats[arfs_rule->rxq]->rq.arfs_add++;
} else {
arfs_modify_rule_rq(priv, arfs_rule->rule,
arfs_rule->rxq);
@@ -652,8 +658,10 @@ static struct arfs_rule *arfs_alloc_rule(struct mlx5e_priv *priv,
struct arfs_tuple *tuple;
rule = kzalloc(sizeof(*rule), GFP_ATOMIC);
- if (!rule)
+ if (!rule) {
+ priv->channel_stats[rxq]->rq.arfs_err++;
return NULL;
+ }
rule->priv = priv;
rule->rxq = rxq;
@@ -746,6 +754,9 @@ int mlx5e_rx_flow_steer(struct net_device *dev, const struct sk_buff *skb,
spin_unlock_bh(&arfs->arfs_lock);
return arfs_rule->filter_id;
}
+
+ priv->channel_stats[rxq_index]->rq.arfs_request_in++;
+ priv->channel_stats[arfs_rule->rxq]->rq.arfs_request_out++;
arfs_rule->rxq = rxq_index;
} else {
arfs_rule = arfs_alloc_rule(priv, arfs_t, &fk, rxq_index, flow_id);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 52141729444e..348fb140858c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -180,7 +180,13 @@ static const struct counter_desc sw_stats_desc[] = {
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_cqe_compress_blks) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_cqe_compress_pkts) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_congst_umr) },
+#ifdef CONFIG_MLX5_EN_ARFS
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_arfs_add) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_arfs_request_in) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_arfs_request_out) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_arfs_expired) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_arfs_err) },
+#endif
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_recover) },
#ifdef CONFIG_PAGE_POOL_STATS
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_pp_alloc_fast) },
@@ -231,7 +237,6 @@ static const struct counter_desc sw_stats_desc[] = {
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_xsk_cqe_compress_blks) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_xsk_cqe_compress_pkts) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_xsk_congst_umr) },
- { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_xsk_arfs_err) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_xsk_xmit) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_xsk_mpwqe) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_xsk_inlnw) },
@@ -321,7 +326,6 @@ static void mlx5e_stats_grp_sw_update_stats_xskrq(struct mlx5e_sw_stats *s,
s->rx_xsk_cqe_compress_blks += xskrq_stats->cqe_compress_blks;
s->rx_xsk_cqe_compress_pkts += xskrq_stats->cqe_compress_pkts;
s->rx_xsk_congst_umr += xskrq_stats->congst_umr;
- s->rx_xsk_arfs_err += xskrq_stats->arfs_err;
}
static void mlx5e_stats_grp_sw_update_stats_rq_stats(struct mlx5e_sw_stats *s,
@@ -354,7 +358,13 @@ static void mlx5e_stats_grp_sw_update_stats_rq_stats(struct mlx5e_sw_stats *s,
s->rx_cqe_compress_blks += rq_stats->cqe_compress_blks;
s->rx_cqe_compress_pkts += rq_stats->cqe_compress_pkts;
s->rx_congst_umr += rq_stats->congst_umr;
+#ifdef CONFIG_MLX5_EN_ARFS
+ s->rx_arfs_add += rq_stats->arfs_add;
+ s->rx_arfs_request_in += rq_stats->arfs_request_in;
+ s->rx_arfs_request_out += rq_stats->arfs_request_out;
+ s->rx_arfs_expired += rq_stats->arfs_expired;
s->rx_arfs_err += rq_stats->arfs_err;
+#endif
s->rx_recover += rq_stats->recover;
#ifdef CONFIG_PAGE_POOL_STATS
s->rx_pp_alloc_fast += rq_stats->pp_alloc_fast;
@@ -1990,7 +2000,13 @@ static const struct counter_desc rq_stats_desc[] = {
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, cqe_compress_blks) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, cqe_compress_pkts) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, congst_umr) },
+#ifdef CONFIG_MLX5_EN_ARFS
+ { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, arfs_add) },
+ { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, arfs_request_in) },
+ { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, arfs_request_out) },
+ { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, arfs_expired) },
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, arfs_err) },
+#endif
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, recover) },
#ifdef CONFIG_PAGE_POOL_STATS
{ MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, pp_alloc_fast) },
@@ -2092,7 +2108,6 @@ static const struct counter_desc xskrq_stats_desc[] = {
{ MLX5E_DECLARE_XSKRQ_STAT(struct mlx5e_rq_stats, cqe_compress_blks) },
{ MLX5E_DECLARE_XSKRQ_STAT(struct mlx5e_rq_stats, cqe_compress_pkts) },
{ MLX5E_DECLARE_XSKRQ_STAT(struct mlx5e_rq_stats, congst_umr) },
- { MLX5E_DECLARE_XSKRQ_STAT(struct mlx5e_rq_stats, arfs_err) },
};
static const struct counter_desc xsksq_stats_desc[] = {
@@ -2168,7 +2183,6 @@ static const struct counter_desc ptp_rq_stats_desc[] = {
{ MLX5E_DECLARE_PTP_RQ_STAT(struct mlx5e_rq_stats, cqe_compress_blks) },
{ MLX5E_DECLARE_PTP_RQ_STAT(struct mlx5e_rq_stats, cqe_compress_pkts) },
{ MLX5E_DECLARE_PTP_RQ_STAT(struct mlx5e_rq_stats, congst_umr) },
- { MLX5E_DECLARE_PTP_RQ_STAT(struct mlx5e_rq_stats, arfs_err) },
{ MLX5E_DECLARE_PTP_RQ_STAT(struct mlx5e_rq_stats, recover) },
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
index 409e9a47e433..176fa5976259 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
@@ -194,7 +194,13 @@ struct mlx5e_sw_stats {
u64 rx_cqe_compress_blks;
u64 rx_cqe_compress_pkts;
u64 rx_congst_umr;
+#ifdef CONFIG_MLX5_EN_ARFS
+ u64 rx_arfs_add;
+ u64 rx_arfs_request_in;
+ u64 rx_arfs_request_out;
+ u64 rx_arfs_expired;
u64 rx_arfs_err;
+#endif
u64 rx_recover;
u64 ch_events;
u64 ch_poll;
@@ -256,7 +262,6 @@ struct mlx5e_sw_stats {
u64 rx_xsk_cqe_compress_blks;
u64 rx_xsk_cqe_compress_pkts;
u64 rx_xsk_congst_umr;
- u64 rx_xsk_arfs_err;
u64 tx_xsk_xmit;
u64 tx_xsk_mpwqe;
u64 tx_xsk_inlnw;
@@ -358,7 +363,13 @@ struct mlx5e_rq_stats {
u64 cqe_compress_blks;
u64 cqe_compress_pkts;
u64 congst_umr;
+#ifdef CONFIG_MLX5_EN_ARFS
+ u64 arfs_add;
+ u64 arfs_request_in;
+ u64 arfs_request_out;
+ u64 arfs_expired;
u64 arfs_err;
+#endif
u64 recover;
#ifdef CONFIG_PAGE_POOL_STATS
u64 pp_alloc_fast;
--
2.41.0
next prev parent reply other threads:[~2023-08-16 21:00 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-16 21:00 [pull request][net-next 00/15] mlx5 updates 2023-08-16 Saeed Mahameed
2023-08-16 21:00 ` [net-next 01/15] net/mlx5e: aRFS, Prevent repeated kernel rule migrations requests Saeed Mahameed
2023-08-16 21:00 ` [net-next 02/15] net/mlx5e: aRFS, Warn if aRFS table does not exist for aRFS rule Saeed Mahameed
2023-08-16 21:00 ` Saeed Mahameed [this message]
2023-08-16 21:00 ` [net-next 04/15] net/mlx5e: Fix spelling mistake "Faided" -> "Failed" Saeed Mahameed
2023-08-16 21:00 ` [net-next 05/15] net/mlx5: IRQ, consolidate irq and affinity mask allocation Saeed Mahameed
2023-08-16 21:00 ` [net-next 06/15] net/mlx5: DR, Fix code indentation Saeed Mahameed
2023-08-16 21:00 ` [net-next 07/15] net/mlx5: DR, Remove unneeded local variable Saeed Mahameed
2023-08-16 21:00 ` [net-next 08/15] net/mlx5: Remove health syndrome enum duplication Saeed Mahameed
2023-08-16 21:00 ` [net-next 09/15] net/mlx5: Update dead links in Kconfig documentation Saeed Mahameed
2023-08-16 21:00 ` [net-next 10/15] net/mlx5: Call mlx5_esw_offloads_rep_load/unload() for uplink port directly Saeed Mahameed
2023-08-16 21:00 ` [net-next 11/15] net/mlx5: Remove VPORT_UPLINK handling from devlink_port.c Saeed Mahameed
2023-08-16 21:00 ` [net-next 12/15] net/mlx5: Rename devlink port ops struct for PFs/VFs Saeed Mahameed
2023-08-16 21:00 ` [net-next 13/15] net/mlx5: DR, Supporting inline WQE when possible Saeed Mahameed
2023-08-16 21:00 ` [net-next 14/15] net/mlx5: Convert PCI error values to generic errnos Saeed Mahameed
2023-08-18 22:28 ` Jakub Kicinski
2023-08-19 2:55 ` Jakub Kicinski
2023-08-21 17:53 ` Saeed Mahameed
2023-08-16 21:00 ` [net-next 15/15] net/mlx5: Devcom, only use devcom after NULL check in mlx5_devcom_send_event() Saeed Mahameed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230816210049.54733-4-saeed@kernel.org \
--to=saeed@kernel.org \
--cc=afaris@nvidia.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=saeedm@nvidia.com \
--cc=shayd@nvidia.com \
--cc=tariqt@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).