From: Saeed Mahameed <saeed@kernel.org>
To: "David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org, Tariq Toukan <tariqt@nvidia.com>,
Saeed Mahameed <saeedm@nvidia.com>
Subject: [net-next 05/15] net/mlx5e: RX, Restrict bulk size for small Striding RQs
Date: Wed, 16 Feb 2022 23:56:22 -0800 [thread overview]
Message-ID: <20220217075632.831542-6-saeed@kernel.org> (raw)
In-Reply-To: <20220217075632.831542-1-saeed@kernel.org>
From: Tariq Toukan <tariqt@nvidia.com>
In RQs of type multi-packet WQE (Striding RQ), each WQE is relatively
large (typically 256KB) but their number is relatively small (8 in
default).
Re-mapping the descriptors' buffers before re-posting them is done via
UMR (User-Mode Memory Registration) operations.
On the one hand, posting UMR WQEs in bulks reduces communication overhead
with the HW and better utilizes its processing units.
On the other hand, delaying the WQE repost operations for a small RQ
(say, of 4 WQEs) might drastically hit its performance, causing packet
drops due to no receive buffer, for high or bursty incoming packets rate.
Here we restrict the bulk size for too small RQs. Effectively, with the current
constants, RQ of size 4 (minimum allowed) would have no bulking, while larger
RQs will continue working with bulks of 2.
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 +
drivers/net/ethernet/mellanox/mlx5/core/en/params.c | 6 ++++++
drivers/net/ethernet/mellanox/mlx5/core/en/params.h | 1 +
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 1 +
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 3 +--
5 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index e29ac77e9bec..2704c7537481 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -714,6 +714,7 @@ struct mlx5e_rq {
u8 umr_in_progress;
u8 umr_last_bulk;
u8 umr_completed;
+ u8 min_wqe_bulk;
struct mlx5e_shampo_hd *shampo;
} mpwqe;
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
index 2d9707c024b0..0bd8698f7226 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
@@ -178,6 +178,12 @@ u8 mlx5e_mpwqe_get_log_num_strides(struct mlx5_core_dev *mdev,
mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk);
}
+u8 mlx5e_mpwqe_get_min_wqe_bulk(unsigned int wq_sz)
+{
+#define UMR_WQE_BULK (2)
+ return min_t(unsigned int, UMR_WQE_BULK, wq_sz / 2 - 1);
+}
+
u16 mlx5e_get_rq_headroom(struct mlx5_core_dev *mdev,
struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
index 433e6967692d..47a368112e31 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
@@ -129,6 +129,7 @@ u8 mlx5e_mpwqe_get_log_stride_size(struct mlx5_core_dev *mdev,
u8 mlx5e_mpwqe_get_log_num_strides(struct mlx5_core_dev *mdev,
struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk);
+u8 mlx5e_mpwqe_get_min_wqe_bulk(unsigned int wq_sz);
u16 mlx5e_get_rq_headroom(struct mlx5_core_dev *mdev,
struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index b157c7aac4ca..b2ed2f6d4a92 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -595,6 +595,7 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params,
rq->mpwqe.log_stride_sz = mlx5e_mpwqe_get_log_stride_size(mdev, params, xsk);
rq->mpwqe.num_strides =
BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params, xsk));
+ rq->mpwqe.min_wqe_bulk = mlx5e_mpwqe_get_min_wqe_bulk(wq_sz);
rq->buff.frame0_sz = (1 << rq->mpwqe.log_stride_sz);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 3fe4f06f3e71..4cb7c7135b6a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -960,8 +960,7 @@ INDIRECT_CALLABLE_SCOPE bool mlx5e_post_rx_mpwqes(struct mlx5e_rq *rq)
if (unlikely(rq->mpwqe.umr_in_progress > rq->mpwqe.umr_last_bulk))
rq->stats->congst_umr++;
-#define UMR_WQE_BULK (2)
- if (likely(missing < UMR_WQE_BULK))
+ if (likely(missing < rq->mpwqe.min_wqe_bulk))
return false;
if (rq->page_pool)
--
2.34.1
next prev parent reply other threads:[~2022-02-17 7:56 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-17 7:56 [pull request][net-next 00/15] mlx5 updates 2022-02-16 Saeed Mahameed
2022-02-17 7:56 ` [net-next 01/15] net/mlx5e: Fix spelling mistake "supoported" -> "supported" Saeed Mahameed
2022-02-17 11:10 ` patchwork-bot+netdevbpf
2022-02-17 7:56 ` [net-next 02/15] net/mlx5e: Add support for using xdp->data_meta Saeed Mahameed
2022-02-17 7:56 ` [net-next 03/15] net/mlx5e: Generalize packet merge error message Saeed Mahameed
2022-02-17 7:56 ` [net-next 04/15] net/mlx5e: Default to Striding RQ when not conflicting with CQE compression Saeed Mahameed
2022-02-17 7:56 ` Saeed Mahameed [this message]
2022-02-17 7:56 ` [net-next 06/15] net/mlx5e: E-Switch, Add PTP counters for uplink representor Saeed Mahameed
2022-02-17 7:56 ` [net-next 07/15] net/mlx5e: E-Switch, Add support for tx_port_ts in switchdev mode Saeed Mahameed
2022-02-17 7:56 ` [net-next 08/15] net/mlx5e: TC, Move flow hashtable to be per rep Saeed Mahameed
2022-02-17 7:56 ` [net-next 09/15] net/mlx5e: Pass actions param to actions_match_supported() Saeed Mahameed
2022-02-17 7:56 ` [net-next 10/15] net/mlx5e: Add post act offload/unoffload API Saeed Mahameed
2022-02-17 7:56 ` [net-next 11/15] net/mlx5e: Create new flow attr for multi table actions Saeed Mahameed
2022-02-17 7:56 ` [net-next 12/15] net/mlx5e: Use multi table support for CT and sample actions Saeed Mahameed
2022-02-17 7:56 ` [net-next 13/15] net/mlx5e: TC, Clean redundant counter flag from tc action parsers Saeed Mahameed
2022-02-17 7:56 ` [net-next 14/15] net/mlx5e: TC, Make post_act parse CT and sample actions Saeed Mahameed
2022-02-17 7:56 ` [net-next 15/15] net/mlx5e: TC, Allow sample action with CT Saeed Mahameed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220217075632.831542-6-saeed@kernel.org \
--to=saeed@kernel.org \
--cc=davem@davemloft.net \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=saeedm@nvidia.com \
--cc=tariqt@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).