From: Saeed Mahameed <saeed@kernel.org>
To: "David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org, Tariq Toukan <tariqt@nvidia.com>,
Yevgeny Kliteynik <kliteyn@nvidia.com>,
Alex Vesker <valex@mellanox.com>,
Saeed Mahameed <saeedm@nvidia.com>
Subject: [net-next 15/15] net/mlx5: DR, Add support for isolate_vl_tc QP
Date: Mon, 19 Apr 2021 20:20:18 -0700 [thread overview]
Message-ID: <20210420032018.58639-16-saeed@kernel.org> (raw)
In-Reply-To: <20210420032018.58639-1-saeed@kernel.org>
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
When using SW steering, rule insertion rate depends on the RDMA RC QP
performance used for writing to the ICM. During stress this QP is competing
on the HW resources with all the other QPs that are used to send data.
To protect SW steering QP's performance in such cases, we set this QP to
use isolated VL. The VL number is reserved by FW and is not exposed to the
driver.
Support for this QP on isolated VL exists only when both force-loopback and
isolate_vl_tc capabilities are set.
Signed-off-by: Alex Vesker <valex@mellanox.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c | 2 ++
drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c | 7 +++++++
.../net/ethernet/mellanox/mlx5/core/steering/dr_types.h | 2 ++
include/linux/mlx5/mlx5_ifc.h | 4 +++-
4 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
index 68d898e144fb..5970cb8fc0c0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
@@ -130,6 +130,8 @@ int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
MLX5_CAP_ROCE(mdev, fl_rc_qp_when_roce_enabled);
}
+ caps->isolate_vl_tc = MLX5_CAP_GEN(mdev, isolate_vl_tc_new);
+
if (caps->flex_protocols & MLX5_FLEX_PARSER_ICMP_V4_ENABLED) {
caps->flex_parser_id_icmp_dw0 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw0);
caps->flex_parser_id_icmp_dw1 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw1);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
index 69d623bedefe..12cf323a5943 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
@@ -46,6 +46,7 @@ struct dr_qp_init_attr {
u32 pdn;
u32 max_send_wr;
struct mlx5_uars_page *uar;
+ u8 isolate_vl_tc:1;
};
static int dr_parse_cqe(struct mlx5dr_cq *dr_cq, struct mlx5_cqe64 *cqe64)
@@ -158,6 +159,7 @@ static struct mlx5dr_qp *dr_create_rc_qp(struct mlx5_core_dev *mdev,
qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
MLX5_SET(qpc, qpc, st, MLX5_QP_ST_RC);
MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
+ MLX5_SET(qpc, qpc, isolate_vl_tc, attr->isolate_vl_tc);
MLX5_SET(qpc, qpc, pd, attr->pdn);
MLX5_SET(qpc, qpc, uar_page, attr->uar->index);
MLX5_SET(qpc, qpc, log_page_size,
@@ -924,6 +926,11 @@ int mlx5dr_send_ring_alloc(struct mlx5dr_domain *dmn)
init_attr.pdn = dmn->pdn;
init_attr.uar = dmn->uar;
init_attr.max_send_wr = QUEUE_SIZE;
+
+ /* Isolated VL is applicable only if force loopback is supported */
+ if (dr_send_allow_fl(&dmn->info.caps))
+ init_attr.isolate_vl_tc = dmn->info.caps.isolate_vl_tc;
+
spin_lock_init(&dmn->send_ring->lock);
dmn->send_ring->qp = dr_create_rc_qp(dmn->mdev, &init_attr);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
index 8de70566f85b..67460c42a99b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
@@ -790,6 +790,7 @@ struct mlx5dr_cmd_caps {
struct mlx5dr_cmd_vport_cap *vports_caps;
bool prio_tag_required;
struct mlx5dr_roce_cap roce_caps;
+ u8 isolate_vl_tc:1;
};
struct mlx5dr_domain_rx_tx {
@@ -1164,6 +1165,7 @@ struct mlx5dr_cmd_qp_create_attr {
u32 sq_wqe_cnt;
u32 rq_wqe_cnt;
u32 rq_wqe_shift;
+ u8 isolate_vl_tc:1;
};
int mlx5dr_cmd_query_gid(struct mlx5_core_dev *mdev, u8 vhca_port_num,
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 4d9569c4b96c..52b7cabcde08 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1319,7 +1319,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 log_max_srq_sz[0x8];
u8 log_max_qp_sz[0x8];
u8 event_cap[0x1];
- u8 reserved_at_91[0x7];
+ u8 reserved_at_91[0x2];
+ u8 isolate_vl_tc_new[0x1];
+ u8 reserved_at_94[0x4];
u8 prio_tag_required[0x1];
u8 reserved_at_99[0x2];
u8 log_max_qp[0x5];
--
2.30.2
prev parent reply other threads:[~2021-04-20 3:21 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-20 3:20 [pull request][net-next 00/15] mlx5 updates 2021-04-19 Saeed Mahameed
2021-04-20 3:20 ` [net-next 01/15] net/mlx5e: Fix lost changes during code movements Saeed Mahameed
2021-04-20 23:30 ` patchwork-bot+netdevbpf
2021-04-20 3:20 ` [net-next 02/15] net/mlx5e: Fix possible non-initialized struct usage Saeed Mahameed
2021-04-20 3:20 ` [net-next 03/15] net/mlx5e: RX, Add checks for calculated Striding RQ attributes Saeed Mahameed
2021-04-20 3:20 ` [net-next 04/15] net/mlx5: DR, Rename an argument in dr_rdma_segments Saeed Mahameed
2021-04-20 3:20 ` [net-next 05/15] net/mlx5: DR, Fix SQ/RQ in doorbell bitmask Saeed Mahameed
2021-04-20 3:20 ` [net-next 06/15] net/mlx5: E-Switch, Improve error messages in term table creation Saeed Mahameed
2021-04-20 3:20 ` [net-next 07/15] net/mlx5: mlx5_ifc updates for flex parser Saeed Mahameed
2021-04-20 4:54 ` Samudrala, Sridhar
2021-04-20 17:07 ` Yevgeny Kliteynik
2021-04-21 15:08 ` Samudrala, Sridhar
2021-04-21 17:52 ` Yevgeny Kliteynik
2021-04-20 3:20 ` [net-next 08/15] net/mlx5: DR, Remove protocol-specific flex_parser_3 definitions Saeed Mahameed
2021-04-20 3:20 ` [net-next 09/15] net/mlx5: DR, Add support for dynamic flex parser Saeed Mahameed
2021-04-20 3:20 ` [net-next 10/15] net/mlx5: DR, Set STEv0 ICMP flex parser dynamically Saeed Mahameed
2021-04-20 3:20 ` [net-next 11/15] net/mlx5: DR, Add support for matching on geneve TLV option Saeed Mahameed
2021-04-20 3:20 ` [net-next 12/15] net/mlx5: DR, Set flex parser for TNL_MPLS dynamically Saeed Mahameed
2021-04-20 3:20 ` [net-next 13/15] net/mlx5: DR, Add support for matching tunnel GTP-U Saeed Mahameed
2021-04-20 3:20 ` [net-next 14/15] net/mlx5: DR, Add support for force-loopback QP Saeed Mahameed
2021-04-20 3:20 ` Saeed Mahameed [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210420032018.58639-16-saeed@kernel.org \
--to=saeed@kernel.org \
--cc=davem@davemloft.net \
--cc=kliteyn@nvidia.com \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=saeedm@nvidia.com \
--cc=tariqt@nvidia.com \
--cc=valex@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).