* [PATCH net 0/7] mlx5 HWS fixes 2025-08-17
@ 2025-08-17 20:23 Mark Bloch
2025-08-17 20:23 ` [PATCH net 1/7] net/mlx5: HWS, fix bad parameter in CQ creation Mark Bloch
` (7 more replies)
0 siblings, 8 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Mark Bloch
Hi,
The following patch set focuses on hardware steering fixes
found by the team.
Alex Vesker (1):
net/mlx5: HWS, Fix table creation UID
Vlad Dogaru (1):
net/mlx5: CT: Use the correct counter offset
Yevgeny Kliteynik (5):
net/mlx5: HWS, fix bad parameter in CQ creation
net/mlx5: HWS, fix simple rules rehash error flow
net/mlx5: HWS, fix complex rules rehash error flow
net/mlx5: HWS, prevent rehash from filling up the queues
net/mlx5: HWS, don't rehash on every kind of insertion failure
.../mellanox/mlx5/core/en/tc/ct_fs_hmfs.c | 2 +
.../mellanox/mlx5/core/steering/hws/bwc.c | 81 ++++++++++++++-----
.../mlx5/core/steering/hws/bwc_complex.c | 41 +++++++---
.../mellanox/mlx5/core/steering/hws/cmd.c | 1 +
.../mellanox/mlx5/core/steering/hws/cmd.h | 1 +
.../mellanox/mlx5/core/steering/hws/fs_hws.c | 1 +
.../mellanox/mlx5/core/steering/hws/matcher.c | 5 +-
.../mellanox/mlx5/core/steering/hws/mlx5hws.h | 1 +
.../mellanox/mlx5/core/steering/hws/send.c | 1 -
.../mellanox/mlx5/core/steering/hws/table.c | 13 ++-
.../mellanox/mlx5/core/steering/hws/table.h | 3 +-
11 files changed, 112 insertions(+), 38 deletions(-)
base-commit: 715c7a36d59f54162a26fac1d1ed8dc087a24cf1
--
2.34.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net 1/7] net/mlx5: HWS, fix bad parameter in CQ creation
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-17 20:23 ` [PATCH net 2/7] net/mlx5: HWS, fix simple rules rehash error flow Mark Bloch
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Yevgeny Kliteynik, Vlad Dogaru,
Mark Bloch, Itamar Gozlan
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
'cqe_sz' valid value should be 0 for 64-byte CQE.
Fixes: 2ca62599aa0b ("net/mlx5: HWS, added send engine and context handling")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Vlad Dogaru <vdogaru@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c
index c4b22be19a9b..b0595c9b09e4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c
@@ -964,7 +964,6 @@ static int hws_send_ring_open_cq(struct mlx5_core_dev *mdev,
return -ENOMEM;
MLX5_SET(cqc, cqc_data, uar_page, mdev->priv.uar->index);
- MLX5_SET(cqc, cqc_data, cqe_sz, queue->num_entries);
MLX5_SET(cqc, cqc_data, log_cq_size, ilog2(queue->num_entries));
err = hws_send_ring_alloc_cq(mdev, numa_node, queue, cqc_data, cq);
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net 2/7] net/mlx5: HWS, fix simple rules rehash error flow
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
2025-08-17 20:23 ` [PATCH net 1/7] net/mlx5: HWS, fix bad parameter in CQ creation Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-17 20:23 ` [PATCH net 3/7] net/mlx5: HWS, fix complex " Mark Bloch
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Yevgeny Kliteynik, Vlad Dogaru,
Mark Bloch
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
Moving rules from matcher to matcher should not fail.
However, if it does fail due to various reasons, the error flow
should allow the kernel to continue functioning (albeit with broken
steering rules) instead of going into series of soft lock-ups or
some other problematic behaviour.
This patch fixes the error flow for moving simple rules:
- If new rule creation fails before it was even enqeued, do not
poll for completion
- If TIMEOUT happened while moving the rule, no point trying
to poll for completions for other rules. Something is broken,
completion won't come, just abort the rehash sequence.
- If some other completion with error received, don't give up.
Continue handling rest of the rules to minimize the damage.
- Make sure that the first error code that was received will
be actually returned to the caller instead of replacing it
with the generic error code.
All the aforementioned issues stem from the same bad error flow,
so no point fixing them one by one and leaving partially broken
code - fixing them in one patch.
Fixes: ef94799a8741 ("net/mlx5: HWS, rework rehash loop")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Vlad Dogaru <vdogaru@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
.../mellanox/mlx5/core/steering/hws/bwc.c | 61 +++++++++++++------
1 file changed, 43 insertions(+), 18 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
index 92de4b761a83..0219a49b2326 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
@@ -74,9 +74,9 @@ static void hws_bwc_matcher_init_attr(struct mlx5hws_bwc_matcher *bwc_matcher,
static int
hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)
{
- bool move_error = false, poll_error = false, drain_error = false;
struct mlx5hws_context *ctx = bwc_matcher->matcher->tbl->ctx;
struct mlx5hws_matcher *matcher = bwc_matcher->matcher;
+ int drain_error = 0, move_error = 0, poll_error = 0;
u16 bwc_queues = mlx5hws_bwc_queues(ctx);
struct mlx5hws_rule_attr rule_attr;
struct mlx5hws_bwc_rule *bwc_rule;
@@ -99,11 +99,15 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)
ret = mlx5hws_matcher_resize_rule_move(matcher,
bwc_rule->rule,
&rule_attr);
- if (unlikely(ret && !move_error)) {
- mlx5hws_err(ctx,
- "Moving BWC rule: move failed (%d), attempting to move rest of the rules\n",
- ret);
- move_error = true;
+ if (unlikely(ret)) {
+ if (!move_error) {
+ mlx5hws_err(ctx,
+ "Moving BWC rule: move failed (%d), attempting to move rest of the rules\n",
+ ret);
+ move_error = ret;
+ }
+ /* Rule wasn't queued, no need to poll */
+ continue;
}
pending_rules++;
@@ -111,11 +115,19 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)
rule_attr.queue_id,
&pending_rules,
false);
- if (unlikely(ret && !poll_error)) {
- mlx5hws_err(ctx,
- "Moving BWC rule: poll failed (%d), attempting to move rest of the rules\n",
- ret);
- poll_error = true;
+ if (unlikely(ret)) {
+ if (ret == -ETIMEDOUT) {
+ mlx5hws_err(ctx,
+ "Moving BWC rule: timeout polling for completions (%d), aborting rehash\n",
+ ret);
+ return ret;
+ }
+ if (!poll_error) {
+ mlx5hws_err(ctx,
+ "Moving BWC rule: polling for completions failed (%d), attempting to move rest of the rules\n",
+ ret);
+ poll_error = ret;
+ }
}
}
@@ -126,17 +138,30 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)
rule_attr.queue_id,
&pending_rules,
true);
- if (unlikely(ret && !drain_error)) {
- mlx5hws_err(ctx,
- "Moving BWC rule: drain failed (%d), attempting to move rest of the rules\n",
- ret);
- drain_error = true;
+ if (unlikely(ret)) {
+ if (ret == -ETIMEDOUT) {
+ mlx5hws_err(ctx,
+ "Moving bwc rule: timeout draining completions (%d), aborting rehash\n",
+ ret);
+ return ret;
+ }
+ if (!drain_error) {
+ mlx5hws_err(ctx,
+ "Moving bwc rule: drain failed (%d), attempting to move rest of the rules\n",
+ ret);
+ drain_error = ret;
+ }
}
}
}
- if (move_error || poll_error || drain_error)
- ret = -EINVAL;
+ /* Return the first error that happened */
+ if (unlikely(move_error))
+ return move_error;
+ if (unlikely(poll_error))
+ return poll_error;
+ if (unlikely(drain_error))
+ return drain_error;
return ret;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net 3/7] net/mlx5: HWS, fix complex rules rehash error flow
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
2025-08-17 20:23 ` [PATCH net 1/7] net/mlx5: HWS, fix bad parameter in CQ creation Mark Bloch
2025-08-17 20:23 ` [PATCH net 2/7] net/mlx5: HWS, fix simple rules rehash error flow Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-17 20:23 ` [PATCH net 4/7] net/mlx5: HWS, prevent rehash from filling up the queues Mark Bloch
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Yevgeny Kliteynik, Vlad Dogaru,
Mark Bloch
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
Moving rules from matcher to matcher should not fail.
However, if it does fail due to various reasons, the error flow
should allow the kernel to continue functioning (albeit with broken
steering rules) instead of going into series of soft lock-ups or
some other problematic behaviour.
Similar to the simple rules, complex rules rehash logic suffers
from the same problems. This patch fixes the error flow for moving
complex rules:
- If new rule creation fails before it was even enqeued, do not
poll for completion
- If TIMEOUT happened while moving the rule, no point trying
to poll for completions for other rules. Something is broken,
completion won't come, just abort the rehash sequence.
- If some other completion with error received, don't give up.
Continue handling rest of the rules to minimize the damage.
- Make sure that the first error code that was received will
be actually returned to the caller instead of replacing it
with the generic error code.
All the aforementioned issues stem from the same bad error flow,
so no point fixing them one by one and leaving partially broken
code - fixing them in one patch.
Fixes: 17e0accac577 ("net/mlx5: HWS, support complex matchers")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Vlad Dogaru <vdogaru@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
.../mlx5/core/steering/hws/bwc_complex.c | 41 +++++++++++++------
1 file changed, 28 insertions(+), 13 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c
index ca7501c57468..14e79579c719 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c
@@ -1328,11 +1328,11 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher)
{
struct mlx5hws_context *ctx = bwc_matcher->matcher->tbl->ctx;
struct mlx5hws_matcher *matcher = bwc_matcher->matcher;
- bool move_error = false, poll_error = false;
u16 bwc_queues = mlx5hws_bwc_queues(ctx);
struct mlx5hws_bwc_rule *tmp_bwc_rule;
struct mlx5hws_rule_attr rule_attr;
struct mlx5hws_table *isolated_tbl;
+ int move_error = 0, poll_error = 0;
struct mlx5hws_rule *tmp_rule;
struct list_head *rules_list;
u32 expected_completions = 1;
@@ -1391,11 +1391,15 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher)
ret = mlx5hws_matcher_resize_rule_move(matcher,
tmp_rule,
&rule_attr);
- if (unlikely(ret && !move_error)) {
- mlx5hws_err(ctx,
- "Moving complex BWC rule failed (%d), attempting to move rest of the rules\n",
- ret);
- move_error = true;
+ if (unlikely(ret)) {
+ if (!move_error) {
+ mlx5hws_err(ctx,
+ "Moving complex BWC rule: move failed (%d), attempting to move rest of the rules\n",
+ ret);
+ move_error = ret;
+ }
+ /* Rule wasn't queued, no need to poll */
+ continue;
}
expected_completions = 1;
@@ -1403,11 +1407,19 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher)
rule_attr.queue_id,
&expected_completions,
true);
- if (unlikely(ret && !poll_error)) {
- mlx5hws_err(ctx,
- "Moving complex BWC rule: poll failed (%d), attempting to move rest of the rules\n",
- ret);
- poll_error = true;
+ if (unlikely(ret)) {
+ if (ret == -ETIMEDOUT) {
+ mlx5hws_err(ctx,
+ "Moving complex BWC rule: timeout polling for completions (%d), aborting rehash\n",
+ ret);
+ return ret;
+ }
+ if (!poll_error) {
+ mlx5hws_err(ctx,
+ "Moving complex BWC rule: polling for completions failed (%d), attempting to move rest of the rules\n",
+ ret);
+ poll_error = ret;
+ }
}
/* Done moving the rule to the new matcher,
@@ -1422,8 +1434,11 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher)
}
}
- if (move_error || poll_error)
- ret = -EINVAL;
+ /* Return the first error that happened */
+ if (unlikely(move_error))
+ return move_error;
+ if (unlikely(poll_error))
+ return poll_error;
return ret;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net 4/7] net/mlx5: HWS, prevent rehash from filling up the queues
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
` (2 preceding siblings ...)
2025-08-17 20:23 ` [PATCH net 3/7] net/mlx5: HWS, fix complex " Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-17 20:23 ` [PATCH net 5/7] net/mlx5: HWS, don't rehash on every kind of insertion failure Mark Bloch
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Yevgeny Kliteynik, Vlad Dogaru,
Mark Bloch
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
While moving the rules during rehash, CQ is not drained. The flush
and drain happens only when all the rules of a certain queue have been
moved. This behaviour can lead to accumulating large quantity of rules
that haven't got their completion yet, and eventually will fill up
the queue and will cause the rehash to fail.
Fix this problem by requiring drain once the number of outstanding
completions reaches a certain threshold.
Fixes: ef94799a8741 ("net/mlx5: HWS, rework rehash loop")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Vlad Dogaru <vdogaru@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
index 0219a49b2326..2a59be11fe55 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
@@ -84,6 +84,7 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)
struct list_head *rules_list;
u32 pending_rules;
int i, ret = 0;
+ bool drain;
mlx5hws_bwc_rule_fill_attr(bwc_matcher, 0, 0, &rule_attr);
@@ -111,10 +112,12 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)
}
pending_rules++;
+ drain = pending_rules >=
+ hws_bwc_get_burst_th(ctx, rule_attr.queue_id);
ret = mlx5hws_bwc_queue_poll(ctx,
rule_attr.queue_id,
&pending_rules,
- false);
+ drain);
if (unlikely(ret)) {
if (ret == -ETIMEDOUT) {
mlx5hws_err(ctx,
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net 5/7] net/mlx5: HWS, don't rehash on every kind of insertion failure
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
` (3 preceding siblings ...)
2025-08-17 20:23 ` [PATCH net 4/7] net/mlx5: HWS, prevent rehash from filling up the queues Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-17 20:23 ` [PATCH net 6/7] net/mlx5: HWS, Fix table creation UID Mark Bloch
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Yevgeny Kliteynik, Vlad Dogaru,
Mark Bloch, Erez Shitrit
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
If rule creation failed due to a full queue, due to timeout
in polling for completion, or due to matcher being in resize,
don't try to initiate rehash sequence - rehash would have
failed anyway.
Fixes: 2111bb970c78 ("net/mlx5: HWS, added backward-compatible API handling")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Vlad Dogaru <vdogaru@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
.../mellanox/mlx5/core/steering/hws/bwc.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
index 2a59be11fe55..adeccc588e5d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c
@@ -1063,6 +1063,21 @@ int mlx5hws_bwc_rule_create_simple(struct mlx5hws_bwc_rule *bwc_rule,
return 0; /* rule inserted successfully */
}
+ /* Rule insertion could fail due to queue being full, timeout, or
+ * matcher in resize. In such cases, no point in trying to rehash.
+ */
+ if (ret == -EBUSY || ret == -ETIMEDOUT || ret == -EAGAIN) {
+ mutex_unlock(queue_lock);
+ mlx5hws_err(ctx,
+ "BWC rule insertion failed - %s (%d)\n",
+ ret == -EBUSY ? "queue is full" :
+ ret == -ETIMEDOUT ? "timeout" :
+ ret == -EAGAIN ? "matcher in resize" : "N/A",
+ ret);
+ hws_bwc_rule_cnt_dec(bwc_rule);
+ return ret;
+ }
+
/* At this point the rule wasn't added.
* It could be because there was collision, or some other problem.
* Try rehash by size and insert rule again - last chance.
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net 6/7] net/mlx5: HWS, Fix table creation UID
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
` (4 preceding siblings ...)
2025-08-17 20:23 ` [PATCH net 5/7] net/mlx5: HWS, don't rehash on every kind of insertion failure Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-17 20:23 ` [PATCH net 7/7] net/mlx5: CT: Use the correct counter offset Mark Bloch
2025-08-20 3:11 ` [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 patchwork-bot+netdevbpf
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Alex Vesker, Yevgeny Kliteynik,
Mark Bloch, Hamdan Agbariya
From: Alex Vesker <valex@nvidia.com>
During table creation, caller passes a UID using ft_attr. The UID
value was ignored, which leads to problems when the caller sets the
UID to a non-zero value, such as SHARED_RESOURCE_UID (0xffff) - the
internal FT objects will be created with UID=0.
Fixes: 0869701cba3d ("net/mlx5: HWS, added FW commands handling")
Signed-off-by: Alex Vesker <valex@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
.../ethernet/mellanox/mlx5/core/steering/hws/cmd.c | 1 +
.../ethernet/mellanox/mlx5/core/steering/hws/cmd.h | 1 +
.../mellanox/mlx5/core/steering/hws/fs_hws.c | 1 +
.../mellanox/mlx5/core/steering/hws/matcher.c | 5 ++++-
.../mellanox/mlx5/core/steering/hws/mlx5hws.h | 1 +
.../mellanox/mlx5/core/steering/hws/table.c | 13 ++++++++++---
.../mellanox/mlx5/core/steering/hws/table.h | 3 ++-
7 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c
index 9c83753e4592..0bdcab2e5cf3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c
@@ -55,6 +55,7 @@ int mlx5hws_cmd_flow_table_create(struct mlx5_core_dev *mdev,
MLX5_SET(create_flow_table_in, in, opcode, MLX5_CMD_OP_CREATE_FLOW_TABLE);
MLX5_SET(create_flow_table_in, in, table_type, ft_attr->type);
+ MLX5_SET(create_flow_table_in, in, uid, ft_attr->uid);
ft_ctx = MLX5_ADDR_OF(create_flow_table_in, in, flow_table_context);
MLX5_SET(flow_table_context, ft_ctx, level, ft_attr->level);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.h
index fa6bff210266..122ccc671628 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.h
@@ -36,6 +36,7 @@ struct mlx5hws_cmd_set_fte_attr {
struct mlx5hws_cmd_ft_create_attr {
u8 type;
u8 level;
+ u16 uid;
bool rtc_valid;
bool decap_en;
bool reformat_en;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c
index 57592b92e24b..131e74b2b774 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c
@@ -267,6 +267,7 @@ static int mlx5_cmd_hws_create_flow_table(struct mlx5_flow_root_namespace *ns,
tbl_attr.type = MLX5HWS_TABLE_TYPE_FDB;
tbl_attr.level = ft_attr->level;
+ tbl_attr.uid = ft_attr->uid;
tbl = mlx5hws_table_create(ctx, &tbl_attr);
if (!tbl) {
mlx5_core_err(ns->dev, "Failed creating hws flow_table\n");
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/matcher.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/matcher.c
index f3ea09caba2b..32f87fdf3213 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/matcher.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/matcher.c
@@ -85,6 +85,7 @@ static int hws_matcher_create_end_ft_isolated(struct mlx5hws_matcher *matcher)
ret = mlx5hws_table_create_default_ft(tbl->ctx->mdev,
tbl,
+ 0,
&matcher->end_ft_id);
if (ret) {
mlx5hws_err(tbl->ctx, "Isolated matcher: failed to create end flow table\n");
@@ -112,7 +113,9 @@ static int hws_matcher_create_end_ft(struct mlx5hws_matcher *matcher)
if (mlx5hws_matcher_is_isolated(matcher))
ret = hws_matcher_create_end_ft_isolated(matcher);
else
- ret = mlx5hws_table_create_default_ft(tbl->ctx->mdev, tbl,
+ ret = mlx5hws_table_create_default_ft(tbl->ctx->mdev,
+ tbl,
+ 0,
&matcher->end_ft_id);
if (ret) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h
index 59c14745ed0c..2498ceff2060 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h
@@ -75,6 +75,7 @@ struct mlx5hws_context_attr {
struct mlx5hws_table_attr {
enum mlx5hws_table_type type;
u32 level;
+ u16 uid;
};
enum mlx5hws_matcher_flow_src {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.c
index 568f691733f3..6113383ae47b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.c
@@ -9,6 +9,7 @@ u32 mlx5hws_table_get_id(struct mlx5hws_table *tbl)
}
static void hws_table_init_next_ft_attr(struct mlx5hws_table *tbl,
+ u16 uid,
struct mlx5hws_cmd_ft_create_attr *ft_attr)
{
ft_attr->type = tbl->fw_ft_type;
@@ -16,7 +17,9 @@ static void hws_table_init_next_ft_attr(struct mlx5hws_table *tbl,
ft_attr->level = tbl->ctx->caps->fdb_ft.max_level - 1;
else
ft_attr->level = tbl->ctx->caps->nic_ft.max_level - 1;
+
ft_attr->rtc_valid = true;
+ ft_attr->uid = uid;
}
static void hws_table_set_cap_attr(struct mlx5hws_table *tbl,
@@ -119,12 +122,12 @@ static int hws_table_connect_to_default_miss_tbl(struct mlx5hws_table *tbl, u32
int mlx5hws_table_create_default_ft(struct mlx5_core_dev *mdev,
struct mlx5hws_table *tbl,
- u32 *ft_id)
+ u16 uid, u32 *ft_id)
{
struct mlx5hws_cmd_ft_create_attr ft_attr = {0};
int ret;
- hws_table_init_next_ft_attr(tbl, &ft_attr);
+ hws_table_init_next_ft_attr(tbl, uid, &ft_attr);
hws_table_set_cap_attr(tbl, &ft_attr);
ret = mlx5hws_cmd_flow_table_create(mdev, &ft_attr, ft_id);
@@ -189,7 +192,10 @@ static int hws_table_init(struct mlx5hws_table *tbl)
}
mutex_lock(&ctx->ctrl_lock);
- ret = mlx5hws_table_create_default_ft(tbl->ctx->mdev, tbl, &tbl->ft_id);
+ ret = mlx5hws_table_create_default_ft(tbl->ctx->mdev,
+ tbl,
+ tbl->uid,
+ &tbl->ft_id);
if (ret) {
mlx5hws_err(tbl->ctx, "Failed to create flow table object\n");
mutex_unlock(&ctx->ctrl_lock);
@@ -239,6 +245,7 @@ struct mlx5hws_table *mlx5hws_table_create(struct mlx5hws_context *ctx,
tbl->ctx = ctx;
tbl->type = attr->type;
tbl->level = attr->level;
+ tbl->uid = attr->uid;
ret = hws_table_init(tbl);
if (ret) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.h
index 0400cce0c317..1246f9bd8422 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.h
@@ -18,6 +18,7 @@ struct mlx5hws_table {
enum mlx5hws_table_type type;
u32 fw_ft_type;
u32 level;
+ u16 uid;
struct list_head matchers_list;
struct list_head tbl_list_node;
struct mlx5hws_default_miss default_miss;
@@ -47,7 +48,7 @@ u32 mlx5hws_table_get_res_fw_ft_type(enum mlx5hws_table_type tbl_type,
int mlx5hws_table_create_default_ft(struct mlx5_core_dev *mdev,
struct mlx5hws_table *tbl,
- u32 *ft_id);
+ u16 uid, u32 *ft_id);
void mlx5hws_table_destroy_default_ft(struct mlx5hws_table *tbl,
u32 ft_id);
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net 7/7] net/mlx5: CT: Use the correct counter offset
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
` (5 preceding siblings ...)
2025-08-17 20:23 ` [PATCH net 6/7] net/mlx5: HWS, Fix table creation UID Mark Bloch
@ 2025-08-17 20:23 ` Mark Bloch
2025-08-20 3:11 ` [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 patchwork-bot+netdevbpf
7 siblings, 0 replies; 9+ messages in thread
From: Mark Bloch @ 2025-08-17 20:23 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Tariq Toukan, Leon Romanovsky, Saeed Mahameed, netdev, linux-rdma,
linux-kernel, Gal Pressman, Vlad Dogaru, Yevgeny Kliteynik,
Mark Bloch, Cosmin Ratiu, Jianbo Liu
From: Vlad Dogaru <vdogaru@nvidia.com>
Specifying the counter action is not enough, as it is used by multiple
counters that were allocated in a bulk. By omitting the offset, rules
will be associated with a different counter from the same bulk.
Subsequently, the CT subsystem checks the correct counter, assumes that
no traffic has triggered the rule, and ages out the rule. The end result
is intermittent offloading of long lived connections, as rules are aged
out then promptly re-added.
Fix this by specifying the correct offset along with the counter rule.
Fixes: 34eea5b12a10 ("net/mlx5e: CT: Add initial support for Hardware Steering")
Signed-off-by: Vlad Dogaru <vdogaru@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c
index a4263137fef5..01d522b02947 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c
@@ -173,6 +173,8 @@ static void mlx5_ct_fs_hmfs_fill_rule_actions(struct mlx5_ct_fs_hmfs *fs_hmfs,
memset(rule_actions, 0, NUM_CT_HMFS_RULES * sizeof(*rule_actions));
rule_actions[0].action = mlx5_fc_get_hws_action(fs_hmfs->ctx, attr->counter);
+ rule_actions[0].counter.offset =
+ attr->counter->id - attr->counter->bulk->base_id;
/* Modify header is special, it may require extra arguments outside the action itself. */
if (mh_action->mh_data) {
rule_actions[1].modify_header.offset = mh_action->mh_data->offset;
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH net 0/7] mlx5 HWS fixes 2025-08-17
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
` (6 preceding siblings ...)
2025-08-17 20:23 ` [PATCH net 7/7] net/mlx5: CT: Use the correct counter offset Mark Bloch
@ 2025-08-20 3:11 ` patchwork-bot+netdevbpf
7 siblings, 0 replies; 9+ messages in thread
From: patchwork-bot+netdevbpf @ 2025-08-20 3:11 UTC (permalink / raw)
To: Mark Bloch
Cc: edumazet, kuba, pabeni, andrew+netdev, davem, tariqt, leon,
saeedm, netdev, linux-rdma, linux-kernel, gal
Hello:
This series was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:
On Sun, 17 Aug 2025 23:23:16 +0300 you wrote:
> Hi,
>
> The following patch set focuses on hardware steering fixes
> found by the team.
>
> Alex Vesker (1):
> net/mlx5: HWS, Fix table creation UID
>
> [...]
Here is the summary with links:
- [net,1/7] net/mlx5: HWS, fix bad parameter in CQ creation
https://git.kernel.org/netdev/net/c/2462c1b92172
- [net,2/7] net/mlx5: HWS, fix simple rules rehash error flow
https://git.kernel.org/netdev/net/c/615b690612b7
- [net,3/7] net/mlx5: HWS, fix complex rules rehash error flow
https://git.kernel.org/netdev/net/c/4a842b1bf18a
- [net,4/7] net/mlx5: HWS, prevent rehash from filling up the queues
https://git.kernel.org/netdev/net/c/1a72298d27ce
- [net,5/7] net/mlx5: HWS, don't rehash on every kind of insertion failure
https://git.kernel.org/netdev/net/c/7c60952f8358
- [net,6/7] net/mlx5: HWS, Fix table creation UID
https://git.kernel.org/netdev/net/c/8a51507320eb
- [net,7/7] net/mlx5: CT: Use the correct counter offset
https://git.kernel.org/netdev/net/c/d2d6f950cb43
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-08-20 3:11 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-17 20:23 [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 Mark Bloch
2025-08-17 20:23 ` [PATCH net 1/7] net/mlx5: HWS, fix bad parameter in CQ creation Mark Bloch
2025-08-17 20:23 ` [PATCH net 2/7] net/mlx5: HWS, fix simple rules rehash error flow Mark Bloch
2025-08-17 20:23 ` [PATCH net 3/7] net/mlx5: HWS, fix complex " Mark Bloch
2025-08-17 20:23 ` [PATCH net 4/7] net/mlx5: HWS, prevent rehash from filling up the queues Mark Bloch
2025-08-17 20:23 ` [PATCH net 5/7] net/mlx5: HWS, don't rehash on every kind of insertion failure Mark Bloch
2025-08-17 20:23 ` [PATCH net 6/7] net/mlx5: HWS, Fix table creation UID Mark Bloch
2025-08-17 20:23 ` [PATCH net 7/7] net/mlx5: CT: Use the correct counter offset Mark Bloch
2025-08-20 3:11 ` [PATCH net 0/7] mlx5 HWS fixes 2025-08-17 patchwork-bot+netdevbpf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).