* [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation
@ 2026-07-28 21:47 Bobby Eshleman
2026-07-29 12:32 ` Tariq Toukan
0 siblings, 1 reply; 3+ messages in thread
From: Bobby Eshleman @ 2026-07-28 21:47 UTC (permalink / raw)
To: Saeed Mahameed, Tariq Toukan, Mark Bloch, Leon Romanovsky,
Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Dragos Tatulea, Mina Almasry
Cc: netdev, linux-rdma, linux-kernel, Bobby Eshleman
From: Bobby Eshleman <bobbyeshleman@meta.com>
mlx5e_queue_start() deactivates and re-activates all channels but closes
only the queue being restarted. mlx5e_activate_txqsq() then
unconditionally calls netdev_tx_reset_queue(), zeroing the BQL counters
of channels that kept their in-flight TX WQEs. The next completion then
over-charges and trips the BUG_ON() in dql_completed():
kernel BUG at lib/dynamic_queue_limits.c:99!
RIP: 0010:dql_completed+0x23d/0x280
Call Trace:
<IRQ>
mlx5e_poll_tx_cq+0x668/0xa60
mlx5e_napi_poll+0x5b/0x7b0
net_rx_action+0x15a/0x580
Reset BQL only when the SQ has no bytes in flight (sq->cc == sq->pc).
In the case that reset is skipped, the outstanding WQEs will eventually
complete and rebalance the dql. The dql->limit is carried across the
reset.
Fixes: b2588ea40ec9 ("net/mlx5e: Implement queue mgmt ops and single channel swap")
Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 +++-
drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 3 ++-
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 4a8351f95b27..44834890a409 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -1940,7 +1940,9 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
{
sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix);
set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
- netdev_tx_reset_queue(sq->txq);
+ /* Reset BQL only when the SQ has no bytes in flight. */
+ if (sq->cc == sq->pc)
+ netdev_tx_reset_queue(sq->txq);
netif_tx_start_queue(sq->txq);
netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
index 0b5e600e4a6a..d0ad1b8d6988 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
@@ -909,7 +909,8 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq)
sq->cc = sqcc;
/* Do not update BQL for TXQs that got replaced by new active ones, as
- * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq().
+ * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq()
+ * if the queue has zero in-flight WQEs.
*/
if (sq == sq->priv->txq2sq[sq->txq_ix])
netdev_tx_completed_queue(sq->txq, npkts, nbytes);
---
base-commit: b515dc54795ef370be3cb396e7c12ad91686b6d1
change-id: 20260728-mlx5-bql-84e02a5a46c8
Best regards,
--
Bobby Eshleman <bobbyeshleman@meta.com>
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation
2026-07-28 21:47 [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation Bobby Eshleman
@ 2026-07-29 12:32 ` Tariq Toukan
2026-07-29 16:33 ` Bobby Eshleman
0 siblings, 1 reply; 3+ messages in thread
From: Tariq Toukan @ 2026-07-29 12:32 UTC (permalink / raw)
To: Bobby Eshleman, Saeed Mahameed, Tariq Toukan, Mark Bloch,
Leon Romanovsky, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Dragos Tatulea, Mina Almasry
Cc: netdev, linux-rdma, linux-kernel, Bobby Eshleman
On 29/07/2026 0:47, Bobby Eshleman wrote:
> From: Bobby Eshleman <bobbyeshleman@meta.com>
>
> mlx5e_queue_start() deactivates and re-activates all channels but closes
> only the queue being restarted. mlx5e_activate_txqsq() then
> unconditionally calls netdev_tx_reset_queue(), zeroing the BQL counters
> of channels that kept their in-flight TX WQEs. The next completion then
> over-charges and trips the BUG_ON() in dql_completed():
>
> kernel BUG at lib/dynamic_queue_limits.c:99!
> RIP: 0010:dql_completed+0x23d/0x280
> Call Trace:
> <IRQ>
> mlx5e_poll_tx_cq+0x668/0xa60
> mlx5e_napi_poll+0x5b/0x7b0
> net_rx_action+0x15a/0x580
>
> Reset BQL only when the SQ has no bytes in flight (sq->cc == sq->pc).
>
> In the case that reset is skipped, the outstanding WQEs will eventually
> complete and rebalance the dql. The dql->limit is carried across the
> reset.
>
> Fixes: b2588ea40ec9 ("net/mlx5e: Implement queue mgmt ops and single channel swap")
> Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
> ---
> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 +++-
> drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 3 ++-
> 2 files changed, 5 insertions(+), 2 deletions(-)
>
Thanks for your patch.
Overall the fix LGTM, just one comment below.
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> index 4a8351f95b27..44834890a409 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> @@ -1940,7 +1940,9 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
> {
> sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix);
> set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
> - netdev_tx_reset_queue(sq->txq);
> + /* Reset BQL only when the SQ has no bytes in flight. */
> + if (sq->cc == sq->pc)
> + netdev_tx_reset_queue(sq->txq);
> netif_tx_start_queue(sq->txq);
> netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi);
> }
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> index 0b5e600e4a6a..d0ad1b8d6988 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> @@ -909,7 +909,8 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq)
> sq->cc = sqcc;
>
> /* Do not update BQL for TXQs that got replaced by new active ones, as
> - * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq().
> + * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq()
> + * if the queue has zero in-flight WQEs.
> */
Please drop this hunk. In this path the replacement SQ is always freshly
created, so cc == pc == 0 and the reset always happens - the qualifier
invites the reader to worry about a case that cannot occur here. The
original wording is accurate as-is.
> if (sq == sq->priv->txq2sq[sq->txq_ix])
> netdev_tx_completed_queue(sq->txq, npkts, nbytes);
>
> ---
> base-commit: b515dc54795ef370be3cb396e7c12ad91686b6d1
> change-id: 20260728-mlx5-bql-84e02a5a46c8
>
> Best regards,
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation
2026-07-29 12:32 ` Tariq Toukan
@ 2026-07-29 16:33 ` Bobby Eshleman
0 siblings, 0 replies; 3+ messages in thread
From: Bobby Eshleman @ 2026-07-29 16:33 UTC (permalink / raw)
To: Tariq Toukan
Cc: Saeed Mahameed, Mark Bloch, Leon Romanovsky, Andrew Lunn,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Dragos Tatulea, Mina Almasry, netdev, linux-rdma, linux-kernel,
Bobby Eshleman
On Wed, Jul 29, 2026 at 03:32:56PM +0300, Tariq Toukan wrote:
>
>
> On 29/07/2026 0:47, Bobby Eshleman wrote:
> > From: Bobby Eshleman <bobbyeshleman@meta.com>
> >
> > mlx5e_queue_start() deactivates and re-activates all channels but closes
> > only the queue being restarted. mlx5e_activate_txqsq() then
> > unconditionally calls netdev_tx_reset_queue(), zeroing the BQL counters
> > of channels that kept their in-flight TX WQEs. The next completion then
> > over-charges and trips the BUG_ON() in dql_completed():
> >
> > kernel BUG at lib/dynamic_queue_limits.c:99!
> > RIP: 0010:dql_completed+0x23d/0x280
> > Call Trace:
> > <IRQ>
> > mlx5e_poll_tx_cq+0x668/0xa60
> > mlx5e_napi_poll+0x5b/0x7b0
> > net_rx_action+0x15a/0x580
> >
> > Reset BQL only when the SQ has no bytes in flight (sq->cc == sq->pc).
> >
> > In the case that reset is skipped, the outstanding WQEs will eventually
> > complete and rebalance the dql. The dql->limit is carried across the
> > reset.
> >
> > Fixes: b2588ea40ec9 ("net/mlx5e: Implement queue mgmt ops and single channel swap")
> > Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
> > ---
> > drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 +++-
> > drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 3 ++-
> > 2 files changed, 5 insertions(+), 2 deletions(-)
> >
>
> Thanks for your patch.
>
> Overall the fix LGTM, just one comment below.
>
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> > index 4a8351f95b27..44834890a409 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> > @@ -1940,7 +1940,9 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
> > {
> > sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix);
> > set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
> > - netdev_tx_reset_queue(sq->txq);
> > + /* Reset BQL only when the SQ has no bytes in flight. */
> > + if (sq->cc == sq->pc)
> > + netdev_tx_reset_queue(sq->txq);
> > netif_tx_start_queue(sq->txq);
> > netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi);
> > }
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > index 0b5e600e4a6a..d0ad1b8d6988 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > @@ -909,7 +909,8 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq)
> > sq->cc = sqcc;
> > /* Do not update BQL for TXQs that got replaced by new active ones, as
> > - * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq().
> > + * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq()
> > + * if the queue has zero in-flight WQEs.
> > */
>
> Please drop this hunk. In this path the replacement SQ is always freshly
> created, so cc == pc == 0 and the reset always happens - the qualifier
> invites the reader to worry about a case that cannot occur here. The
> original wording is accurate as-is.
Ah makes sense, thanks. Will re-send with the change!
Best,
Bobby
>
> > if (sq == sq->priv->txq2sq[sq->txq_ix])
> > netdev_tx_completed_queue(sq->txq, npkts, nbytes);
> >
> > ---
> > base-commit: b515dc54795ef370be3cb396e7c12ad91686b6d1
> > change-id: 20260728-mlx5-bql-84e02a5a46c8
> >
> > Best regards,
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-07-29 16:33 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-28 21:47 [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation Bobby Eshleman
2026-07-29 12:32 ` Tariq Toukan
2026-07-29 16:33 ` Bobby Eshleman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox