[PATCH net] net/mlx5e: fix BQL reset on SQ re-activation

[PATCH net] net/mlx5e: fix BQL reset on SQ re-activation

[Bobby Eshleman]

From: Bobby Eshleman <bobbyeshleman@meta.com>

mlx5e_queue_start() deactivates and re-activates all channels but closes
only the queue being restarted. mlx5e_activate_txqsq() then
unconditionally calls netdev_tx_reset_queue(), zeroing the BQL counters
of channels that kept their in-flight TX WQEs. The next completion then
over-charges and trips the BUG_ON() in dql_completed():

  kernel BUG at lib/dynamic_queue_limits.c:99!
  RIP: 0010:dql_completed+0x23d/0x280
  Call Trace:
   <IRQ>
   mlx5e_poll_tx_cq+0x668/0xa60
   mlx5e_napi_poll+0x5b/0x7b0
   net_rx_action+0x15a/0x580

Reset BQL only when the SQ has no bytes in flight (sq->cc == sq->pc).

In the case that reset is skipped, the outstanding WQEs will eventually
complete and rebalance the dql. The dql->limit is carried across the
reset.

Fixes: b2588ea40ec9 ("net/mlx5e: Implement queue mgmt ops and single channel swap")
Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 +++-
 drivers/net/ethernet/mellanox/mlx5/core/en_tx.c   | 3 ++-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 4a8351f95b27..44834890a409 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -1940,7 +1940,9 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
 {
 	sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix);
 	set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
-	netdev_tx_reset_queue(sq->txq);
+	/* Reset BQL only when the SQ has no bytes in flight. */
+	if (sq->cc == sq->pc)
+		netdev_tx_reset_queue(sq->txq);
 	netif_tx_start_queue(sq->txq);
 	netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi);
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
index 0b5e600e4a6a..d0ad1b8d6988 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
@@ -909,7 +909,8 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq)
 	sq->cc = sqcc;
 
 	/* Do not update BQL for TXQs that got replaced by new active ones, as
-	 * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq().
+	 * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq()
+	 * if the queue has zero in-flight WQEs.
 	 */
 	if (sq == sq->priv->txq2sq[sq->txq_ix])
 		netdev_tx_completed_queue(sq->txq, npkts, nbytes);

---
base-commit: b515dc54795ef370be3cb396e7c12ad91686b6d1
change-id: 20260728-mlx5-bql-84e02a5a46c8

Best regards,
-- 
Bobby Eshleman <bobbyeshleman@meta.com>

Re: [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation

[Tariq Toukan]

On 29/07/2026 0:47, Bobby Eshleman wrote:
> From: Bobby Eshleman <bobbyeshleman@meta.com>
> 
> mlx5e_queue_start() deactivates and re-activates all channels but closes
> only the queue being restarted. mlx5e_activate_txqsq() then
> unconditionally calls netdev_tx_reset_queue(), zeroing the BQL counters
> of channels that kept their in-flight TX WQEs. The next completion then
> over-charges and trips the BUG_ON() in dql_completed():
> 
>    kernel BUG at lib/dynamic_queue_limits.c:99!
>    RIP: 0010:dql_completed+0x23d/0x280
>    Call Trace:
>     <IRQ>
>     mlx5e_poll_tx_cq+0x668/0xa60
>     mlx5e_napi_poll+0x5b/0x7b0
>     net_rx_action+0x15a/0x580
> 
> Reset BQL only when the SQ has no bytes in flight (sq->cc == sq->pc).
> 
> In the case that reset is skipped, the outstanding WQEs will eventually
> complete and rebalance the dql. The dql->limit is carried across the
> reset.
> 
> Fixes: b2588ea40ec9 ("net/mlx5e: Implement queue mgmt ops and single channel swap")
> Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
> ---
>   drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 +++-
>   drivers/net/ethernet/mellanox/mlx5/core/en_tx.c   | 3 ++-
>   2 files changed, 5 insertions(+), 2 deletions(-)
> 

Thanks for your patch.

Overall the fix LGTM, just one comment below.

> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> index 4a8351f95b27..44834890a409 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> @@ -1940,7 +1940,9 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
>   {
>   	sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix);
>   	set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
> -	netdev_tx_reset_queue(sq->txq);
> +	/* Reset BQL only when the SQ has no bytes in flight. */
> +	if (sq->cc == sq->pc)
> +		netdev_tx_reset_queue(sq->txq);
>   	netif_tx_start_queue(sq->txq);
>   	netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi);
>   }
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> index 0b5e600e4a6a..d0ad1b8d6988 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> @@ -909,7 +909,8 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq)
>   	sq->cc = sqcc;
>   
>   	/* Do not update BQL for TXQs that got replaced by new active ones, as
> -	 * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq().
> +	 * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq()
> +	 * if the queue has zero in-flight WQEs.
>   	 */

Please drop this hunk. In this path the replacement SQ is always freshly 
created, so cc == pc == 0 and the reset always happens - the qualifier 
invites the reader to worry about a case that cannot occur here. The 
original wording is accurate as-is.

>   	if (sq == sq->priv->txq2sq[sq->txq_ix])
>   		netdev_tx_completed_queue(sq->txq, npkts, nbytes);
> 
> ---
> base-commit: b515dc54795ef370be3cb396e7c12ad91686b6d1
> change-id: 20260728-mlx5-bql-84e02a5a46c8
> 
> Best regards,

Re: [PATCH net] net/mlx5e: fix BQL reset on SQ re-activation

[Bobby Eshleman]

On Wed, Jul 29, 2026 at 03:32:56PM +0300, Tariq Toukan wrote:
> 
> 
> On 29/07/2026 0:47, Bobby Eshleman wrote:
> > From: Bobby Eshleman <bobbyeshleman@meta.com>
> > 
> > mlx5e_queue_start() deactivates and re-activates all channels but closes
> > only the queue being restarted. mlx5e_activate_txqsq() then
> > unconditionally calls netdev_tx_reset_queue(), zeroing the BQL counters
> > of channels that kept their in-flight TX WQEs. The next completion then
> > over-charges and trips the BUG_ON() in dql_completed():
> > 
> >    kernel BUG at lib/dynamic_queue_limits.c:99!
> >    RIP: 0010:dql_completed+0x23d/0x280
> >    Call Trace:
> >     <IRQ>
> >     mlx5e_poll_tx_cq+0x668/0xa60
> >     mlx5e_napi_poll+0x5b/0x7b0
> >     net_rx_action+0x15a/0x580
> > 
> > Reset BQL only when the SQ has no bytes in flight (sq->cc == sq->pc).
> > 
> > In the case that reset is skipped, the outstanding WQEs will eventually
> > complete and rebalance the dql. The dql->limit is carried across the
> > reset.
> > 
> > Fixes: b2588ea40ec9 ("net/mlx5e: Implement queue mgmt ops and single channel swap")
> > Signed-off-by: Bobby Eshleman <bobbyeshleman@meta.com>
> > ---
> >   drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 4 +++-
> >   drivers/net/ethernet/mellanox/mlx5/core/en_tx.c   | 3 ++-
> >   2 files changed, 5 insertions(+), 2 deletions(-)
> > 
> 
> Thanks for your patch.
> 
> Overall the fix LGTM, just one comment below.
> 
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> > index 4a8351f95b27..44834890a409 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> > @@ -1940,7 +1940,9 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
> >   {
> >   	sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix);
> >   	set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
> > -	netdev_tx_reset_queue(sq->txq);
> > +	/* Reset BQL only when the SQ has no bytes in flight. */
> > +	if (sq->cc == sq->pc)
> > +		netdev_tx_reset_queue(sq->txq);
> >   	netif_tx_start_queue(sq->txq);
> >   	netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi);
> >   }
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > index 0b5e600e4a6a..d0ad1b8d6988 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > @@ -909,7 +909,8 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq)
> >   	sq->cc = sqcc;
> >   	/* Do not update BQL for TXQs that got replaced by new active ones, as
> > -	 * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq().
> > +	 * netdev_tx_reset_queue() is called for them in mlx5e_activate_txqsq()
> > +	 * if the queue has zero in-flight WQEs.
> >   	 */
> 
> Please drop this hunk. In this path the replacement SQ is always freshly
> created, so cc == pc == 0 and the reset always happens - the qualifier
> invites the reader to worry about a case that cannot occur here. The
> original wording is accurate as-is.

Ah makes sense, thanks. Will re-send with the change!

Best,
Bobby

> 
> >   	if (sq == sq->priv->txq2sq[sq->txq_ix])
> >   		netdev_tx_completed_queue(sq->txq, npkts, nbytes);
> > 
> > ---
> > base-commit: b515dc54795ef370be3cb396e7c12ad91686b6d1
> > change-id: 20260728-mlx5-bql-84e02a5a46c8
> > 
> > Best regards,
>