Re: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings

[Simon Horman <horms@kernel.org>]

On Thu, Jan 16, 2025 at 11:23:42AM -0800, Michael Chan wrote:
> From: Somnath Kotur <somnath.kotur@broadcom.com>
> 
> In order to use queue_stop/queue_start to support the new Steering
> Tags, we need to free the TX ring and TX completion ring if it is a
> combined channel with TX/RX sharing the same NAPI.  Otherwise
> TX completions will not have the updated Steering Tag.  With that
> we can now add napi_disable() and napi_enable() during queue_stop()/
> queue_start().  This will guarantee that NAPI will stop processing
> the completion entries in case there are additional pending entries
> in the completion rings after queue_stop().
> 
> There could be some NQEs sitting unprocessed while NAPI is disabled
> thereby leaving the NQ unarmed.  Explicitly re-arm the NQ after
> napi_enable() in queue start so that NAPI will resume properly.
> 
> Error handling in bnxt_queue_start() requires a reset.  If a TX
> ring cannot be allocated or initialized properly, it will cause
> TX timeout.  The reset will also free any partially allocated
> rings.
> 
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
> Cc: David Wei <dw@davidwei.uk>
> 
> v2:
> Add reset for error handling in queue_start().
> Fix compile error.
> 
> Discussion about adding napi_disable()/napi_enable():
> 
> https://lore.kernel.org/netdev/5336d624-8d8b-40a6-b732-b020e4a119a2@davidwei.uk/#t
> ---
>  drivers/net/ethernet/broadcom/bnxt/bnxt.c | 124 ++++++++++++++++++++--
>  1 file changed, 115 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c

...

> @@ -15616,6 +15694,7 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>  	struct bnxt_rx_ring_info *rxr, *clone;
>  	struct bnxt_cp_ring_info *cpr;
>  	struct bnxt_vnic_info *vnic;
> +	struct bnxt_napi *bnapi;
>  	int i, rc;
>  
>  	rxr = &bp->rx_ring[idx];
> @@ -15633,25 +15712,40 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>  
>  	bnxt_copy_rx_ring(bp, rxr, clone);
>  
> +	bnapi = rxr->bnapi;
>  	rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
>  	if (rc)
> -		return rc;
> +		goto err_reset_rx;
>  
>  	rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
>  	if (rc)
> -		goto err_free_hwrm_rx_ring;
> +		goto err_reset_rx;
>  
>  	rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
>  	if (rc)
> -		goto err_free_hwrm_cp_ring;
> +		goto err_reset_rx;
>  
>  	bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
>  	if (bp->flags & BNXT_FLAG_AGG_RINGS)
>  		bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
>  
> -	cpr = &rxr->bnapi->cp_ring;
> +	cpr = &bnapi->cp_ring;
>  	cpr->sw_stats->rx.rx_resets++;

Hi Somnath and Michael,

cpr is initialised here. However, it appears that the above
"goto err_reset_rx" directives will result in cpr being dereferenced.

Flagged by W=1 builds with clang-19, and Smatch.

>  
> +	if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
> +		cpr->sw_stats->tx.tx_resets++;
> +		rc = bnxt_tx_queue_start(bp, idx);
> +		if (rc) {
> +			netdev_warn(bp->dev,
> +				    "tx queue restart failed: rc=%d\n", rc);
> +			bnapi->tx_fault = 1;
> +			goto err_reset;
> +		}
> +	}
> +
> +	napi_enable(&bnapi->napi);
> +	bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
> +
>  	for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
>  		vnic = &bp->vnic_info[i];
>  
> @@ -15668,10 +15762,12 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>  
>  	return 0;
>  
> -err_free_hwrm_cp_ring:
> -	bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
> -err_free_hwrm_rx_ring:
> -	bnxt_hwrm_rx_ring_free(bp, rxr, false);
> +err_reset_rx:
> +	rxr->bnapi->in_reset = true;
> +err_reset:
> +	napi_enable(&bnapi->napi);
> +	bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
> +	bnxt_reset_task(bp, true);
>  	return rc;
>  }

...