From: Michael Chan <michael.chan@broadcom.com>
To: davem@davemloft.net
Cc: netdev@vger.kernel.org, edumazet@kernel.org, kuba@kernel.org,
pabeni@redhat.com, andrew+netdev@lunn.ch,
pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com
Subject: [PATCH net-next v5 9/9] bnxt_en: Fix ring accounting/validation when IRQs are constrained
Date: Thu, 19 Nov 2026 17:29:21 -0800 [thread overview]
Message-ID: <20261120012921.2537903-10-michael.chan@broadcom.com> (raw)
In-Reply-To: <20261120012921.2537903-1-michael.chan@broadcom.com>
Similar to the previous patch that handles ring shortages from the FW,
this patch handles MSI-X vector shortages. We use the similar logic
and infrastructure to adjust the rings if there are not enough MSI-X
vectors to satisfy the current ring configuration.
Add a new bnxt_adj_rings_to_irqs() helper to adjust rings to the
available MSI-X vectors. The helper will be used when static or
dynamic MSI-X vectors are constrained. Note that the helper will
return success early if rings have not been reserved yet.
We can now centralize ring adjustments during ring reservations and
IRQ allocations. Other ring adjustments and checks scattered elsewhere
can now be deleted.
We now remove the error path at the end of bnxt_reserve_rings() to abort
and reset the TCs if the rings cannot satisfy the TC requirements. The
open will now fail and the user will have to reduce the TCs and retry.
This failure should be very rare because we first call
bnxt_check_rings() to make sure there is high likelihood of success
before committing.
Just like the previous patch, we have similar existing issues regarding
RSS tables for non-default RSS contexts and user defined n-tuple filters
when RX rings shrink due to MSI-X constraints. The fixes for these are
deferred to a separate patchset.
Fixes: 1ee581c24dfd ("bnxt_en: Adjust TX rings if reservation is less than requested")
Fixes: 674f50a5b026 ("bnxt_en: Implement new method to reserve rings.")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
v5:
New patch split from patch #8.
Fixed a Sashiko issue in bnxt_init_int_mode() with no rings reserved.
Rebased to handle dynamic MSI-X vectors after:
af449230bd8d ("bnxt_en: Don't keep MSI-X vectors added before FW reserves them")
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 110 ++++++++++++++--------
1 file changed, 73 insertions(+), 37 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 565099088969..8ec9576371cb 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -11569,9 +11569,74 @@ static int bnxt_get_num_msix(struct bnxt *bp)
return bnxt_nq_rings_in_use(bp);
}
+/* Adjust rings to fit bp->total_irqs */
+static int bnxt_adj_rings_to_irqs(struct bnxt *bp)
+{
+ int req_rx_rings, req_tx_rings, req_cp_rings, req_tx_per_tc, req_tx_xdp;
+ int ulp_msix = bnxt_get_ulp_msix_num(bp);
+ int min_tx, rc;
+
+ /* Nothing to adjust if rings have not been setup yet, e.g. after
+ * bnxt_clear_reservations() during devlink reload, or on a VF with
+ * 0 rings provisioned by the PF. The caller must ensure that genuine
+ * ring shortage cannot reach here.
+ */
+ if (!bp->tx_nr_rings)
+ return 0;
+
+ req_tx_rings = bp->tx_nr_rings;
+ req_tx_per_tc = bp->tx_nr_rings_per_tc;
+ req_tx_xdp = bp->tx_nr_rings_xdp;
+ req_rx_rings = bp->rx_nr_rings;
+ req_cp_rings = bp->cp_nr_rings;
+
+ /* Trim rings based upon num of vectors allocated */
+ rc = bnxt_trim_rings(bp, &bp->rx_nr_rings, &bp->tx_nr_rings,
+ bp->total_irqs - ulp_msix,
+ bp->flags & BNXT_FLAG_SHARED_RINGS);
+ if (rc)
+ goto adj_irqs_err;
+
+ min_tx = bnxt_min_tx_rings(bp);
+ if (bp->tx_nr_rings < min_tx) {
+ netdev_err(bp->dev, "Not enough MSI-X to satisfy minimum TX rings\n");
+ rc = -ENOMEM;
+ goto adj_irqs_err;
+ }
+
+ bnxt_adj_rings(bp);
+
+ if (bp->bnapi && (req_tx_rings != bp->tx_nr_rings ||
+ req_rx_rings != bp->rx_nr_rings)) {
+ netdev_err(bp->dev, "Cannot shrink rings once NAPI is allocated\n");
+ rc = -ENOSPC;
+ goto adj_irqs_err;
+ }
+
+ if (req_tx_rings != bp->tx_nr_rings)
+ netdev_warn(bp->dev, "TX rings reduced from %d to %d\n",
+ req_tx_rings, bp->tx_nr_rings);
+ if (req_rx_rings != bp->rx_nr_rings) {
+ netdev_warn(bp->dev, "RX rings reduced from %d to %d\n",
+ req_rx_rings, bp->rx_nr_rings);
+ bnxt_check_rss_tbl_lost(bp, req_rx_rings, bp->rx_nr_rings);
+ if (!netif_is_rxfh_configured(bp->dev))
+ bnxt_set_dflt_rss_indir_tbl(bp, NULL);
+ }
+ return 0;
+
+adj_irqs_err:
+ bp->tx_nr_rings = req_tx_rings;
+ bp->tx_nr_rings_per_tc = req_tx_per_tc;
+ bp->tx_nr_rings_xdp = req_tx_xdp;
+ bp->rx_nr_rings = req_rx_rings;
+ bp->cp_nr_rings = req_cp_rings;
+ return rc;
+}
+
static int bnxt_init_int_mode(struct bnxt *bp)
{
- int i, total_vecs, max, rc, min = 1, ulp_msix, tx_cp, tbl_size;
+ int i, total_vecs, max, rc, min = 1, ulp_msix, tbl_size;
total_vecs = bnxt_get_num_msix(bp);
max = bnxt_get_max_func_irqs(bp);
@@ -11606,18 +11671,10 @@ static int bnxt_init_int_mode(struct bnxt *bp)
bp->irq_tbl[i].vector = pci_irq_vector(bp->pdev, i);
bp->total_irqs = total_vecs;
- /* Trim rings based upon num of vectors allocated */
- rc = bnxt_trim_rings(bp, &bp->rx_nr_rings, &bp->tx_nr_rings,
- total_vecs - ulp_msix, min == 1);
+ rc = bnxt_adj_rings_to_irqs(bp);
if (rc)
goto msix_setup_exit;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings, bp->tx_nr_rings_xdp,
- bp->rx_nr_rings);
- bp->cp_nr_rings = (min == 1) ?
- max_t(int, tx_cp, bp->rx_nr_rings) :
- tx_cp + bp->rx_nr_rings;
-
return 0;
msix_setup_exit:
@@ -11658,7 +11715,6 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
{
bool irq_cleared = false;
bool irq_change = false;
- int tcs = bp->num_tc;
int irqs_required;
int rc;
@@ -11672,9 +11728,11 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
irqs_required = bnxt_get_num_msix(bp);
if (irqs_required > bp->total_irqs &&
bnxt_change_msix(bp, irqs_required) != irqs_required) {
- netdev_err(bp->dev, "Unable to allocate %d MSIX vectors, maximum available %d\n",
- irqs_required, bp->total_irqs);
- return -ENOSPC;
+ netdev_warn(bp->dev, "Able to allocate only %d MSIX vectors out of %d requested\n",
+ bp->total_irqs, irqs_required);
+ rc = bnxt_adj_rings_to_irqs(bp);
+ if (rc)
+ return rc;
}
}
return 0;
@@ -11700,23 +11758,12 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
*/
irqs_required = bnxt_get_num_msix(bp);
if (bnxt_change_msix(bp, irqs_required) != irqs_required)
- rc = -ENOSPC;
+ rc = bnxt_adj_rings_to_irqs(bp);
}
if (rc) {
netdev_err(bp->dev, "ring reservation/IRQ init failure rc: %d\n", rc);
return rc;
}
- if (tcs && (bp->tx_nr_rings_per_tc * tcs !=
- bp->tx_nr_rings - bp->tx_nr_rings_xdp)) {
- netdev_err(bp->dev, "tx ring reservation failure\n");
- netdev_reset_tc(bp->dev);
- bp->num_tc = 0;
- if (bp->tx_nr_rings_xdp)
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings_xdp;
- else
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
- return -ENOMEM;
- }
return 0;
}
@@ -13256,8 +13303,6 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
if (rc)
return rc;
- if (irq_re_init)
- bnxt_adj_rings(bp);
rc = bnxt_alloc_mem(bp, irq_re_init);
if (rc) {
netdev_err(bp->dev, "bnxt_alloc_mem err: %x\n", rc);
@@ -16951,7 +16996,6 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
if (rc && rc != -ECOMM)
netdev_warn(bp->dev, "Unable to reserve tx rings\n");
- bnxt_adj_rings(bp);
if (sh)
bnxt_adj_dflt_rings(bp, true);
@@ -16960,7 +17004,6 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
rc = __bnxt_reserve_rings(bp);
if (rc && rc != -ECOMM)
netdev_warn(bp->dev, "2nd rings reservation failed.\n");
- bnxt_adj_rings(bp);
}
if (rc) {
bp->tx_nr_rings = 0;
@@ -16990,8 +17033,6 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)
if (rc)
goto init_dflt_ring_err;
- bnxt_adj_rings(bp);
-
bnxt_set_dflt_rfs(bp);
init_dflt_ring_err:
@@ -17385,11 +17426,6 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (rc)
goto init_err_pci_clean;
- /* No TC has been set yet and rings may have been trimmed due to
- * limited MSIX, so we re-initialize the TX rings per TC.
- */
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
-
if (BNXT_PF(bp)) {
if (!bnxt_pf_wq) {
bnxt_pf_wq =
--
2.51.0
prev parent reply other threads:[~2026-11-20 1:30 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 1/9] bnxt_en: Clear bp->total_irqs when disabling IRQs Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 2/9] bnxt_en: Fix bnxt_reinit_features() when irq_re_init is true Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 3/9] bnxt_en: Refactor RSS table check logic Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 4/9] bnxt_en: Refactor IRQs required logic Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 5/9] bnxt_en: Reinit IRQ when configuring LRO/GRO/HDS Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 6/9] bnxt_en: Remove support for the old NITRO A0 chip Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 7/9] bnxt_en: Refactor bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 8/9] bnxt_en: Fix ring accounting/validation when rings are constrained by FW Michael Chan
2026-11-20 1:29 ` Michael Chan [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120012921.2537903-10-michael.chan@broadcom.com \
--to=michael.chan@broadcom.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrew.gospodarek@broadcom.com \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=pavan.chebbi@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox