* [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes
@ 2026-11-20 1:29 Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 1/9] bnxt_en: Clear bp->total_irqs when disabling IRQs Michael Chan
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
The main purpose of this patchset is to fix ring accounting bugs
specifically when FW is unable to reserve all the rings requested
by the driver or when there are not enough MSI-X vectors. This
issue was first reported by Sashiko when reviewing the driver
patches to support kTLS (patch #3 of 15):
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260810051358.1244418-7-michael.chan@broadcom.com
kTLS support requires the new MPC rings, so the patches touch
the existing ring reservation logic and exposed the issues to
Sashiko. The issues are real but only likely to be hit with a
large number of VFs, stressing the ring resources to the limit.
These issues were not found in the lab or by users, so Jakub
suggested splitting these out and posting them to net-next.
The patchset includes some refactoring and some related fixes
in this code path. The driver will now scale down the rings
properly when FW is unable reserve all the rings requested by
the driver or when there are not enough MSI-X vectors. The
default RSS context is also reset if the RX rings have been
reduced if needed. Note that additional RSS contexts and
n-tuple filters have similar problems if the RX rings are
reduced and the fixes are deferred to a future patchset.
Patch #6 is new and removes support for the obsolete NITRO A0,
to make the fixes in patch #8 simpler.
Patch #7 is new in v4 to refactor 2 helpers.
Patch #9 is new in v5 to separate the MSI-X logic to a separate
patch after this commit landed that modified the dynamic MSI-X
logic:
af449230bd8d ("bnxt_en: Don't keep MSI-X vectors added before FW reserves them")
The patches were tested on 5750X and 5760X chips using production FW
version 237.x.x.x.
v5:
Added a new patch #9 to handle MSI-X separately.
Fixed 2 Sashiko reported issues in patch #1 and #8 (fixed in #9)
v4:
Improved patch #1 to fix a possible NULL bp->irq_tbl issue reported
by Sashiko.
Added a new patch #7 to fix the ring trimming issue in patch #8
reported by Sashiko.
https://lore.kernel.org/netdev/20261016064427.496944-1-michael.chan@broadcom.com/
v3:
Split from the original -net patchset (patchs 1 to 6)
Updated to address some more Sashiko comments.
Added the new patch #6 to remove NITRO A0.
https://lore.kernel.org/netdev/20261012220827.340998-1-michael.chan@broadcom.com/
v2:
Updated to address some Sashiko comments.
https://lore.kernel.org/netdev/20260928041712.3467803-1-michael.chan@broadcom.com/
v1:
https://lore.kernel.org/netdev/20260831024342.2161156-1-michael.chan@broadcom.com/
Michael Chan (9):
bnxt_en: Clear bp->total_irqs when disabling IRQs
bnxt_en: Fix bnxt_reinit_features() when irq_re_init is true
bnxt_en: Refactor RSS table check logic
bnxt_en: Refactor IRQs required logic
bnxt_en: Reinit IRQ when configuring LRO/GRO/HDS
bnxt_en: Remove support for the old NITRO A0 chip
bnxt_en: Refactor bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers
bnxt_en: Fix ring accounting/validation when rings are constrained by
FW
bnxt_en: Fix ring accounting/validation when IRQs are constrained
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 522 +++++++++---------
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 14 +-
.../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 22 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 7 +-
4 files changed, 274 insertions(+), 291 deletions(-)
--
2.51.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next v5 1/9] bnxt_en: Clear bp->total_irqs when disabling IRQs
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 2/9] bnxt_en: Fix bnxt_reinit_features() when irq_re_init is true Michael Chan
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
bnxt_init_int_mode() and bnxt_clear_int_mode() will free bp->irq_tbl
but bp->total_irqs retains the old value. If a subsequent
reinitialization happens, bnxt_reserve_rings() may see that
bp->total_irqs does not match a new irqs_required. It will then
try to adjust if dynamic MSI-X is supported and call
bnxt_change_msix(). It will then crash when dereferencing the NULL
bp->irq_tbl.
Fix it by clearing bp->total_irqs when freeing bp->irq_tbl. Dynamic
MSI-X adjustments should only proceed if bp->irq_tbl is valid which
means that MSI-X has been initialized and can be adjusted.
bnxt_ulp_start() and bnxt_ulp_irq_restart() also need a similar check
before filling the ULP MSI-X entries.
Add a bp->irq_tbl_size to prevent OOB bp->irq_tbl[] array access in
case MSI-X capabilities change during re-init. For dynamic MSI-X,
allocate the biggest bp->irq_tbl that is configured in the NVRAM
without clamping it with maximum CP/NQ rings to allow dynamic MSI-X
to grow to the max.
In the AER path, if MSI-X capabilities change, bnxt_reserve_rings()
will initialize MSI-X if BNXT_NEW_RM() is true. Add a check in
bnxt_io_resume() to skip doing it again later.
Fixes: e68256c8a73c ("bnxt_en: Support dynamic MSIX")
Fixes: c2d20a3814d1 ("bnxt_en: delay pci_alloc_irq_vectors() in the AER path")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
v5:
Fix a Sashiko issue: possible NULL bp->irq_tble issue in bnxt_ulp_start().
v4:
Clear bp->total_irqs in bnxt_clear_int_mode() to fix a possible NULL
bp->irq_tbl issue reported by Sashiko and Gregory Price on patch #8.
https://lore.kernel.org/netdev/20261016064427.496944-2-michael.chan@broadcom.com/
v3:
Added an additional Fixes tag.
https://lore.kernel.org/netdev/20261012220827.340998-2-michael.chan@broadcom.com/
v2:
https://lore.kernel.org/netdev/20260928041712.3467803-2-michael.chan@broadcom.com/
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 19 +++++++++++++++----
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 +
drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 7 ++++++-
3 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index e78cf7ad3cbc..0808911186c5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -11529,6 +11529,12 @@ static int bnxt_change_msix(struct bnxt *bp, int total)
struct msi_map map;
int i;
+ if (!bp->irq_tbl)
+ return 0;
+
+ if (total > bp->irq_tbl_size)
+ return bp->total_irqs;
+
/* add MSIX to the end if needed */
for (i = bp->total_irqs; i < total; i++) {
map = pci_msix_alloc_irq_at(bp->pdev, i, NULL);
@@ -11672,12 +11678,13 @@ static int bnxt_init_int_mode(struct bnxt *bp)
tbl_size = total_vecs;
if (pci_msix_can_alloc_dyn(bp->pdev))
- tbl_size = max;
+ tbl_size = bp->max_irqs;
bp->irq_tbl = kzalloc_objs(*bp->irq_tbl, tbl_size);
if (!bp->irq_tbl) {
rc = -ENOMEM;
goto msix_setup_exit;
}
+ bp->irq_tbl_size = tbl_size;
for (i = 0; i < total_vecs; i++)
bp->irq_tbl[i].vector = pci_irq_vector(bp->pdev, i);
@@ -11700,6 +11707,8 @@ static int bnxt_init_int_mode(struct bnxt *bp)
netdev_err(bp->dev, "bnxt_init_int_mode err: %x\n", rc);
kfree(bp->irq_tbl);
bp->irq_tbl = NULL;
+ bp->total_irqs = 0;
+ bp->irq_tbl_size = 0;
pci_free_irq_vectors(bp->pdev);
return rc;
}
@@ -11710,6 +11719,8 @@ static void bnxt_clear_int_mode(struct bnxt *bp)
kfree(bp->irq_tbl);
bp->irq_tbl = NULL;
+ bp->total_irqs = 0;
+ bp->irq_tbl_size = 0;
}
int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
@@ -11751,7 +11762,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
if (irq_re_init && BNXT_NEW_RM(bp) && irqs_required != bp->total_irqs) {
irq_change = true;
- if (!pci_msix_can_alloc_dyn(bp->pdev)) {
+ if (!pci_msix_can_alloc_dyn(bp->pdev) || !bp->irq_tbl) {
bnxt_ulp_irq_stop(bp);
bnxt_clear_int_mode(bp);
irq_cleared = true;
@@ -15137,7 +15148,7 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
hwr.cp += bnxt_get_ulp_msix_num(bp);
hwr.cp = min_t(int, hwr.cp, bnxt_get_max_func_irqs(bp));
}
- if (hwr.cp > bp->total_irqs) {
+ if (bp->irq_tbl && hwr.cp > bp->total_irqs) {
int old_total = bp->total_irqs;
int total_msix = bnxt_change_msix(bp, hwr.cp);
@@ -17842,7 +17853,7 @@ static void bnxt_io_resume(struct pci_dev *pdev)
err = bnxt_open(netdev);
} else {
err = bnxt_reserve_rings(bp, true);
- if (!err)
+ if (!err && !bp->irq_tbl)
err = bnxt_init_int_mode(bp);
}
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 41d05efd6307..75428a585577 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2489,6 +2489,7 @@ struct bnxt {
*/
unsigned long *ring_affinity_set;
int max_irqs;
+ int irq_tbl_size;
int total_irqs;
int ulp_num_msix_want;
u8 mac_addr[ETH_ALEN];
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
index a515c368bac0..0c27a6ec69f7 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
@@ -334,6 +334,11 @@ void bnxt_ulp_start(struct bnxt *bp)
goto clear_flag_continue;
}
+ if (edev->ulp_tbl->msix_requested && !bp->irq_tbl) {
+ /* Stay stopped if there are no vectors */
+ mutex_unlock(&edev->en_dev_lock);
+ continue;
+ }
if (edev->ulp_tbl->msix_requested)
bnxt_fill_msix_vecs(bp, edev->msix_entries);
@@ -396,7 +401,7 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err)
if (!ops || !ops->ulp_irq_restart)
return;
- if (!err) {
+ if (!err && bp->irq_tbl) {
ent = kzalloc_objs(*ent, ulp->msix_requested);
if (!ent)
return;
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 2/9] bnxt_en: Fix bnxt_reinit_features() when irq_re_init is true
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 1/9] bnxt_en: Clear bp->total_irqs when disabling IRQs Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 3/9] bnxt_en: Refactor RSS table check logic Michael Chan
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
bnxt_set_features() takes a snapshot of bp->flags, closes the device,
and then blindly restores the entire bp->flags. bp->flags contains
statistics flags that may be cleared and the memory freed when
irq_re_init is true. Do not restore these statistics flags until
bnxt_open_nic() reallocates the memory. bnxt_open_nic() can
potentially fail to allocate the stats memory and the restored flags
can potentially cause NULL dereference of the stats memory.
Fixes: 93e90104bd12 ("bnxt_en: Create and setup the additional VNIC for adding ntuple filters")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 0808911186c5..95a2227e45cc 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -14130,7 +14130,8 @@ static int bnxt_reinit_features(struct bnxt *bp, bool irq_re_init,
bool link_re_init, u32 flags, bool update_tpa)
{
bnxt_close_nic(bp, irq_re_init, link_re_init);
- bp->flags = flags;
+ bp->flags = (bp->flags & ~BNXT_FLAG_ALL_CONFIG_FEATS) |
+ (flags & BNXT_FLAG_ALL_CONFIG_FEATS);
if (update_tpa)
bnxt_set_ring_params(bp);
return bnxt_open_nic(bp, irq_re_init, link_re_init);
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 3/9] bnxt_en: Refactor RSS table check logic
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 1/9] bnxt_en: Clear bp->total_irqs when disabling IRQs Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 2/9] bnxt_en: Fix bnxt_reinit_features() when irq_re_init is true Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 4/9] bnxt_en: Refactor IRQs required logic Michael Chan
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, Joe Damato
We have logic to see if we need to reset the user configured RSS
table for the default RSS context if the RX rings have changed
in __bnxt_reserve_rings(). Refactor this code into a helper that
will be used by other callers later in the patch series. There is
no change in behavior.
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Reviewed-by: Joe Damato <joe@dama.to>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 95a2227e45cc..35dabc5adf04 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -8187,6 +8187,20 @@ static bool bnxt_rings_ok(struct bnxt *bp, struct bnxt_hw_rings *hwr)
hwr->stat && (hwr->cp_p5 || !(bp->flags & BNXT_FLAG_CHIP_P5_PLUS));
}
+/* Check to see if we need to reset the user configured RSS table
+ * (default RSS context) because of change in RX rings from
+ * old_rx to new_rx.
+ */
+static void bnxt_check_rss_tbl_lost(struct bnxt *bp, int old_rx, int new_rx)
+{
+ if (netif_is_rxfh_configured(bp->dev) &&
+ (bnxt_get_nr_rss_ctxs(bp, old_rx) !=
+ bnxt_get_nr_rss_ctxs(bp, new_rx) ||
+ bnxt_get_max_rss_ring(bp) >= new_rx)) {
+ ethtool_rxfh_indir_lost(bp->dev);
+ }
+}
+
static int bnxt_get_avail_msix(struct bnxt *bp, int num);
static int __bnxt_reserve_rings(struct bnxt *bp)
@@ -8276,12 +8290,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
if (rx_rings != bp->rx_nr_rings) {
netdev_warn(bp->dev, "Able to reserve only %d out of %d requested RX rings\n",
rx_rings, bp->rx_nr_rings);
- if (netif_is_rxfh_configured(bp->dev) &&
- (bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings) !=
- bnxt_get_nr_rss_ctxs(bp, rx_rings) ||
- bnxt_get_max_rss_ring(bp) >= rx_rings)) {
- ethtool_rxfh_indir_lost(bp->dev);
- }
+ bnxt_check_rss_tbl_lost(bp, bp->rx_nr_rings, rx_rings);
}
bp->rx_nr_rings = rx_rings;
bp->cp_nr_rings = hwr.cp;
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 4/9] bnxt_en: Refactor IRQs required logic
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
` (2 preceding siblings ...)
2026-11-20 1:29 ` [PATCH net-next v5 3/9] bnxt_en: Refactor RSS table check logic Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 5/9] bnxt_en: Reinit IRQ when configuring LRO/GRO/HDS Michael Chan
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, Joe Damato
Refactor the logic to determine the required IRQ vectors into a new
helper. Later patches in the series will make additional use of the
helper. There is no change in behavior.
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Reviewed-by: Joe Damato <joe@dama.to>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 26 +++++++++++++----------
1 file changed, 15 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 35dabc5adf04..f4eff048eea5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -11732,9 +11732,22 @@ static void bnxt_clear_int_mode(struct bnxt *bp)
bp->irq_tbl_size = 0;
}
-int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
+static int bnxt_irqs_required(struct bnxt *bp)
{
struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+
+ if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) {
+ int ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want);
+
+ if (ulp_msix > bp->ulp_num_msix_want)
+ ulp_msix = bp->ulp_num_msix_want;
+ return ulp_msix + bp->cp_nr_rings;
+ }
+ return bnxt_get_num_msix(bp);
+}
+
+int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
+{
bool irq_cleared = false;
bool irq_change = false;
int tcs = bp->num_tc;
@@ -11758,16 +11771,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
}
return 0;
}
-
- if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) {
- int ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want);
-
- if (ulp_msix > bp->ulp_num_msix_want)
- ulp_msix = bp->ulp_num_msix_want;
- irqs_required = ulp_msix + bp->cp_nr_rings;
- } else {
- irqs_required = bnxt_get_num_msix(bp);
- }
+ irqs_required = bnxt_irqs_required(bp);
if (irq_re_init && BNXT_NEW_RM(bp) && irqs_required != bp->total_irqs) {
irq_change = true;
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 5/9] bnxt_en: Reinit IRQ when configuring LRO/GRO/HDS
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
` (3 preceding siblings ...)
2026-11-20 1:29 ` [PATCH net-next v5 4/9] bnxt_en: Refactor IRQs required logic Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 6/9] bnxt_en: Remove support for the old NITRO A0 chip Michael Chan
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
When configuring LRO/GRO/HDS, a new set of RX Aggregation rings may be
required. It is possible that the FW cannot grant the desired number
of Agg rings, causing the driver to reduce the number of ethtool
channels to adjust. This means that the number of IRQs/NAPIs may
change and we must set irq_re_init to true to make that work. Without
this patch, when the driver is eventually shutdown, some memory for the
unused NAPIs may never be freed properly if the rings have shrunk.
Fixes: 87c8f8496a05 ("bnxt_en: add support for tcp-data-split ethtool command")
Fixes: 674f50a5b026 ("bnxt_en: Implement new method to reserve rings.")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 11 ++++++++---
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 10 ++++++++--
2 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index f4eff048eea5..887a67d42bb3 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -14154,10 +14154,11 @@ static int bnxt_set_features(struct net_device *dev, netdev_features_t features)
{
bool update_tpa = false, update_ntuple = false;
struct bnxt *bp = netdev_priv(dev);
+ bool irq_re_init = false;
u32 flags = bp->flags;
+ bool re_init = false;
u32 changes;
int rc = 0;
- bool re_init = false;
bp->tx_wake_thresh = max_t(int, bp->tx_ring_size / 2,
bnxt_min_tx_desc_cnt(bp, features));
@@ -14184,8 +14185,12 @@ static int bnxt_set_features(struct net_device *dev, netdev_features_t features)
update_tpa = true;
if ((bp->flags & BNXT_FLAG_TPA) == 0 ||
(flags & BNXT_FLAG_TPA) == 0 ||
- (bp->flags & BNXT_FLAG_CHIP_P5_PLUS))
+ (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)) {
re_init = true;
+ if (!(bp->flags & BNXT_FLAG_AGG_RINGS) &&
+ (flags & BNXT_FLAG_AGG_RINGS))
+ irq_re_init = true;
+ }
}
if (changes & ~BNXT_FLAG_TPA)
@@ -14208,7 +14213,7 @@ static int bnxt_set_features(struct net_device *dev, netdev_features_t features)
return bnxt_reinit_features(bp, true, false, flags, update_tpa);
if (re_init)
- return bnxt_reinit_features(bp, false, false, flags, update_tpa);
+ return bnxt_reinit_features(bp, irq_re_init, false, flags, update_tpa);
if (update_tpa) {
bp->flags = flags;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index ef4bea4a0f8c..0739cd80a966 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -853,6 +853,7 @@ static int bnxt_set_ringparam(struct net_device *dev,
{
u8 tcp_data_split = kernel_ering->tcp_data_split;
struct bnxt *bp = netdev_priv(dev);
+ bool irq_re_init = false;
u8 hds_config_mod;
int rc;
@@ -876,8 +877,13 @@ static int bnxt_set_ringparam(struct net_device *dev,
return -EINVAL;
}
+ if (hds_config_mod &&
+ tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED &&
+ !(bp->flags & BNXT_FLAG_AGG_RINGS))
+ irq_re_init = true;
+
if (netif_running(dev))
- bnxt_close_nic(bp, false, false);
+ bnxt_close_nic(bp, irq_re_init, false);
if (hds_config_mod) {
if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_ENABLED)
@@ -891,7 +897,7 @@ static int bnxt_set_ringparam(struct net_device *dev,
bnxt_set_ring_params(bp);
if (netif_running(dev)) {
- rc = bnxt_open_nic(bp, false, false);
+ rc = bnxt_open_nic(bp, irq_re_init, false);
if (rc)
return rc;
}
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 6/9] bnxt_en: Remove support for the old NITRO A0 chip
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
` (4 preceding siblings ...)
2026-11-20 1:29 ` [PATCH net-next v5 5/9] bnxt_en: Reinit IRQ when configuring LRO/GRO/HDS Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 7/9] bnxt_en: Refactor bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers Michael Chan
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, Joe Damato
Nitro A0 was a very old pre-production chip that requires significant
workarounds. For example, it requires an extra (hidden) RX ring and CP
ring to work, adding a lot of complexity. The workaround code is very
buggy and breaks in many cases, including ethtool ring changes
initiated by the user.
Remove all Nitro A0 workaround code completely. Print an error and
abort if the chip is detected. This will simplify the next patch that
fixes ring accounting bugs.
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Reviewed-by: Joe Damato <joe@dama.to>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 185 ++----------------
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 11 +-
.../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 12 +-
3 files changed, 19 insertions(+), 189 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 887a67d42bb3..b49e02fcde29 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3229,82 +3229,6 @@ static int bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
return rx_pkts;
}
-static int bnxt_poll_nitroa0(struct napi_struct *napi, int budget)
-{
- struct bnxt_napi *bnapi = container_of(napi, struct bnxt_napi, napi);
- struct bnxt *bp = bnapi->bp;
- struct bnxt_cp_ring_info *cpr = &bnapi->cp_ring;
- struct bnxt_rx_ring_info *rxr = bnapi->rx_ring;
- struct tx_cmp *txcmp;
- struct rx_cmp_ext *rxcmp1;
- u32 cp_cons, tmp_raw_cons;
- u32 raw_cons = cpr->cp_raw_cons;
- bool flush_xdp = false;
- u32 rx_pkts = 0;
- u8 event = 0;
-
- while (1) {
- int rc;
-
- cp_cons = RING_CMP(raw_cons);
- txcmp = &cpr->cp_desc_ring[CP_RING(cp_cons)][CP_IDX(cp_cons)];
-
- if (!TX_CMP_VALID(txcmp, raw_cons))
- break;
-
- /* The valid test of the entry must be done first before
- * reading any further.
- */
- dma_rmb();
- if ((TX_CMP_TYPE(txcmp) & 0x30) == 0x10) {
- tmp_raw_cons = NEXT_RAW_CMP(raw_cons);
- cp_cons = RING_CMP(tmp_raw_cons);
- rxcmp1 = (struct rx_cmp_ext *)
- &cpr->cp_desc_ring[CP_RING(cp_cons)][CP_IDX(cp_cons)];
-
- if (!RX_CMP_VALID(rxcmp1, tmp_raw_cons))
- break;
-
- /* force an error to recycle the buffer */
- rxcmp1->rx_cmp_cfa_code_errors_v2 |=
- cpu_to_le32(RX_CMPL_ERRORS_CRC_ERROR);
-
- rc = bnxt_rx_pkt(bp, cpr, &raw_cons, &event);
- if (likely(rc == -EIO) && budget)
- rx_pkts++;
- else if (rc == -EBUSY) /* partial completion */
- break;
- if (event & BNXT_REDIRECT_EVENT)
- flush_xdp = true;
- } else if (unlikely(TX_CMP_TYPE(txcmp) ==
- CMPL_BASE_TYPE_HWRM_DONE)) {
- bnxt_hwrm_handler(bp, txcmp);
- } else {
- netdev_err(bp->dev,
- "Invalid completion received on special ring\n");
- }
- raw_cons = NEXT_RAW_CMP(raw_cons);
-
- if (rx_pkts == budget)
- break;
- }
-
- cpr->cp_raw_cons = raw_cons;
- BNXT_DB_CQ(&cpr->cp_db, cpr->cp_raw_cons);
- bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
-
- if (event & BNXT_AGG_EVENT)
- bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
- if (flush_xdp)
- xdp_do_flush();
-
- if (!bnxt_has_work(bp, cpr) && rx_pkts < budget) {
- napi_complete_done(napi, rx_pkts);
- BNXT_DB_CQ_ARM(&cpr->cp_db, cpr->cp_raw_cons);
- }
- return rx_pkts;
-}
-
static int bnxt_poll(struct napi_struct *napi, int budget)
{
struct bnxt_napi *bnapi = container_of(napi, struct bnxt_napi, napi);
@@ -4811,9 +4735,6 @@ static int bnxt_alloc_vnics(struct bnxt *bp)
}
#endif
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- num_vnics++;
-
bp->vnic_info = kzalloc_objs(struct bnxt_vnic_info, num_vnics);
if (!bp->vnic_info)
return -ENOMEM;
@@ -6346,11 +6267,8 @@ int bnxt_hwrm_l2_filter_alloc(struct bnxt *bp, struct bnxt_l2_filter *fltr)
return rc;
req->target_id = cpu_to_le16(target_id);
- req->flags = cpu_to_le32(CFA_L2_FILTER_ALLOC_REQ_FLAGS_PATH_RX);
-
- if (!BNXT_CHIP_TYPE_NITRO_A0(bp))
- req->flags |=
- cpu_to_le32(CFA_L2_FILTER_ALLOC_REQ_FLAGS_OUTERMOST);
+ req->flags = cpu_to_le32(CFA_L2_FILTER_ALLOC_REQ_FLAGS_PATH_RX |
+ CFA_L2_FILTER_ALLOC_REQ_FLAGS_OUTERMOST);
req->dst_id = cpu_to_le16(fltr->base.fw_vnic_id);
req->enables =
cpu_to_le32(CFA_L2_FILTER_ALLOC_REQ_ENABLES_L2_ADDR |
@@ -6707,10 +6625,7 @@ void bnxt_set_dflt_rss_indir_tbl(struct bnxt *bp,
if (!bp->rx_nr_rings)
return;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- max_rings = bp->rx_nr_rings - 1;
- else
- max_rings = bp->rx_nr_rings;
+ max_rings = bp->rx_nr_rings;
max_entries = bnxt_get_rxfh_indir_size(bp->dev);
if (rss_ctx)
@@ -6750,8 +6665,6 @@ int bnxt_get_nr_rss_ctxs(struct bnxt *bp, int rx_rings)
return bnxt_calc_nr_ring_pages(rx_rings - 1,
BNXT_RSS_TABLE_ENTRIES_P5);
}
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- return 2;
return 1;
}
@@ -6776,8 +6689,6 @@ static void bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic)
vnic->default_rx_ring = min_j;
else if (vnic->flags & BNXT_VNIC_RFS_FLAG)
vnic->default_rx_ring = vnic->vnic_id - 1;
- else if ((vnic->vnic_id == 1) && BNXT_CHIP_TYPE_NITRO_A0(bp))
- vnic->default_rx_ring = bp->rx_nr_rings - 1;
else
vnic->default_rx_ring = 0;
}
@@ -7032,13 +6943,7 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic)
req->rss_rule = cpu_to_le16(0xffff);
}
- if (BNXT_CHIP_TYPE_NITRO_A0(bp) &&
- (vnic->fw_rss_cos_lb_ctx[0] != INVALID_HW_RING_ID)) {
- req->cos_rule = cpu_to_le16(vnic->fw_rss_cos_lb_ctx[1]);
- req->enables |= cpu_to_le32(VNIC_CFG_REQ_ENABLES_COS_RULE);
- } else {
- req->cos_rule = cpu_to_le16(0xffff);
- }
+ req->cos_rule = cpu_to_le16(0xffff);
ring = vnic->default_rx_ring;
grp_idx = bp->rx_ring[ring].bnapi->index;
@@ -8636,9 +8541,6 @@ static void bnxt_hwrm_stat_ctx_free(struct bnxt *bp)
if (!bp->bnapi)
return;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- return;
-
if (hwrm_req_init(bp, req, HWRM_STAT_CTX_FREE))
return;
if (BNXT_FW_MAJ(bp) <= 20) {
@@ -8675,9 +8577,6 @@ static int bnxt_hwrm_stat_ctx_alloc(struct bnxt *bp)
struct hwrm_stat_ctx_alloc_input *req;
int rc, i;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- return 0;
-
rc = hwrm_req_init(bp, req, HWRM_STAT_CTX_ALLOC);
if (rc)
return rc;
@@ -10508,9 +10407,6 @@ static int bnxt_hwrm_ver_get(struct bnxt *bp)
bp->chip_num = le16_to_cpu(resp->chip_num);
bp->chip_rev = resp->chip_rev;
- if (bp->chip_num == CHIP_NUM_58700 && !resp->chip_rev &&
- !resp->chip_metal)
- bp->flags |= BNXT_FLAG_CHIP_NITRO_A0;
dev_caps_cfg = le32_to_cpu(resp->dev_caps_cfg);
if ((dev_caps_cfg & VER_GET_RESP_DEV_CAPS_CFG_SHORT_CMD_SUPPORTED) &&
@@ -10925,16 +10821,6 @@ static int __bnxt_setup_vnic(struct bnxt *bp, struct bnxt_vnic_info *vnic)
}
bp->rsscos_nr_ctxs++;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp)) {
- rc = bnxt_hwrm_vnic_ctx_alloc(bp, vnic, 1);
- if (rc) {
- netdev_err(bp->dev, "hwrm vnic %d cos ctx alloc failure rc: %x\n",
- vnic->vnic_id, rc);
- goto vnic_setup_err;
- }
- bp->rsscos_nr_ctxs++;
- }
-
skip_rss_ctx:
bnxt_fill_hw_rss_tbl(bp, vnic);
/* configure default vnic, ring grp */
@@ -11235,30 +11121,6 @@ static bool bnxt_promisc_ok(struct bnxt *bp)
return true;
}
-static int bnxt_setup_nitroa0_vnic(struct bnxt *bp)
-{
- struct bnxt_vnic_info *vnic = &bp->vnic_info[1];
- unsigned int rc = 0;
-
- rc = bnxt_hwrm_vnic_alloc(bp, vnic, bp->rx_nr_rings - 1, 1);
- if (rc) {
- netdev_err(bp->dev, "Cannot allocate special vnic for NS2 A0: %x\n",
- rc);
- return rc;
- }
-
- /* Setup the proper default RX ring */
- bnxt_fill_hw_rss_tbl(bp, vnic);
-
- rc = bnxt_hwrm_vnic_cfg(bp, vnic);
- if (rc) {
- netdev_err(bp->dev, "Cannot allocate special vnic for NS2 A0: %x\n",
- rc);
- return rc;
- }
- return rc;
-}
-
static int bnxt_cfg_rx_mode(struct bnxt *, struct netdev_hw_addr_list *, bool);
static bool bnxt_mc_list_updated(struct bnxt *, u32 *,
const struct netdev_hw_addr_list *);
@@ -11267,7 +11129,6 @@ static int bnxt_init_chip(struct bnxt *bp, bool irq_re_init)
{
struct bnxt_vnic_info *vnic = &bp->vnic_info[BNXT_VNIC_DEFAULT];
int rc = 0;
- unsigned int rx_nr_rings = bp->rx_nr_rings;
if (irq_re_init) {
rc = bnxt_hwrm_stat_ctx_alloc(bp);
@@ -11290,11 +11151,8 @@ static int bnxt_init_chip(struct bnxt *bp, bool irq_re_init)
goto err_out;
}
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- rx_nr_rings--;
-
/* default vnic 0 */
- rc = bnxt_hwrm_vnic_alloc(bp, vnic, 0, rx_nr_rings);
+ rc = bnxt_hwrm_vnic_alloc(bp, vnic, 0, bp->rx_nr_rings);
if (rc) {
netdev_err(bp->dev, "hwrm vnic alloc failure rc: %x\n", rc);
goto err_out;
@@ -11369,13 +11227,6 @@ static int bnxt_init_chip(struct bnxt *bp, bool irq_re_init)
netdev_warn(bp->dev, "HWRM set coalescing failure rc: %x\n",
rc);
- if (BNXT_CHIP_TYPE_NITRO_A0(bp)) {
- rc = bnxt_setup_nitroa0_vnic(bp);
- if (rc)
- netdev_err(bp->dev, "Special vnic setup failure for NS2 A0 rc: %x\n",
- rc);
- }
-
if (BNXT_VF(bp)) {
bnxt_hwrm_func_qcfg(bp);
netdev_update_features(bp->dev);
@@ -12147,26 +11998,19 @@ static void bnxt_del_napi(struct bnxt *bp)
static void bnxt_init_napi(struct bnxt *bp)
{
int (*poll_fn)(struct napi_struct *, int) = bnxt_poll;
- unsigned int cp_nr_rings = bp->cp_nr_rings;
struct bnxt_napi *bnapi;
int i;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
poll_fn = bnxt_poll_p5;
- else if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- cp_nr_rings--;
set_bit(BNXT_STATE_NAPI_DISABLED, &bp->state);
- for (i = 0; i < cp_nr_rings; i++) {
+ for (i = 0; i < bp->cp_nr_rings; i++) {
bnapi = bp->bnapi[i];
netif_napi_add_config_locked(bp->dev, &bnapi->napi, poll_fn,
bnapi->index);
}
- if (BNXT_CHIP_TYPE_NITRO_A0(bp)) {
- bnapi = bp->bnapi[cp_nr_rings];
- netif_napi_add_locked(bp->dev, &bnapi->napi, bnxt_poll_nitroa0);
- }
}
static void bnxt_disable_napi(struct bnxt *bp)
@@ -14040,7 +13884,7 @@ static bool bnxt_rfs_supported(struct bnxt *bp)
/* 212 firmware is broken for aRFS */
if (BNXT_FW_MAJ(bp) == 212)
return false;
- if (BNXT_PF(bp) && !BNXT_CHIP_TYPE_NITRO_A0(bp))
+ if (BNXT_PF(bp))
return true;
if (bp->rss_cap & BNXT_RSS_CAP_NEW_RSS_CAP)
return true;
@@ -16894,10 +16738,6 @@ static void _bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx,
if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS))
*max_cp = min_t(int, *max_cp, max_irq);
max_ring_grps = hw_resc->max_hw_ring_grps;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp) && BNXT_PF(bp)) {
- *max_cp -= 1;
- *max_rx -= 2;
- }
if (bp->flags & BNXT_FLAG_AGG_RINGS)
*max_rx >>= 1;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
@@ -17047,10 +16887,6 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
netdev_warn(bp->dev, "2nd rings reservation failed.\n");
bnxt_adj_tx_rings(bp);
}
- if (BNXT_CHIP_TYPE_NITRO_A0(bp)) {
- bp->rx_nr_rings++;
- bp->cp_nr_rings++;
- }
if (rc) {
bp->tx_nr_rings = 0;
bp->rx_nr_rings = 0;
@@ -17321,6 +17157,13 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (rc)
goto init_err_pci_clean;
+ if (bp->chip_num == CHIP_NUM_58700 && !bp->chip_rev &&
+ !bp->ver_resp.chip_metal) {
+ dev_err(&pdev->dev, "58700 A0 unsupported, aborting\n");
+ rc = -ENODEV;
+ goto init_err_pci_clean;
+ }
+
if (BNXT_PF(bp))
bnxt_vpd_read_info(bp);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 75428a585577..51a269d06c49 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2295,7 +2295,6 @@ struct bnxt {
#define BNXT_FLAG_DSN_VALID 0x200000
#define BNXT_FLAG_DOUBLE_DB 0x400000
#define BNXT_FLAG_UDP_GSO_CAP 0x800000
- #define BNXT_FLAG_CHIP_NITRO_A0 0x1000000
#define BNXT_FLAG_DIM 0x2000000
#define BNXT_FLAG_ROCE_MIRROR_CAP 0x4000000
#define BNXT_FLAG_TX_COAL_CMPL 0x8000000
@@ -2323,10 +2322,8 @@ struct bnxt {
#define BNXT_PHY_CFG_ABLE(bp) ((BNXT_SINGLE_PF(bp) || \
BNXT_SH_PORT_CFG_OK(bp)) && \
(bp)->link_info.phy_state == BNXT_PHY_STATE_ENABLED)
-#define BNXT_CHIP_TYPE_NITRO_A0(bp) ((bp)->flags & BNXT_FLAG_CHIP_NITRO_A0)
#define BNXT_RX_PAGE_MODE(bp) ((bp)->flags & BNXT_FLAG_RX_PAGE_MODE)
-#define BNXT_SUPPORTS_TPA(bp) (!BNXT_CHIP_TYPE_NITRO_A0(bp) && \
- (!((bp)->flags & BNXT_FLAG_CHIP_P5_PLUS) ||\
+#define BNXT_SUPPORTS_TPA(bp) ((!((bp)->flags & BNXT_FLAG_CHIP_P5_PLUS) ||\
(bp)->max_tpa_v2) && !is_kdump_kernel())
#define BNXT_RX_JUMBO_MODE(bp) ((bp)->flags & BNXT_FLAG_JUMBO)
@@ -2347,13 +2344,11 @@ struct bnxt {
(BNXT_CHIP_NUM_57X1X((bp)->chip_num) || \
BNXT_CHIP_NUM_5745X((bp)->chip_num) || \
BNXT_CHIP_NUM_588XX((bp)->chip_num) || \
- (BNXT_CHIP_NUM_58700((bp)->chip_num) && \
- !BNXT_CHIP_TYPE_NITRO_A0(bp)))
+ BNXT_CHIP_NUM_58700((bp)->chip_num))
/* Chip class phase 3.x */
#define BNXT_CHIP_P3(bp) \
- (BNXT_CHIP_NUM_57X0X((bp)->chip_num) || \
- BNXT_CHIP_TYPE_NITRO_A0(bp))
+ (BNXT_CHIP_NUM_57X0X((bp)->chip_num))
#define BNXT_CHIP_P4_PLUS(bp) \
(BNXT_CHIP_P4(bp) || BNXT_CHIP_P5_PLUS(bp))
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 0739cd80a966..ac9361e4448c 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -949,13 +949,9 @@ static void bnxt_get_channels(struct net_device *dev,
channel->max_other = 0;
if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
channel->combined_count = bp->rx_nr_rings;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp))
- channel->combined_count--;
} else {
- if (!BNXT_CHIP_TYPE_NITRO_A0(bp)) {
- channel->rx_count = bp->rx_nr_rings;
- channel->tx_count = bp->tx_nr_rings_per_tc;
- }
+ channel->rx_count = bp->rx_nr_rings;
+ channel->tx_count = bp->tx_nr_rings_per_tc;
}
}
@@ -980,10 +976,6 @@ static int bnxt_set_channels(struct net_device *dev,
(channel->rx_count || channel->tx_count))
return -EINVAL;
- if (BNXT_CHIP_TYPE_NITRO_A0(bp) && (channel->rx_count ||
- channel->tx_count))
- return -EINVAL;
-
if (channel->combined_count)
sh = true;
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 7/9] bnxt_en: Refactor bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
` (5 preceding siblings ...)
2026-11-20 1:29 ` [PATCH net-next v5 6/9] bnxt_en: Remove support for the old NITRO A0 chip Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 8/9] bnxt_en: Fix ring accounting/validation when rings are constrained by FW Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 9/9] bnxt_en: Fix ring accounting/validation when IRQs are constrained Michael Chan
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
Add 2 parameters tx_xdp and rx to these 2 helpers. tx_xdp will be
passed in by the caller instead of always using bp->tx_nr_rings_xdp.
All current callers will pass in bp->tx_nr_rings_xdp so there is no
change in behavior. The rx parameter is unused until the next patch
to clamp the XDP rings. The callers now pass in the correct rx
value for this purpose.
To be consistent in bnxt_num_cp_to_tx(), tcs is now correctly set to 1
when bp->num_tc is 0.
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 28 +++++++++++------------
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +-
2 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index b49e02fcde29..d76227d29282 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -8180,7 +8180,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
rc = bnxt_trim_rings(bp, &rx_rings, &hwr.tx, hwr.cp, sh);
if (bp->flags & BNXT_FLAG_AGG_RINGS)
hwr.rx = rx_rings << 1;
- tx_cp = bnxt_num_tx_to_cp(bp, hwr.tx);
+ tx_cp = bnxt_num_tx_to_cp(bp, hwr.tx, bp->tx_nr_rings_xdp, rx_rings);
hwr.cp = sh ? max_t(int, tx_cp, rx_rings) : tx_cp + rx_rings;
if (hwr.tx != bp->tx_nr_rings) {
netdev_warn(bp->dev,
@@ -11313,27 +11313,24 @@ static int __bnxt_num_tx_to_cp(struct bnxt *bp, int tx, int tx_sets, int tx_xdp)
return (tx - tx_xdp) / tx_sets + tx_xdp;
}
-int bnxt_num_tx_to_cp(struct bnxt *bp, int tx)
+int bnxt_num_tx_to_cp(struct bnxt *bp, int tx, int tx_xdp, int rx)
{
- int tcs = bp->num_tc;
+ int tcs = bp->num_tc ? bp->num_tc : 1;
- if (!tcs)
- tcs = 1;
- return __bnxt_num_tx_to_cp(bp, tx, tcs, bp->tx_nr_rings_xdp);
+ return __bnxt_num_tx_to_cp(bp, tx, tcs, tx_xdp);
}
-static int bnxt_num_cp_to_tx(struct bnxt *bp, int tx_cp)
+static int bnxt_num_cp_to_tx(struct bnxt *bp, int tx_cp, int tx_xdp, int rx)
{
- int tcs = bp->num_tc;
+ int tcs = bp->num_tc ? bp->num_tc : 1;
- return (tx_cp - bp->tx_nr_rings_xdp) * tcs +
- bp->tx_nr_rings_xdp;
+ return (tx_cp - tx_xdp) * tcs + tx_xdp;
}
static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
bool sh)
{
- int tx_cp = bnxt_num_tx_to_cp(bp, *tx);
+ int tx_cp = bnxt_num_tx_to_cp(bp, *tx, bp->tx_nr_rings_xdp, *rx);
if (tx_cp != *tx) {
int tx_saved = tx_cp, rc;
@@ -11342,7 +11339,8 @@ static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
if (rc)
return rc;
if (tx_cp != tx_saved)
- *tx = bnxt_num_cp_to_tx(bp, tx_cp);
+ *tx = bnxt_num_cp_to_tx(bp, tx_cp, bp->tx_nr_rings_xdp,
+ *rx);
return 0;
}
return __bnxt_trim_rings(bp, rx, tx, max, sh);
@@ -11556,7 +11554,8 @@ static int bnxt_init_int_mode(struct bnxt *bp)
if (rc)
goto msix_setup_exit;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
+ tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings, bp->tx_nr_rings_xdp,
+ bp->rx_nr_rings);
bp->cp_nr_rings = (min == 1) ?
max_t(int, tx_cp, bp->rx_nr_rings) :
tx_cp + bp->rx_nr_rings;
@@ -15716,7 +15715,8 @@ static int bnxt_change_mtu(struct net_device *dev, int new_mtu)
void bnxt_set_cp_rings(struct bnxt *bp, bool sh)
{
- int tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
+ int tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings, bp->tx_nr_rings_xdp,
+ bp->rx_nr_rings);
bp->cp_nr_rings = sh ? max_t(int, tx_cp, bp->rx_nr_rings) :
tx_cp + bp->rx_nr_rings;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 51a269d06c49..5ca98394a9ab 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -3003,7 +3003,7 @@ int bnxt_hwrm_set_coal(struct bnxt *);
size_t bnxt_copy_ctx_mem(struct bnxt *bp, struct bnxt_ctx_mem_type *ctxm,
void *buf, size_t offset);
void bnxt_free_ctx_mem(struct bnxt *bp, bool force);
-int bnxt_num_tx_to_cp(struct bnxt *bp, int tx);
+int bnxt_num_tx_to_cp(struct bnxt *bp, int tx, int tx_xdp, int rx);
unsigned int bnxt_get_max_func_stat_ctxs(struct bnxt *bp);
unsigned int bnxt_get_avail_stat_ctxs_for_en(struct bnxt *bp);
unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp);
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 8/9] bnxt_en: Fix ring accounting/validation when rings are constrained by FW
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
` (6 preceding siblings ...)
2026-11-20 1:29 ` [PATCH net-next v5 7/9] bnxt_en: Refactor bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 9/9] bnxt_en: Fix ring accounting/validation when IRQs are constrained Michael Chan
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
When __bnxt_reserve_rings() reserves fewer TX rings than requested,
and an XDP program is attached, the driver used to blindly subtract
bp->tx_nr_rings_xdp from bp->tx_nr_rings, potentially causing an integer
underflow.
This patch mainly fixes the existing bnxt_adj_tx_rings(),
bnxt_rings_ok(), some ring helpers, and __bnxt_reserve_rings():
1. bnxt_adj_tx_rings() is now renamed bnxt_adj_rings() to reflect that
all rings (tx, rx, cp) may adjust if needed. It will now correctly
scale down TX rings for XDP and TCs evenly. In combined channel mode,
RX and TX rings are now clamped to match each other - required by
XDP's 1:1 TX/RX mapping and now applied to non-XDP mode for
consistency. CP rings will also be reduced if necessary. Any
leftover rings after integer division are intentionally left unused.
2. bnxt_rings_ok() is now updated to robustly check for the absolute
minimum viable ring configuration to support the TC and XDP settings.
We now call bnxt_rings_ok() first before calling bnxt_adj_rings() to
guarantee no math underflow or ring truncation to 0.
3. bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers are used while
trimming the rings with possible ring shortages. These helpers now
use the passed in tx_xdp (now used as boolean) and rx parameters to
accurately scale and clamp the XDP rings.
4. __bnxt_reserve_rings() now only calls bnxt_adj_rings() if
bp->bnapi has not been allocated. In this context, we may be expanding
and asking for more rings than FW can grant and we need to call
bnxt_adj_rings() for possible adjustments. We now abort
__bnxt_reserve_rings() for all fatal conditions through the common
reserve_rings_exit label. We now leave the ring states unchanged and
the ULP resources reconciled during abort.
We also need to reset the RSS table of the default context as needed
if the RX rings have shrunk. As noted in the cover letter, we have
similar existing issues regarding RSS tables for non-default RSS
contexts and user defined n-tuple filters when RX rings shrink due
to FW or MSI-X constraints. The fixes for these are deferred to a
separate patchset.
The next patch will fix similar issues when MSI-X vectors are
constrained.
These existing issues were detected by Sashiko when reviewing the
new kTLS patchset (patch #3 of 15):
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260810051358.1244418-7-michael.chan@broadcom.com
Fixes: 1ee581c24dfd ("bnxt_en: Adjust TX rings if reservation is less than requested")
Fixes: 674f50a5b026 ("bnxt_en: Implement new method to reserve rings.")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
v5:
Split out the MSI-X changes to the new patch #9.
v4:
Fix bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers to fix the
ring trimming issue reported by Sashiko.
https://lore.kernel.org/netdev/20261016064427.496944-9-michael.chan@broadcom.com/
v3:
No longer have to deal with dropped NITRO A0
Fix the dynamic IRQ code path in bnxt_reserve_rings().
Use the proper __bnxt_num_tx_to_cp() in bnxt_rings_ok().
https://lore.kernel.org/netdev/20261012220827.340998-8-michael.chan@broadcom.com/
v2:
Reduce RX and CP rings if needed when TX rings are reduced to keep it
consistent.
Handle NITRO_A0 special RX requirements to keep it consistent.
Fix all unwind issues in these code paths.
https://lore.kernel.org/netdev/20260928041712.3467803-7-michael.chan@broadcom.com/
v1:
https://lore.kernel.org/netdev/20260831024342.2161156-2-michael.chan@broadcom.com/
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 137 +++++++++++++++++-----
1 file changed, 106 insertions(+), 31 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index d76227d29282..565099088969 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -8086,10 +8086,32 @@ static void bnxt_copy_reserved_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)
}
}
+static int bnxt_min_tx_rings(struct bnxt *bp)
+{
+ int min_tx = bp->num_tc ? bp->num_tc : 1;
+
+ return bp->tx_nr_rings_xdp ? min_tx + 1 : min_tx;
+}
+
static bool bnxt_rings_ok(struct bnxt *bp, struct bnxt_hw_rings *hwr)
{
- return hwr->tx && hwr->rx && hwr->cp && hwr->grp && hwr->vnic &&
- hwr->stat && (hwr->cp_p5 || !(bp->flags & BNXT_FLAG_CHIP_P5_PLUS));
+ int min_tx, min_tx_xdp = 0;
+ int min_rx = 1;
+ int min_cp;
+ int tx_cp;
+
+ min_tx = bnxt_min_tx_rings(bp);
+ if (bp->tx_nr_rings_xdp)
+ min_tx_xdp = 1;
+
+ tx_cp = bnxt_num_tx_to_cp(bp, min_tx, min_tx_xdp, min_rx);
+ if (bp->flags & BNXT_FLAG_SHARED_RINGS)
+ min_cp = max(tx_cp, min_rx);
+ else
+ min_cp = tx_cp + min_rx;
+ return hwr->tx >= min_tx && hwr->rx >= min_rx && hwr->cp >= min_cp &&
+ hwr->grp && hwr->vnic && hwr->stat &&
+ (hwr->cp_p5 || !(bp->flags & BNXT_FLAG_CHIP_P5_PLUS));
}
/* Check to see if we need to reset the user configured RSS table
@@ -8107,10 +8129,13 @@ static void bnxt_check_rss_tbl_lost(struct bnxt *bp, int old_rx, int new_rx)
}
static int bnxt_get_avail_msix(struct bnxt *bp, int num);
+static void bnxt_adj_rings(struct bnxt *bp);
static int __bnxt_reserve_rings(struct bnxt *bp)
{
struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA];
+ int req_rx_rings = bp->rx_nr_rings;
+ int req_tx_rings = bp->tx_nr_rings;
struct bnxt_hw_rings hwr = {0};
int rx_rings, old_rx_rings, rc;
int cp = bp->cp_nr_rings;
@@ -8153,7 +8178,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
rc = bnxt_hwrm_reserve_rings(bp, &hwr);
if (rc)
- return rc;
+ goto reserve_rings_exit;
bnxt_copy_reserved_rings(bp, &hwr);
@@ -8162,8 +8187,10 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
if (hwr.rx >= 2) {
rx_rings = hwr.rx >> 1;
} else {
- if (netif_running(bp->dev))
- return -ENOMEM;
+ if (netif_running(bp->dev)) {
+ rc = -ENOMEM;
+ goto reserve_rings_exit;
+ }
bp->flags &= ~BNXT_FLAG_AGG_RINGS;
bp->flags |= BNXT_FLAG_NO_AGG_RINGS;
@@ -8178,49 +8205,67 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
hwr.stat -= bnxt_get_ulp_stat_ctxs(bp);
hwr.cp = min_t(int, hwr.cp, hwr.stat);
rc = bnxt_trim_rings(bp, &rx_rings, &hwr.tx, hwr.cp, sh);
+ if (rc)
+ goto reserve_rings_exit;
+
+ if (bp->bnapi && (rx_rings < req_rx_rings ||
+ hwr.tx < req_tx_rings)) {
+ netdev_err(bp->dev, "Unexpected ring shrinkage %d/%d RX/TX to %d/%d\n",
+ req_rx_rings, req_tx_rings, rx_rings, hwr.tx);
+ rc = -ENOSPC;
+ goto reserve_rings_exit;
+ }
if (bp->flags & BNXT_FLAG_AGG_RINGS)
hwr.rx = rx_rings << 1;
tx_cp = bnxt_num_tx_to_cp(bp, hwr.tx, bp->tx_nr_rings_xdp, rx_rings);
hwr.cp = sh ? max_t(int, tx_cp, rx_rings) : tx_cp + rx_rings;
- if (hwr.tx != bp->tx_nr_rings) {
- netdev_warn(bp->dev,
- "Able to reserve only %d out of %d requested TX rings\n",
- hwr.tx, bp->tx_nr_rings);
+
+ if (!bnxt_rings_ok(bp, &hwr)) {
+ rc = -ENOMEM;
+ goto reserve_rings_exit;
}
bp->tx_nr_rings = hwr.tx;
+ bp->rx_nr_rings = rx_rings;
+ bp->cp_nr_rings = hwr.cp;
+ if (!bp->bnapi)
+ bnxt_adj_rings(bp);
+
+ if (req_tx_rings != bp->tx_nr_rings) {
+ netdev_warn(bp->dev,
+ "Able to reserve only %d out of %d requested TX rings\n",
+ bp->tx_nr_rings, req_tx_rings);
+ }
/* If we cannot reserve all the RX rings, reset the RSS map only
* if absolutely necessary
*/
- if (rx_rings != bp->rx_nr_rings) {
+ if (req_rx_rings != bp->rx_nr_rings) {
netdev_warn(bp->dev, "Able to reserve only %d out of %d requested RX rings\n",
- rx_rings, bp->rx_nr_rings);
- bnxt_check_rss_tbl_lost(bp, bp->rx_nr_rings, rx_rings);
+ bp->rx_nr_rings, req_rx_rings);
+ bnxt_check_rss_tbl_lost(bp, req_rx_rings, bp->rx_nr_rings);
}
- bp->rx_nr_rings = rx_rings;
- bp->cp_nr_rings = hwr.cp;
/* Fall back if we cannot reserve enough HW RSS contexts */
if ((bp->rss_cap & BNXT_RSS_CAP_LARGE_RSS_CTX) &&
hwr.rss_ctx < bnxt_get_total_rss_ctxs(bp, &hwr))
bp->rss_cap &= ~BNXT_RSS_CAP_LARGE_RSS_CTX;
- if (!bnxt_rings_ok(bp, &hwr))
- return -ENOMEM;
-
- if (old_rx_rings != bp->hw_resc.resv_rx_rings &&
+ if ((old_rx_rings != bp->hw_resc.resv_rx_rings ||
+ req_rx_rings != bp->rx_nr_rings) &&
!netif_is_rxfh_configured(bp->dev))
bnxt_set_dflt_rss_indir_tbl(bp, NULL);
+reserve_rings_exit:
if (!bnxt_ulp_registered(edev) && BNXT_NEW_RM(bp)) {
int resv_msix, resv_ctx, ulp_ctxs;
struct bnxt_hw_resc *hw_resc;
hw_resc = &bp->hw_resc;
- resv_msix = hw_resc->resv_irqs - bp->cp_nr_rings;
+ resv_msix = max_t(int, hw_resc->resv_irqs - bp->cp_nr_rings, 0);
ulp_msix = min_t(int, resv_msix, ulp_msix);
bnxt_set_ulp_msix_num(bp, ulp_msix);
- resv_ctx = hw_resc->resv_stat_ctxs - bp->cp_nr_rings;
+ resv_ctx = max_t(int, hw_resc->resv_stat_ctxs - bp->cp_nr_rings,
+ 0);
ulp_ctxs = min(resv_ctx, bnxt_get_ulp_stat_ctxs(bp));
bnxt_set_ulp_stat_ctxs(bp, ulp_ctxs);
}
@@ -11317,14 +11362,27 @@ int bnxt_num_tx_to_cp(struct bnxt *bp, int tx, int tx_xdp, int rx)
{
int tcs = bp->num_tc ? bp->num_tc : 1;
+ if (tx_xdp) {
+ tx_xdp = tx / (tcs + 1);
+ tx_xdp = min(tx_xdp, rx);
+ }
return __bnxt_num_tx_to_cp(bp, tx, tcs, tx_xdp);
}
static int bnxt_num_cp_to_tx(struct bnxt *bp, int tx_cp, int tx_xdp, int rx)
{
int tcs = bp->num_tc ? bp->num_tc : 1;
+ int cp_per_tc;
- return (tx_cp - tx_xdp) * tcs + tx_xdp;
+ if (tx_xdp) {
+ /* The CP rings are divided equally between XDP TX rings and
+ * the regular TX rings.
+ */
+ cp_per_tc = tx_cp / 2;
+ cp_per_tc = min(cp_per_tc, rx);
+ return cp_per_tc * (tcs + 1);
+ }
+ return tx_cp * tcs;
}
static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
@@ -13156,13 +13214,29 @@ static void bnxt_set_xdp_tx_rings(struct bnxt *bp)
bp->tx_nr_rings += bp->tx_nr_rings_xdp;
}
-static void bnxt_adj_tx_rings(struct bnxt *bp)
+static void bnxt_adj_rings(struct bnxt *bp)
{
- /* Make adjustments if reserved TX rings are less than requested */
- bp->tx_nr_rings -= bp->tx_nr_rings_xdp;
- bp->tx_nr_rings_per_tc = bnxt_tx_nr_rings_per_tc(bp);
- if (bp->tx_nr_rings_xdp)
- bnxt_set_xdp_tx_rings(bp);
+ int tcs = bp->num_tc ? bp->num_tc : 1;
+ int rx = bp->rx_nr_rings;
+
+ /* Make adjustments to rings based on TC/XDP/shared rings policies */
+ if (bp->tx_nr_rings_xdp) {
+ tcs++;
+ bp->tx_nr_rings_per_tc = bp->tx_nr_rings / tcs;
+ bp->tx_nr_rings_xdp = bp->tx_nr_rings_per_tc;
+ } else {
+ bp->tx_nr_rings_per_tc = bnxt_tx_nr_rings_per_tc(bp);
+ }
+
+ if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
+ bp->tx_nr_rings_per_tc = min(bp->tx_nr_rings_per_tc, rx);
+ bp->rx_nr_rings = bp->tx_nr_rings_per_tc;
+ if (bp->tx_nr_rings_xdp)
+ bp->tx_nr_rings_xdp = bp->tx_nr_rings_per_tc;
+ }
+
+ bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tcs;
+ bnxt_set_cp_rings(bp, bp->flags & BNXT_FLAG_SHARED_RINGS);
}
static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
@@ -13182,7 +13256,8 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
if (rc)
return rc;
- bnxt_adj_tx_rings(bp);
+ if (irq_re_init)
+ bnxt_adj_rings(bp);
rc = bnxt_alloc_mem(bp, irq_re_init);
if (rc) {
netdev_err(bp->dev, "bnxt_alloc_mem err: %x\n", rc);
@@ -16876,7 +16951,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
if (rc && rc != -ECOMM)
netdev_warn(bp->dev, "Unable to reserve tx rings\n");
- bnxt_adj_tx_rings(bp);
+ bnxt_adj_rings(bp);
if (sh)
bnxt_adj_dflt_rings(bp, true);
@@ -16885,7 +16960,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
rc = __bnxt_reserve_rings(bp);
if (rc && rc != -ECOMM)
netdev_warn(bp->dev, "2nd rings reservation failed.\n");
- bnxt_adj_tx_rings(bp);
+ bnxt_adj_rings(bp);
}
if (rc) {
bp->tx_nr_rings = 0;
@@ -16915,7 +16990,7 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)
if (rc)
goto init_dflt_ring_err;
- bnxt_adj_tx_rings(bp);
+ bnxt_adj_rings(bp);
bnxt_set_dflt_rfs(bp);
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH net-next v5 9/9] bnxt_en: Fix ring accounting/validation when IRQs are constrained
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
` (7 preceding siblings ...)
2026-11-20 1:29 ` [PATCH net-next v5 8/9] bnxt_en: Fix ring accounting/validation when rings are constrained by FW Michael Chan
@ 2026-11-20 1:29 ` Michael Chan
8 siblings, 0 replies; 10+ messages in thread
From: Michael Chan @ 2026-11-20 1:29 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek
Similar to the previous patch that handles ring shortages from the FW,
this patch handles MSI-X vector shortages. We use the similar logic
and infrastructure to adjust the rings if there are not enough MSI-X
vectors to satisfy the current ring configuration.
Add a new bnxt_adj_rings_to_irqs() helper to adjust rings to the
available MSI-X vectors. The helper will be used when static or
dynamic MSI-X vectors are constrained. Note that the helper will
return success early if rings have not been reserved yet.
We can now centralize ring adjustments during ring reservations and
IRQ allocations. Other ring adjustments and checks scattered elsewhere
can now be deleted.
We now remove the error path at the end of bnxt_reserve_rings() to abort
and reset the TCs if the rings cannot satisfy the TC requirements. The
open will now fail and the user will have to reduce the TCs and retry.
This failure should be very rare because we first call
bnxt_check_rings() to make sure there is high likelihood of success
before committing.
Just like the previous patch, we have similar existing issues regarding
RSS tables for non-default RSS contexts and user defined n-tuple filters
when RX rings shrink due to MSI-X constraints. The fixes for these are
deferred to a separate patchset.
Fixes: 1ee581c24dfd ("bnxt_en: Adjust TX rings if reservation is less than requested")
Fixes: 674f50a5b026 ("bnxt_en: Implement new method to reserve rings.")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
v5:
New patch split from patch #8.
Fixed a Sashiko issue in bnxt_init_int_mode() with no rings reserved.
Rebased to handle dynamic MSI-X vectors after:
af449230bd8d ("bnxt_en: Don't keep MSI-X vectors added before FW reserves them")
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 110 ++++++++++++++--------
1 file changed, 73 insertions(+), 37 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 565099088969..8ec9576371cb 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -11569,9 +11569,74 @@ static int bnxt_get_num_msix(struct bnxt *bp)
return bnxt_nq_rings_in_use(bp);
}
+/* Adjust rings to fit bp->total_irqs */
+static int bnxt_adj_rings_to_irqs(struct bnxt *bp)
+{
+ int req_rx_rings, req_tx_rings, req_cp_rings, req_tx_per_tc, req_tx_xdp;
+ int ulp_msix = bnxt_get_ulp_msix_num(bp);
+ int min_tx, rc;
+
+ /* Nothing to adjust if rings have not been setup yet, e.g. after
+ * bnxt_clear_reservations() during devlink reload, or on a VF with
+ * 0 rings provisioned by the PF. The caller must ensure that genuine
+ * ring shortage cannot reach here.
+ */
+ if (!bp->tx_nr_rings)
+ return 0;
+
+ req_tx_rings = bp->tx_nr_rings;
+ req_tx_per_tc = bp->tx_nr_rings_per_tc;
+ req_tx_xdp = bp->tx_nr_rings_xdp;
+ req_rx_rings = bp->rx_nr_rings;
+ req_cp_rings = bp->cp_nr_rings;
+
+ /* Trim rings based upon num of vectors allocated */
+ rc = bnxt_trim_rings(bp, &bp->rx_nr_rings, &bp->tx_nr_rings,
+ bp->total_irqs - ulp_msix,
+ bp->flags & BNXT_FLAG_SHARED_RINGS);
+ if (rc)
+ goto adj_irqs_err;
+
+ min_tx = bnxt_min_tx_rings(bp);
+ if (bp->tx_nr_rings < min_tx) {
+ netdev_err(bp->dev, "Not enough MSI-X to satisfy minimum TX rings\n");
+ rc = -ENOMEM;
+ goto adj_irqs_err;
+ }
+
+ bnxt_adj_rings(bp);
+
+ if (bp->bnapi && (req_tx_rings != bp->tx_nr_rings ||
+ req_rx_rings != bp->rx_nr_rings)) {
+ netdev_err(bp->dev, "Cannot shrink rings once NAPI is allocated\n");
+ rc = -ENOSPC;
+ goto adj_irqs_err;
+ }
+
+ if (req_tx_rings != bp->tx_nr_rings)
+ netdev_warn(bp->dev, "TX rings reduced from %d to %d\n",
+ req_tx_rings, bp->tx_nr_rings);
+ if (req_rx_rings != bp->rx_nr_rings) {
+ netdev_warn(bp->dev, "RX rings reduced from %d to %d\n",
+ req_rx_rings, bp->rx_nr_rings);
+ bnxt_check_rss_tbl_lost(bp, req_rx_rings, bp->rx_nr_rings);
+ if (!netif_is_rxfh_configured(bp->dev))
+ bnxt_set_dflt_rss_indir_tbl(bp, NULL);
+ }
+ return 0;
+
+adj_irqs_err:
+ bp->tx_nr_rings = req_tx_rings;
+ bp->tx_nr_rings_per_tc = req_tx_per_tc;
+ bp->tx_nr_rings_xdp = req_tx_xdp;
+ bp->rx_nr_rings = req_rx_rings;
+ bp->cp_nr_rings = req_cp_rings;
+ return rc;
+}
+
static int bnxt_init_int_mode(struct bnxt *bp)
{
- int i, total_vecs, max, rc, min = 1, ulp_msix, tx_cp, tbl_size;
+ int i, total_vecs, max, rc, min = 1, ulp_msix, tbl_size;
total_vecs = bnxt_get_num_msix(bp);
max = bnxt_get_max_func_irqs(bp);
@@ -11606,18 +11671,10 @@ static int bnxt_init_int_mode(struct bnxt *bp)
bp->irq_tbl[i].vector = pci_irq_vector(bp->pdev, i);
bp->total_irqs = total_vecs;
- /* Trim rings based upon num of vectors allocated */
- rc = bnxt_trim_rings(bp, &bp->rx_nr_rings, &bp->tx_nr_rings,
- total_vecs - ulp_msix, min == 1);
+ rc = bnxt_adj_rings_to_irqs(bp);
if (rc)
goto msix_setup_exit;
- tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings, bp->tx_nr_rings_xdp,
- bp->rx_nr_rings);
- bp->cp_nr_rings = (min == 1) ?
- max_t(int, tx_cp, bp->rx_nr_rings) :
- tx_cp + bp->rx_nr_rings;
-
return 0;
msix_setup_exit:
@@ -11658,7 +11715,6 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
{
bool irq_cleared = false;
bool irq_change = false;
- int tcs = bp->num_tc;
int irqs_required;
int rc;
@@ -11672,9 +11728,11 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
irqs_required = bnxt_get_num_msix(bp);
if (irqs_required > bp->total_irqs &&
bnxt_change_msix(bp, irqs_required) != irqs_required) {
- netdev_err(bp->dev, "Unable to allocate %d MSIX vectors, maximum available %d\n",
- irqs_required, bp->total_irqs);
- return -ENOSPC;
+ netdev_warn(bp->dev, "Able to allocate only %d MSIX vectors out of %d requested\n",
+ bp->total_irqs, irqs_required);
+ rc = bnxt_adj_rings_to_irqs(bp);
+ if (rc)
+ return rc;
}
}
return 0;
@@ -11700,23 +11758,12 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
*/
irqs_required = bnxt_get_num_msix(bp);
if (bnxt_change_msix(bp, irqs_required) != irqs_required)
- rc = -ENOSPC;
+ rc = bnxt_adj_rings_to_irqs(bp);
}
if (rc) {
netdev_err(bp->dev, "ring reservation/IRQ init failure rc: %d\n", rc);
return rc;
}
- if (tcs && (bp->tx_nr_rings_per_tc * tcs !=
- bp->tx_nr_rings - bp->tx_nr_rings_xdp)) {
- netdev_err(bp->dev, "tx ring reservation failure\n");
- netdev_reset_tc(bp->dev);
- bp->num_tc = 0;
- if (bp->tx_nr_rings_xdp)
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings_xdp;
- else
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
- return -ENOMEM;
- }
return 0;
}
@@ -13256,8 +13303,6 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
if (rc)
return rc;
- if (irq_re_init)
- bnxt_adj_rings(bp);
rc = bnxt_alloc_mem(bp, irq_re_init);
if (rc) {
netdev_err(bp->dev, "bnxt_alloc_mem err: %x\n", rc);
@@ -16951,7 +16996,6 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
if (rc && rc != -ECOMM)
netdev_warn(bp->dev, "Unable to reserve tx rings\n");
- bnxt_adj_rings(bp);
if (sh)
bnxt_adj_dflt_rings(bp, true);
@@ -16960,7 +17004,6 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
rc = __bnxt_reserve_rings(bp);
if (rc && rc != -ECOMM)
netdev_warn(bp->dev, "2nd rings reservation failed.\n");
- bnxt_adj_rings(bp);
}
if (rc) {
bp->tx_nr_rings = 0;
@@ -16990,8 +17033,6 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)
if (rc)
goto init_dflt_ring_err;
- bnxt_adj_rings(bp);
-
bnxt_set_dflt_rfs(bp);
init_dflt_ring_err:
@@ -17385,11 +17426,6 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (rc)
goto init_err_pci_clean;
- /* No TC has been set yet and rings may have been trimmed due to
- * limited MSIX, so we re-initialize the TX rings per TC.
- */
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
-
if (BNXT_PF(bp)) {
if (!bnxt_pf_wq) {
bnxt_pf_wq =
--
2.51.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-11-20 1:30 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-20 1:29 [PATCH net-next v5 0/9] bnxt_en: Ring accounting fixes Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 1/9] bnxt_en: Clear bp->total_irqs when disabling IRQs Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 2/9] bnxt_en: Fix bnxt_reinit_features() when irq_re_init is true Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 3/9] bnxt_en: Refactor RSS table check logic Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 4/9] bnxt_en: Refactor IRQs required logic Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 5/9] bnxt_en: Reinit IRQ when configuring LRO/GRO/HDS Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 6/9] bnxt_en: Remove support for the old NITRO A0 chip Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 7/9] bnxt_en: Refactor bnxt_num_cp_to_tx()/bnxt_num_tx_to_cp() helpers Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 8/9] bnxt_en: Fix ring accounting/validation when rings are constrained by FW Michael Chan
2026-11-20 1:29 ` [PATCH net-next v5 9/9] bnxt_en: Fix ring accounting/validation when IRQs are constrained Michael Chan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox