* [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support
@ 2025-01-16 19:23 Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 01/10] bnxt_en: Set NAPR 1.2 support when registering with firmware Michael Chan
` (9 more replies)
0 siblings, 10 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas
The first patch adds NPAR 1.2 support. Patches 2 to 10 add TPH
(TLP Processing Hints) support. These TPH driver patches are new
revisions originally posted as part of the TPH PCI patch series.
Additional driver refactoring has been done so that we can free
and allocate RX completion ring and the TX rings if the channel is
a combined channel. We also add napi_disable() and napi_enable()
during queue_stop() and queue_start() respectively, and reset for
error handling in queue_start().
v2:
Major change is error handling in patch #9.
v1:
https://lore.kernel.org/netdev/20250113063927.4017173-1-michael.chan@broadcom.com/
Discussion about adding napi_disable()/napi_enable():
https://lore.kernel.org/netdev/5336d624-8d8b-40a6-b732-b020e4a119a2@davidwei.uk/#t
Previous driver series fixing rtnl_lock and empty release function:
https://lore.kernel.org/netdev/20241115200412.1340286-1-wei.huang2@amd.com/
v5 of the PCI series using netdev_rx_queue_restart():
https://lore.kernel.org/netdev/20240916205103.3882081-5-wei.huang2@amd.com/
v1 of the PCI series using open/close:
https://lore.kernel.org/netdev/20240509162741.1937586-9-wei.huang2@amd.com/
Manoj Panicker (1):
bnxt_en: Add TPH support in BNXT driver
Michael Chan (5):
bnxt_en: Set NAPR 1.2 support when registering with firmware
bnxt_en Refactor completion ring allocation logic for P5_PLUS chips
bnxt_en: Refactor TX ring allocation logic
bnxt_en: Refactor RX/RX AGG ring parameters setup for P5_PLUS
bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings
Somnath Kotur (4):
bnxt_en: Refactor completion ring free routine
bnxt_en: Refactor bnxt_free_tx_rings() to free per TX ring
bnxt_en: Reallocate RX completion ring for TPH support
bnxt_en: Extend queue stop/start for TX rings
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 519 ++++++++++++++++------
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 8 +
2 files changed, 396 insertions(+), 131 deletions(-)
--
2.30.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH net-next v2 01/10] bnxt_en: Set NAPR 1.2 support when registering with firmware
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 02/10] bnxt_en Refactor completion ring allocation logic for P5_PLUS chips Michael Chan
` (8 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Somnath Kotur
NPAR 1.2 adds a transparent VLAN tag for all packets between the NIC
and the switch. Because of that, RX VLAN acceleration cannot be
supported for any additional host configured VLANs. The driver has
to acknowledge that it can support no RX VLAN acceleration and
set the NPAR 1.2 supported flag when registering with the FW.
Otherwise, the FW call will fail and the driver will abort on these
NPAR 1.2 NICs with this error:
bnxt_en 0000:26:00.0 (unnamed net_device) (uninitialized): hwrm req_type 0x1d seq id 0xb error 0x2
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++++
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 +
2 files changed, 6 insertions(+)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index caddb5cbc024..5fda41acbb5a 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -5543,6 +5543,8 @@ int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp, unsigned long *bmap, int bmap_size,
if (bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY)
flags |= FUNC_DRV_RGTR_REQ_FLAGS_ERROR_RECOVERY_SUPPORT |
FUNC_DRV_RGTR_REQ_FLAGS_MASTER_SUPPORT;
+ if (bp->fw_cap & BNXT_FW_CAP_NPAR_1_2)
+ flags |= FUNC_DRV_RGTR_REQ_FLAGS_NPAR_1_2_SUPPORT;
req->flags = cpu_to_le32(flags);
req->ver_maj_8b = DRV_VER_MAJ;
req->ver_min_8b = DRV_VER_MIN;
@@ -8343,6 +8345,7 @@ static int bnxt_hwrm_func_qcfg(struct bnxt *bp)
switch (resp->port_partition_type) {
case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_0:
+ case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_2:
case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_5:
case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR2_0:
bp->port_partition_type = resp->port_partition_type;
@@ -9507,6 +9510,8 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)
bp->fw_cap |= BNXT_FW_CAP_HOT_RESET_IF;
if (BNXT_PF(bp) && (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_FW_LIVEPATCH_SUPPORTED))
bp->fw_cap |= BNXT_FW_CAP_LIVEPATCH;
+ if (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_NPAR_1_2_SUPPORTED)
+ bp->fw_cap |= BNXT_FW_CAP_NPAR_1_2;
if (BNXT_PF(bp) && (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_DFLT_VLAN_TPID_PCP_SUPPORTED))
bp->fw_cap |= BNXT_FW_CAP_DFLT_VLAN_TPID_PCP;
if (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_BS_V2_SUPPORTED)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 8f481dd9c224..826ae030fc09 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2492,6 +2492,7 @@ struct bnxt {
#define BNXT_FW_CAP_CFA_RFS_RING_TBL_IDX_V3 BIT_ULL(39)
#define BNXT_FW_CAP_VNIC_RE_FLUSH BIT_ULL(40)
#define BNXT_FW_CAP_SW_MAX_RESOURCE_LIMITS BIT_ULL(41)
+ #define BNXT_FW_CAP_NPAR_1_2 BIT_ULL(42)
u32 fw_dbg_cap;
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 02/10] bnxt_en Refactor completion ring allocation logic for P5_PLUS chips
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 01/10] bnxt_en: Set NAPR 1.2 support when registering with firmware Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 03/10] bnxt_en: Refactor TX ring allocation logic Michael Chan
` (7 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Ajit Khaparde
Add a new bnxt_hwrm_cp_ring_alloc_p5() function to handle allocating
one completion ring on P5_PLUS chips. This simplifies the existing code
and will be useful later in the series.
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
v2: Use const for a variable
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 44 +++++++++++------------
1 file changed, 21 insertions(+), 23 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 5fda41acbb5a..72fff9c10413 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -7177,6 +7177,25 @@ static int bnxt_hwrm_rx_agg_ring_alloc(struct bnxt *bp,
return 0;
}
+static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,
+ struct bnxt_cp_ring_info *cpr)
+{
+ const u32 type = HWRM_RING_ALLOC_CMPL;
+ struct bnxt_napi *bnapi = cpr->bnapi;
+ struct bnxt_ring_struct *ring;
+ u32 map_idx = bnapi->index;
+ int rc;
+
+ ring = &cpr->cp_ring_struct;
+ ring->handle = BNXT_SET_NQ_HDL(cpr);
+ rc = hwrm_ring_alloc_send_msg(bp, ring, type, map_idx);
+ if (rc)
+ return rc;
+ bnxt_set_db(bp, &cpr->cp_db, type, map_idx, ring->fw_ring_id);
+ bnxt_db_cq(bp, &cpr->cp_db, cpr->cp_raw_cons);
+ return 0;
+}
+
static int bnxt_hwrm_ring_alloc(struct bnxt *bp)
{
bool agg_rings = !!(bp->flags & BNXT_FLAG_AGG_RINGS);
@@ -7220,19 +7239,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp)
u32 map_idx;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
- struct bnxt_cp_ring_info *cpr2 = txr->tx_cpr;
- struct bnxt_napi *bnapi = txr->bnapi;
- u32 type2 = HWRM_RING_ALLOC_CMPL;
-
- ring = &cpr2->cp_ring_struct;
- ring->handle = BNXT_SET_NQ_HDL(cpr2);
- map_idx = bnapi->index;
- rc = hwrm_ring_alloc_send_msg(bp, ring, type2, map_idx);
+ rc = bnxt_hwrm_cp_ring_alloc_p5(bp, txr->tx_cpr);
if (rc)
goto err_out;
- bnxt_set_db(bp, &cpr2->cp_db, type2, map_idx,
- ring->fw_ring_id);
- bnxt_db_cq(bp, &cpr2->cp_db, cpr2->cp_raw_cons);
}
ring = &txr->tx_ring_struct;
map_idx = i;
@@ -7252,20 +7261,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp)
if (!agg_rings)
bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
- struct bnxt_cp_ring_info *cpr2 = rxr->rx_cpr;
- struct bnxt_napi *bnapi = rxr->bnapi;
- u32 type2 = HWRM_RING_ALLOC_CMPL;
- struct bnxt_ring_struct *ring;
- u32 map_idx = bnapi->index;
-
- ring = &cpr2->cp_ring_struct;
- ring->handle = BNXT_SET_NQ_HDL(cpr2);
- rc = hwrm_ring_alloc_send_msg(bp, ring, type2, map_idx);
+ rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
if (rc)
goto err_out;
- bnxt_set_db(bp, &cpr2->cp_db, type2, map_idx,
- ring->fw_ring_id);
- bnxt_db_cq(bp, &cpr2->cp_db, cpr2->cp_raw_cons);
}
}
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 03/10] bnxt_en: Refactor TX ring allocation logic
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 01/10] bnxt_en: Set NAPR 1.2 support when registering with firmware Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 02/10] bnxt_en Refactor completion ring allocation logic for P5_PLUS chips Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 04/10] bnxt_en: Refactor completion ring free routine Michael Chan
` (6 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Ajit Khaparde
Add a new bnxt_hwrm_tx_ring_alloc() function to handle allocating
a transmit ring. This will be useful later in the series.
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
v2: Use const for a variable
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 72fff9c10413..042d394c8235 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -7196,6 +7196,20 @@ static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,
return 0;
}
+static int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr, u32 tx_idx)
+{
+ struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
+ const u32 type = HWRM_RING_ALLOC_TX;
+ int rc;
+
+ rc = hwrm_ring_alloc_send_msg(bp, ring, type, tx_idx);
+ if (rc)
+ return rc;
+ bnxt_set_db(bp, &txr->tx_db, type, tx_idx, ring->fw_ring_id);
+ return 0;
+}
+
static int bnxt_hwrm_ring_alloc(struct bnxt *bp)
{
bool agg_rings = !!(bp->flags & BNXT_FLAG_AGG_RINGS);
@@ -7232,23 +7246,17 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp)
}
}
- type = HWRM_RING_ALLOC_TX;
for (i = 0; i < bp->tx_nr_rings; i++) {
struct bnxt_tx_ring_info *txr = &bp->tx_ring[i];
- struct bnxt_ring_struct *ring;
- u32 map_idx;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
rc = bnxt_hwrm_cp_ring_alloc_p5(bp, txr->tx_cpr);
if (rc)
goto err_out;
}
- ring = &txr->tx_ring_struct;
- map_idx = i;
- rc = hwrm_ring_alloc_send_msg(bp, ring, type, map_idx);
+ rc = bnxt_hwrm_tx_ring_alloc(bp, txr, i);
if (rc)
goto err_out;
- bnxt_set_db(bp, &txr->tx_db, type, map_idx, ring->fw_ring_id);
}
for (i = 0; i < bp->rx_nr_rings; i++) {
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 04/10] bnxt_en: Refactor completion ring free routine
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (2 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 03/10] bnxt_en: Refactor TX ring allocation logic Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 05/10] bnxt_en: Refactor bnxt_free_tx_rings() to free per TX ring Michael Chan
` (5 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Somnath Kotur,
Kalesh AP
From: Somnath Kotur <somnath.kotur@broadcom.com>
Add a wrapper routine to free L2 completion rings. This will be
useful later in the series.
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 26 ++++++++++++++---------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 042d394c8235..c651fe42cd57 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -7383,6 +7383,20 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp,
bp->grp_info[grp_idx].agg_fw_ring_id = INVALID_HW_RING_ID;
}
+static void bnxt_hwrm_cp_ring_free(struct bnxt *bp,
+ struct bnxt_cp_ring_info *cpr)
+{
+ struct bnxt_ring_struct *ring;
+
+ ring = &cpr->cp_ring_struct;
+ if (ring->fw_ring_id == INVALID_HW_RING_ID)
+ return;
+
+ hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_L2_CMPL,
+ INVALID_HW_RING_ID);
+ ring->fw_ring_id = INVALID_HW_RING_ID;
+}
+
static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)
{
u32 type;
@@ -7428,17 +7442,9 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)
struct bnxt_ring_struct *ring;
int j;
- for (j = 0; j < cpr->cp_ring_count && cpr->cp_ring_arr; j++) {
- struct bnxt_cp_ring_info *cpr2 = &cpr->cp_ring_arr[j];
+ for (j = 0; j < cpr->cp_ring_count && cpr->cp_ring_arr; j++)
+ bnxt_hwrm_cp_ring_free(bp, &cpr->cp_ring_arr[j]);
- ring = &cpr2->cp_ring_struct;
- if (ring->fw_ring_id == INVALID_HW_RING_ID)
- continue;
- hwrm_ring_free_send_msg(bp, ring,
- RING_FREE_REQ_RING_TYPE_L2_CMPL,
- INVALID_HW_RING_ID);
- ring->fw_ring_id = INVALID_HW_RING_ID;
- }
ring = &cpr->cp_ring_struct;
if (ring->fw_ring_id != INVALID_HW_RING_ID) {
hwrm_ring_free_send_msg(bp, ring, type,
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 05/10] bnxt_en: Refactor bnxt_free_tx_rings() to free per TX ring
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (3 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 04/10] bnxt_en: Refactor completion ring free routine Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 06/10] bnxt_en: Refactor RX/RX AGG ring parameters setup for P5_PLUS Michael Chan
` (4 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Somnath Kotur
From: Somnath Kotur <somnath.kotur@broadcom.com>
Modify bnxt_free_tx_rings() to free the skbs per TX ring.
This will be useful later in the series.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 115 ++++++++++++----------
1 file changed, 61 insertions(+), 54 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index c651fe42cd57..50459ffc48c8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3313,74 +3313,81 @@ static int bnxt_poll_p5(struct napi_struct *napi, int budget)
return work_done;
}
-static void bnxt_free_tx_skbs(struct bnxt *bp)
+static void bnxt_free_one_tx_ring_skbs(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr, int idx)
{
int i, max_idx;
struct pci_dev *pdev = bp->pdev;
- if (!bp->tx_ring)
- return;
-
max_idx = bp->tx_nr_pages * TX_DESC_CNT;
- for (i = 0; i < bp->tx_nr_rings; i++) {
- struct bnxt_tx_ring_info *txr = &bp->tx_ring[i];
- int j;
- if (!txr->tx_buf_ring)
+ for (i = 0; i < max_idx;) {
+ struct bnxt_sw_tx_bd *tx_buf = &txr->tx_buf_ring[i];
+ struct sk_buff *skb;
+ int j, last;
+
+ if (idx < bp->tx_nr_rings_xdp &&
+ tx_buf->action == XDP_REDIRECT) {
+ dma_unmap_single(&pdev->dev,
+ dma_unmap_addr(tx_buf, mapping),
+ dma_unmap_len(tx_buf, len),
+ DMA_TO_DEVICE);
+ xdp_return_frame(tx_buf->xdpf);
+ tx_buf->action = 0;
+ tx_buf->xdpf = NULL;
+ i++;
continue;
+ }
- for (j = 0; j < max_idx;) {
- struct bnxt_sw_tx_bd *tx_buf = &txr->tx_buf_ring[j];
- struct sk_buff *skb;
- int k, last;
-
- if (i < bp->tx_nr_rings_xdp &&
- tx_buf->action == XDP_REDIRECT) {
- dma_unmap_single(&pdev->dev,
- dma_unmap_addr(tx_buf, mapping),
- dma_unmap_len(tx_buf, len),
- DMA_TO_DEVICE);
- xdp_return_frame(tx_buf->xdpf);
- tx_buf->action = 0;
- tx_buf->xdpf = NULL;
- j++;
- continue;
- }
+ skb = tx_buf->skb;
+ if (!skb) {
+ i++;
+ continue;
+ }
- skb = tx_buf->skb;
- if (!skb) {
- j++;
- continue;
- }
+ tx_buf->skb = NULL;
- tx_buf->skb = NULL;
+ if (tx_buf->is_push) {
+ dev_kfree_skb(skb);
+ i += 2;
+ continue;
+ }
- if (tx_buf->is_push) {
- dev_kfree_skb(skb);
- j += 2;
- continue;
- }
+ dma_unmap_single(&pdev->dev,
+ dma_unmap_addr(tx_buf, mapping),
+ skb_headlen(skb),
+ DMA_TO_DEVICE);
- dma_unmap_single(&pdev->dev,
- dma_unmap_addr(tx_buf, mapping),
- skb_headlen(skb),
- DMA_TO_DEVICE);
+ last = tx_buf->nr_frags;
+ i += 2;
+ for (j = 0; j < last; j++, i++) {
+ int ring_idx = i & bp->tx_ring_mask;
+ skb_frag_t *frag = &skb_shinfo(skb)->frags[j];
- last = tx_buf->nr_frags;
- j += 2;
- for (k = 0; k < last; k++, j++) {
- int ring_idx = j & bp->tx_ring_mask;
- skb_frag_t *frag = &skb_shinfo(skb)->frags[k];
-
- tx_buf = &txr->tx_buf_ring[ring_idx];
- dma_unmap_page(
- &pdev->dev,
- dma_unmap_addr(tx_buf, mapping),
- skb_frag_size(frag), DMA_TO_DEVICE);
- }
- dev_kfree_skb(skb);
+ tx_buf = &txr->tx_buf_ring[ring_idx];
+ dma_unmap_page(&pdev->dev,
+ dma_unmap_addr(tx_buf, mapping),
+ skb_frag_size(frag), DMA_TO_DEVICE);
}
- netdev_tx_reset_queue(netdev_get_tx_queue(bp->dev, i));
+ dev_kfree_skb(skb);
+ }
+ netdev_tx_reset_queue(netdev_get_tx_queue(bp->dev, idx));
+}
+
+static void bnxt_free_tx_skbs(struct bnxt *bp)
+{
+ int i;
+
+ if (!bp->tx_ring)
+ return;
+
+ for (i = 0; i < bp->tx_nr_rings; i++) {
+ struct bnxt_tx_ring_info *txr = &bp->tx_ring[i];
+
+ if (!txr->tx_buf_ring)
+ continue;
+
+ bnxt_free_one_tx_ring_skbs(bp, txr, i);
}
}
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 06/10] bnxt_en: Refactor RX/RX AGG ring parameters setup for P5_PLUS
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (4 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 05/10] bnxt_en: Refactor bnxt_free_tx_rings() to free per TX ring Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 07/10] bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings Michael Chan
` (3 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Hongguang Gao,
Ajit Khaparde
There is some common code for setting up RX and RX AGG ring allocation
parameters for P5_PLUS chips. Refactor the logic into a new function.
Reviewed-by: Hongguang Gao <hongguang.gao@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 58 +++++++++++------------
1 file changed, 28 insertions(+), 30 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 50459ffc48c8..919f6efd0571 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -6922,6 +6922,28 @@ static void bnxt_hwrm_ring_grp_free(struct bnxt *bp)
hwrm_req_drop(bp, req);
}
+static void bnxt_set_rx_ring_params_p5(struct bnxt *bp, u32 ring_type,
+ struct hwrm_ring_alloc_input *req,
+ struct bnxt_ring_struct *ring)
+{
+ struct bnxt_ring_grp_info *grp_info = &bp->grp_info[ring->grp_idx];
+ u32 enables = RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID;
+
+ if (ring_type == HWRM_RING_ALLOC_AGG) {
+ req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX_AGG;
+ req->rx_ring_id = cpu_to_le16(grp_info->rx_fw_ring_id);
+ req->rx_buf_size = cpu_to_le16(BNXT_RX_PAGE_SIZE);
+ enables |= RING_ALLOC_REQ_ENABLES_RX_RING_ID_VALID;
+ } else {
+ req->rx_buf_size = cpu_to_le16(bp->rx_buf_use_size);
+ if (NET_IP_ALIGN == 2)
+ req->flags =
+ cpu_to_le16(RING_ALLOC_REQ_FLAGS_RX_SOP_PAD);
+ }
+ req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx);
+ req->enables |= cpu_to_le32(enables);
+}
+
static int hwrm_ring_alloc_send_msg(struct bnxt *bp,
struct bnxt_ring_struct *ring,
u32 ring_type, u32 map_index)
@@ -6973,37 +6995,13 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp,
break;
}
case HWRM_RING_ALLOC_RX:
- req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX;
- req->length = cpu_to_le32(bp->rx_ring_mask + 1);
- if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
- u16 flags = 0;
-
- /* Association of rx ring with stats context */
- grp_info = &bp->grp_info[ring->grp_idx];
- req->rx_buf_size = cpu_to_le16(bp->rx_buf_use_size);
- req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx);
- req->enables |= cpu_to_le32(
- RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID);
- if (NET_IP_ALIGN == 2)
- flags = RING_ALLOC_REQ_FLAGS_RX_SOP_PAD;
- req->flags = cpu_to_le16(flags);
- }
- break;
case HWRM_RING_ALLOC_AGG:
- if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
- req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX_AGG;
- /* Association of agg ring with rx ring */
- grp_info = &bp->grp_info[ring->grp_idx];
- req->rx_ring_id = cpu_to_le16(grp_info->rx_fw_ring_id);
- req->rx_buf_size = cpu_to_le16(BNXT_RX_PAGE_SIZE);
- req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx);
- req->enables |= cpu_to_le32(
- RING_ALLOC_REQ_ENABLES_RX_RING_ID_VALID |
- RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID);
- } else {
- req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX;
- }
- req->length = cpu_to_le32(bp->rx_agg_ring_mask + 1);
+ req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX;
+ req->length = (ring_type == HWRM_RING_ALLOC_RX) ?
+ cpu_to_le32(bp->rx_ring_mask + 1) :
+ cpu_to_le32(bp->rx_agg_ring_mask + 1);
+ if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
+ bnxt_set_rx_ring_params_p5(bp, ring_type, req, ring);
break;
case HWRM_RING_ALLOC_CMPL:
req->ring_type = RING_ALLOC_REQ_RING_TYPE_L2_CMPL;
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 07/10] bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (5 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 06/10] bnxt_en: Refactor RX/RX AGG ring parameters setup for P5_PLUS Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 08/10] bnxt_en: Reallocate RX completion ring for TPH support Michael Chan
` (2 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Hongguang Gao,
Ajit Khaparde
Newer firmware can use the NQ ring ID associated with each RX/RX AGG
ring to enable PCIe Steering Tags on P5_PLUS chips. When allocating
RX/RX AGG rings, pass along NQ ring ID for the firmware to use. This
information helps optimize DMA writes by directing them to the cache
closer to the CPU consuming the data, potentially improving the
processing speed. This change is backward-compatible with older
firmware, which will simply disregard the information.
Reviewed-by: Hongguang Gao <hongguang.gao@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 919f6efd0571..da5acb8b0495 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -6927,7 +6927,8 @@ static void bnxt_set_rx_ring_params_p5(struct bnxt *bp, u32 ring_type,
struct bnxt_ring_struct *ring)
{
struct bnxt_ring_grp_info *grp_info = &bp->grp_info[ring->grp_idx];
- u32 enables = RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID;
+ u32 enables = RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID |
+ RING_ALLOC_REQ_ENABLES_NQ_RING_ID_VALID;
if (ring_type == HWRM_RING_ALLOC_AGG) {
req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX_AGG;
@@ -6941,6 +6942,7 @@ static void bnxt_set_rx_ring_params_p5(struct bnxt *bp, u32 ring_type,
cpu_to_le16(RING_ALLOC_REQ_FLAGS_RX_SOP_PAD);
}
req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx);
+ req->nq_ring_id = cpu_to_le16(grp_info->cp_fw_ring_id);
req->enables |= cpu_to_le32(enables);
}
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 08/10] bnxt_en: Reallocate RX completion ring for TPH support
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (6 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 07/10] bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver Michael Chan
9 siblings, 0 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Somnath Kotur,
David Wei
From: Somnath Kotur <somnath.kotur@broadcom.com>
In order to program the correct Steering Tag during an IRQ affinity
change, we need to free/re-allocate the RX completion ring during
queue_restart. Call FW to free the Rx completion ring and clear the
ring entries in queue_stop(). Re-allocate it in queue_start().
While modifying bnxt_queue_start(), remove the unnecessary zeroing of
rxr->rx_next_cons. It gets overwritten by the clone in
bnxt_queue_start().
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
Cc: David Wei <dw@davidwei.uk>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 26 +++++++++++++++++++++--
1 file changed, 24 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index da5acb8b0495..53279904cdb5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -7404,6 +7404,19 @@ static void bnxt_hwrm_cp_ring_free(struct bnxt *bp,
ring->fw_ring_id = INVALID_HW_RING_ID;
}
+static void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)
+{
+ struct bnxt_ring_struct *ring = &cpr->cp_ring_struct;
+ int i, size = ring->ring_mem.page_size;
+
+ cpr->cp_raw_cons = 0;
+ cpr->toggle = 0;
+
+ for (i = 0; i < bp->cp_nr_pages; i++)
+ if (cpr->cp_desc_ring[i])
+ memset(cpr->cp_desc_ring[i], 0, size);
+}
+
static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)
{
u32 type;
@@ -15623,10 +15636,15 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
if (rc)
return rc;
- rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
+
+ rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
if (rc)
goto err_free_hwrm_rx_ring;
+ rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
+ if (rc)
+ goto err_free_hwrm_cp_ring;
+
bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
if (bp->flags & BNXT_FLAG_AGG_RINGS)
bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
@@ -15650,6 +15668,8 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
return 0;
+err_free_hwrm_cp_ring:
+ bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
err_free_hwrm_rx_ring:
bnxt_hwrm_rx_ring_free(bp, rxr, false);
return rc;
@@ -15674,11 +15694,13 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx)
cancel_work_sync(&rxr->bnapi->cp_ring.dim.work);
bnxt_hwrm_rx_ring_free(bp, rxr, false);
bnxt_hwrm_rx_agg_ring_free(bp, rxr, false);
- rxr->rx_next_cons = 0;
page_pool_disable_direct_recycling(rxr->page_pool);
if (bnxt_separate_head_pool())
page_pool_disable_direct_recycling(rxr->head_pool);
+ bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
+ bnxt_clear_one_cp_ring(bp, rxr->rx_cpr);
+
memcpy(qmem, rxr, sizeof(*rxr));
bnxt_init_rx_ring_struct(bp, qmem);
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (7 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 08/10] bnxt_en: Reallocate RX completion ring for TPH support Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-17 6:56 ` Michal Swiatkowski
` (3 more replies)
2025-01-16 19:23 ` [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver Michael Chan
9 siblings, 4 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Somnath Kotur,
Ajit Khaparde, David Wei
From: Somnath Kotur <somnath.kotur@broadcom.com>
In order to use queue_stop/queue_start to support the new Steering
Tags, we need to free the TX ring and TX completion ring if it is a
combined channel with TX/RX sharing the same NAPI. Otherwise
TX completions will not have the updated Steering Tag. With that
we can now add napi_disable() and napi_enable() during queue_stop()/
queue_start(). This will guarantee that NAPI will stop processing
the completion entries in case there are additional pending entries
in the completion rings after queue_stop().
There could be some NQEs sitting unprocessed while NAPI is disabled
thereby leaving the NQ unarmed. Explicitly re-arm the NQ after
napi_enable() in queue start so that NAPI will resume properly.
Error handling in bnxt_queue_start() requires a reset. If a TX
ring cannot be allocated or initialized properly, it will cause
TX timeout. The reset will also free any partially allocated
rings.
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
Cc: David Wei <dw@davidwei.uk>
v2:
Add reset for error handling in queue_start().
Fix compile error.
Discussion about adding napi_disable()/napi_enable():
https://lore.kernel.org/netdev/5336d624-8d8b-40a6-b732-b020e4a119a2@davidwei.uk/#t
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 124 ++++++++++++++++++++--
1 file changed, 115 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 53279904cdb5..0a10a4cffcc8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -7346,6 +7346,22 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,
return 0;
}
+static void bnxt_hwrm_tx_ring_free(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ bool close_path)
+{
+ struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
+ u32 cmpl_ring_id;
+
+ if (ring->fw_ring_id == INVALID_HW_RING_ID)
+ return;
+
+ cmpl_ring_id = close_path ? bnxt_cp_ring_for_tx(bp, txr) :
+ INVALID_HW_RING_ID;
+ hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_TX,
+ cmpl_ring_id);
+ ring->fw_ring_id = INVALID_HW_RING_ID;
+}
+
static void bnxt_hwrm_rx_ring_free(struct bnxt *bp,
struct bnxt_rx_ring_info *rxr,
bool close_path)
@@ -11252,6 +11268,68 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
return 0;
}
+static void bnxt_tx_queue_stop(struct bnxt *bp, int idx)
+{
+ struct bnxt_tx_ring_info *txr;
+ struct netdev_queue *txq;
+ struct bnxt_napi *bnapi;
+ int i;
+
+ bnapi = bp->bnapi[idx];
+ bnxt_for_each_napi_tx(i, bnapi, txr) {
+ WRITE_ONCE(txr->dev_state, BNXT_DEV_STATE_CLOSING);
+ synchronize_net();
+
+ if (!(bnapi->flags & BNXT_NAPI_FLAG_XDP)) {
+ txq = netdev_get_tx_queue(bp->dev, txr->txq_index);
+ if (txq) {
+ __netif_tx_lock_bh(txq);
+ netif_tx_stop_queue(txq);
+ __netif_tx_unlock_bh(txq);
+ }
+ }
+ bnxt_hwrm_tx_ring_free(bp, txr, true);
+ bnxt_hwrm_cp_ring_free(bp, txr->tx_cpr);
+ bnxt_free_one_tx_ring_skbs(bp, txr, txr->txq_index);
+ bnxt_clear_one_cp_ring(bp, txr->tx_cpr);
+ }
+}
+
+static int bnxt_tx_queue_start(struct bnxt *bp, int idx)
+{
+ struct bnxt_tx_ring_info *txr;
+ struct netdev_queue *txq;
+ struct bnxt_napi *bnapi;
+ int rc, i;
+
+ bnapi = bp->bnapi[idx];
+ bnxt_for_each_napi_tx(i, bnapi, txr) {
+ rc = bnxt_hwrm_cp_ring_alloc_p5(bp, txr->tx_cpr);
+ if (rc)
+ return rc;
+
+ rc = bnxt_hwrm_tx_ring_alloc(bp, txr, false);
+ if (rc)
+ return rc;
+
+ txr->tx_prod = 0;
+ txr->tx_cons = 0;
+ txr->tx_hw_cons = 0;
+
+ WRITE_ONCE(txr->dev_state, 0);
+ synchronize_net();
+
+ if (bnapi->flags & BNXT_NAPI_FLAG_XDP)
+ continue;
+
+ txq = netdev_get_tx_queue(bp->dev, txr->txq_index);
+ if (txq)
+ netif_tx_start_queue(txq);
+ }
+
+ return 0;
+}
+
static void bnxt_free_irq(struct bnxt *bp)
{
struct bnxt_irq *irq;
@@ -15616,6 +15694,7 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
struct bnxt_rx_ring_info *rxr, *clone;
struct bnxt_cp_ring_info *cpr;
struct bnxt_vnic_info *vnic;
+ struct bnxt_napi *bnapi;
int i, rc;
rxr = &bp->rx_ring[idx];
@@ -15633,25 +15712,40 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
bnxt_copy_rx_ring(bp, rxr, clone);
+ bnapi = rxr->bnapi;
rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
if (rc)
- return rc;
+ goto err_reset_rx;
rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
if (rc)
- goto err_free_hwrm_rx_ring;
+ goto err_reset_rx;
rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
if (rc)
- goto err_free_hwrm_cp_ring;
+ goto err_reset_rx;
bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
if (bp->flags & BNXT_FLAG_AGG_RINGS)
bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
- cpr = &rxr->bnapi->cp_ring;
+ cpr = &bnapi->cp_ring;
cpr->sw_stats->rx.rx_resets++;
+ if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
+ cpr->sw_stats->tx.tx_resets++;
+ rc = bnxt_tx_queue_start(bp, idx);
+ if (rc) {
+ netdev_warn(bp->dev,
+ "tx queue restart failed: rc=%d\n", rc);
+ bnapi->tx_fault = 1;
+ goto err_reset;
+ }
+ }
+
+ napi_enable(&bnapi->napi);
+ bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
+
for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
vnic = &bp->vnic_info[i];
@@ -15668,10 +15762,12 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
return 0;
-err_free_hwrm_cp_ring:
- bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
-err_free_hwrm_rx_ring:
- bnxt_hwrm_rx_ring_free(bp, rxr, false);
+err_reset_rx:
+ rxr->bnapi->in_reset = true;
+err_reset:
+ napi_enable(&bnapi->napi);
+ bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
+ bnxt_reset_task(bp, true);
return rc;
}
@@ -15679,7 +15775,9 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx)
{
struct bnxt *bp = netdev_priv(dev);
struct bnxt_rx_ring_info *rxr;
+ struct bnxt_cp_ring_info *cpr;
struct bnxt_vnic_info *vnic;
+ struct bnxt_napi *bnapi;
int i;
for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
@@ -15691,15 +15789,23 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx)
/* Make sure NAPI sees that the VNIC is disabled */
synchronize_net();
rxr = &bp->rx_ring[idx];
- cancel_work_sync(&rxr->bnapi->cp_ring.dim.work);
+ bnapi = rxr->bnapi;
+ cpr = &bnapi->cp_ring;
+ cancel_work_sync(&cpr->dim.work);
bnxt_hwrm_rx_ring_free(bp, rxr, false);
bnxt_hwrm_rx_agg_ring_free(bp, rxr, false);
page_pool_disable_direct_recycling(rxr->page_pool);
if (bnxt_separate_head_pool())
page_pool_disable_direct_recycling(rxr->head_pool);
+ if (bp->flags & BNXT_FLAG_SHARED_RINGS)
+ bnxt_tx_queue_stop(bp, idx);
+
+ napi_disable(&bnapi->napi);
+
bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
bnxt_clear_one_cp_ring(bp, rxr->rx_cpr);
+ bnxt_db_nq(bp, &cpr->cp_db, cpr->cp_raw_cons);
memcpy(qmem, rxr, sizeof(*rxr));
bnxt_init_rx_ring_struct(bp, qmem);
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
` (8 preceding siblings ...)
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
@ 2025-01-16 19:23 ` Michael Chan
2025-01-17 7:04 ` Michal Swiatkowski
2025-01-17 15:41 ` Simon Horman
9 siblings, 2 replies; 18+ messages in thread
From: Michael Chan @ 2025-01-16 19:23 UTC (permalink / raw)
To: davem
Cc: netdev, edumazet, kuba, pabeni, andrew+netdev, pavan.chebbi,
andrew.gospodarek, michal.swiatkowski, helgaas, Manoj Panicker,
Somnath Kotur, Wei Huang, Ajit Khaparde
From: Manoj Panicker <manoj.panicker2@amd.com>
Add TPH support to the Broadcom BNXT device driver. This allows the
driver to utilize TPH functions for retrieving and configuring Steering
Tags when changing interrupt affinity. With compatible NIC firmware,
network traffic will be tagged correctly with Steering Tags, resulting
in significant memory bandwidth savings and other advantages as
demonstrated by real network benchmarks on TPH-capable platforms.
Co-developed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Co-developed-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Wei Huang <wei.huang2@amd.com>
Signed-off-by: Manoj Panicker <manoj.panicker2@amd.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
Cc: Bjorn Helgaas <helgaas@kernel.org>
Previous driver series fixing rtnl_lock and empty release function:
https://lore.kernel.org/netdev/20241115200412.1340286-1-wei.huang2@amd.com/
v5 of the PCI series using netdev_rx_queue_restart():
https://lore.kernel.org/netdev/20240916205103.3882081-5-wei.huang2@amd.com/
v1 of the PCI series using open/close:
https://lore.kernel.org/netdev/20240509162741.1937586-9-wei.huang2@amd.com/
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 105 ++++++++++++++++++++++
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 7 ++
2 files changed, 112 insertions(+)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 0a10a4cffcc8..8c24642b8812 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -55,6 +55,8 @@
#include <net/page_pool/helpers.h>
#include <linux/align.h>
#include <net/netdev_queues.h>
+#include <net/netdev_rx_queue.h>
+#include <linux/pci-tph.h>
#include "bnxt_hsi.h"
#include "bnxt.h"
@@ -11330,6 +11332,83 @@ static int bnxt_tx_queue_start(struct bnxt *bp, int idx)
return 0;
}
+static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
+ const cpumask_t *mask)
+{
+ struct bnxt_irq *irq;
+ u16 tag;
+ int err;
+
+ irq = container_of(notify, struct bnxt_irq, affinity_notify);
+
+ if (!irq->bp->tph_mode)
+ return;
+
+ cpumask_copy(irq->cpu_mask, mask);
+
+ if (irq->ring_nr >= irq->bp->rx_nr_rings)
+ return;
+
+ if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
+ cpumask_first(irq->cpu_mask), &tag))
+ return;
+
+ if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag))
+ return;
+
+ rtnl_lock();
+ if (netif_running(irq->bp->dev)) {
+ err = netdev_rx_queue_restart(irq->bp->dev, irq->ring_nr);
+ if (err)
+ netdev_err(irq->bp->dev,
+ "RX queue restart failed: err=%d\n", err);
+ }
+ rtnl_unlock();
+}
+
+static void bnxt_irq_affinity_release(struct kref *ref)
+{
+ struct irq_affinity_notify *notify =
+ container_of(ref, struct irq_affinity_notify, kref);
+ struct bnxt_irq *irq;
+
+ irq = container_of(notify, struct bnxt_irq, affinity_notify);
+
+ if (!irq->bp->tph_mode)
+ return;
+
+ if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, 0)) {
+ netdev_err(irq->bp->dev,
+ "Setting ST=0 for MSIX entry %d failed\n",
+ irq->msix_nr);
+ return;
+ }
+}
+
+static void bnxt_release_irq_notifier(struct bnxt_irq *irq)
+{
+ irq_set_affinity_notifier(irq->vector, NULL);
+}
+
+static void bnxt_register_irq_notifier(struct bnxt *bp, struct bnxt_irq *irq)
+{
+ struct irq_affinity_notify *notify;
+
+ irq->bp = bp;
+
+ /* Nothing to do if TPH is not enabled */
+ if (!bp->tph_mode)
+ return;
+
+ /* Register IRQ affinity notifier */
+ notify = &irq->affinity_notify;
+ notify->irq = irq->vector;
+ notify->notify = bnxt_irq_affinity_notify;
+ notify->release = bnxt_irq_affinity_release;
+
+ irq_set_affinity_notifier(irq->vector, notify);
+}
+
static void bnxt_free_irq(struct bnxt *bp)
{
struct bnxt_irq *irq;
@@ -11352,11 +11431,18 @@ static void bnxt_free_irq(struct bnxt *bp)
free_cpumask_var(irq->cpu_mask);
irq->have_cpumask = 0;
}
+
+ bnxt_release_irq_notifier(irq);
+
free_irq(irq->vector, bp->bnapi[i]);
}
irq->requested = 0;
}
+
+ /* Disable TPH support */
+ pcie_disable_tph(bp->pdev);
+ bp->tph_mode = 0;
}
static int bnxt_request_irq(struct bnxt *bp)
@@ -11376,6 +11462,12 @@ static int bnxt_request_irq(struct bnxt *bp)
#ifdef CONFIG_RFS_ACCEL
rmap = bp->dev->rx_cpu_rmap;
#endif
+
+ /* Enable TPH support as part of IRQ request */
+ rc = pcie_enable_tph(bp->pdev, PCI_TPH_ST_IV_MODE);
+ if (!rc)
+ bp->tph_mode = PCI_TPH_ST_IV_MODE;
+
for (i = 0, j = 0; i < bp->cp_nr_rings; i++) {
int map_idx = bnxt_cp_num_to_irq_num(bp, i);
struct bnxt_irq *irq = &bp->irq_tbl[map_idx];
@@ -11399,8 +11491,11 @@ static int bnxt_request_irq(struct bnxt *bp)
if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) {
int numa_node = dev_to_node(&bp->pdev->dev);
+ u16 tag;
irq->have_cpumask = 1;
+ irq->msix_nr = map_idx;
+ irq->ring_nr = i;
cpumask_set_cpu(cpumask_local_spread(i, numa_node),
irq->cpu_mask);
rc = irq_update_affinity_hint(irq->vector, irq->cpu_mask);
@@ -11410,6 +11505,16 @@ static int bnxt_request_irq(struct bnxt *bp)
irq->vector);
break;
}
+
+ bnxt_register_irq_notifier(bp, irq);
+
+ /* Init ST table entry */
+ if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
+ cpumask_first(irq->cpu_mask),
+ &tag))
+ continue;
+
+ pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag);
}
}
return rc;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 826ae030fc09..02dc2ed9c75d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -1234,6 +1234,11 @@ struct bnxt_irq {
u8 have_cpumask:1;
char name[IFNAMSIZ + BNXT_IRQ_NAME_EXTRA];
cpumask_var_t cpu_mask;
+
+ struct bnxt *bp;
+ int msix_nr;
+ int ring_nr;
+ struct irq_affinity_notify affinity_notify;
};
#define HWRM_RING_ALLOC_TX 0x1
@@ -2229,6 +2234,8 @@ struct bnxt {
struct net_device *dev;
struct pci_dev *pdev;
+ u8 tph_mode;
+
atomic_t intr_sem;
u32 flags;
--
2.30.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
@ 2025-01-17 6:56 ` Michal Swiatkowski
2025-01-17 15:29 ` Simon Horman
` (2 subsequent siblings)
3 siblings, 0 replies; 18+ messages in thread
From: Michal Swiatkowski @ 2025-01-17 6:56 UTC (permalink / raw)
To: Michael Chan
Cc: davem, netdev, edumazet, kuba, pabeni, andrew+netdev,
pavan.chebbi, andrew.gospodarek, helgaas, Somnath Kotur,
Ajit Khaparde, David Wei
On Thu, Jan 16, 2025 at 11:23:42AM -0800, Michael Chan wrote:
> From: Somnath Kotur <somnath.kotur@broadcom.com>
>
> In order to use queue_stop/queue_start to support the new Steering
> Tags, we need to free the TX ring and TX completion ring if it is a
> combined channel with TX/RX sharing the same NAPI. Otherwise
> TX completions will not have the updated Steering Tag. With that
> we can now add napi_disable() and napi_enable() during queue_stop()/
> queue_start(). This will guarantee that NAPI will stop processing
> the completion entries in case there are additional pending entries
> in the completion rings after queue_stop().
>
> There could be some NQEs sitting unprocessed while NAPI is disabled
> thereby leaving the NQ unarmed. Explicitly re-arm the NQ after
> napi_enable() in queue start so that NAPI will resume properly.
>
> Error handling in bnxt_queue_start() requires a reset. If a TX
> ring cannot be allocated or initialized properly, it will cause
> TX timeout. The reset will also free any partially allocated
> rings.
>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
> Cc: David Wei <dw@davidwei.uk>
>
> v2:
> Add reset for error handling in queue_start().
> Fix compile error.
>
> Discussion about adding napi_disable()/napi_enable():
>
> https://lore.kernel.org/netdev/5336d624-8d8b-40a6-b732-b020e4a119a2@davidwei.uk/#t
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 124 ++++++++++++++++++++--
> 1 file changed, 115 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> index 53279904cdb5..0a10a4cffcc8 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> @@ -7346,6 +7346,22 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,
> return 0;
> }
>
[...]
> static void bnxt_free_irq(struct bnxt *bp)
> {
> struct bnxt_irq *irq;
> @@ -15616,6 +15694,7 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
> struct bnxt_rx_ring_info *rxr, *clone;
> struct bnxt_cp_ring_info *cpr;
> struct bnxt_vnic_info *vnic;
> + struct bnxt_napi *bnapi;
> int i, rc;
>
> rxr = &bp->rx_ring[idx];
> @@ -15633,25 +15712,40 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>
> bnxt_copy_rx_ring(bp, rxr, clone);
>
> + bnapi = rxr->bnapi;
> rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
> if (rc)
> - return rc;
> + goto err_reset_rx;
>
> rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
> if (rc)
> - goto err_free_hwrm_rx_ring;
> + goto err_reset_rx;
>
> rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
> if (rc)
> - goto err_free_hwrm_cp_ring;
> + goto err_reset_rx;
>
> bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
> if (bp->flags & BNXT_FLAG_AGG_RINGS)
> bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
>
> - cpr = &rxr->bnapi->cp_ring;
> + cpr = &bnapi->cp_ring;
> cpr->sw_stats->rx.rx_resets++;
>
> + if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
> + cpr->sw_stats->tx.tx_resets++;
> + rc = bnxt_tx_queue_start(bp, idx);
> + if (rc) {
> + netdev_warn(bp->dev,
> + "tx queue restart failed: rc=%d\n", rc);
> + bnapi->tx_fault = 1;
> + goto err_reset;
> + }
> + }
> +
> + napi_enable(&bnapi->napi);
> + bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
> +
> for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
> vnic = &bp->vnic_info[i];
>
> @@ -15668,10 +15762,12 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>
> return 0;
>
> -err_free_hwrm_cp_ring:
> - bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
> -err_free_hwrm_rx_ring:
> - bnxt_hwrm_rx_ring_free(bp, rxr, false);
It looked good to have partial freeing here, but as reset can do the
same it is fine to drop it.
Thanks for fixing the error path.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> +err_reset_rx:
> + rxr->bnapi->in_reset = true;
> +err_reset:
> + napi_enable(&bnapi->napi);
> + bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
> + bnxt_reset_task(bp, true);
> return rc;
> }
>
> @@ -15679,7 +15775,9 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx)
> {
> struct bnxt *bp = netdev_priv(dev);
> struct bnxt_rx_ring_info *rxr;
> + struct bnxt_cp_ring_info *cpr;
> struct bnxt_vnic_info *vnic;
> + struct bnxt_napi *bnapi;
> int i;
>
> for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
> @@ -15691,15 +15789,23 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx)
> /* Make sure NAPI sees that the VNIC is disabled */
> synchronize_net();
> rxr = &bp->rx_ring[idx];
> - cancel_work_sync(&rxr->bnapi->cp_ring.dim.work);
> + bnapi = rxr->bnapi;
> + cpr = &bnapi->cp_ring;
> + cancel_work_sync(&cpr->dim.work);
> bnxt_hwrm_rx_ring_free(bp, rxr, false);
> bnxt_hwrm_rx_agg_ring_free(bp, rxr, false);
> page_pool_disable_direct_recycling(rxr->page_pool);
> if (bnxt_separate_head_pool())
> page_pool_disable_direct_recycling(rxr->head_pool);
>
> + if (bp->flags & BNXT_FLAG_SHARED_RINGS)
> + bnxt_tx_queue_stop(bp, idx);
> +
> + napi_disable(&bnapi->napi);
> +
> bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
> bnxt_clear_one_cp_ring(bp, rxr->rx_cpr);
> + bnxt_db_nq(bp, &cpr->cp_db, cpr->cp_raw_cons);
>
> memcpy(qmem, rxr, sizeof(*rxr));
> bnxt_init_rx_ring_struct(bp, qmem);
> --
> 2.30.1
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver
2025-01-16 19:23 ` [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver Michael Chan
@ 2025-01-17 7:04 ` Michal Swiatkowski
2025-01-21 22:15 ` Panicker, Manoj
2025-01-17 15:41 ` Simon Horman
1 sibling, 1 reply; 18+ messages in thread
From: Michal Swiatkowski @ 2025-01-17 7:04 UTC (permalink / raw)
To: Michael Chan
Cc: davem, netdev, edumazet, kuba, pabeni, andrew+netdev,
pavan.chebbi, andrew.gospodarek, helgaas, Manoj Panicker,
Somnath Kotur, Wei Huang, Ajit Khaparde
On Thu, Jan 16, 2025 at 11:23:43AM -0800, Michael Chan wrote:
> From: Manoj Panicker <manoj.panicker2@amd.com>
>
> Add TPH support to the Broadcom BNXT device driver. This allows the
> driver to utilize TPH functions for retrieving and configuring Steering
> Tags when changing interrupt affinity. With compatible NIC firmware,
> network traffic will be tagged correctly with Steering Tags, resulting
> in significant memory bandwidth savings and other advantages as
> demonstrated by real network benchmarks on TPH-capable platforms.
>
> Co-developed-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Co-developed-by: Wei Huang <wei.huang2@amd.com>
> Signed-off-by: Wei Huang <wei.huang2@amd.com>
> Signed-off-by: Manoj Panicker <manoj.panicker2@amd.com>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
> Cc: Bjorn Helgaas <helgaas@kernel.org>
>
> Previous driver series fixing rtnl_lock and empty release function:
>
> https://lore.kernel.org/netdev/20241115200412.1340286-1-wei.huang2@amd.com/
>
> v5 of the PCI series using netdev_rx_queue_restart():
>
> https://lore.kernel.org/netdev/20240916205103.3882081-5-wei.huang2@amd.com/
>
> v1 of the PCI series using open/close:
>
> https://lore.kernel.org/netdev/20240509162741.1937586-9-wei.huang2@amd.com/
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 105 ++++++++++++++++++++++
> drivers/net/ethernet/broadcom/bnxt/bnxt.h | 7 ++
> 2 files changed, 112 insertions(+)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> index 0a10a4cffcc8..8c24642b8812 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> @@ -55,6 +55,8 @@
> #include <net/page_pool/helpers.h>
> #include <linux/align.h>
> #include <net/netdev_queues.h>
> +#include <net/netdev_rx_queue.h>
> +#include <linux/pci-tph.h>
>
> #include "bnxt_hsi.h"
> #include "bnxt.h"
> @@ -11330,6 +11332,83 @@ static int bnxt_tx_queue_start(struct bnxt *bp, int idx)
> return 0;
> }
>
> +static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
> + const cpumask_t *mask)
> +{
> + struct bnxt_irq *irq;
> + u16 tag;
> + int err;
> +
> + irq = container_of(notify, struct bnxt_irq, affinity_notify);
> +
> + if (!irq->bp->tph_mode)
> + return;
> +
Can it not be set? The notifier is registered only if it is set, can
mode change while irq notifier is registered? Maybe I am missing sth,
but it looks like it can't.
> + cpumask_copy(irq->cpu_mask, mask);
> +
> + if (irq->ring_nr >= irq->bp->rx_nr_rings)
> + return;
> +
> + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
> + cpumask_first(irq->cpu_mask), &tag))
> + return;
> +
> + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag))
> + return;
> +
> + rtnl_lock();
> + if (netif_running(irq->bp->dev)) {
> + err = netdev_rx_queue_restart(irq->bp->dev, irq->ring_nr);
> + if (err)
> + netdev_err(irq->bp->dev,
> + "RX queue restart failed: err=%d\n", err);
> + }
> + rtnl_unlock();
> +}
> +
> +static void bnxt_irq_affinity_release(struct kref *ref)
> +{
> + struct irq_affinity_notify *notify =
> + container_of(ref, struct irq_affinity_notify, kref);
> + struct bnxt_irq *irq;
> +
> + irq = container_of(notify, struct bnxt_irq, affinity_notify);
> +
> + if (!irq->bp->tph_mode)
The same here.
> + return;
> +
> + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, 0)) {
> + netdev_err(irq->bp->dev,
> + "Setting ST=0 for MSIX entry %d failed\n",
> + irq->msix_nr);
> + return;
> + }
> +}
> +
> +static void bnxt_release_irq_notifier(struct bnxt_irq *irq)
> +{
> + irq_set_affinity_notifier(irq->vector, NULL);
> +}
> +
> +static void bnxt_register_irq_notifier(struct bnxt *bp, struct bnxt_irq *irq)
> +{
> + struct irq_affinity_notify *notify;
> +
> + irq->bp = bp;
> +
> + /* Nothing to do if TPH is not enabled */
> + if (!bp->tph_mode)
> + return;
> +
> + /* Register IRQ affinity notifier */
> + notify = &irq->affinity_notify;
> + notify->irq = irq->vector;
> + notify->notify = bnxt_irq_affinity_notify;
> + notify->release = bnxt_irq_affinity_release;
> +
> + irq_set_affinity_notifier(irq->vector, notify);
> +}
> +
> static void bnxt_free_irq(struct bnxt *bp)
> {
> struct bnxt_irq *irq;
> @@ -11352,11 +11431,18 @@ static void bnxt_free_irq(struct bnxt *bp)
> free_cpumask_var(irq->cpu_mask);
> irq->have_cpumask = 0;
> }
> +
> + bnxt_release_irq_notifier(irq);
> +
> free_irq(irq->vector, bp->bnapi[i]);
> }
>
> irq->requested = 0;
> }
> +
> + /* Disable TPH support */
> + pcie_disable_tph(bp->pdev);
> + bp->tph_mode = 0;
> }
>
> static int bnxt_request_irq(struct bnxt *bp)
> @@ -11376,6 +11462,12 @@ static int bnxt_request_irq(struct bnxt *bp)
> #ifdef CONFIG_RFS_ACCEL
> rmap = bp->dev->rx_cpu_rmap;
> #endif
> +
> + /* Enable TPH support as part of IRQ request */
> + rc = pcie_enable_tph(bp->pdev, PCI_TPH_ST_IV_MODE);
> + if (!rc)
> + bp->tph_mode = PCI_TPH_ST_IV_MODE;
> +
> for (i = 0, j = 0; i < bp->cp_nr_rings; i++) {
> int map_idx = bnxt_cp_num_to_irq_num(bp, i);
> struct bnxt_irq *irq = &bp->irq_tbl[map_idx];
> @@ -11399,8 +11491,11 @@ static int bnxt_request_irq(struct bnxt *bp)
>
> if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) {
> int numa_node = dev_to_node(&bp->pdev->dev);
> + u16 tag;
>
> irq->have_cpumask = 1;
> + irq->msix_nr = map_idx;
> + irq->ring_nr = i;
> cpumask_set_cpu(cpumask_local_spread(i, numa_node),
> irq->cpu_mask);
> rc = irq_update_affinity_hint(irq->vector, irq->cpu_mask);
> @@ -11410,6 +11505,16 @@ static int bnxt_request_irq(struct bnxt *bp)
> irq->vector);
> break;
> }
> +
> + bnxt_register_irq_notifier(bp, irq);
> +
> + /* Init ST table entry */
> + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
> + cpumask_first(irq->cpu_mask),
> + &tag))
> + continue;
> +
> + pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag);
> }
> }
> return rc;
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> index 826ae030fc09..02dc2ed9c75d 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> @@ -1234,6 +1234,11 @@ struct bnxt_irq {
> u8 have_cpumask:1;
> char name[IFNAMSIZ + BNXT_IRQ_NAME_EXTRA];
> cpumask_var_t cpu_mask;
> +
> + struct bnxt *bp;
> + int msix_nr;
> + int ring_nr;
> + struct irq_affinity_notify affinity_notify;
> };
>
> #define HWRM_RING_ALLOC_TX 0x1
> @@ -2229,6 +2234,8 @@ struct bnxt {
> struct net_device *dev;
> struct pci_dev *pdev;
>
> + u8 tph_mode;
> +
> atomic_t intr_sem;
>
> u32 flags;
> --
> 2.30.1
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
2025-01-17 6:56 ` Michal Swiatkowski
@ 2025-01-17 15:29 ` Simon Horman
2025-01-17 18:07 ` kernel test robot
2025-01-30 5:39 ` Dan Carpenter
3 siblings, 0 replies; 18+ messages in thread
From: Simon Horman @ 2025-01-17 15:29 UTC (permalink / raw)
To: Michael Chan
Cc: davem, netdev, edumazet, kuba, pabeni, andrew+netdev,
pavan.chebbi, andrew.gospodarek, michal.swiatkowski, helgaas,
Somnath Kotur, Ajit Khaparde, David Wei
On Thu, Jan 16, 2025 at 11:23:42AM -0800, Michael Chan wrote:
> From: Somnath Kotur <somnath.kotur@broadcom.com>
>
> In order to use queue_stop/queue_start to support the new Steering
> Tags, we need to free the TX ring and TX completion ring if it is a
> combined channel with TX/RX sharing the same NAPI. Otherwise
> TX completions will not have the updated Steering Tag. With that
> we can now add napi_disable() and napi_enable() during queue_stop()/
> queue_start(). This will guarantee that NAPI will stop processing
> the completion entries in case there are additional pending entries
> in the completion rings after queue_stop().
>
> There could be some NQEs sitting unprocessed while NAPI is disabled
> thereby leaving the NQ unarmed. Explicitly re-arm the NQ after
> napi_enable() in queue start so that NAPI will resume properly.
>
> Error handling in bnxt_queue_start() requires a reset. If a TX
> ring cannot be allocated or initialized properly, it will cause
> TX timeout. The reset will also free any partially allocated
> rings.
>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
> Cc: David Wei <dw@davidwei.uk>
>
> v2:
> Add reset for error handling in queue_start().
> Fix compile error.
>
> Discussion about adding napi_disable()/napi_enable():
>
> https://lore.kernel.org/netdev/5336d624-8d8b-40a6-b732-b020e4a119a2@davidwei.uk/#t
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 124 ++++++++++++++++++++--
> 1 file changed, 115 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
...
> @@ -15616,6 +15694,7 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
> struct bnxt_rx_ring_info *rxr, *clone;
> struct bnxt_cp_ring_info *cpr;
> struct bnxt_vnic_info *vnic;
> + struct bnxt_napi *bnapi;
> int i, rc;
>
> rxr = &bp->rx_ring[idx];
> @@ -15633,25 +15712,40 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>
> bnxt_copy_rx_ring(bp, rxr, clone);
>
> + bnapi = rxr->bnapi;
> rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
> if (rc)
> - return rc;
> + goto err_reset_rx;
>
> rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
> if (rc)
> - goto err_free_hwrm_rx_ring;
> + goto err_reset_rx;
>
> rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
> if (rc)
> - goto err_free_hwrm_cp_ring;
> + goto err_reset_rx;
>
> bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
> if (bp->flags & BNXT_FLAG_AGG_RINGS)
> bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
>
> - cpr = &rxr->bnapi->cp_ring;
> + cpr = &bnapi->cp_ring;
> cpr->sw_stats->rx.rx_resets++;
Hi Somnath and Michael,
cpr is initialised here. However, it appears that the above
"goto err_reset_rx" directives will result in cpr being dereferenced.
Flagged by W=1 builds with clang-19, and Smatch.
>
> + if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
> + cpr->sw_stats->tx.tx_resets++;
> + rc = bnxt_tx_queue_start(bp, idx);
> + if (rc) {
> + netdev_warn(bp->dev,
> + "tx queue restart failed: rc=%d\n", rc);
> + bnapi->tx_fault = 1;
> + goto err_reset;
> + }
> + }
> +
> + napi_enable(&bnapi->napi);
> + bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
> +
> for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
> vnic = &bp->vnic_info[i];
>
> @@ -15668,10 +15762,12 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
>
> return 0;
>
> -err_free_hwrm_cp_ring:
> - bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr);
> -err_free_hwrm_rx_ring:
> - bnxt_hwrm_rx_ring_free(bp, rxr, false);
> +err_reset_rx:
> + rxr->bnapi->in_reset = true;
> +err_reset:
> + napi_enable(&bnapi->napi);
> + bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
> + bnxt_reset_task(bp, true);
> return rc;
> }
...
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver
2025-01-16 19:23 ` [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver Michael Chan
2025-01-17 7:04 ` Michal Swiatkowski
@ 2025-01-17 15:41 ` Simon Horman
1 sibling, 0 replies; 18+ messages in thread
From: Simon Horman @ 2025-01-17 15:41 UTC (permalink / raw)
To: Michael Chan
Cc: davem, netdev, edumazet, kuba, pabeni, andrew+netdev,
pavan.chebbi, andrew.gospodarek, michal.swiatkowski, helgaas,
Manoj Panicker, Somnath Kotur, Wei Huang, Ajit Khaparde
On Thu, Jan 16, 2025 at 11:23:43AM -0800, Michael Chan wrote:
> From: Manoj Panicker <manoj.panicker2@amd.com>
>
> Add TPH support to the Broadcom BNXT device driver. This allows the
> driver to utilize TPH functions for retrieving and configuring Steering
> Tags when changing interrupt affinity. With compatible NIC firmware,
> network traffic will be tagged correctly with Steering Tags, resulting
> in significant memory bandwidth savings and other advantages as
> demonstrated by real network benchmarks on TPH-capable platforms.
>
> Co-developed-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Co-developed-by: Wei Huang <wei.huang2@amd.com>
> Signed-off-by: Wei Huang <wei.huang2@amd.com>
> Signed-off-by: Manoj Panicker <manoj.panicker2@amd.com>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
> Cc: Bjorn Helgaas <helgaas@kernel.org>
>
> Previous driver series fixing rtnl_lock and empty release function:
>
> https://lore.kernel.org/netdev/20241115200412.1340286-1-wei.huang2@amd.com/
>
> v5 of the PCI series using netdev_rx_queue_restart():
>
> https://lore.kernel.org/netdev/20240916205103.3882081-5-wei.huang2@amd.com/
>
> v1 of the PCI series using open/close:
>
> https://lore.kernel.org/netdev/20240509162741.1937586-9-wei.huang2@amd.com/
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 105 ++++++++++++++++++++++
> drivers/net/ethernet/broadcom/bnxt/bnxt.h | 7 ++
> 2 files changed, 112 insertions(+)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> index 0a10a4cffcc8..8c24642b8812 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> @@ -55,6 +55,8 @@
> #include <net/page_pool/helpers.h>
> #include <linux/align.h>
> #include <net/netdev_queues.h>
> +#include <net/netdev_rx_queue.h>
> +#include <linux/pci-tph.h>
>
> #include "bnxt_hsi.h"
> #include "bnxt.h"
Hi Manoj, Michael, all,
Modpost complains that:
WARNING: modpost: module bnxt_en uses symbol netdev_rx_queue_restart from namespace NETDEV_INTERNAL, but does not import it.
And looking into this I see:
* netdev: define NETDEV_INTERNAL
https://git.kernel.org/netdev/net-next/c/0b7bdc7fab57
Which adds the following text to Documentation/networking/netdevices.rst
NETDEV_INTERNAL symbol namespace
================================
Symbols exported as NETDEV_INTERNAL can only be used in networking
core and drivers which exclusively flow via the main networking list and trees.
Note that the inverse is not true, most symbols outside of NETDEV_INTERNAL
are not expected to be used by random code outside netdev either.
Symbols may lack the designation because they predate the namespaces,
or simply due to an oversight.
Which I think is satisfied here. So I think this problem can be
addressed by adding the following about here (completely untested!).
MODULE_IMPORT_NS("NETDEV_INTERNAL");
> @@ -11330,6 +11332,83 @@ static int bnxt_tx_queue_start(struct bnxt *bp, int idx)
> return 0;
> }
>
> +static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
> + const cpumask_t *mask)
> +{
> + struct bnxt_irq *irq;
> + u16 tag;
> + int err;
> +
> + irq = container_of(notify, struct bnxt_irq, affinity_notify);
> +
> + if (!irq->bp->tph_mode)
> + return;
> +
> + cpumask_copy(irq->cpu_mask, mask);
> +
> + if (irq->ring_nr >= irq->bp->rx_nr_rings)
> + return;
> +
> + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
> + cpumask_first(irq->cpu_mask), &tag))
> + return;
> +
> + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag))
> + return;
> +
> + rtnl_lock();
> + if (netif_running(irq->bp->dev)) {
> + err = netdev_rx_queue_restart(irq->bp->dev, irq->ring_nr);
> + if (err)
> + netdev_err(irq->bp->dev,
> + "RX queue restart failed: err=%d\n", err);
> + }
> + rtnl_unlock();
> +}
...
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
2025-01-17 6:56 ` Michal Swiatkowski
2025-01-17 15:29 ` Simon Horman
@ 2025-01-17 18:07 ` kernel test robot
2025-01-30 5:39 ` Dan Carpenter
3 siblings, 0 replies; 18+ messages in thread
From: kernel test robot @ 2025-01-17 18:07 UTC (permalink / raw)
To: Michael Chan, davem
Cc: llvm, oe-kbuild-all, netdev, edumazet, kuba, pabeni,
andrew+netdev, pavan.chebbi, andrew.gospodarek,
michal.swiatkowski, helgaas, Somnath Kotur, Ajit Khaparde,
David Wei
Hi Michael,
kernel test robot noticed the following build warnings:
[auto build test WARNING on net-next/main]
url: https://github.com/intel-lab-lkp/linux/commits/Michael-Chan/bnxt_en-Set-NAPR-1-2-support-when-registering-with-firmware/20250117-032822
base: net-next/main
patch link: https://lore.kernel.org/r/20250116192343.34535-10-michael.chan%40broadcom.com
patch subject: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
config: i386-randconfig-012-20250117 (https://download.01.org/0day-ci/archive/20250118/202501180109.z7pduJQw-lkp@intel.com/config)
compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250118/202501180109.z7pduJQw-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202501180109.z7pduJQw-lkp@intel.com/
All warnings (new ones prefixed by >>):
>> drivers/net/ethernet/broadcom/bnxt/bnxt.c:15725:2: warning: variable 'cpr' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
15725 | if (rc)
| ^~~~~~~
include/linux/compiler.h:55:28: note: expanded from macro 'if'
55 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/compiler.h:57:30: note: expanded from macro '__trace_if_var'
57 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15769:22: note: uninitialized use occurs here
15769 | bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
| ^~~
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15725:2: note: remove the 'if' if its condition is always false
15725 | if (rc)
| ^~~~~~~
15726 | goto err_reset_rx;
| ~~~~~~~~~~~~~~~~~
include/linux/compiler.h:55:23: note: expanded from macro 'if'
55 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15721:2: warning: variable 'cpr' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
15721 | if (rc)
| ^~~~~~~
include/linux/compiler.h:55:28: note: expanded from macro 'if'
55 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/compiler.h:57:30: note: expanded from macro '__trace_if_var'
57 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15769:22: note: uninitialized use occurs here
15769 | bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
| ^~~
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15721:2: note: remove the 'if' if its condition is always false
15721 | if (rc)
| ^~~~~~~
15722 | goto err_reset_rx;
| ~~~~~~~~~~~~~~~~~
include/linux/compiler.h:55:23: note: expanded from macro 'if'
55 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15717:2: warning: variable 'cpr' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
15717 | if (rc)
| ^~~~~~~
include/linux/compiler.h:55:28: note: expanded from macro 'if'
55 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/compiler.h:57:30: note: expanded from macro '__trace_if_var'
57 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15769:22: note: uninitialized use occurs here
15769 | bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
| ^~~
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15717:2: note: remove the 'if' if its condition is always false
15717 | if (rc)
| ^~~~~~~
15718 | goto err_reset_rx;
| ~~~~~~~~~~~~~~~~~
include/linux/compiler.h:55:23: note: expanded from macro 'if'
55 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15695:31: note: initialize the variable 'cpr' to silence this warning
15695 | struct bnxt_cp_ring_info *cpr;
| ^
| = NULL
3 warnings generated.
vim +15725 drivers/net/ethernet/broadcom/bnxt/bnxt.c
2d694c27d32efc David Wei 2024-06-18 15690
2d694c27d32efc David Wei 2024-06-18 15691 static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
2d694c27d32efc David Wei 2024-06-18 15692 {
2d694c27d32efc David Wei 2024-06-18 15693 struct bnxt *bp = netdev_priv(dev);
2d694c27d32efc David Wei 2024-06-18 15694 struct bnxt_rx_ring_info *rxr, *clone;
2d694c27d32efc David Wei 2024-06-18 15695 struct bnxt_cp_ring_info *cpr;
b9d2956e869c78 David Wei 2024-08-07 15696 struct bnxt_vnic_info *vnic;
5d2310614bc883 Somnath Kotur 2025-01-16 15697 struct bnxt_napi *bnapi;
b9d2956e869c78 David Wei 2024-08-07 15698 int i, rc;
2d694c27d32efc David Wei 2024-06-18 15699
2d694c27d32efc David Wei 2024-06-18 15700 rxr = &bp->rx_ring[idx];
2d694c27d32efc David Wei 2024-06-18 15701 clone = qmem;
2d694c27d32efc David Wei 2024-06-18 15702
2d694c27d32efc David Wei 2024-06-18 15703 rxr->rx_prod = clone->rx_prod;
2d694c27d32efc David Wei 2024-06-18 15704 rxr->rx_agg_prod = clone->rx_agg_prod;
2d694c27d32efc David Wei 2024-06-18 15705 rxr->rx_sw_agg_prod = clone->rx_sw_agg_prod;
2d694c27d32efc David Wei 2024-06-18 15706 rxr->rx_next_cons = clone->rx_next_cons;
bd649c5cc95816 David Wei 2024-12-03 15707 rxr->rx_tpa = clone->rx_tpa;
bd649c5cc95816 David Wei 2024-12-03 15708 rxr->rx_tpa_idx_map = clone->rx_tpa_idx_map;
2d694c27d32efc David Wei 2024-06-18 15709 rxr->page_pool = clone->page_pool;
bd649c5cc95816 David Wei 2024-12-03 15710 rxr->head_pool = clone->head_pool;
b537633ce57b29 Taehee Yoo 2024-07-21 15711 rxr->xdp_rxq = clone->xdp_rxq;
2d694c27d32efc David Wei 2024-06-18 15712
2d694c27d32efc David Wei 2024-06-18 15713 bnxt_copy_rx_ring(bp, rxr, clone);
2d694c27d32efc David Wei 2024-06-18 15714
5d2310614bc883 Somnath Kotur 2025-01-16 15715 bnapi = rxr->bnapi;
2d694c27d32efc David Wei 2024-06-18 15716 rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
2d694c27d32efc David Wei 2024-06-18 15717 if (rc)
5d2310614bc883 Somnath Kotur 2025-01-16 15718 goto err_reset_rx;
377e78a9e08ce5 Somnath Kotur 2025-01-16 15719
377e78a9e08ce5 Somnath Kotur 2025-01-16 15720 rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
2d694c27d32efc David Wei 2024-06-18 15721 if (rc)
5d2310614bc883 Somnath Kotur 2025-01-16 15722 goto err_reset_rx;
2d694c27d32efc David Wei 2024-06-18 15723
377e78a9e08ce5 Somnath Kotur 2025-01-16 15724 rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
377e78a9e08ce5 Somnath Kotur 2025-01-16 @15725 if (rc)
5d2310614bc883 Somnath Kotur 2025-01-16 15726 goto err_reset_rx;
377e78a9e08ce5 Somnath Kotur 2025-01-16 15727
2d694c27d32efc David Wei 2024-06-18 15728 bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
2d694c27d32efc David Wei 2024-06-18 15729 if (bp->flags & BNXT_FLAG_AGG_RINGS)
2d694c27d32efc David Wei 2024-06-18 15730 bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
2d694c27d32efc David Wei 2024-06-18 15731
5d2310614bc883 Somnath Kotur 2025-01-16 15732 cpr = &bnapi->cp_ring;
2d694c27d32efc David Wei 2024-06-18 15733 cpr->sw_stats->rx.rx_resets++;
2d694c27d32efc David Wei 2024-06-18 15734
5d2310614bc883 Somnath Kotur 2025-01-16 15735 if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
5d2310614bc883 Somnath Kotur 2025-01-16 15736 cpr->sw_stats->tx.tx_resets++;
5d2310614bc883 Somnath Kotur 2025-01-16 15737 rc = bnxt_tx_queue_start(bp, idx);
5d2310614bc883 Somnath Kotur 2025-01-16 15738 if (rc) {
5d2310614bc883 Somnath Kotur 2025-01-16 15739 netdev_warn(bp->dev,
5d2310614bc883 Somnath Kotur 2025-01-16 15740 "tx queue restart failed: rc=%d\n", rc);
5d2310614bc883 Somnath Kotur 2025-01-16 15741 bnapi->tx_fault = 1;
5d2310614bc883 Somnath Kotur 2025-01-16 15742 goto err_reset;
5d2310614bc883 Somnath Kotur 2025-01-16 15743 }
5d2310614bc883 Somnath Kotur 2025-01-16 15744 }
5d2310614bc883 Somnath Kotur 2025-01-16 15745
5d2310614bc883 Somnath Kotur 2025-01-16 15746 napi_enable(&bnapi->napi);
5d2310614bc883 Somnath Kotur 2025-01-16 15747 bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
5d2310614bc883 Somnath Kotur 2025-01-16 15748
b9d2956e869c78 David Wei 2024-08-07 15749 for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
b9d2956e869c78 David Wei 2024-08-07 15750 vnic = &bp->vnic_info[i];
5ac066b7b062ee Somnath Kotur 2024-11-22 15751
5ac066b7b062ee Somnath Kotur 2024-11-22 15752 rc = bnxt_hwrm_vnic_set_rss_p5(bp, vnic, true);
5ac066b7b062ee Somnath Kotur 2024-11-22 15753 if (rc) {
5ac066b7b062ee Somnath Kotur 2024-11-22 15754 netdev_err(bp->dev, "hwrm vnic %d set rss failure rc: %d\n",
5ac066b7b062ee Somnath Kotur 2024-11-22 15755 vnic->vnic_id, rc);
5ac066b7b062ee Somnath Kotur 2024-11-22 15756 return rc;
5ac066b7b062ee Somnath Kotur 2024-11-22 15757 }
b9d2956e869c78 David Wei 2024-08-07 15758 vnic->mru = bp->dev->mtu + ETH_HLEN + VLAN_HLEN;
b9d2956e869c78 David Wei 2024-08-07 15759 bnxt_hwrm_vnic_update(bp, vnic,
b9d2956e869c78 David Wei 2024-08-07 15760 VNIC_UPDATE_REQ_ENABLES_MRU_VALID);
b9d2956e869c78 David Wei 2024-08-07 15761 }
b9d2956e869c78 David Wei 2024-08-07 15762
2d694c27d32efc David Wei 2024-06-18 15763 return 0;
2d694c27d32efc David Wei 2024-06-18 15764
5d2310614bc883 Somnath Kotur 2025-01-16 15765 err_reset_rx:
5d2310614bc883 Somnath Kotur 2025-01-16 15766 rxr->bnapi->in_reset = true;
5d2310614bc883 Somnath Kotur 2025-01-16 15767 err_reset:
5d2310614bc883 Somnath Kotur 2025-01-16 15768 napi_enable(&bnapi->napi);
5d2310614bc883 Somnath Kotur 2025-01-16 15769 bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
5d2310614bc883 Somnath Kotur 2025-01-16 15770 bnxt_reset_task(bp, true);
2d694c27d32efc David Wei 2024-06-18 15771 return rc;
2d694c27d32efc David Wei 2024-06-18 15772 }
2d694c27d32efc David Wei 2024-06-18 15773
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 18+ messages in thread
* RE: [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver
2025-01-17 7:04 ` Michal Swiatkowski
@ 2025-01-21 22:15 ` Panicker, Manoj
0 siblings, 0 replies; 18+ messages in thread
From: Panicker, Manoj @ 2025-01-21 22:15 UTC (permalink / raw)
To: Michal Swiatkowski, Michael Chan
Cc: davem@davemloft.net, netdev@vger.kernel.org, edumazet@google.com,
kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch,
pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com,
helgaas@kernel.org, Somnath Kotur, Huang2, Wei, Ajit Khaparde
Hello Michal,
The driver changes currently attempt to enable TPH only for the Interrupt Vector Mode operation when interrupts are being enabled. The check you pointed out should fall through to the rest of the notifier code since the notifier is registered only when the tph_mode is set currently. I believe the check was added in anticipation of further changes in TPH support like NoST mode, for example, but that is not part of this submission.
Thanks
Manoj
-----Original Message-----
From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Sent: Thursday, January 16, 2025 11:04 PM
To: Michael Chan <michael.chan@broadcom.com>
Cc: davem@davemloft.net; netdev@vger.kernel.org; edumazet@google.com; kuba@kernel.org; pabeni@redhat.com; andrew+netdev@lunn.ch; pavan.chebbi@broadcom.com; andrew.gospodarek@broadcom.com; helgaas@kernel.org; Panicker, Manoj <Manoj.Panicker2@amd.com>; Somnath Kotur <somnath.kotur@broadcom.com>; Huang2, Wei <Wei.Huang2@amd.com>; Ajit Khaparde <ajit.khaparde@broadcom.com>
Subject: Re: [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
On Thu, Jan 16, 2025 at 11:23:43AM -0800, Michael Chan wrote:
> From: Manoj Panicker <manoj.panicker2@amd.com>
>
> Add TPH support to the Broadcom BNXT device driver. This allows the
> driver to utilize TPH functions for retrieving and configuring
> Steering Tags when changing interrupt affinity. With compatible NIC
> firmware, network traffic will be tagged correctly with Steering Tags,
> resulting in significant memory bandwidth savings and other advantages
> as demonstrated by real network benchmarks on TPH-capable platforms.
>
> Co-developed-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
> Co-developed-by: Wei Huang <wei.huang2@amd.com>
> Signed-off-by: Wei Huang <wei.huang2@amd.com>
> Signed-off-by: Manoj Panicker <manoj.panicker2@amd.com>
> Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
> Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> ---
> Cc: Bjorn Helgaas <helgaas@kernel.org>
>
> Previous driver series fixing rtnl_lock and empty release function:
>
> https://lore.kernel.org/netdev/20241115200412.1340286-1-wei.huang2@amd
> .com/
>
> v5 of the PCI series using netdev_rx_queue_restart():
>
> https://lore.kernel.org/netdev/20240916205103.3882081-5-wei.huang2@amd
> .com/
>
> v1 of the PCI series using open/close:
>
> https://lore.kernel.org/netdev/20240509162741.1937586-9-wei.huang2@amd
> .com/
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 105 ++++++++++++++++++++++
> drivers/net/ethernet/broadcom/bnxt/bnxt.h | 7 ++
> 2 files changed, 112 insertions(+)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> index 0a10a4cffcc8..8c24642b8812 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> @@ -55,6 +55,8 @@
> #include <net/page_pool/helpers.h>
> #include <linux/align.h>
> #include <net/netdev_queues.h>
> +#include <net/netdev_rx_queue.h>
> +#include <linux/pci-tph.h>
>
> #include "bnxt_hsi.h"
> #include "bnxt.h"
> @@ -11330,6 +11332,83 @@ static int bnxt_tx_queue_start(struct bnxt *bp, int idx)
> return 0;
> }
>
> +static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
> + const cpumask_t *mask) {
> + struct bnxt_irq *irq;
> + u16 tag;
> + int err;
> +
> + irq = container_of(notify, struct bnxt_irq, affinity_notify);
> +
> + if (!irq->bp->tph_mode)
> + return;
> +
Can it not be set? The notifier is registered only if it is set, can mode change while irq notifier is registered? Maybe I am missing sth, but it looks like it can't.
> + cpumask_copy(irq->cpu_mask, mask);
> +
> + if (irq->ring_nr >= irq->bp->rx_nr_rings)
> + return;
> +
> + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
> + cpumask_first(irq->cpu_mask), &tag))
> + return;
> +
> + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag))
> + return;
> +
> + rtnl_lock();
> + if (netif_running(irq->bp->dev)) {
> + err = netdev_rx_queue_restart(irq->bp->dev, irq->ring_nr);
> + if (err)
> + netdev_err(irq->bp->dev,
> + "RX queue restart failed: err=%d\n", err);
> + }
> + rtnl_unlock();
> +}
> +
> +static void bnxt_irq_affinity_release(struct kref *ref) {
> + struct irq_affinity_notify *notify =
> + container_of(ref, struct irq_affinity_notify, kref);
> + struct bnxt_irq *irq;
> +
> + irq = container_of(notify, struct bnxt_irq, affinity_notify);
> +
> + if (!irq->bp->tph_mode)
The same here.
> + return;
> +
> + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, 0)) {
> + netdev_err(irq->bp->dev,
> + "Setting ST=0 for MSIX entry %d failed\n",
> + irq->msix_nr);
> + return;
> + }
> +}
> +
> +static void bnxt_release_irq_notifier(struct bnxt_irq *irq) {
> + irq_set_affinity_notifier(irq->vector, NULL); }
> +
> +static void bnxt_register_irq_notifier(struct bnxt *bp, struct
> +bnxt_irq *irq) {
> + struct irq_affinity_notify *notify;
> +
> + irq->bp = bp;
> +
> + /* Nothing to do if TPH is not enabled */
> + if (!bp->tph_mode)
> + return;
> +
> + /* Register IRQ affinity notifier */
> + notify = &irq->affinity_notify;
> + notify->irq = irq->vector;
> + notify->notify = bnxt_irq_affinity_notify;
> + notify->release = bnxt_irq_affinity_release;
> +
> + irq_set_affinity_notifier(irq->vector, notify); }
> +
> static void bnxt_free_irq(struct bnxt *bp) {
> struct bnxt_irq *irq;
> @@ -11352,11 +11431,18 @@ static void bnxt_free_irq(struct bnxt *bp)
> free_cpumask_var(irq->cpu_mask);
> irq->have_cpumask = 0;
> }
> +
> + bnxt_release_irq_notifier(irq);
> +
> free_irq(irq->vector, bp->bnapi[i]);
> }
>
> irq->requested = 0;
> }
> +
> + /* Disable TPH support */
> + pcie_disable_tph(bp->pdev);
> + bp->tph_mode = 0;
> }
>
> static int bnxt_request_irq(struct bnxt *bp) @@ -11376,6 +11462,12 @@
> static int bnxt_request_irq(struct bnxt *bp) #ifdef CONFIG_RFS_ACCEL
> rmap = bp->dev->rx_cpu_rmap;
> #endif
> +
> + /* Enable TPH support as part of IRQ request */
> + rc = pcie_enable_tph(bp->pdev, PCI_TPH_ST_IV_MODE);
> + if (!rc)
> + bp->tph_mode = PCI_TPH_ST_IV_MODE;
> +
> for (i = 0, j = 0; i < bp->cp_nr_rings; i++) {
> int map_idx = bnxt_cp_num_to_irq_num(bp, i);
> struct bnxt_irq *irq = &bp->irq_tbl[map_idx]; @@
> -11399,8 +11491,11 @@ static int bnxt_request_irq(struct bnxt *bp)
>
> if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) {
> int numa_node = dev_to_node(&bp->pdev->dev);
> + u16 tag;
>
> irq->have_cpumask = 1;
> + irq->msix_nr = map_idx;
> + irq->ring_nr = i;
> cpumask_set_cpu(cpumask_local_spread(i, numa_node),
> irq->cpu_mask);
> rc = irq_update_affinity_hint(irq->vector,
> irq->cpu_mask); @@ -11410,6 +11505,16 @@ static int bnxt_request_irq(struct bnxt *bp)
> irq->vector);
> break;
> }
> +
> + bnxt_register_irq_notifier(bp, irq);
> +
> + /* Init ST table entry */
> + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
> + cpumask_first(irq->cpu_mask),
> + &tag))
> + continue;
> +
> + pcie_tph_set_st_entry(irq->bp->pdev,
> + irq->msix_nr, tag);
> }
> }
> return rc;
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> index 826ae030fc09..02dc2ed9c75d 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
> @@ -1234,6 +1234,11 @@ struct bnxt_irq {
> u8 have_cpumask:1;
> char name[IFNAMSIZ + BNXT_IRQ_NAME_EXTRA];
> cpumask_var_t cpu_mask;
> +
> + struct bnxt *bp;
> + int msix_nr;
> + int ring_nr;
> + struct irq_affinity_notify affinity_notify;
> };
>
> #define HWRM_RING_ALLOC_TX 0x1
> @@ -2229,6 +2234,8 @@ struct bnxt {
> struct net_device *dev;
> struct pci_dev *pdev;
>
> + u8 tph_mode;
> +
> atomic_t intr_sem;
>
> u32 flags;
> --
> 2.30.1
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
` (2 preceding siblings ...)
2025-01-17 18:07 ` kernel test robot
@ 2025-01-30 5:39 ` Dan Carpenter
3 siblings, 0 replies; 18+ messages in thread
From: Dan Carpenter @ 2025-01-30 5:39 UTC (permalink / raw)
To: oe-kbuild, Michael Chan, davem
Cc: lkp, oe-kbuild-all, netdev, edumazet, kuba, pabeni, andrew+netdev,
pavan.chebbi, andrew.gospodarek, michal.swiatkowski, helgaas,
Somnath Kotur, Ajit Khaparde, David Wei
Hi Michael,
kernel test robot noticed the following build warnings:
url: https://github.com/intel-lab-lkp/linux/commits/Michael-Chan/bnxt_en-Set-NAPR-1-2-support-when-registering-with-firmware/20250117-032822
base: net-next/main
patch link: https://lore.kernel.org/r/20250116192343.34535-10-michael.chan%40broadcom.com
patch subject: [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings
config: x86_64-randconfig-r071-20250126 (https://download.01.org/0day-ci/archive/20250130/202501300808.5bqgHMA6-lkp@intel.com/config)
compiler: gcc-11 (Debian 11.3.0-12) 11.3.0
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
| Closes: https://lore.kernel.org/r/202501300808.5bqgHMA6-lkp@intel.com/
New smatch warnings:
drivers/net/ethernet/broadcom/bnxt/bnxt.c:15769 bnxt_queue_start() error: uninitialized symbol 'cpr'.
vim +/cpr +15769 drivers/net/ethernet/broadcom/bnxt/bnxt.c
2d694c27d32efc David Wei 2024-06-18 15691 static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx)
2d694c27d32efc David Wei 2024-06-18 15692 {
2d694c27d32efc David Wei 2024-06-18 15693 struct bnxt *bp = netdev_priv(dev);
2d694c27d32efc David Wei 2024-06-18 15694 struct bnxt_rx_ring_info *rxr, *clone;
2d694c27d32efc David Wei 2024-06-18 15695 struct bnxt_cp_ring_info *cpr;
b9d2956e869c78 David Wei 2024-08-07 15696 struct bnxt_vnic_info *vnic;
5d2310614bc883 Somnath Kotur 2025-01-16 15697 struct bnxt_napi *bnapi;
b9d2956e869c78 David Wei 2024-08-07 15698 int i, rc;
2d694c27d32efc David Wei 2024-06-18 15699
2d694c27d32efc David Wei 2024-06-18 15700 rxr = &bp->rx_ring[idx];
2d694c27d32efc David Wei 2024-06-18 15701 clone = qmem;
2d694c27d32efc David Wei 2024-06-18 15702
2d694c27d32efc David Wei 2024-06-18 15703 rxr->rx_prod = clone->rx_prod;
2d694c27d32efc David Wei 2024-06-18 15704 rxr->rx_agg_prod = clone->rx_agg_prod;
2d694c27d32efc David Wei 2024-06-18 15705 rxr->rx_sw_agg_prod = clone->rx_sw_agg_prod;
2d694c27d32efc David Wei 2024-06-18 15706 rxr->rx_next_cons = clone->rx_next_cons;
bd649c5cc95816 David Wei 2024-12-03 15707 rxr->rx_tpa = clone->rx_tpa;
bd649c5cc95816 David Wei 2024-12-03 15708 rxr->rx_tpa_idx_map = clone->rx_tpa_idx_map;
2d694c27d32efc David Wei 2024-06-18 15709 rxr->page_pool = clone->page_pool;
bd649c5cc95816 David Wei 2024-12-03 15710 rxr->head_pool = clone->head_pool;
b537633ce57b29 Taehee Yoo 2024-07-21 15711 rxr->xdp_rxq = clone->xdp_rxq;
2d694c27d32efc David Wei 2024-06-18 15712
2d694c27d32efc David Wei 2024-06-18 15713 bnxt_copy_rx_ring(bp, rxr, clone);
2d694c27d32efc David Wei 2024-06-18 15714
5d2310614bc883 Somnath Kotur 2025-01-16 15715 bnapi = rxr->bnapi;
2d694c27d32efc David Wei 2024-06-18 15716 rc = bnxt_hwrm_rx_ring_alloc(bp, rxr);
2d694c27d32efc David Wei 2024-06-18 15717 if (rc)
5d2310614bc883 Somnath Kotur 2025-01-16 15718 goto err_reset_rx;
cpr isn't initialized
377e78a9e08ce5 Somnath Kotur 2025-01-16 15719
377e78a9e08ce5 Somnath Kotur 2025-01-16 15720 rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr);
2d694c27d32efc David Wei 2024-06-18 15721 if (rc)
5d2310614bc883 Somnath Kotur 2025-01-16 15722 goto err_reset_rx;
2d694c27d32efc David Wei 2024-06-18 15723
377e78a9e08ce5 Somnath Kotur 2025-01-16 15724 rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr);
377e78a9e08ce5 Somnath Kotur 2025-01-16 15725 if (rc)
5d2310614bc883 Somnath Kotur 2025-01-16 15726 goto err_reset_rx;
377e78a9e08ce5 Somnath Kotur 2025-01-16 15727
2d694c27d32efc David Wei 2024-06-18 15728 bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod);
2d694c27d32efc David Wei 2024-06-18 15729 if (bp->flags & BNXT_FLAG_AGG_RINGS)
2d694c27d32efc David Wei 2024-06-18 15730 bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod);
2d694c27d32efc David Wei 2024-06-18 15731
5d2310614bc883 Somnath Kotur 2025-01-16 15732 cpr = &bnapi->cp_ring;
2d694c27d32efc David Wei 2024-06-18 15733 cpr->sw_stats->rx.rx_resets++;
2d694c27d32efc David Wei 2024-06-18 15734
5d2310614bc883 Somnath Kotur 2025-01-16 15735 if (bp->flags & BNXT_FLAG_SHARED_RINGS) {
5d2310614bc883 Somnath Kotur 2025-01-16 15736 cpr->sw_stats->tx.tx_resets++;
5d2310614bc883 Somnath Kotur 2025-01-16 15737 rc = bnxt_tx_queue_start(bp, idx);
5d2310614bc883 Somnath Kotur 2025-01-16 15738 if (rc) {
5d2310614bc883 Somnath Kotur 2025-01-16 15739 netdev_warn(bp->dev,
5d2310614bc883 Somnath Kotur 2025-01-16 15740 "tx queue restart failed: rc=%d\n", rc);
5d2310614bc883 Somnath Kotur 2025-01-16 15741 bnapi->tx_fault = 1;
5d2310614bc883 Somnath Kotur 2025-01-16 15742 goto err_reset;
5d2310614bc883 Somnath Kotur 2025-01-16 15743 }
5d2310614bc883 Somnath Kotur 2025-01-16 15744 }
5d2310614bc883 Somnath Kotur 2025-01-16 15745
5d2310614bc883 Somnath Kotur 2025-01-16 15746 napi_enable(&bnapi->napi);
5d2310614bc883 Somnath Kotur 2025-01-16 15747 bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
5d2310614bc883 Somnath Kotur 2025-01-16 15748
b9d2956e869c78 David Wei 2024-08-07 15749 for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) {
b9d2956e869c78 David Wei 2024-08-07 15750 vnic = &bp->vnic_info[i];
5ac066b7b062ee Somnath Kotur 2024-11-22 15751
5ac066b7b062ee Somnath Kotur 2024-11-22 15752 rc = bnxt_hwrm_vnic_set_rss_p5(bp, vnic, true);
5ac066b7b062ee Somnath Kotur 2024-11-22 15753 if (rc) {
5ac066b7b062ee Somnath Kotur 2024-11-22 15754 netdev_err(bp->dev, "hwrm vnic %d set rss failure rc: %d\n",
5ac066b7b062ee Somnath Kotur 2024-11-22 15755 vnic->vnic_id, rc);
5ac066b7b062ee Somnath Kotur 2024-11-22 15756 return rc;
5ac066b7b062ee Somnath Kotur 2024-11-22 15757 }
b9d2956e869c78 David Wei 2024-08-07 15758 vnic->mru = bp->dev->mtu + ETH_HLEN + VLAN_HLEN;
b9d2956e869c78 David Wei 2024-08-07 15759 bnxt_hwrm_vnic_update(bp, vnic,
b9d2956e869c78 David Wei 2024-08-07 15760 VNIC_UPDATE_REQ_ENABLES_MRU_VALID);
b9d2956e869c78 David Wei 2024-08-07 15761 }
b9d2956e869c78 David Wei 2024-08-07 15762
2d694c27d32efc David Wei 2024-06-18 15763 return 0;
2d694c27d32efc David Wei 2024-06-18 15764
5d2310614bc883 Somnath Kotur 2025-01-16 15765 err_reset_rx:
5d2310614bc883 Somnath Kotur 2025-01-16 15766 rxr->bnapi->in_reset = true;
5d2310614bc883 Somnath Kotur 2025-01-16 15767 err_reset:
5d2310614bc883 Somnath Kotur 2025-01-16 15768 napi_enable(&bnapi->napi);
5d2310614bc883 Somnath Kotur 2025-01-16 @15769 bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Uninitialized variable
5d2310614bc883 Somnath Kotur 2025-01-16 15770 bnxt_reset_task(bp, true);
2d694c27d32efc David Wei 2024-06-18 15771 return rc;
2d694c27d32efc David Wei 2024-06-18 15772 }
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2025-01-30 5:39 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-01-16 19:23 [PATCH net-next v2 00/10] bnxt_en: Add NPAR 1.2 and TPH support Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 01/10] bnxt_en: Set NAPR 1.2 support when registering with firmware Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 02/10] bnxt_en Refactor completion ring allocation logic for P5_PLUS chips Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 03/10] bnxt_en: Refactor TX ring allocation logic Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 04/10] bnxt_en: Refactor completion ring free routine Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 05/10] bnxt_en: Refactor bnxt_free_tx_rings() to free per TX ring Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 06/10] bnxt_en: Refactor RX/RX AGG ring parameters setup for P5_PLUS Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 07/10] bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 08/10] bnxt_en: Reallocate RX completion ring for TPH support Michael Chan
2025-01-16 19:23 ` [PATCH net-next v2 09/10] bnxt_en: Extend queue stop/start for TX rings Michael Chan
2025-01-17 6:56 ` Michal Swiatkowski
2025-01-17 15:29 ` Simon Horman
2025-01-17 18:07 ` kernel test robot
2025-01-30 5:39 ` Dan Carpenter
2025-01-16 19:23 ` [PATCH net-next v2 10/10] bnxt_en: Add TPH support in BNXT driver Michael Chan
2025-01-17 7:04 ` Michal Swiatkowski
2025-01-21 22:15 ` Panicker, Manoj
2025-01-17 15:41 ` Simon Horman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).