From: Jakub Kicinski <kuba@kernel.org>
To: davem@davemloft.net, michael.chan@broadcom.com
Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com,
andrew+netdev@lunn.ch, horms@kernel.org,
pavan.chebbi@broadcom.com, Jakub Kicinski <kuba@kernel.org>
Subject: [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node
Date: Tue, 17 Nov 2026 18:39:04 -0800 [thread overview]
Message-ID: <20261118023905.3050661-6-kuba@kernel.org> (raw)
In-Reply-To: <20261118023905.3050661-1-kuba@kernel.org>
Descriptor rings, completion rings, NQs and the TX push buffer are
accessed by the host in the fast path, and so is the SW state around
them, which we already allocate on the node of the CPU expected to
service the queue. The DMA memory itself comes from the device's node,
though, as that's what dma_alloc_coherent() always uses.
Use dma_alloc_coherent_node() for the ring pages and the TX push buffer.
Pass the queue's node for completion rings and NQs as well, they used
NUMA_NO_NODE so far since @nid only affected the SW rings.
Ring page tables are only read by the device, keep them on the device's
node.
Note that queue restart only reallocates the Rx descriptor rings.
Completion rings, NQs and Tx rings will follow IRQ affinity changes
only on the next full reconfiguration.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 24 +++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index f98507944350..134b24842c4f 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3790,10 +3790,10 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
for (i = 0; i < rmem->nr_pages; i++) {
u64 extra_bits = valid_bit;
- rmem->pg_arr[i] = dma_alloc_coherent(&pdev->dev,
- rmem->page_size,
- &rmem->dma_arr[i],
- GFP_KERNEL);
+ rmem->pg_arr[i] = dma_alloc_coherent_node(&pdev->dev,
+ rmem->page_size,
+ &rmem->dma_arr[i],
+ GFP_KERNEL, nid);
if (!rmem->pg_arr[i])
return -ENOMEM;
@@ -3812,7 +3812,6 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
}
}
- /* Only the SW ring follows @nid, DMA memory is on the device's node */
if (rmem->vmem_size) {
*rmem->vmem = vzalloc_node(rmem->vmem_size, nid);
if (!(*rmem->vmem))
@@ -4208,10 +4207,11 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)
/* One pre-allocated DMA buffer to backup
* TX push operation
*/
- txr->tx_push = dma_alloc_coherent(&pdev->dev,
- bp->tx_push_size,
- &txr->tx_push_mapping,
- GFP_KERNEL);
+ txr->tx_push =
+ dma_alloc_coherent_node(&pdev->dev,
+ bp->tx_push_size,
+ &txr->tx_push_mapping,
+ GFP_KERNEL, nid);
if (!txr->tx_push)
return -ENOMEM;
@@ -4353,7 +4353,7 @@ static int bnxt_alloc_cp_sub_ring(struct bnxt *bp,
rmem->pg_arr = (void **)cpr->cp_desc_ring;
rmem->dma_arr = cpr->cp_desc_mapping;
rmem->flags = BNXT_RMEM_RING_PTE_FLAG;
- rc = bnxt_alloc_ring(bp, rmem, NUMA_NO_NODE);
+ rc = bnxt_alloc_ring(bp, rmem, nid);
if (rc) {
bnxt_free_ring(bp, rmem);
bnxt_free_cp_arrays(cpr);
@@ -4384,7 +4384,8 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
cpr->bnapi = bnapi;
ring = &cpr->cp_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem, NUMA_NO_NODE);
+ nid = bnxt_cp_num_to_node(bp, i);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, nid);
if (rc)
return rc;
@@ -4406,7 +4407,6 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
tx = 1;
}
- nid = bnxt_cp_num_to_node(bp, i);
cpr->cp_ring_arr = kcalloc_node(cp_count, sizeof(*cpr),
GFP_KERNEL, nid);
if (!cpr->cp_ring_arr)
--
2.56.1
next prev parent reply other threads:[~2026-11-18 2:39 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
2026-11-18 2:39 ` Jakub Kicinski [this message]
2026-11-18 2:39 ` [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another NUMA node Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118023905.3050661-6-kuba@kernel.org \
--to=kuba@kernel.org \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=horms@kernel.org \
--cc=michael.chan@broadcom.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=pavan.chebbi@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.