* [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node
@ 2026-11-18 2:38 Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
` (5 more replies)
0 siblings, 6 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:38 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Teach bnxt to allocate the memory touched by the fast path
on the NUMA node where NAPI is being serviced.
On modern systems the PCIe controller is part of the CPU.
On NUMA systems we get to pick which NUMA node the queue and NAPI
related memory will get allocated from. Allocating SW state on
the node where NAPI will run is just common sense.
For memory touched by both device and the CPU we need to pick
which side will pay the extra price (either device or CPU
will have to "reach" futher). This series allocates all state
and all descriptor rings on the NAPI CPU.
Testing a dual-IO-die AMD Venice CPU, where NIC is on node 1
and with NAPI+kperf pinned to node 0, this series gives:
placing page pools on the right node: -5.1% CPU load
placing page pools and completion rings: -8.9% CPU load
Moving SW state is in the noise in a tight-loop kperf banchmark
as the state fits in the cache, anyway.
This series was the motivation for cleaning up the DMA API
allocations with explicit NUMA node:
commit a0aa9ceca03f ("Merge tag 'dma_alloc_coherent_node' of git://git.kernel.org/pub/scm/linux/kernel/git/kuba/linux")
Jakub Kicinski (6):
eth: bnxt_en: order event data before setting the sp_event bit
eth: bnxt_en: follow explicitly set IRQ affinity when placing page
pool
eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node
eth: bnxt_en: explicitly keep device-only memory on the device's node
eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node
eth: bnxt_en: restart Rx queues when their IRQ moves to another NUMA
node
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 7 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 203 ++++++++++++++--------
2 files changed, 141 insertions(+), 69 deletions(-)
--
2.56.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
@ 2026-11-18 2:39 ` Jakub Kicinski
2026-11-18 12:27 ` Björn Töpel
2026-11-18 2:39 ` [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
` (4 subsequent siblings)
5 siblings, 1 reply; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:39 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Callers of bnxt_queue_sp_work() may store data for bnxt_sp_task()
right before raising the event, e.g. the IRQ affinity notifier
publishes the new TPH Steering Tag. set_bit() is not ordered, so on
weakly ordered architectures the event bit may become visible first.
If bnxt_sp_task() is already running it can consume the event and
read the stale data. The queue_work() which follows only re-runs
the task with the event bit already cleared, so the update is lost.
Add smp_mb__before_atomic() before setting the bit, it pairs with
the fully ordered test_and_clear_bit() in bnxt_sp_task().
Reported by an LLM reviewer while working on subsequent patches.
Very unlikely to matter in practice.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 6846a27bfef4..5ad2a3033505 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -394,6 +394,10 @@ static void __bnxt_queue_sp_work(struct bnxt *bp)
static void bnxt_queue_sp_work(struct bnxt *bp, unsigned int event)
{
+ /* Make the data the event carries visible before the event bit,
+ * pairs with test_and_clear_bit() in bnxt_sp_task().
+ */
+ smp_mb__before_atomic();
set_bit(event, &bp->sp_event);
__bnxt_queue_sp_work(bp);
}
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
@ 2026-11-18 2:39 ` Jakub Kicinski
2026-11-18 12:32 ` Björn Töpel
2026-11-18 14:42 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
` (3 subsequent siblings)
5 siblings, 2 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:39 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
The driver picks the NUMA node for each Rx ring's page pool with
cpumask_local_spread() at open. IRQ affinity configured by the user
is already recorded by the driver and persists across close/open
cycles. Use it instead of cpumask_local_spread().
The affinity notifier runs without any of the locks held by
the allocation paths, and copying a multi-word cpumask is not atomic.
Like for the TPH Steering Tag, have the notifier compute the node
(of the first online CPU in the mask) and publish it with WRITE_ONCE().
Queue restart (bnxt_queue_mem_alloc()) needs to resolve the NUMA
node instead of copying the nid of the old page pool.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 5 +++
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 48 +++++++++++++++++++----
2 files changed, 46 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 41d05efd6307..86fc2e8a5816 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2488,6 +2488,11 @@ struct bnxt {
* rather than being our own default placement.
*/
unsigned long *ring_affinity_set;
+ /* Node of the first online CPU in the mask configured from
+ * the outside, NUMA_NO_NODE if none. Written by the affinity
+ * notifier, read locklessly by the allocation paths.
+ */
+ int *ring_nid;
int max_irqs;
int total_irqs;
int ulp_num_msix_want;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 5ad2a3033505..8e8ee02ca8d9 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3818,6 +3818,21 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
return 0;
}
+/* Node of the CPU expected to service cp ring @n. If the IRQ affinity was
+ * configured from the outside (by the user or irqbalance) the affinity
+ * notifier recorded the node to use, otherwise follow our default spread.
+ */
+static int bnxt_cp_num_to_node(struct bnxt *bp, int n)
+{
+ int nid = READ_ONCE(bp->ring_nid[n]);
+ unsigned int cpu;
+
+ if (nid != NUMA_NO_NODE)
+ return nid;
+ cpu = cpumask_local_spread(n, dev_to_node(&bp->pdev->dev));
+ return cpu_to_node(cpu);
+}
+
static void bnxt_free_one_tpa_info(struct bnxt *bp,
struct bnxt_rx_ring_info *rxr)
{
@@ -4020,8 +4035,7 @@ static int bnxt_alloc_rx_agg_bmap(struct bnxt *bp, struct bnxt_rx_ring_info *rxr
static int bnxt_alloc_rx_rings(struct bnxt *bp)
{
- int numa_node = dev_to_node(&bp->pdev->dev);
- int i, rc = 0, agg_rings = 0, cpu;
+ int i, rc = 0, agg_rings = 0;
if (!bp->rx_ring)
return -ENOMEM;
@@ -4036,8 +4050,7 @@ static int bnxt_alloc_rx_rings(struct bnxt *bp)
ring = &rxr->rx_ring_struct;
- cpu = cpumask_local_spread(i, numa_node);
- cpu_node = cpu_to_node(cpu);
+ cpu_node = bnxt_cp_num_to_node(bp, i);
netdev_dbg(bp->dev, "Allocating page pool for rx_ring[%d] on numa_node: %d\n",
i, cpu_node);
rc = bnxt_alloc_rx_page_pool(bp, rxr, cpu_node);
@@ -11870,6 +11883,8 @@ static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
const cpumask_t *mask)
{
struct bnxt_irq *irq;
+ unsigned int cpu;
+ int nid;
u16 tag;
irq = container_of(notify, struct bnxt_irq, affinity_notify);
@@ -11877,6 +11892,14 @@ static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
cpumask_copy(irq->bp->ring_cpu_mask[irq->ring_nr], mask);
set_bit(irq->ring_nr, irq->bp->ring_affinity_set);
+ /* Allocation paths don't hold any lock which would exclude us,
+ * publish the node to allocate on instead of making them parse
+ * the mask while it may be getting overwritten.
+ */
+ cpu = cpumask_first_and(mask, cpu_online_mask);
+ nid = cpu < nr_cpu_ids ? cpu_to_node(cpu) : NUMA_NO_NODE;
+ WRITE_ONCE(irq->bp->ring_nid[irq->ring_nr], nid);
+
#ifdef CONFIG_RFS_ACCEL
if (irq->bp->dev->rx_cpu_rmap && irq->ring_nr < irq->bp->rx_nr_rings) {
int err;
@@ -11954,9 +11977,16 @@ static int bnxt_alloc_ring_cpu_masks(struct bnxt *bp)
if (!bp->ring_affinity_set)
return -ENOMEM;
- for (i = 0; i < bp->max_irqs; i++)
+ bp->ring_nid = kmalloc_array(bp->max_irqs, sizeof(*bp->ring_nid),
+ GFP_KERNEL);
+ if (!bp->ring_nid)
+ return -ENOMEM;
+
+ for (i = 0; i < bp->max_irqs; i++) {
if (!zalloc_cpumask_var(&bp->ring_cpu_mask[i], GFP_KERNEL))
return -ENOMEM;
+ bp->ring_nid[i] = NUMA_NO_NODE;
+ }
return 0;
}
@@ -11971,6 +12001,8 @@ static void bnxt_free_ring_cpu_masks(struct bnxt *bp)
for (i = 0; i < bp->max_irqs; i++)
free_cpumask_var(bp->ring_cpu_mask[i]);
+ kfree(bp->ring_nid);
+ bp->ring_nid = NULL;
bitmap_free(bp->ring_affinity_set);
bp->ring_affinity_set = NULL;
kfree(bp->ring_cpu_mask);
@@ -12066,6 +12098,7 @@ static int bnxt_request_irq(struct bnxt *bp)
if (!test_bit(i, bp->ring_affinity_set) ||
!cpumask_intersects(cpu_mask, cpu_online_mask)) {
clear_bit(i, bp->ring_affinity_set);
+ WRITE_ONCE(bp->ring_nid[i], NUMA_NO_NODE);
cpumask_clear(cpu_mask);
cpumask_set_cpu(cpumask_local_spread(i, numa_node),
cpu_mask);
@@ -16431,7 +16464,7 @@ static int bnxt_queue_mem_alloc(struct net_device *dev,
struct bnxt_rx_ring_info *rxr, *clone;
struct bnxt *bp = netdev_priv(dev);
struct bnxt_ring_struct *ring;
- int rc;
+ int nid, rc;
if (!bp->rx_ring)
return -ENETDOWN;
@@ -16452,7 +16485,8 @@ static int bnxt_queue_mem_alloc(struct net_device *dev,
clone->rx_tpa = NULL;
clone->rx_tpa_idx_map = NULL;
- rc = bnxt_alloc_rx_page_pool(bp, clone, rxr->page_pool->p.nid);
+ nid = bnxt_cp_num_to_node(bp, idx);
+ rc = bnxt_alloc_rx_page_pool(bp, clone, nid);
if (rc)
return rc;
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
@ 2026-11-18 2:39 ` Jakub Kicinski
2026-11-18 12:37 ` Björn Töpel
2026-11-18 14:49 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
` (2 subsequent siblings)
5 siblings, 2 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:39 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Allocate the SW rings, agg bitmap, TPA state, completion ring
page arrays, completion sub-ring structs, SW stats and the SW USO
header buffer on the NUMA node of the CPU which will service the queue.
This lowers the cost of cache misses for NAPI fast path (and xmit, for Tx).
Previous change already allocated the page pools on the right node.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 87 ++++++++++++++---------
1 file changed, 52 insertions(+), 35 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 8e8ee02ca8d9..7c694d8e99f5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3765,7 +3765,8 @@ static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
}
}
-static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
+static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
+ int nid)
{
struct pci_dev *pdev = bp->pdev;
u64 valid_bit = 0;
@@ -3810,8 +3811,9 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
}
}
+ /* Only the SW ring follows @nid, DMA memory is on the device's node */
if (rmem->vmem_size) {
- *rmem->vmem = vzalloc(rmem->vmem_size);
+ *rmem->vmem = vzalloc_node(rmem->vmem_size, nid);
if (!(*rmem->vmem))
return -ENOMEM;
}
@@ -3862,25 +3864,27 @@ static void bnxt_free_tpa_info(struct bnxt *bp)
}
static int bnxt_alloc_one_tpa_info(struct bnxt *bp,
- struct bnxt_rx_ring_info *rxr)
+ struct bnxt_rx_ring_info *rxr, int nid)
{
struct rx_agg_cmp *agg;
int i;
- rxr->rx_tpa = kzalloc_objs(struct bnxt_tpa_info,
- bp->max_tpa_roundup_size);
+ rxr->rx_tpa = kcalloc_node(bp->max_tpa_roundup_size,
+ sizeof(*rxr->rx_tpa), GFP_KERNEL, nid);
if (!rxr->rx_tpa)
return -ENOMEM;
if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS))
return 0;
for (i = 0; i < bp->max_tpa_roundup_size; i++) {
- agg = kzalloc_objs(*agg, MAX_SKB_FRAGS);
+ agg = kcalloc_node(MAX_SKB_FRAGS, sizeof(*agg),
+ GFP_KERNEL, nid);
if (!agg)
return -ENOMEM;
rxr->rx_tpa[i].agg_arr = agg;
}
- rxr->rx_tpa_idx_map = kzalloc_obj(*rxr->rx_tpa_idx_map);
+ rxr->rx_tpa_idx_map = kzalloc_node(sizeof(*rxr->rx_tpa_idx_map),
+ GFP_KERNEL, nid);
if (!rxr->rx_tpa_idx_map)
return -ENOMEM;
@@ -3907,8 +3911,10 @@ static int bnxt_alloc_tpa_info(struct bnxt *bp)
for (i = 0; i < bp->rx_nr_rings; i++) {
struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i];
+ int nid;
- rc = bnxt_alloc_one_tpa_info(bp, rxr);
+ nid = bnxt_cp_num_to_node(bp, i);
+ rc = bnxt_alloc_one_tpa_info(bp, rxr, nid);
if (rc)
return rc;
}
@@ -4020,13 +4026,14 @@ static void bnxt_enable_rx_page_pool(struct bnxt_rx_ring_info *rxr)
page_pool_enable_direct_recycling(rxr->page_pool, &rxr->bnapi->napi);
}
-static int bnxt_alloc_rx_agg_bmap(struct bnxt *bp, struct bnxt_rx_ring_info *rxr)
+static int bnxt_alloc_rx_agg_bmap(struct bnxt *bp,
+ struct bnxt_rx_ring_info *rxr, int nid)
{
u16 mem_size;
rxr->rx_agg_bmap_size = bp->rx_agg_ring_mask + 1;
mem_size = rxr->rx_agg_bmap_size / 8;
- rxr->rx_agg_bmap = kzalloc(mem_size, GFP_KERNEL);
+ rxr->rx_agg_bmap = kzalloc_node(mem_size, GFP_KERNEL, nid);
if (!rxr->rx_agg_bmap)
return -ENOMEM;
@@ -4070,19 +4077,19 @@ static int bnxt_alloc_rx_rings(struct bnxt *bp)
return rc;
}
- rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, cpu_node);
if (rc)
return rc;
ring->grp_idx = i;
if (agg_rings) {
ring = &rxr->rx_agg_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, cpu_node);
if (rc)
return rc;
ring->grp_idx = i;
- rc = bnxt_alloc_rx_agg_bmap(bp, rxr);
+ rc = bnxt_alloc_rx_agg_bmap(bp, rxr, cpu_node);
if (rc)
return rc;
}
@@ -4107,9 +4114,9 @@ static void bnxt_free_tx_inline_buf(struct bnxt_tx_ring_info *txr,
static int bnxt_alloc_tx_inline_buf(struct bnxt_tx_ring_info *txr,
struct pci_dev *pdev,
- unsigned int size)
+ unsigned int size, int nid)
{
- txr->tx_inline_buf = kmalloc(size, GFP_KERNEL);
+ txr->tx_inline_buf = kmalloc_node(size, GFP_KERNEL, nid);
if (!txr->tx_inline_buf)
return -ENOMEM;
@@ -4183,11 +4190,13 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)
for (i = 0, j = 0; i < bp->tx_nr_rings; i++) {
struct bnxt_tx_ring_info *txr = &bp->tx_ring[i];
struct bnxt_ring_struct *ring;
+ int nid;
u8 qidx;
ring = &txr->tx_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ nid = bnxt_cp_num_to_node(bp, txr->bnapi->index);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, nid);
if (rc)
return rc;
@@ -4213,7 +4222,7 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)
if (!(bp->flags & BNXT_FLAG_UDP_GSO_CAP)) {
rc = bnxt_alloc_tx_inline_buf(txr, pdev,
BNXT_SW_USO_MAX_SEGS *
- TSO_HEADER_SIZE);
+ TSO_HEADER_SIZE, nid);
if (rc)
return rc;
}
@@ -4240,12 +4249,15 @@ static void bnxt_free_cp_arrays(struct bnxt_cp_ring_info *cpr)
ring->ring_mem.dma_arr = NULL;
}
-static int bnxt_alloc_cp_arrays(struct bnxt_cp_ring_info *cpr, int n)
+static int bnxt_alloc_cp_arrays(struct bnxt_cp_ring_info *cpr, int n,
+ int nid)
{
- cpr->cp_desc_ring = kzalloc_objs(*cpr->cp_desc_ring, n);
+ cpr->cp_desc_ring = kcalloc_node(n, sizeof(*cpr->cp_desc_ring),
+ GFP_KERNEL, nid);
if (!cpr->cp_desc_ring)
return -ENOMEM;
- cpr->cp_desc_mapping = kzalloc_objs(*cpr->cp_desc_mapping, n);
+ cpr->cp_desc_mapping = kcalloc_node(n, sizeof(*cpr->cp_desc_mapping),
+ GFP_KERNEL, nid);
if (!cpr->cp_desc_mapping)
return -ENOMEM;
return 0;
@@ -4272,11 +4284,12 @@ static int bnxt_alloc_all_cp_arrays(struct bnxt *bp)
for (i = 0; i < bp->cp_nr_rings; i++) {
struct bnxt_napi *bnapi = bp->bnapi[i];
- int rc;
+ int nid, rc;
if (!bnapi)
continue;
- rc = bnxt_alloc_cp_arrays(&bnapi->cp_ring, n);
+ nid = bnxt_cp_num_to_node(bp, i);
+ rc = bnxt_alloc_cp_arrays(&bnapi->cp_ring, n, nid);
if (rc)
return rc;
}
@@ -4321,13 +4334,13 @@ static void bnxt_free_cp_rings(struct bnxt *bp)
}
static int bnxt_alloc_cp_sub_ring(struct bnxt *bp,
- struct bnxt_cp_ring_info *cpr)
+ struct bnxt_cp_ring_info *cpr, int nid)
{
struct bnxt_ring_mem_info *rmem;
struct bnxt_ring_struct *ring;
int rc;
- rc = bnxt_alloc_cp_arrays(cpr, bp->cp_nr_pages);
+ rc = bnxt_alloc_cp_arrays(cpr, bp->cp_nr_pages, nid);
if (rc) {
bnxt_free_cp_arrays(cpr);
return -ENOMEM;
@@ -4339,7 +4352,7 @@ static int bnxt_alloc_cp_sub_ring(struct bnxt *bp,
rmem->pg_arr = (void **)cpr->cp_desc_ring;
rmem->dma_arr = cpr->cp_desc_mapping;
rmem->flags = BNXT_RMEM_RING_PTE_FLAG;
- rc = bnxt_alloc_ring(bp, rmem);
+ rc = bnxt_alloc_ring(bp, rmem, NUMA_NO_NODE);
if (rc) {
bnxt_free_ring(bp, rmem);
bnxt_free_cp_arrays(cpr);
@@ -4360,7 +4373,7 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
struct bnxt_napi *bnapi = bp->bnapi[i];
struct bnxt_cp_ring_info *cpr, *cpr2;
struct bnxt_ring_struct *ring;
- int cp_count = 0, k;
+ int cp_count = 0, k, nid;
int rx = 0, tx = 0;
if (!bnapi)
@@ -4370,7 +4383,7 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
cpr->bnapi = bnapi;
ring = &cpr->cp_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, NUMA_NO_NODE);
if (rc)
return rc;
@@ -4392,14 +4405,16 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
tx = 1;
}
- cpr->cp_ring_arr = kzalloc_objs(*cpr, cp_count);
+ nid = bnxt_cp_num_to_node(bp, i);
+ cpr->cp_ring_arr = kcalloc_node(cp_count, sizeof(*cpr),
+ GFP_KERNEL, nid);
if (!cpr->cp_ring_arr)
return -ENOMEM;
cpr->cp_ring_count = cp_count;
for (k = 0; k < cp_count; k++) {
cpr2 = &cpr->cp_ring_arr[k];
- rc = bnxt_alloc_cp_sub_ring(bp, cpr2);
+ rc = bnxt_alloc_cp_sub_ring(bp, cpr2, nid);
if (rc)
return rc;
cpr2->bnapi = bnapi;
@@ -5411,8 +5426,10 @@ static int bnxt_alloc_stats(struct bnxt *bp)
for (i = 0; i < bp->cp_nr_rings; i++) {
struct bnxt_napi *bnapi = bp->bnapi[i];
struct bnxt_cp_ring_info *cpr = &bnapi->cp_ring;
+ int nid = bnxt_cp_num_to_node(bp, i);
- cpr->sw_stats = kzalloc_obj(*cpr->sw_stats);
+ cpr->sw_stats = kzalloc_node(sizeof(*cpr->sw_stats), GFP_KERNEL,
+ nid);
if (!cpr->sw_stats)
return -ENOMEM;
@@ -9201,7 +9218,7 @@ static int bnxt_alloc_ctx_mem_blk(struct bnxt *bp,
rmem->flags = BNXT_RMEM_VALID_PTE_FLAG;
if (rmem->depth >= 1)
rmem->flags |= BNXT_RMEM_USE_FULL_PAGE_FLAG;
- return bnxt_alloc_ring(bp, rmem);
+ return bnxt_alloc_ring(bp, rmem, NUMA_NO_NODE);
}
static int bnxt_alloc_ctx_pg_tbls(struct bnxt *bp,
@@ -16501,23 +16518,23 @@ static int bnxt_queue_mem_alloc(struct net_device *dev,
goto err_rxq_info_unreg;
ring = &clone->rx_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, nid);
if (rc)
goto err_free_rx_ring;
if (bp->flags & BNXT_FLAG_AGG_RINGS) {
ring = &clone->rx_agg_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, nid);
if (rc)
goto err_free_rx_agg_ring;
- rc = bnxt_alloc_rx_agg_bmap(bp, clone);
+ rc = bnxt_alloc_rx_agg_bmap(bp, clone, nid);
if (rc)
goto err_free_rx_agg_ring;
}
if (bp->flags & BNXT_FLAG_TPA) {
- rc = bnxt_alloc_one_tpa_info(bp, clone);
+ rc = bnxt_alloc_one_tpa_info(bp, clone, nid);
if (rc)
goto err_free_tpa_info;
}
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
` (2 preceding siblings ...)
2026-11-18 2:39 ` [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
@ 2026-11-18 2:39 ` Jakub Kicinski
2026-11-18 12:38 ` Björn Töpel
2026-11-18 14:50 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another " Jakub Kicinski
5 siblings, 2 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:39 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Context memory (and the FW crash dump buffer which reuses the same
helpers) is used by the device, once set up the host only touches it
on the slow path (diagnostics, FW trace). Pass the device's node to
bnxt_alloc_ring() rather than NUMA_NO_NODE to make it clear that this
memory is not meant to follow any queue.
Ring page tables are only read by the device, annotate their
allocation so that they stay put once the ring pages start following
the queue.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 7c694d8e99f5..f98507944350 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3779,6 +3779,7 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
if (rmem->flags & BNXT_RMEM_USE_FULL_PAGE_FLAG)
pg_tbl_size = rmem->page_size;
+ /* Only read by the device, keep it on the device's node */
rmem->pg_tbl = dma_alloc_coherent(&pdev->dev, pg_tbl_size,
&rmem->pg_tbl_map,
GFP_KERNEL);
@@ -9218,7 +9219,8 @@ static int bnxt_alloc_ctx_mem_blk(struct bnxt *bp,
rmem->flags = BNXT_RMEM_VALID_PTE_FLAG;
if (rmem->depth >= 1)
rmem->flags |= BNXT_RMEM_USE_FULL_PAGE_FLAG;
- return bnxt_alloc_ring(bp, rmem, NUMA_NO_NODE);
+ /* Host only uses it on the slow path, keep it on the device's node */
+ return bnxt_alloc_ring(bp, rmem, dev_to_node(&bp->pdev->dev));
}
static int bnxt_alloc_ctx_pg_tbls(struct bnxt *bp,
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
` (3 preceding siblings ...)
2026-11-18 2:39 ` [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
@ 2026-11-18 2:39 ` Jakub Kicinski
2026-11-18 12:39 ` Björn Töpel
2026-11-18 14:51 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another " Jakub Kicinski
5 siblings, 2 replies; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:39 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Descriptor rings, completion rings, NQs and the TX push buffer are
accessed by the host in the fast path, and so is the SW state around
them, which we already allocate on the node of the CPU expected to
service the queue. The DMA memory itself comes from the device's node,
though, as that's what dma_alloc_coherent() always uses.
Use dma_alloc_coherent_node() for the ring pages and the TX push buffer.
Pass the queue's node for completion rings and NQs as well, they used
NUMA_NO_NODE so far since @nid only affected the SW rings.
Ring page tables are only read by the device, keep them on the device's
node.
Note that queue restart only reallocates the Rx descriptor rings.
Completion rings, NQs and Tx rings will follow IRQ affinity changes
only on the next full reconfiguration.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 24 +++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index f98507944350..134b24842c4f 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3790,10 +3790,10 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
for (i = 0; i < rmem->nr_pages; i++) {
u64 extra_bits = valid_bit;
- rmem->pg_arr[i] = dma_alloc_coherent(&pdev->dev,
- rmem->page_size,
- &rmem->dma_arr[i],
- GFP_KERNEL);
+ rmem->pg_arr[i] = dma_alloc_coherent_node(&pdev->dev,
+ rmem->page_size,
+ &rmem->dma_arr[i],
+ GFP_KERNEL, nid);
if (!rmem->pg_arr[i])
return -ENOMEM;
@@ -3812,7 +3812,6 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
}
}
- /* Only the SW ring follows @nid, DMA memory is on the device's node */
if (rmem->vmem_size) {
*rmem->vmem = vzalloc_node(rmem->vmem_size, nid);
if (!(*rmem->vmem))
@@ -4208,10 +4207,11 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)
/* One pre-allocated DMA buffer to backup
* TX push operation
*/
- txr->tx_push = dma_alloc_coherent(&pdev->dev,
- bp->tx_push_size,
- &txr->tx_push_mapping,
- GFP_KERNEL);
+ txr->tx_push =
+ dma_alloc_coherent_node(&pdev->dev,
+ bp->tx_push_size,
+ &txr->tx_push_mapping,
+ GFP_KERNEL, nid);
if (!txr->tx_push)
return -ENOMEM;
@@ -4353,7 +4353,7 @@ static int bnxt_alloc_cp_sub_ring(struct bnxt *bp,
rmem->pg_arr = (void **)cpr->cp_desc_ring;
rmem->dma_arr = cpr->cp_desc_mapping;
rmem->flags = BNXT_RMEM_RING_PTE_FLAG;
- rc = bnxt_alloc_ring(bp, rmem, NUMA_NO_NODE);
+ rc = bnxt_alloc_ring(bp, rmem, nid);
if (rc) {
bnxt_free_ring(bp, rmem);
bnxt_free_cp_arrays(cpr);
@@ -4384,7 +4384,8 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
cpr->bnapi = bnapi;
ring = &cpr->cp_ring_struct;
- rc = bnxt_alloc_ring(bp, &ring->ring_mem, NUMA_NO_NODE);
+ nid = bnxt_cp_num_to_node(bp, i);
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem, nid);
if (rc)
return rc;
@@ -4406,7 +4407,6 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
tx = 1;
}
- nid = bnxt_cp_num_to_node(bp, i);
cpr->cp_ring_arr = kcalloc_node(cp_count, sizeof(*cpr),
GFP_KERNEL, nid);
if (!cpr->cp_ring_arr)
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another NUMA node
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
` (4 preceding siblings ...)
2026-11-18 2:39 ` [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node Jakub Kicinski
@ 2026-11-18 2:39 ` Jakub Kicinski
2026-11-18 12:50 ` Björn Töpel
5 siblings, 1 reply; 17+ messages in thread
From: Jakub Kicinski @ 2026-11-18 2:39 UTC (permalink / raw)
To: davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Rx queue memory is allocated on the node of the CPU servicing the queue
when the device is opened. On a running device it only follows IRQ
affinity changes if TPH is enabled, as a side effect of the queue
restart which applies the new TPH Steering Tag.
Schedule the affinity update work whenever the device supports the
queue API, not just when TPH Steering Tag changed. Queue restart places
all the memory it reallocates, other than the ring page tables, on
the page pool's node, so compare the page pool's nid to decide if
the queue has to move.
Since we need queue state of the currently allocated queue to compare
nid - check that the device is still up after errors.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 46 ++++++++++++++---------
2 files changed, 29 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 86fc2e8a5816..4b80b20288ba 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2656,7 +2656,7 @@ struct bnxt {
#define BNXT_RING_COAL_NOW_SP_EVENT 17
#define BNXT_FW_RESET_NOTIFY_SP_EVENT 18
#define BNXT_FW_EXCEPTION_SP_EVENT 19
-#define BNXT_TPH_UPDATE_SP_EVENT 20
+#define BNXT_IRQ_AFFINITY_SP_EVENT 20
#define BNXT_LINK_CFG_CHANGE_SP_EVENT 21
#define BNXT_THERMAL_THRESHOLD_SP_EVENT 22
#define BNXT_FW_ECHO_REQUEST_SP_EVENT 23
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 134b24842c4f..5f4a8db1f8b5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -11931,18 +11931,17 @@ static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify,
}
#endif
- if (!irq->bp->tph_mode)
+ /* Only Rx queues can be restarted to follow the IRQ */
+ if (!BNXT_SUPPORTS_QUEUE_API(irq->bp) ||
+ irq->ring_nr >= irq->bp->rx_nr_rings)
return;
- if (irq->ring_nr >= irq->bp->rx_nr_rings)
- return;
+ if (irq->bp->tph_mode &&
+ !pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
+ cpumask_first(mask), &tag))
+ WRITE_ONCE(irq->new_tag, tag);
- if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM,
- cpumask_first(mask), &tag))
- return;
-
- WRITE_ONCE(irq->new_tag, tag);
- bnxt_queue_sp_work(irq->bp, BNXT_TPH_UPDATE_SP_EVENT);
+ bnxt_queue_sp_work(irq->bp, BNXT_IRQ_AFFINITY_SP_EVENT);
}
static void bnxt_irq_affinity_release(struct kref *ref)
@@ -14663,7 +14662,10 @@ static void bnxt_rtnl_unlock_sp(struct bnxt *bp)
rtnl_unlock();
}
-static void bnxt_tph_update(struct bnxt *bp)
+/* Restart the Rx queues whose IRQ moved, if the move changed the TPH
+ * Steering Tag or the node the queue memory should be allocated on.
+ */
+static void bnxt_irq_affinity_update(struct bnxt *bp)
{
struct net_device *dev = bp->dev;
int i;
@@ -14673,6 +14675,8 @@ static void bnxt_tph_update(struct bnxt *bp)
goto unlock;
for (i = 0; i < bp->rx_nr_rings; i++) {
+ struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i];
+ bool new_tag = false, new_node = false;
struct bnxt_irq *irq;
int map_idx, err;
u16 tag;
@@ -14680,20 +14684,26 @@ static void bnxt_tph_update(struct bnxt *bp)
map_idx = bnxt_cp_num_to_irq_num(bp, i);
irq = &bp->irq_tbl[map_idx];
tag = READ_ONCE(irq->new_tag);
- if (irq->tag == tag)
+ if (irq->tag != tag &&
+ !pcie_tph_set_st_entry(bp->pdev, irq->msix_nr, tag))
+ new_tag = true;
+ if (rxr->page_pool->p.nid != bnxt_cp_num_to_node(bp, i))
+ new_node = true;
+ if (!new_tag && !new_node)
continue;
- if (pcie_tph_set_st_entry(bp->pdev, irq->msix_nr, tag))
- continue;
-
- err = netdev_rx_queue_restart(dev, irq->ring_nr);
+ err = netdev_rx_queue_restart(dev, i);
if (err) {
netdev_err(dev, "RX queue restart failed: err=%d\n",
err);
+ /* Failing to start a new queue may close the device */
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state))
+ break;
continue;
}
- irq->tag = tag;
+ if (new_tag)
+ irq->tag = tag;
}
unlock:
@@ -15137,8 +15147,8 @@ static void bnxt_sp_task(struct work_struct *work)
bnxt_devlink_health_fw_report(bp);
}
- if (test_and_clear_bit(BNXT_TPH_UPDATE_SP_EVENT, &bp->sp_event))
- bnxt_tph_update(bp);
+ if (test_and_clear_bit(BNXT_IRQ_AFFINITY_SP_EVENT, &bp->sp_event))
+ bnxt_irq_affinity_update(bp);
smp_mb__before_atomic();
clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit
2026-11-18 2:39 ` [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
@ 2026-11-18 12:27 ` Björn Töpel
0 siblings, 0 replies; 17+ messages in thread
From: Björn Töpel @ 2026-11-18 12:27 UTC (permalink / raw)
To: Jakub Kicinski, davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Jakub Kicinski <kuba@kernel.org> writes:
> Callers of bnxt_queue_sp_work() may store data for bnxt_sp_task()
> right before raising the event, e.g. the IRQ affinity notifier
> publishes the new TPH Steering Tag. set_bit() is not ordered, so on
> weakly ordered architectures the event bit may become visible first.
> If bnxt_sp_task() is already running it can consume the event and
> read the stale data. The queue_work() which follows only re-runs
> the task with the event bit already cleared, so the update is lost.
>
> Add smp_mb__before_atomic() before setting the bit, it pairs with
> the fully ordered test_and_clear_bit() in bnxt_sp_task().
>
> Reported by an LLM reviewer while working on subsequent patches.
> Very unlikely to matter in practice.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> index 6846a27bfef4..5ad2a3033505 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> @@ -394,6 +394,10 @@ static void __bnxt_queue_sp_work(struct bnxt *bp)
>
> static void bnxt_queue_sp_work(struct bnxt *bp, unsigned int event)
> {
> + /* Make the data the event carries visible before the event bit,
> + * pairs with test_and_clear_bit() in bnxt_sp_task().
> + */
> + smp_mb__before_atomic();
> set_bit(event, &bp->sp_event);
> __bnxt_queue_sp_work(bp);
There are direct callers of __bnxt_queue_sp_work(), that'll miss the
fence. Fixup those callers?
Björn
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool
2026-11-18 2:39 ` [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
@ 2026-11-18 12:32 ` Björn Töpel
2026-11-18 14:42 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Björn Töpel @ 2026-11-18 12:32 UTC (permalink / raw)
To: Jakub Kicinski, davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Jakub Kicinski <kuba@kernel.org> writes:
> The driver picks the NUMA node for each Rx ring's page pool with
> cpumask_local_spread() at open. IRQ affinity configured by the user
> is already recorded by the driver and persists across close/open
> cycles. Use it instead of cpumask_local_spread().
>
> The affinity notifier runs without any of the locks held by
> the allocation paths, and copying a multi-word cpumask is not atomic.
> Like for the TPH Steering Tag, have the notifier compute the node
> (of the first online CPU in the mask) and publish it with WRITE_ONCE().
>
> Queue restart (bnxt_queue_mem_alloc()) needs to resolve the NUMA
> node instead of copying the nid of the old page pool.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node
2026-11-18 2:39 ` [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
@ 2026-11-18 12:37 ` Björn Töpel
2026-11-18 14:49 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Björn Töpel @ 2026-11-18 12:37 UTC (permalink / raw)
To: Jakub Kicinski, davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Jakub Kicinski <kuba@kernel.org> writes:
> Allocate the SW rings, agg bitmap, TPA state, completion ring
> page arrays, completion sub-ring structs, SW stats and the SW USO
> header buffer on the NUMA node of the CPU which will service the queue.
> This lowers the cost of cache misses for NAPI fast path (and xmit, for Tx).
>
> Previous change already allocated the page pools on the right node.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node
2026-11-18 2:39 ` [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
@ 2026-11-18 12:38 ` Björn Töpel
2026-11-18 14:50 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Björn Töpel @ 2026-11-18 12:38 UTC (permalink / raw)
To: Jakub Kicinski, davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Jakub Kicinski <kuba@kernel.org> writes:
> Context memory (and the FW crash dump buffer which reuses the same
> helpers) is used by the device, once set up the host only touches it
> on the slow path (diagnostics, FW trace). Pass the device's node to
> bnxt_alloc_ring() rather than NUMA_NO_NODE to make it clear that this
> memory is not meant to follow any queue.
>
> Ring page tables are only read by the device, annotate their
> allocation so that they stay put once the ring pages start following
> the queue.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node
2026-11-18 2:39 ` [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node Jakub Kicinski
@ 2026-11-18 12:39 ` Björn Töpel
2026-11-18 14:51 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Björn Töpel @ 2026-11-18 12:39 UTC (permalink / raw)
To: Jakub Kicinski, davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Jakub Kicinski <kuba@kernel.org> writes:
> Descriptor rings, completion rings, NQs and the TX push buffer are
> accessed by the host in the fast path, and so is the SW state around
> them, which we already allocate on the node of the CPU expected to
> service the queue. The DMA memory itself comes from the device's node,
> though, as that's what dma_alloc_coherent() always uses.
>
> Use dma_alloc_coherent_node() for the ring pages and the TX push buffer.
> Pass the queue's node for completion rings and NQs as well, they used
> NUMA_NO_NODE so far since @nid only affected the SW rings.
>
> Ring page tables are only read by the device, keep them on the device's
> node.
>
> Note that queue restart only reallocates the Rx descriptor rings.
> Completion rings, NQs and Tx rings will follow IRQ affinity changes
> only on the next full reconfiguration.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another NUMA node
2026-11-18 2:39 ` [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another " Jakub Kicinski
@ 2026-11-18 12:50 ` Björn Töpel
0 siblings, 0 replies; 17+ messages in thread
From: Björn Töpel @ 2026-11-18 12:50 UTC (permalink / raw)
To: Jakub Kicinski, davem, michael.chan
Cc: netdev, edumazet, pabeni, andrew+netdev, horms, pavan.chebbi,
Jakub Kicinski
Jakub Kicinski <kuba@kernel.org> writes:
> Rx queue memory is allocated on the node of the CPU servicing the queue
> when the device is opened. On a running device it only follows IRQ
> affinity changes if TPH is enabled, as a side effect of the queue
> restart which applies the new TPH Steering Tag.
>
> Schedule the affinity update work whenever the device supports the
> queue API, not just when TPH Steering Tag changed. Queue restart places
> all the memory it reallocates, other than the ring page tables, on
> the page pool's node, so compare the page pool's nid to decide if
> the queue has to move.
>
> Since we need queue state of the currently allocated queue to compare
> nid - check that the device is still up after errors.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool
2026-11-18 2:39 ` [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
2026-11-18 12:32 ` Björn Töpel
@ 2026-11-18 14:42 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Joe Damato @ 2026-11-18 14:42 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, michael.chan, netdev, edumazet, pabeni, andrew+netdev,
horms, pavan.chebbi
On Tue, Nov 17, 2026 at 06:39:01PM -0800, Jakub Kicinski wrote:
> The driver picks the NUMA node for each Rx ring's page pool with
> cpumask_local_spread() at open. IRQ affinity configured by the user
> is already recorded by the driver and persists across close/open
> cycles. Use it instead of cpumask_local_spread().
>
> The affinity notifier runs without any of the locks held by
> the allocation paths, and copying a multi-word cpumask is not atomic.
> Like for the TPH Steering Tag, have the notifier compute the node
> (of the first online CPU in the mask) and publish it with WRITE_ONCE().
>
> Queue restart (bnxt_queue_mem_alloc()) needs to resolve the NUMA
> node instead of copying the nid of the old page pool.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.h | 5 +++
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 48 +++++++++++++++++++----
> 2 files changed, 46 insertions(+), 7 deletions(-)
Reviewed-by: Joe Damato <joe@dama.to>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node
2026-11-18 2:39 ` [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
2026-11-18 12:37 ` Björn Töpel
@ 2026-11-18 14:49 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Joe Damato @ 2026-11-18 14:49 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, michael.chan, netdev, edumazet, pabeni, andrew+netdev,
horms, pavan.chebbi
On Tue, Nov 17, 2026 at 06:39:02PM -0800, Jakub Kicinski wrote:
> Allocate the SW rings, agg bitmap, TPA state, completion ring
> page arrays, completion sub-ring structs, SW stats and the SW USO
> header buffer on the NUMA node of the CPU which will service the queue.
> This lowers the cost of cache misses for NAPI fast path (and xmit, for Tx).
>
> Previous change already allocated the page pools on the right node.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 87 ++++++++++++++---------
> 1 file changed, 52 insertions(+), 35 deletions(-)
Reviewed-by: Joe Damato <joe@dama.to>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node
2026-11-18 2:39 ` [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
2026-11-18 12:38 ` Björn Töpel
@ 2026-11-18 14:50 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Joe Damato @ 2026-11-18 14:50 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, michael.chan, netdev, edumazet, pabeni, andrew+netdev,
horms, pavan.chebbi
On Tue, Nov 17, 2026 at 06:39:03PM -0800, Jakub Kicinski wrote:
> Context memory (and the FW crash dump buffer which reuses the same
> helpers) is used by the device, once set up the host only touches it
> on the slow path (diagnostics, FW trace). Pass the device's node to
> bnxt_alloc_ring() rather than NUMA_NO_NODE to make it clear that this
> memory is not meant to follow any queue.
>
> Ring page tables are only read by the device, annotate their
> allocation so that they stay put once the ring pages start following
> the queue.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
Reviewed-by: Joe Damato <joe@dama.to>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node
2026-11-18 2:39 ` [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node Jakub Kicinski
2026-11-18 12:39 ` Björn Töpel
@ 2026-11-18 14:51 ` Joe Damato
1 sibling, 0 replies; 17+ messages in thread
From: Joe Damato @ 2026-11-18 14:51 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, michael.chan, netdev, edumazet, pabeni, andrew+netdev,
horms, pavan.chebbi
On Tue, Nov 17, 2026 at 06:39:04PM -0800, Jakub Kicinski wrote:
> Descriptor rings, completion rings, NQs and the TX push buffer are
> accessed by the host in the fast path, and so is the SW state around
> them, which we already allocate on the node of the CPU expected to
> service the queue. The DMA memory itself comes from the device's node,
> though, as that's what dma_alloc_coherent() always uses.
>
> Use dma_alloc_coherent_node() for the ring pages and the TX push buffer.
> Pass the queue's node for completion rings and NQs as well, they used
> NUMA_NO_NODE so far since @nid only affected the SW rings.
>
> Ring page tables are only read by the device, keep them on the device's
> node.
>
> Note that queue restart only reallocates the Rx descriptor rings.
> Completion rings, NQs and Tx rings will follow IRQ affinity changes
> only on the next full reconfiguration.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 24 +++++++++++------------
> 1 file changed, 12 insertions(+), 12 deletions(-)
Reviewed-by: Joe Damato <joe@dama.to>
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2026-11-18 14:51 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 2:38 [PATCH net-next 0/6] eth: bnxt_en: allocate memory on the right NUMA node Jakub Kicinski
2026-11-18 2:39 ` [PATCH net-next 1/6] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
2026-11-18 12:27 ` Björn Töpel
2026-11-18 2:39 ` [PATCH net-next 2/6] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
2026-11-18 12:32 ` Björn Töpel
2026-11-18 14:42 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 3/6] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
2026-11-18 12:37 ` Björn Töpel
2026-11-18 14:49 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 4/6] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
2026-11-18 12:38 ` Björn Töpel
2026-11-18 14:50 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 5/6] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node Jakub Kicinski
2026-11-18 12:39 ` Björn Töpel
2026-11-18 14:51 ` Joe Damato
2026-11-18 2:39 ` [PATCH net-next 6/6] eth: bnxt_en: restart Rx queues when their IRQ moves to another " Jakub Kicinski
2026-11-18 12:50 ` Björn Töpel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox