From: Jakub Kicinski <kuba@kernel.org>
To: davem@davemloft.net, michael.chan@broadcom.com
Cc: netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com,
andrew+netdev@lunn.ch, horms@kernel.org,
pavan.chebbi@broadcom.com, bjorn@kernel.org, joe@dama.to,
Jakub Kicinski <kuba@kernel.org>
Subject: [PATCH net-next v2 0/7] eth: bnxt_en: allocate memory on the right NUMA node
Date: Thu, 19 Nov 2026 19:52:07 -0800 [thread overview]
Message-ID: <20261120035214.369158-1-kuba@kernel.org> (raw)
Teach bnxt to allocate the memory touched by the fast path
on the NUMA node where NAPI is being serviced.
On modern systems the PCIe controller is part of the CPU.
On NUMA systems we get to pick which NUMA node the queue and NAPI
related memory will get allocated from. Allocating SW state on
the node where NAPI will run is just common sense.
For memory touched by both device and the CPU we need to pick
which side will pay the extra price (either device or CPU
will have to "reach" futher). This series allocates all state
and all descriptor rings on the NAPI CPU.
Testing a dual-IO-die AMD Venice CPU, where NIC is on node 1
and with NAPI+kperf pinned to node 0, this series gives:
placing page pools on the right node: -5.1% CPU load
placing page pools and completion rings: -8.9% CPU load
Moving SW state is in the noise in a tight-loop kperf banchmark
as the state fits in the cache, anyway.
This series was the motivation for cleaning up the DMA API
allocations with explicit NUMA node:
commit a0aa9ceca03f ("Merge tag 'dma_alloc_coherent_node' of git://git.kernel.org/pub/scm/linux/kernel/git/kuba/linux")
I held off reallocating the Rx completion ring on queue restart,
we need to fix the IRQ coalescing there first:
https://lore.kernel.org/all/20261120034645.364630-1-kuba@kernel.org/
v2:
- convert the sp_event producers which set the bit directly (Björn)
- correct the comment on ring_nid, it's not only written by the notifier
- allocate ring_nid with kmalloc_objs()
- add a prep patch with a local bp variable in the IRQ affinity notifier
- split the TPH condition in the IRQ affinity notifier (Joe)
- only compare the node of queues with explicitly set IRQ affinity
v1: https://lore.kernel.org/20261118023905.3050661-1-kuba@kernel.org
Jakub Kicinski (7):
eth: bnxt_en: order event data before setting the sp_event bit
eth: bnxt_en: follow explicitly set IRQ affinity when placing page
pool
eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node
eth: bnxt_en: explicitly keep device-only memory on the device's node
eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node
eth: bnxt_en: add a local bp variable in the IRQ affinity notifier
eth: bnxt_en: restart Rx queues when their IRQ moves to another NUMA
node
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 7 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 250 ++++++++++++++--------
2 files changed, 169 insertions(+), 88 deletions(-)
--
2.56.1
next reply other threads:[~2026-11-20 3:52 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 3:52 Jakub Kicinski [this message]
2026-11-20 3:52 ` [PATCH net-next v2 1/7] eth: bnxt_en: order event data before setting the sp_event bit Jakub Kicinski
2026-11-20 3:52 ` [PATCH net-next v2 2/7] eth: bnxt_en: follow explicitly set IRQ affinity when placing page pool Jakub Kicinski
2026-11-20 3:52 ` [PATCH net-next v2 3/7] eth: bnxt_en: allocate per-queue SW state on the queue's NUMA node Jakub Kicinski
2026-11-20 3:52 ` [PATCH net-next v2 4/7] eth: bnxt_en: explicitly keep device-only memory on the device's node Jakub Kicinski
2026-11-20 3:52 ` [PATCH net-next v2 5/7] eth: bnxt_en: allocate fast path DMA rings on the queue's NUMA node Jakub Kicinski
2026-11-20 3:52 ` [PATCH net-next v2 6/7] eth: bnxt_en: add a local bp variable in the IRQ affinity notifier Jakub Kicinski
2026-11-20 3:52 ` [PATCH net-next v2 7/7] eth: bnxt_en: restart Rx queues when their IRQ moves to another NUMA node Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120035214.369158-1-kuba@kernel.org \
--to=kuba@kernel.org \
--cc=andrew+netdev@lunn.ch \
--cc=bjorn@kernel.org \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=horms@kernel.org \
--cc=joe@dama.to \
--cc=michael.chan@broadcom.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=pavan.chebbi@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.