* [PATCH net-next v2 00/15] ibmveth: Add multi-queue RX support
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe
Hi,
Power11 PHYP firmware adds Virtual Ethernet multi-queue (MQ) RX for
the ibmveth device: multiple logical-LAN RX queues, per-queue buffer
posting, and completion delivery. Guest Linux did not use that
platform support; ibmveth still registered one RX queue even when
PHYP was MQ-capable.
This series adds the ibmveth MQ client. When PHYP advertises the
capability through H_ILLAN_ATTRIBUTES, the driver registers
multiple RX queues, receives on per-queue NAPI, and exposes queue
count through ethtool. Older firmware without the bit is unchanged.
Please apply to net-next.
Background
ibmveth today registers one logical LAN, one set of buffer pools, and
one NAPI context. PHYP MQ mode gives each RX queue its own handle:
buffers are posted with H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, subordinate
queues register through H_REG_LOGICAL_LAN_QUEUE, and traffic can
land on any active queue. Queue selection is firmware-defined; v1
does not program RSS or hash tables. The driver needs per-queue
pools, IRQs, and poll state to match.
Queue-aware hcalls are selected only when probe sets multi_queue
from H_ILLAN_ATTRIBUTES; legacy firmware keeps the original hcall
path unchanged through the entire series.
This splits the work so review follows the actual bring-up sequence:
1. Hypercall definitions and MQ data structures (patches 1-2)
2. Refactor open/close into helpers - RX, per-queue pools,
IRQ, TX, PHYP (3-9)
3. Turn on the MQ datapath at probe/open (10)
4. Per-queue RX/TX stats and sysfs pool readout (11-12)
5. Runtime RX queue resize via ethtool -L (13-14)
6. LPAR stability fix (15)
- Helper patches (3-9) reshape ibmveth_open()/close() into
queue-aware helpers. Runtime behaviour is unchanged through that
block: num_rx_queues stays 1 and multi_queue is false until patch 10.
- Patch 10 is the switch: probe sets multi_queue from firmware, raises
num_rx_queues, registers subordinates, and replenishes every active
queue.
- Patch 15 fixes poll hangs after aggressive ethtool -L cycling and
NAPI/close deadlocks on ip link down.
Testing
Tested on ppc64le PowerVM LPAR with MQ-capable firmware:
* Aggressive ethtool -L cycling (16/1/8/11/1/3/16/8/1) with ping
* MQ path: ethtool -L under iperf3 load, link down/up during traffic
* Legacy firmware (no MQ bit): full open/close/stress on the
refactored helper path to confirm single-queue behaviour is
unchanged
Changes in v2
v1 resubmit as 15 patches (Patchwork limit): same code and LPAR testing;
squashed split plus checkpatch fixes in patch 15 only.
v1: https://lore.kernel.org/r/cover.1782758799.git.mmc@linux.ibm.com
Patchwork: https://patchwork.kernel.org/project/netdevbpf/list/?series=1119106
Future work
* IRQ affinity hints for subordinate queue IRQs returned by PHYP
* Summed global no_buffer drop counter across all RX queues in MQ mode
Comments and suggestions on patch split, design, and testing are
welcome.
Mingming Cao <mmc@linux.ibm.com>
Mingming Cao (15):
ibmveth: Refactor RX resource allocation for MQ RX bring-up
ibmveth: Refactor buffer pool management for per-queue MQ RX
ibmveth: Refactor RX interrupt control for MQ RX queues
ibmveth: Refactor TX resource allocation in open/close paths
ibmveth: Add RX queue register/deregister helpers for MQ
ibmveth: Refactor open/close into MQ-ready resource pipeline
ibmveth: Add queue-aware RX buffer submit helper for MQ
ibmveth: Enable multi-queue RX receive path
ibmveth: Add per-queue RX statistics collection and reporting
ibmveth: Add per-queue TX statistics reporting
ibmveth: Expose per-queue buffer pool details via sysfs
ibmveth: Add helpers for incremental MQ RX queue resize
ibmveth: Implement incremental MQ RX queue resize
ibmveth: Wire ethtool set_channels to MQ RX queue resize
ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
drivers/net/ethernet/ibm/ibmveth.c | 2350 +++++++++++++++++++++++-----
drivers/net/ethernet/ibm/ibmveth.h | 25 +-
2 files changed, 2014 insertions(+), 361 deletions(-)
^ permalink raw reply
* [PATCH net-next v2 01/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
ibmveth_open() allocates the filter list and every RX queue inline.
That's already ~160 lines and would get ugly once we loop over
num_rx_queues, especially on error unwind.
Pull the RX bits into helpers:
ibmveth_alloc_filter_list() / ibmveth_free_filter_list()
— shared multicast filter list (one per adapter, not per queue)
ibmveth_alloc_rx_queues() / ibmveth_cleanup_rx_resources()
— per-queue buffer lists and RX rings, looping [0, num_rx_queues)
alloc_rx_queues() rolls back on failure so open() does not need nested
goto chains for every queue index.
This is the first of several helper-only patches (pools, IRQ, TX, PHYP
registration, open/close wiring, buffer submit) that reshape bring-up
ahead of MQ datapath commit later in the series.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 168 +++++++++++++++++++++++++++++
1 file changed, 168 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 8f9f927bff23..b8adc9935471 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -147,6 +147,174 @@ static unsigned int ibmveth_real_max_tx_queues(void)
return min(n_cpu, IBMVETH_MAX_QUEUES);
}
+/**
+ * ibmveth_alloc_filter_list - Allocate and map filter list
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+
+ adapter->filter_list_addr = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->filter_list_addr) {
+ netdev_err(netdev, "unable to allocate filter pages\n");
+ return -ENOMEM;
+ }
+
+ adapter->filter_list_dma = dma_map_single(dev,
+ adapter->filter_list_addr,
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->filter_list_dma)) {
+ netdev_err(netdev, "unable to map filter list pages\n");
+ free_page((unsigned long)adapter->filter_list_addr);
+ adapter->filter_list_addr = NULL;
+ return -ENOMEM;
+ }
+
+ netdev_dbg(netdev, "filter list @ 0x%p (DMA: 0x%llx)\n",
+ adapter->filter_list_addr,
+ (unsigned long long)adapter->filter_list_dma);
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_filter_list - Free filter list resources
+ * @adapter: ibmveth adapter structure
+ */
+static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+
+ if (adapter->filter_list_dma) {
+ dma_unmap_single(dev, adapter->filter_list_dma, 4096,
+ DMA_BIDIRECTIONAL);
+ adapter->filter_list_dma = 0;
+ }
+
+ if (adapter->filter_list_addr) {
+ free_page((unsigned long)adapter->filter_list_addr);
+ adapter->filter_list_addr = NULL;
+ }
+}
+
+/**
+ * ibmveth_alloc_rx_queues - Allocate per-queue RX resources
+ * @adapter: ibmveth adapter structure
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused
+ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ adapter->buffer_list_addr[i] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[i]) {
+ netdev_err(netdev, "unable to allocate buffer list for queue %d\n", i);
+ goto err_cleanup;
+ }
+
+ adapter->rx_queue[i].queue_len =
+ sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+ adapter->rx_queue[i].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[i].queue_len,
+ &adapter->rx_queue[i].queue_dma,
+ GFP_KERNEL);
+ if (!adapter->rx_queue[i].queue_addr) {
+ netdev_err(netdev, "unable to allocate RX queue for queue %d\n", i);
+ goto err_cleanup;
+ }
+
+ adapter->buffer_list_dma[i] = dma_map_single(dev,
+ adapter->buffer_list_addr[i],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
+ netdev_err(netdev, "unable to map buffer list for queue %d\n", i);
+ adapter->buffer_list_dma[i] = 0;
+ goto err_cleanup;
+ }
+
+ adapter->rx_queue[i].index = 0;
+ adapter->rx_queue[i].num_slots = rxq_entries;
+ adapter->rx_queue[i].toggle = 1;
+
+ netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
+ i, adapter->buffer_list_addr[i],
+ (unsigned long long)adapter->buffer_list_dma[i],
+ adapter->rx_queue[i].queue_addr,
+ (unsigned long long)adapter->rx_queue[i].queue_dma,
+ (unsigned long long)rxq_entries);
+ }
+
+ netdev_dbg(netdev, "allocated %d RX queue(s) with %d entries each\n",
+ adapter->num_rx_queues, rxq_entries);
+
+ return 0;
+
+err_cleanup:
+ /* Clean up previously allocated queues */
+ for (; i >= 0; i--) {
+ if (adapter->buffer_list_dma[i]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[i],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[i] = 0;
+ }
+ if (adapter->rx_queue[i].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+ adapter->rx_queue[i].queue_addr,
+ adapter->rx_queue[i].queue_dma);
+ adapter->rx_queue[i].queue_addr = NULL;
+ }
+ if (adapter->buffer_list_addr[i]) {
+ free_page((unsigned long)adapter->buffer_list_addr[i]);
+ adapter->buffer_list_addr[i] = NULL;
+ }
+ }
+
+ return -ENOMEM;
+}
+
+/**
+ * ibmveth_cleanup_rx_resources - Free all RX queue resources
+ * @adapter: ibmveth adapter structure
+ */
+static void __maybe_unused ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+ int i;
+
+ netdev_dbg(adapter->netdev, "cleaning up %d RX queue(s)\n",
+ adapter->num_rx_queues);
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->buffer_list_dma[i]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[i],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[i] = 0;
+ }
+
+ if (adapter->rx_queue[i].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+ adapter->rx_queue[i].queue_addr,
+ adapter->rx_queue[i].queue_dma);
+ adapter->rx_queue[i].queue_addr = NULL;
+ }
+
+ if (adapter->buffer_list_addr[i]) {
+ free_page((unsigned long)adapter->buffer_list_addr[i]);
+ adapter->buffer_list_addr[i] = NULL;
+ }
+ }
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 02/15] ibmveth: Refactor buffer pool management for per-queue MQ RX
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
This is the key memory-model change for MQ RX.
Legacy ibmveth uses five adapter-level RX buffer pools (512 B
through 64 KiB slots). pool_active[] enables the standard-MTU pools by
default; larger pools activate when MTU requires them. With single-queue
RX that set is shared on one completion path.
MQ requires the same pool model per queue: buffers post with
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against a queue handle and completions
return on that queue. Sharing pools across queues would mix ownership and
break queue-local replenish/drain/teardown.
Refactor around queue-local pools with static geometry (still defined at
probe on queue 0, copied to queues 1..N at alloc time):
rx_buff_pool[queue][pool]
ibmveth_alloc_queue_buffer_pools()
ibmveth_free_queue_buffer_pools()
ibmveth_alloc_buffer_pools() / ibmveth_free_buffer_pools()
Queue 0 remains the template for pool geometry (size, buff_size,
threshold, active). For queues 1..N we copy metadata from queue 0, then
allocate actual backing arrays/skbs per queue.
At the default 1500-byte MTU, pool 4 (64 KiB buffers) is not needed and
costs guest memory when allocated per queue in MQ mode. Clear
pool_active[4] so open() skips it; ibmveth_change_mtu() still enables
larger pools when MTU warrants jumbo frames.
Error handling is also made queue-safe:
- if allocation fails in one pool, unwind only what was allocated for
that queue, then unwind prior queues in the caller
- free paths release pools based on real allocations
(free_map/dma_addr/skbuff), not only pool->active
That allocation-based free check is intentional: later resize and failure
paths can leave memory allocated even when active was already cleared.
Freeing by allocation state avoids leaks and double-free corner cases.
This split keeps the per-queue pool design isolated and reviewable ahead
of the MQ datapath enable commit later in the series.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 127 +++++++++++++++++++++++++++++
drivers/net/ethernet/ibm/ibmveth.h | 2 +-
2 files changed, 128 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b8adc9935471..95068fb20dba 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -611,6 +611,133 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
}
}
+/**
+ * ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Allocates all active buffer pools for the specified queue.
+ * Pool metadata must be initialized before calling this function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int ibmveth_alloc_queue_buffer_pools(struct ibmveth_adapter *adapter,
+ int queue)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ if (!adapter->rx_buff_pool[queue][i].active)
+ continue;
+
+ if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[queue][i])) {
+ netdev_err(netdev,
+ "unable to allocate buffer pool %d for queue %d (size=%u, count=%u)\n",
+ i, queue,
+ adapter->rx_buff_pool[queue][i].buff_size,
+ adapter->rx_buff_pool[queue][i].size);
+ adapter->rx_buff_pool[queue][i].active = 0;
+
+ /* Free pools allocated so far for this queue */
+ while (--i >= 0) {
+ if (adapter->rx_buff_pool[queue][i].active)
+ ibmveth_free_buffer_pool(adapter,
+ &adapter->rx_buff_pool[queue][i]);
+ }
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Frees all active buffer pools for the specified queue.
+ */
+static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
+ int queue)
+{
+ int i;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[queue][i];
+
+ /* Free pool if it has allocated memory, regardless of active flag.
+ * Pools may have memory allocated but not marked active during
+ * queue scale-up, so we must check for actual allocations.
+ */
+ if (pool->free_map || pool->dma_addr || pool->skbuff)
+ ibmveth_free_buffer_pool(adapter, pool);
+ }
+}
+
+/**
+ * ibmveth_alloc_buffer_pools - Allocate buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Initializes pool metadata for queues 1-N from queue 0 settings,
+ * then allocates buffer pools for all queues using the helper function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i, q, rc;
+
+ /* Initialize pool metadata for queues 1-15 from queue 0 settings */
+ for (q = 1; q < adapter->num_rx_queues; q++) {
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *src = &adapter->rx_buff_pool[0][i];
+ struct ibmveth_buff_pool *dst = &adapter->rx_buff_pool[q][i];
+
+ dst->size = src->size;
+ dst->index = src->index;
+ dst->buff_size = src->buff_size;
+ dst->threshold = src->threshold;
+ dst->active = src->active;
+ }
+ }
+
+ /* Allocate actual buffers for all queues */
+ for (q = 0; q < adapter->num_rx_queues; q++) {
+ rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
+ if (rc) {
+ /* Free pools for all previous queues */
+ while (--q >= 0)
+ ibmveth_free_queue_buffer_pools(adapter, q);
+ return rc;
+ }
+ }
+
+ netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
+ adapter->num_rx_queues);
+ return 0;
+}
+
+/**
+ * ibmveth_free_buffer_pools - Free buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees buffer pools for all queues using the helper function.
+ */
+static void __maybe_unused ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
+{
+ int q;
+
+ /* Free buffer pools for all queues */
+ for (q = 0; q < adapter->num_rx_queues; q++)
+ ibmveth_free_queue_buffer_pools(adapter, q);
+
+ netdev_dbg(adapter->netdev, "freed buffer pools for %d queue(s)\n",
+ adapter->num_rx_queues);
+}
+
/**
* ibmveth_remove_buffer_from_pool - remove a buffer from a pool
* @adapter: adapter instance
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index f0dffe42e8fe..d2ceeccd5fbd 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -286,7 +286,7 @@ static inline long h_illan_attributes(unsigned long unit_address,
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
static int pool_count[] = { 256, 512, 256, 256, 256 };
static int pool_count_cmo[] = { 256, 512, 256, 256, 64 };
-static int pool_active[] = { 1, 1, 0, 0, 1};
+static int pool_active[] = { 1, 1, 0, 0, 0};
#define IBM_VETH_INVALID_MAP ((u16)0xffff)
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 04/15] ibmveth: Refactor TX resource allocation in open/close paths
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Same story as the RX refactor: pull TX LTB alloc out of open/close.
ibmveth_alloc_tx_resources() / ibmveth_free_tx_resources() walk
real_num_tx_queues so ethtool TX channel changes keep working. Hooked
into open/close in the next patch.
No MQ RX behaviour change — TX was already multi-queue capable via
ethtool -L. This patch only tidies the open/close path ahead of the
RX helper wiring in the next patch.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 43 ++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b5ae979c1f82..63b0184c622a 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1038,6 +1038,49 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
return 0;
}
+/**
+ * ibmveth_alloc_tx_resources - Allocate TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Allocates TX Long Term Buffers (LTBs) for all TX queues.
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int __maybe_unused
+ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++) {
+ if (ibmveth_allocate_tx_ltb(adapter, i))
+ goto err_free_ltbs;
+ }
+
+ return 0;
+
+err_free_ltbs:
+ while (--i >= 0)
+ ibmveth_free_tx_ltb(adapter, i);
+ return -ENOMEM;
+}
+
+/**
+ * ibmveth_free_tx_resources - Free TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees TX Long Term Buffers (LTBs) for all TX queues.
+ */
+static void __maybe_unused
+ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++)
+ ibmveth_free_tx_ltb(adapter, i);
+}
+
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
union ibmveth_buf_desc rxq_desc,
u64 mac_address)
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 03/15] ibmveth: Refactor RX interrupt control for MQ RX queues
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Queue 0 and subordinate RX queues use different interrupt control
interfaces in PHYP:
- queue 0: h_vio_signal() after h_register_logical_lan()
- queue N: H_VIOCTL against the queue handle/hwirq mapping
The current code is single-queue oriented and cannot safely scale to
multiple RX queues in poll completion and open/close IRQ setup.
Introduce queue-indexed interrupt helpers:
ibmveth_enable_irq(adapter, queue_index)
ibmveth_disable_irq(adapter, queue_index)
ibmveth_setup_rx_interrupts()
ibmveth_cleanup_rx_interrupts()
These helpers centralize queue0-vs-subordinate dispatch and make IRQ
lifecycle symmetric across open/close and future resize paths.
request_irq() is wired with &adapter->napi[i] as dev_id per queue, so
interrupt ownership follows the NAPI instance that services that RX
queue.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 160 +++++++++++++++++++++++++++++
1 file changed, 160 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 95068fb20dba..b5ae979c1f82 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -315,6 +315,166 @@ static void __maybe_unused ibmveth_cleanup_rx_resources(struct ibmveth_adapter *
}
}
+/**
+ * ibmveth_toggle_irq - Common helper to enable/disable queue interrupts
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ * @enable: true to enable, false to disable
+ *
+ * For queue 0 (primary), uses h_vio_signal() as it's registered via
+ * h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL
+ * with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control.
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_toggle_irq(struct ibmveth_adapter *adapter, int queue_index, bool enable)
+{
+ unsigned long rc;
+ unsigned long irq = adapter->queue_irq[queue_index];
+ const char *action = enable ? "enable" : "disable";
+
+ if (queue_index == 0) {
+ /* Primary queue: use h_vio_signal() */
+ rc = h_vio_signal(adapter->vdev->unit_address,
+ enable ? VIO_IRQ_ENABLE : VIO_IRQ_DISABLE);
+ } else {
+ /* Subordinate queues: use H_VIOCTL with hardware IRQ */
+ struct irq_data *irq_data = irq_get_irq_data(irq);
+ irq_hw_number_t hwirq;
+ u64 vioctl_cmd = enable ? H_ENABLE_VIO_INTERRUPT : H_DISABLE_VIO_INTERRUPT;
+
+ if (!irq_data) {
+ netdev_err(adapter->netdev,
+ "Failed to get IRQ data for queue %d (virq=%lu)\n",
+ queue_index, irq);
+ return -EINVAL;
+ }
+
+ hwirq = irqd_to_hwirq(irq_data);
+ rc = plpar_hcall_norets(H_VIOCTL,
+ adapter->vdev->unit_address,
+ vioctl_cmd,
+ hwirq, 0, 0);
+
+ if (rc == H_PARAMETER) {
+ /* H_PARAMETER is non-fatal when IRQ is already in the requested state. */
+ netdev_warn_once(adapter->netdev,
+ "H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n",
+ action, queue_index, hwirq);
+ return 0;
+ }
+ }
+
+ if (rc)
+ netdev_err(adapter->netdev,
+ "Failed to %s IRQ for queue %d, rc=%ld\n",
+ action, queue_index, rc);
+ return rc;
+}
+
+/**
+ * ibmveth_disable_irq - Disable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_disable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+ return ibmveth_toggle_irq(adapter, queue_index, false);
+}
+
+/**
+ * ibmveth_enable_irq - Enable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+ return ibmveth_toggle_irq(adapter, queue_index, true);
+}
+
+/**
+ * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
+ * @adapter: ibmveth adapter structure
+ *
+ * Registers interrupt handlers for all RX queues and enables NAPI polling.
+ * On error, cleans up any successfully registered IRQs before returning.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused
+ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i, rc;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (!adapter->queue_irq[i]) {
+ netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
+ rc = -EINVAL;
+ goto err_free_irqs;
+ }
+
+ rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt,
+ 0, netdev->name, &adapter->napi[i]);
+ if (rc) {
+ netdev_err(netdev,
+ "request_irq() failed for irq 0x%x queue %d: %d\n",
+ adapter->queue_irq[i], i, rc);
+ goto err_free_irqs;
+ }
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ napi_enable(&adapter->napi[i]);
+
+ return 0;
+
+err_free_irqs:
+ while (--i >= 0)
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ return rc;
+}
+
+/**
+ * ibmveth_cleanup_rx_interrupts - Disable NAPI and free IRQs
+ * @adapter: ibmveth adapter structure
+ *
+ * Disables NAPI polling and frees interrupt handlers for all RX queues.
+ */
+static void
+ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i])
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ }
+
+ /* Dispose IRQ mappings for subordinate queues (1-15).
+ * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
+ */
+ for (i = 1; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i]) {
+ irq_dispose_mapping(adapter->queue_irq[i]);
+ adapter->queue_irq[i] = 0;
+ }
+ }
+
+ /* Clear queue 0 IRQ number */
+ adapter->queue_irq[0] = 0;
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 05/15] ibmveth: Add RX queue register/deregister helpers for MQ
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
MQ RX replaces a single adapter-level register/free pair with a mixed
PHYP model: queue 0 via h_register_logical_lan*(), subordinates via
H_REG_LOGICAL_LAN_QUEUE. Subordinate registration returns queue handles
and hardware IRQ numbers that must be mapped to Linux virqs and unwound
on failure.
Add queue lifecycle helpers to isolate that control plane:
ibmveth_register_logical_lan_queue()
ibmveth_register_single_rx_queue()
ibmveth_deregister_single_rx_queue()
ibmveth_register_rx_queues()
ibmveth_free_all_queues()
ibmveth_dispose_subordinate_irq_mappings()
These helpers are called only when multi_queue is enabled (patch 11).
Until then open/close still use the legacy register and buffer hcall
path; legacy firmware is unchanged.
When multi_queue is enabled, queue 0 uses
h_register_logical_lan_with_handle() so all queues share the per-queue
buffer hcall path. register_rx_queues() registers with PHYP only;
interrupt delivery is enabled later from ibmveth_setup_rx_interrupts()
after request_irq(). Partial registration failure disposes subordinate virq
mappings before ibmveth_free_all_queues() clears handles;
free_all_queues() clears queue handles only — IRQ mappings are released
by dispose_subordinate_irq_mappings() or cleanup_rx_interrupts().
This commit also centralizes hcall accounting on the register/free paths.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 337 ++++++++++++++++++++++++++++-
1 file changed, 332 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 63b0184c622a..7fc11a4e1f61 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -21,6 +21,8 @@
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
#include <linux/mm.h>
#include <linux/pm.h>
#include <linux/ethtool.h>
@@ -399,6 +401,28 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
return ibmveth_toggle_irq(adapter, queue_index, true);
}
+/**
+ * ibmveth_dispose_subordinate_irq_mappings - Drop virq mappings for queues 1..N
+ * @adapter: ibmveth adapter structure
+ *
+ * Subordinate queues get mappings from irq_create_mapping() during PHYP
+ * registration. Queue 0 uses netdev->irq from device tree and is left alone.
+ * Call after free_irq() when handlers were installed, or alone when open
+ * fails during register_rx_queues() before request_irq().
+ */
+static void
+ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
+{
+ int i;
+
+ for (i = 1; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i]) {
+ irq_dispose_mapping(adapter->queue_irq[i]);
+ adapter->queue_irq[i] = 0;
+ }
+ }
+}
+
/**
* ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
* @adapter: ibmveth adapter structure
@@ -1082,8 +1106,8 @@ ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
}
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
- union ibmveth_buf_desc rxq_desc,
- u64 mac_address)
+ union ibmveth_buf_desc rxq_desc,
+ u64 mac_address)
{
int rc, try_again = 1;
@@ -1093,13 +1117,29 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
* try again, but only once.
*/
retry:
- rc = h_register_logical_lan(adapter->vdev->unit_address,
- adapter->buffer_list_dma[0], rxq_desc.desc,
- adapter->filter_list_dma, mac_address);
+ /* In multi-queue mode, obtain a queue handle for queue 0 so all RX
+ * queues can use the same per-queue buffer hypercalls.
+ */
+ if (adapter->multi_queue) {
+ rc = h_register_logical_lan_with_handle(adapter->vdev->unit_address,
+ adapter->buffer_list_dma[0],
+ rxq_desc.desc,
+ adapter->filter_list_dma,
+ mac_address,
+ &adapter->queue_handle[0]);
+ } else {
+ rc = h_register_logical_lan(adapter->vdev->unit_address,
+ adapter->buffer_list_dma[0],
+ rxq_desc.desc,
+ adapter->filter_list_dma,
+ mac_address);
+ }
+ adapter->hcall_stats.reg_lan++;
if (rc != H_SUCCESS && try_again) {
do {
rc = h_free_logical_lan(adapter->vdev->unit_address);
+ adapter->hcall_stats.free_lan++;
} while (H_IS_LONG_BUSY(rc) || (rc == H_BUSY));
try_again = 0;
@@ -1136,6 +1176,293 @@ static void __maybe_unused ibmveth_free_rx_qstats(struct ibmveth_adapter *adapte
adapter->rx_qstats = NULL;
}
+/**
+ * ibmveth_register_logical_lan_queue - Register subordinate queue with hypervisor
+ * @adapter: ibmveth adapter structure
+ * @rxq_desc: Receive queue descriptor
+ * @queue_index: RX queue index (1..N for subordinate queues)
+ *
+ * Registers a subordinate receive queue using H_REG_LOGICAL_LAN_QUEUE.
+ * On success, stores the queue handle and virtual IRQ in the adapter.
+ * Retries once if registration fails (handles kexec case). If IRQ mapping
+ * fails after a successful hypervisor registration, the queue is freed
+ * before returning.
+ *
+ * Return: H_SUCCESS on success, negative errno on IRQ mapping failure,
+ * hypervisor error code otherwise
+ */
+static int
+ibmveth_register_logical_lan_queue(struct ibmveth_adapter *adapter,
+ union ibmveth_buf_desc rxq_desc,
+ int queue_index)
+{
+ unsigned long handle, hwirq;
+ unsigned int virq;
+ long lpar_rc;
+ int try_again = 1;
+
+retry:
+ netdev_dbg(adapter->netdev,
+ "Attempting to register queue %d: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+ queue_index, adapter->vdev->unit_address,
+ (unsigned long long)adapter->buffer_list_dma[queue_index],
+ (unsigned long long)rxq_desc.desc);
+
+ lpar_rc = h_reg_logical_lan_queue(adapter->vdev->unit_address,
+ adapter->buffer_list_dma[queue_index],
+ rxq_desc.desc, &handle, &hwirq);
+ adapter->hcall_stats.reg_lan_queue++;
+
+ if (lpar_rc == H_SUCCESS) {
+ virq = irq_create_mapping(NULL, hwirq);
+ if (!virq) {
+ unsigned long free_rc;
+
+ netdev_err(adapter->netdev,
+ "Failed to map IRQ for queue %d (hwirq=%lu)\n",
+ queue_index, hwirq);
+ do {
+ free_rc = h_free_logical_lan_queue(adapter->vdev->unit_address,
+ handle);
+ } while (H_IS_LONG_BUSY(free_rc) || (free_rc == H_BUSY));
+ adapter->hcall_stats.free_lan_queue++;
+ if (free_rc != H_SUCCESS)
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan_queue failed for queue %d after IRQ map failure: rc=0x%lx\n",
+ queue_index, free_rc);
+ return -EINVAL;
+ }
+
+ adapter->queue_handle[queue_index] = handle;
+ adapter->queue_irq[queue_index] = virq;
+
+ netdev_dbg(adapter->netdev,
+ "queue %d registered: handle=0x%llx irq=%u\n",
+ queue_index, adapter->queue_handle[queue_index],
+ adapter->queue_irq[queue_index]);
+ return H_SUCCESS;
+ }
+
+ if (lpar_rc == H_FUNCTION) {
+ if (adapter->multi_queue) {
+ netdev_info(adapter->netdev,
+ "Multi queue mode not supported by firmware, falling back to single queue\n");
+ adapter->multi_queue = 0;
+ } else {
+ netdev_err(adapter->netdev,
+ "Unexpected H_FUNCTION for queue %d registration (MQ mode already disabled)\n",
+ queue_index);
+ }
+ return lpar_rc;
+ }
+
+ if (try_again) {
+ try_again = 0;
+ goto retry;
+ }
+
+ netdev_err(adapter->netdev,
+ "h_reg_logical_lan_queue failed with %ld after retry\n",
+ lpar_rc);
+ netdev_err(adapter->netdev,
+ "queue %d params: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+ queue_index, adapter->vdev->unit_address,
+ (unsigned long long)adapter->buffer_list_dma[queue_index],
+ (unsigned long long)rxq_desc.desc);
+
+ return lpar_rc;
+}
+
+/**
+ * ibmveth_register_single_rx_queue - Register one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to register (1..N)
+ * @mac_address: MAC address (unused; reserved for API symmetry)
+ *
+ * Builds the queue descriptor and registers with the hypervisor via
+ * ibmveth_register_logical_lan_queue().
+ *
+ * Return: 0 on success, -EINVAL if @queue_idx is invalid, -EIO on failure
+ */
+static int
+ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx, u64 mac_address)
+{
+ struct net_device *netdev = adapter->netdev;
+ union ibmveth_buf_desc rxq_desc;
+ long lpar_rc;
+
+ (void)mac_address;
+
+ if (WARN_ON(queue_idx < 1 || queue_idx >= IBMVETH_MAX_RX_QUEUES))
+ return -EINVAL;
+
+ rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+ adapter->rx_queue[queue_idx].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[queue_idx].queue_dma;
+
+ lpar_rc = ibmveth_register_logical_lan_queue(adapter, rxq_desc,
+ queue_idx);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev, "Failed to register queue %d: rc=0x%lx\n",
+ queue_idx, lpar_rc);
+ return -EIO;
+ }
+
+ netdev_dbg(netdev, "Registered queue %d with handle 0x%llx\n",
+ queue_idx, adapter->queue_handle[queue_idx]);
+
+ return 0;
+}
+
+/**
+ * ibmveth_deregister_single_rx_queue - Deregister one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to deregister (1..N)
+ *
+ * Deregisters a single queue via H_FREE_LOGICAL_LAN_QUEUE and disposes
+ * the IRQ mapping for subordinate queues. Queue 0 is freed only through
+ * ibmveth_free_all_queues() (H_FREE_LOGICAL_LAN).
+ */
+static void __maybe_unused
+ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ unsigned long lpar_rc;
+
+ if (!adapter->queue_handle[queue_idx])
+ return;
+
+ do {
+ lpar_rc = h_free_logical_lan_queue(adapter->vdev->unit_address,
+ adapter->queue_handle[queue_idx]);
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+ adapter->hcall_stats.free_lan_queue++;
+
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan_queue failed for queue %d: rc=0x%lx\n",
+ queue_idx, lpar_rc);
+ }
+
+ adapter->queue_handle[queue_idx] = 0;
+
+ if (queue_idx > 0 && adapter->queue_irq[queue_idx]) {
+ irq_dispose_mapping(adapter->queue_irq[queue_idx]);
+ adapter->queue_irq[queue_idx] = 0;
+ }
+
+ netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
+}
+
+/**
+ * ibmveth_free_all_queues - Free all RX queues at once
+ * @adapter: ibmveth adapter structure
+ *
+ * Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall.
+ * Used during interface close and registration error cleanup.
+ *
+ * Clears queue handles only; queue_irq[] is released by
+ * ibmveth_cleanup_rx_interrupts() on close, or by
+ * ibmveth_dispose_subordinate_irq_mappings() on partial register failure.
+ */
+static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
+{
+ unsigned long lpar_rc;
+ int i;
+
+ netdev_dbg(adapter->netdev, "freeing all RX queues at once\n");
+
+ do {
+ lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
+ adapter->hcall_stats.free_lan++;
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan failed: %ld\n", lpar_rc);
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ adapter->queue_handle[i] = 0;
+}
+
+/**
+ * ibmveth_register_rx_queues - Register RX queues with hypervisor
+ * @adapter: ibmveth adapter structure
+ * @mac_address: MAC address for device registration
+ *
+ * Registers queue 0 via ibmveth_register_logical_lan(), then subordinate
+ * queues 1..N when multi-queue mode is enabled.
+ *
+ * Return: 0 on success, -ENONET if queue 0 registration fails, -EIO on
+ * subordinate queue registration failure
+ */
+static int
+ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
+{
+ struct net_device *netdev = adapter->netdev;
+ union ibmveth_buf_desc rxq_desc;
+ unsigned long lpar_rc;
+ int i, rc;
+
+ rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+ adapter->rx_queue[0].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
+ adapter->queue_irq[0] = netdev->irq;
+
+ rc = ibmveth_disable_irq(adapter, 0);
+ if (rc != H_SUCCESS)
+ netdev_dbg(netdev,
+ "Failed to disable IRQ for queue 0 before registration, rc=%d\n",
+ rc);
+
+ lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev, "h_register_logical_lan failed: %ld\n", lpar_rc);
+ netdev_err(netdev,
+ "buffer TCE:0x%llx filter TCE:0x%llx rxq desc:0x%llx MAC:0x%llx\n",
+ adapter->buffer_list_dma[0],
+ adapter->filter_list_dma,
+ rxq_desc.desc, mac_address);
+ return -ENONET;
+ }
+
+ if (adapter->num_rx_queues == 1 || !adapter->multi_queue) {
+ netdev_dbg(netdev,
+ "registered 1 RX queue with hypervisor (single-queue mode)\n");
+ return 0;
+ }
+
+ netdev_dbg(netdev, "Registering %d subordinate queues (1-%d)\n",
+ adapter->num_rx_queues - 1, adapter->num_rx_queues - 1);
+
+ for (i = 1; i < adapter->num_rx_queues; i++) {
+ rc = ibmveth_register_single_rx_queue(adapter, i, mac_address);
+ if (rc) {
+ if (!adapter->queue_handle[i] || !adapter->queue_irq[i]) {
+ netdev_err(netdev,
+ "Invalid hypervisor return for queue %d: handle=0x%llx irq=%u\n",
+ i, adapter->queue_handle[i],
+ adapter->queue_irq[i]);
+ }
+ goto err_unregister;
+ }
+ }
+
+ netdev_dbg(netdev,
+ "registered %d RX queues with hypervisor (multi-queue mode)\n",
+ adapter->num_rx_queues);
+
+ return 0;
+
+err_unregister:
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+ ibmveth_free_all_queues(adapter);
+ return rc;
+}
+
static int ibmveth_open(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 06/15] ibmveth: Refactor open/close into MQ-ready resource pipeline
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Patches 4-8 added alloc/free helpers for RX rings, buffer pools, IRQs,
TX LTBs, and PHYP registration, but open() and close() still duplicated
most of that logic inline. This patch wires the helpers in and makes
open/close the readable bring-up/teardown sequence MQ will extend.
ibmveth_open() runs:
1. ibmveth_alloc_rx_qstats()
2. ibmveth_alloc_filter_list()
3. ibmveth_alloc_rx_queues() - buffer lists + RX rings [0, N)
4. ibmveth_alloc_buffer_pools() - guest RX memory before PHYP
5. ibmveth_register_rx_queues() - PHYP registration (no IRQ enable)
6. netif_set_real_num_rx_queues()
7. ibmveth_setup_rx_interrupts() - request_irq, PHYP enable on MQ
8. initial replenish - queue 0 only today
9. ibmveth_alloc_tx_resources()
Each step has a matching out_* label on failure so unwind walks back
through free_all_queues(), cleanup_rx_resources(), and the other helpers
instead of open() carrying its own DMA unmap/free_page/goto maze (~200
lines removed).
ibmveth_close() mirrors that in reverse: stop TX, disable hypervisor IRQs
per queue, free TX LTBs, tear down NAPI/IRQ handlers, drop buffer pools,
H_FREE_LOGICAL_LAN via ibmveth_free_all_queues(), then free
RX/filter/qstats memory.
request_irq() now passes &napi[i] as dev_id on every queue so the
interrupt and poll paths can derive the queue index from the napi pointer
(napi - adapter->napi).
Drop __maybe_unused from the helpers added in patches 4-8 — they are
called from open/close from this patch onward.
Runtime still single-queue until the MQ enable commit later in the series;
replenish still kicks off via ibmveth_interrupt() on queue 0 as before.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 351 +++++++++++------------------
1 file changed, 137 insertions(+), 214 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 7fc11a4e1f61..fa2d4777ffc7 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -155,7 +155,7 @@ static unsigned int ibmveth_real_max_tx_queues(void)
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
+static int ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
struct net_device *netdev = adapter->netdev;
@@ -187,7 +187,7 @@ static int __maybe_unused ibmveth_alloc_filter_list(struct ibmveth_adapter *adap
* ibmveth_free_filter_list - Free filter list resources
* @adapter: ibmveth adapter structure
*/
-static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
+static void ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
@@ -203,6 +203,33 @@ static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adap
}
}
+/**
+ * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
+ sizeof(struct ibmveth_rx_queue_stats),
+ GFP_KERNEL);
+ if (!adapter->rx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_rx_qstats - Free per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->rx_qstats);
+ adapter->rx_qstats = NULL;
+}
+
/**
* ibmveth_alloc_rx_queues - Allocate per-queue RX resources
* @adapter: ibmveth adapter structure
@@ -210,7 +237,7 @@ static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adap
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused
+static int
ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
{
struct device *dev = &adapter->vdev->dev;
@@ -288,7 +315,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
* ibmveth_cleanup_rx_resources - Free all RX queue resources
* @adapter: ibmveth adapter structure
*/
-static void __maybe_unused ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
+static void ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
int i;
@@ -424,21 +451,22 @@ ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
}
/**
- * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
+ * ibmveth_setup_rx_interrupts - Register IRQ handlers and enable NAPI
* @adapter: ibmveth adapter structure
*
* Registers interrupt handlers for all RX queues and enables NAPI polling.
- * On error, cleans up any successfully registered IRQs before returning.
+ * For multi-queue mode, enables hypervisor interrupt delivery only after
+ * every queue has a Linux handler installed.
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused
+static int
ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
- int i, rc;
+ int i, rc, num = adapter->num_rx_queues;
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < num; i++) {
if (!adapter->queue_irq[i]) {
netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
rc = -EINVAL;
@@ -455,14 +483,34 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
}
}
- for (i = 0; i < adapter->num_rx_queues; i++)
+ for (i = 0; i < num; i++)
napi_enable(&adapter->napi[i]);
+ if (adapter->multi_queue && num > 1) {
+ for (i = 0; i < num; i++) {
+ rc = ibmveth_enable_irq(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d, rc=%d\n",
+ i, rc);
+ while (--i >= 0)
+ ibmveth_disable_irq(adapter, i);
+ rc = -EIO;
+ goto err_disable_napi;
+ }
+ }
+ }
+
return 0;
+err_disable_napi:
+ for (i = 0; i < num; i++)
+ napi_disable(&adapter->napi[i]);
+ i = num;
err_free_irqs:
while (--i >= 0)
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
return rc;
}
@@ -485,15 +533,7 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
}
- /* Dispose IRQ mappings for subordinate queues (1-15).
- * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
- */
- for (i = 1; i < adapter->num_rx_queues; i++) {
- if (adapter->queue_irq[i]) {
- irq_dispose_mapping(adapter->queue_irq[i]);
- adapter->queue_irq[i] = 0;
- }
- }
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
/* Clear queue 0 IRQ number */
adapter->queue_irq[0] = 0;
@@ -869,7 +909,7 @@ static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
+static int ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
int i, q, rc;
@@ -910,7 +950,7 @@ static int __maybe_unused ibmveth_alloc_buffer_pools(struct ibmveth_adapter *ada
*
* Frees buffer pools for all queues using the helper function.
*/
-static void __maybe_unused ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
+static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
{
int q;
@@ -1070,7 +1110,7 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
*
* Return: 0 on success, -ENOMEM on failure
*/
-static int __maybe_unused
+static int
ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
@@ -1095,7 +1135,7 @@ ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
*
* Frees TX Long Term Buffers (LTBs) for all TX queues.
*/
-static void __maybe_unused
+static void
ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
@@ -1149,33 +1189,6 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
return rc;
}
-/**
- * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
- * @adapter: ibmveth adapter structure
- *
- * Return: 0 on success, -ENOMEM on failure
- */
-static int __maybe_unused ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
-{
- adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
- sizeof(struct ibmveth_rx_queue_stats),
- GFP_KERNEL);
- if (!adapter->rx_qstats)
- return -ENOMEM;
-
- return 0;
-}
-
-/**
- * ibmveth_free_rx_qstats - Free per-queue RX statistics
- * @adapter: ibmveth adapter structure
- */
-static void __maybe_unused ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
-{
- kfree(adapter->rx_qstats);
- adapter->rx_qstats = NULL;
-}
-
/**
* ibmveth_register_logical_lan_queue - Register subordinate queue with hypervisor
* @adapter: ibmveth adapter structure
@@ -1466,208 +1479,108 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
static int ibmveth_open(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- u64 mac_address;
+ u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
int rxq_entries = 1;
- unsigned long lpar_rc;
int rc;
- union ibmveth_buf_desc rxq_desc;
int i;
- struct device *dev;
netdev_dbg(netdev, "open starting\n");
- napi_enable(&adapter->napi[0]);
-
- for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
rxq_entries += adapter->rx_buff_pool[0][i].size;
- rc = -ENOMEM;
- adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
- if (!adapter->buffer_list_addr[0]) {
- netdev_err(netdev, "unable to allocate list pages\n");
+ rc = ibmveth_alloc_rx_qstats(adapter);
+ if (rc)
goto out;
- }
- adapter->filter_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
- if (!adapter->filter_list_addr) {
- netdev_err(netdev, "unable to allocate filter pages\n");
- goto out_free_buffer_list;
- }
-
- dev = &adapter->vdev->dev;
+ rc = ibmveth_alloc_filter_list(adapter);
+ if (rc)
+ goto out_free_rx_qstats;
- adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
- rxq_entries;
- adapter->rx_queue[0].queue_addr =
- dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
- &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
- if (!adapter->rx_queue[0].queue_addr)
+ rc = ibmveth_alloc_rx_queues(adapter, rxq_entries);
+ if (rc)
goto out_free_filter_list;
- adapter->buffer_list_dma[0] = dma_map_single(dev,
- adapter->buffer_list_addr[0],
- 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
- netdev_err(netdev, "unable to map buffer list pages\n");
+ rc = ibmveth_alloc_buffer_pools(adapter);
+ if (rc)
goto out_free_queue_mem;
- }
- adapter->filter_list_dma = dma_map_single(dev,
- adapter->filter_list_addr, 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->filter_list_dma)) {
- netdev_err(netdev, "unable to map filter list pages\n");
- goto out_unmap_buffer_list;
- }
+ rc = ibmveth_register_rx_queues(adapter, mac_address);
+ if (rc)
+ goto out_free_buffer_pools;
- for (i = 0; i < netdev->real_num_tx_queues; i++) {
- if (ibmveth_allocate_tx_ltb(adapter, i))
- goto out_free_tx_ltb;
+ rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
+ if (rc) {
+ netdev_err(netdev, "failed to set number of rx queues\n");
+ goto out_unregister_queues;
}
- adapter->rx_queue[0].index = 0;
- adapter->rx_queue[0].num_slots = rxq_entries;
- adapter->rx_queue[0].toggle = 1;
-
- mac_address = ether_addr_to_u64(netdev->dev_addr);
-
- rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
- adapter->rx_queue[0].queue_len;
- rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
-
- netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr[0]);
- netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
- netdev_dbg(netdev, "receive q @ 0x%p\n", adapter->rx_queue[0].queue_addr);
-
- h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
-
- lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
-
- if (lpar_rc != H_SUCCESS) {
- netdev_err(netdev, "h_register_logical_lan failed with %ld\n",
- lpar_rc);
- netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
- "desc:0x%llx MAC:0x%llx\n",
- adapter->buffer_list_dma[0],
- adapter->filter_list_dma,
- rxq_desc.desc,
- mac_address);
- rc = -ENONET;
- goto out_unmap_filter_list;
- }
+ rc = ibmveth_setup_rx_interrupts(adapter);
+ if (rc)
+ goto out_unregister_queues;
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (!adapter->rx_buff_pool[0][i].active)
- continue;
- if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
- netdev_err(netdev, "unable to alloc pool\n");
- adapter->rx_buff_pool[0][i].active = 0;
- rc = -ENOMEM;
- goto out_free_buffer_pools;
+ if (adapter->num_rx_queues > 1) {
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ netdev_dbg(netdev, "initial replenish cycle for queue %d\n", i);
+ ibmveth_replenish_task(adapter, i);
}
+ } else {
+ netdev_dbg(netdev, "initial replenish cycle\n");
+ ibmveth_interrupt(adapter->queue_irq[0], &adapter->napi[0]);
}
- netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
- rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
- netdev);
- if (rc != 0) {
- netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
- netdev->irq, rc);
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
- goto out_free_buffer_pools;
- }
-
- rc = -ENOMEM;
-
- netdev_dbg(netdev, "initial replenish cycle\n");
- ibmveth_interrupt(netdev->irq, netdev);
+ rc = ibmveth_alloc_tx_resources(adapter);
+ if (rc)
+ goto out_cleanup_rx_interrupts;
netif_tx_start_all_queues(netdev);
netdev_dbg(netdev, "open complete\n");
-
return 0;
+out_cleanup_rx_interrupts:
+ ibmveth_cleanup_rx_interrupts(adapter);
+out_free_tx_resources:
+ ibmveth_free_tx_resources(adapter);
out_free_buffer_pools:
- while (--i >= 0) {
- if (adapter->rx_buff_pool[0][i].active)
- ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[0][i]);
- }
-out_unmap_filter_list:
- dma_unmap_single(dev, adapter->filter_list_dma, 4096,
- DMA_BIDIRECTIONAL);
-
-out_free_tx_ltb:
- while (--i >= 0) {
- ibmveth_free_tx_ltb(adapter, i);
- }
-
-out_unmap_buffer_list:
- dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
- DMA_BIDIRECTIONAL);
+ ibmveth_free_buffer_pools(adapter);
+out_unregister_queues:
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+ ibmveth_free_all_queues(adapter);
out_free_queue_mem:
- dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
- adapter->rx_queue[0].queue_addr,
- adapter->rx_queue[0].queue_dma);
+ ibmveth_cleanup_rx_resources(adapter);
out_free_filter_list:
- free_page((unsigned long)adapter->filter_list_addr);
-out_free_buffer_list:
- free_page((unsigned long)adapter->buffer_list_addr[0]);
+ ibmveth_free_filter_list(adapter);
+out_free_rx_qstats:
+ ibmveth_free_rx_qstats(adapter);
out:
- napi_disable(&adapter->napi[0]);
return rc;
}
static int ibmveth_close(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- struct device *dev = &adapter->vdev->dev;
- long lpar_rc;
int i;
netdev_dbg(netdev, "close starting\n");
- napi_disable(&adapter->napi[0]);
-
netif_tx_stop_all_queues(netdev);
- h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
-
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
- if (lpar_rc != H_SUCCESS) {
- netdev_err(netdev, "h_free_logical_lan failed with %lx, "
- "continuing with close\n", lpar_rc);
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i]) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
}
- free_irq(netdev->irq, netdev);
-
+ ibmveth_free_tx_resources(adapter);
+ ibmveth_cleanup_rx_interrupts(adapter);
ibmveth_update_rx_no_buffer(adapter);
-
- dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
- DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->buffer_list_addr[0]);
-
- dma_unmap_single(dev, adapter->filter_list_dma, 4096,
- DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->filter_list_addr);
-
- dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
- adapter->rx_queue[0].queue_addr,
- adapter->rx_queue[0].queue_dma);
-
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (adapter->rx_buff_pool[0][i].active)
- ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[0][i]);
-
- for (i = 0; i < netdev->real_num_tx_queues; i++)
- ibmveth_free_tx_ltb(adapter, i);
+ ibmveth_free_all_queues(adapter);
+ ibmveth_free_buffer_pools(adapter);
+ ibmveth_cleanup_rx_resources(adapter);
+ ibmveth_free_filter_list(adapter);
+ ibmveth_free_rx_qstats(adapter);
netdev_dbg(netdev, "close complete\n");
@@ -2423,15 +2336,21 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
{
- struct net_device *netdev = dev_instance;
+ struct napi_struct *napi = dev_instance;
+ struct net_device *netdev = napi->dev;
struct ibmveth_adapter *adapter = netdev_priv(netdev);
unsigned long lpar_rc;
+ int qindex;
- if (napi_schedule_prep(&adapter->napi[0])) {
- lpar_rc = h_vio_signal(adapter->vdev->unit_address,
- VIO_IRQ_DISABLE);
+ qindex = napi - adapter->napi;
+
+ if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
+ return IRQ_NONE;
+
+ if (napi_schedule_prep(napi)) {
+ lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
- __napi_schedule(&adapter->napi[0]);
+ __napi_schedule(napi);
}
return IRQ_HANDLED;
}
@@ -2537,8 +2456,10 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
#ifdef CONFIG_NET_POLL_CONTROLLER
static void ibmveth_poll_controller(struct net_device *dev)
{
- ibmveth_replenish_task(netdev_priv(dev));
- ibmveth_interrupt(dev->irq, dev);
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+
+ ibmveth_replenish_task(adapter);
+ ibmveth_interrupt(dev->irq, &adapter->napi[0]);
}
#endif
@@ -2951,7 +2872,7 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
rtnl_unlock();
/* kick the interrupt handler to allocate/deallocate pools */
- ibmveth_interrupt(netdev->irq, netdev);
+ ibmveth_interrupt(netdev->irq, &adapter->napi[0]);
return count;
unlock_err:
@@ -2991,7 +2912,9 @@ static struct kobj_type ktype_veth_pool = {
static int ibmveth_resume(struct device *dev)
{
struct net_device *netdev = dev_get_drvdata(dev);
- ibmveth_interrupt(netdev->irq, netdev);
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
+ ibmveth_interrupt(netdev->irq, &adapter->napi[0]);
return 0;
}
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 07/15] ibmveth: Add queue-aware RX buffer submit helper for MQ
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Replenish is the last open-path hypervisor call that still needs
per-queue awareness before MQ receive is enabled. Today
ibmveth_replenish_buffer_pool() calls h_add_logical_lan_buffer() or
h_add_logical_lan_buffers() directly; MQ posts via
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[].
Add ibmveth_add_logical_lan_buffers() to pick the hcall:
multi_queue uses h_add_logical_lan_buffers_queue() (up to 12 buffers,
IOBAs packed with odd counts in the upper 32 bits); legacy uses the
existing single- and multi-buffer hcalls. Count add_buf/add_bufs/
add_bufs_queue in hcall_stats.
Thread queue_index through replenish_task() and replenish_buffer_pool()
so they index rx_buff_pool[queue_index][pool]. All callers still pass
queue 0; legacy hcalls remain the live path until MQ probe enables
multi_queue.
Also split H_FUNCTION handling: legacy batch falls back to single-buffer
mode; multi_queue logs an error on unsupported firmware.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 134 ++++++++++++++++++++---------
1 file changed, 94 insertions(+), 40 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index fa2d4777ffc7..b3b3886c3eed 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -597,11 +597,73 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length)
asm("dcbf %0,%1,1" :: "b" (addr), "r" (offset));
}
+/**
+ * ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor
+ * @adapter: ibmveth adapter structure
+ * @descs: array of buffer descriptors to add
+ * @filled: number of valid descriptors in the array
+ * @buff_size: size of each buffer (multi-queue mode only)
+ * @queue_index: RX queue index
+ *
+ * Return: hypervisor return code
+ */
+static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
+ union ibmveth_buf_desc *descs,
+ int filled,
+ unsigned long buff_size,
+ int queue_index)
+{
+ struct vio_dev *vdev = adapter->vdev;
+ unsigned long rc;
+
+ if (adapter->multi_queue) {
+ unsigned long buffersznum = (buff_size << 32) | filled;
+ unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL / 2] = {0};
+ int i;
+
+ /* Pack descriptor addresses into ioba pairs.
+ * Each ioba holds two 32-bit addresses packed into 64 bits:
+ * - Even descriptors (0,2,4...) go in high 32 bits
+ * - Odd descriptors (1,3,5...) go in low 32 bits
+ */
+ for (i = 0; i < filled && i < IBMVETH_MAX_RX_PER_HCALL; i++) {
+ int pair_idx = i / 2; /* Which pair: 0-5 */
+ int is_high = (i % 2 == 0); /* High or low 32 bits */
+
+ if (is_high)
+ ioba[pair_idx] = (unsigned long)descs[i].fields.address << 32;
+ else
+ ioba[pair_idx] |= descs[i].fields.address;
+ }
+
+ rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
+ adapter->queue_handle[queue_index],
+ buffersznum,
+ ioba[0], ioba[1], ioba[2],
+ ioba[3], ioba[4], ioba[5]);
+ adapter->hcall_stats.add_bufs_queue++;
+ } else if (filled == 1) {
+ rc = h_add_logical_lan_buffer(vdev->unit_address,
+ descs[0].desc);
+ adapter->hcall_stats.add_buf++;
+ } else {
+ rc = h_add_logical_lan_buffers(vdev->unit_address,
+ descs[0].desc, descs[1].desc,
+ descs[2].desc, descs[3].desc,
+ descs[4].desc, descs[5].desc,
+ descs[6].desc, descs[7].desc);
+ adapter->hcall_stats.add_bufs++;
+ }
+
+ return rc;
+}
+
/* replenish the buffers for a pool. note that we don't need to
* skb_reserve these since they are used for incoming...
*/
static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
- struct ibmveth_buff_pool *pool)
+ struct ibmveth_buff_pool *pool,
+ int queue_index)
{
union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL] = {0};
u32 remaining = pool->size - atomic_read(&pool->available);
@@ -687,24 +749,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
if (!filled)
break;
- /* single buffer case*/
- if (filled == 1)
- lpar_rc = h_add_logical_lan_buffer(vdev->unit_address,
- descs[0].desc);
- else
- /* Multi-buffer hcall */
- lpar_rc = h_add_logical_lan_buffers(vdev->unit_address,
- descs[0].desc,
- descs[1].desc,
- descs[2].desc,
- descs[3].desc,
- descs[4].desc,
- descs[5].desc,
- descs[6].desc,
- descs[7].desc);
+ lpar_rc = ibmveth_add_logical_lan_buffers(adapter, descs,
+ filled,
+ pool->buff_size,
+ queue_index);
+
if (lpar_rc != H_SUCCESS) {
dev_warn_ratelimited(dev,
- "RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n",
+ "RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n",
+ adapter->multi_queue ? "_queue" : "",
filled, lpar_rc, batch);
goto hcall_failure;
}
@@ -745,24 +798,19 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
}
adapter->replenish_add_buff_failure += filled;
- /*
- * If multi rx buffers hcall is no longer supported by FW
- * e.g. in the case of Live Partition Migration
- */
- if (batch > 1 && lpar_rc == H_FUNCTION) {
- /*
- * Instead of retry submit single buffer individually
- * here just set the max rx buffer per hcall to 1
- * buffers will be respleshed next time
- * when ibmveth_replenish_buffer_pool() is called again
- * with single-buffer case
- */
- netdev_info(adapter->netdev,
- "RX Multi buffers not supported by FW, rc=%lu\n",
- lpar_rc);
- adapter->rx_buffers_per_hcall = 1;
- netdev_info(adapter->netdev,
- "Next rx replesh will fall back to single-buffer hcall\n");
+ if (lpar_rc == H_FUNCTION) {
+ if (adapter->multi_queue) {
+ netdev_err(adapter->netdev,
+ "Unexpected H_FUNCTION from multi-queue buffer add (queue=%d, batch=%d)\n",
+ queue_index, batch);
+ break;
+ } else if (batch > 1) {
+ netdev_warn(adapter->netdev,
+ "H_FUNCTION from legacy batch buffer add (batch=%d), falling back to single buffer mode\n",
+ batch);
+ adapter->rx_buffers_per_hcall = 1;
+ continue;
+ }
}
break;
}
@@ -784,18 +832,24 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
}
/* replenish routine */
-static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
+static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
+ int queue_index)
{
int i;
+ if (queue_index >= adapter->num_rx_queues)
+ return;
+
adapter->replenish_task_cycles++;
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
- struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[queue_index][i];
if (pool->active &&
(atomic_read(&pool->available) < pool->threshold))
- ibmveth_replenish_buffer_pool(adapter, pool);
+ ibmveth_replenish_buffer_pool(adapter, pool,
+ queue_index);
}
ibmveth_update_rx_no_buffer(adapter);
@@ -2307,7 +2361,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
}
- ibmveth_replenish_task(adapter);
+ ibmveth_replenish_task(adapter, 0);
if (frames_processed == budget)
goto out;
@@ -2458,7 +2512,7 @@ static void ibmveth_poll_controller(struct net_device *dev)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
- ibmveth_replenish_task(adapter);
+ ibmveth_replenish_task(adapter, 0);
ibmveth_interrupt(dev->irq, &adapter->napi[0]);
}
#endif
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 08/15] ibmveth: Enable multi-queue RX receive path
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
This is the first patch that sets multi_queue from H_ILLAN_ATTRIBUTES
and switches registration, buffer posting, and receive to the MQ
hcall path. It also raises num_rx_queues and enables per-queue NAPI.
This is where MQ actually receives packets. If firmware sets
IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT in H_ILLAN_ATTRIBUTES, probe sets
multi_queue and num_rx_queues to min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES), matching the existing TX default (cap 8).
Up to IBMVETH_MAX_RX_QUEUES (16) remains available via ethtool -L.
Otherwise we stay at one queue like today.
Raise IBMVETH_MAX_RX_QUEUES to 16 here so adapter arrays and NAPI state
can hold every queue before num_rx_queues is increased.
Register a NAPI struct per possible queue at probe, use
alloc_etherdev_mqs(), and call netif_set_real_num_rx_queues() after PHYP
registration on open.
With MQ enabled, open runs initial replenish on every active queue before
starting TX; legacy still kicks replenish via queue-0 interrupt/NAPI only.
PHYP can deliver to any registered queue immediately, so unprimed queues
see no-buffer drops until their NAPI path runs.
Datapath: derive queue_index from the NAPI instance, thread it through
harvest/replenish/pool access, and enable/disable IRQ per queue on NAPI
completion. Add per-queue replenish_lock around buffer posting (same-queue
NAPI vs netpoll/resize). poll_controller() and get_desired_dma() walk all
queues.
Update KUnit tests for the queue_index argument added to
ibmveth_remove_buffer_from_pool() and ibmveth_rxq_get_buffer().
Legacy firmware without the MQ bit is unchanged.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 257 ++++++++++++++++++-----------
drivers/net/ethernet/ibm/ibmveth.h | 10 +-
2 files changed, 171 insertions(+), 96 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b3b3886c3eed..863e5c68b42c 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -30,6 +30,7 @@
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/slab.h>
+#include <linux/spinlock.h>
#include <asm/hvcall.h>
#include <linux/atomic.h>
#include <asm/vio.h>
@@ -101,45 +102,58 @@ static struct ibmveth_stat ibmveth_stats[] = {
};
/* simple methods of getting data from the current rxq entry */
-static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
+static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return be32_to_cpu(adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].flags_off);
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+
+ return be32_to_cpu(rxq->queue_addr[rxq->index].flags_off);
}
-static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return (ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_TOGGLE) >>
- IBMVETH_RXQ_TOGGLE_SHIFT;
+ return (ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_TOGGLE) >>
+ IBMVETH_RXQ_TOGGLE_SHIFT;
}
-static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
+ return ibmveth_rxq_toggle(adapter, queue_index) ==
+ adapter->rx_queue[queue_index].toggle;
}
-static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_VALID;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_VALID;
}
-static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_OFF_MASK;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_OFF_MASK;
}
-static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_LRG_PKT;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_LRG_PKT;
}
-static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return be32_to_cpu(adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].length);
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+
+ return be32_to_cpu(rxq->queue_addr[rxq->index].length);
}
-static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_CSUM_GOOD;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_CSUM_GOOD;
}
static unsigned int ibmveth_real_max_tx_queues(void)
@@ -274,6 +288,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
adapter->rx_queue[i].index = 0;
adapter->rx_queue[i].num_slots = rxq_entries;
adapter->rx_queue[i].toggle = 1;
+ spin_lock_init(&adapter->rx_queue[i].replenish_lock);
netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
i, adapter->buffer_list_addr[i],
@@ -826,15 +841,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
*/
static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
{
- __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ __be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
+ u64 drops = be64_to_cpup(p);
- adapter->rx_no_buffer = be64_to_cpup(p);
+ if (i == 0)
+ adapter->rx_no_buffer = drops;
+ }
}
/* replenish routine */
static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
int queue_index)
{
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+ unsigned long flags;
int i;
if (queue_index >= adapter->num_rx_queues)
@@ -842,6 +865,8 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
adapter->replenish_task_cycles++;
+ spin_lock_irqsave(&rxq->replenish_lock, flags);
+
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
struct ibmveth_buff_pool *pool =
&adapter->rx_buff_pool[queue_index][i];
@@ -853,6 +878,8 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
}
ibmveth_update_rx_no_buffer(adapter);
+
+ spin_unlock_irqrestore(&rxq->replenish_lock, flags);
}
/* empty and free ana buffer pool - also used to do cleanup in error paths */
@@ -1028,7 +1055,8 @@ static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
* * %-EFAULT - pool and index map to null skb
*/
static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
- u64 correlator, bool reuse)
+ u64 correlator, int queue_index,
+ bool reuse)
{
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
@@ -1036,12 +1064,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
struct sk_buff *skb;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
schedule_work(&adapter->work);
return -EINVAL;
}
- skb = adapter->rx_buff_pool[0][pool].skbuff[index];
+ skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
if (WARN_ON(!skb)) {
schedule_work(&adapter->work);
return -EFAULT;
@@ -1055,42 +1083,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* remove the skb pointer to mark free. actual freeing is done
* by upper level networking after gro_receive
*/
- adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
+ adapter->rx_buff_pool[queue_index][pool].skbuff[index] = NULL;
dma_unmap_single(&adapter->vdev->dev,
- adapter->rx_buff_pool[0][pool].dma_addr[index],
- adapter->rx_buff_pool[0][pool].buff_size,
+ adapter->rx_buff_pool[queue_index][pool].dma_addr[index],
+ adapter->rx_buff_pool[queue_index][pool].buff_size,
DMA_FROM_DEVICE);
}
- free_index = adapter->rx_buff_pool[0][pool].producer_index;
- adapter->rx_buff_pool[0][pool].producer_index++;
- if (adapter->rx_buff_pool[0][pool].producer_index >=
- adapter->rx_buff_pool[0][pool].size)
- adapter->rx_buff_pool[0][pool].producer_index = 0;
- adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
+ free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;
+ adapter->rx_buff_pool[queue_index][pool].producer_index++;
+ if (adapter->rx_buff_pool[queue_index][pool].producer_index >=
+ adapter->rx_buff_pool[queue_index][pool].size)
+ adapter->rx_buff_pool[queue_index][pool].producer_index = 0;
+ adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;
mb();
- atomic_dec(&adapter->rx_buff_pool[0][pool].available);
+ atomic_dec(&adapter->rx_buff_pool[queue_index][pool].available);
return 0;
}
/* get the current buffer on the rx queue */
-static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
+static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- u64 correlator = adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].correlator;
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+ u64 correlator = rxq->queue_addr[rxq->index].correlator;
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
schedule_work(&adapter->work);
return NULL;
}
- return adapter->rx_buff_pool[0][pool].skbuff[index];
+ return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
}
/**
@@ -1106,19 +1136,20 @@ static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *ada
* * other - non-zero return from ibmveth_remove_buffer_from_pool
*/
static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
- bool reuse)
+ int queue_index, bool reuse)
{
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
u64 cor;
int rc;
- cor = adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].correlator;
- rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
+ cor = rxq->queue_addr[rxq->index].correlator;
+ rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
if (unlikely(rc))
return rc;
- if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
- adapter->rx_queue[0].index = 0;
- adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
+ if (++rxq->index == rxq->num_slots) {
+ rxq->index = 0;
+ rxq->toggle = !rxq->toggle;
}
return 0;
@@ -2268,34 +2299,40 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
static int ibmveth_poll(struct napi_struct *napi, int budget)
{
- struct ibmveth_adapter *adapter =
- container_of(napi, struct ibmveth_adapter, napi[0]);
- struct net_device *netdev = adapter->netdev;
+ struct net_device *netdev = napi->dev;
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
int frames_processed = 0;
unsigned long lpar_rc;
+ int queue_index, rc;
u16 mss = 0;
+ queue_index = napi - adapter->napi;
+
+ if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
+ return 0;
+
restart_poll:
while (frames_processed < budget) {
- if (!ibmveth_rxq_pending_buffer(adapter))
+ if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
break;
smp_rmb();
- if (!ibmveth_rxq_buffer_valid(adapter)) {
+ if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
wmb(); /* suggested by larson1 */
adapter->rx_invalid_buffer++;
netdev_dbg(netdev, "recycling invalid buffer\n");
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (unlikely(rc))
break;
} else {
struct sk_buff *skb, *new_skb;
- int length = ibmveth_rxq_frame_length(adapter);
- int offset = ibmveth_rxq_frame_offset(adapter);
- int csum_good = ibmveth_rxq_csum_good(adapter);
- int lrg_pkt = ibmveth_rxq_large_packet(adapter);
+ int length = ibmveth_rxq_frame_length(adapter, queue_index);
+ int offset = ibmveth_rxq_frame_offset(adapter, queue_index);
+ int csum_good = ibmveth_rxq_csum_good(adapter, queue_index);
+ int lrg_pkt = ibmveth_rxq_large_packet(adapter, queue_index);
__sum16 iph_check = 0;
- skb = ibmveth_rxq_get_buffer(adapter);
+ skb = ibmveth_rxq_get_buffer(adapter, queue_index);
if (unlikely(!skb))
break;
@@ -2320,12 +2357,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
length);
if (rx_flush)
ibmveth_flush_buffer(skb->data,
- length + offset);
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+ length + offset);
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (unlikely(rc))
break;
skb = new_skb;
} else {
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, false)))
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, false);
+ if (unlikely(rc))
break;
skb_reserve(skb, offset);
}
@@ -2361,7 +2400,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
}
- ibmveth_replenish_task(adapter, 0);
+ ibmveth_replenish_task(adapter, queue_index);
if (frames_processed == budget)
goto out;
@@ -2372,15 +2411,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
/* We think we are done - reenable interrupts,
* then check once more to make sure we are done.
*/
- lpar_rc = h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_ENABLE);
- if (WARN_ON(lpar_rc != H_SUCCESS)) {
+ lpar_rc = ibmveth_enable_irq(adapter, queue_index);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n",
+ queue_index, lpar_rc);
schedule_work(&adapter->work);
goto out;
}
- if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
- lpar_rc = h_vio_signal(adapter->vdev->unit_address,
- VIO_IRQ_DISABLE);
+ if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+ napi_schedule(napi)) {
+ lpar_rc = ibmveth_disable_irq(adapter, queue_index);
+ WARN_ON(lpar_rc != H_SUCCESS);
goto restart_poll;
}
@@ -2511,9 +2554,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
static void ibmveth_poll_controller(struct net_device *dev)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i;
- ibmveth_replenish_task(adapter, 0);
- ibmveth_interrupt(dev->irq, &adapter->napi[0]);
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ ibmveth_replenish_task(adapter, i);
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ ibmveth_interrupt(adapter->queue_irq[i], &adapter->napi[i]);
}
#endif
@@ -2531,8 +2578,7 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
struct ibmveth_adapter *adapter;
struct iommu_table *tbl;
unsigned long ret;
- int i;
- int rxqentries = 1;
+ int i, q;
tbl = get_iommu_table_base(&vdev->dev);
@@ -2547,18 +2593,22 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
/* add size of mapped tx buffers */
ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- /* add the size of the active receive buffers */
- if (adapter->rx_buff_pool[0][i].active)
- ret +=
- adapter->rx_buff_pool[0][i].size *
- IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
- buff_size, tbl);
- rxqentries += adapter->rx_buff_pool[0][i].size;
- }
- /* add the size of the receive queue entries */
- ret += IOMMU_PAGE_ALIGN(
- rxqentries * sizeof(struct ibmveth_rx_q_entry), tbl);
+ for (q = 0; q < adapter->num_rx_queues; q++) {
+ int rxqentries = 1;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ /* add the size of the active receive buffers */
+ if (adapter->rx_buff_pool[q][i].active)
+ ret += adapter->rx_buff_pool[q][i].size *
+ IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[q][i].buff_size,
+ tbl);
+ rxqentries += adapter->rx_buff_pool[q][i].size;
+ }
+
+ /* add the size of the receive queue entries */
+ ret += IOMMU_PAGE_ALIGN(rxqentries *
+ sizeof(struct ibmveth_rx_q_entry), tbl);
+ }
return ret;
}
@@ -2660,7 +2710,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
return -EINVAL;
}
- netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter), IBMVETH_MAX_QUEUES, 1);
+ netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter),
+ IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES);
if (!netdev)
return -ENOMEM;
@@ -2673,7 +2724,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
ibmveth_init_link_settings(netdev);
- netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+ netif_napi_add_weight(netdev, &adapter->napi[i], ibmveth_poll, 16);
netdev->irq = dev->irq;
netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2705,16 +2757,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev->features |= NETIF_F_FRAGLIST;
}
- /* Initialize queue count - always 1 for now */
- adapter->multi_queue = 0;
- adapter->num_rx_queues = 1;
+ if (ret == H_SUCCESS &&
+ (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) {
+ adapter->multi_queue = 1;
+ adapter->num_rx_queues = min(num_online_cpus(), IBMVETH_DEFAULT_QUEUES);
+ netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n",
+ adapter->num_rx_queues);
+ } else {
+ adapter->multi_queue = 0;
+ adapter->num_rx_queues = 1;
+ }
if (ret == H_SUCCESS &&
(ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
- adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
+ if (adapter->multi_queue)
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_QUEUE;
+ else
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
+
netdev_dbg(netdev,
"RX Multi-buffer hcall supported by FW, batch set to %u\n",
- adapter->rx_buffers_per_hcall);
+ adapter->rx_buffers_per_hcall);
} else {
adapter->rx_buffers_per_hcall = 1;
netdev_dbg(netdev,
@@ -3057,17 +3120,23 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
correlator = ((u64)IBMVETH_NUM_BUFF_POOLS << 32) | 0;
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, true));
correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, true));
correlator = (u64)0 | 0;
pool->skbuff[0] = NULL;
- KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EFAULT,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EFAULT,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, true));
flush_work(&adapter->work);
}
@@ -3111,15 +3180,15 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
adapter->rx_queue[0].queue_addr[0].correlator = (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
- KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
adapter->rx_queue[0].queue_addr[0].correlator =
(u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
- KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
pool->skbuff[0] = skb;
adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
- KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter, 0));
flush_work(&adapter->work);
}
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index d2ceeccd5fbd..f7b20fd01acb 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -14,6 +14,8 @@
#ifndef _IBMVETH_H
#define _IBMVETH_H
+#include <linux/spinlock_types.h>
+
/* constants for H_MULTICAST_CTRL */
#define IbmVethMcastReceptionModifyBit 0x80000UL
#define IbmVethMcastReceptionEnableBit 0x20000UL
@@ -28,6 +30,7 @@
#define IbmVethMcastRemoveFilter 0x2UL
#define IbmVethMcastClearFilterTable 0x3UL
+#define IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT 0x0000000000080000UL
#define IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT 0x0000000000040000UL
#define IBMVETH_ILLAN_LRG_SR_ENABLED 0x0000000000010000UL
#define IBMVETH_ILLAN_LRG_SND_SUPPORT 0x0000000000008000UL
@@ -279,9 +282,11 @@ static inline long h_illan_attributes(unsigned long unit_address,
#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
#define IBMVETH_MAX_QUEUES 16U
#define IBMVETH_DEFAULT_QUEUES 8U
-#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_MAX_RX_QUEUES 16U
#define IBMVETH_DEFAULT_RX_QUEUES 1U
-#define IBMVETH_MAX_RX_PER_HCALL 8U
+#define IBMVETH_MAX_RX_REGULAR 8U
+#define IBMVETH_MAX_RX_QUEUE 12U
+#define IBMVETH_MAX_RX_PER_HCALL 12U
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
static int pool_count[] = { 256, 512, 256, 256, 256 };
@@ -336,6 +341,7 @@ struct ibmveth_rx_q {
dma_addr_t queue_dma;
u32 queue_len;
struct ibmveth_rx_q_entry *queue_addr;
+ spinlock_t replenish_lock; /* serializes per-queue buffer replenish */
};
struct ibmveth_adapter {
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 09/15] ibmveth: Add per-queue RX statistics collection and reporting
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Count per-queue RX stats in poll, replenish, and the IRQ handler:
packets, bytes, polls, large_packets, invalid_buffers, no_buffer_drops,
and interrupts. Stop updating netdev->stats.rx_* in poll; totals are
summed from rx_qstats[] in get_stats64(). Per-queue TX stats follow in
the next patch.
Expose the counters via:
- ethtool -S: per-queue rxN_* strings and aggregated invalid/large
packet globals via ibmveth_aggregate_rx_qstats(). pool%d_* reports
queue-0 pool geometry (size, active, available) only: static probe
config used as the template for every queue. Live per-queue pool
usage is exported through sysfs in the next patch.
- get_stats64: sum rx_qstats[] so ip -s and /proc/net/dev report total RX
- ethtool hcall_stats counters and count send_lan on successful TX hcalls
Fix get_channels() reporting: max_rx is IBMVETH_MAX_RX_QUEUES only when
MQ firmware is enabled, rx_count tracks adapter->num_rx_queues.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 152 ++++++++++++++++++++++++++---
1 file changed, 141 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 863e5c68b42c..1c08082ffbd6 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -98,7 +98,15 @@ static struct ibmveth_stat ibmveth_stats[] = {
{ "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) },
{ "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) },
{ "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) },
- { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }
+ { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) },
+ { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
+ { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) },
+ { "hcall_add_bufs_queue", IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) },
+ { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) },
+ { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) },
+ { "hcall_free_lan_queue", IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) },
+ { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) },
+ { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) },
};
/* simple methods of getting data from the current rxq entry */
@@ -847,6 +855,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
u64 drops = be64_to_cpup(p);
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[i].no_buffer_drops = drops;
if (i == 0)
adapter->rx_no_buffer = drops;
}
@@ -1925,22 +1935,71 @@ static int ibmveth_set_features(struct net_device *dev,
return rc1 ? rc1 : rc2;
}
+/**
+ * ibmveth_aggregate_rx_qstats - Sum per-queue RX stats into globals
+ * @adapter: ibmveth adapter
+ *
+ * Cold path only (ethtool). Keeps legacy global counters meaningful for
+ * tools that read the adapter-level fields in ibmveth_stats[].
+ */
+static void ibmveth_aggregate_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ u64 total_invalid = 0;
+ u64 total_large = 0;
+ int i;
+
+ if (!adapter->rx_qstats)
+ return;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ total_invalid += adapter->rx_qstats[i].invalid_buffers;
+ total_large += adapter->rx_qstats[i].large_packets;
+ }
+
+ adapter->rx_invalid_buffer = total_invalid;
+ adapter->rx_large_packets = total_large;
+}
+
static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ u8 *p = data;
int i;
if (stringset != ETH_SS_STATS)
return;
- for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++, data += ETH_GSTRING_LEN)
- memcpy(data, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+ for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++) {
+ memcpy(p, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+ p += ETH_GSTRING_LEN;
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ ethtool_sprintf(&p, "rx%d_packets", i);
+ ethtool_sprintf(&p, "rx%d_bytes", i);
+ ethtool_sprintf(&p, "rx%d_interrupts", i);
+ ethtool_sprintf(&p, "rx%d_polls", i);
+ ethtool_sprintf(&p, "rx%d_large_packets", i);
+ ethtool_sprintf(&p, "rx%d_invalid_buffers", i);
+ ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
+ }
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ ethtool_sprintf(&p, "pool%d_size", i);
+ ethtool_sprintf(&p, "pool%d_active", i);
+ ethtool_sprintf(&p, "pool%d_available", i);
+ }
}
static int ibmveth_get_sset_count(struct net_device *dev, int sset)
{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+
switch (sset) {
case ETH_SS_STATS:
- return ARRAY_SIZE(ibmveth_stats);
+ return ARRAY_SIZE(ibmveth_stats) +
+ adapter->num_rx_queues * IBMVETH_NUM_RX_QSTATS +
+ IBMVETH_NUM_BUFF_POOLS * 3;
default:
return -EOPNOTSUPP;
}
@@ -1949,21 +2008,48 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
static void ibmveth_get_ethtool_stats(struct net_device *dev,
struct ethtool_stats *stats, u64 *data)
{
- int i;
struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i, j;
+
+ ibmveth_aggregate_rx_qstats(adapter);
for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++)
data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset);
+
+ for (j = 0; j < adapter->num_rx_queues; j++) {
+ if (adapter->rx_qstats) {
+ data[i++] = adapter->rx_qstats[j].packets;
+ data[i++] = adapter->rx_qstats[j].bytes;
+ data[i++] = adapter->rx_qstats[j].interrupts;
+ data[i++] = adapter->rx_qstats[j].polls;
+ data[i++] = adapter->rx_qstats[j].large_packets;
+ data[i++] = adapter->rx_qstats[j].invalid_buffers;
+ data[i++] = adapter->rx_qstats[j].no_buffer_drops;
+ } else {
+ i += IBMVETH_NUM_RX_QSTATS;
+ }
+ }
+
+ for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+ data[i++] = adapter->rx_buff_pool[0][j].size;
+ data[i++] = adapter->rx_buff_pool[0][j].active;
+ data[i++] = atomic_read(&adapter->rx_buff_pool[0][j].available);
+ }
}
static void ibmveth_get_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
channels->max_tx = ibmveth_real_max_tx_queues();
channels->tx_count = netdev->real_num_tx_queues;
- channels->max_rx = netdev->real_num_rx_queues;
- channels->rx_count = netdev->real_num_rx_queues;
+ if (adapter->multi_queue)
+ channels->max_rx = IBMVETH_MAX_RX_QUEUES;
+ else
+ channels->max_rx = 1;
+ channels->rx_count = adapter->num_rx_queues;
}
static int ibmveth_set_channels(struct net_device *netdev,
@@ -2061,6 +2147,7 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,
return 1;
}
+ adapter->hcall_stats.send_lan++;
return 0;
}
@@ -2311,6 +2398,9 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
return 0;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].polls++;
+
restart_poll:
while (frames_processed < budget) {
if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
@@ -2319,7 +2409,10 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
smp_rmb();
if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
wmb(); /* suggested by larson1 */
- adapter->rx_invalid_buffer++;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].invalid_buffers++;
+ else
+ adapter->rx_invalid_buffer++;
netdev_dbg(netdev, "recycling invalid buffer\n");
rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
if (unlikely(rc))
@@ -2384,7 +2477,10 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
if ((length > netdev->mtu + ETH_HLEN) ||
lrg_pkt || iph_check == 0xffff) {
ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
- adapter->rx_large_packets++;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].large_packets++;
+ else
+ adapter->rx_large_packets++;
}
if (csum_good) {
@@ -2394,8 +2490,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
napi_gro_receive(napi, skb); /* send it up */
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += length;
+ if (adapter->rx_qstats) {
+ adapter->rx_qstats[queue_index].packets++;
+ adapter->rx_qstats[queue_index].bytes += length;
+ }
+
frames_processed++;
}
}
@@ -2444,6 +2543,9 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
return IRQ_NONE;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[qindex].interrupts++;
+
if (napi_schedule_prep(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
@@ -2656,6 +2758,33 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,
return vlan_features_check(skb, features);
}
+/**
+ * ibmveth_get_stats64 - Return aggregated per-queue RX statistics
+ * @dev: network device
+ * @stats: rtnl link statistics storage
+ *
+ * Sums per-queue rx_qstats into rx_packets/rx_bytes for multi-queue mode.
+ * TX counters continue to come from netdev->stats (updated in start_xmit).
+ */
+static void ibmveth_get_stats64(struct net_device *dev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i;
+
+ if (adapter->rx_qstats) {
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ stats->rx_packets += adapter->rx_qstats[i].packets;
+ stats->rx_bytes += adapter->rx_qstats[i].bytes;
+ }
+ }
+
+ stats->tx_packets = dev->stats.tx_packets;
+ stats->tx_bytes = dev->stats.tx_bytes;
+ stats->tx_dropped = dev->stats.tx_dropped;
+ stats->tx_errors = dev->stats.tx_errors;
+}
+
static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_open = ibmveth_open,
.ndo_stop = ibmveth_close,
@@ -2668,6 +2797,7 @@ static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_validate_addr = eth_validate_addr,
.ndo_set_mac_address = ibmveth_set_mac_addr,
.ndo_features_check = ibmveth_features_check,
+ .ndo_get_stats64 = ibmveth_get_stats64,
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = ibmveth_poll_controller,
#endif
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 11/15] ibmveth: Expose per-queue buffer pool details via sysfs
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Add a read-only buffer_pools sysfs attribute under the VIO device that
lists size, buff_size, active, and available for every RX queue and
pool: runtime per-queue buffer pressure during MQ operation. ethtool -S
pool%d_* (previous patch) reports queue-0 static probe geometry only;
sysfs is the right place for dynamic per-queue pool state at scale.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 56 ++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 4e3f49b6346f..ecc472ee8f71 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -2896,6 +2896,52 @@ static const struct net_device_ops ibmveth_netdev_ops = {
#endif
};
+static const struct attribute_group ibmveth_attr_group;
+
+static ssize_t buffer_pools_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct net_device *netdev = dev_get_drvdata(dev);
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+ int len = 0;
+ int i, j;
+
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "Queue Pool Size BuffSize Active Available\n");
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "----- ---- ---- -------- ------ ---------\n");
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[i][j];
+
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "%5d %4d %4u %8u %6d %9d\n",
+ i, j, pool->size, pool->buff_size,
+ pool->active,
+ atomic_read(&pool->available));
+
+ if (len >= PAGE_SIZE - 100)
+ goto out;
+ }
+ }
+
+out:
+ return len;
+}
+static DEVICE_ATTR_RO(buffer_pools);
+
+static struct attribute *ibmveth_attrs[] = {
+ &dev_attr_buffer_pools.attr,
+ NULL,
+};
+
+static const struct attribute_group ibmveth_attr_group = {
+ .attrs = ibmveth_attrs,
+};
+
static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
{
int rc, i, mac_len;
@@ -3056,6 +3102,14 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev_dbg(netdev, "registered\n");
+ rc = sysfs_create_group(&dev->dev.kobj, &ibmveth_attr_group);
+ if (rc) {
+ netdev_err(netdev, "failed to create sysfs attributes rc=%d\n", rc);
+ unregister_netdev(netdev);
+ free_netdev(netdev);
+ return rc;
+ }
+
return 0;
}
@@ -3067,6 +3121,8 @@ static void ibmveth_remove(struct vio_dev *dev)
cancel_work_sync(&adapter->work);
+ sysfs_remove_group(&dev->dev.kobj, &ibmveth_attr_group);
+
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
kobject_put(&adapter->rx_buff_pool[0][i].kobj);
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 10/15] ibmveth: Add per-queue TX statistics reporting
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Track transmit counters per TX queue to avoid cache line contention in
the xmit hot path and expose per-queue visibility via ethtool -S and
ndo_get_stats64() aggregation.
Global tx_large_packets and tx_send_failed continue to be aggregated on
the ethtool read path for backward compatibility with existing tools.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 129 +++++++++++++++++++++++++----
drivers/net/ethernet/ibm/ibmveth.h | 13 +++
2 files changed, 124 insertions(+), 18 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 1c08082ffbd6..4e3f49b6346f 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -252,6 +252,33 @@ static void ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
adapter->rx_qstats = NULL;
}
+/**
+ * ibmveth_alloc_tx_qstats - Allocate per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->tx_qstats = kcalloc(IBMVETH_MAX_QUEUES,
+ sizeof(struct ibmveth_tx_queue_stats),
+ GFP_KERNEL);
+ if (!adapter->tx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_tx_qstats - Free per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->tx_qstats);
+ adapter->tx_qstats = NULL;
+}
+
/**
* ibmveth_alloc_rx_queues - Allocate per-queue RX resources
* @adapter: ibmveth adapter structure
@@ -1628,6 +1655,10 @@ static int ibmveth_open(struct net_device *netdev)
if (rc)
goto out_cleanup_rx_interrupts;
+ rc = ibmveth_alloc_tx_qstats(adapter);
+ if (rc)
+ goto out_free_tx_resources;
+
netif_tx_start_all_queues(netdev);
netdev_dbg(netdev, "open complete\n");
@@ -1668,6 +1699,7 @@ static int ibmveth_close(struct net_device *netdev)
}
}
+ ibmveth_free_tx_qstats(adapter);
ibmveth_free_tx_resources(adapter);
ibmveth_cleanup_rx_interrupts(adapter);
ibmveth_update_rx_no_buffer(adapter);
@@ -1960,6 +1992,32 @@ static void ibmveth_aggregate_rx_qstats(struct ibmveth_adapter *adapter)
adapter->rx_large_packets = total_large;
}
+/**
+ * ibmveth_aggregate_tx_qstats - Sum per-queue TX stats into globals
+ * @adapter: ibmveth adapter
+ *
+ * Cold path only (ethtool). Keeps legacy global counters meaningful for
+ * tools that read the adapter-level fields in ibmveth_stats[].
+ */
+static void ibmveth_aggregate_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ u64 total_large = 0;
+ u64 total_send_failed = 0;
+ int i;
+
+ if (!adapter->tx_qstats)
+ return;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++) {
+ total_large += adapter->tx_qstats[i].large_packets;
+ total_send_failed += adapter->tx_qstats[i].send_failures;
+ }
+
+ adapter->tx_large_packets = total_large;
+ adapter->tx_send_failed = total_send_failed;
+}
+
static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
@@ -1984,6 +2042,15 @@ static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
}
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ ethtool_sprintf(&p, "tx%d_packets", i);
+ ethtool_sprintf(&p, "tx%d_bytes", i);
+ ethtool_sprintf(&p, "tx%d_large_packets", i);
+ ethtool_sprintf(&p, "tx%d_dropped_packets", i);
+ ethtool_sprintf(&p, "tx%d_send_failures", i);
+ ethtool_sprintf(&p, "tx%d_checksum_offload", i);
+ }
+
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
ethtool_sprintf(&p, "pool%d_size", i);
ethtool_sprintf(&p, "pool%d_active", i);
@@ -1999,6 +2066,7 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
case ETH_SS_STATS:
return ARRAY_SIZE(ibmveth_stats) +
adapter->num_rx_queues * IBMVETH_NUM_RX_QSTATS +
+ dev->real_num_tx_queues * IBMVETH_NUM_TX_QSTATS +
IBMVETH_NUM_BUFF_POOLS * 3;
default:
return -EOPNOTSUPP;
@@ -2012,6 +2080,7 @@ static void ibmveth_get_ethtool_stats(struct net_device *dev,
int i, j;
ibmveth_aggregate_rx_qstats(adapter);
+ ibmveth_aggregate_tx_qstats(adapter);
for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++)
data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset);
@@ -2030,6 +2099,19 @@ static void ibmveth_get_ethtool_stats(struct net_device *dev,
}
}
+ for (j = 0; j < dev->real_num_tx_queues; j++) {
+ if (adapter->tx_qstats) {
+ data[i++] = adapter->tx_qstats[j].packets;
+ data[i++] = adapter->tx_qstats[j].bytes;
+ data[i++] = adapter->tx_qstats[j].large_packets;
+ data[i++] = adapter->tx_qstats[j].dropped_packets;
+ data[i++] = adapter->tx_qstats[j].send_failures;
+ data[i++] = adapter->tx_qstats[j].checksum_offload;
+ } else {
+ i += IBMVETH_NUM_TX_QSTATS;
+ }
+ }
+
for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
data[i++] = adapter->rx_buff_pool[0][j].size;
data[i++] = adapter->rx_buff_pool[0][j].active;
@@ -2152,8 +2234,10 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,
}
static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
- struct net_device *netdev)
+ struct ibmveth_adapter *adapter,
+ int queue_num)
{
+ struct net_device *netdev = adapter->netdev;
struct ethhdr *ether_header;
int ret = 0;
@@ -2161,7 +2245,8 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
if (ether_addr_equal(ether_header->h_dest, netdev->dev_addr)) {
netdev_dbg(netdev, "veth doesn't support loopback packets, dropping packet.\n");
- netdev->stats.tx_dropped++;
+ if (adapter->tx_qstats)
+ adapter->tx_qstats[queue_num].dropped_packets++;
ret = -EOPNOTSUPP;
}
@@ -2177,7 +2262,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
int i, queue_num = skb_get_queue_mapping(skb);
unsigned long mss = 0;
- if (ibmveth_is_packet_unsupported(skb, netdev))
+ if (ibmveth_is_packet_unsupported(skb, adapter, queue_num))
goto out;
/* veth can't checksum offload UDP */
if (skb->ip_summed == CHECKSUM_PARTIAL &&
@@ -2188,7 +2273,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
skb_checksum_help(skb)) {
netdev_err(netdev, "tx: failed to checksum packet\n");
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
@@ -2200,6 +2285,8 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
desc_flags |= (IBMVETH_BUF_NO_CSUM | IBMVETH_BUF_CSUM_GOOD);
+ adapter->tx_qstats[queue_num].checksum_offload++;
+
/* Need to zero out the checksum */
buf[0] = 0;
buf[1] = 0;
@@ -2211,7 +2298,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (skb->ip_summed == CHECKSUM_PARTIAL && skb_is_gso(skb)) {
if (adapter->fw_large_send_support) {
mss = (unsigned long)skb_shinfo(skb)->gso_size;
- adapter->tx_large_packets++;
+ adapter->tx_qstats[queue_num].large_packets++;
} else if (!skb_is_gso_v6(skb)) {
/* Put -1 in the IP checksum to tell phyp it
* is a largesend packet. Put the mss in
@@ -2220,7 +2307,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
ip_hdr(skb)->check = 0xffff;
tcp_hdr(skb)->check =
cpu_to_be16(skb_shinfo(skb)->gso_size);
- adapter->tx_large_packets++;
+ adapter->tx_qstats[queue_num].large_packets++;
}
}
@@ -2228,7 +2315,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (unlikely(skb->len > adapter->tx_ltb_size)) {
netdev_err(adapter->netdev, "tx: packet size (%u) exceeds ltb (%u)\n",
skb->len, adapter->tx_ltb_size);
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
@@ -2245,7 +2332,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (unlikely(total_bytes != skb->len)) {
netdev_err(adapter->netdev, "tx: incorrect packet len copied into ltb (%u != %u)\n",
skb->len, total_bytes);
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
desc.fields.flags_len = desc_flags | skb->len;
@@ -2254,11 +2341,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
dma_wmb();
if (ibmveth_send(adapter, desc.desc, mss)) {
- adapter->tx_send_failed++;
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].send_failures++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
} else {
- netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += skb->len;
+ adapter->tx_qstats[queue_num].packets++;
+ adapter->tx_qstats[queue_num].bytes += skb->len;
}
out:
@@ -2759,12 +2846,13 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,
}
/**
- * ibmveth_get_stats64 - Return aggregated per-queue RX statistics
+ * ibmveth_get_stats64 - Return aggregated per-queue statistics
* @dev: network device
* @stats: rtnl link statistics storage
*
- * Sums per-queue rx_qstats into rx_packets/rx_bytes for multi-queue mode.
- * TX counters continue to come from netdev->stats (updated in start_xmit).
+ * Sums per-queue rx_qstats and tx_qstats into the rtnl counters.
+ * Callers use ndo_get_stats64(); avoid updating netdev->stats on the
+ * xmit/poll paths to keep per-queue counters off the hot cache line.
*/
static void ibmveth_get_stats64(struct net_device *dev,
struct rtnl_link_stats64 *stats)
@@ -2779,9 +2867,14 @@ static void ibmveth_get_stats64(struct net_device *dev,
}
}
- stats->tx_packets = dev->stats.tx_packets;
- stats->tx_bytes = dev->stats.tx_bytes;
- stats->tx_dropped = dev->stats.tx_dropped;
+ if (adapter->tx_qstats) {
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ stats->tx_packets += adapter->tx_qstats[i].packets;
+ stats->tx_bytes += adapter->tx_qstats[i].bytes;
+ stats->tx_dropped += adapter->tx_qstats[i].dropped_packets;
+ }
+ }
+
stats->tx_errors = dev->stats.tx_errors;
}
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index f7b20fd01acb..390c660af979 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -316,9 +316,21 @@ struct ibmveth_rx_queue_stats {
u64 no_buffer_drops;
};
+struct ibmveth_tx_queue_stats {
+ u64 packets;
+ u64 bytes;
+ u64 large_packets;
+ u64 dropped_packets;
+ u64 send_failures;
+ u64 checksum_offload;
+};
+
#define IBMVETH_NUM_RX_QSTATS \
(sizeof(struct ibmveth_rx_queue_stats) / sizeof(u64))
+#define IBMVETH_NUM_TX_QSTATS \
+ (sizeof(struct ibmveth_tx_queue_stats) / sizeof(u64))
+
struct ibmveth_buff_pool {
u32 size;
u32 index;
@@ -386,6 +398,7 @@ struct ibmveth_adapter {
/* Multi-queue statistics */
struct ibmveth_hcall_stats hcall_stats;
struct ibmveth_rx_queue_stats *rx_qstats;
+ struct ibmveth_tx_queue_stats *tx_qstats;
/* Ethtool settings */
u8 duplex;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 12/15] ibmveth: Add helpers for incremental MQ RX queue resize
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Patches 15-17 add runtime RX queue resize via ethtool -L: single-queue
helpers here, ibmveth_resize_rx_queues_incremental() next, then ethtool
set_channels wiring.
Design: rx queue count must be changeable without a full close/open.
Close tears down the whole logical LAN (H_FREE_LOGICAL_LAN), dropping
every queue and disrupting traffic on queues that should stay up.
Incremental resize is viable because MQ PHYP registers subordinate
queues independently (H_REG_LOGICAL_LAN_QUEUE and per-queue free) while
queue 0 keeps the adapter handle; earlier per-queue bring-up helpers
already split pools, IRQs, and PHYP registration by queue index. Resize
then grows or shrinks by touching only the indices that change, leaving
surviving queues registered with buffers and IRQs intact.
This patch adds the single-queue Linux-side lifecycle helpers the resize
path calls for each new or removed index:
ibmveth_drain_rx_queue()
ibmveth_alloc_single_rx_queue()
ibmveth_free_single_rx_queue()
ibmveth_setup_single_rx_interrupt()
ibmveth_cleanup_single_rx_interrupt()
Scale-up copies pool geometry from queue 0 and uses
ibmveth_alloc_queue_buffer_pools() so only active pools are allocated
for the new queue index.
No user-visible behavior yet: helpers are added but not called until
the next patch implements ibmveth_resize_rx_queues_incremental().
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 223 +++++++++++++++++++++++++++++
1 file changed, 223 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index ecc472ee8f71..cd0acd1715da 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -589,6 +589,54 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
adapter->queue_irq[0] = 0;
}
+/**
+ * ibmveth_setup_single_rx_interrupt - Setup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to setup
+ *
+ * Registers the IRQ handler for one queue. Used during incremental
+ * scale-up when adding new RX queues; the caller enables NAPI via
+ * napi_enable() after ibmveth_enable_irq().
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_setup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ struct net_device *netdev = adapter->netdev;
+ int rc;
+
+ rc = request_irq(adapter->queue_irq[queue_idx], ibmveth_interrupt,
+ 0, netdev->name, &adapter->napi[queue_idx]);
+ if (rc) {
+ netdev_err(netdev, "request_irq() failed for queue %d: %d\n",
+ queue_idx, rc);
+ return rc;
+ }
+
+ netdev_dbg(netdev, "Setup IRQ %d for queue %d\n",
+ adapter->queue_irq[queue_idx], queue_idx);
+ return 0;
+}
+
+/**
+ * ibmveth_cleanup_single_rx_interrupt - Cleanup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to cleanup
+ *
+ * Frees the IRQ handler for one queue. Used during incremental scale-down.
+ */
+static void
+ibmveth_cleanup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ if (adapter->queue_irq[queue_idx]) {
+ free_irq(adapter->queue_irq[queue_idx], &adapter->napi[queue_idx]);
+ netdev_dbg(adapter->netdev, "Freed IRQ for queue %d\n", queue_idx);
+ }
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
@@ -1080,6 +1128,138 @@ static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
adapter->num_rx_queues);
}
+/**
+ * ibmveth_alloc_single_rx_queue - Allocate resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to allocate
+ * @rxq_entries: Number of RX queue entries
+ *
+ * Allocates buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-up without affecting existing queues.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx,
+ int rxq_entries)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+ int i, rc = -ENOMEM;
+
+ adapter->buffer_list_addr[queue_idx] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[queue_idx]) {
+ netdev_err(netdev, "unable to allocate buffer list for queue %d\n",
+ queue_idx);
+ return -ENOMEM;
+ }
+
+ adapter->rx_queue[queue_idx].queue_len =
+ sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+ adapter->rx_queue[queue_idx].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ &adapter->rx_queue[queue_idx].queue_dma,
+ GFP_KERNEL);
+ if (!adapter->rx_queue[queue_idx].queue_addr) {
+ netdev_err(netdev, "unable to allocate RX queue for queue %d\n",
+ queue_idx);
+ goto out_free_buflist;
+ }
+
+ adapter->buffer_list_dma[queue_idx] =
+ dma_map_single(dev, adapter->buffer_list_addr[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[queue_idx])) {
+ netdev_err(netdev, "unable to map buffer list for queue %d\n",
+ queue_idx);
+ goto out_free_rxq;
+ }
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ adapter->rx_buff_pool[queue_idx][i].size =
+ adapter->rx_buff_pool[0][i].size;
+ adapter->rx_buff_pool[queue_idx][i].buff_size =
+ adapter->rx_buff_pool[0][i].buff_size;
+ adapter->rx_buff_pool[queue_idx][i].threshold =
+ adapter->rx_buff_pool[0][i].threshold;
+ adapter->rx_buff_pool[queue_idx][i].active =
+ adapter->rx_buff_pool[0][i].active;
+ }
+
+ rc = ibmveth_alloc_queue_buffer_pools(adapter, queue_idx);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to allocate buffer pools for queue %d\n",
+ queue_idx);
+ goto out_unmap_buflist;
+ }
+
+ adapter->rx_queue[queue_idx].index = 0;
+ adapter->rx_queue[queue_idx].num_slots = rxq_entries;
+ adapter->rx_queue[queue_idx].toggle = 1;
+ spin_lock_init(&adapter->rx_queue[queue_idx].replenish_lock);
+
+ netdev_dbg(netdev,
+ "Allocated queue %d: buffer_list @ %p (DMA: 0x%llx), rx_queue @ %p (DMA: 0x%llx), %d entries\n",
+ queue_idx, adapter->buffer_list_addr[queue_idx],
+ (unsigned long long)adapter->buffer_list_dma[queue_idx],
+ adapter->rx_queue[queue_idx].queue_addr,
+ (unsigned long long)adapter->rx_queue[queue_idx].queue_dma,
+ rxq_entries);
+
+ return 0;
+
+out_unmap_buflist:
+ dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[queue_idx] = 0;
+out_free_rxq:
+ dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ adapter->rx_queue[queue_idx].queue_addr,
+ adapter->rx_queue[queue_idx].queue_dma);
+ adapter->rx_queue[queue_idx].queue_addr = NULL;
+out_free_buflist:
+ free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+ adapter->buffer_list_addr[queue_idx] = NULL;
+ return rc;
+}
+
+/**
+ * ibmveth_free_single_rx_queue - Free resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to free
+ *
+ * Frees buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-down without affecting remaining queues.
+ */
+static void
+ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
+{
+ struct device *dev = &adapter->vdev->dev;
+
+ ibmveth_free_queue_buffer_pools(adapter, queue_idx);
+
+ if (adapter->buffer_list_dma[queue_idx]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[queue_idx] = 0;
+ }
+
+ if (adapter->rx_queue[queue_idx].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ adapter->rx_queue[queue_idx].queue_addr,
+ adapter->rx_queue[queue_idx].queue_dma);
+ adapter->rx_queue[queue_idx].queue_addr = NULL;
+ }
+
+ if (adapter->buffer_list_addr[queue_idx]) {
+ free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+ adapter->buffer_list_addr[queue_idx] = NULL;
+ }
+
+ netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
+}
+
/**
* ibmveth_remove_buffer_from_pool - remove a buffer from a pool
* @adapter: adapter instance
@@ -1192,6 +1372,49 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
return 0;
}
+/**
+ * ibmveth_drain_rx_queue - Drain pending buffers from an RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Queue index to drain
+ *
+ * Recycles all pending buffers back to the per-queue buffer pools.
+ * Must be called with NAPI disabled for this queue.
+ *
+ * Return: Number of buffers drained
+ */
+static int
+ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
+{
+ struct net_device *netdev = adapter->netdev;
+ int drained = 0;
+ int limit = adapter->rx_queue[queue_index].num_slots;
+ int rc;
+
+ netdev_dbg(netdev, "Draining RX queue %d (limit: %d slots)\n",
+ queue_index, limit);
+
+ while (drained < limit &&
+ ibmveth_rxq_pending_buffer(adapter, queue_index)) {
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to harvest buffer from queue %d during drain: %d\n",
+ queue_index, rc);
+ break;
+ }
+ drained++;
+ }
+
+ if (drained > 0)
+ netdev_dbg(netdev, "Drained %d buffer(s) from RX queue %d\n",
+ drained, queue_index);
+ else
+ netdev_dbg(netdev, "No buffers to drain from RX queue %d\n",
+ queue_index);
+
+ return drained;
+}
+
static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
{
dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 13/15] ibmveth: Implement incremental MQ RX queue resize
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Add ibmveth_resize_rx_queues_incremental() to grow or shrink
adapter->num_rx_queues while the netdev stays up.
Scale-up, per new queue index:
alloc RX resources and per-queue pools
register subordinate queue with PHYP
request_irq(), then ibmveth_enable_irq(), then napi_enable
update num_rx_queues, replenish new queues
netif_set_real_num_rx_queues()
Scale-down disables NAPI on excess queues, drains pending buffers,
disables PHYP IRQ delivery and waits for in-flight handlers with
synchronize_irq() before lowering num_rx_queues, then tears down
IRQ/PHYP/memory.
Reject out-of-range new_count. On scale-down netif failure, re-enable
NAPI on queues not yet torn down. Refresh VIO CMO entitlement after a
successful resize when FW_FEATURE_CMO is enabled.
Scale-up rollback mirrors scale-down: drain posted buffers and wait for
in-flight handlers before deregistering with PHYP.
In replenish_task(), skip queues with queue_index >= num_rx_queues and
require pool->free_map before replenishing so in-flight handlers avoid
queues being torn down without clearing probe-time pool->active on free.
Queue 0 is never removed here. Scale-up failure unwinds only queues
added in this call. ethtool -L wiring is next.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 183 ++++++++++++++++++++++++++++-
1 file changed, 178 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index cd0acd1715da..ac4d89a66a8d 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -945,18 +945,22 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
unsigned long flags;
int i;
- if (queue_index >= adapter->num_rx_queues)
- return;
-
adapter->replenish_task_cycles++;
+ if (queue_index >= adapter->num_rx_queues) {
+ netdev_dbg(adapter->netdev,
+ "Skipping replenish for freed queue %d (num_queues=%d)\n",
+ queue_index, adapter->num_rx_queues);
+ return;
+ }
+
spin_lock_irqsave(&rxq->replenish_lock, flags);
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
struct ibmveth_buff_pool *pool =
&adapter->rx_buff_pool[queue_index][i];
- if (pool->active &&
+ if (pool->active && pool->free_map &&
(atomic_read(&pool->available) < pool->threshold))
ibmveth_replenish_buffer_pool(adapter, pool,
queue_index);
@@ -1682,7 +1686,7 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
* the IRQ mapping for subordinate queues. Queue 0 is freed only through
* ibmveth_free_all_queues() (H_FREE_LOGICAL_LAN).
*/
-static void __maybe_unused
+static void
ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
int queue_idx)
{
@@ -1714,6 +1718,175 @@ ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
}
+/**
+ * ibmveth_resize_rx_queues_incremental - Resize RX queue count incrementally
+ * @adapter: ibmveth adapter structure
+ * @new_count: Target number of RX queues
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Adds or removes RX queues without tearing down the entire adapter.
+ * Active queues continue receiving during scale-up; scale-down drains
+ * excess queues before deregistering them with the hypervisor.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
+ int new_count, int rxq_entries)
+{
+ struct net_device *netdev = adapter->netdev;
+ u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
+ int old_count = adapter->num_rx_queues;
+ int failed_queue;
+ int rc, i;
+
+ if (old_count == new_count) {
+ netdev_dbg(netdev, "RX queue count unchanged (%d), nothing to do\n",
+ old_count);
+ return 0;
+ }
+
+ if (new_count < 1 || new_count > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev, "Invalid RX queue count %d (must be 1-%d)\n",
+ new_count, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ netdev_info(netdev, "Incrementally resizing RX queues: %d to %d\n",
+ old_count, new_count);
+
+ if (new_count > old_count) {
+ netdev_dbg(netdev, "Scale-up: adding queues %d-%d\n",
+ old_count, new_count - 1);
+
+ for (i = old_count; i < new_count; i++) {
+ rc = ibmveth_alloc_single_rx_queue(adapter, i, rxq_entries);
+ if (rc) {
+ netdev_err(netdev, "Failed to allocate queue %d: %d\n",
+ i, rc);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_register_single_rx_queue(adapter, i,
+ mac_address);
+ if (rc) {
+ netdev_err(netdev, "Failed to register queue %d: %d\n",
+ i, rc);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_setup_single_rx_interrupt(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to setup IRQ for queue %d: %d\n",
+ i, rc);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_enable_irq(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d: %d\n",
+ i, rc);
+ ibmveth_cleanup_single_rx_interrupt(adapter, i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ napi_enable(&adapter->napi[i]);
+ }
+
+ adapter->num_rx_queues = new_count;
+
+ for (i = old_count; i < new_count; i++)
+ ibmveth_replenish_task(adapter, i);
+
+ rc = netif_set_real_num_rx_queues(netdev, new_count);
+ if (rc) {
+ netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+ new_count, rc);
+ goto cleanup_new_queues;
+ }
+ } else {
+ netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
+ new_count, old_count - 1);
+
+ for (i = new_count; i < old_count; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = new_count; i < old_count; i++)
+ ibmveth_drain_rx_queue(adapter, i);
+
+ synchronize_net();
+
+ rc = netif_set_real_num_rx_queues(netdev, new_count);
+ if (rc) {
+ netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+ new_count, rc);
+ for (i = new_count; i < old_count; i++)
+ napi_enable(&adapter->napi[i]);
+ return rc;
+ }
+
+ /* Disable hypervisor interrupts and wait for handlers to complete
+ * before updating num_rx_queues.
+ */
+ for (i = new_count; i < old_count; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ adapter->num_rx_queues = new_count;
+
+ for (i = new_count; i < old_count; i++) {
+ ibmveth_cleanup_single_rx_interrupt(adapter, i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ }
+ }
+
+ netdev_info(netdev, "Successfully resized to %d RX queues (incremental)\n",
+ adapter->num_rx_queues);
+
+ if (firmware_has_feature(FW_FEATURE_CMO))
+ vio_cmo_set_dev_desired(adapter->vdev,
+ ibmveth_get_desired_dma(adapter->vdev));
+
+ return 0;
+
+cleanup_new_queues:
+ failed_queue = i;
+ netdev_err(netdev,
+ "Scale-up failed at queue %d, cleaning up queues %d-%d\n",
+ failed_queue, old_count, failed_queue - 1);
+ for (i = old_count; i < failed_queue; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = old_count; i < failed_queue; i++)
+ ibmveth_drain_rx_queue(adapter, i);
+
+ synchronize_net();
+
+ for (i = old_count; i < failed_queue; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = old_count; i < failed_queue; i++) {
+ ibmveth_cleanup_single_rx_interrupt(adapter, i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ }
+ adapter->num_rx_queues = old_count;
+ netdev_warn(netdev, "Keeping %d queues after scale-up failure\n",
+ old_count);
+ return rc;
+}
+
/**
* ibmveth_free_all_queues - Free all RX queues at once
* @adapter: ibmveth adapter structure
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 14/15] ibmveth: Wire ethtool set_channels to MQ RX queue resize
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
Expose incremental RX resize through ethtool channel control.
get_channels() reports rx_count from adapter->num_rx_queues and max_rx
as IBMVETH_MAX_RX_QUEUES when MQ firmware is enabled, else 1.
set_channels() validates rx_count is within 1..IBMVETH_MAX_RX_QUEUES.
When rx_count changes and the interface is up, call
ibmveth_resize_rx_queues_incremental(). When the interface is down,
store the requested rx_count in adapter->num_rx_queues so the next open
registers that many queues. Non-MQ firmware returns -EOPNOTSUPP for
rx > 1.
TX queue changes keep existing stop/wake behavior when tx_count changes.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 58 +++++++++++++++++++++++++++---
1 file changed, 54 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index ac4d89a66a8d..50a332ab83fd 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -2534,19 +2534,69 @@ static int ibmveth_set_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- unsigned int old = netdev->real_num_tx_queues,
- goal = channels->tx_count;
+ unsigned int old_rx = adapter->num_rx_queues;
+ unsigned int goal_rx = channels->rx_count;
+ unsigned int old = netdev->real_num_tx_queues;
+ unsigned int goal = channels->tx_count;
+ int rxq_entries = adapter->rx_queue[0].num_slots;
int rc, i;
/* If ndo_open has not been called yet then don't allocate, just set
* desired netdev_queue's and return
*/
- if (!(netdev->flags & IFF_UP))
+ if (!(netdev->flags & IFF_UP)) {
+ if (goal_rx > 1 && !adapter->multi_queue) {
+ netdev_err(netdev,
+ "Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n",
+ goal_rx);
+ return -EOPNOTSUPP;
+ }
+
+ if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev,
+ "Invalid RX queue count %u (must be 1-%d)\n",
+ goal_rx, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ /* Stash desired RX count; open() publishes it via
+ * netif_set_real_num_rx_queues() after queue registration.
+ */
+ if (goal_rx != adapter->num_rx_queues)
+ adapter->num_rx_queues = goal_rx;
+
return netif_set_real_num_tx_queues(netdev, goal);
+ }
+
+ if (goal_rx > 1 && !adapter->multi_queue) {
+ netdev_err(netdev,
+ "Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n",
+ goal_rx);
+ return -EOPNOTSUPP;
+ }
+
+ if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev,
+ "Invalid RX queue count %u (must be 1-%d)\n",
+ goal_rx, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ if (goal_rx != old_rx) {
+ rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
+ rxq_entries);
+ if (rc) {
+ netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
+ return rc;
+ }
+ }
/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
* but we may need to alloc/free the ltb's.
*/
+ if (goal == old)
+ return 0;
+
netif_tx_stop_all_queues(netdev);
/* Allocate any queue that we need */
@@ -2580,7 +2630,7 @@ static int ibmveth_set_channels(struct net_device *netdev,
netif_tx_wake_all_queues(netdev);
- return rc;
+ return 0;
}
static const struct ethtool_ops netdev_ethtool_ops = {
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* [PATCH net-next v2 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
From: Mingming Cao @ 2026-07-01 22:23 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
In-Reply-To: <20260701222327.61325-1-mmc@linux.ibm.com>
After aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor
with a correlator that no longer matches the per-queue buffer pools. Poll
treated this as fatal: ibmveth_rxq_get_buffer() WARNed and returned NULL
without advancing the ring, then restart_poll retried the same slot forever.
Advance past bad correlators instead of spinning: validate correlators
without WARN_ON, skip invalid slots in poll (count as invalid_buffers),
and advance the RX ring when remove_buffer_from_pool cannot map the
correlator. Rate-limit the bad correlator message.
Complete NAPI when the interface is down or napi_disable is pending so
ibmveth_cleanup_rx_interrupts() can finish. Do not restart_poll in that
window. Close keeps hypervisor IRQ disable before napi_disable (via
cleanup_rx_interrupts()).
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 76 ++++++++++++++++++++++--------
1 file changed, 57 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 50a332ab83fd..d7bf01271161 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -158,6 +158,25 @@ static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter,
return be32_to_cpu(rxq->queue_addr[rxq->index].length);
}
+static inline bool
+ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter, int queue_index,
+ u64 correlator)
+{
+ unsigned int pool = correlator >> 32;
+ unsigned int index = correlator & 0xffffffffUL;
+
+ return pool < IBMVETH_NUM_BUFF_POOLS &&
+ index < adapter->rx_buff_pool[queue_index][pool].size;
+}
+
+static inline void ibmveth_rxq_advance(struct ibmveth_rx_q *rxq)
+{
+ if (++rxq->index == rxq->num_slots) {
+ rxq->index = 0;
+ rxq->toggle = !rxq->toggle;
+ }
+}
+
static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
int queue_index)
{
@@ -1284,17 +1303,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
unsigned int free_index;
struct sk_buff *skb;
- if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
- schedule_work(&adapter->work);
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
return -EINVAL;
- }
skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
- if (WARN_ON(!skb)) {
- schedule_work(&adapter->work);
+ if (!skb)
return -EFAULT;
- }
/* if we are going to reuse the buffer then keep the pointers around
* but mark index as available. replenish will see the skb pointer and
@@ -1335,11 +1349,8 @@ static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *ada
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
- if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
- schedule_work(&adapter->work);
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
return NULL;
- }
return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
}
@@ -1365,14 +1376,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
cor = rxq->queue_addr[rxq->index].correlator;
rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
- if (unlikely(rc))
+ if (unlikely(rc)) {
+ if (rc == -EINVAL || rc == -EFAULT)
+ goto advance;
return rc;
-
- if (++rxq->index == rxq->num_slots) {
- rxq->index = 0;
- rxq->toggle = !rxq->toggle;
}
+advance:
+ ibmveth_rxq_advance(rxq);
+
return 0;
}
@@ -2931,11 +2943,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
return 0;
+ if (!netif_running(netdev) || napi_disable_pending(napi)) {
+ napi_complete_done(napi, 0);
+ return 0;
+ }
+
if (adapter->rx_qstats)
adapter->rx_qstats[queue_index].polls++;
restart_poll:
while (frames_processed < budget) {
+ if (!netif_running(netdev) || napi_disable_pending(napi))
+ break;
+
if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
break;
@@ -2959,8 +2979,21 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
__sum16 iph_check = 0;
skb = ibmveth_rxq_get_buffer(adapter, queue_index);
- if (unlikely(!skb))
- break;
+ if (unlikely(!skb)) {
+ if (net_ratelimit())
+ netdev_err(netdev,
+ "bad correlator on queue %d, skipping slot\n",
+ queue_index);
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].invalid_buffers++;
+ else
+ adapter->rx_invalid_buffer++;
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index,
+ true);
+ if (unlikely(rc))
+ break;
+ continue;
+ }
/* if the large packet bit is set in the rx queue
* descriptor, the mss will be written by PHYP eight
@@ -3034,8 +3067,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
ibmveth_replenish_task(adapter, queue_index);
- if (frames_processed == budget)
+ if (frames_processed == budget) {
+ if (!netif_running(netdev) || napi_disable_pending(napi))
+ napi_complete_done(napi, frames_processed);
goto out;
+ }
if (!napi_complete_done(napi, frames_processed))
goto out;
@@ -3053,6 +3089,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+ netif_running(netdev) &&
+ !napi_disable_pending(napi) &&
napi_schedule(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, queue_index);
WARN_ON(lpar_rc != H_SUCCESS);
--
2.39.3 (Apple Git-146)
^ permalink raw reply related
* Re: [PATCH net-next] macsec: no longer rely on RTNL in macsec_fill_info()
From: Kuniyuki Iwashima @ 2026-07-01 22:27 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, Simon Horman,
netdev, eric.dumazet, Sabrina Dubroca, Andrew Lunn
In-Reply-To: <20260701094341.3218199-1-edumazet@google.com>
On Wed, Jul 1, 2026 at 2:43 AM Eric Dumazet <edumazet@google.com> wrote:
>
> Add READ_ONCE()/WRITE_ONCE() annotations on fields that can be
> changed concurrently in macsec_changelink() and macsec_update_offload():
>
> - secy->key_len
> - secy->xpn
> - tx_sc->encoding_sa
> - tx_sc->encrypt
> - secy->protect_frames
> - tx_sc->send_sci
> - tx_sc->end_station
> - tx_sc->scb
> - secy->replay_protect
> - secy->validate_frames
> - secy->replay_window
> - macsec->offload
>
> This allows macsec_fill_info() to run locklessly without RTNL.
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
^ permalink raw reply
* Re: [PATCH net v2] net: airoha: fix MIB stats collection to be lossless
From: Lorenzo Bianconi @ 2026-07-01 22:43 UTC (permalink / raw)
To: Aniket Negi
Cc: netdev, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, linux-arm-kernel, linux-mediatek,
linux-kernel
In-Reply-To: <20260701173941.314795-1-aniket.negi03@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 14296 bytes --]
> The current driver resets hardware MIB counters after every read via
> REG_FE_GDM_MIB_CLEAR. This creates a race window: packets arriving
> between the read and the clear are silently lost from statistics.
>
> Fix this by removing the MIB clear and switching to a delta-based
> software tracking approach:
>
> - 64-bit H+L registers (tx/rx ok pkts, ok bytes, E64..L1023):
> read the absolute hardware total directly each poll.
>
> - 32-bit registers (drops, bc, mc, errors, runt, long, ...):
> store the previous raw register value in mib_prev and accumulate
> (u32)(curr - prev) into a 64-bit software counter. Unsigned
> subtraction handles wrap-around transparently.
>
> - tx_len[0]/rx_len[0] ({0,64} RMON bucket) combines RUNT_CNT
> (32-bit, delta-tracked via mib_prev.tx_runt_cnt) and E64_CNT
> (64-bit, absolute). A u64 accumulator tx_runt_accum64 holds the
> running RUNT delta sum so that each poll sets:
> tx_len[0] = tx_runt_accum64 + E64_abs
> without double-counting the E64 value.
>
> Merge airoha_dev_get_hw_stats() into airoha_update_hw_stats(),
> moving the port spin_lock inside so callers do not need a separate
> wrapper.
>
> Signed-off-by: Aniket Negi <aniket.negi03@gmail.com>
Hi Aniket,
just few nits inline. Fixing them:
Acked-by: Lorenzo Bianconi <lorenzo@kernel.org>
> ---
>
> Changes in v2:
> - Store _CNT_L register reads in val before adding to stats, improving
> readability (suggested by Lorenzo Bianconi)
> - Fix double-counting bug in the RUNT+E64 combined bucket: previously
> "+=" for E64 re-added the full absolute counter each poll; now a
> dedicated tx_runt_accum64/rx_runt_accum64 accumulator holds the
> running RUNT delta, and tx_len[0] is assigned (not accumulated) each
> poll as runt_accum64 + E64_abs
> - Replace 7-element tx_len[]/rx_len[] shadow arrays in mib_prev with
> focused tx_runt_cnt/tx_long_cnt and rx_runt_cnt/rx_long_cnt fields;
> only RUNT and LONG are 32-bit and need wrap-around tracking
> - Rename inner struct hw_prev_stats to mib_prev; rename accumulator
> fields to tx_runt_accum64/rx_runt_accum64 for clarity
> - Fix comment alignment in mib_prev struct block
> - Rename airoha_dev_get_hw_stats() to airoha_update_hw_stats() and
> move the port spin_lock inside, removing the separate wrapper
>
> drivers/net/ethernet/airoha/airoha_eth.c | 115 +++++++++++++----------
> drivers/net/ethernet/airoha/airoha_eth.h | 27 ++++++
> 2 files changed, 92 insertions(+), 50 deletions(-)
>
> diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c
> index 59001fd4b6f7..4b7c547de165 100644
> --- a/drivers/net/ethernet/airoha/airoha_eth.c
> +++ b/drivers/net/ethernet/airoha/airoha_eth.c
> @@ -1686,12 +1686,14 @@ static void airoha_qdma_stop_napi(struct airoha_qdma *qdma)
> }
> }
>
> -static void airoha_dev_get_hw_stats(struct airoha_gdm_dev *dev)
> +static void airoha_update_hw_stats(struct airoha_gdm_dev *dev)
> {
> struct airoha_gdm_port *port = dev->port;
> struct airoha_eth *eth = dev->eth;
> u32 val, i = 0;
>
> + spin_lock(&port->stats_lock);
> +
> /* Read relevant MIB for GDM with multiple port attached */
> if (port->id == AIROHA_GDM3_IDX || port->id == AIROHA_GDM4_IDX)
> airoha_fe_rmw(eth, REG_FE_GDM_MIB_CFG(port->id),
> @@ -1701,152 +1703,165 @@ static void airoha_dev_get_hw_stats(struct airoha_gdm_dev *dev)
>
> u64_stats_update_begin(&dev->stats.syncp);
>
> - /* TX */
> + /* TX - 64-bit H+L registers: hw accumulates the total, read directly. */
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_OK_PKT_CNT_H(port->id));
> - dev->stats.tx_ok_pkts += ((u64)val << 32);
> + dev->stats.tx_ok_pkts = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_OK_PKT_CNT_L(port->id));
> dev->stats.tx_ok_pkts += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_OK_BYTE_CNT_H(port->id));
> - dev->stats.tx_ok_bytes += ((u64)val << 32);
> + dev->stats.tx_ok_bytes = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_OK_BYTE_CNT_L(port->id));
> dev->stats.tx_ok_bytes += val;
>
> + /* TX - 32-bit registers: accumulate delta to handle wrap-around. */
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_DROP_CNT(port->id));
> - dev->stats.tx_drops += val;
> + dev->stats.tx_drops += (u32)(val - dev->stats.mib_prev.tx_drops);
> + dev->stats.mib_prev.tx_drops = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_BC_CNT(port->id));
> - dev->stats.tx_broadcast += val;
> + dev->stats.tx_broadcast += (u32)(val - dev->stats.mib_prev.tx_broadcast);
> + dev->stats.mib_prev.tx_broadcast = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_MC_CNT(port->id));
> - dev->stats.tx_multicast += val;
> + dev->stats.tx_multicast += (u32)(val - dev->stats.mib_prev.tx_multicast);
> + dev->stats.mib_prev.tx_multicast = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_RUNT_CNT(port->id));
> - dev->stats.tx_len[i] += val;
> + dev->stats.mib_prev.tx_runt_accum64 +=
I guess dev->stats.mib_prev.tx_runt64
> + (u32)(val - dev->stats.mib_prev.tx_runt_cnt);
dev->stats.mib_prev.tx_runt64 += (u32)(val -
dev->stats.mib_prev.tx_runt);
> + dev->stats.mib_prev.tx_runt_cnt = val;
> +
> + /* tx_len[0]: RUNT (32-bit, delta) + E64 (64-bit, absolute) → {0, 64} bucket.
> + * Accumulate RUNT delta in tx_runt_accum64, then assign tx_len[0] as
> + * accum + E64_abs so each call gives the correct combined total.
> + */
no new-line here.
> +
> + dev->stats.tx_len[i] = dev->stats.mib_prev.tx_runt_accum64;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_E64_CNT_H(port->id));
> - dev->stats.tx_len[i] += ((u64)val << 32);
> + dev->stats.tx_len[i] += (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_E64_CNT_L(port->id));
> dev->stats.tx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L64_CNT_H(port->id));
> - dev->stats.tx_len[i] += ((u64)val << 32);
> + dev->stats.tx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L64_CNT_L(port->id));
> dev->stats.tx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L127_CNT_H(port->id));
> - dev->stats.tx_len[i] += ((u64)val << 32);
> + dev->stats.tx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L127_CNT_L(port->id));
> dev->stats.tx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L255_CNT_H(port->id));
> - dev->stats.tx_len[i] += ((u64)val << 32);
> + dev->stats.tx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L255_CNT_L(port->id));
> dev->stats.tx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L511_CNT_H(port->id));
> - dev->stats.tx_len[i] += ((u64)val << 32);
> + dev->stats.tx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L511_CNT_L(port->id));
> dev->stats.tx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L1023_CNT_H(port->id));
> - dev->stats.tx_len[i] += ((u64)val << 32);
> + dev->stats.tx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_L1023_CNT_L(port->id));
> dev->stats.tx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_TX_ETH_LONG_CNT(port->id));
> - dev->stats.tx_len[i++] += val;
> + dev->stats.tx_len[i++] += (u32)(val - dev->stats.mib_prev.tx_long_cnt);
> + dev->stats.mib_prev.tx_long_cnt = val;
>
> /* RX */
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_OK_PKT_CNT_H(port->id));
> - dev->stats.rx_ok_pkts += ((u64)val << 32);
> + dev->stats.rx_ok_pkts = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_OK_PKT_CNT_L(port->id));
> dev->stats.rx_ok_pkts += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_OK_BYTE_CNT_H(port->id));
> - dev->stats.rx_ok_bytes += ((u64)val << 32);
> + dev->stats.rx_ok_bytes = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_OK_BYTE_CNT_L(port->id));
> dev->stats.rx_ok_bytes += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_DROP_CNT(port->id));
> - dev->stats.rx_drops += val;
> + dev->stats.rx_drops += (u32)(val - dev->stats.mib_prev.rx_drops);
> + dev->stats.mib_prev.rx_drops = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_BC_CNT(port->id));
> - dev->stats.rx_broadcast += val;
> + dev->stats.rx_broadcast += (u32)(val - dev->stats.mib_prev.rx_broadcast);
> + dev->stats.mib_prev.rx_broadcast = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_MC_CNT(port->id));
> - dev->stats.rx_multicast += val;
> + dev->stats.rx_multicast += (u32)(val - dev->stats.mib_prev.rx_multicast);
> + dev->stats.mib_prev.rx_multicast = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ERROR_DROP_CNT(port->id));
> - dev->stats.rx_errors += val;
> + dev->stats.rx_errors += (u32)(val - dev->stats.mib_prev.rx_errors);
> + dev->stats.mib_prev.rx_errors = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_CRC_ERR_CNT(port->id));
> - dev->stats.rx_crc_error += val;
> + dev->stats.rx_crc_error += (u32)(val - dev->stats.mib_prev.rx_crc_error);
> + dev->stats.mib_prev.rx_crc_error = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_OVERFLOW_DROP_CNT(port->id));
> - dev->stats.rx_over_errors += val;
> + dev->stats.rx_over_errors += (u32)(val - dev->stats.mib_prev.rx_over_errors);
> + dev->stats.mib_prev.rx_over_errors = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_FRAG_CNT(port->id));
> - dev->stats.rx_fragment += val;
> + dev->stats.rx_fragment += (u32)(val - dev->stats.mib_prev.rx_fragment);
> + dev->stats.mib_prev.rx_fragment = val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_JABBER_CNT(port->id));
> - dev->stats.rx_jabber += val;
> + dev->stats.rx_jabber += (u32)(val - dev->stats.mib_prev.rx_jabber);
> + dev->stats.mib_prev.rx_jabber = val;
>
> i = 0;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_RUNT_CNT(port->id));
> - dev->stats.rx_len[i] += val;
> + dev->stats.mib_prev.rx_runt_accum64 +=
> + (u32)(val - dev->stats.mib_prev.rx_runt_cnt);
ditto.
> + dev->stats.mib_prev.rx_runt_cnt = val;
> +
> + /* rx_len[0]: RUNT (32-bit, delta) + E64 (64-bit, absolute) → {0, 64} bucket.
> + * then assign rx_len[0] = rx_runt_accum64 + E64_abs.
> + */
>
ditto.
> + dev->stats.rx_len[i] = dev->stats.mib_prev.rx_runt_accum64;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_E64_CNT_H(port->id));
> - dev->stats.rx_len[i] += ((u64)val << 32);
> + dev->stats.rx_len[i] += (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_E64_CNT_L(port->id));
> dev->stats.rx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L64_CNT_H(port->id));
> - dev->stats.rx_len[i] += ((u64)val << 32);
> + dev->stats.rx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L64_CNT_L(port->id));
> dev->stats.rx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L127_CNT_H(port->id));
> - dev->stats.rx_len[i] += ((u64)val << 32);
> + dev->stats.rx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L127_CNT_L(port->id));
> dev->stats.rx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L255_CNT_H(port->id));
> - dev->stats.rx_len[i] += ((u64)val << 32);
> + dev->stats.rx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L255_CNT_L(port->id));
> dev->stats.rx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L511_CNT_H(port->id));
> - dev->stats.rx_len[i] += ((u64)val << 32);
> + dev->stats.rx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L511_CNT_L(port->id));
> dev->stats.rx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L1023_CNT_H(port->id));
> - dev->stats.rx_len[i] += ((u64)val << 32);
> + dev->stats.rx_len[i] = (u64)val << 32;
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_L1023_CNT_L(port->id));
> dev->stats.rx_len[i++] += val;
>
> val = airoha_fe_rr(eth, REG_FE_GDM_RX_ETH_LONG_CNT(port->id));
> - dev->stats.rx_len[i++] += val;
> + dev->stats.rx_len[i] += (u32)(val - dev->stats.mib_prev.rx_long_cnt);
> + dev->stats.mib_prev.rx_long_cnt = val;
>
> u64_stats_update_end(&dev->stats.syncp);
> -}
> -
> -static void airoha_update_hw_stats(struct airoha_gdm_dev *dev)
> -{
> - struct airoha_gdm_port *port = dev->port;
> - int i;
> -
> - spin_lock(&port->stats_lock);
> -
> - for (i = 0; i < ARRAY_SIZE(port->devs); i++) {
> - if (port->devs[i])
> - airoha_dev_get_hw_stats(port->devs[i]);
> - }
> -
> - /* Reset MIB counters */
> - airoha_fe_set(dev->eth, REG_FE_GDM_MIB_CLEAR(port->id),
> - FE_GDM_MIB_RX_CLEAR_MASK | FE_GDM_MIB_TX_CLEAR_MASK);
>
> spin_unlock(&port->stats_lock);
> }
> diff --git a/drivers/net/ethernet/airoha/airoha_eth.h b/drivers/net/ethernet/airoha/airoha_eth.h
> index f6d01a8e8da1..3af1c49dd62d 100644
> --- a/drivers/net/ethernet/airoha/airoha_eth.h
> +++ b/drivers/net/ethernet/airoha/airoha_eth.h
> @@ -245,6 +245,33 @@ struct airoha_hw_stats {
> u64 rx_fragment;
> u64 rx_jabber;
> u64 rx_len[7];
> +
> + struct {
> + /* Previous HW register values for 32-bit counter delta
> + * tracking. Storing the last seen value and accumulating
> + * (u32)(curr - prev) into the 64-bit software counter
> + * handles wrap-around transparently via unsigned arithmetic.
> + * tx_runt_accum64/rx_runt_accum64 hold the running sum of
> + * runt deltas. These fields are never reported to userspace.
> + */
> + u32 tx_drops;
> + u32 tx_broadcast;
> + u32 tx_multicast;
> + u32 tx_runt_cnt;
u32 tx_runt;
> + u32 tx_long_cnt;
u32 tx_long;
> + u64 tx_runt_accum64;
64 tx_runt64;
> + u32 rx_drops;
> + u32 rx_broadcast;
> + u32 rx_multicast;
> + u32 rx_errors;
> + u32 rx_crc_error;
> + u32 rx_over_errors;
> + u32 rx_fragment;
> + u32 rx_jabber;
> + u32 rx_runt_cnt;
u32 rx_runt;
> + u32 rx_long_cnt;
u32 rx_long;
> + u64 rx_runt_accum64;
u64 rx_runt64;
> + } mib_prev;
> };
>
> enum {
>
> base-commit: a225f8c20712713406ae47024b8df42deacddd4a
> --
> 2.43.0
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply
* Re: [PATCH bpf v2 1/4] bpf, sockmap: Reject unhashed UDP sockets on sockmap update
From: John Fastabend @ 2026-07-01 22:59 UTC (permalink / raw)
To: Jakub Sitnicki
Cc: Michal Luczaj, Eric Dumazet, Kuniyuki Iwashima, Paolo Abeni,
Willem de Bruijn, Jiayuan Chen, David S. Miller, Jakub Kicinski,
Simon Horman, Alexei Starovoitov, Cong Wang, Daniel Borkmann,
Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Martin KaFai Lau, Song Liu, Yonghong Song, Jiri Olsa,
Emil Tsalapatis, Shuah Khan, netdev, bpf, linux-kernel,
linux-kselftest
In-Reply-To: <87pl1937c7.fsf@cloudflare.com>
On Mon, Jun 29, 2026 at 01:38:00PM +0200, Jakub Sitnicki wrote:
>On Fri, Jun 26, 2026 at 10:36 PM +02, Michal Luczaj wrote:
>> UDP sockets get SOCK_RCU_FREE set when (auto-)bound. This means
>> sk_is_refcounted(unbound) = true, while sk_is_refcounted(bound) = false.
>>
>> Because sockmap accepts unbound UDP sockets, a BPF program can increment a
>> socket's refcount via lookup. If the socket is subsequently bound, the
>> transition from unbound to bound causes bpf_sk_release() to skip the
>> decrement of the refcount, causing a memory leak.
>>
>> unreferenced object 0xffff88810bc2eb40 (size 1984):
>> comm "test_progs", pid 2451, jiffies 4295320596
>> hex dump (first 32 bytes):
>> 7f 00 00 01 7f 00 00 01 d2 04 1b b7 04 d2 00 00 ................
>> 02 00 01 40 00 00 00 00 00 00 00 00 00 00 00 00 ...@............
>> backtrace (crc bdee079d):
>> kmem_cache_alloc_noprof+0x557/0x660
>> sk_prot_alloc+0x69/0x240
>> sk_alloc+0x30/0x460
>> inet_create+0x2ce/0xf80
>> __sock_create+0x25b/0x5c0
>> __sys_socket+0x119/0x1d0
>> __x64_sys_socket+0x72/0xd0
>> do_syscall_64+0xa1/0x5f0
>> entry_SYSCALL_64_after_hwframe+0x76/0x7e
>>
>> Instead of special-casing for refcounted sockets, reject unhashed UDP
>> sockets during sockmap updates, as there is no benefit to supporting those.
>> This effectively reverts the commit under Fixes, with two exceptions:
>>
>> 1. sock_map_sk_state_allowed() maintains a fall-through `return true`.
>> 2. In the spirit of commit b8b8315e39ff ("bpf, sockmap: Remove unhash
>> handler for BPF sockmap usage"), the proto::unhash BPF handler is not
>> reintroduced.
>>
>> Historical note: this issue is related to commit 67312adc96b5 ("bpf: reject
>> unhashed sockets in bpf_sk_assign").
>>
>> Fixes: 0c48eefae712 ("sock_map: Lift socket state restriction for datagram sockets")
>> Suggested-by: Kuniyuki Iwashima <kuniyu@google.com>
>> Signed-off-by: Michal Luczaj <mhal@rbox.co>
>> ---
>
>Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
For me as well.
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
^ permalink raw reply
* [PATCH net-next v4 0/2] udp: fix FOU/GUE over multicast
From: Anton Danilov @ 2026-07-01 23:10 UTC (permalink / raw)
To: netdev
Cc: Willem de Bruijn, David S . Miller, David Ahern, Eric Dumazet,
Kuniyuki Iwashima, Jakub Kicinski, Paolo Abeni, Simon Horman,
Shuah Khan, linux-kselftest
UDP encapsulation (FOU, GUE) has never worked correctly with multicast
destination addresses. When a FOU-encapsulated packet arrives at a
multicast address, it enters __udp4_lib_mcast_deliver() which calls
consume_skb() on packets that need resubmission to the inner protocol
handler, silently dropping them instead.
The unicast delivery path handles this correctly by returning -ret,
but the multicast path was never updated to support UDP encapsulation
resubmit.
This causes silent packet loss for FOU/GRETAP tunnels configured with
multicast remote addresses. The loss ratio depends on the early demux
cache hit rate - packets that hit early demux bypass the multicast path
and work correctly, masking the issue.
Reproducing the issue:
ip netns add ns_a && ip netns add ns_b
ip link add veth0 type veth peer name veth1
ip link set veth0 netns ns_a && ip link set veth1 netns ns_b
ip -n ns_a addr add 10.0.0.1/24 dev veth0 && ip -n ns_a link set veth0 up
ip -n ns_b addr add 10.0.0.2/24 dev veth1 && ip -n ns_b link set veth1 up
# Multicast routes
ip -n ns_a route add 239.0.0.0/8 dev veth0
ip -n ns_b route add 239.0.0.0/8 dev veth1
# Disable early demux to expose the issue (otherwise it's partially masked)
ip netns exec ns_b sysctl -w net.ipv4.ip_early_demux=0
# Join multicast group on receiver
ip -n ns_b addr add 239.0.0.1/32 dev veth1 autojoin
# Sender: GRETAP with FOU encap
ip -n ns_a link add eoudp0 type gretap \
remote 239.0.0.1 local 10.0.0.1 \
encap fou encap-sport 4797 encap-dport 4797 key 239.0.0.1
ip -n ns_a link set eoudp0 up
ip -n ns_a addr add 192.168.99.1/24 dev eoudp0
# Receiver: FOU listener + GRETAP
ip netns exec ns_b ip fou add port 4797 ipproto 47
ip -n ns_b link add eoudp0 type gretap \
remote 239.0.0.1 local 10.0.0.2 \
encap fou encap-sport 4797 encap-dport 4797 key 239.0.0.1
ip -n ns_b link set eoudp0 up
ip -n ns_b addr add 192.168.99.2/24 dev eoudp0
# Static neigh: ARP replies can't traverse unidirectional mcast tunnel
recv_mac=$(ip -n ns_b link show eoudp0 | awk '/ether/{print $2}')
ip -n ns_a neigh add 192.168.99.2 lladdr $recv_mac dev eoudp0
# Test: ping through the FOU/GRETAP tunnel
ip netns exec ns_a ping -c 100 192.168.99.2
# -> without this patch: 0 packets received on eoudp0
# -> with this patch: all packets received on eoudp0
AI assistance (Claude, claude-opus-4-6) was used during root cause
analysis of the kernel source code (tracing the call chain from
udp_queue_rcv_skb through encap_rcv to ip_protocol_deliver_rcu,
comparing unicast/GSO/multicast paths) and during patch and selftest
authoring. The fix approach was identified by observing that the
unicast path (udp_unicast_rcv_skb) already handles encap resubmit
correctly via return -ret, while the multicast path did not.
v4:
- Promoted from RFC to PATCH; no functional changes since v3.
v3 was posted as RFC and consequently dropped from patchwork,
which explains the lack of review feedback.
v3: https://lore.kernel.org/netdev/cover.1777934869.git.littlesmilingcloud@gmail.com/
- Use return -ret instead of calling ip_protocol_deliver_rcu()
directly, matching the unicast path and avoiding call stack
growth with nested encapsulations (Kuniyuki Iwashima)
- Only change the first-socket path; the clone loop is not
reachable for tunnel sockets (no SO_REUSEADDR/SO_REUSEPORT)
- Replace Python packet generator with ping through a properly
configured FOU/GRETAP tunnel in the selftest
- Add static neighbor entry (ARP replies cannot traverse the
unidirectional multicast tunnel)
v2: https://lore.kernel.org/netdev/ad_dal164gVmImWl@dau-home-pc/
- Moved inline Python packet generator into a separate helper
- Fixed author email typo in Signed-off-by
v1 (RFC): https://lore.kernel.org/netdev/ad7MsSJOuUU6EGwS@dau-home-pc/
Anton Danilov (2):
udp: fix encapsulation packet resubmit in multicast deliver
selftests: net: add FOU multicast encapsulation resubmit test
net/ipv4/udp.c | 6 +-
net/ipv6/udp.c | 6 +-
tools/testing/selftests/net/Makefile | 1 +
.../testing/selftests/net/fou_mcast_encap.sh | 112 ++++++++++++++++++
4 files changed, 121 insertions(+), 4 deletions(-)
create mode 100755 tools/testing/selftests/net/fou_mcast_encap.sh
--
2.47.3
^ permalink raw reply
* [PATCH net-next v4 1/2] udp: fix encapsulation packet resubmit in multicast deliver
From: Anton Danilov @ 2026-07-01 23:10 UTC (permalink / raw)
To: netdev
Cc: Willem de Bruijn, David S . Miller, David Ahern, Eric Dumazet,
Kuniyuki Iwashima, Jakub Kicinski, Paolo Abeni, Simon Horman,
Shuah Khan, linux-kselftest
In-Reply-To: <cover.1782945956.git.littlesmilingcloud@gmail.com>
When a UDP encapsulation socket (e.g., FOU) receives a multicast
packet, __udp4_lib_mcast_deliver() and __udp6_lib_mcast_deliver()
call consume_skb() when udp_queue_rcv_skb() returns a positive value.
A positive return value from udp_queue_rcv_skb() indicates that the
encap_rcv handler (e.g., fou_udp_recv) has consumed the UDP header
and wants the packet to be resubmitted to the IP protocol handler
for further processing (e.g., as a GRE packet).
The unicast path in udp_unicast_rcv_skb() handles this correctly by
returning -ret, which propagates up to ip_protocol_deliver_rcu() for
resubmission. However, the multicast path destroys the packet via
consume_skb() instead of resubmitting it, causing silent packet loss.
This affects any UDP encapsulation (FOU, GUE) combined with multicast
destination addresses.
Fix this by returning -ret instead of calling consume_skb() when the
return value is positive, matching the behavior of the unicast path.
This avoids growing the call stack compared to calling
ip_protocol_deliver_rcu() directly.
Signed-off-by: Anton Danilov <littlesmilingcloud@gmail.com>
Assisted-by: Claude:claude-opus-4-6
---
net/ipv4/udp.c | 6 ++++--
net/ipv6/udp.c | 6 ++++--
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 70f6cbd4ef73..b0910659391e 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -2475,6 +2475,7 @@ static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
struct udp_hslot *hslot;
struct sk_buff *nskb;
bool use_hash2;
+ int ret;
hash2_any = 0;
hash2 = 0;
@@ -2519,8 +2520,9 @@ static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
}
if (first) {
- if (udp_queue_rcv_skb(first, skb) > 0)
- consume_skb(skb);
+ ret = udp_queue_rcv_skb(first, skb);
+ if (ret > 0)
+ return -ret;
} else {
kfree_skb(skb);
__UDP_INC_STATS(net, UDP_MIB_IGNOREDMULTI);
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 15e032194ecc..ff2e389e286b 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -949,6 +949,7 @@ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
struct udp_hslot *hslot;
struct sk_buff *nskb;
bool use_hash2;
+ int ret;
hash2_any = 0;
hash2 = 0;
@@ -998,8 +999,9 @@ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
}
if (first) {
- if (udpv6_queue_rcv_skb(first, skb) > 0)
- consume_skb(skb);
+ ret = udpv6_queue_rcv_skb(first, skb);
+ if (ret > 0)
+ return -ret;
} else {
kfree_skb(skb);
__UDP6_INC_STATS(net, UDP_MIB_IGNOREDMULTI);
--
2.47.3
^ permalink raw reply related
* [PATCH net-next v4 2/2] selftests: net: add FOU multicast encapsulation resubmit test
From: Anton Danilov @ 2026-07-01 23:10 UTC (permalink / raw)
To: netdev
Cc: Willem de Bruijn, David S . Miller, David Ahern, Eric Dumazet,
Kuniyuki Iwashima, Jakub Kicinski, Paolo Abeni, Simon Horman,
Shuah Khan, linux-kselftest
In-Reply-To: <cover.1782945956.git.littlesmilingcloud@gmail.com>
Add a selftest to verify that FOU-encapsulated packets addressed to a
multicast destination are correctly resubmitted to the inner protocol
handler (GRE) via the UDP multicast delivery path.
The test creates two network namespaces connected by a veth pair with
a FOU/GRETAP tunnel using a multicast remote address (239.0.0.1).
Ping is sent through the tunnel and received packets are counted on
the receiver's tunnel interface.
A static neighbor entry is configured on the sender because ARP
replies from the receiver cannot traverse the unidirectional multicast
tunnel back to the sender.
The early demux optimization (net.ipv4.ip_early_demux) is disabled on
the receiver to force packets through __udp4_lib_mcast_deliver(),
which is the code path being tested.
Signed-off-by: Anton Danilov <littlesmilingcloud@gmail.com>
Assisted-by: Claude:claude-opus-4-6
---
tools/testing/selftests/net/Makefile | 1 +
.../testing/selftests/net/fou_mcast_encap.sh | 112 ++++++++++++++++++
2 files changed, 113 insertions(+)
create mode 100755 tools/testing/selftests/net/fou_mcast_encap.sh
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 708d960ae07d..7e9ae937cffa 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -39,6 +39,7 @@ TEST_PROGS := \
fib_rule_tests.sh \
fib_tests.sh \
fin_ack_lat.sh \
+ fou_mcast_encap.sh \
fq_band_pktlimit.sh \
gre_gso.sh \
gre_ipv6_lladdr.sh \
diff --git a/tools/testing/selftests/net/fou_mcast_encap.sh b/tools/testing/selftests/net/fou_mcast_encap.sh
new file mode 100755
index 000000000000..8db9633f4c28
--- /dev/null
+++ b/tools/testing/selftests/net/fou_mcast_encap.sh
@@ -0,0 +1,112 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Test that UDP encapsulation (FOU) correctly handles packet resubmit
+# when packets are delivered via the multicast UDP delivery path.
+#
+# When a FOU-encapsulated packet arrives with a multicast destination IP,
+# __udp4_lib_mcast_deliver() must resubmit it to the inner protocol
+# handler (e.g., GRE) rather than consuming it. This test verifies that
+# by creating a FOU/GRETAP tunnel with a multicast remote address and
+# sending ping through it.
+#
+# The early demux optimization can mask this issue by routing packets via
+# the unicast path (udp_unicast_rcv_skb), so we disable it to force
+# packets through __udp4_lib_mcast_deliver().
+
+source lib.sh
+
+NSENDER=""
+NRECV=""
+
+cleanup() {
+ cleanup_all_ns
+}
+
+trap cleanup EXIT
+
+setup() {
+ setup_ns NSENDER NRECV
+
+ ip link add veth_s type veth peer name veth_r
+ ip link set veth_s netns "$NSENDER"
+ ip link set veth_r netns "$NRECV"
+
+ ip -n "$NSENDER" addr add 10.0.0.1/24 dev veth_s
+ ip -n "$NSENDER" link set veth_s up
+
+ ip -n "$NRECV" addr add 10.0.0.2/24 dev veth_r
+ ip -n "$NRECV" link set veth_r up
+
+ # Disable early demux to force multicast delivery path
+ ip netns exec "$NRECV" sysctl -wq net.ipv4.ip_early_demux=0
+
+ # Join multicast group on receiver
+ ip -n "$NRECV" addr add 239.0.0.1/32 dev veth_r autojoin
+
+ # Multicast routes
+ ip -n "$NRECV" route add 239.0.0.0/8 dev veth_r
+ ip -n "$NSENDER" route add 239.0.0.0/8 dev veth_s
+
+ # Sender: GRETAP with FOU encap (no FOU listener needed on TX side)
+ ip -n "$NSENDER" link add eoudp0 type gretap \
+ remote 239.0.0.1 local 10.0.0.1 \
+ encap fou encap-sport 4797 encap-dport 4797 \
+ key 239.0.0.1
+ ip -n "$NSENDER" link set eoudp0 up
+ ip -n "$NSENDER" addr add 192.168.99.1/24 dev eoudp0
+
+ # Receiver: FOU listener + GRETAP
+ ip netns exec "$NRECV" ip fou add port 4797 ipproto 47
+ ip -n "$NRECV" link add eoudp0 type gretap \
+ remote 239.0.0.1 local 10.0.0.2 \
+ encap fou encap-sport 4797 encap-dport 4797 \
+ key 239.0.0.1
+ ip -n "$NRECV" link set eoudp0 up
+ ip -n "$NRECV" addr add 192.168.99.2/24 dev eoudp0
+
+ # Static neigh entry on sender: ARP replies cannot traverse the
+ # multicast tunnel back, so pre-populate the neighbor cache.
+ local recv_mac
+ recv_mac=$(ip -n "$NRECV" link show eoudp0 | awk '/ether/{print $2}')
+ ip -n "$NSENDER" neigh add 192.168.99.2 lladdr "$recv_mac" dev eoudp0
+}
+
+get_rx_packets() {
+ ip -n "$NRECV" -s link show eoudp0 | awk '/RX:/{getline; print $2}'
+}
+
+test_fou_mcast_encap() {
+ local count=100
+ local rx_before
+ local rx_after
+ local rx_delta
+
+ # Warmup: let any initial broadcast/ARP traffic settle
+ ip netns exec "$NSENDER" ping -c 1 -W 1 192.168.99.2 >/dev/null 2>&1
+ sleep 1
+
+ rx_before=$(get_rx_packets)
+ ip netns exec "$NSENDER" ping -c $count -W 1 192.168.99.2 >/dev/null 2>&1
+ sleep 1
+ rx_after=$(get_rx_packets)
+
+ rx_delta=$((rx_after - rx_before))
+
+ if [ "$rx_delta" -ge "$count" ]; then
+ echo "PASS: received $rx_delta/$count packets via multicast FOU/GRETAP"
+ return "$ksft_pass"
+ elif [ "$rx_delta" -gt 0 ]; then
+ echo "FAIL: only $rx_delta/$count packets received (partial delivery)"
+ return "$ksft_fail"
+ else
+ echo "FAIL: 0/$count packets received (multicast encap resubmit broken)"
+ return "$ksft_fail"
+ fi
+}
+
+echo "TEST: FOU/GRETAP multicast encapsulation resubmit"
+
+setup
+test_fou_mcast_encap
+exit $?
--
2.47.3
^ permalink raw reply related
* Re: [PATCH bpf v2 4/4] selftests/bpf: Fail unbound UDP on sockmap update
From: Michal Luczaj @ 2026-07-01 23:19 UTC (permalink / raw)
To: Kuniyuki Iwashima
Cc: Eric Dumazet, Paolo Abeni, Willem de Bruijn, John Fastabend,
Jakub Sitnicki, Jiayuan Chen, David S. Miller, Jakub Kicinski,
Simon Horman, Alexei Starovoitov, Cong Wang, Daniel Borkmann,
Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Martin KaFai Lau, Song Liu, Yonghong Song, Jiri Olsa,
Emil Tsalapatis, Shuah Khan, netdev, bpf, linux-kernel,
linux-kselftest
In-Reply-To: <CAAVpQUAJYkOBEtgUpL5UgnwM9Hu3Y57a9xZWRy7zHg5XkGnW5w@mail.gmail.com>
On 6/26/26 23:03, Kuniyuki Iwashima wrote:
> On Fri, Jun 26, 2026 at 1:37 PM Michal Luczaj <mhal@rbox.co> wrote:
>>
>> sockmap now rejects unbound UDP sockets. Adjust test_maps.
>>
>> This effectively reverts commit c39aa2159974 ("bpf, selftests: Fix
>> test_maps now that sockmap supports UDP").
>>
>> Signed-off-by: Michal Luczaj <mhal@rbox.co>
>> ---
>> tools/testing/selftests/bpf/test_maps.c | 6 +++---
>> 1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
>> index c32da7bd8be2..81cd5d0d69c1 100644
>> --- a/tools/testing/selftests/bpf/test_maps.c
>> +++ b/tools/testing/selftests/bpf/test_maps.c
>> @@ -759,12 +759,12 @@ static void test_sockmap(unsigned int tasks, void *data)
>> goto out_sockmap;
>> }
>>
>> - /* Test update with unsupported UDP socket */
>> + /* Test update with unsupported unbound UDP socket */
>> udp = socket(AF_INET, SOCK_DGRAM, 0);
>> i = 0;
>> err = bpf_map_update_elem(fd, &i, &udp, BPF_ANY);
>> - if (err) {
>> - printf("Failed socket update SOCK_DGRAM '%i:%i'\n",
>> + if (!err) {
>> + printf("Failed allowed unbound SOCK_DGRAM socket update '%i:%i'\n",
>
> nit: Maybe s/Failed/Unexpectedly succeeded/ ?
Sure. I've tried to align with the other printfs ("failed allowed..."), but
I agree it was unclear.
> If we want to avoid breakage, this patch needs to be squashed to
> the fix patch, but it's discouraged in netdev, not sure about bpf tree.
I guess I'll just post v3 as is, and squash if requested.
thanks,
Michal
^ permalink raw reply
* Re: Ethtool : PRBS feature
From: Lee Trager @ 2026-07-01 23:28 UTC (permalink / raw)
To: Andrew Lunn, Srinivasan, Vijay
Cc: Das, Shubham, Alexander Duyck, Maxime Chevallier,
netdev@vger.kernel.org, mkubecek@suse.cz, D H, Siddaraju,
Chintalapalle, Balaji, Lindberg, Magnus,
niklas.damberg@ericsson.com, Wirandi, Jonas
In-Reply-To: <26d44b09-d72c-4e23-84a6-3d0ea4a521ef@lunn.ch>
On 7/1/26 3:02 PM, Andrew Lunn wrote:
> On Wed, Jul 01, 2026 at 09:38:08PM +0000, Srinivasan, Vijay wrote:
>> Hi Andrew,
>> I think there is a disconnect here.
> Which proves my point. The specification is not sufficient if you have
> to keep correcting me.
>
> The kAPI should be understandable by somebody who has a general
> networking background. Please write a specification with that
> assumption in mind. Don't assume the reader is a test engineer who has
> used PRBS for half his life. Assume it is a brand new test engineer
> who is hearing PRBS for the first time. That is what most engineers on
> the netdev list are. Me included.
I think part of the disconnect is that PRBS testing is a signal
integrity test, not a network test. In this case the phy happens to be
Ethernet but it could just as easily be PCIE or USB. That is why it was
heavily suggested to me at netdev 0x19 that this should be done on the
generic phy layer, not netdev.
Lee
^ permalink raw reply
* [PATCH bpf v3 3/4] selftests/bpf: Adapt sockmap update error handling
From: Michal Luczaj @ 2026-07-01 23:28 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
John Fastabend, Jakub Sitnicki, Jiayuan Chen, Eric Dumazet,
Kuniyuki Iwashima, Paolo Abeni, Willem de Bruijn, David S. Miller,
Jakub Kicinski, Simon Horman, Cong Wang
Cc: Michal Luczaj, bpf, linux-kselftest, linux-kernel, netdev
In-Reply-To: <20260702-sockmap-lookup-udp-leak-v3-0-ff8de8782468@rbox.co>
Update sockmap_listen to accommodate the recent change in sockmap that
rejects unbound UDP sockets.
TCP: Reject unbound and bound (unless established or listening).
UDP: Accept only bound sockets.
While at it, migrate to ASSERT_* and enforce reverse xmas tree.
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Michal Luczaj <mhal@rbox.co>
---
.../selftests/bpf/prog_tests/sockmap_listen.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c b/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c
index cc0c68bab907..b87118aab7c4 100644
--- a/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c
+++ b/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c
@@ -53,8 +53,8 @@ static void test_insert_opened(struct test_sockmap_listen *skel __always_unused,
int family, int sotype, int mapfd)
{
u32 key = 0;
- u64 value;
int err, s;
+ u64 value;
s = xsocket(family, sotype, 0);
if (s == -1)
@@ -63,11 +63,8 @@ static void test_insert_opened(struct test_sockmap_listen *skel __always_unused,
errno = 0;
value = s;
err = bpf_map_update_elem(mapfd, &key, &value, BPF_NOEXIST);
- if (sotype == SOCK_STREAM) {
- if (!err || errno != EOPNOTSUPP)
- FAIL_ERRNO("map_update: expected EOPNOTSUPP");
- } else if (err)
- FAIL_ERRNO("map_update: expected success");
+ ASSERT_ERR(err, "map_update");
+ ASSERT_EQ(errno, EOPNOTSUPP, "errno");
xclose(s);
}
@@ -77,8 +74,8 @@ static void test_insert_bound(struct test_sockmap_listen *skel __always_unused,
struct sockaddr_storage addr;
socklen_t len = 0;
u32 key = 0;
- u64 value;
int err, s;
+ u64 value;
init_addr_loopback(family, &addr, &len);
@@ -93,8 +90,12 @@ static void test_insert_bound(struct test_sockmap_listen *skel __always_unused,
errno = 0;
value = s;
err = bpf_map_update_elem(mapfd, &key, &value, BPF_NOEXIST);
- if (!err || errno != EOPNOTSUPP)
- FAIL_ERRNO("map_update: expected EOPNOTSUPP");
+ if (sotype == SOCK_STREAM) {
+ ASSERT_ERR(err, "map_update");
+ ASSERT_EQ(errno, EOPNOTSUPP, "errno");
+ } else if (err) {
+ ASSERT_OK(err, "map_update");
+ }
close:
xclose(s);
}
@@ -1289,7 +1290,7 @@ static void test_ops(struct test_sockmap_listen *skel, struct bpf_map *map,
/* insert */
TEST(test_insert_invalid),
TEST(test_insert_opened),
- TEST(test_insert_bound, SOCK_STREAM),
+ TEST(test_insert_bound),
TEST(test_insert),
/* delete */
TEST(test_delete_after_insert),
--
2.54.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox