* [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support
@ 2026-07-06 19:35 Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
` (14 more replies)
0 siblings, 15 replies; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe
Hi,
Power11 PHYP firmware adds Virtual Ethernet multi-queue (MQ) RX for
the ibmveth device: multiple logical-LAN RX queues, per-queue buffer
posting, and completion delivery. Guest Linux did not use that
platform support; ibmveth still registered one RX queue even when
PHYP was MQ-capable.
This series adds the ibmveth MQ client. When PHYP advertises the
capability through H_ILLAN_ATTRIBUTES, the driver registers
multiple RX queues, receives on per-queue NAPI, and exposes queue
count through ethtool. Older firmware without the bit is unchanged.
Please apply to net-next.
Background
ibmveth today registers one logical LAN, one set of buffer pools, and
one NAPI context. PHYP MQ mode gives each RX queue its own handle:
buffers are posted with H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, subordinate
queues register through H_REG_LOGICAL_LAN_QUEUE, and traffic can
land on any active queue. Queue selection is firmware-defined; v1
does not program RSS or hash tables. The driver needs per-queue
pools, IRQs, and poll state to match.
Queue-aware hcalls are selected only when probe sets multi_queue
from H_ILLAN_ATTRIBUTES; legacy firmware keeps the original hcall
path unchanged through the entire series.
This splits the work so review follows the actual bring-up sequence:
1. Hypercall definitions and MQ data structures (patches 1-2)
2. Refactor open/close into helpers - RX, per-queue pools,
IRQ, TX, PHYP (3-9)
3. Turn on the MQ datapath at probe/open (10)
4. Per-queue RX/TX stats and sysfs pool readout (11-12)
5. Runtime RX queue resize via ethtool -L (13-14)
6. LPAR stability fix (15)
- Helper patches (3-9) reshape ibmveth_open()/close() into
queue-aware helpers. Runtime behaviour is unchanged through that
block: num_rx_queues stays 1 and multi_queue is false until patch 10.
- Patch 10 is the switch: probe sets multi_queue from firmware, raises
num_rx_queues, registers subordinates, and replenishes every active
queue.
- Patch 15 fixes poll hangs after aggressive ethtool -L cycling and
NAPI/close deadlocks on ip link down.
Testing
Tested on ppc64le PowerVM LPAR with MQ-capable firmware:
* Aggressive ethtool -L cycling (16/1/8/11/1/3/16/8/1) with ping
* MQ path: ethtool -L under iperf3 load, link down/up during traffic
* Legacy firmware (no MQ bit): full open/close/stress on the
refactored helper path to confirm single-queue behaviour is
unchanged
Changes in v3
v2 resubmit with complete patch series: v2 accidentally omitted patches
1-2 (hypercall definitions and MQ structures). This v3 includes all 15
patches in correct order. No code changes from v2; same LPAR testing
results. v2: https://lore.kernel.org/r/20260701222327.61325-1-mmc@linux.ibm.com
Changes in v2
v1 resubmit as 15 patches (Patchwork limit): same code and LPAR testing;
squashed split plus checkpatch fixes in patch 15 only.
v1: https://lore.kernel.org/r/cover.1782758799.git.mmc@linux.ibm.com
Patchwork: https://patchwork.kernel.org/project/netdevbpf/list/?series=1119106
Future work
* IRQ affinity hints for subordinate queue IRQs returned by PHYP
* Summed global no_buffer drop counter across all RX queues in MQ mode
Comments and suggestions on patch split, design, and testing are
welcome.
Mingming Cao <mmc@linux.ibm.com>
Mingming Cao (15):
ibmveth: Add MQ RX hypercall wrappers and call definitions
ibmveth: Prepare MQ RX adapter and statistics structures
ibmveth: Refactor RX resource allocation for MQ RX bring-up
ibmveth: Refactor buffer pool management for per-queue MQ RX
ibmveth: Refactor RX interrupt control for MQ RX queues
ibmveth: Refactor TX resource allocation in open/close paths
ibmveth: Add RX queue register/deregister helpers for MQ
ibmveth: Refactor open/close into MQ-ready resource pipeline
ibmveth: Add queue-aware RX buffer submit helper for MQ
ibmveth: Enable multi-queue RX receive path
ibmveth: Add per-queue RX and TX statistics collection and reporting
ibmveth: Expose per-queue buffer pool details via sysfs
ibmveth: Implement incremental MQ RX queue resize
ibmveth: Wire ethtool set_channels to MQ RX queue resize
ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
arch/powerpc/include/asm/hvcall.h | 6 +-
drivers/net/ethernet/ibm/ibmveth.c | 2452 +++++++++++++++++++++++-----
drivers/net/ethernet/ibm/ibmveth.h | 226 ++-
3 files changed, 2285 insertions(+), 399 deletions(-)
--
2.39.3 (Apple Git-146)
# Made with Bob
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH net-next v3 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 02/15] ibmveth: Prepare MQ RX adapter and statistics structures Mingming Cao
` (13 subsequent siblings)
14 siblings, 0 replies; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Single-queue ibmveth only needs h_register_logical_lan() plus legacy
buffer add/free calls. MQ RX uses per-queue handles, so the driver must
also be able to register/deregister subordinate queues and post/free
buffers against a specific queue handle.
Add the PHYP call IDs for:
H_REG_LOGICAL_LAN_QUEUE
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
H_FREE_LOGICAL_LAN_BUFFER_QUEUE
H_FREE_LOGICAL_LAN_QUEUE
and add ibmveth.h wrapper helpers (h_reg_logical_lan_queue(),
h_add_logical_lan_buffers_queue(), h_free_logical_lan_buffer_queue(),
h_free_logical_lan_queue()) with argument ordering and return semantics
matching the existing ibmveth hcall wrappers.
This patch is intentionally plumbing only: no runtime behavior change
yet. Legacy firmware keeps H_REGISTER_LOGICAL_LAN and the existing
buffer hcalls. The new wrappers are used only when a later commit sets
multi_queue from H_ILLAN_ATTRIBUTES.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
arch/powerpc/include/asm/hvcall.h | 6 +-
drivers/net/ethernet/ibm/ibmveth.c | 3 +-
drivers/net/ethernet/ibm/ibmveth.h | 158 +++++++++++++++++++++++++++++
3 files changed, 165 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
index dff90a7d7f70..bf2f1b0356c4 100644
--- a/arch/powerpc/include/asm/hvcall.h
+++ b/arch/powerpc/include/asm/hvcall.h
@@ -362,7 +362,11 @@
#define H_GUEST_DELETE 0x488
#define H_PKS_WRAP_OBJECT 0x490
#define H_PKS_UNWRAP_OBJECT 0x494
-#define MAX_HCALL_OPCODE H_PKS_UNWRAP_OBJECT
+#define H_REG_LOGICAL_LAN_QUEUE 0x49C
+#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0
+#define H_FREE_LOGICAL_LAN_BUFFER_QUEUE 0x4A4
+#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8
+#define MAX_HCALL_OPCODE H_FREE_LOGICAL_LAN_QUEUE
/* Scope args for H_SCM_UNBIND_ALL */
#define H_UNBIND_SCOPE_ALL (0x1)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 73e051d26b9d..af287eeafc0c 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -584,7 +584,8 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
}
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
- union ibmveth_buf_desc rxq_desc, u64 mac_address)
+ union ibmveth_buf_desc rxq_desc,
+ u64 mac_address)
{
int rc, try_again = 1;
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index d87713668ed3..45cfb0d054e3 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -66,6 +66,164 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address,
desc5, desc6, desc7, desc8);
}
+/**
+ * h_reg_logical_lan_queue - Register a subordinate receive queue
+ * @unit_address: Device unit address
+ * @buffer_list: DMA address of 4KB page for tracking registered buffers
+ * @rec_queue: Buffer descriptor of receive queue
+ *
+ * Registers a subordinate receive queue with the hypervisor.
+ *
+ * Return:
+ * H_SUCCESS (0) on success
+ * H_PARAMETER if parameters are invalid
+ *
+ * On success, hypervisor returns:
+ * R3: H_SUCCESS
+ * R4: Queue handle
+ * R5: IRQ number for this queue
+ */
+static inline long h_reg_logical_lan_queue(unsigned long unit_address,
+ unsigned long buffer_list,
+ unsigned long rec_queue,
+ unsigned long *queue_handle,
+ unsigned long *irq)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+ long rc;
+
+ rc = plpar_hcall9(H_REG_LOGICAL_LAN_QUEUE,
+ retbuf, unit_address,
+ buffer_list, rec_queue);
+
+ if (rc == H_SUCCESS) {
+ if (queue_handle)
+ *queue_handle = retbuf[0];
+ if (irq)
+ *irq = retbuf[1];
+ }
+
+ return rc;
+}
+
+/**
+ * h_add_logical_lan_buffers_queue - Add buffers to subordinate queue
+ * @unit_address: Device unit address
+ * @queue_handle: Queue handle from h_reg_logical_lan_queue()
+ * @buffersznum: Buffer size (upper 32 bits) | count (lower 32 bits)
+ * @ioba12: Buffer addresses 1 and 2 packed (addr1 | addr2 << 32)
+ * @ioba34: Buffer addresses 3 and 4 packed
+ * @ioba56: Buffer addresses 5 and 6 packed
+ * @ioba78: Buffer addresses 7 and 8 packed
+ * @ioba910: Buffer addresses 9 and 10 packed
+ * @ioba1112: Buffer addresses 11 and 12 packed
+ *
+ * Return:
+ * H_SUCCESS - All buffers added successfully
+ * H_PARAMETER - Invalid parameters
+ * H_HARDWARE - Hardware error
+ */
+static inline long h_add_logical_lan_buffers_queue(unsigned long unit_address,
+ unsigned long queue_handle,
+ unsigned long buffersznum,
+ unsigned long ioba12,
+ unsigned long ioba34,
+ unsigned long ioba56,
+ unsigned long ioba78,
+ unsigned long ioba910,
+ unsigned long ioba1112)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+
+ return plpar_hcall9(H_ADD_LOGICAL_LAN_BUFFERS_QUEUE,
+ retbuf, unit_address,
+ queue_handle, buffersznum,
+ ioba12, ioba34, ioba56,
+ ioba78, ioba910, ioba1112);
+}
+
+/**
+ * h_free_logical_lan_buffer_queue - Free buffer from subordinate queue
+ * @unit_address: Device unit address
+ * @buf_size: Size of buffer to remove from pool
+ * @queue_handle: Queue handle from h_reg_logical_lan_queue()
+ *
+ * Removes a buffer of specified size from the subordinate queue's buffer pool.
+ *
+ * Return:
+ * H_SUCCESS - Buffer removed successfully
+ * H_PARAMETER - Invalid parameters
+ * H_HARDWARE - Hardware error
+ * H_NOT_FOUND - Buffer pool does not exist
+ */
+static inline long h_free_logical_lan_buffer_queue(unsigned long unit_address,
+ unsigned long buf_size,
+ unsigned long queue_handle)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+
+ return plpar_hcall9(H_FREE_LOGICAL_LAN_BUFFER_QUEUE,
+ retbuf, unit_address, buf_size, queue_handle);
+}
+
+/**
+ * h_free_logical_lan_queue - Deregister subordinate receive queue
+ * @unit_address: Device unit address
+ * @queue_handle: Queue handle from h_reg_logical_lan_queue()
+ *
+ * Deregisters and frees all structures associated with the subordinate queue.
+ *
+ * Return:
+ * H_SUCCESS - Queue freed successfully
+ * H_PARAMETER - Invalid parameters
+ * H_HARDWARE - Hardware error
+ * H_STATE - VIOA not in valid state
+ * H_BUSY / H_LONG_BUSY_* - Resource busy, retry
+ */
+static inline long h_free_logical_lan_queue(unsigned long unit_address,
+ unsigned long queue_handle)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+
+ return plpar_hcall9(H_FREE_LOGICAL_LAN_QUEUE,
+ retbuf, unit_address, queue_handle);
+}
+
+/**
+ * h_register_logical_lan_with_handle - Register primary queue and get handle
+ * @unit_address: Device unit address
+ * @buffer_list: DMA address of buffer list
+ * @rec_queue: Buffer descriptor of receive queue
+ * @filter_list: DMA address of filter list
+ * @mac_address: MAC address
+ * @queue_handle: Output parameter for queue handle
+ *
+ * Registers the primary receive queue (queue 0) with the hypervisor and
+ * returns the queue handle. This is needed in multi-queue mode to use
+ * h_add_logical_lan_buffers_queue() for all queues including queue 0.
+ *
+ * Return: H_SUCCESS (0) on success, error code otherwise
+ */
+static inline long h_register_logical_lan_with_handle(unsigned long unit_address,
+ unsigned long buffer_list,
+ unsigned long rec_queue,
+ unsigned long filter_list,
+ unsigned long mac_address,
+ u64 *queue_handle)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+ long rc;
+
+ rc = plpar_hcall9(H_REGISTER_LOGICAL_LAN, retbuf,
+ unit_address, buffer_list, rec_queue,
+ filter_list, mac_address);
+
+ if (rc == H_SUCCESS && queue_handle)
+ *queue_handle = retbuf[0];
+
+ return rc;
+}
+
/* FW allows us to send 6 descriptors but we only use one so mark
* the other 5 as unused (0)
*/
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 02/15] ibmveth: Prepare MQ RX adapter and statistics structures
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
` (12 subsequent siblings)
14 siblings, 0 replies; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Extend ibmveth_adapter for multi-queue RX: per-queue buffer pools,
queue handles, NAPI/IRQ arrays, and MQ capability flags. Add
rx_qstats[] layout and related constants for per-queue statistics
(reporting wired in a later patch).
No runtime behaviour change; num_rx_queues remains 1 until the MQ
receive path is enabled.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 222 +++++++++++++++++------------
drivers/net/ethernet/ibm/ibmveth.h | 45 +++++-
2 files changed, 168 insertions(+), 99 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index af287eeafc0c..8f9f927bff23 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -101,7 +101,7 @@ static struct ibmveth_stat ibmveth_stats[] = {
/* simple methods of getting data from the current rxq entry */
static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
{
- return be32_to_cpu(adapter->rx_queue.queue_addr[adapter->rx_queue.index].flags_off);
+ return be32_to_cpu(adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].flags_off);
}
static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
@@ -112,7 +112,7 @@ static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
{
- return ibmveth_rxq_toggle(adapter) == adapter->rx_queue.toggle;
+ return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
}
static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
@@ -132,7 +132,7 @@ static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
{
- return be32_to_cpu(adapter->rx_queue.queue_addr[adapter->rx_queue.index].length);
+ return be32_to_cpu(adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].length);
}
static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
@@ -386,7 +386,7 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
*/
static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
{
- __be64 *p = adapter->buffer_list_addr + 4096 - 8;
+ __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
adapter->rx_no_buffer = be64_to_cpup(p);
}
@@ -399,7 +399,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
adapter->replenish_task_cycles++;
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
- struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[i];
+ struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
if (pool->active &&
(atomic_read(&pool->available) < pool->threshold))
@@ -463,12 +463,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
struct sk_buff *skb;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
schedule_work(&adapter->work);
return -EINVAL;
}
- skb = adapter->rx_buff_pool[pool].skbuff[index];
+ skb = adapter->rx_buff_pool[0][pool].skbuff[index];
if (WARN_ON(!skb)) {
schedule_work(&adapter->work);
return -EFAULT;
@@ -482,24 +482,24 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* remove the skb pointer to mark free. actual freeing is done
* by upper level networking after gro_receive
*/
- adapter->rx_buff_pool[pool].skbuff[index] = NULL;
+ adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
dma_unmap_single(&adapter->vdev->dev,
- adapter->rx_buff_pool[pool].dma_addr[index],
- adapter->rx_buff_pool[pool].buff_size,
+ adapter->rx_buff_pool[0][pool].dma_addr[index],
+ adapter->rx_buff_pool[0][pool].buff_size,
DMA_FROM_DEVICE);
}
- free_index = adapter->rx_buff_pool[pool].producer_index;
- adapter->rx_buff_pool[pool].producer_index++;
- if (adapter->rx_buff_pool[pool].producer_index >=
- adapter->rx_buff_pool[pool].size)
- adapter->rx_buff_pool[pool].producer_index = 0;
- adapter->rx_buff_pool[pool].free_map[free_index] = index;
+ free_index = adapter->rx_buff_pool[0][pool].producer_index;
+ adapter->rx_buff_pool[0][pool].producer_index++;
+ if (adapter->rx_buff_pool[0][pool].producer_index >=
+ adapter->rx_buff_pool[0][pool].size)
+ adapter->rx_buff_pool[0][pool].producer_index = 0;
+ adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
mb();
- atomic_dec(&(adapter->rx_buff_pool[pool].available));
+ atomic_dec(&adapter->rx_buff_pool[0][pool].available);
return 0;
}
@@ -507,17 +507,17 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* get the current buffer on the rx queue */
static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
{
- u64 correlator = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator;
+ u64 correlator = adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].correlator;
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
schedule_work(&adapter->work);
return NULL;
}
- return adapter->rx_buff_pool[pool].skbuff[index];
+ return adapter->rx_buff_pool[0][pool].skbuff[index];
}
/**
@@ -538,14 +538,14 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
u64 cor;
int rc;
- cor = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator;
+ cor = adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].correlator;
rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
if (unlikely(rc))
return rc;
- if (++adapter->rx_queue.index == adapter->rx_queue.num_slots) {
- adapter->rx_queue.index = 0;
- adapter->rx_queue.toggle = !adapter->rx_queue.toggle;
+ if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
+ adapter->rx_queue[0].index = 0;
+ adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
}
return 0;
@@ -596,7 +596,7 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
*/
retry:
rc = h_register_logical_lan(adapter->vdev->unit_address,
- adapter->buffer_list_dma, rxq_desc.desc,
+ adapter->buffer_list_dma[0], rxq_desc.desc,
adapter->filter_list_dma, mac_address);
if (rc != H_SUCCESS && try_again) {
@@ -611,6 +611,33 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
return rc;
}
+/**
+ * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int __maybe_unused ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
+ sizeof(struct ibmveth_rx_queue_stats),
+ GFP_KERNEL);
+ if (!adapter->rx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_rx_qstats - Free per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void __maybe_unused ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->rx_qstats);
+ adapter->rx_qstats = NULL;
+}
+
static int ibmveth_open(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
@@ -624,14 +651,14 @@ static int ibmveth_open(struct net_device *netdev)
netdev_dbg(netdev, "open starting\n");
- napi_enable(&adapter->napi);
+ napi_enable(&adapter->napi[0]);
for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- rxq_entries += adapter->rx_buff_pool[i].size;
+ rxq_entries += adapter->rx_buff_pool[0][i].size;
rc = -ENOMEM;
- adapter->buffer_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
- if (!adapter->buffer_list_addr) {
+ adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[0]) {
netdev_err(netdev, "unable to allocate list pages\n");
goto out;
}
@@ -644,17 +671,18 @@ static int ibmveth_open(struct net_device *netdev)
dev = &adapter->vdev->dev;
- adapter->rx_queue.queue_len = sizeof(struct ibmveth_rx_q_entry) *
+ adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
rxq_entries;
- adapter->rx_queue.queue_addr =
- dma_alloc_coherent(dev, adapter->rx_queue.queue_len,
- &adapter->rx_queue.queue_dma, GFP_KERNEL);
- if (!adapter->rx_queue.queue_addr)
+ adapter->rx_queue[0].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
+ &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
+ if (!adapter->rx_queue[0].queue_addr)
goto out_free_filter_list;
- adapter->buffer_list_dma = dma_map_single(dev,
- adapter->buffer_list_addr, 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->buffer_list_dma)) {
+ adapter->buffer_list_dma[0] = dma_map_single(dev,
+ adapter->buffer_list_addr[0],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
netdev_err(netdev, "unable to map buffer list pages\n");
goto out_free_queue_mem;
}
@@ -671,19 +699,19 @@ static int ibmveth_open(struct net_device *netdev)
goto out_free_tx_ltb;
}
- adapter->rx_queue.index = 0;
- adapter->rx_queue.num_slots = rxq_entries;
- adapter->rx_queue.toggle = 1;
+ adapter->rx_queue[0].index = 0;
+ adapter->rx_queue[0].num_slots = rxq_entries;
+ adapter->rx_queue[0].toggle = 1;
mac_address = ether_addr_to_u64(netdev->dev_addr);
rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
- adapter->rx_queue.queue_len;
- rxq_desc.fields.address = adapter->rx_queue.queue_dma;
+ adapter->rx_queue[0].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
- netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr);
+ netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr[0]);
netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
- netdev_dbg(netdev, "receive q @ 0x%p\n", adapter->rx_queue.queue_addr);
+ netdev_dbg(netdev, "receive q @ 0x%p\n", adapter->rx_queue[0].queue_addr);
h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
@@ -694,7 +722,7 @@ static int ibmveth_open(struct net_device *netdev)
lpar_rc);
netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
"desc:0x%llx MAC:0x%llx\n",
- adapter->buffer_list_dma,
+ adapter->buffer_list_dma[0],
adapter->filter_list_dma,
rxq_desc.desc,
mac_address);
@@ -703,11 +731,11 @@ static int ibmveth_open(struct net_device *netdev)
}
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (!adapter->rx_buff_pool[i].active)
+ if (!adapter->rx_buff_pool[0][i].active)
continue;
- if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[i])) {
+ if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
netdev_err(netdev, "unable to alloc pool\n");
- adapter->rx_buff_pool[i].active = 0;
+ adapter->rx_buff_pool[0][i].active = 0;
rc = -ENOMEM;
goto out_free_buffer_pools;
}
@@ -739,9 +767,9 @@ static int ibmveth_open(struct net_device *netdev)
out_free_buffer_pools:
while (--i >= 0) {
- if (adapter->rx_buff_pool[i].active)
+ if (adapter->rx_buff_pool[0][i].active)
ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[i]);
+ &adapter->rx_buff_pool[0][i]);
}
out_unmap_filter_list:
dma_unmap_single(dev, adapter->filter_list_dma, 4096,
@@ -753,18 +781,18 @@ static int ibmveth_open(struct net_device *netdev)
}
out_unmap_buffer_list:
- dma_unmap_single(dev, adapter->buffer_list_dma, 4096,
+ dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
DMA_BIDIRECTIONAL);
out_free_queue_mem:
- dma_free_coherent(dev, adapter->rx_queue.queue_len,
- adapter->rx_queue.queue_addr,
- adapter->rx_queue.queue_dma);
+ dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
+ adapter->rx_queue[0].queue_addr,
+ adapter->rx_queue[0].queue_dma);
out_free_filter_list:
free_page((unsigned long)adapter->filter_list_addr);
out_free_buffer_list:
- free_page((unsigned long)adapter->buffer_list_addr);
+ free_page((unsigned long)adapter->buffer_list_addr[0]);
out:
- napi_disable(&adapter->napi);
+ napi_disable(&adapter->napi[0]);
return rc;
}
@@ -777,7 +805,7 @@ static int ibmveth_close(struct net_device *netdev)
netdev_dbg(netdev, "close starting\n");
- napi_disable(&adapter->napi);
+ napi_disable(&adapter->napi[0]);
netif_tx_stop_all_queues(netdev);
@@ -796,22 +824,22 @@ static int ibmveth_close(struct net_device *netdev)
ibmveth_update_rx_no_buffer(adapter);
- dma_unmap_single(dev, adapter->buffer_list_dma, 4096,
+ dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->buffer_list_addr);
+ free_page((unsigned long)adapter->buffer_list_addr[0]);
dma_unmap_single(dev, adapter->filter_list_dma, 4096,
DMA_BIDIRECTIONAL);
free_page((unsigned long)adapter->filter_list_addr);
- dma_free_coherent(dev, adapter->rx_queue.queue_len,
- adapter->rx_queue.queue_addr,
- adapter->rx_queue.queue_dma);
+ dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
+ adapter->rx_queue[0].queue_addr,
+ adapter->rx_queue[0].queue_dma);
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (adapter->rx_buff_pool[i].active)
+ if (adapter->rx_buff_pool[0][i].active)
ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[i]);
+ &adapter->rx_buff_pool[0][i]);
for (i = 0; i < netdev->real_num_tx_queues; i++)
ibmveth_free_tx_ltb(adapter, i);
@@ -1449,7 +1477,7 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
static int ibmveth_poll(struct napi_struct *napi, int budget)
{
struct ibmveth_adapter *adapter =
- container_of(napi, struct ibmveth_adapter, napi);
+ container_of(napi, struct ibmveth_adapter, napi[0]);
struct net_device *netdev = adapter->netdev;
int frames_processed = 0;
unsigned long lpar_rc;
@@ -1574,11 +1602,11 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
struct ibmveth_adapter *adapter = netdev_priv(netdev);
unsigned long lpar_rc;
- if (napi_schedule_prep(&adapter->napi)) {
+ if (napi_schedule_prep(&adapter->napi[0])) {
lpar_rc = h_vio_signal(adapter->vdev->unit_address,
VIO_IRQ_DISABLE);
WARN_ON(lpar_rc != H_SUCCESS);
- __napi_schedule(&adapter->napi);
+ __napi_schedule(&adapter->napi[0]);
}
return IRQ_HANDLED;
}
@@ -1646,7 +1674,7 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
int need_restart = 0;
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size)
+ if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size)
break;
if (i == IBMVETH_NUM_BUFF_POOLS)
@@ -1661,9 +1689,9 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
/* Look for an active buffer pool that can hold the new MTU */
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- adapter->rx_buff_pool[i].active = 1;
+ adapter->rx_buff_pool[0][i].active = 1;
- if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size) {
+ if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size) {
WRITE_ONCE(dev->mtu, new_mtu);
vio_cmo_set_dev_desired(viodev,
ibmveth_get_desired_dma
@@ -1721,12 +1749,12 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
/* add the size of the active receive buffers */
- if (adapter->rx_buff_pool[i].active)
+ if (adapter->rx_buff_pool[0][i].active)
ret +=
- adapter->rx_buff_pool[i].size *
- IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[i].
+ adapter->rx_buff_pool[0][i].size *
+ IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
buff_size, tbl);
- rxqentries += adapter->rx_buff_pool[i].size;
+ rxqentries += adapter->rx_buff_pool[0][i].size;
}
/* add the size of the receive queue entries */
ret += IOMMU_PAGE_ALIGN(
@@ -1845,7 +1873,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
ibmveth_init_link_settings(netdev);
- netif_napi_add_weight(netdev, &adapter->napi, ibmveth_poll, 16);
+ netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
netdev->irq = dev->irq;
netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -1877,6 +1905,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev->features |= NETIF_F_FRAGLIST;
}
+ /* Initialize queue count - always 1 for now */
+ adapter->multi_queue = 0;
+ adapter->num_rx_queues = 1;
+
if (ret == H_SUCCESS &&
(ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
@@ -1899,10 +1931,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
memcpy(pool_count, pool_count_cmo, sizeof(pool_count));
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- struct kobject *kobj = &adapter->rx_buff_pool[i].kobj;
+ struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
int error;
- ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+ ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
error = kobject_init_and_add(kobj, &ktype_veth_pool,
@@ -1950,7 +1982,7 @@ static void ibmveth_remove(struct vio_dev *dev)
cancel_work_sync(&adapter->work);
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- kobject_put(&adapter->rx_buff_pool[i].kobj);
+ kobject_put(&adapter->rx_buff_pool[0][i].kobj);
unregister_netdev(netdev);
@@ -2036,11 +2068,11 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
/* Make sure there is a buffer pool with buffers that
can hold a packet of the size of the MTU */
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (pool == &adapter->rx_buff_pool[i])
+ if (pool == &adapter->rx_buff_pool[0][i])
continue;
- if (!adapter->rx_buff_pool[i].active)
+ if (!adapter->rx_buff_pool[0][i].active)
continue;
- if (mtu <= adapter->rx_buff_pool[i].buff_size)
+ if (mtu <= adapter->rx_buff_pool[0][i].buff_size)
break;
}
@@ -2214,11 +2246,11 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
/* Set sane values for buffer pools */
for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+ ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
- pool = &adapter->rx_buff_pool[0];
+ pool = &adapter->rx_buff_pool[0][0];
pool->skbuff = kunit_kcalloc(test, pool->size, sizeof(void *), GFP_KERNEL);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
@@ -2226,7 +2258,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
- correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0].size;
+ correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
@@ -2259,30 +2291,32 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
INIT_WORK(&adapter->work, ibmveth_reset_kunit);
- adapter->rx_queue.queue_len = 1;
- adapter->rx_queue.index = 0;
- adapter->rx_queue.queue_addr = kunit_kzalloc(test, sizeof(struct ibmveth_rx_q_entry),
- GFP_KERNEL);
- KUNIT_ASSERT_NOT_ERR_OR_NULL(test, adapter->rx_queue.queue_addr);
+ adapter->rx_queue[0].queue_len = 1;
+ adapter->rx_queue[0].index = 0;
+ adapter->rx_queue[0].queue_addr =
+ kunit_kzalloc(test, sizeof(struct ibmveth_rx_q_entry),
+ GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, adapter->rx_queue[0].queue_addr);
/* Set sane values for buffer pools */
for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+ ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
- pool = &adapter->rx_buff_pool[0];
+ pool = &adapter->rx_buff_pool[0][0];
pool->skbuff = kunit_kcalloc(test, pool->size, sizeof(void *), GFP_KERNEL);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
- adapter->rx_queue.queue_addr[0].correlator = (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
+ adapter->rx_queue[0].queue_addr[0].correlator = (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
- adapter->rx_queue.queue_addr[0].correlator = (u64)0 << 32 | adapter->rx_buff_pool[0].size;
+ adapter->rx_queue[0].queue_addr[0].correlator =
+ (u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
pool->skbuff[0] = skb;
- adapter->rx_queue.queue_addr[0].correlator = (u64)0 << 32 | 0;
+ adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
flush_work(&adapter->work);
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 45cfb0d054e3..f0dffe42e8fe 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -279,6 +279,8 @@ static inline long h_illan_attributes(unsigned long unit_address,
#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
#define IBMVETH_MAX_QUEUES 16U
#define IBMVETH_DEFAULT_QUEUES 8U
+#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_DEFAULT_RX_QUEUES 1U
#define IBMVETH_MAX_RX_PER_HCALL 8U
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
@@ -288,6 +290,30 @@ static int pool_active[] = { 1, 1, 0, 0, 1};
#define IBM_VETH_INVALID_MAP ((u16)0xffff)
+struct ibmveth_hcall_stats {
+ u64 reg_lan_queue; /* H_REG_LOGICAL_LAN_QUEUE */
+ u64 reg_lan; /* H_REGISTER_LOGICAL_LAN */
+ u64 add_bufs_queue; /* H_ADD_LOGICAL_LAN_BUFFERS_QUEUE */
+ u64 add_bufs; /* H_ADD_LOGICAL_LAN_BUFFERS */
+ u64 add_buf; /* H_ADD_LOGICAL_LAN_BUFFER */
+ u64 free_lan_queue; /* H_FREE_LOGICAL_LAN_QUEUE */
+ u64 free_lan; /* H_FREE_LOGICAL_LAN */
+ u64 send_lan; /* H_SEND_LOGICAL_LAN */
+};
+
+struct ibmveth_rx_queue_stats {
+ u64 packets;
+ u64 bytes;
+ u64 interrupts;
+ u64 polls;
+ u64 large_packets;
+ u64 invalid_buffers;
+ u64 no_buffer_drops;
+};
+
+#define IBMVETH_NUM_RX_QSTATS \
+ (sizeof(struct ibmveth_rx_queue_stats) / sizeof(u64))
+
struct ibmveth_buff_pool {
u32 size;
u32 index;
@@ -315,18 +341,22 @@ struct ibmveth_rx_q {
struct ibmveth_adapter {
struct vio_dev *vdev;
struct net_device *netdev;
- struct napi_struct napi;
+ struct napi_struct napi[IBMVETH_MAX_RX_QUEUES];
struct work_struct work;
unsigned int mcastFilterSize;
- void *buffer_list_addr;
+ void *buffer_list_addr[IBMVETH_MAX_RX_QUEUES];
void *filter_list_addr;
void *tx_ltb_ptr[IBMVETH_MAX_QUEUES];
unsigned int tx_ltb_size;
dma_addr_t tx_ltb_dma[IBMVETH_MAX_QUEUES];
- dma_addr_t buffer_list_dma;
+ dma_addr_t buffer_list_dma[IBMVETH_MAX_RX_QUEUES];
dma_addr_t filter_list_dma;
- struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS];
- struct ibmveth_rx_q rx_queue;
+ struct ibmveth_buff_pool rx_buff_pool[IBMVETH_MAX_RX_QUEUES][IBMVETH_NUM_BUFF_POOLS];
+ struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES];
+ u64 queue_handle[IBMVETH_MAX_RX_QUEUES];
+ unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES];
+ int multi_queue;
+ unsigned int num_rx_queues;
int rx_csum;
int large_send;
bool is_active_trunk;
@@ -346,6 +376,11 @@ struct ibmveth_adapter {
u64 tx_send_failed;
u64 tx_large_packets;
u64 rx_large_packets;
+
+ /* Multi-queue statistics */
+ struct ibmveth_hcall_stats hcall_stats;
+ struct ibmveth_rx_queue_stats *rx_qstats;
+
/* Ethtool settings */
u8 duplex;
u32 speed;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 02/15] ibmveth: Prepare MQ RX adapter and statistics structures Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
` (11 subsequent siblings)
14 siblings, 0 replies; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
ibmveth_open() allocates the filter list and every RX queue inline.
That's already ~160 lines and would get ugly once we loop over
num_rx_queues, especially on error unwind.
Pull the RX bits into helpers:
ibmveth_alloc_filter_list() / ibmveth_free_filter_list()
— shared multicast filter list (one per adapter, not per queue)
ibmveth_alloc_rx_queues() / ibmveth_cleanup_rx_resources()
— per-queue buffer lists and RX rings, looping [0, num_rx_queues)
alloc_rx_queues() rolls back on failure so open() does not need nested
goto chains for every queue index.
This is the first of several helper-only patches (pools, IRQ, TX, PHYP
registration, open/close wiring, buffer submit) that reshape bring-up
ahead of MQ datapath commit later in the series.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 168 +++++++++++++++++++++++++++++
1 file changed, 168 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 8f9f927bff23..b8adc9935471 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -147,6 +147,174 @@ static unsigned int ibmveth_real_max_tx_queues(void)
return min(n_cpu, IBMVETH_MAX_QUEUES);
}
+/**
+ * ibmveth_alloc_filter_list - Allocate and map filter list
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+
+ adapter->filter_list_addr = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->filter_list_addr) {
+ netdev_err(netdev, "unable to allocate filter pages\n");
+ return -ENOMEM;
+ }
+
+ adapter->filter_list_dma = dma_map_single(dev,
+ adapter->filter_list_addr,
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->filter_list_dma)) {
+ netdev_err(netdev, "unable to map filter list pages\n");
+ free_page((unsigned long)adapter->filter_list_addr);
+ adapter->filter_list_addr = NULL;
+ return -ENOMEM;
+ }
+
+ netdev_dbg(netdev, "filter list @ 0x%p (DMA: 0x%llx)\n",
+ adapter->filter_list_addr,
+ (unsigned long long)adapter->filter_list_dma);
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_filter_list - Free filter list resources
+ * @adapter: ibmveth adapter structure
+ */
+static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+
+ if (adapter->filter_list_dma) {
+ dma_unmap_single(dev, adapter->filter_list_dma, 4096,
+ DMA_BIDIRECTIONAL);
+ adapter->filter_list_dma = 0;
+ }
+
+ if (adapter->filter_list_addr) {
+ free_page((unsigned long)adapter->filter_list_addr);
+ adapter->filter_list_addr = NULL;
+ }
+}
+
+/**
+ * ibmveth_alloc_rx_queues - Allocate per-queue RX resources
+ * @adapter: ibmveth adapter structure
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused
+ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ adapter->buffer_list_addr[i] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[i]) {
+ netdev_err(netdev, "unable to allocate buffer list for queue %d\n", i);
+ goto err_cleanup;
+ }
+
+ adapter->rx_queue[i].queue_len =
+ sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+ adapter->rx_queue[i].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[i].queue_len,
+ &adapter->rx_queue[i].queue_dma,
+ GFP_KERNEL);
+ if (!adapter->rx_queue[i].queue_addr) {
+ netdev_err(netdev, "unable to allocate RX queue for queue %d\n", i);
+ goto err_cleanup;
+ }
+
+ adapter->buffer_list_dma[i] = dma_map_single(dev,
+ adapter->buffer_list_addr[i],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
+ netdev_err(netdev, "unable to map buffer list for queue %d\n", i);
+ adapter->buffer_list_dma[i] = 0;
+ goto err_cleanup;
+ }
+
+ adapter->rx_queue[i].index = 0;
+ adapter->rx_queue[i].num_slots = rxq_entries;
+ adapter->rx_queue[i].toggle = 1;
+
+ netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
+ i, adapter->buffer_list_addr[i],
+ (unsigned long long)adapter->buffer_list_dma[i],
+ adapter->rx_queue[i].queue_addr,
+ (unsigned long long)adapter->rx_queue[i].queue_dma,
+ (unsigned long long)rxq_entries);
+ }
+
+ netdev_dbg(netdev, "allocated %d RX queue(s) with %d entries each\n",
+ adapter->num_rx_queues, rxq_entries);
+
+ return 0;
+
+err_cleanup:
+ /* Clean up previously allocated queues */
+ for (; i >= 0; i--) {
+ if (adapter->buffer_list_dma[i]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[i],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[i] = 0;
+ }
+ if (adapter->rx_queue[i].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+ adapter->rx_queue[i].queue_addr,
+ adapter->rx_queue[i].queue_dma);
+ adapter->rx_queue[i].queue_addr = NULL;
+ }
+ if (adapter->buffer_list_addr[i]) {
+ free_page((unsigned long)adapter->buffer_list_addr[i]);
+ adapter->buffer_list_addr[i] = NULL;
+ }
+ }
+
+ return -ENOMEM;
+}
+
+/**
+ * ibmveth_cleanup_rx_resources - Free all RX queue resources
+ * @adapter: ibmveth adapter structure
+ */
+static void __maybe_unused ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+ int i;
+
+ netdev_dbg(adapter->netdev, "cleaning up %d RX queue(s)\n",
+ adapter->num_rx_queues);
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->buffer_list_dma[i]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[i],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[i] = 0;
+ }
+
+ if (adapter->rx_queue[i].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+ adapter->rx_queue[i].queue_addr,
+ adapter->rx_queue[i].queue_dma);
+ adapter->rx_queue[i].queue_addr = NULL;
+ }
+
+ if (adapter->buffer_list_addr[i]) {
+ free_page((unsigned long)adapter->buffer_list_addr[i]);
+ adapter->buffer_list_addr[i] = NULL;
+ }
+ }
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (2 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
` (10 subsequent siblings)
14 siblings, 0 replies; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
This is the key memory-model change for MQ RX.
Legacy ibmveth uses five adapter-level RX buffer pools (512 B
through 64 KiB slots). pool_active[] enables the standard-MTU pools by
default; larger pools activate when MTU requires them. With single-queue
RX that set is shared on one completion path.
MQ requires the same pool model per queue: buffers post with
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against a queue handle and completions
return on that queue. Sharing pools across queues would mix ownership and
break queue-local replenish/drain/teardown.
Refactor around queue-local pools with static geometry (still defined at
probe on queue 0, copied to queues 1..N at alloc time):
rx_buff_pool[queue][pool]
ibmveth_alloc_queue_buffer_pools()
ibmveth_free_queue_buffer_pools()
ibmveth_alloc_buffer_pools() / ibmveth_free_buffer_pools()
Queue 0 remains the template for pool geometry (size, buff_size,
threshold, active). For queues 1..N we copy metadata from queue 0, then
allocate actual backing arrays/skbs per queue.
At the default 1500-byte MTU, pool 4 (64 KiB buffers) is not needed and
costs guest memory when allocated per queue in MQ mode. Clear
pool_active[4] so open() skips it; ibmveth_change_mtu() still enables
larger pools when MTU warrants jumbo frames.
Error handling is also made queue-safe:
- if allocation fails in one pool, unwind only what was allocated for
that queue, then unwind prior queues in the caller
- free paths release pools based on real allocations
(free_map/dma_addr/skbuff), not only pool->active
That allocation-based free check is intentional: later resize and failure
paths can leave memory allocated even when active was already cleared.
Freeing by allocation state avoids leaks and double-free corner cases.
This split keeps the per-queue pool design isolated and reviewable ahead
of the MQ datapath enable commit later in the series.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 127 +++++++++++++++++++++++++++++
drivers/net/ethernet/ibm/ibmveth.h | 2 +-
2 files changed, 128 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b8adc9935471..95068fb20dba 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -611,6 +611,133 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
}
}
+/**
+ * ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Allocates all active buffer pools for the specified queue.
+ * Pool metadata must be initialized before calling this function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int ibmveth_alloc_queue_buffer_pools(struct ibmveth_adapter *adapter,
+ int queue)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ if (!adapter->rx_buff_pool[queue][i].active)
+ continue;
+
+ if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[queue][i])) {
+ netdev_err(netdev,
+ "unable to allocate buffer pool %d for queue %d (size=%u, count=%u)\n",
+ i, queue,
+ adapter->rx_buff_pool[queue][i].buff_size,
+ adapter->rx_buff_pool[queue][i].size);
+ adapter->rx_buff_pool[queue][i].active = 0;
+
+ /* Free pools allocated so far for this queue */
+ while (--i >= 0) {
+ if (adapter->rx_buff_pool[queue][i].active)
+ ibmveth_free_buffer_pool(adapter,
+ &adapter->rx_buff_pool[queue][i]);
+ }
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Frees all active buffer pools for the specified queue.
+ */
+static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
+ int queue)
+{
+ int i;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[queue][i];
+
+ /* Free pool if it has allocated memory, regardless of active flag.
+ * Pools may have memory allocated but not marked active during
+ * queue scale-up, so we must check for actual allocations.
+ */
+ if (pool->free_map || pool->dma_addr || pool->skbuff)
+ ibmveth_free_buffer_pool(adapter, pool);
+ }
+}
+
+/**
+ * ibmveth_alloc_buffer_pools - Allocate buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Initializes pool metadata for queues 1-N from queue 0 settings,
+ * then allocates buffer pools for all queues using the helper function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i, q, rc;
+
+ /* Initialize pool metadata for queues 1-15 from queue 0 settings */
+ for (q = 1; q < adapter->num_rx_queues; q++) {
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *src = &adapter->rx_buff_pool[0][i];
+ struct ibmveth_buff_pool *dst = &adapter->rx_buff_pool[q][i];
+
+ dst->size = src->size;
+ dst->index = src->index;
+ dst->buff_size = src->buff_size;
+ dst->threshold = src->threshold;
+ dst->active = src->active;
+ }
+ }
+
+ /* Allocate actual buffers for all queues */
+ for (q = 0; q < adapter->num_rx_queues; q++) {
+ rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
+ if (rc) {
+ /* Free pools for all previous queues */
+ while (--q >= 0)
+ ibmveth_free_queue_buffer_pools(adapter, q);
+ return rc;
+ }
+ }
+
+ netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
+ adapter->num_rx_queues);
+ return 0;
+}
+
+/**
+ * ibmveth_free_buffer_pools - Free buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees buffer pools for all queues using the helper function.
+ */
+static void __maybe_unused ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
+{
+ int q;
+
+ /* Free buffer pools for all queues */
+ for (q = 0; q < adapter->num_rx_queues; q++)
+ ibmveth_free_queue_buffer_pools(adapter, q);
+
+ netdev_dbg(adapter->netdev, "freed buffer pools for %d queue(s)\n",
+ adapter->num_rx_queues);
+}
+
/**
* ibmveth_remove_buffer_from_pool - remove a buffer from a pool
* @adapter: adapter instance
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index f0dffe42e8fe..d2ceeccd5fbd 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -286,7 +286,7 @@ static inline long h_illan_attributes(unsigned long unit_address,
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
static int pool_count[] = { 256, 512, 256, 256, 256 };
static int pool_count_cmo[] = { 256, 512, 256, 256, 64 };
-static int pool_active[] = { 1, 1, 0, 0, 1};
+static int pool_active[] = { 1, 1, 0, 0, 0};
#define IBM_VETH_INVALID_MAP ((u16)0xffff)
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (3 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-14 12:43 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
` (9 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Queue 0 and subordinate RX queues use different interrupt control
interfaces in PHYP:
- queue 0: h_vio_signal() after h_register_logical_lan()
- queue N: H_VIOCTL against the queue handle/hwirq mapping
The current code is single-queue oriented and cannot safely scale to
multiple RX queues in poll completion and open/close IRQ setup.
Introduce queue-indexed interrupt helpers:
ibmveth_enable_irq(adapter, queue_index)
ibmveth_disable_irq(adapter, queue_index)
ibmveth_setup_rx_interrupts()
ibmveth_cleanup_rx_interrupts()
These helpers centralize queue0-vs-subordinate dispatch and make IRQ
lifecycle symmetric across open/close and future resize paths.
request_irq() is wired with &adapter->napi[i] as dev_id per queue, so
interrupt ownership follows the NAPI instance that services that RX
queue.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 160 +++++++++++++++++++++++++++++
1 file changed, 160 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 95068fb20dba..b5ae979c1f82 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -315,6 +315,166 @@ static void __maybe_unused ibmveth_cleanup_rx_resources(struct ibmveth_adapter *
}
}
+/**
+ * ibmveth_toggle_irq - Common helper to enable/disable queue interrupts
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ * @enable: true to enable, false to disable
+ *
+ * For queue 0 (primary), uses h_vio_signal() as it's registered via
+ * h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL
+ * with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control.
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_toggle_irq(struct ibmveth_adapter *adapter, int queue_index, bool enable)
+{
+ unsigned long rc;
+ unsigned long irq = adapter->queue_irq[queue_index];
+ const char *action = enable ? "enable" : "disable";
+
+ if (queue_index == 0) {
+ /* Primary queue: use h_vio_signal() */
+ rc = h_vio_signal(adapter->vdev->unit_address,
+ enable ? VIO_IRQ_ENABLE : VIO_IRQ_DISABLE);
+ } else {
+ /* Subordinate queues: use H_VIOCTL with hardware IRQ */
+ struct irq_data *irq_data = irq_get_irq_data(irq);
+ irq_hw_number_t hwirq;
+ u64 vioctl_cmd = enable ? H_ENABLE_VIO_INTERRUPT : H_DISABLE_VIO_INTERRUPT;
+
+ if (!irq_data) {
+ netdev_err(adapter->netdev,
+ "Failed to get IRQ data for queue %d (virq=%lu)\n",
+ queue_index, irq);
+ return -EINVAL;
+ }
+
+ hwirq = irqd_to_hwirq(irq_data);
+ rc = plpar_hcall_norets(H_VIOCTL,
+ adapter->vdev->unit_address,
+ vioctl_cmd,
+ hwirq, 0, 0);
+
+ if (rc == H_PARAMETER) {
+ /* H_PARAMETER is non-fatal when IRQ is already in the requested state. */
+ netdev_warn_once(adapter->netdev,
+ "H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n",
+ action, queue_index, hwirq);
+ return 0;
+ }
+ }
+
+ if (rc)
+ netdev_err(adapter->netdev,
+ "Failed to %s IRQ for queue %d, rc=%ld\n",
+ action, queue_index, rc);
+ return rc;
+}
+
+/**
+ * ibmveth_disable_irq - Disable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_disable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+ return ibmveth_toggle_irq(adapter, queue_index, false);
+}
+
+/**
+ * ibmveth_enable_irq - Enable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+ return ibmveth_toggle_irq(adapter, queue_index, true);
+}
+
+/**
+ * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
+ * @adapter: ibmveth adapter structure
+ *
+ * Registers interrupt handlers for all RX queues and enables NAPI polling.
+ * On error, cleans up any successfully registered IRQs before returning.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int __maybe_unused
+ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i, rc;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (!adapter->queue_irq[i]) {
+ netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
+ rc = -EINVAL;
+ goto err_free_irqs;
+ }
+
+ rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt,
+ 0, netdev->name, &adapter->napi[i]);
+ if (rc) {
+ netdev_err(netdev,
+ "request_irq() failed for irq 0x%x queue %d: %d\n",
+ adapter->queue_irq[i], i, rc);
+ goto err_free_irqs;
+ }
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ napi_enable(&adapter->napi[i]);
+
+ return 0;
+
+err_free_irqs:
+ while (--i >= 0)
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ return rc;
+}
+
+/**
+ * ibmveth_cleanup_rx_interrupts - Disable NAPI and free IRQs
+ * @adapter: ibmveth adapter structure
+ *
+ * Disables NAPI polling and frees interrupt handlers for all RX queues.
+ */
+static void
+ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i])
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ }
+
+ /* Dispose IRQ mappings for subordinate queues (1-15).
+ * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
+ */
+ for (i = 1; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i]) {
+ irq_dispose_mapping(adapter->queue_irq[i]);
+ adapter->queue_irq[i] = 0;
+ }
+ }
+
+ /* Clear queue 0 IRQ number */
+ adapter->queue_irq[0] = 0;
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 06/15] ibmveth: Refactor TX resource allocation in open/close paths
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (4 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 07/15] ibmveth: Add RX queue register/deregister helpers for MQ Mingming Cao
` (8 subsequent siblings)
14 siblings, 0 replies; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Same story as the RX refactor: pull TX LTB alloc out of open/close.
ibmveth_alloc_tx_resources() / ibmveth_free_tx_resources() walk
real_num_tx_queues so ethtool TX channel changes keep working. Hooked
into open/close in the next patch.
No MQ RX behaviour change — TX was already multi-queue capable via
ethtool -L. This patch only tidies the open/close path ahead of the
RX helper wiring in the next patch.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 43 ++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b5ae979c1f82..63b0184c622a 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1038,6 +1038,49 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
return 0;
}
+/**
+ * ibmveth_alloc_tx_resources - Allocate TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Allocates TX Long Term Buffers (LTBs) for all TX queues.
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int __maybe_unused
+ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++) {
+ if (ibmveth_allocate_tx_ltb(adapter, i))
+ goto err_free_ltbs;
+ }
+
+ return 0;
+
+err_free_ltbs:
+ while (--i >= 0)
+ ibmveth_free_tx_ltb(adapter, i);
+ return -ENOMEM;
+}
+
+/**
+ * ibmveth_free_tx_resources - Free TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees TX Long Term Buffers (LTBs) for all TX queues.
+ */
+static void __maybe_unused
+ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++)
+ ibmveth_free_tx_ltb(adapter, i);
+}
+
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
union ibmveth_buf_desc rxq_desc,
u64 mac_address)
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 07/15] ibmveth: Add RX queue register/deregister helpers for MQ
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (5 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-14 12:43 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 08/15] ibmveth: Refactor open/close into MQ-ready resource pipeline Mingming Cao
` (7 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
MQ RX replaces a single adapter-level register/free pair with a mixed
PHYP model: queue 0 via h_register_logical_lan*(), subordinates via
H_REG_LOGICAL_LAN_QUEUE. Subordinate registration returns queue handles
and hardware IRQ numbers that must be mapped to Linux virqs and unwound
on failure.
Add queue lifecycle helpers to isolate that control plane:
ibmveth_register_logical_lan_queue()
ibmveth_register_single_rx_queue()
ibmveth_deregister_single_rx_queue()
ibmveth_register_rx_queues()
ibmveth_free_all_queues()
ibmveth_dispose_subordinate_irq_mappings()
These helpers are called only when multi_queue is enabled (patch 11).
Until then open/close still use the legacy register and buffer hcall
path; legacy firmware is unchanged.
When multi_queue is enabled, queue 0 uses
h_register_logical_lan_with_handle() so all queues share the per-queue
buffer hcall path. register_rx_queues() registers with PHYP only;
interrupt delivery is enabled later from ibmveth_setup_rx_interrupts()
after request_irq(). Partial registration failure disposes subordinate virq
mappings before ibmveth_free_all_queues() clears handles;
free_all_queues() clears queue handles only — IRQ mappings are released
by dispose_subordinate_irq_mappings() or cleanup_rx_interrupts().
This commit also centralizes hcall accounting on the register/free paths.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 337 ++++++++++++++++++++++++++++-
1 file changed, 332 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 63b0184c622a..7fc11a4e1f61 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -21,6 +21,8 @@
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
#include <linux/mm.h>
#include <linux/pm.h>
#include <linux/ethtool.h>
@@ -399,6 +401,28 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
return ibmveth_toggle_irq(adapter, queue_index, true);
}
+/**
+ * ibmveth_dispose_subordinate_irq_mappings - Drop virq mappings for queues 1..N
+ * @adapter: ibmveth adapter structure
+ *
+ * Subordinate queues get mappings from irq_create_mapping() during PHYP
+ * registration. Queue 0 uses netdev->irq from device tree and is left alone.
+ * Call after free_irq() when handlers were installed, or alone when open
+ * fails during register_rx_queues() before request_irq().
+ */
+static void
+ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
+{
+ int i;
+
+ for (i = 1; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i]) {
+ irq_dispose_mapping(adapter->queue_irq[i]);
+ adapter->queue_irq[i] = 0;
+ }
+ }
+}
+
/**
* ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
* @adapter: ibmveth adapter structure
@@ -1082,8 +1106,8 @@ ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
}
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
- union ibmveth_buf_desc rxq_desc,
- u64 mac_address)
+ union ibmveth_buf_desc rxq_desc,
+ u64 mac_address)
{
int rc, try_again = 1;
@@ -1093,13 +1117,29 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
* try again, but only once.
*/
retry:
- rc = h_register_logical_lan(adapter->vdev->unit_address,
- adapter->buffer_list_dma[0], rxq_desc.desc,
- adapter->filter_list_dma, mac_address);
+ /* In multi-queue mode, obtain a queue handle for queue 0 so all RX
+ * queues can use the same per-queue buffer hypercalls.
+ */
+ if (adapter->multi_queue) {
+ rc = h_register_logical_lan_with_handle(adapter->vdev->unit_address,
+ adapter->buffer_list_dma[0],
+ rxq_desc.desc,
+ adapter->filter_list_dma,
+ mac_address,
+ &adapter->queue_handle[0]);
+ } else {
+ rc = h_register_logical_lan(adapter->vdev->unit_address,
+ adapter->buffer_list_dma[0],
+ rxq_desc.desc,
+ adapter->filter_list_dma,
+ mac_address);
+ }
+ adapter->hcall_stats.reg_lan++;
if (rc != H_SUCCESS && try_again) {
do {
rc = h_free_logical_lan(adapter->vdev->unit_address);
+ adapter->hcall_stats.free_lan++;
} while (H_IS_LONG_BUSY(rc) || (rc == H_BUSY));
try_again = 0;
@@ -1136,6 +1176,293 @@ static void __maybe_unused ibmveth_free_rx_qstats(struct ibmveth_adapter *adapte
adapter->rx_qstats = NULL;
}
+/**
+ * ibmveth_register_logical_lan_queue - Register subordinate queue with hypervisor
+ * @adapter: ibmveth adapter structure
+ * @rxq_desc: Receive queue descriptor
+ * @queue_index: RX queue index (1..N for subordinate queues)
+ *
+ * Registers a subordinate receive queue using H_REG_LOGICAL_LAN_QUEUE.
+ * On success, stores the queue handle and virtual IRQ in the adapter.
+ * Retries once if registration fails (handles kexec case). If IRQ mapping
+ * fails after a successful hypervisor registration, the queue is freed
+ * before returning.
+ *
+ * Return: H_SUCCESS on success, negative errno on IRQ mapping failure,
+ * hypervisor error code otherwise
+ */
+static int
+ibmveth_register_logical_lan_queue(struct ibmveth_adapter *adapter,
+ union ibmveth_buf_desc rxq_desc,
+ int queue_index)
+{
+ unsigned long handle, hwirq;
+ unsigned int virq;
+ long lpar_rc;
+ int try_again = 1;
+
+retry:
+ netdev_dbg(adapter->netdev,
+ "Attempting to register queue %d: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+ queue_index, adapter->vdev->unit_address,
+ (unsigned long long)adapter->buffer_list_dma[queue_index],
+ (unsigned long long)rxq_desc.desc);
+
+ lpar_rc = h_reg_logical_lan_queue(adapter->vdev->unit_address,
+ adapter->buffer_list_dma[queue_index],
+ rxq_desc.desc, &handle, &hwirq);
+ adapter->hcall_stats.reg_lan_queue++;
+
+ if (lpar_rc == H_SUCCESS) {
+ virq = irq_create_mapping(NULL, hwirq);
+ if (!virq) {
+ unsigned long free_rc;
+
+ netdev_err(adapter->netdev,
+ "Failed to map IRQ for queue %d (hwirq=%lu)\n",
+ queue_index, hwirq);
+ do {
+ free_rc = h_free_logical_lan_queue(adapter->vdev->unit_address,
+ handle);
+ } while (H_IS_LONG_BUSY(free_rc) || (free_rc == H_BUSY));
+ adapter->hcall_stats.free_lan_queue++;
+ if (free_rc != H_SUCCESS)
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan_queue failed for queue %d after IRQ map failure: rc=0x%lx\n",
+ queue_index, free_rc);
+ return -EINVAL;
+ }
+
+ adapter->queue_handle[queue_index] = handle;
+ adapter->queue_irq[queue_index] = virq;
+
+ netdev_dbg(adapter->netdev,
+ "queue %d registered: handle=0x%llx irq=%u\n",
+ queue_index, adapter->queue_handle[queue_index],
+ adapter->queue_irq[queue_index]);
+ return H_SUCCESS;
+ }
+
+ if (lpar_rc == H_FUNCTION) {
+ if (adapter->multi_queue) {
+ netdev_info(adapter->netdev,
+ "Multi queue mode not supported by firmware, falling back to single queue\n");
+ adapter->multi_queue = 0;
+ } else {
+ netdev_err(adapter->netdev,
+ "Unexpected H_FUNCTION for queue %d registration (MQ mode already disabled)\n",
+ queue_index);
+ }
+ return lpar_rc;
+ }
+
+ if (try_again) {
+ try_again = 0;
+ goto retry;
+ }
+
+ netdev_err(adapter->netdev,
+ "h_reg_logical_lan_queue failed with %ld after retry\n",
+ lpar_rc);
+ netdev_err(adapter->netdev,
+ "queue %d params: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+ queue_index, adapter->vdev->unit_address,
+ (unsigned long long)adapter->buffer_list_dma[queue_index],
+ (unsigned long long)rxq_desc.desc);
+
+ return lpar_rc;
+}
+
+/**
+ * ibmveth_register_single_rx_queue - Register one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to register (1..N)
+ * @mac_address: MAC address (unused; reserved for API symmetry)
+ *
+ * Builds the queue descriptor and registers with the hypervisor via
+ * ibmveth_register_logical_lan_queue().
+ *
+ * Return: 0 on success, -EINVAL if @queue_idx is invalid, -EIO on failure
+ */
+static int
+ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx, u64 mac_address)
+{
+ struct net_device *netdev = adapter->netdev;
+ union ibmveth_buf_desc rxq_desc;
+ long lpar_rc;
+
+ (void)mac_address;
+
+ if (WARN_ON(queue_idx < 1 || queue_idx >= IBMVETH_MAX_RX_QUEUES))
+ return -EINVAL;
+
+ rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+ adapter->rx_queue[queue_idx].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[queue_idx].queue_dma;
+
+ lpar_rc = ibmveth_register_logical_lan_queue(adapter, rxq_desc,
+ queue_idx);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev, "Failed to register queue %d: rc=0x%lx\n",
+ queue_idx, lpar_rc);
+ return -EIO;
+ }
+
+ netdev_dbg(netdev, "Registered queue %d with handle 0x%llx\n",
+ queue_idx, adapter->queue_handle[queue_idx]);
+
+ return 0;
+}
+
+/**
+ * ibmveth_deregister_single_rx_queue - Deregister one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to deregister (1..N)
+ *
+ * Deregisters a single queue via H_FREE_LOGICAL_LAN_QUEUE and disposes
+ * the IRQ mapping for subordinate queues. Queue 0 is freed only through
+ * ibmveth_free_all_queues() (H_FREE_LOGICAL_LAN).
+ */
+static void __maybe_unused
+ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ unsigned long lpar_rc;
+
+ if (!adapter->queue_handle[queue_idx])
+ return;
+
+ do {
+ lpar_rc = h_free_logical_lan_queue(adapter->vdev->unit_address,
+ adapter->queue_handle[queue_idx]);
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+ adapter->hcall_stats.free_lan_queue++;
+
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan_queue failed for queue %d: rc=0x%lx\n",
+ queue_idx, lpar_rc);
+ }
+
+ adapter->queue_handle[queue_idx] = 0;
+
+ if (queue_idx > 0 && adapter->queue_irq[queue_idx]) {
+ irq_dispose_mapping(adapter->queue_irq[queue_idx]);
+ adapter->queue_irq[queue_idx] = 0;
+ }
+
+ netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
+}
+
+/**
+ * ibmveth_free_all_queues - Free all RX queues at once
+ * @adapter: ibmveth adapter structure
+ *
+ * Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall.
+ * Used during interface close and registration error cleanup.
+ *
+ * Clears queue handles only; queue_irq[] is released by
+ * ibmveth_cleanup_rx_interrupts() on close, or by
+ * ibmveth_dispose_subordinate_irq_mappings() on partial register failure.
+ */
+static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
+{
+ unsigned long lpar_rc;
+ int i;
+
+ netdev_dbg(adapter->netdev, "freeing all RX queues at once\n");
+
+ do {
+ lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
+ adapter->hcall_stats.free_lan++;
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan failed: %ld\n", lpar_rc);
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ adapter->queue_handle[i] = 0;
+}
+
+/**
+ * ibmveth_register_rx_queues - Register RX queues with hypervisor
+ * @adapter: ibmveth adapter structure
+ * @mac_address: MAC address for device registration
+ *
+ * Registers queue 0 via ibmveth_register_logical_lan(), then subordinate
+ * queues 1..N when multi-queue mode is enabled.
+ *
+ * Return: 0 on success, -ENONET if queue 0 registration fails, -EIO on
+ * subordinate queue registration failure
+ */
+static int
+ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
+{
+ struct net_device *netdev = adapter->netdev;
+ union ibmveth_buf_desc rxq_desc;
+ unsigned long lpar_rc;
+ int i, rc;
+
+ rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+ adapter->rx_queue[0].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
+ adapter->queue_irq[0] = netdev->irq;
+
+ rc = ibmveth_disable_irq(adapter, 0);
+ if (rc != H_SUCCESS)
+ netdev_dbg(netdev,
+ "Failed to disable IRQ for queue 0 before registration, rc=%d\n",
+ rc);
+
+ lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev, "h_register_logical_lan failed: %ld\n", lpar_rc);
+ netdev_err(netdev,
+ "buffer TCE:0x%llx filter TCE:0x%llx rxq desc:0x%llx MAC:0x%llx\n",
+ adapter->buffer_list_dma[0],
+ adapter->filter_list_dma,
+ rxq_desc.desc, mac_address);
+ return -ENONET;
+ }
+
+ if (adapter->num_rx_queues == 1 || !adapter->multi_queue) {
+ netdev_dbg(netdev,
+ "registered 1 RX queue with hypervisor (single-queue mode)\n");
+ return 0;
+ }
+
+ netdev_dbg(netdev, "Registering %d subordinate queues (1-%d)\n",
+ adapter->num_rx_queues - 1, adapter->num_rx_queues - 1);
+
+ for (i = 1; i < adapter->num_rx_queues; i++) {
+ rc = ibmveth_register_single_rx_queue(adapter, i, mac_address);
+ if (rc) {
+ if (!adapter->queue_handle[i] || !adapter->queue_irq[i]) {
+ netdev_err(netdev,
+ "Invalid hypervisor return for queue %d: handle=0x%llx irq=%u\n",
+ i, adapter->queue_handle[i],
+ adapter->queue_irq[i]);
+ }
+ goto err_unregister;
+ }
+ }
+
+ netdev_dbg(netdev,
+ "registered %d RX queues with hypervisor (multi-queue mode)\n",
+ adapter->num_rx_queues);
+
+ return 0;
+
+err_unregister:
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+ ibmveth_free_all_queues(adapter);
+ return rc;
+}
+
static int ibmveth_open(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 08/15] ibmveth: Refactor open/close into MQ-ready resource pipeline
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (6 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 07/15] ibmveth: Add RX queue register/deregister helpers for MQ Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-14 12:47 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 09/15] ibmveth: Add queue-aware RX buffer submit helper for MQ Mingming Cao
` (6 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Patches 4-8 added alloc/free helpers for RX rings, buffer pools, IRQs,
TX LTBs, and PHYP registration, but open() and close() still duplicated
most of that logic inline. This patch wires the helpers in and makes
open/close the readable bring-up/teardown sequence MQ will extend.
ibmveth_open() runs:
1. ibmveth_alloc_rx_qstats()
2. ibmveth_alloc_filter_list()
3. ibmveth_alloc_rx_queues() - buffer lists + RX rings [0, N)
4. ibmveth_alloc_buffer_pools() - guest RX memory before PHYP
5. ibmveth_register_rx_queues() - PHYP registration (no IRQ enable)
6. netif_set_real_num_rx_queues()
7. ibmveth_setup_rx_interrupts() - request_irq, PHYP enable on MQ
8. initial replenish - queue 0 only today
9. ibmveth_alloc_tx_resources()
Each step has a matching out_* label on failure so unwind walks back
through free_all_queues(), cleanup_rx_resources(), and the other helpers
instead of open() carrying its own DMA unmap/free_page/goto maze (~200
lines removed).
ibmveth_close() mirrors that in reverse: stop TX, disable hypervisor IRQs
per queue, free TX LTBs, tear down NAPI/IRQ handlers, drop buffer pools,
H_FREE_LOGICAL_LAN via ibmveth_free_all_queues(), then free
RX/filter/qstats memory.
request_irq() now passes &napi[i] as dev_id on every queue so the
interrupt and poll paths can derive the queue index from the napi pointer
(napi - adapter->napi).
Drop __maybe_unused from the helpers added in patches 4-8 — they are
called from open/close from this patch onward.
Runtime still single-queue until the MQ enable commit later in the series;
replenish still kicks off via ibmveth_interrupt() on queue 0 as before.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 351 +++++++++++------------------
1 file changed, 137 insertions(+), 214 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 7fc11a4e1f61..fa2d4777ffc7 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -155,7 +155,7 @@ static unsigned int ibmveth_real_max_tx_queues(void)
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
+static int ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
struct net_device *netdev = adapter->netdev;
@@ -187,7 +187,7 @@ static int __maybe_unused ibmveth_alloc_filter_list(struct ibmveth_adapter *adap
* ibmveth_free_filter_list - Free filter list resources
* @adapter: ibmveth adapter structure
*/
-static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
+static void ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
@@ -203,6 +203,33 @@ static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adap
}
}
+/**
+ * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
+ sizeof(struct ibmveth_rx_queue_stats),
+ GFP_KERNEL);
+ if (!adapter->rx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_rx_qstats - Free per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->rx_qstats);
+ adapter->rx_qstats = NULL;
+}
+
/**
* ibmveth_alloc_rx_queues - Allocate per-queue RX resources
* @adapter: ibmveth adapter structure
@@ -210,7 +237,7 @@ static void __maybe_unused ibmveth_free_filter_list(struct ibmveth_adapter *adap
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused
+static int
ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
{
struct device *dev = &adapter->vdev->dev;
@@ -288,7 +315,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
* ibmveth_cleanup_rx_resources - Free all RX queue resources
* @adapter: ibmveth adapter structure
*/
-static void __maybe_unused ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
+static void ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
{
struct device *dev = &adapter->vdev->dev;
int i;
@@ -424,21 +451,22 @@ ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
}
/**
- * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
+ * ibmveth_setup_rx_interrupts - Register IRQ handlers and enable NAPI
* @adapter: ibmveth adapter structure
*
* Registers interrupt handlers for all RX queues and enables NAPI polling.
- * On error, cleans up any successfully registered IRQs before returning.
+ * For multi-queue mode, enables hypervisor interrupt delivery only after
+ * every queue has a Linux handler installed.
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused
+static int
ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
- int i, rc;
+ int i, rc, num = adapter->num_rx_queues;
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < num; i++) {
if (!adapter->queue_irq[i]) {
netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
rc = -EINVAL;
@@ -455,14 +483,34 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
}
}
- for (i = 0; i < adapter->num_rx_queues; i++)
+ for (i = 0; i < num; i++)
napi_enable(&adapter->napi[i]);
+ if (adapter->multi_queue && num > 1) {
+ for (i = 0; i < num; i++) {
+ rc = ibmveth_enable_irq(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d, rc=%d\n",
+ i, rc);
+ while (--i >= 0)
+ ibmveth_disable_irq(adapter, i);
+ rc = -EIO;
+ goto err_disable_napi;
+ }
+ }
+ }
+
return 0;
+err_disable_napi:
+ for (i = 0; i < num; i++)
+ napi_disable(&adapter->napi[i]);
+ i = num;
err_free_irqs:
while (--i >= 0)
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
return rc;
}
@@ -485,15 +533,7 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
}
- /* Dispose IRQ mappings for subordinate queues (1-15).
- * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
- */
- for (i = 1; i < adapter->num_rx_queues; i++) {
- if (adapter->queue_irq[i]) {
- irq_dispose_mapping(adapter->queue_irq[i]);
- adapter->queue_irq[i] = 0;
- }
- }
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
/* Clear queue 0 IRQ number */
adapter->queue_irq[0] = 0;
@@ -869,7 +909,7 @@ static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
*
* Return: 0 on success, negative error code on failure
*/
-static int __maybe_unused ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
+static int ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
int i, q, rc;
@@ -910,7 +950,7 @@ static int __maybe_unused ibmveth_alloc_buffer_pools(struct ibmveth_adapter *ada
*
* Frees buffer pools for all queues using the helper function.
*/
-static void __maybe_unused ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
+static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
{
int q;
@@ -1070,7 +1110,7 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
*
* Return: 0 on success, -ENOMEM on failure
*/
-static int __maybe_unused
+static int
ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
@@ -1095,7 +1135,7 @@ ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
*
* Frees TX Long Term Buffers (LTBs) for all TX queues.
*/
-static void __maybe_unused
+static void
ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
@@ -1149,33 +1189,6 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
return rc;
}
-/**
- * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
- * @adapter: ibmveth adapter structure
- *
- * Return: 0 on success, -ENOMEM on failure
- */
-static int __maybe_unused ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
-{
- adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
- sizeof(struct ibmveth_rx_queue_stats),
- GFP_KERNEL);
- if (!adapter->rx_qstats)
- return -ENOMEM;
-
- return 0;
-}
-
-/**
- * ibmveth_free_rx_qstats - Free per-queue RX statistics
- * @adapter: ibmveth adapter structure
- */
-static void __maybe_unused ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
-{
- kfree(adapter->rx_qstats);
- adapter->rx_qstats = NULL;
-}
-
/**
* ibmveth_register_logical_lan_queue - Register subordinate queue with hypervisor
* @adapter: ibmveth adapter structure
@@ -1466,208 +1479,108 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
static int ibmveth_open(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- u64 mac_address;
+ u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
int rxq_entries = 1;
- unsigned long lpar_rc;
int rc;
- union ibmveth_buf_desc rxq_desc;
int i;
- struct device *dev;
netdev_dbg(netdev, "open starting\n");
- napi_enable(&adapter->napi[0]);
-
- for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
rxq_entries += adapter->rx_buff_pool[0][i].size;
- rc = -ENOMEM;
- adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
- if (!adapter->buffer_list_addr[0]) {
- netdev_err(netdev, "unable to allocate list pages\n");
+ rc = ibmveth_alloc_rx_qstats(adapter);
+ if (rc)
goto out;
- }
- adapter->filter_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
- if (!adapter->filter_list_addr) {
- netdev_err(netdev, "unable to allocate filter pages\n");
- goto out_free_buffer_list;
- }
-
- dev = &adapter->vdev->dev;
+ rc = ibmveth_alloc_filter_list(adapter);
+ if (rc)
+ goto out_free_rx_qstats;
- adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
- rxq_entries;
- adapter->rx_queue[0].queue_addr =
- dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
- &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
- if (!adapter->rx_queue[0].queue_addr)
+ rc = ibmveth_alloc_rx_queues(adapter, rxq_entries);
+ if (rc)
goto out_free_filter_list;
- adapter->buffer_list_dma[0] = dma_map_single(dev,
- adapter->buffer_list_addr[0],
- 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
- netdev_err(netdev, "unable to map buffer list pages\n");
+ rc = ibmveth_alloc_buffer_pools(adapter);
+ if (rc)
goto out_free_queue_mem;
- }
- adapter->filter_list_dma = dma_map_single(dev,
- adapter->filter_list_addr, 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->filter_list_dma)) {
- netdev_err(netdev, "unable to map filter list pages\n");
- goto out_unmap_buffer_list;
- }
+ rc = ibmveth_register_rx_queues(adapter, mac_address);
+ if (rc)
+ goto out_free_buffer_pools;
- for (i = 0; i < netdev->real_num_tx_queues; i++) {
- if (ibmveth_allocate_tx_ltb(adapter, i))
- goto out_free_tx_ltb;
+ rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
+ if (rc) {
+ netdev_err(netdev, "failed to set number of rx queues\n");
+ goto out_unregister_queues;
}
- adapter->rx_queue[0].index = 0;
- adapter->rx_queue[0].num_slots = rxq_entries;
- adapter->rx_queue[0].toggle = 1;
-
- mac_address = ether_addr_to_u64(netdev->dev_addr);
-
- rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
- adapter->rx_queue[0].queue_len;
- rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
-
- netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr[0]);
- netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
- netdev_dbg(netdev, "receive q @ 0x%p\n", adapter->rx_queue[0].queue_addr);
-
- h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
-
- lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
-
- if (lpar_rc != H_SUCCESS) {
- netdev_err(netdev, "h_register_logical_lan failed with %ld\n",
- lpar_rc);
- netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
- "desc:0x%llx MAC:0x%llx\n",
- adapter->buffer_list_dma[0],
- adapter->filter_list_dma,
- rxq_desc.desc,
- mac_address);
- rc = -ENONET;
- goto out_unmap_filter_list;
- }
+ rc = ibmveth_setup_rx_interrupts(adapter);
+ if (rc)
+ goto out_unregister_queues;
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (!adapter->rx_buff_pool[0][i].active)
- continue;
- if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
- netdev_err(netdev, "unable to alloc pool\n");
- adapter->rx_buff_pool[0][i].active = 0;
- rc = -ENOMEM;
- goto out_free_buffer_pools;
+ if (adapter->num_rx_queues > 1) {
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ netdev_dbg(netdev, "initial replenish cycle for queue %d\n", i);
+ ibmveth_replenish_task(adapter, i);
}
+ } else {
+ netdev_dbg(netdev, "initial replenish cycle\n");
+ ibmveth_interrupt(adapter->queue_irq[0], &adapter->napi[0]);
}
- netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
- rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
- netdev);
- if (rc != 0) {
- netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
- netdev->irq, rc);
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
- goto out_free_buffer_pools;
- }
-
- rc = -ENOMEM;
-
- netdev_dbg(netdev, "initial replenish cycle\n");
- ibmveth_interrupt(netdev->irq, netdev);
+ rc = ibmveth_alloc_tx_resources(adapter);
+ if (rc)
+ goto out_cleanup_rx_interrupts;
netif_tx_start_all_queues(netdev);
netdev_dbg(netdev, "open complete\n");
-
return 0;
+out_cleanup_rx_interrupts:
+ ibmveth_cleanup_rx_interrupts(adapter);
+out_free_tx_resources:
+ ibmveth_free_tx_resources(adapter);
out_free_buffer_pools:
- while (--i >= 0) {
- if (adapter->rx_buff_pool[0][i].active)
- ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[0][i]);
- }
-out_unmap_filter_list:
- dma_unmap_single(dev, adapter->filter_list_dma, 4096,
- DMA_BIDIRECTIONAL);
-
-out_free_tx_ltb:
- while (--i >= 0) {
- ibmveth_free_tx_ltb(adapter, i);
- }
-
-out_unmap_buffer_list:
- dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
- DMA_BIDIRECTIONAL);
+ ibmveth_free_buffer_pools(adapter);
+out_unregister_queues:
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+ ibmveth_free_all_queues(adapter);
out_free_queue_mem:
- dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
- adapter->rx_queue[0].queue_addr,
- adapter->rx_queue[0].queue_dma);
+ ibmveth_cleanup_rx_resources(adapter);
out_free_filter_list:
- free_page((unsigned long)adapter->filter_list_addr);
-out_free_buffer_list:
- free_page((unsigned long)adapter->buffer_list_addr[0]);
+ ibmveth_free_filter_list(adapter);
+out_free_rx_qstats:
+ ibmveth_free_rx_qstats(adapter);
out:
- napi_disable(&adapter->napi[0]);
return rc;
}
static int ibmveth_close(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- struct device *dev = &adapter->vdev->dev;
- long lpar_rc;
int i;
netdev_dbg(netdev, "close starting\n");
- napi_disable(&adapter->napi[0]);
-
netif_tx_stop_all_queues(netdev);
- h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
-
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
- if (lpar_rc != H_SUCCESS) {
- netdev_err(netdev, "h_free_logical_lan failed with %lx, "
- "continuing with close\n", lpar_rc);
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i]) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
}
- free_irq(netdev->irq, netdev);
-
+ ibmveth_free_tx_resources(adapter);
+ ibmveth_cleanup_rx_interrupts(adapter);
ibmveth_update_rx_no_buffer(adapter);
-
- dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
- DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->buffer_list_addr[0]);
-
- dma_unmap_single(dev, adapter->filter_list_dma, 4096,
- DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->filter_list_addr);
-
- dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
- adapter->rx_queue[0].queue_addr,
- adapter->rx_queue[0].queue_dma);
-
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (adapter->rx_buff_pool[0][i].active)
- ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[0][i]);
-
- for (i = 0; i < netdev->real_num_tx_queues; i++)
- ibmveth_free_tx_ltb(adapter, i);
+ ibmveth_free_all_queues(adapter);
+ ibmveth_free_buffer_pools(adapter);
+ ibmveth_cleanup_rx_resources(adapter);
+ ibmveth_free_filter_list(adapter);
+ ibmveth_free_rx_qstats(adapter);
netdev_dbg(netdev, "close complete\n");
@@ -2423,15 +2336,21 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
{
- struct net_device *netdev = dev_instance;
+ struct napi_struct *napi = dev_instance;
+ struct net_device *netdev = napi->dev;
struct ibmveth_adapter *adapter = netdev_priv(netdev);
unsigned long lpar_rc;
+ int qindex;
- if (napi_schedule_prep(&adapter->napi[0])) {
- lpar_rc = h_vio_signal(adapter->vdev->unit_address,
- VIO_IRQ_DISABLE);
+ qindex = napi - adapter->napi;
+
+ if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
+ return IRQ_NONE;
+
+ if (napi_schedule_prep(napi)) {
+ lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
- __napi_schedule(&adapter->napi[0]);
+ __napi_schedule(napi);
}
return IRQ_HANDLED;
}
@@ -2537,8 +2456,10 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
#ifdef CONFIG_NET_POLL_CONTROLLER
static void ibmveth_poll_controller(struct net_device *dev)
{
- ibmveth_replenish_task(netdev_priv(dev));
- ibmveth_interrupt(dev->irq, dev);
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+
+ ibmveth_replenish_task(adapter);
+ ibmveth_interrupt(dev->irq, &adapter->napi[0]);
}
#endif
@@ -2951,7 +2872,7 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
rtnl_unlock();
/* kick the interrupt handler to allocate/deallocate pools */
- ibmveth_interrupt(netdev->irq, netdev);
+ ibmveth_interrupt(netdev->irq, &adapter->napi[0]);
return count;
unlock_err:
@@ -2991,7 +2912,9 @@ static struct kobj_type ktype_veth_pool = {
static int ibmveth_resume(struct device *dev)
{
struct net_device *netdev = dev_get_drvdata(dev);
- ibmveth_interrupt(netdev->irq, netdev);
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
+ ibmveth_interrupt(netdev->irq, &adapter->napi[0]);
return 0;
}
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 09/15] ibmveth: Add queue-aware RX buffer submit helper for MQ
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (7 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 08/15] ibmveth: Refactor open/close into MQ-ready resource pipeline Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-14 12:50 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
` (5 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Replenish is the last open-path hypervisor call that still needs
per-queue awareness before MQ receive is enabled. Today
ibmveth_replenish_buffer_pool() calls h_add_logical_lan_buffer() or
h_add_logical_lan_buffers() directly; MQ posts via
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[].
Add ibmveth_add_logical_lan_buffers() to pick the hcall:
multi_queue uses h_add_logical_lan_buffers_queue() (up to 12 buffers,
IOBAs packed with odd counts in the upper 32 bits); legacy uses the
existing single- and multi-buffer hcalls. Count add_buf/add_bufs/
add_bufs_queue in hcall_stats.
Thread queue_index through replenish_task() and replenish_buffer_pool()
so they index rx_buff_pool[queue_index][pool]. All callers still pass
queue 0; legacy hcalls remain the live path until MQ probe enables
multi_queue.
Also split H_FUNCTION handling: legacy batch falls back to single-buffer
mode; multi_queue logs an error on unsupported firmware.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 134 ++++++++++++++++++++---------
1 file changed, 94 insertions(+), 40 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index fa2d4777ffc7..b3b3886c3eed 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -597,11 +597,73 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length)
asm("dcbf %0,%1,1" :: "b" (addr), "r" (offset));
}
+/**
+ * ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor
+ * @adapter: ibmveth adapter structure
+ * @descs: array of buffer descriptors to add
+ * @filled: number of valid descriptors in the array
+ * @buff_size: size of each buffer (multi-queue mode only)
+ * @queue_index: RX queue index
+ *
+ * Return: hypervisor return code
+ */
+static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
+ union ibmveth_buf_desc *descs,
+ int filled,
+ unsigned long buff_size,
+ int queue_index)
+{
+ struct vio_dev *vdev = adapter->vdev;
+ unsigned long rc;
+
+ if (adapter->multi_queue) {
+ unsigned long buffersznum = (buff_size << 32) | filled;
+ unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL / 2] = {0};
+ int i;
+
+ /* Pack descriptor addresses into ioba pairs.
+ * Each ioba holds two 32-bit addresses packed into 64 bits:
+ * - Even descriptors (0,2,4...) go in high 32 bits
+ * - Odd descriptors (1,3,5...) go in low 32 bits
+ */
+ for (i = 0; i < filled && i < IBMVETH_MAX_RX_PER_HCALL; i++) {
+ int pair_idx = i / 2; /* Which pair: 0-5 */
+ int is_high = (i % 2 == 0); /* High or low 32 bits */
+
+ if (is_high)
+ ioba[pair_idx] = (unsigned long)descs[i].fields.address << 32;
+ else
+ ioba[pair_idx] |= descs[i].fields.address;
+ }
+
+ rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
+ adapter->queue_handle[queue_index],
+ buffersznum,
+ ioba[0], ioba[1], ioba[2],
+ ioba[3], ioba[4], ioba[5]);
+ adapter->hcall_stats.add_bufs_queue++;
+ } else if (filled == 1) {
+ rc = h_add_logical_lan_buffer(vdev->unit_address,
+ descs[0].desc);
+ adapter->hcall_stats.add_buf++;
+ } else {
+ rc = h_add_logical_lan_buffers(vdev->unit_address,
+ descs[0].desc, descs[1].desc,
+ descs[2].desc, descs[3].desc,
+ descs[4].desc, descs[5].desc,
+ descs[6].desc, descs[7].desc);
+ adapter->hcall_stats.add_bufs++;
+ }
+
+ return rc;
+}
+
/* replenish the buffers for a pool. note that we don't need to
* skb_reserve these since they are used for incoming...
*/
static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
- struct ibmveth_buff_pool *pool)
+ struct ibmveth_buff_pool *pool,
+ int queue_index)
{
union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL] = {0};
u32 remaining = pool->size - atomic_read(&pool->available);
@@ -687,24 +749,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
if (!filled)
break;
- /* single buffer case*/
- if (filled == 1)
- lpar_rc = h_add_logical_lan_buffer(vdev->unit_address,
- descs[0].desc);
- else
- /* Multi-buffer hcall */
- lpar_rc = h_add_logical_lan_buffers(vdev->unit_address,
- descs[0].desc,
- descs[1].desc,
- descs[2].desc,
- descs[3].desc,
- descs[4].desc,
- descs[5].desc,
- descs[6].desc,
- descs[7].desc);
+ lpar_rc = ibmveth_add_logical_lan_buffers(adapter, descs,
+ filled,
+ pool->buff_size,
+ queue_index);
+
if (lpar_rc != H_SUCCESS) {
dev_warn_ratelimited(dev,
- "RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n",
+ "RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n",
+ adapter->multi_queue ? "_queue" : "",
filled, lpar_rc, batch);
goto hcall_failure;
}
@@ -745,24 +798,19 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
}
adapter->replenish_add_buff_failure += filled;
- /*
- * If multi rx buffers hcall is no longer supported by FW
- * e.g. in the case of Live Partition Migration
- */
- if (batch > 1 && lpar_rc == H_FUNCTION) {
- /*
- * Instead of retry submit single buffer individually
- * here just set the max rx buffer per hcall to 1
- * buffers will be respleshed next time
- * when ibmveth_replenish_buffer_pool() is called again
- * with single-buffer case
- */
- netdev_info(adapter->netdev,
- "RX Multi buffers not supported by FW, rc=%lu\n",
- lpar_rc);
- adapter->rx_buffers_per_hcall = 1;
- netdev_info(adapter->netdev,
- "Next rx replesh will fall back to single-buffer hcall\n");
+ if (lpar_rc == H_FUNCTION) {
+ if (adapter->multi_queue) {
+ netdev_err(adapter->netdev,
+ "Unexpected H_FUNCTION from multi-queue buffer add (queue=%d, batch=%d)\n",
+ queue_index, batch);
+ break;
+ } else if (batch > 1) {
+ netdev_warn(adapter->netdev,
+ "H_FUNCTION from legacy batch buffer add (batch=%d), falling back to single buffer mode\n",
+ batch);
+ adapter->rx_buffers_per_hcall = 1;
+ continue;
+ }
}
break;
}
@@ -784,18 +832,24 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
}
/* replenish routine */
-static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
+static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
+ int queue_index)
{
int i;
+ if (queue_index >= adapter->num_rx_queues)
+ return;
+
adapter->replenish_task_cycles++;
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
- struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[queue_index][i];
if (pool->active &&
(atomic_read(&pool->available) < pool->threshold))
- ibmveth_replenish_buffer_pool(adapter, pool);
+ ibmveth_replenish_buffer_pool(adapter, pool,
+ queue_index);
}
ibmveth_update_rx_no_buffer(adapter);
@@ -2307,7 +2361,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
}
- ibmveth_replenish_task(adapter);
+ ibmveth_replenish_task(adapter, 0);
if (frames_processed == budget)
goto out;
@@ -2458,7 +2512,7 @@ static void ibmveth_poll_controller(struct net_device *dev)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
- ibmveth_replenish_task(adapter);
+ ibmveth_replenish_task(adapter, 0);
ibmveth_interrupt(dev->irq, &adapter->napi[0]);
}
#endif
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 10/15] ibmveth: Enable multi-queue RX receive path
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (8 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 09/15] ibmveth: Add queue-aware RX buffer submit helper for MQ Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-14 12:55 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 11/15] ibmveth: Add per-queue RX and TX statistics collection and reporting Mingming Cao
` (4 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
This is the first patch that sets multi_queue from H_ILLAN_ATTRIBUTES
and switches registration, buffer posting, and receive to the MQ
hcall path. It also raises num_rx_queues and enables per-queue NAPI.
This is where MQ actually receives packets. If firmware sets
IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT in H_ILLAN_ATTRIBUTES, probe sets
multi_queue and num_rx_queues to min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES), matching the existing TX default (cap 8).
Up to IBMVETH_MAX_RX_QUEUES (16) remains available via ethtool -L.
Otherwise we stay at one queue like today.
Raise IBMVETH_MAX_RX_QUEUES to 16 here so adapter arrays and NAPI state
can hold every queue before num_rx_queues is increased.
Register a NAPI struct per possible queue at probe, use
alloc_etherdev_mqs(), and call netif_set_real_num_rx_queues() after PHYP
registration on open.
With MQ enabled, open runs initial replenish on every active queue before
starting TX; legacy still kicks replenish via queue-0 interrupt/NAPI only.
PHYP can deliver to any registered queue immediately, so unprimed queues
see no-buffer drops until their NAPI path runs.
Datapath: derive queue_index from the NAPI instance, thread it through
harvest/replenish/pool access, and enable/disable IRQ per queue on NAPI
completion. Add per-queue replenish_lock around buffer posting (same-queue
NAPI vs netpoll/resize). poll_controller() and get_desired_dma() walk all
queues.
Update KUnit tests for the queue_index argument added to
ibmveth_remove_buffer_from_pool() and ibmveth_rxq_get_buffer().
Legacy firmware without the MQ bit is unchanged.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 257 ++++++++++++++++++-----------
drivers/net/ethernet/ibm/ibmveth.h | 10 +-
2 files changed, 171 insertions(+), 96 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b3b3886c3eed..863e5c68b42c 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -30,6 +30,7 @@
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/slab.h>
+#include <linux/spinlock.h>
#include <asm/hvcall.h>
#include <linux/atomic.h>
#include <asm/vio.h>
@@ -101,45 +102,58 @@ static struct ibmveth_stat ibmveth_stats[] = {
};
/* simple methods of getting data from the current rxq entry */
-static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
+static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return be32_to_cpu(adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].flags_off);
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+
+ return be32_to_cpu(rxq->queue_addr[rxq->index].flags_off);
}
-static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return (ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_TOGGLE) >>
- IBMVETH_RXQ_TOGGLE_SHIFT;
+ return (ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_TOGGLE) >>
+ IBMVETH_RXQ_TOGGLE_SHIFT;
}
-static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
+ return ibmveth_rxq_toggle(adapter, queue_index) ==
+ adapter->rx_queue[queue_index].toggle;
}
-static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_VALID;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_VALID;
}
-static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_OFF_MASK;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_OFF_MASK;
}
-static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_LRG_PKT;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_LRG_PKT;
}
-static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return be32_to_cpu(adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].length);
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+
+ return be32_to_cpu(rxq->queue_addr[rxq->index].length);
}
-static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_CSUM_GOOD;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_CSUM_GOOD;
}
static unsigned int ibmveth_real_max_tx_queues(void)
@@ -274,6 +288,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
adapter->rx_queue[i].index = 0;
adapter->rx_queue[i].num_slots = rxq_entries;
adapter->rx_queue[i].toggle = 1;
+ spin_lock_init(&adapter->rx_queue[i].replenish_lock);
netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
i, adapter->buffer_list_addr[i],
@@ -826,15 +841,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
*/
static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
{
- __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ __be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
+ u64 drops = be64_to_cpup(p);
- adapter->rx_no_buffer = be64_to_cpup(p);
+ if (i == 0)
+ adapter->rx_no_buffer = drops;
+ }
}
/* replenish routine */
static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
int queue_index)
{
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+ unsigned long flags;
int i;
if (queue_index >= adapter->num_rx_queues)
@@ -842,6 +865,8 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
adapter->replenish_task_cycles++;
+ spin_lock_irqsave(&rxq->replenish_lock, flags);
+
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
struct ibmveth_buff_pool *pool =
&adapter->rx_buff_pool[queue_index][i];
@@ -853,6 +878,8 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
}
ibmveth_update_rx_no_buffer(adapter);
+
+ spin_unlock_irqrestore(&rxq->replenish_lock, flags);
}
/* empty and free ana buffer pool - also used to do cleanup in error paths */
@@ -1028,7 +1055,8 @@ static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
* * %-EFAULT - pool and index map to null skb
*/
static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
- u64 correlator, bool reuse)
+ u64 correlator, int queue_index,
+ bool reuse)
{
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
@@ -1036,12 +1064,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
struct sk_buff *skb;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
schedule_work(&adapter->work);
return -EINVAL;
}
- skb = adapter->rx_buff_pool[0][pool].skbuff[index];
+ skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
if (WARN_ON(!skb)) {
schedule_work(&adapter->work);
return -EFAULT;
@@ -1055,42 +1083,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* remove the skb pointer to mark free. actual freeing is done
* by upper level networking after gro_receive
*/
- adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
+ adapter->rx_buff_pool[queue_index][pool].skbuff[index] = NULL;
dma_unmap_single(&adapter->vdev->dev,
- adapter->rx_buff_pool[0][pool].dma_addr[index],
- adapter->rx_buff_pool[0][pool].buff_size,
+ adapter->rx_buff_pool[queue_index][pool].dma_addr[index],
+ adapter->rx_buff_pool[queue_index][pool].buff_size,
DMA_FROM_DEVICE);
}
- free_index = adapter->rx_buff_pool[0][pool].producer_index;
- adapter->rx_buff_pool[0][pool].producer_index++;
- if (adapter->rx_buff_pool[0][pool].producer_index >=
- adapter->rx_buff_pool[0][pool].size)
- adapter->rx_buff_pool[0][pool].producer_index = 0;
- adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
+ free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;
+ adapter->rx_buff_pool[queue_index][pool].producer_index++;
+ if (adapter->rx_buff_pool[queue_index][pool].producer_index >=
+ adapter->rx_buff_pool[queue_index][pool].size)
+ adapter->rx_buff_pool[queue_index][pool].producer_index = 0;
+ adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;
mb();
- atomic_dec(&adapter->rx_buff_pool[0][pool].available);
+ atomic_dec(&adapter->rx_buff_pool[queue_index][pool].available);
return 0;
}
/* get the current buffer on the rx queue */
-static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
+static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- u64 correlator = adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].correlator;
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+ u64 correlator = rxq->queue_addr[rxq->index].correlator;
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
schedule_work(&adapter->work);
return NULL;
}
- return adapter->rx_buff_pool[0][pool].skbuff[index];
+ return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
}
/**
@@ -1106,19 +1136,20 @@ static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *ada
* * other - non-zero return from ibmveth_remove_buffer_from_pool
*/
static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
- bool reuse)
+ int queue_index, bool reuse)
{
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
u64 cor;
int rc;
- cor = adapter->rx_queue[0].queue_addr[adapter->rx_queue[0].index].correlator;
- rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
+ cor = rxq->queue_addr[rxq->index].correlator;
+ rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
if (unlikely(rc))
return rc;
- if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
- adapter->rx_queue[0].index = 0;
- adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
+ if (++rxq->index == rxq->num_slots) {
+ rxq->index = 0;
+ rxq->toggle = !rxq->toggle;
}
return 0;
@@ -2268,34 +2299,40 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
static int ibmveth_poll(struct napi_struct *napi, int budget)
{
- struct ibmveth_adapter *adapter =
- container_of(napi, struct ibmveth_adapter, napi[0]);
- struct net_device *netdev = adapter->netdev;
+ struct net_device *netdev = napi->dev;
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
int frames_processed = 0;
unsigned long lpar_rc;
+ int queue_index, rc;
u16 mss = 0;
+ queue_index = napi - adapter->napi;
+
+ if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
+ return 0;
+
restart_poll:
while (frames_processed < budget) {
- if (!ibmveth_rxq_pending_buffer(adapter))
+ if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
break;
smp_rmb();
- if (!ibmveth_rxq_buffer_valid(adapter)) {
+ if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
wmb(); /* suggested by larson1 */
adapter->rx_invalid_buffer++;
netdev_dbg(netdev, "recycling invalid buffer\n");
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (unlikely(rc))
break;
} else {
struct sk_buff *skb, *new_skb;
- int length = ibmveth_rxq_frame_length(adapter);
- int offset = ibmveth_rxq_frame_offset(adapter);
- int csum_good = ibmveth_rxq_csum_good(adapter);
- int lrg_pkt = ibmveth_rxq_large_packet(adapter);
+ int length = ibmveth_rxq_frame_length(adapter, queue_index);
+ int offset = ibmveth_rxq_frame_offset(adapter, queue_index);
+ int csum_good = ibmveth_rxq_csum_good(adapter, queue_index);
+ int lrg_pkt = ibmveth_rxq_large_packet(adapter, queue_index);
__sum16 iph_check = 0;
- skb = ibmveth_rxq_get_buffer(adapter);
+ skb = ibmveth_rxq_get_buffer(adapter, queue_index);
if (unlikely(!skb))
break;
@@ -2320,12 +2357,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
length);
if (rx_flush)
ibmveth_flush_buffer(skb->data,
- length + offset);
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+ length + offset);
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (unlikely(rc))
break;
skb = new_skb;
} else {
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, false)))
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, false);
+ if (unlikely(rc))
break;
skb_reserve(skb, offset);
}
@@ -2361,7 +2400,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
}
- ibmveth_replenish_task(adapter, 0);
+ ibmveth_replenish_task(adapter, queue_index);
if (frames_processed == budget)
goto out;
@@ -2372,15 +2411,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
/* We think we are done - reenable interrupts,
* then check once more to make sure we are done.
*/
- lpar_rc = h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_ENABLE);
- if (WARN_ON(lpar_rc != H_SUCCESS)) {
+ lpar_rc = ibmveth_enable_irq(adapter, queue_index);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n",
+ queue_index, lpar_rc);
schedule_work(&adapter->work);
goto out;
}
- if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
- lpar_rc = h_vio_signal(adapter->vdev->unit_address,
- VIO_IRQ_DISABLE);
+ if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+ napi_schedule(napi)) {
+ lpar_rc = ibmveth_disable_irq(adapter, queue_index);
+ WARN_ON(lpar_rc != H_SUCCESS);
goto restart_poll;
}
@@ -2511,9 +2554,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
static void ibmveth_poll_controller(struct net_device *dev)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i;
- ibmveth_replenish_task(adapter, 0);
- ibmveth_interrupt(dev->irq, &adapter->napi[0]);
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ ibmveth_replenish_task(adapter, i);
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ ibmveth_interrupt(adapter->queue_irq[i], &adapter->napi[i]);
}
#endif
@@ -2531,8 +2578,7 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
struct ibmveth_adapter *adapter;
struct iommu_table *tbl;
unsigned long ret;
- int i;
- int rxqentries = 1;
+ int i, q;
tbl = get_iommu_table_base(&vdev->dev);
@@ -2547,18 +2593,22 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
/* add size of mapped tx buffers */
ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- /* add the size of the active receive buffers */
- if (adapter->rx_buff_pool[0][i].active)
- ret +=
- adapter->rx_buff_pool[0][i].size *
- IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
- buff_size, tbl);
- rxqentries += adapter->rx_buff_pool[0][i].size;
- }
- /* add the size of the receive queue entries */
- ret += IOMMU_PAGE_ALIGN(
- rxqentries * sizeof(struct ibmveth_rx_q_entry), tbl);
+ for (q = 0; q < adapter->num_rx_queues; q++) {
+ int rxqentries = 1;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ /* add the size of the active receive buffers */
+ if (adapter->rx_buff_pool[q][i].active)
+ ret += adapter->rx_buff_pool[q][i].size *
+ IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[q][i].buff_size,
+ tbl);
+ rxqentries += adapter->rx_buff_pool[q][i].size;
+ }
+
+ /* add the size of the receive queue entries */
+ ret += IOMMU_PAGE_ALIGN(rxqentries *
+ sizeof(struct ibmveth_rx_q_entry), tbl);
+ }
return ret;
}
@@ -2660,7 +2710,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
return -EINVAL;
}
- netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter), IBMVETH_MAX_QUEUES, 1);
+ netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter),
+ IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES);
if (!netdev)
return -ENOMEM;
@@ -2673,7 +2724,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
ibmveth_init_link_settings(netdev);
- netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+ netif_napi_add_weight(netdev, &adapter->napi[i], ibmveth_poll, 16);
netdev->irq = dev->irq;
netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2705,16 +2757,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev->features |= NETIF_F_FRAGLIST;
}
- /* Initialize queue count - always 1 for now */
- adapter->multi_queue = 0;
- adapter->num_rx_queues = 1;
+ if (ret == H_SUCCESS &&
+ (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) {
+ adapter->multi_queue = 1;
+ adapter->num_rx_queues = min(num_online_cpus(), IBMVETH_DEFAULT_QUEUES);
+ netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n",
+ adapter->num_rx_queues);
+ } else {
+ adapter->multi_queue = 0;
+ adapter->num_rx_queues = 1;
+ }
if (ret == H_SUCCESS &&
(ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
- adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
+ if (adapter->multi_queue)
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_QUEUE;
+ else
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
+
netdev_dbg(netdev,
"RX Multi-buffer hcall supported by FW, batch set to %u\n",
- adapter->rx_buffers_per_hcall);
+ adapter->rx_buffers_per_hcall);
} else {
adapter->rx_buffers_per_hcall = 1;
netdev_dbg(netdev,
@@ -3057,17 +3120,23 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
correlator = ((u64)IBMVETH_NUM_BUFF_POOLS << 32) | 0;
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, true));
correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, true));
correlator = (u64)0 | 0;
pool->skbuff[0] = NULL;
- KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EFAULT,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EFAULT,
+ ibmveth_remove_buffer_from_pool(adapter, correlator, 0, true));
flush_work(&adapter->work);
}
@@ -3111,15 +3180,15 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
adapter->rx_queue[0].queue_addr[0].correlator = (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
- KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
adapter->rx_queue[0].queue_addr[0].correlator =
(u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
- KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
pool->skbuff[0] = skb;
adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
- KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter, 0));
flush_work(&adapter->work);
}
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index d2ceeccd5fbd..f7b20fd01acb 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -14,6 +14,8 @@
#ifndef _IBMVETH_H
#define _IBMVETH_H
+#include <linux/spinlock_types.h>
+
/* constants for H_MULTICAST_CTRL */
#define IbmVethMcastReceptionModifyBit 0x80000UL
#define IbmVethMcastReceptionEnableBit 0x20000UL
@@ -28,6 +30,7 @@
#define IbmVethMcastRemoveFilter 0x2UL
#define IbmVethMcastClearFilterTable 0x3UL
+#define IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT 0x0000000000080000UL
#define IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT 0x0000000000040000UL
#define IBMVETH_ILLAN_LRG_SR_ENABLED 0x0000000000010000UL
#define IBMVETH_ILLAN_LRG_SND_SUPPORT 0x0000000000008000UL
@@ -279,9 +282,11 @@ static inline long h_illan_attributes(unsigned long unit_address,
#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
#define IBMVETH_MAX_QUEUES 16U
#define IBMVETH_DEFAULT_QUEUES 8U
-#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_MAX_RX_QUEUES 16U
#define IBMVETH_DEFAULT_RX_QUEUES 1U
-#define IBMVETH_MAX_RX_PER_HCALL 8U
+#define IBMVETH_MAX_RX_REGULAR 8U
+#define IBMVETH_MAX_RX_QUEUE 12U
+#define IBMVETH_MAX_RX_PER_HCALL 12U
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
static int pool_count[] = { 256, 512, 256, 256, 256 };
@@ -336,6 +341,7 @@ struct ibmveth_rx_q {
dma_addr_t queue_dma;
u32 queue_len;
struct ibmveth_rx_q_entry *queue_addr;
+ spinlock_t replenish_lock; /* serializes per-queue buffer replenish */
};
struct ibmveth_adapter {
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 11/15] ibmveth: Add per-queue RX and TX statistics collection and reporting
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (9 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
@ 2026-07-06 19:35 ` Mingming Cao
2026-07-14 12:59 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs Mingming Cao
` (3 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:35 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Count per-queue RX stats in poll, replenish, and the IRQ handler:
packets, bytes, polls, large_packets, invalid_buffers, no_buffer_drops,
and interrupts. Track transmit counters per TX queue to avoid cache
line contention in the xmit hot path.
Expose counters via ethtool -S (rxN_* and txN_*), get_stats64()
aggregation for ip -s and /proc/net/dev, and hcall_stats counters.
Fix get_channels() reporting: max_rx is IBMVETH_MAX_RX_QUEUES only when
MQ firmware is enabled, rx_count tracks adapter->num_rx_queues.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 269 ++++++++++++++++++++++++++---
drivers/net/ethernet/ibm/ibmveth.h | 13 ++
2 files changed, 259 insertions(+), 23 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 863e5c68b42c..4e3f49b6346f 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -98,7 +98,15 @@ static struct ibmveth_stat ibmveth_stats[] = {
{ "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) },
{ "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) },
{ "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) },
- { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }
+ { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) },
+ { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
+ { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) },
+ { "hcall_add_bufs_queue", IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) },
+ { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) },
+ { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) },
+ { "hcall_free_lan_queue", IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) },
+ { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) },
+ { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) },
};
/* simple methods of getting data from the current rxq entry */
@@ -244,6 +252,33 @@ static void ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
adapter->rx_qstats = NULL;
}
+/**
+ * ibmveth_alloc_tx_qstats - Allocate per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->tx_qstats = kcalloc(IBMVETH_MAX_QUEUES,
+ sizeof(struct ibmveth_tx_queue_stats),
+ GFP_KERNEL);
+ if (!adapter->tx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_tx_qstats - Free per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->tx_qstats);
+ adapter->tx_qstats = NULL;
+}
+
/**
* ibmveth_alloc_rx_queues - Allocate per-queue RX resources
* @adapter: ibmveth adapter structure
@@ -847,6 +882,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
u64 drops = be64_to_cpup(p);
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[i].no_buffer_drops = drops;
if (i == 0)
adapter->rx_no_buffer = drops;
}
@@ -1618,6 +1655,10 @@ static int ibmveth_open(struct net_device *netdev)
if (rc)
goto out_cleanup_rx_interrupts;
+ rc = ibmveth_alloc_tx_qstats(adapter);
+ if (rc)
+ goto out_free_tx_resources;
+
netif_tx_start_all_queues(netdev);
netdev_dbg(netdev, "open complete\n");
@@ -1658,6 +1699,7 @@ static int ibmveth_close(struct net_device *netdev)
}
}
+ ibmveth_free_tx_qstats(adapter);
ibmveth_free_tx_resources(adapter);
ibmveth_cleanup_rx_interrupts(adapter);
ibmveth_update_rx_no_buffer(adapter);
@@ -1925,22 +1967,107 @@ static int ibmveth_set_features(struct net_device *dev,
return rc1 ? rc1 : rc2;
}
+/**
+ * ibmveth_aggregate_rx_qstats - Sum per-queue RX stats into globals
+ * @adapter: ibmveth adapter
+ *
+ * Cold path only (ethtool). Keeps legacy global counters meaningful for
+ * tools that read the adapter-level fields in ibmveth_stats[].
+ */
+static void ibmveth_aggregate_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ u64 total_invalid = 0;
+ u64 total_large = 0;
+ int i;
+
+ if (!adapter->rx_qstats)
+ return;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ total_invalid += adapter->rx_qstats[i].invalid_buffers;
+ total_large += adapter->rx_qstats[i].large_packets;
+ }
+
+ adapter->rx_invalid_buffer = total_invalid;
+ adapter->rx_large_packets = total_large;
+}
+
+/**
+ * ibmveth_aggregate_tx_qstats - Sum per-queue TX stats into globals
+ * @adapter: ibmveth adapter
+ *
+ * Cold path only (ethtool). Keeps legacy global counters meaningful for
+ * tools that read the adapter-level fields in ibmveth_stats[].
+ */
+static void ibmveth_aggregate_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ u64 total_large = 0;
+ u64 total_send_failed = 0;
+ int i;
+
+ if (!adapter->tx_qstats)
+ return;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++) {
+ total_large += adapter->tx_qstats[i].large_packets;
+ total_send_failed += adapter->tx_qstats[i].send_failures;
+ }
+
+ adapter->tx_large_packets = total_large;
+ adapter->tx_send_failed = total_send_failed;
+}
+
static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ u8 *p = data;
int i;
if (stringset != ETH_SS_STATS)
return;
- for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++, data += ETH_GSTRING_LEN)
- memcpy(data, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+ for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++) {
+ memcpy(p, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+ p += ETH_GSTRING_LEN;
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ ethtool_sprintf(&p, "rx%d_packets", i);
+ ethtool_sprintf(&p, "rx%d_bytes", i);
+ ethtool_sprintf(&p, "rx%d_interrupts", i);
+ ethtool_sprintf(&p, "rx%d_polls", i);
+ ethtool_sprintf(&p, "rx%d_large_packets", i);
+ ethtool_sprintf(&p, "rx%d_invalid_buffers", i);
+ ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
+ }
+
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ ethtool_sprintf(&p, "tx%d_packets", i);
+ ethtool_sprintf(&p, "tx%d_bytes", i);
+ ethtool_sprintf(&p, "tx%d_large_packets", i);
+ ethtool_sprintf(&p, "tx%d_dropped_packets", i);
+ ethtool_sprintf(&p, "tx%d_send_failures", i);
+ ethtool_sprintf(&p, "tx%d_checksum_offload", i);
+ }
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ ethtool_sprintf(&p, "pool%d_size", i);
+ ethtool_sprintf(&p, "pool%d_active", i);
+ ethtool_sprintf(&p, "pool%d_available", i);
+ }
}
static int ibmveth_get_sset_count(struct net_device *dev, int sset)
{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+
switch (sset) {
case ETH_SS_STATS:
- return ARRAY_SIZE(ibmveth_stats);
+ return ARRAY_SIZE(ibmveth_stats) +
+ adapter->num_rx_queues * IBMVETH_NUM_RX_QSTATS +
+ dev->real_num_tx_queues * IBMVETH_NUM_TX_QSTATS +
+ IBMVETH_NUM_BUFF_POOLS * 3;
default:
return -EOPNOTSUPP;
}
@@ -1949,21 +2076,62 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
static void ibmveth_get_ethtool_stats(struct net_device *dev,
struct ethtool_stats *stats, u64 *data)
{
- int i;
struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i, j;
+
+ ibmveth_aggregate_rx_qstats(adapter);
+ ibmveth_aggregate_tx_qstats(adapter);
for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++)
data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset);
+
+ for (j = 0; j < adapter->num_rx_queues; j++) {
+ if (adapter->rx_qstats) {
+ data[i++] = adapter->rx_qstats[j].packets;
+ data[i++] = adapter->rx_qstats[j].bytes;
+ data[i++] = adapter->rx_qstats[j].interrupts;
+ data[i++] = adapter->rx_qstats[j].polls;
+ data[i++] = adapter->rx_qstats[j].large_packets;
+ data[i++] = adapter->rx_qstats[j].invalid_buffers;
+ data[i++] = adapter->rx_qstats[j].no_buffer_drops;
+ } else {
+ i += IBMVETH_NUM_RX_QSTATS;
+ }
+ }
+
+ for (j = 0; j < dev->real_num_tx_queues; j++) {
+ if (adapter->tx_qstats) {
+ data[i++] = adapter->tx_qstats[j].packets;
+ data[i++] = adapter->tx_qstats[j].bytes;
+ data[i++] = adapter->tx_qstats[j].large_packets;
+ data[i++] = adapter->tx_qstats[j].dropped_packets;
+ data[i++] = adapter->tx_qstats[j].send_failures;
+ data[i++] = adapter->tx_qstats[j].checksum_offload;
+ } else {
+ i += IBMVETH_NUM_TX_QSTATS;
+ }
+ }
+
+ for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+ data[i++] = adapter->rx_buff_pool[0][j].size;
+ data[i++] = adapter->rx_buff_pool[0][j].active;
+ data[i++] = atomic_read(&adapter->rx_buff_pool[0][j].available);
+ }
}
static void ibmveth_get_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
channels->max_tx = ibmveth_real_max_tx_queues();
channels->tx_count = netdev->real_num_tx_queues;
- channels->max_rx = netdev->real_num_rx_queues;
- channels->rx_count = netdev->real_num_rx_queues;
+ if (adapter->multi_queue)
+ channels->max_rx = IBMVETH_MAX_RX_QUEUES;
+ else
+ channels->max_rx = 1;
+ channels->rx_count = adapter->num_rx_queues;
}
static int ibmveth_set_channels(struct net_device *netdev,
@@ -2061,12 +2229,15 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,
return 1;
}
+ adapter->hcall_stats.send_lan++;
return 0;
}
static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
- struct net_device *netdev)
+ struct ibmveth_adapter *adapter,
+ int queue_num)
{
+ struct net_device *netdev = adapter->netdev;
struct ethhdr *ether_header;
int ret = 0;
@@ -2074,7 +2245,8 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
if (ether_addr_equal(ether_header->h_dest, netdev->dev_addr)) {
netdev_dbg(netdev, "veth doesn't support loopback packets, dropping packet.\n");
- netdev->stats.tx_dropped++;
+ if (adapter->tx_qstats)
+ adapter->tx_qstats[queue_num].dropped_packets++;
ret = -EOPNOTSUPP;
}
@@ -2090,7 +2262,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
int i, queue_num = skb_get_queue_mapping(skb);
unsigned long mss = 0;
- if (ibmveth_is_packet_unsupported(skb, netdev))
+ if (ibmveth_is_packet_unsupported(skb, adapter, queue_num))
goto out;
/* veth can't checksum offload UDP */
if (skb->ip_summed == CHECKSUM_PARTIAL &&
@@ -2101,7 +2273,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
skb_checksum_help(skb)) {
netdev_err(netdev, "tx: failed to checksum packet\n");
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
@@ -2113,6 +2285,8 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
desc_flags |= (IBMVETH_BUF_NO_CSUM | IBMVETH_BUF_CSUM_GOOD);
+ adapter->tx_qstats[queue_num].checksum_offload++;
+
/* Need to zero out the checksum */
buf[0] = 0;
buf[1] = 0;
@@ -2124,7 +2298,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (skb->ip_summed == CHECKSUM_PARTIAL && skb_is_gso(skb)) {
if (adapter->fw_large_send_support) {
mss = (unsigned long)skb_shinfo(skb)->gso_size;
- adapter->tx_large_packets++;
+ adapter->tx_qstats[queue_num].large_packets++;
} else if (!skb_is_gso_v6(skb)) {
/* Put -1 in the IP checksum to tell phyp it
* is a largesend packet. Put the mss in
@@ -2133,7 +2307,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
ip_hdr(skb)->check = 0xffff;
tcp_hdr(skb)->check =
cpu_to_be16(skb_shinfo(skb)->gso_size);
- adapter->tx_large_packets++;
+ adapter->tx_qstats[queue_num].large_packets++;
}
}
@@ -2141,7 +2315,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (unlikely(skb->len > adapter->tx_ltb_size)) {
netdev_err(adapter->netdev, "tx: packet size (%u) exceeds ltb (%u)\n",
skb->len, adapter->tx_ltb_size);
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
@@ -2158,7 +2332,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (unlikely(total_bytes != skb->len)) {
netdev_err(adapter->netdev, "tx: incorrect packet len copied into ltb (%u != %u)\n",
skb->len, total_bytes);
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
desc.fields.flags_len = desc_flags | skb->len;
@@ -2167,11 +2341,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
dma_wmb();
if (ibmveth_send(adapter, desc.desc, mss)) {
- adapter->tx_send_failed++;
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].send_failures++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
} else {
- netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += skb->len;
+ adapter->tx_qstats[queue_num].packets++;
+ adapter->tx_qstats[queue_num].bytes += skb->len;
}
out:
@@ -2311,6 +2485,9 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
return 0;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].polls++;
+
restart_poll:
while (frames_processed < budget) {
if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
@@ -2319,7 +2496,10 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
smp_rmb();
if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
wmb(); /* suggested by larson1 */
- adapter->rx_invalid_buffer++;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].invalid_buffers++;
+ else
+ adapter->rx_invalid_buffer++;
netdev_dbg(netdev, "recycling invalid buffer\n");
rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
if (unlikely(rc))
@@ -2384,7 +2564,10 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
if ((length > netdev->mtu + ETH_HLEN) ||
lrg_pkt || iph_check == 0xffff) {
ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
- adapter->rx_large_packets++;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].large_packets++;
+ else
+ adapter->rx_large_packets++;
}
if (csum_good) {
@@ -2394,8 +2577,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
napi_gro_receive(napi, skb); /* send it up */
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += length;
+ if (adapter->rx_qstats) {
+ adapter->rx_qstats[queue_index].packets++;
+ adapter->rx_qstats[queue_index].bytes += length;
+ }
+
frames_processed++;
}
}
@@ -2444,6 +2630,9 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
return IRQ_NONE;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[qindex].interrupts++;
+
if (napi_schedule_prep(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(lpar_rc != H_SUCCESS);
@@ -2656,6 +2845,39 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,
return vlan_features_check(skb, features);
}
+/**
+ * ibmveth_get_stats64 - Return aggregated per-queue statistics
+ * @dev: network device
+ * @stats: rtnl link statistics storage
+ *
+ * Sums per-queue rx_qstats and tx_qstats into the rtnl counters.
+ * Callers use ndo_get_stats64(); avoid updating netdev->stats on the
+ * xmit/poll paths to keep per-queue counters off the hot cache line.
+ */
+static void ibmveth_get_stats64(struct net_device *dev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i;
+
+ if (adapter->rx_qstats) {
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ stats->rx_packets += adapter->rx_qstats[i].packets;
+ stats->rx_bytes += adapter->rx_qstats[i].bytes;
+ }
+ }
+
+ if (adapter->tx_qstats) {
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ stats->tx_packets += adapter->tx_qstats[i].packets;
+ stats->tx_bytes += adapter->tx_qstats[i].bytes;
+ stats->tx_dropped += adapter->tx_qstats[i].dropped_packets;
+ }
+ }
+
+ stats->tx_errors = dev->stats.tx_errors;
+}
+
static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_open = ibmveth_open,
.ndo_stop = ibmveth_close,
@@ -2668,6 +2890,7 @@ static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_validate_addr = eth_validate_addr,
.ndo_set_mac_address = ibmveth_set_mac_addr,
.ndo_features_check = ibmveth_features_check,
+ .ndo_get_stats64 = ibmveth_get_stats64,
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = ibmveth_poll_controller,
#endif
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index f7b20fd01acb..390c660af979 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -316,9 +316,21 @@ struct ibmveth_rx_queue_stats {
u64 no_buffer_drops;
};
+struct ibmveth_tx_queue_stats {
+ u64 packets;
+ u64 bytes;
+ u64 large_packets;
+ u64 dropped_packets;
+ u64 send_failures;
+ u64 checksum_offload;
+};
+
#define IBMVETH_NUM_RX_QSTATS \
(sizeof(struct ibmveth_rx_queue_stats) / sizeof(u64))
+#define IBMVETH_NUM_TX_QSTATS \
+ (sizeof(struct ibmveth_tx_queue_stats) / sizeof(u64))
+
struct ibmveth_buff_pool {
u32 size;
u32 index;
@@ -386,6 +398,7 @@ struct ibmveth_adapter {
/* Multi-queue statistics */
struct ibmveth_hcall_stats hcall_stats;
struct ibmveth_rx_queue_stats *rx_qstats;
+ struct ibmveth_tx_queue_stats *tx_qstats;
/* Ethtool settings */
u8 duplex;
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (10 preceding siblings ...)
2026-07-06 19:35 ` [PATCH net-next v3 11/15] ibmveth: Add per-queue RX and TX statistics collection and reporting Mingming Cao
@ 2026-07-06 19:36 ` Mingming Cao
2026-07-14 13:01 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
` (2 subsequent siblings)
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:36 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Add a read-only buffer_pools sysfs attribute under the VIO device that
lists size, buff_size, active, and available for every RX queue and
pool: runtime per-queue buffer pressure during MQ operation. ethtool -S
pool%d_* (previous patch) reports queue-0 static probe geometry only;
sysfs is the right place for dynamic per-queue pool state at scale.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 56 ++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 4e3f49b6346f..ecc472ee8f71 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -2896,6 +2896,52 @@ static const struct net_device_ops ibmveth_netdev_ops = {
#endif
};
+static const struct attribute_group ibmveth_attr_group;
+
+static ssize_t buffer_pools_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct net_device *netdev = dev_get_drvdata(dev);
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+ int len = 0;
+ int i, j;
+
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "Queue Pool Size BuffSize Active Available\n");
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "----- ---- ---- -------- ------ ---------\n");
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[i][j];
+
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "%5d %4d %4u %8u %6d %9d\n",
+ i, j, pool->size, pool->buff_size,
+ pool->active,
+ atomic_read(&pool->available));
+
+ if (len >= PAGE_SIZE - 100)
+ goto out;
+ }
+ }
+
+out:
+ return len;
+}
+static DEVICE_ATTR_RO(buffer_pools);
+
+static struct attribute *ibmveth_attrs[] = {
+ &dev_attr_buffer_pools.attr,
+ NULL,
+};
+
+static const struct attribute_group ibmveth_attr_group = {
+ .attrs = ibmveth_attrs,
+};
+
static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
{
int rc, i, mac_len;
@@ -3056,6 +3102,14 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev_dbg(netdev, "registered\n");
+ rc = sysfs_create_group(&dev->dev.kobj, &ibmveth_attr_group);
+ if (rc) {
+ netdev_err(netdev, "failed to create sysfs attributes rc=%d\n", rc);
+ unregister_netdev(netdev);
+ free_netdev(netdev);
+ return rc;
+ }
+
return 0;
}
@@ -3067,6 +3121,8 @@ static void ibmveth_remove(struct vio_dev *dev)
cancel_work_sync(&adapter->work);
+ sysfs_remove_group(&dev->dev.kobj, &ibmveth_attr_group);
+
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
kobject_put(&adapter->rx_buff_pool[0][i].kobj);
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (11 preceding siblings ...)
2026-07-06 19:36 ` [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs Mingming Cao
@ 2026-07-06 19:36 ` Mingming Cao
2026-07-14 13:03 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
2026-07-06 19:36 ` [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after " Mingming Cao
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:36 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Add helpers to register/deregister individual RX queues and scale
queue count up or down while the netdev stays up. Surviving queues
keep PHYP handles, buffer pools, and IRQ state across resize.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 406 ++++++++++++++++++++++++++++-
1 file changed, 401 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index ecc472ee8f71..ac4d89a66a8d 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -589,6 +589,54 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
adapter->queue_irq[0] = 0;
}
+/**
+ * ibmveth_setup_single_rx_interrupt - Setup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to setup
+ *
+ * Registers the IRQ handler for one queue. Used during incremental
+ * scale-up when adding new RX queues; the caller enables NAPI via
+ * napi_enable() after ibmveth_enable_irq().
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_setup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ struct net_device *netdev = adapter->netdev;
+ int rc;
+
+ rc = request_irq(adapter->queue_irq[queue_idx], ibmveth_interrupt,
+ 0, netdev->name, &adapter->napi[queue_idx]);
+ if (rc) {
+ netdev_err(netdev, "request_irq() failed for queue %d: %d\n",
+ queue_idx, rc);
+ return rc;
+ }
+
+ netdev_dbg(netdev, "Setup IRQ %d for queue %d\n",
+ adapter->queue_irq[queue_idx], queue_idx);
+ return 0;
+}
+
+/**
+ * ibmveth_cleanup_single_rx_interrupt - Cleanup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to cleanup
+ *
+ * Frees the IRQ handler for one queue. Used during incremental scale-down.
+ */
+static void
+ibmveth_cleanup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ if (adapter->queue_irq[queue_idx]) {
+ free_irq(adapter->queue_irq[queue_idx], &adapter->napi[queue_idx]);
+ netdev_dbg(adapter->netdev, "Freed IRQ for queue %d\n", queue_idx);
+ }
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
@@ -897,18 +945,22 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
unsigned long flags;
int i;
- if (queue_index >= adapter->num_rx_queues)
- return;
-
adapter->replenish_task_cycles++;
+ if (queue_index >= adapter->num_rx_queues) {
+ netdev_dbg(adapter->netdev,
+ "Skipping replenish for freed queue %d (num_queues=%d)\n",
+ queue_index, adapter->num_rx_queues);
+ return;
+ }
+
spin_lock_irqsave(&rxq->replenish_lock, flags);
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
struct ibmveth_buff_pool *pool =
&adapter->rx_buff_pool[queue_index][i];
- if (pool->active &&
+ if (pool->active && pool->free_map &&
(atomic_read(&pool->available) < pool->threshold))
ibmveth_replenish_buffer_pool(adapter, pool,
queue_index);
@@ -1080,6 +1132,138 @@ static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
adapter->num_rx_queues);
}
+/**
+ * ibmveth_alloc_single_rx_queue - Allocate resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to allocate
+ * @rxq_entries: Number of RX queue entries
+ *
+ * Allocates buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-up without affecting existing queues.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx,
+ int rxq_entries)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+ int i, rc = -ENOMEM;
+
+ adapter->buffer_list_addr[queue_idx] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[queue_idx]) {
+ netdev_err(netdev, "unable to allocate buffer list for queue %d\n",
+ queue_idx);
+ return -ENOMEM;
+ }
+
+ adapter->rx_queue[queue_idx].queue_len =
+ sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+ adapter->rx_queue[queue_idx].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ &adapter->rx_queue[queue_idx].queue_dma,
+ GFP_KERNEL);
+ if (!adapter->rx_queue[queue_idx].queue_addr) {
+ netdev_err(netdev, "unable to allocate RX queue for queue %d\n",
+ queue_idx);
+ goto out_free_buflist;
+ }
+
+ adapter->buffer_list_dma[queue_idx] =
+ dma_map_single(dev, adapter->buffer_list_addr[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[queue_idx])) {
+ netdev_err(netdev, "unable to map buffer list for queue %d\n",
+ queue_idx);
+ goto out_free_rxq;
+ }
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ adapter->rx_buff_pool[queue_idx][i].size =
+ adapter->rx_buff_pool[0][i].size;
+ adapter->rx_buff_pool[queue_idx][i].buff_size =
+ adapter->rx_buff_pool[0][i].buff_size;
+ adapter->rx_buff_pool[queue_idx][i].threshold =
+ adapter->rx_buff_pool[0][i].threshold;
+ adapter->rx_buff_pool[queue_idx][i].active =
+ adapter->rx_buff_pool[0][i].active;
+ }
+
+ rc = ibmveth_alloc_queue_buffer_pools(adapter, queue_idx);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to allocate buffer pools for queue %d\n",
+ queue_idx);
+ goto out_unmap_buflist;
+ }
+
+ adapter->rx_queue[queue_idx].index = 0;
+ adapter->rx_queue[queue_idx].num_slots = rxq_entries;
+ adapter->rx_queue[queue_idx].toggle = 1;
+ spin_lock_init(&adapter->rx_queue[queue_idx].replenish_lock);
+
+ netdev_dbg(netdev,
+ "Allocated queue %d: buffer_list @ %p (DMA: 0x%llx), rx_queue @ %p (DMA: 0x%llx), %d entries\n",
+ queue_idx, adapter->buffer_list_addr[queue_idx],
+ (unsigned long long)adapter->buffer_list_dma[queue_idx],
+ adapter->rx_queue[queue_idx].queue_addr,
+ (unsigned long long)adapter->rx_queue[queue_idx].queue_dma,
+ rxq_entries);
+
+ return 0;
+
+out_unmap_buflist:
+ dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[queue_idx] = 0;
+out_free_rxq:
+ dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ adapter->rx_queue[queue_idx].queue_addr,
+ adapter->rx_queue[queue_idx].queue_dma);
+ adapter->rx_queue[queue_idx].queue_addr = NULL;
+out_free_buflist:
+ free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+ adapter->buffer_list_addr[queue_idx] = NULL;
+ return rc;
+}
+
+/**
+ * ibmveth_free_single_rx_queue - Free resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to free
+ *
+ * Frees buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-down without affecting remaining queues.
+ */
+static void
+ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
+{
+ struct device *dev = &adapter->vdev->dev;
+
+ ibmveth_free_queue_buffer_pools(adapter, queue_idx);
+
+ if (adapter->buffer_list_dma[queue_idx]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[queue_idx] = 0;
+ }
+
+ if (adapter->rx_queue[queue_idx].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ adapter->rx_queue[queue_idx].queue_addr,
+ adapter->rx_queue[queue_idx].queue_dma);
+ adapter->rx_queue[queue_idx].queue_addr = NULL;
+ }
+
+ if (adapter->buffer_list_addr[queue_idx]) {
+ free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+ adapter->buffer_list_addr[queue_idx] = NULL;
+ }
+
+ netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
+}
+
/**
* ibmveth_remove_buffer_from_pool - remove a buffer from a pool
* @adapter: adapter instance
@@ -1192,6 +1376,49 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
return 0;
}
+/**
+ * ibmveth_drain_rx_queue - Drain pending buffers from an RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Queue index to drain
+ *
+ * Recycles all pending buffers back to the per-queue buffer pools.
+ * Must be called with NAPI disabled for this queue.
+ *
+ * Return: Number of buffers drained
+ */
+static int
+ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
+{
+ struct net_device *netdev = adapter->netdev;
+ int drained = 0;
+ int limit = adapter->rx_queue[queue_index].num_slots;
+ int rc;
+
+ netdev_dbg(netdev, "Draining RX queue %d (limit: %d slots)\n",
+ queue_index, limit);
+
+ while (drained < limit &&
+ ibmveth_rxq_pending_buffer(adapter, queue_index)) {
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to harvest buffer from queue %d during drain: %d\n",
+ queue_index, rc);
+ break;
+ }
+ drained++;
+ }
+
+ if (drained > 0)
+ netdev_dbg(netdev, "Drained %d buffer(s) from RX queue %d\n",
+ drained, queue_index);
+ else
+ netdev_dbg(netdev, "No buffers to drain from RX queue %d\n",
+ queue_index);
+
+ return drained;
+}
+
static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
{
dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
@@ -1459,7 +1686,7 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
* the IRQ mapping for subordinate queues. Queue 0 is freed only through
* ibmveth_free_all_queues() (H_FREE_LOGICAL_LAN).
*/
-static void __maybe_unused
+static void
ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
int queue_idx)
{
@@ -1491,6 +1718,175 @@ ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
}
+/**
+ * ibmveth_resize_rx_queues_incremental - Resize RX queue count incrementally
+ * @adapter: ibmveth adapter structure
+ * @new_count: Target number of RX queues
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Adds or removes RX queues without tearing down the entire adapter.
+ * Active queues continue receiving during scale-up; scale-down drains
+ * excess queues before deregistering them with the hypervisor.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
+ int new_count, int rxq_entries)
+{
+ struct net_device *netdev = adapter->netdev;
+ u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
+ int old_count = adapter->num_rx_queues;
+ int failed_queue;
+ int rc, i;
+
+ if (old_count == new_count) {
+ netdev_dbg(netdev, "RX queue count unchanged (%d), nothing to do\n",
+ old_count);
+ return 0;
+ }
+
+ if (new_count < 1 || new_count > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev, "Invalid RX queue count %d (must be 1-%d)\n",
+ new_count, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ netdev_info(netdev, "Incrementally resizing RX queues: %d to %d\n",
+ old_count, new_count);
+
+ if (new_count > old_count) {
+ netdev_dbg(netdev, "Scale-up: adding queues %d-%d\n",
+ old_count, new_count - 1);
+
+ for (i = old_count; i < new_count; i++) {
+ rc = ibmveth_alloc_single_rx_queue(adapter, i, rxq_entries);
+ if (rc) {
+ netdev_err(netdev, "Failed to allocate queue %d: %d\n",
+ i, rc);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_register_single_rx_queue(adapter, i,
+ mac_address);
+ if (rc) {
+ netdev_err(netdev, "Failed to register queue %d: %d\n",
+ i, rc);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_setup_single_rx_interrupt(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to setup IRQ for queue %d: %d\n",
+ i, rc);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_enable_irq(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d: %d\n",
+ i, rc);
+ ibmveth_cleanup_single_rx_interrupt(adapter, i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ napi_enable(&adapter->napi[i]);
+ }
+
+ adapter->num_rx_queues = new_count;
+
+ for (i = old_count; i < new_count; i++)
+ ibmveth_replenish_task(adapter, i);
+
+ rc = netif_set_real_num_rx_queues(netdev, new_count);
+ if (rc) {
+ netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+ new_count, rc);
+ goto cleanup_new_queues;
+ }
+ } else {
+ netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
+ new_count, old_count - 1);
+
+ for (i = new_count; i < old_count; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = new_count; i < old_count; i++)
+ ibmveth_drain_rx_queue(adapter, i);
+
+ synchronize_net();
+
+ rc = netif_set_real_num_rx_queues(netdev, new_count);
+ if (rc) {
+ netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+ new_count, rc);
+ for (i = new_count; i < old_count; i++)
+ napi_enable(&adapter->napi[i]);
+ return rc;
+ }
+
+ /* Disable hypervisor interrupts and wait for handlers to complete
+ * before updating num_rx_queues.
+ */
+ for (i = new_count; i < old_count; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ adapter->num_rx_queues = new_count;
+
+ for (i = new_count; i < old_count; i++) {
+ ibmveth_cleanup_single_rx_interrupt(adapter, i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ }
+ }
+
+ netdev_info(netdev, "Successfully resized to %d RX queues (incremental)\n",
+ adapter->num_rx_queues);
+
+ if (firmware_has_feature(FW_FEATURE_CMO))
+ vio_cmo_set_dev_desired(adapter->vdev,
+ ibmveth_get_desired_dma(adapter->vdev));
+
+ return 0;
+
+cleanup_new_queues:
+ failed_queue = i;
+ netdev_err(netdev,
+ "Scale-up failed at queue %d, cleaning up queues %d-%d\n",
+ failed_queue, old_count, failed_queue - 1);
+ for (i = old_count; i < failed_queue; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = old_count; i < failed_queue; i++)
+ ibmveth_drain_rx_queue(adapter, i);
+
+ synchronize_net();
+
+ for (i = old_count; i < failed_queue; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = old_count; i < failed_queue; i++) {
+ ibmveth_cleanup_single_rx_interrupt(adapter, i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ }
+ adapter->num_rx_queues = old_count;
+ netdev_warn(netdev, "Keeping %d queues after scale-up failure\n",
+ old_count);
+ return rc;
+}
+
/**
* ibmveth_free_all_queues - Free all RX queues at once
* @adapter: ibmveth adapter structure
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to MQ RX queue resize
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (12 preceding siblings ...)
2026-07-06 19:36 ` [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
@ 2026-07-06 19:36 ` Mingming Cao
2026-07-14 13:07 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after " Mingming Cao
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:36 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
Expose incremental RX resize through ethtool channel control.
get_channels() reports rx_count from adapter->num_rx_queues and max_rx
as IBMVETH_MAX_RX_QUEUES when MQ firmware is enabled, else 1.
set_channels() validates rx_count is within 1..IBMVETH_MAX_RX_QUEUES.
When rx_count changes and the interface is up, call
ibmveth_resize_rx_queues_incremental(). When the interface is down,
store the requested rx_count in adapter->num_rx_queues so the next open
registers that many queues. Non-MQ firmware returns -EOPNOTSUPP for
rx > 1.
TX queue changes keep existing stop/wake behavior when tx_count changes.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 58 +++++++++++++++++++++++++++---
1 file changed, 54 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index ac4d89a66a8d..50a332ab83fd 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -2534,19 +2534,69 @@ static int ibmveth_set_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- unsigned int old = netdev->real_num_tx_queues,
- goal = channels->tx_count;
+ unsigned int old_rx = adapter->num_rx_queues;
+ unsigned int goal_rx = channels->rx_count;
+ unsigned int old = netdev->real_num_tx_queues;
+ unsigned int goal = channels->tx_count;
+ int rxq_entries = adapter->rx_queue[0].num_slots;
int rc, i;
/* If ndo_open has not been called yet then don't allocate, just set
* desired netdev_queue's and return
*/
- if (!(netdev->flags & IFF_UP))
+ if (!(netdev->flags & IFF_UP)) {
+ if (goal_rx > 1 && !adapter->multi_queue) {
+ netdev_err(netdev,
+ "Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n",
+ goal_rx);
+ return -EOPNOTSUPP;
+ }
+
+ if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev,
+ "Invalid RX queue count %u (must be 1-%d)\n",
+ goal_rx, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ /* Stash desired RX count; open() publishes it via
+ * netif_set_real_num_rx_queues() after queue registration.
+ */
+ if (goal_rx != adapter->num_rx_queues)
+ adapter->num_rx_queues = goal_rx;
+
return netif_set_real_num_tx_queues(netdev, goal);
+ }
+
+ if (goal_rx > 1 && !adapter->multi_queue) {
+ netdev_err(netdev,
+ "Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n",
+ goal_rx);
+ return -EOPNOTSUPP;
+ }
+
+ if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev,
+ "Invalid RX queue count %u (must be 1-%d)\n",
+ goal_rx, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ if (goal_rx != old_rx) {
+ rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
+ rxq_entries);
+ if (rc) {
+ netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
+ return rc;
+ }
+ }
/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
* but we may need to alloc/free the ltb's.
*/
+ if (goal == old)
+ return 0;
+
netif_tx_stop_all_queues(netdev);
/* Allocate any queue that we need */
@@ -2580,7 +2630,7 @@ static int ibmveth_set_channels(struct net_device *netdev,
netif_tx_wake_all_queues(netdev);
- return rc;
+ return 0;
}
static const struct ethtool_ops netdev_ethtool_ops = {
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (13 preceding siblings ...)
2026-07-06 19:36 ` [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
@ 2026-07-06 19:36 ` Mingming Cao
2026-07-14 13:10 ` Simon Horman
14 siblings, 1 reply; 26+ messages in thread
From: Mingming Cao @ 2026-07-06 19:36 UTC (permalink / raw)
To: netdev
Cc: horms, bjking1, haren, ricklind, mmc, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
After aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor
with a correlator that no longer matches the per-queue buffer pools. Poll
treated this as fatal: ibmveth_rxq_get_buffer() WARNed and returned NULL
without advancing the ring, then restart_poll retried the same slot
forever.
Advance past bad correlators instead of spinning: validate correlators
without WARN_ON, skip invalid slots in poll (count as invalid_buffers),
and advance the RX ring when remove_buffer_from_pool cannot map the
correlator. Rate-limit the bad correlator message.
Complete NAPI when the interface is down or napi_disable is pending so
ibmveth_cleanup_rx_interrupts() can finish. Do not restart_poll in that
window. Close keeps hypervisor IRQ disable before napi_disable (via
cleanup_rx_interrupts()).
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
---
drivers/net/ethernet/ibm/ibmveth.c | 77 ++++++++++++++++++++++--------
1 file changed, 58 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 50a332ab83fd..eb84b4e5f69f 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -158,6 +158,24 @@ static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter,
return be32_to_cpu(rxq->queue_addr[rxq->index].length);
}
+static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
+ int queue_index, u64 correlator)
+{
+ unsigned int pool = correlator >> 32;
+ unsigned int index = correlator & 0xffffffffUL;
+
+ return pool < IBMVETH_NUM_BUFF_POOLS &&
+ index < adapter->rx_buff_pool[queue_index][pool].size;
+}
+
+static void ibmveth_rxq_advance(struct ibmveth_rx_q *rxq)
+{
+ if (++rxq->index == rxq->num_slots) {
+ rxq->index = 0;
+ rxq->toggle = !rxq->toggle;
+ }
+}
+
static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
int queue_index)
{
@@ -1284,17 +1302,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
unsigned int free_index;
struct sk_buff *skb;
- if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
- schedule_work(&adapter->work);
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
return -EINVAL;
- }
skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
- if (WARN_ON(!skb)) {
- schedule_work(&adapter->work);
+ if (!skb)
return -EFAULT;
- }
/* if we are going to reuse the buffer then keep the pointers around
* but mark index as available. replenish will see the skb pointer and
@@ -1335,11 +1348,8 @@ static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *ada
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
- if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
- schedule_work(&adapter->work);
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
return NULL;
- }
return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
}
@@ -1365,14 +1375,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
cor = rxq->queue_addr[rxq->index].correlator;
rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
- if (unlikely(rc))
+ if (unlikely(rc)) {
+ if (rc == -EINVAL || rc == -EFAULT)
+ goto advance;
return rc;
-
- if (++rxq->index == rxq->num_slots) {
- rxq->index = 0;
- rxq->toggle = !rxq->toggle;
}
+advance:
+ ibmveth_rxq_advance(rxq);
+
return 0;
}
@@ -2931,11 +2942,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
return 0;
+ if (!netif_running(netdev) || napi_disable_pending(napi)) {
+ napi_complete_done(napi, 0);
+ return 0;
+ }
+
if (adapter->rx_qstats)
adapter->rx_qstats[queue_index].polls++;
restart_poll:
while (frames_processed < budget) {
+ if (!netif_running(netdev) || napi_disable_pending(napi))
+ break;
+
if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
break;
@@ -2959,8 +2978,23 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
__sum16 iph_check = 0;
skb = ibmveth_rxq_get_buffer(adapter, queue_index);
- if (unlikely(!skb))
- break;
+ if (unlikely(!skb)) {
+ if (net_ratelimit())
+ netdev_err(netdev,
+ "bad correlator on queue %d, skipping slot\n",
+ queue_index);
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index]
+ .invalid_buffers++;
+ else
+ adapter->rx_invalid_buffer++;
+ rc = ibmveth_rxq_harvest_buffer(adapter,
+ queue_index,
+ true);
+ if (unlikely(rc))
+ break;
+ continue;
+ }
/* if the large packet bit is set in the rx queue
* descriptor, the mss will be written by PHYP eight
@@ -3034,8 +3068,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
ibmveth_replenish_task(adapter, queue_index);
- if (frames_processed == budget)
+ if (frames_processed == budget) {
+ if (!netif_running(netdev) || napi_disable_pending(napi))
+ napi_complete_done(napi, frames_processed);
goto out;
+ }
if (!napi_complete_done(napi, frames_processed))
goto out;
@@ -3053,6 +3090,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+ netif_running(netdev) &&
+ !napi_disable_pending(napi) &&
napi_schedule(napi)) {
lpar_rc = ibmveth_disable_irq(adapter, queue_index);
WARN_ON(lpar_rc != H_SUCCESS);
--
2.39.3 (Apple Git-146)
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues
2026-07-06 19:35 ` [PATCH net-next v3 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
@ 2026-07-14 12:43 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 12:43 UTC (permalink / raw)
To: Mingming Cao
Cc: netdev, bjking1, haren, ricklind, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
On Mon, Jul 06, 2026 at 12:35:53PM -0700, Mingming Cao wrote:
> Queue 0 and subordinate RX queues use different interrupt control
> interfaces in PHYP:
>
> - queue 0: h_vio_signal() after h_register_logical_lan()
> - queue N: H_VIOCTL against the queue handle/hwirq mapping
>
> The current code is single-queue oriented and cannot safely scale to
> multiple RX queues in poll completion and open/close IRQ setup.
>
> Introduce queue-indexed interrupt helpers:
>
> ibmveth_enable_irq(adapter, queue_index)
> ibmveth_disable_irq(adapter, queue_index)
> ibmveth_setup_rx_interrupts()
> ibmveth_cleanup_rx_interrupts()
>
> These helpers centralize queue0-vs-subordinate dispatch and make IRQ
> lifecycle symmetric across open/close and future resize paths.
>
> request_irq() is wired with &adapter->napi[i] as dev_id per queue, so
> interrupt ownership follows the NAPI instance that services that RX
> queue.
>
> Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
> Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Although it is added later in this patch set, I believe linux/irqdomain.h
needs to be included in this patch so that irq_dispose_mapping is defined.
Also, I think it would be best to add these helpers in the same patch(es)
that they are first used. As this will avoid temporal compiler warnings
about declared but otherwise unused functions. Similarly for patch 13/15.
Overall, please make sure that when each patch of the series is applied
no new warnings or errors are introduced for allmodconfig W=1 builds.
FWIIW, I exercised this using gcc 16.1 from
https://www.kernel.org/pub/tools/crosstool/
...
--
pw-bot: changes-requested
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 07/15] ibmveth: Add RX queue register/deregister helpers for MQ
2026-07-06 19:35 ` [PATCH net-next v3 07/15] ibmveth: Add RX queue register/deregister helpers for MQ Mingming Cao
@ 2026-07-14 12:43 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 12:43 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Add RX queue register/deregister helpers for MQ
This patch introduces new helpers to handle RX queue registration and
deregistration for MQ support. It replaces the single adapter-level
register/free paradigm with per-queue register calls to the hypervisor,
fetching queue handles and IRQs. These new paths are only fully exercised
when multi-queue mode is enabled.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 63b0184c622a4..7fc11a4e1f61d 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -401,6 +401,28 @@ ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
[ ... ]
> +static int __maybe_unused
> +ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
> +{
> + struct net_device *netdev = adapter->netdev;
> + int i, rc;
> +
> + for (i = 0; i < adapter->num_rx_queues; i++) {
> + if (!adapter->queue_irq[i]) {
> + netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
> + rc = -EINVAL;
> + goto err_free_irqs;
> + }
> +
> + rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt,
> + 0, netdev->name, &adapter->napi[i]);
> + if (rc) {
> + netdev_err(netdev,
> + "request_irq() failed for irq 0x%x queue %d: %d\n",
> + adapter->queue_irq[i], i, rc);
> + goto err_free_irqs;
> + }
> + }
> +
> + for (i = 0; i < adapter->num_rx_queues; i++)
> + napi_enable(&adapter->napi[i]);
[Severity: Medium]
Could this sequence race with packet reception during interface bring-up?
By calling request_irq() before napi_enable(), the hardware interrupt is
unmasked while the NAPI state is still NAPI_STATE_DISABLE. If an interrupt
fires in this window, could ibmveth_interrupt() fail to schedule the NAPI
poll, potentially leading to dropped events or an interrupt storm?
[ ... ]
> @@ -1136,6 +1176,293 @@ static void __maybe_unused ibmveth_free_rx_qstats(struct ibmveth_adapter *adapte
[ ... ]
> +static int
> +ibmveth_register_logical_lan_queue(struct ibmveth_adapter *adapter,
> + union ibmveth_buf_desc rxq_desc,
> + int queue_index)
> +{
[ ... ]
> + if (lpar_rc == H_FUNCTION) {
> + if (adapter->multi_queue) {
> + netdev_info(adapter->netdev,
> + "Multi queue mode not supported by firmware, falling back to single queue\n");
> + adapter->multi_queue = 0;
> + } else {
> + netdev_err(adapter->netdev,
> + "Unexpected H_FUNCTION for queue %d registration (MQ mode already disabled)\n",
> + queue_index);
> + }
> + return lpar_rc;
[Severity: Medium]
Does returning lpar_rc here cause a regression for the single queue fallback?
Since this error propagates up to ibmveth_register_rx_queues(), it seems it
will unconditionally jump to err_unregister and fail the interface open
rather than allowing it to continue with a single queue.
> + }
> +
> + if (try_again) {
> + try_again = 0;
> + goto retry;
[Severity: Low]
Does this retry loop provide any benefit? Unlike the primary queue which
calls h_free_logical_lan() before retrying to clear stale state, this
jumps back to retry the exact same hypercall with the exact same arguments,
which seems like it will immediately fail again.
> + }
[ ... ]
> +static int
> +ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
> +{
[ ... ]
> + for (i = 1; i < adapter->num_rx_queues; i++) {
> + rc = ibmveth_register_single_rx_queue(adapter, i, mac_address);
> + if (rc) {
> + if (!adapter->queue_handle[i] || !adapter->queue_irq[i]) {
> + netdev_err(netdev,
> + "Invalid hypervisor return for queue %d: handle=0x%llx irq=%u\n",
> + i, adapter->queue_handle[i],
> + adapter->queue_irq[i]);
> + }
[Severity: Low]
Is this validation check placed correctly? Because it sits inside the
if (rc) error block, it will log "Invalid hypervisor return" whenever
registration fails normally (e.g., due to resource constraints). It appears
the hypervisor is not expected to populate the handle and irq on failure.
> + goto err_unregister;
> + }
> + }
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 08/15] ibmveth: Refactor open/close into MQ-ready resource pipeline
2026-07-06 19:35 ` [PATCH net-next v3 08/15] ibmveth: Refactor open/close into MQ-ready resource pipeline Mingming Cao
@ 2026-07-14 12:47 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 12:47 UTC (permalink / raw)
To: Mingming Cao
Cc: netdev, bjking1, haren, ricklind, kuba, edumazet, pabeni,
linuxppc-dev, maddy, mpe, Dave Marquardt
On Mon, Jul 06, 2026 at 12:35:56PM -0700, Mingming Cao wrote:
...
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
...
> /**
> * ibmveth_register_logical_lan_queue - Register subordinate queue with hypervisor
> * @adapter: ibmveth adapter structure
> @@ -1466,208 +1479,108 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
> static int ibmveth_open(struct net_device *netdev)
> {
> struct ibmveth_adapter *adapter = netdev_priv(netdev);
> - u64 mac_address;
> + u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
> int rxq_entries = 1;
> - unsigned long lpar_rc;
> int rc;
> - union ibmveth_buf_desc rxq_desc;
> int i;
> - struct device *dev;
>
> netdev_dbg(netdev, "open starting\n");
>
> - napi_enable(&adapter->napi[0]);
> -
> - for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
> + for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
> rxq_entries += adapter->rx_buff_pool[0][i].size;
>
> - rc = -ENOMEM;
> - adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
> - if (!adapter->buffer_list_addr[0]) {
> - netdev_err(netdev, "unable to allocate list pages\n");
> + rc = ibmveth_alloc_rx_qstats(adapter);
> + if (rc)
> goto out;
> - }
>
> - adapter->filter_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
> - if (!adapter->filter_list_addr) {
> - netdev_err(netdev, "unable to allocate filter pages\n");
> - goto out_free_buffer_list;
> - }
> -
> - dev = &adapter->vdev->dev;
> + rc = ibmveth_alloc_filter_list(adapter);
> + if (rc)
> + goto out_free_rx_qstats;
>
> - adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
> - rxq_entries;
> - adapter->rx_queue[0].queue_addr =
> - dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
> - &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
> - if (!adapter->rx_queue[0].queue_addr)
> + rc = ibmveth_alloc_rx_queues(adapter, rxq_entries);
> + if (rc)
> goto out_free_filter_list;
>
> - adapter->buffer_list_dma[0] = dma_map_single(dev,
> - adapter->buffer_list_addr[0],
> - 4096, DMA_BIDIRECTIONAL);
> - if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
> - netdev_err(netdev, "unable to map buffer list pages\n");
> + rc = ibmveth_alloc_buffer_pools(adapter);
> + if (rc)
> goto out_free_queue_mem;
> - }
>
> - adapter->filter_list_dma = dma_map_single(dev,
> - adapter->filter_list_addr, 4096, DMA_BIDIRECTIONAL);
> - if (dma_mapping_error(dev, adapter->filter_list_dma)) {
> - netdev_err(netdev, "unable to map filter list pages\n");
> - goto out_unmap_buffer_list;
> - }
> + rc = ibmveth_register_rx_queues(adapter, mac_address);
> + if (rc)
> + goto out_free_buffer_pools;
>
> - for (i = 0; i < netdev->real_num_tx_queues; i++) {
> - if (ibmveth_allocate_tx_ltb(adapter, i))
> - goto out_free_tx_ltb;
> + rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
> + if (rc) {
> + netdev_err(netdev, "failed to set number of rx queues\n");
> + goto out_unregister_queues;
> }
>
> - adapter->rx_queue[0].index = 0;
> - adapter->rx_queue[0].num_slots = rxq_entries;
> - adapter->rx_queue[0].toggle = 1;
> -
> - mac_address = ether_addr_to_u64(netdev->dev_addr);
> -
> - rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
> - adapter->rx_queue[0].queue_len;
> - rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
> -
> - netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr[0]);
> - netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
> - netdev_dbg(netdev, "receive q @ 0x%p\n", adapter->rx_queue[0].queue_addr);
> -
> - h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
> -
> - lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
> -
> - if (lpar_rc != H_SUCCESS) {
> - netdev_err(netdev, "h_register_logical_lan failed with %ld\n",
> - lpar_rc);
> - netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
> - "desc:0x%llx MAC:0x%llx\n",
> - adapter->buffer_list_dma[0],
> - adapter->filter_list_dma,
> - rxq_desc.desc,
> - mac_address);
> - rc = -ENONET;
> - goto out_unmap_filter_list;
> - }
> + rc = ibmveth_setup_rx_interrupts(adapter);
> + if (rc)
> + goto out_unregister_queues;
>
> - for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> - if (!adapter->rx_buff_pool[0][i].active)
> - continue;
> - if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
> - netdev_err(netdev, "unable to alloc pool\n");
> - adapter->rx_buff_pool[0][i].active = 0;
> - rc = -ENOMEM;
> - goto out_free_buffer_pools;
> + if (adapter->num_rx_queues > 1) {
> + for (i = 0; i < adapter->num_rx_queues; i++) {
> + netdev_dbg(netdev, "initial replenish cycle for queue %d\n", i);
> + ibmveth_replenish_task(adapter, i);
ibmveth_replenish_task() only has one parameter
until a later patch in this series.
> }
> + } else {
> + netdev_dbg(netdev, "initial replenish cycle\n");
> + ibmveth_interrupt(adapter->queue_irq[0], &adapter->napi[0]);
> }
>
> - netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
> - rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
> - netdev);
> - if (rc != 0) {
> - netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
> - netdev->irq, rc);
> - do {
> - lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> - } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> -
> - goto out_free_buffer_pools;
> - }
> -
> - rc = -ENOMEM;
> -
> - netdev_dbg(netdev, "initial replenish cycle\n");
> - ibmveth_interrupt(netdev->irq, netdev);
> + rc = ibmveth_alloc_tx_resources(adapter);
> + if (rc)
> + goto out_cleanup_rx_interrupts;
>
> netif_tx_start_all_queues(netdev);
>
> netdev_dbg(netdev, "open complete\n");
> -
> return 0;
>
> +out_cleanup_rx_interrupts:
> + ibmveth_cleanup_rx_interrupts(adapter);
> +out_free_tx_resources:
> + ibmveth_free_tx_resources(adapter);
The out_free_tx_resources label is unused until a later patch of this
series, so it should be added in that patch rather than this one.
And it's not clear to me that ibmveth_free_tx_resources() should
be called when jumping to out_cleanup_rx_interrupts as
in that case ibmveth_alloc_tx_resources() hasn't run successfully.
The AI-generated review on sashiko.dev also highlights the error handling
here:
"Are the unwind labels ordered incorrectly here?
"If ibmveth_setup_rx_interrupts() fails, it jumps to
out_unregister_queues, which is placed after out_free_buffer_pools. Does
this mean we skip freeing the buffer pools and leak memory?
"Also, if ibmveth_alloc_tx_resources() fails, it internally frees
partially allocated LTBs. It then jumps to out_cleanup_rx_interrupts and
falls through to out_free_tx_resources. Because ibmveth_free_tx_ltb()
calls dma_unmap_single() unconditionally without checking or zeroing
tx_ltb_dma, will this cause a double free and an invalid DMA unmap?
"Finally, in the fall-through path from out_cleanup_rx_interrupts,
out_free_buffer_pools is executed before out_unregister_queues (which
calls ibmveth_free_all_queues() to unregister the logical LAN). Does this
free and unmap the RX buffers while the hypervisor's logical LAN is still
active, potentially allowing the hypervisor to DMA incoming packets into
freed memory?
> out_free_buffer_pools:
> - while (--i >= 0) {
> - if (adapter->rx_buff_pool[0][i].active)
> - ibmveth_free_buffer_pool(adapter,
> - &adapter->rx_buff_pool[0][i]);
> - }
> -out_unmap_filter_list:
> - dma_unmap_single(dev, adapter->filter_list_dma, 4096,
> - DMA_BIDIRECTIONAL);
> -
> -out_free_tx_ltb:
> - while (--i >= 0) {
> - ibmveth_free_tx_ltb(adapter, i);
> - }
> -
> -out_unmap_buffer_list:
> - dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
> - DMA_BIDIRECTIONAL);
> + ibmveth_free_buffer_pools(adapter);
> +out_unregister_queues:
> + ibmveth_dispose_subordinate_irq_mappings(adapter);
> + ibmveth_free_all_queues(adapter);
> out_free_queue_mem:
> - dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
> - adapter->rx_queue[0].queue_addr,
> - adapter->rx_queue[0].queue_dma);
> + ibmveth_cleanup_rx_resources(adapter);
> out_free_filter_list:
> - free_page((unsigned long)adapter->filter_list_addr);
> -out_free_buffer_list:
> - free_page((unsigned long)adapter->buffer_list_addr[0]);
> + ibmveth_free_filter_list(adapter);
> +out_free_rx_qstats:
> + ibmveth_free_rx_qstats(adapter);
> out:
> - napi_disable(&adapter->napi[0]);
> return rc;
> }
...
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 09/15] ibmveth: Add queue-aware RX buffer submit helper for MQ
2026-07-06 19:35 ` [PATCH net-next v3 09/15] ibmveth: Add queue-aware RX buffer submit helper for MQ Mingming Cao
@ 2026-07-14 12:50 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 12:50 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Add queue-aware RX buffer submit helper for MQ
This patch updates the ibmveth driver's buffer replenishment path to
be queue-aware for multi-queue receive support. It adds a helper to select
the correct hypervisor call and handles fallback to single-buffer mode if
batch operations are unsupported.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index fa2d4777ffc7b..b3b3886c3eedb 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -597,11 +597,73 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length)
[ ... ]
> +static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
> + union ibmveth_buf_desc *descs,
> + int filled,
> + unsigned long buff_size,
> + int queue_index)
> +{
> + struct vio_dev *vdev = adapter->vdev;
> + unsigned long rc;
> +
> + if (adapter->multi_queue) {
> + unsigned long buffersznum = (buff_size << 32) | filled;
> + unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL / 2] = {0};
> + int i;
> +
> + /* Pack descriptor addresses into ioba pairs.
> + * Each ioba holds two 32-bit addresses packed into 64 bits:
> + * - Even descriptors (0,2,4...) go in high 32 bits
> + * - Odd descriptors (1,3,5...) go in low 32 bits
> + */
> + for (i = 0; i < filled && i < IBMVETH_MAX_RX_PER_HCALL; i++) {
> + int pair_idx = i / 2; /* Which pair: 0-5 */
> + int is_high = (i % 2 == 0); /* High or low 32 bits */
> +
> + if (is_high)
> + ioba[pair_idx] = (unsigned long)descs[i].fields.address << 32;
> + else
> + ioba[pair_idx] |= descs[i].fields.address;
> + }
> +
> + rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
> + adapter->queue_handle[queue_index],
> + buffersznum,
> + ioba[0], ioba[1], ioba[2],
> + ioba[3], ioba[4], ioba[5]);
> + adapter->hcall_stats.add_bufs_queue++;
[Severity: Medium]
Will this counter increment lose updates when multiple NAPI poll routines run
concurrently on different CPUs?
Since multi-queue enables concurrent execution, it seems global statistic
counters like hcall_stats.add_bufs_queue and replenish_add_buff_success might
need atomic or per-CPU operations to avoid read-modify-write data races.
> + } else if (filled == 1) {
> + rc = h_add_logical_lan_buffer(vdev->unit_address,
> + descs[0].desc);
> + adapter->hcall_stats.add_buf++;
> + } else {
> + rc = h_add_logical_lan_buffers(vdev->unit_address,
> + descs[0].desc, descs[1].desc,
> + descs[2].desc, descs[3].desc,
> + descs[4].desc, descs[5].desc,
> + descs[6].desc, descs[7].desc);
> + adapter->hcall_stats.add_bufs++;
> + }
> +
> + return rc;
> +}
[ ... ]
> @@ -745,24 +798,19 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
> }
> adapter->replenish_add_buff_failure += filled;
>
> - /*
> - * If multi rx buffers hcall is no longer supported by FW
> - * e.g. in the case of Live Partition Migration
> - */
> - if (batch > 1 && lpar_rc == H_FUNCTION) {
> - /*
[ ... ]
> + if (lpar_rc == H_FUNCTION) {
> + if (adapter->multi_queue) {
> + netdev_err(adapter->netdev,
> + "Unexpected H_FUNCTION from multi-queue buffer add (queue=%d, batch=%d)\n",
> + queue_index, batch);
> + break;
> + } else if (batch > 1) {
> + netdev_warn(adapter->netdev,
> + "H_FUNCTION from legacy batch buffer add (batch=%d), falling back to single buffer mode\n",
> + batch);
> + adapter->rx_buffers_per_hcall = 1;
> + continue;
[Severity: High]
Does this continue statement cause an infinite loop?
Looking at ibmveth_replenish_buffer_pool(), the local batch variable is set
before the while loop:
batch = adapter->rx_buffers_per_hcall;
while (remaining > 0) {
...
If the fallback updates adapter->rx_buffers_per_hcall to 1 and issues a continue,
the loop will restart but the local batch variable will retain its original
value.
Would this lead to the loop repeatedly attempting the unsupported batch size
and failing with H_FUNCTION forever?
> + }
> }
> break;
> }
> @@ -784,18 +832,24 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
> }
>
> /* replenish routine */
> -static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
> +static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
> + int queue_index)
> {
> int i;
>
> + if (queue_index >= adapter->num_rx_queues)
> + return;
> +
> adapter->replenish_task_cycles++;
[Severity: Medium]
Similar to the hcall_stats counters above, could this increment suffer from data
races when multiple queues are processed concurrently across different CPUs?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 10/15] ibmveth: Enable multi-queue RX receive path
2026-07-06 19:35 ` [PATCH net-next v3 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
@ 2026-07-14 12:55 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 12:55 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Enable multi-queue RX receive path
This is the first patch that sets multi_queue from H_ILLAN_ATTRIBUTES
and switches registration, buffer posting, and receive to the MQ
hcall path. It also raises num_rx_queues and enables per-queue NAPI.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index b3b3886c3eedb..863e5c68b42c7 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -826,15 +841,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
> */
> static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
> {
> - __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
> + int i;
> +
> + for (i = 0; i < adapter->num_rx_queues; i++) {
> + __be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
> + u64 drops = be64_to_cpup(p);
>
> - adapter->rx_no_buffer = be64_to_cpup(p);
> + if (i == 0)
> + adapter->rx_no_buffer = drops;
> + }
> }
[Severity: Medium]
Will this logic silently discard the hardware drop counts for all queues
other than queue 0?
Since ethtool -S reports the global adapter->rx_no_buffer statistic, shouldn't
this loop accumulate the drops across all queues instead of only recording
them when evaluating the first queue?
[ ... ]
> @@ -842,6 +865,8 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
> int queue_index)
> {
> + struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
> + unsigned long flags;
> int i;
>
> if (queue_index >= adapter->num_rx_queues)
> @@ -842,6 +865,8 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
>
> adapter->replenish_task_cycles++;
>
> + spin_lock_irqsave(&rxq->replenish_lock, flags);
> +
[Severity: Medium]
With multi-queue RX enabled, can multiple NAPI threads execute this task
concurrently for different queues?
If so, does this non-atomic increment of the global replenish_task_cycles
stat risk data corruption, torn reads on 32-bit platforms, or lost updates?
Similarly, in ibmveth_replenish_buffer_pool():
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_replenish_buffer_pool() {
...
buffers_added += filled;
adapter->replenish_add_buff_success += filled;
remaining -= filled;
...
}
Even though the caller acquires the per-queue rxq->replenish_lock,
replenish_add_buff_success is a global adapter statistic. Does this update
also race with other CPUs holding the locks for their respective queues?
[ ... ]
> @@ -2268,34 +2299,40 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
>
> static int ibmveth_poll(struct napi_struct *napi, int budget)
> {
[ ... ]
> } else {
> struct sk_buff *skb, *new_skb;
> - int length = ibmveth_rxq_frame_length(adapter);
> - int offset = ibmveth_rxq_frame_offset(adapter);
> - int csum_good = ibmveth_rxq_csum_good(adapter);
> - int lrg_pkt = ibmveth_rxq_large_packet(adapter);
> + int length = ibmveth_rxq_frame_length(adapter, queue_index);
> + int offset = ibmveth_rxq_frame_offset(adapter, queue_index);
> + int csum_good = ibmveth_rxq_csum_good(adapter, queue_index);
> + int lrg_pkt = ibmveth_rxq_large_packet(adapter, queue_index);
[Severity: High]
This isn't a bug introduced by this patch, but does this code validate the
length and offset provided by the hardware descriptor before modifying the
SKB bounds?
Later in ibmveth_poll(), these values are used directly:
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_poll() {
...
skb_reserve(skb, offset);
skb_put(skb, length);
...
}
If the hypervisor provides an untrusted or artificially large length, could
the call to skb_put() trigger an skb_over_panic() and crash the guest
kernel?
[ ... ]
Also, I noticed a potential memory leak in the error unwinding path of
ibmveth_open():
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_open() {
...
if (rc) {
netdev_err(netdev, "failed to set number of rx queues\n");
goto out_unregister_queues;
}
...
out_free_buffer_pools:
ibmveth_free_buffer_pools(adapter);
out_unregister_queues:
ibmveth_dispose_subordinate_irq_mappings(adapter);
ibmveth_free_all_queues(adapter);
}
[Severity: Medium]
This is a pre-existing issue, but if netif_set_real_num_rx_queues() fails,
the code jumps to the out_unregister_queues label.
Does this bypass out_free_buffer_pools, leaving the RX buffer pools unfreed
and causing a memory leak when cleaning up after a failure?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 11/15] ibmveth: Add per-queue RX and TX statistics collection and reporting
2026-07-06 19:35 ` [PATCH net-next v3 11/15] ibmveth: Add per-queue RX and TX statistics collection and reporting Mingming Cao
@ 2026-07-14 12:59 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 12:59 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Add per-queue RX and TX statistics collection and reporting
This commit introduces per-queue statistics tracking for both receive and
transmit paths in the ibmveth driver. The counters are aggregated for legacy
interfaces and exposed individually via ethtool stringsets.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 863e5c68b42c7..4e3f49b6346fd 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -1655,6 +1655,10 @@ static int ibmveth_open(struct net_device *netdev)
> if (rc)
> goto out_cleanup_rx_interrupts;
>
> + rc = ibmveth_alloc_tx_qstats(adapter);
> + if (rc)
> + goto out_free_tx_resources;
> +
> netif_tx_start_all_queues(netdev);
>
> netdev_dbg(netdev, "open complete\n");
[ ... ]
> out_cleanup_rx_interrupts:
> ibmveth_cleanup_rx_interrupts(adapter);
> out_free_tx_resources:
> ibmveth_free_tx_resources(adapter);
[Severity: High]
Does the goto out_free_tx_resources on ibmveth_alloc_tx_qstats() failure
bypass the cleanup for RX interrupts?
Since out_free_tx_resources is physically below out_cleanup_rx_interrupts in
ibmveth_open(), failing here leaves the IRQs requested and NAPI enabled. If
an interrupt subsequently fires, could the handler access the freed RX queue
memory and cause a kernel panic?
[ ... ]
> @@ -1699,6 +1699,7 @@ static int ibmveth_close(struct net_device *netdev)
> }
> }
>
> + ibmveth_free_tx_qstats(adapter);
> ibmveth_free_tx_resources(adapter);
> ibmveth_cleanup_rx_interrupts(adapter);
> ibmveth_update_rx_no_buffer(adapter);
[ ... ]
> ibmveth_free_rx_qstats(adapter);
[Severity: High]
If ibmveth_close() is called to bring down the interface, does this
unconditionally free adapter->tx_qstats and adapter->rx_qstats without a
grace period?
Since ibmveth_get_stats64() can be called concurrently under RCU, and
ibmveth_start_xmit() can run concurrently if the datapath is not completely
synchronized, could this lead to a use-after-free or NULL pointer dereference
when these functions access the freed arrays?
[ ... ]
> @@ -1967,22 +1967,107 @@ static int ibmveth_set_features(struct net_device *dev,
[ ... ]
> +static void ibmveth_aggregate_rx_qstats(struct ibmveth_adapter *adapter)
> +{
> + u64 total_invalid = 0;
> + u64 total_large = 0;
> + int i;
> +
> + if (!adapter->rx_qstats)
> + return;
> +
> + for (i = 0; i < adapter->num_rx_queues; i++) {
> + total_invalid += adapter->rx_qstats[i].invalid_buffers;
> + total_large += adapter->rx_qstats[i].large_packets;
> + }
> +
> + adapter->rx_invalid_buffer = total_invalid;
> + adapter->rx_large_packets = total_large;
> +}
[Severity: High]
Does overwriting adapter->rx_invalid_buffer and other persistent global
counters with the dynamically allocated adapter->rx_qstats values cause a
permanent loss of interface statistics?
When the interface is brought down and back up, ibmveth_close() frees the
queue stats arrays and ibmveth_open() reallocates them as zeroed. Will this
cause all historical statistics to be wiped out on every interface reset,
breaking standard persistence behavior?
[ ... ]
> +static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
> +{
[ ... ]
> + for (i = 0; i < adapter->num_rx_queues; i++) {
> + ethtool_sprintf(&p, "rx%d_packets", i);
> + ethtool_sprintf(&p, "rx%d_bytes", i);
[Severity: Low]
Are these private ethtool stringsets duplicating standard kernel statistics?
According to the subsystem guidelines, per-queue packet and byte statistics
that have a standard UAPI representation (like struct netdev_queue_stats_rx
exported via netlink) should not be duplicated as private ethtool strings.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index f7b20fd01acb5..390c660af979e 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -316,9 +316,21 @@ struct ibmveth_rx_queue_stats {
> u64 no_buffer_drops;
> };
>
> +struct ibmveth_tx_queue_stats {
> + u64 packets;
> + u64 bytes;
> + u64 large_packets;
> + u64 dropped_packets;
> + u64 send_failures;
> + u64 checksum_offload;
> +};
[Severity: Medium]
Will this structure layout prevent cache line false sharing?
The commit message mentions tracking transmit counters per TX queue to avoid
cache line contention, but struct ibmveth_tx_queue_stats appears to be
exactly 48 bytes and lacks cacheline alignment annotations like
____cacheline_aligned. When allocated as a contiguous array, will multiple
CPUs processing different queues end up writing to the same hardware
cacheline?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs
2026-07-06 19:36 ` [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs Mingming Cao
@ 2026-07-14 13:01 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 13:01 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Expose per-queue buffer pool details via sysfs
This commit adds a read-only sysfs attribute buffer_pools to the VIO device
to expose per-queue buffer pool details dynamically. It outputs runtime per-queue
buffer pressure metrics like size, active, and available buffers.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 4e3f49b6346fd..ecc472ee8f71f 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -2896,6 +2896,52 @@ static const struct net_device_ops ibmveth_netdev_ops = {
[ ... ]
> +static ssize_t buffer_pools_show(struct device *dev,
> + struct device_attribute *attr,
> + char *buf)
> +{
> + struct net_device *netdev = dev_get_drvdata(dev);
> + struct ibmveth_adapter *adapter = netdev_priv(netdev);
> + int len = 0;
> + int i, j;
> +
> + len += scnprintf(buf + len, PAGE_SIZE - len,
> + "Queue Pool Size BuffSize Active Available\n");
> + len += scnprintf(buf + len, PAGE_SIZE - len,
> + "----- ---- ---- -------- ------ ---------\n");
[Severity: Medium]
Does this buffer_pools_show() implementation violate the sysfs "one value
per file" design rule?
The attribute appears to dump a complex, multi-line ASCII table with headers
and columns rather than exposing a single value per sysfs file.
[ ... ]
> @@ -3056,6 +3102,14 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>
> netdev_dbg(netdev, "registered\n");
>
> + rc = sysfs_create_group(&dev->dev.kobj, &ibmveth_attr_group);
> + if (rc) {
> + netdev_err(netdev, "failed to create sysfs attributes rc=%d\n", rc);
> + unregister_netdev(netdev);
> + free_netdev(netdev);
> + return rc;
> + }
[Severity: High]
If userspace brings up the interface immediately after register_netdev()
succeeds, could adapter->work be scheduled before sysfs_create_group() fails?
If this race occurs, unregister_netdev() does not synchronously cancel the
work.
Could the global workqueue execute the pending adapter->work item on freed
memory after free_netdev() is called in this ibmveth_probe() error path?
[Severity: High]
This is a pre-existing issue in older error paths of ibmveth_probe(), but it
is duplicated here: does this error path leak the adapter->rx_buff_pool
kobjects?
Since kobject_put() is not called on the pool kobjects before free_netdev()
is executed, the kobjects remain attached to the VIO device with dangling
pointers to the freed adapter memory.
Could subsequent sysfs accesses to those pool kobjects trigger a
use-after-free?
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize
2026-07-06 19:36 ` [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
@ 2026-07-14 13:03 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 13:03 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
Date: Mon Jul 6 19:36:01 2026 +0000
ibmveth: Implement incremental MQ RX queue resize
Add helpers to register/deregister individual RX queues and scale
queue count up or down while the netdev stays up. Surviving queues
keep PHYP handles, buffer pools, and IRQ state across resize.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index ecc472ee8f71f..ac4d89a66a8db 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -1080,6 +1132,138 @@ static void ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
> +static int
> +ibmveth_alloc_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx,
> + int rxq_entries)
> +{
[ ... ]
> + for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> + adapter->rx_buff_pool[queue_idx][i].size =
> + adapter->rx_buff_pool[0][i].size;
> + adapter->rx_buff_pool[queue_idx][i].buff_size =
> + adapter->rx_buff_pool[0][i].buff_size;
> + adapter->rx_buff_pool[queue_idx][i].threshold =
> + adapter->rx_buff_pool[0][i].threshold;
> + adapter->rx_buff_pool[queue_idx][i].active =
> + adapter->rx_buff_pool[0][i].active;
> + }
[Severity: Critical]
Does this loop miss initializing pool->index?
If pool->index remains 0 for all buffer pools on the new queue, when
replenishing, 0 is shifted into the correlator.
Then when receiving, ibmveth_remove_buffer_from_pool() will extract pool=0,
which fetches from the wrong pool and could corrupt the free map of pool 0,
leading to a use-after-free or leak.
[ ... ]
> @@ -1192,6 +1376,49 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
> +static int
> +ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
> +{
[ ... ]
> + while (drained < limit &&
> + ibmveth_rxq_pending_buffer(adapter, queue_index)) {
> + rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
Simon says: I think we can consider the issue below in the context of
possible follow-up, because I believe the use of this pattern
predates this patch.
[Severity: High]
Is a read memory barrier needed between reading the toggle bit in
ibmveth_rxq_pending_buffer() and reading the correlator in
ibmveth_rxq_harvest_buffer()?
Since the hypervisor could be actively writing new descriptors while this is
called, could a weakly-ordered CPU reorder the correlator load before the
toggle bit load, causing us to recycle the wrong buffer?
[ ... ]
> @@ -1491,6 +1718,175 @@ ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
> +static int
> +ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
> + int new_count, int rxq_entries)
> +{
[ ... ]
> + rc = ibmveth_enable_irq(adapter, i);
> + if (rc) {
> + netdev_err(netdev,
> + "Failed to enable IRQ for queue %d: %d\n",
> + i, rc);
> + ibmveth_cleanup_single_rx_interrupt(adapter, i);
> + ibmveth_deregister_single_rx_queue(adapter, i);
> + ibmveth_free_single_rx_queue(adapter, i);
> + goto cleanup_new_queues;
> + }
> +
> + napi_enable(&adapter->napi[i]);
> + }
> +
> + adapter->num_rx_queues = new_count;
[Severity: High]
What happens if an interrupt fires on the newly added queue before
adapter->num_rx_queues is updated?
If an interrupt fires between ibmveth_enable_irq() and the update of
adapter->num_rx_queues, ibmveth_interrupt() might evaluate:
WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues)
to true and return IRQ_NONE without masking the interrupt.
Could this lead to an unhandled interrupt storm?
[ ... ]
> + } else {
> + netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
> + new_count, old_count - 1);
> +
> + for (i = new_count; i < old_count; i++)
> + napi_disable(&adapter->napi[i]);
> +
> + for (i = new_count; i < old_count; i++)
> + ibmveth_drain_rx_queue(adapter, i);
> +
> + synchronize_net();
[Severity: High]
Can disabling NAPI before disabling the hardware interrupts create an
interrupt storm window?
If the hypervisor asserts an interrupt during synchronize_net(), the hardware
IRQ handler runs. Since NAPI is disabled, it returns IRQ_HANDLED without
masking the interrupt source at the hypervisor, leading immediately to an
IRQ storm and CPU lockup.
> +
> + rc = netif_set_real_num_rx_queues(netdev, new_count);
> + if (rc) {
> + netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
> + new_count, rc);
> + for (i = new_count; i < old_count; i++)
> + napi_enable(&adapter->napi[i]);
> + return rc;
> + }
[Severity: High]
Is it possible for this error path to permanently stall the RX queues?
It calls napi_enable() but doesn't call ibmveth_replenish_task(). Since the
queues were completely drained of their buffers by ibmveth_drain_rx_queue(),
will the hypervisor have any buffers to place incoming packets?
Without buffers, no interrupts will fire for these queues, and NAPI will
never run again.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to MQ RX queue resize
2026-07-06 19:36 ` [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
@ 2026-07-14 13:07 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 13:07 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Wire ethtool set_channels to MQ RX queue resize
Expose incremental RX resize through ethtool channel control.
get_channels() reports rx_count from adapter->num_rx_queues and max_rx
as IBMVETH_MAX_RX_QUEUES when MQ firmware is enabled, else 1.
set_channels() validates rx_count is within 1..IBMVETH_MAX_RX_QUEUES.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index ac4d89a66a8db..50a332ab83fd3 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -2534,19 +2534,69 @@ static int ibmveth_set_channels(struct net_device *netdev,
> struct ethtool_channels *channels)
> {
> struct ibmveth_adapter *adapter = netdev_priv(netdev);
[ ... ]
> - if (!(netdev->flags & IFF_UP))
> + if (!(netdev->flags & IFF_UP)) {
> + if (goal_rx > 1 && !adapter->multi_queue) {
[ ... ]
> + /* Stash desired RX count; open() publishes it via
> + * netif_set_real_num_rx_queues() after queue registration.
> + */
> + if (goal_rx != adapter->num_rx_queues)
> + adapter->num_rx_queues = goal_rx;
> +
> return netif_set_real_num_tx_queues(netdev, goal);
[Severity: Medium]
If netif_set_real_num_tx_queues() fails and returns an error, the TX
configuration command will fail, but the RX queue count has already been
permanently modified in adapter->num_rx_queues. Will this leave the device in
an inconsistent state where the new RX count is applied on the next open
despite the command failing?
> + }
[ ... ]
> + if (goal_rx != old_rx) {
> + rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
> + rxq_entries);
[Severity: High]
Does this new ethtool scale-down path expose an interrupt storm?
Looking at ibmveth_resize_rx_queues_incremental(), when scaling down queues it
disables NAPI before disabling the hardware interrupt:
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_resize_rx_queues_incremental() {
...
for (i = new_count; i < old_count; i++)
napi_disable(&adapter->napi[i]);
for (i = new_count; i < old_count; i++)
ibmveth_drain_rx_queue(adapter, i);
synchronize_net();
...
/* Disable hypervisor interrupts and wait for handlers to complete
* before updating num_rx_queues.
*/
for (i = new_count; i < old_count; i++) {
ibmveth_disable_irq(adapter, i);
...
}
If the hypervisor delivers a packet during this window, ibmveth_interrupt()
will execute. Since NAPI is disabled, napi_schedule_prep() fails, causing the
handler to return IRQ_HANDLED without disabling the hardware interrupt. Will
this leave the virtual interrupt asserted and force an infinite interrupt loop?
> + if (rc) {
> + netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
> + return rc;
> + }
> + }
>
> /* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
> * but we may need to alloc/free the ltb's.
> */
[Severity: High]
This is a pre-existing issue, but does this function use an uninitialized
variable in the error path?
When decreasing the number of TX queues (goal < old), the allocation loop for
(i = old; i < goal; i++) is skipped, leaving 'i' uninitialized. If
netif_set_real_num_tx_queues() then fails, the error path assigns old = i;.
The cleanup loop then uses this garbage value:
for (i = old; i > goal; i--) {
if (adapter->tx_ltb_ptr[i - 1])
Will this access adapter->tx_ltb_ptr out of bounds and pass arbitrary pointers
to ibmveth_free_tx_ltb()?
> @@ -2580,7 +2630,7 @@ static int ibmveth_set_channels(struct net_device *netdev,
>
> netif_tx_wake_all_queues(netdev);
>
> - return rc;
> + return 0;
[Severity: High]
Does this unconditional return of 0 mask TX queue configuration failures?
If applying the new TX queue count via netif_set_real_num_tx_queues() fails,
the code handles it locally, but the function exits with 0 regardless. Will
this mislead userspace into believing the TX channel count was successfully
modified?
> }
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
2026-07-06 19:36 ` [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after " Mingming Cao
@ 2026-07-14 13:10 ` Simon Horman
0 siblings, 0 replies; 26+ messages in thread
From: Simon Horman @ 2026-07-14 13:10 UTC (permalink / raw)
To: mmc
Cc: Simon Horman, netdev, bjking1, haren, ricklind, kuba, edumazet,
pabeni, linuxppc-dev, maddy, mpe, davemarq
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev
---
ibmveth: Fix MQ RX poll and shutdown hangs after queue resize
This commit fixes polling hangs in the ibmveth driver after queue resize
by skipping over bad correlators instead of retrying endlessly. It also
ensures NAPI completes when the interface is down or being disabled.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 50a332ab83fd3..eb84b4e5f69fd 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -2959,8 +2978,23 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
> __sum16 iph_check = 0;
>
> skb = ibmveth_rxq_get_buffer(adapter, queue_index);
> - if (unlikely(!skb))
> - break;
> + if (unlikely(!skb)) {
> + if (net_ratelimit())
> + netdev_err(netdev,
[Severity: High]
This isn't a bug introduced by this patch, but can the unvalidated packet
lengths and offsets from hardware descriptors trigger an skb_over_panic()
here?
Further down in ibmveth_poll(), the driver reads length and offset directly
from the hardware receive descriptor and passes them to skb_reserve() and
skb_put() on the reused SKB:
int length = ibmveth_rxq_frame_length(adapter, queue_index);
int offset = ibmveth_rxq_frame_offset(adapter, queue_index);
...
} else {
rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, false);
if (unlikely(rc))
break;
skb_reserve(skb, offset);
}
skb_put(skb, length);
If a faulty hypervisor provides an abnormally large length or offset,
skb_put() will extend the tail past the end of the buffer, which triggers an
skb_over_panic() and causes a crash.
Should the driver validate that offset + length is within the known buffer
size before modifying the boundaries?
[ ... ]
> @@ -3034,8 +3068,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>
> ibmveth_replenish_task(adapter, queue_index);
>
> - if (frames_processed == budget)
> + if (frames_processed == budget) {
> + if (!netif_running(netdev) || napi_disable_pending(napi))
> + napi_complete_done(napi, frames_processed);
> goto out;
> + }
[Severity: High]
Does calling napi_complete_done() here conflict with the NAPI core contract?
If the polling loop exhausts its budget (frames_processed == budget), returning
the full budget after calling napi_complete_done() might cause issues with the
NAPI state machine. The NAPI core (__napi_poll) will operate on the instance
assuming it still holds the NAPIF_STATE_SCHED bit.
If napi_disable_pending() is true, the core calls napi_complete(), which will
hit a WARN_ON_ONCE.
If napi_disable_pending() is false but !netif_running() is true, the core will
add the NAPI instance back to the poll_list. Since the driver cleared
NAPIF_STATE_SCHED, a subsequent interrupt can trigger another napi_schedule(),
causing a double list_add_tail() and potentially leading to list corruption.
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2026-07-14 13:15 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-06 19:35 [PATCH net-next v3 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 02/15] ibmveth: Prepare MQ RX adapter and statistics structures Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
2026-07-14 12:43 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
2026-07-06 19:35 ` [PATCH net-next v3 07/15] ibmveth: Add RX queue register/deregister helpers for MQ Mingming Cao
2026-07-14 12:43 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 08/15] ibmveth: Refactor open/close into MQ-ready resource pipeline Mingming Cao
2026-07-14 12:47 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 09/15] ibmveth: Add queue-aware RX buffer submit helper for MQ Mingming Cao
2026-07-14 12:50 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
2026-07-14 12:55 ` Simon Horman
2026-07-06 19:35 ` [PATCH net-next v3 11/15] ibmveth: Add per-queue RX and TX statistics collection and reporting Mingming Cao
2026-07-14 12:59 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 12/15] ibmveth: Expose per-queue buffer pool details via sysfs Mingming Cao
2026-07-14 13:01 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 13/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
2026-07-14 13:03 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 14/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
2026-07-14 13:07 ` Simon Horman
2026-07-06 19:36 ` [PATCH net-next v3 15/15] ibmveth: Fix MQ RX poll and shutdown hangs after " Mingming Cao
2026-07-14 13:10 ` Simon Horman
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.