[PATCH net-next v4 00/14] ibmveth: Add multi-queue RX support

[PATCH net-next v4 00/14] ibmveth: Add multi-queue RX support

[Mingming Cao]

Hi,

Power11 PHYP adds Virtual Ethernet multi-queue (MQ) RX: multiple
logical-LAN RX queues, per-queue buffer posting, and completion
delivery. Guest Linux did not use that; ibmveth still registered one
RX queue even when PHYP was MQ-capable.

This series adds the ibmveth MQ client for net-next. When PHYP
advertises IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT via H_ILLAN_ATTRIBUTES,
probe enables MQ with a default RX count of min(num_online_cpus(), 8)
(same cap as TX today); ethtool -L can raise RX up to 16. Packets are
received on per-queue NAPI. Older firmware without the bit is unchanged.
Queue selection remains firmware-defined (PHYP hash). Ethtool RSS hash
get/set for that algorithm is deferred to a follow-up series so this
one stays MQ datapath only.

User-visible bits: ethtool -l/-L (channels), ethtool -S and
ndo_get_stats64 (per-queue + aggregate), and a read-only debugfs
buffer_pools dump (v3's multi-line sysfs dump moved to debugfs; the
historical queue-0 poolN/ sysfs ABI is unchanged).

Background:

ibmveth today uses one logical LAN, one set of buffer pools, and one
NAPI context. PHYP MQ mode gives each RX queue its own handle (post via
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, subordinate register via
H_REG_LOGICAL_LAN_QUEUE). Legacy firmware keeps the original hcall path.

Series layout (14 patches):

  1-2   Hypercall wrappers; MQ adapter layout (MAX_RX_QUEUES stays 1)
  3-8   Refactor open/close into queue-aware helpers (still SQ runtime)
  9     Enable MQ datapath at probe/open (subordinate register helpers
        land here with first use; probe pool-kobj fail paths put
        already-created pool kobjects)
  10-11 Per-queue RX/TX stats; debugfs buffer_pools
  12-13 Incremental RX resize; ethtool set_channels
  14    Poll/close fixes after aggressive ethtool -L

Patch 2 is layout-only. Statistics types land with first use
(hcall_stats in 7, rx/tx qstats in 10). Patches 3-8 do not change
runtime behaviour until patch 9. v3's separate big-bang open/close
pipeline patch is gone; wiring is incremental across 3-8.

Testing:

ppc64le PowerVM LPAR, MQ-capable firmware:
* ethtool -L cycling (16/1/8/11/1/3/16/8/1) with ping
* ethtool -L under iperf3; link down/up during traffic
* ifdown/ifup under iperf3 RX+TX (SQ and MQ)
* Legacy firmware (no MQ bit): open/close/stress on helper path
* allmodconfig W=1 clean at each patch (bisect-friendly restack)

Changes in v4:

Addresses Simon's v3 review and related fixes:
* First-use helpers/includes (irqdomain.h with first dispose); no
  unused statics; dropped orphan open/close pipeline patch
* Open/close unwind (free LAN before RX pools); no double TX teardown
* MQ open: replenish all queues before PHYP unmask; H_FUNCTION on
  subordinate register is a hard open failure
* Resize/set_channels hardenings; stats probe-lifetime + sum-on-read;
  buffer_pools diagnostic on debugfs
* Patch 9: put already-created pool kobjects on probe failure paths
* Patch 14: correlator skip, skb tailroom check, napi_complete_done
  shutdown return < budget

Comments welcome.

---
Previous versions
v3: https://lore.kernel.org/r/20260706193603.8039-1-mmc@linux.ibm.com
v2: https://lore.kernel.org/r/20260701222327.61325-1-mmc@linux.ibm.com
v1: https://lore.kernel.org/r/cover.1782758799.git.mmc@linux.ibm.com
v3 review (Simon, from 05/15):
  https://lore.kernel.org/r/20260714124327.GJ1364329@horms.kernel.org

[PATCH net-next v4 01/14] ibmveth: Add MQ RX hypercall wrappers and call definitions

[Mingming Cao]

Single-queue ibmveth only needs h_register_logical_lan() plus legacy
buffer add/free calls. MQ RX uses per-queue handles, so the driver must
also be able to register/deregister subordinate queues and post
buffers against a specific queue handle.

Add the PHYP call IDs for:

  H_REG_LOGICAL_LAN_QUEUE
  H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
  H_FREE_LOGICAL_LAN_QUEUE

and add ibmveth.h wrapper helpers (h_reg_logical_lan_queue(),
h_add_logical_lan_buffers_queue(), h_free_logical_lan_queue()) with
argument ordering and return semantics matching the existing ibmveth
hcall wrappers. Also add h_register_logical_lan_with_handle() so
queue 0 can capture the PHYP queue handle in MQ mode.

This patch is intentionally plumbing only: no runtime behavior change
yet. Legacy firmware keeps H_REGISTER_LOGICAL_LAN and the existing
buffer hcalls. The new wrappers are used only when a later commit sets
multi_queue from H_ILLAN_ATTRIBUTES.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Document @queue_handle and @irq in h_reg_logical_lan_queue() kdoc.
- Wrap h_register_logical_lan_with_handle() prototype for readability /
  checkpatch.
- Drop unused H_FREE_LOGICAL_LAN_BUFFER_QUEUE wrapper/opcode (no caller;
  buffer return is local harvest + queue free).

 arch/powerpc/include/asm/hvcall.h  |   5 +-
 drivers/net/ethernet/ibm/ibmveth.h | 137 +++++++++++++++++++++++++++++
 2 files changed, 141 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
index dff90a7d7f70..d79baf2e3827 100644
--- a/arch/powerpc/include/asm/hvcall.h
+++ b/arch/powerpc/include/asm/hvcall.h
@@ -362,7 +362,10 @@
 #define H_GUEST_DELETE		0x488
 #define H_PKS_WRAP_OBJECT	0x490
 #define H_PKS_UNWRAP_OBJECT	0x494
-#define MAX_HCALL_OPCODE	H_PKS_UNWRAP_OBJECT
+#define H_REG_LOGICAL_LAN_QUEUE 0x49C
+#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0
+#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8
+#define MAX_HCALL_OPCODE	H_FREE_LOGICAL_LAN_QUEUE
 
 /* Scope args for H_SCM_UNBIND_ALL */
 #define H_UNBIND_SCOPE_ALL (0x1)
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index d87713668ed3..c88dbeb7cd96 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -66,6 +66,143 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address,
 			    desc5, desc6, desc7, desc8);
 }
 
+/**
+ * h_reg_logical_lan_queue - Register a subordinate receive queue
+ * @unit_address: Device unit address
+ * @buffer_list: DMA address of 4KB page for tracking registered buffers
+ * @rec_queue: Buffer descriptor of receive queue
+ * @queue_handle: Output queue handle on success (may be NULL)
+ * @irq: Output hypervisor IRQ number on success (may be NULL)
+ *
+ * Registers a subordinate receive queue with the hypervisor.
+ *
+ * Return:
+ *   H_SUCCESS (0) on success
+ *   H_PARAMETER if parameters are invalid
+ *
+ * On success, hypervisor returns:
+ *   R3: H_SUCCESS
+ *   R4: Queue handle
+ *   R5: IRQ number for this queue
+ */
+static inline long h_reg_logical_lan_queue(unsigned long unit_address,
+					   unsigned long buffer_list,
+					   unsigned long rec_queue,
+					   unsigned long *queue_handle,
+					   unsigned long *irq)
+{
+	unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+	long rc;
+
+	rc = plpar_hcall9(H_REG_LOGICAL_LAN_QUEUE,
+			  retbuf, unit_address,
+			  buffer_list, rec_queue);
+
+	if (rc == H_SUCCESS) {
+		if (queue_handle)
+			*queue_handle = retbuf[0];
+		if (irq)
+			*irq = retbuf[1];
+	}
+
+	return rc;
+}
+
+/**
+ * h_add_logical_lan_buffers_queue - Add buffers to subordinate queue
+ * @unit_address: Device unit address
+ * @queue_handle: Queue handle from h_reg_logical_lan_queue()
+ * @buffersznum: Buffer size (upper 32 bits) | count (lower 32 bits)
+ * @ioba12: Buffer addresses 1 and 2 packed ((addr1 << 32) | addr2)
+ * @ioba34: Buffer addresses 3 and 4 packed
+ * @ioba56: Buffer addresses 5 and 6 packed
+ * @ioba78: Buffer addresses 7 and 8 packed
+ * @ioba910: Buffer addresses 9 and 10 packed
+ * @ioba1112: Buffer addresses 11 and 12 packed
+ *
+ * Return:
+ *   H_SUCCESS - All buffers added successfully
+ *   H_PARAMETER - Invalid parameters
+ *   H_HARDWARE - Hardware error
+ */
+static inline long h_add_logical_lan_buffers_queue(unsigned long unit_address,
+						   unsigned long queue_handle,
+						   unsigned long buffersznum,
+						   unsigned long ioba12,
+						   unsigned long ioba34,
+						   unsigned long ioba56,
+						   unsigned long ioba78,
+						   unsigned long ioba910,
+						   unsigned long ioba1112)
+{
+	unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+
+	return plpar_hcall9(H_ADD_LOGICAL_LAN_BUFFERS_QUEUE,
+			    retbuf, unit_address,
+			    queue_handle, buffersznum,
+			    ioba12, ioba34, ioba56,
+			    ioba78, ioba910, ioba1112);
+}
+
+/**
+ * h_free_logical_lan_queue - Deregister subordinate receive queue
+ * @unit_address: Device unit address
+ * @queue_handle: Queue handle from h_reg_logical_lan_queue()
+ *
+ * Deregisters and frees all structures associated with the subordinate queue.
+ *
+ * Return:
+ *   H_SUCCESS - Queue freed successfully
+ *   H_PARAMETER - Invalid parameters
+ *   H_HARDWARE - Hardware error
+ *   H_STATE - VIOA not in valid state
+ *   H_BUSY / H_LONG_BUSY_* - Resource busy, retry
+ */
+static inline long h_free_logical_lan_queue(unsigned long unit_address,
+					    unsigned long queue_handle)
+{
+	unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+
+	return plpar_hcall9(H_FREE_LOGICAL_LAN_QUEUE,
+			    retbuf, unit_address, queue_handle);
+}
+
+/**
+ * h_register_logical_lan_with_handle - Register primary queue and get handle
+ * @unit_address: Device unit address
+ * @buffer_list: DMA address of buffer list
+ * @rec_queue: Buffer descriptor of receive queue
+ * @filter_list: DMA address of filter list
+ * @mac_address: MAC address
+ * @queue_handle: Output parameter for queue handle
+ *
+ * Registers the primary receive queue (queue 0) with the hypervisor and
+ * returns the queue handle. This is needed in multi-queue mode to use
+ * h_add_logical_lan_buffers_queue() for all queues including queue 0.
+ *
+ * Return: H_SUCCESS (0) on success, error code otherwise
+ */
+static inline long
+h_register_logical_lan_with_handle(unsigned long unit_address,
+				   unsigned long buffer_list,
+				   unsigned long rec_queue,
+				   unsigned long filter_list,
+				   unsigned long mac_address,
+				   u64 *queue_handle)
+{
+	unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+	long rc;
+
+	rc = plpar_hcall9(H_REGISTER_LOGICAL_LAN, retbuf,
+			  unit_address, buffer_list, rec_queue,
+			  filter_list, mac_address);
+
+	if (rc == H_SUCCESS && queue_handle)
+		*queue_handle = retbuf[0];
+
+	return rc;
+}
+
 /* FW allows us to send 6 descriptors but we only use one so mark
  * the other 5 as unused (0)
  */
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 02/14] ibmveth: Prepare MQ RX adapter data structures

[Mingming Cao]

MQ RX needs per-queue state for NAPI, queue handles/IRQs, RX rings,
buffer-list DMA mappings, and buffer pools. The current driver stores
most of this as single instances tied to queue 0.

Convert those fields to queue-indexed layouts sized by
IBMVETH_MAX_RX_QUEUES:

  rx_queue[]
  napi[]
  queue_handle[] / queue_irq[]
  buffer_list_addr[] / buffer_list_dma[]
  rx_buff_pool[queue][pool]

and add multi_queue / num_rx_queues to track MQ capability and how
many RX queues are active. Keep IBMVETH_MAX_RX_QUEUES at 1 for now so
this remains a structural preparation patch; later enablement raises
the limit when multi-queue RX is actually turned on.

This patch keeps behavior unchanged by mechanically switching existing
references to index 0 (for example rx_queue -> rx_queue[0],
rx_buff_pool[pool] -> rx_buff_pool[0][pool], napi -> napi[0]).
open/poll/close still drive a single RX queue only.

Per-queue statistics structs and hcall_stats are introduced later with
their first use (register helpers / statistics collection).

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Keep IBMVETH_MAX_RX_QUEUES at 1 until MQ enablement (same idea as v3,
  but v3 also planted unused stats types here).
- Layout-only: queue-indexed adapter fields only. Do not introduce
  hcall_stats / rx_qstats here (first-use).
- Defer hcall_stats to patch 7 (register helpers) and rx/tx qstats +
  NUM macros to patch 10 (statistics collection).
- Subject: "Prepare MQ RX adapter data structures" (was "...and
  statistics structures" in earlier drafts).

 drivers/net/ethernet/ibm/ibmveth.c | 212 ++++++++++++++++-------------
 drivers/net/ethernet/ibm/ibmveth.h |  18 ++-
 2 files changed, 128 insertions(+), 102 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 73e051d26b9d..8e758362cb26 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -101,7 +101,9 @@ static struct ibmveth_stat ibmveth_stats[] = {
 /* simple methods of getting data from the current rxq entry */
 static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
 {
-	return be32_to_cpu(adapter->rx_queue.queue_addr[adapter->rx_queue.index].flags_off);
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+
+	return be32_to_cpu(rxq->queue_addr[rxq->index].flags_off);
 }
 
 static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
@@ -112,7 +114,7 @@ static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
 
 static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
 {
-	return ibmveth_rxq_toggle(adapter) == adapter->rx_queue.toggle;
+	return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
 }
 
 static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
@@ -132,7 +134,9 @@ static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
 
 static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
 {
-	return be32_to_cpu(adapter->rx_queue.queue_addr[adapter->rx_queue.index].length);
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+
+	return be32_to_cpu(rxq->queue_addr[rxq->index].length);
 }
 
 static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
@@ -386,7 +390,7 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
  */
 static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
 {
-	__be64 *p = adapter->buffer_list_addr + 4096 - 8;
+	__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
 
 	adapter->rx_no_buffer = be64_to_cpup(p);
 }
@@ -399,7 +403,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
 	adapter->replenish_task_cycles++;
 
 	for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
-		struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[i];
+		struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
 
 		if (pool->active &&
 		    (atomic_read(&pool->available) < pool->threshold))
@@ -463,12 +467,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
 	struct sk_buff *skb;
 
 	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
-	    WARN_ON(index >= adapter->rx_buff_pool[pool].size)) {
+	    WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
 		schedule_work(&adapter->work);
 		return -EINVAL;
 	}
 
-	skb = adapter->rx_buff_pool[pool].skbuff[index];
+	skb = adapter->rx_buff_pool[0][pool].skbuff[index];
 	if (WARN_ON(!skb)) {
 		schedule_work(&adapter->work);
 		return -EFAULT;
@@ -482,42 +486,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
 		/* remove the skb pointer to mark free. actual freeing is done
 		 * by upper level networking after gro_receive
 		 */
-		adapter->rx_buff_pool[pool].skbuff[index] = NULL;
+		adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
 
 		dma_unmap_single(&adapter->vdev->dev,
-				 adapter->rx_buff_pool[pool].dma_addr[index],
-				 adapter->rx_buff_pool[pool].buff_size,
+				 adapter->rx_buff_pool[0][pool].dma_addr[index],
+				 adapter->rx_buff_pool[0][pool].buff_size,
 				 DMA_FROM_DEVICE);
 	}
 
-	free_index = adapter->rx_buff_pool[pool].producer_index;
-	adapter->rx_buff_pool[pool].producer_index++;
-	if (adapter->rx_buff_pool[pool].producer_index >=
-	    adapter->rx_buff_pool[pool].size)
-		adapter->rx_buff_pool[pool].producer_index = 0;
-	adapter->rx_buff_pool[pool].free_map[free_index] = index;
+	free_index = adapter->rx_buff_pool[0][pool].producer_index;
+	adapter->rx_buff_pool[0][pool].producer_index++;
+	if (adapter->rx_buff_pool[0][pool].producer_index >=
+	    adapter->rx_buff_pool[0][pool].size)
+		adapter->rx_buff_pool[0][pool].producer_index = 0;
+	adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
 
 	mb();
 
-	atomic_dec(&(adapter->rx_buff_pool[pool].available));
+	atomic_dec(&adapter->rx_buff_pool[0][pool].available);
 
 	return 0;
 }
 
 /* get the current buffer on the rx queue */
-static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
+static inline struct sk_buff *
+ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
 {
-	u64 correlator = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator;
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+	u64 correlator = rxq->queue_addr[rxq->index].correlator;
 	unsigned int pool = correlator >> 32;
 	unsigned int index = correlator & 0xffffffffUL;
 
 	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
-	    WARN_ON(index >= adapter->rx_buff_pool[pool].size)) {
+	    WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
 		schedule_work(&adapter->work);
 		return NULL;
 	}
 
-	return adapter->rx_buff_pool[pool].skbuff[index];
+	return adapter->rx_buff_pool[0][pool].skbuff[index];
 }
 
 /**
@@ -538,14 +544,16 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
 	u64 cor;
 	int rc;
 
-	cor = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator;
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+
+	cor = rxq->queue_addr[rxq->index].correlator;
 	rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
 	if (unlikely(rc))
 		return rc;
 
-	if (++adapter->rx_queue.index == adapter->rx_queue.num_slots) {
-		adapter->rx_queue.index = 0;
-		adapter->rx_queue.toggle = !adapter->rx_queue.toggle;
+	if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
+		adapter->rx_queue[0].index = 0;
+		adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
 	}
 
 	return 0;
@@ -595,7 +603,7 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
 	 */
 retry:
 	rc = h_register_logical_lan(adapter->vdev->unit_address,
-				    adapter->buffer_list_dma, rxq_desc.desc,
+				    adapter->buffer_list_dma[0], rxq_desc.desc,
 				    adapter->filter_list_dma, mac_address);
 
 	if (rc != H_SUCCESS && try_again) {
@@ -623,14 +631,14 @@ static int ibmveth_open(struct net_device *netdev)
 
 	netdev_dbg(netdev, "open starting\n");
 
-	napi_enable(&adapter->napi);
+	napi_enable(&adapter->napi[0]);
 
 	for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		rxq_entries += adapter->rx_buff_pool[i].size;
+		rxq_entries += adapter->rx_buff_pool[0][i].size;
 
 	rc = -ENOMEM;
-	adapter->buffer_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
-	if (!adapter->buffer_list_addr) {
+	adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
+	if (!adapter->buffer_list_addr[0]) {
 		netdev_err(netdev, "unable to allocate list pages\n");
 		goto out;
 	}
@@ -643,17 +651,18 @@ static int ibmveth_open(struct net_device *netdev)
 
 	dev = &adapter->vdev->dev;
 
-	adapter->rx_queue.queue_len = sizeof(struct ibmveth_rx_q_entry) *
+	adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
 						rxq_entries;
-	adapter->rx_queue.queue_addr =
-		dma_alloc_coherent(dev, adapter->rx_queue.queue_len,
-				   &adapter->rx_queue.queue_dma, GFP_KERNEL);
-	if (!adapter->rx_queue.queue_addr)
+	adapter->rx_queue[0].queue_addr =
+		dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
+				   &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
+	if (!adapter->rx_queue[0].queue_addr)
 		goto out_free_filter_list;
 
-	adapter->buffer_list_dma = dma_map_single(dev,
-			adapter->buffer_list_addr, 4096, DMA_BIDIRECTIONAL);
-	if (dma_mapping_error(dev, adapter->buffer_list_dma)) {
+	adapter->buffer_list_dma[0] =
+		dma_map_single(dev, adapter->buffer_list_addr[0],
+			       4096, DMA_BIDIRECTIONAL);
+	if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
 		netdev_err(netdev, "unable to map buffer list pages\n");
 		goto out_free_queue_mem;
 	}
@@ -670,19 +679,21 @@ static int ibmveth_open(struct net_device *netdev)
 			goto out_free_tx_ltb;
 	}
 
-	adapter->rx_queue.index = 0;
-	adapter->rx_queue.num_slots = rxq_entries;
-	adapter->rx_queue.toggle = 1;
+	adapter->rx_queue[0].index = 0;
+	adapter->rx_queue[0].num_slots = rxq_entries;
+	adapter->rx_queue[0].toggle = 1;
 
 	mac_address = ether_addr_to_u64(netdev->dev_addr);
 
 	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
-					adapter->rx_queue.queue_len;
-	rxq_desc.fields.address = adapter->rx_queue.queue_dma;
+					adapter->rx_queue[0].queue_len;
+	rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
 
-	netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr);
+	netdev_dbg(netdev, "buffer list @ 0x%p\n",
+		   adapter->buffer_list_addr[0]);
 	netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
-	netdev_dbg(netdev, "receive q   @ 0x%p\n", adapter->rx_queue.queue_addr);
+	netdev_dbg(netdev, "receive q   @ 0x%p\n",
+		   adapter->rx_queue[0].queue_addr);
 
 	h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
 
@@ -693,7 +704,7 @@ static int ibmveth_open(struct net_device *netdev)
 			   lpar_rc);
 		netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
 			   "desc:0x%llx MAC:0x%llx\n",
-				     adapter->buffer_list_dma,
+				     adapter->buffer_list_dma[0],
 				     adapter->filter_list_dma,
 				     rxq_desc.desc,
 				     mac_address);
@@ -702,11 +713,11 @@ static int ibmveth_open(struct net_device *netdev)
 	}
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
-		if (!adapter->rx_buff_pool[i].active)
+		if (!adapter->rx_buff_pool[0][i].active)
 			continue;
-		if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[i])) {
+		if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
 			netdev_err(netdev, "unable to alloc pool\n");
-			adapter->rx_buff_pool[i].active = 0;
+			adapter->rx_buff_pool[0][i].active = 0;
 			rc = -ENOMEM;
 			goto out_free_buffer_pools;
 		}
@@ -738,9 +749,9 @@ static int ibmveth_open(struct net_device *netdev)
 
 out_free_buffer_pools:
 	while (--i >= 0) {
-		if (adapter->rx_buff_pool[i].active)
+		if (adapter->rx_buff_pool[0][i].active)
 			ibmveth_free_buffer_pool(adapter,
-						 &adapter->rx_buff_pool[i]);
+						 &adapter->rx_buff_pool[0][i]);
 	}
 out_unmap_filter_list:
 	dma_unmap_single(dev, adapter->filter_list_dma, 4096,
@@ -752,18 +763,18 @@ static int ibmveth_open(struct net_device *netdev)
 	}
 
 out_unmap_buffer_list:
-	dma_unmap_single(dev, adapter->buffer_list_dma, 4096,
+	dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
 			 DMA_BIDIRECTIONAL);
 out_free_queue_mem:
-	dma_free_coherent(dev, adapter->rx_queue.queue_len,
-			  adapter->rx_queue.queue_addr,
-			  adapter->rx_queue.queue_dma);
+	dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
+			  adapter->rx_queue[0].queue_addr,
+			  adapter->rx_queue[0].queue_dma);
 out_free_filter_list:
 	free_page((unsigned long)adapter->filter_list_addr);
 out_free_buffer_list:
-	free_page((unsigned long)adapter->buffer_list_addr);
+	free_page((unsigned long)adapter->buffer_list_addr[0]);
 out:
-	napi_disable(&adapter->napi);
+	napi_disable(&adapter->napi[0]);
 	return rc;
 }
 
@@ -776,7 +787,7 @@ static int ibmveth_close(struct net_device *netdev)
 
 	netdev_dbg(netdev, "close starting\n");
 
-	napi_disable(&adapter->napi);
+	napi_disable(&adapter->napi[0]);
 
 	netif_tx_stop_all_queues(netdev);
 
@@ -795,22 +806,22 @@ static int ibmveth_close(struct net_device *netdev)
 
 	ibmveth_update_rx_no_buffer(adapter);
 
-	dma_unmap_single(dev, adapter->buffer_list_dma, 4096,
+	dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
 			 DMA_BIDIRECTIONAL);
-	free_page((unsigned long)adapter->buffer_list_addr);
+	free_page((unsigned long)adapter->buffer_list_addr[0]);
 
 	dma_unmap_single(dev, adapter->filter_list_dma, 4096,
 			 DMA_BIDIRECTIONAL);
 	free_page((unsigned long)adapter->filter_list_addr);
 
-	dma_free_coherent(dev, adapter->rx_queue.queue_len,
-			  adapter->rx_queue.queue_addr,
-			  adapter->rx_queue.queue_dma);
+	dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
+			  adapter->rx_queue[0].queue_addr,
+			  adapter->rx_queue[0].queue_dma);
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		if (adapter->rx_buff_pool[i].active)
+		if (adapter->rx_buff_pool[0][i].active)
 			ibmveth_free_buffer_pool(adapter,
-						 &adapter->rx_buff_pool[i]);
+						 &adapter->rx_buff_pool[0][i]);
 
 	for (i = 0; i < netdev->real_num_tx_queues; i++)
 		ibmveth_free_tx_ltb(adapter, i);
@@ -1448,7 +1459,7 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
 static int ibmveth_poll(struct napi_struct *napi, int budget)
 {
 	struct ibmveth_adapter *adapter =
-			container_of(napi, struct ibmveth_adapter, napi);
+			container_of(napi, struct ibmveth_adapter, napi[0]);
 	struct net_device *netdev = adapter->netdev;
 	int frames_processed = 0;
 	unsigned long lpar_rc;
@@ -1573,11 +1584,11 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
 	unsigned long lpar_rc;
 
-	if (napi_schedule_prep(&adapter->napi)) {
+	if (napi_schedule_prep(&adapter->napi[0])) {
 		lpar_rc = h_vio_signal(adapter->vdev->unit_address,
 				       VIO_IRQ_DISABLE);
 		WARN_ON(lpar_rc != H_SUCCESS);
-		__napi_schedule(&adapter->napi);
+		__napi_schedule(&adapter->napi[0]);
 	}
 	return IRQ_HANDLED;
 }
@@ -1645,7 +1656,7 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
 	int need_restart = 0;
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size)
+		if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size)
 			break;
 
 	if (i == IBMVETH_NUM_BUFF_POOLS)
@@ -1660,9 +1671,9 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
 
 	/* Look for an active buffer pool that can hold the new MTU */
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
-		adapter->rx_buff_pool[i].active = 1;
+		adapter->rx_buff_pool[0][i].active = 1;
 
-		if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size) {
+		if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size) {
 			WRITE_ONCE(dev->mtu, new_mtu);
 			vio_cmo_set_dev_desired(viodev,
 						ibmveth_get_desired_dma
@@ -1693,8 +1704,7 @@ static void ibmveth_poll_controller(struct net_device *dev)
  *
  * @vdev: struct vio_dev for the device whose desired IO mem is to be returned
  *
- * Return value:
- *	Number of bytes of IO data the driver will need to perform well.
+ * Return: Number of bytes of IO data the driver will need to perform well.
  */
 static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
 {
@@ -1720,12 +1730,12 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
 		/* add the size of the active receive buffers */
-		if (adapter->rx_buff_pool[i].active)
+		if (adapter->rx_buff_pool[0][i].active)
 			ret +=
-			    adapter->rx_buff_pool[i].size *
-			    IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[i].
+			    adapter->rx_buff_pool[0][i].size *
+			    IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
 					     buff_size, tbl);
-		rxqentries += adapter->rx_buff_pool[i].size;
+		rxqentries += adapter->rx_buff_pool[0][i].size;
 	}
 	/* add the size of the receive queue entries */
 	ret += IOMMU_PAGE_ALIGN(
@@ -1844,7 +1854,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 	adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
 	ibmveth_init_link_settings(netdev);
 
-	netif_napi_add_weight(netdev, &adapter->napi, ibmveth_poll, 16);
+	netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
 
 	netdev->irq = dev->irq;
 	netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -1876,6 +1886,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 		netdev->features |= NETIF_F_FRAGLIST;
 	}
 
+	/* Initialize queue count - always 1 for now */
+	adapter->multi_queue = 0;
+	adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
+
 	if (ret == H_SUCCESS &&
 	    (ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
 		adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
@@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 		memcpy(pool_count, pool_count_cmo, sizeof(pool_count));
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
-		struct kobject *kobj = &adapter->rx_buff_pool[i].kobj;
+		struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
 		int error;
 
-		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
 					 pool_count[i], pool_size[i],
 					 pool_active[i]);
 		error = kobject_init_and_add(kobj, &ktype_veth_pool,
@@ -1949,7 +1963,7 @@ static void ibmveth_remove(struct vio_dev *dev)
 	cancel_work_sync(&adapter->work);
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		kobject_put(&adapter->rx_buff_pool[i].kobj);
+		kobject_put(&adapter->rx_buff_pool[0][i].kobj);
 
 	unregister_netdev(netdev);
 
@@ -2035,11 +2049,12 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
 			/* Make sure there is a buffer pool with buffers that
 			   can hold a packet of the size of the MTU */
 			for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
-				if (pool == &adapter->rx_buff_pool[i])
+				if (pool == &adapter->rx_buff_pool[0][i])
 					continue;
-				if (!adapter->rx_buff_pool[i].active)
+				if (!adapter->rx_buff_pool[0][i].active)
 					continue;
-				if (mtu <= adapter->rx_buff_pool[i].buff_size)
+				if (mtu <=
+				    adapter->rx_buff_pool[0][i].buff_size)
 					break;
 			}
 
@@ -2213,11 +2228,11 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
 
 	/* Set sane values for buffer pools */
 	for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
 					 pool_count[i], pool_size[i],
 					 pool_active[i]);
 
-	pool = &adapter->rx_buff_pool[0];
+	pool = &adapter->rx_buff_pool[0][0];
 	pool->skbuff = kunit_kcalloc(test, pool->size, sizeof(void *), GFP_KERNEL);
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
 
@@ -2225,7 +2240,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
 	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
 	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
 
-	correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0].size;
+	correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
 	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
 	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
 
@@ -2258,30 +2273,33 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
 
 	INIT_WORK(&adapter->work, ibmveth_reset_kunit);
 
-	adapter->rx_queue.queue_len = 1;
-	adapter->rx_queue.index = 0;
-	adapter->rx_queue.queue_addr = kunit_kzalloc(test, sizeof(struct ibmveth_rx_q_entry),
-						     GFP_KERNEL);
-	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, adapter->rx_queue.queue_addr);
+	adapter->rx_queue[0].queue_len = 1;
+	adapter->rx_queue[0].index = 0;
+	adapter->rx_queue[0].queue_addr =
+		kunit_kzalloc(test, sizeof(struct ibmveth_rx_q_entry),
+			      GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, adapter->rx_queue[0].queue_addr);
 
 	/* Set sane values for buffer pools */
 	for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
 					 pool_count[i], pool_size[i],
 					 pool_active[i]);
 
-	pool = &adapter->rx_buff_pool[0];
+	pool = &adapter->rx_buff_pool[0][0];
 	pool->skbuff = kunit_kcalloc(test, pool->size, sizeof(void *), GFP_KERNEL);
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
 
-	adapter->rx_queue.queue_addr[0].correlator = (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
+	adapter->rx_queue[0].queue_addr[0].correlator =
+		(u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
 	KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
 
-	adapter->rx_queue.queue_addr[0].correlator = (u64)0 << 32 | adapter->rx_buff_pool[0].size;
+	adapter->rx_queue[0].queue_addr[0].correlator =
+		(u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
 	KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
 
 	pool->skbuff[0] = skb;
-	adapter->rx_queue.queue_addr[0].correlator = (u64)0 << 32 | 0;
+	adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
 	KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
 
 	flush_work(&adapter->work);
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index c88dbeb7cd96..8eaa3942fc40 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -258,6 +258,8 @@ static inline long h_illan_attributes(unsigned long unit_address,
 #define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
 #define IBMVETH_MAX_QUEUES 16U
 #define IBMVETH_DEFAULT_QUEUES 8U
+#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_DEFAULT_RX_QUEUES 1U
 #define IBMVETH_MAX_RX_PER_HCALL 8U
 
 static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
@@ -294,18 +296,23 @@ struct ibmveth_rx_q {
 struct ibmveth_adapter {
 	struct vio_dev *vdev;
 	struct net_device *netdev;
-	struct napi_struct napi;
+	struct napi_struct napi[IBMVETH_MAX_RX_QUEUES];
 	struct work_struct work;
 	unsigned int mcastFilterSize;
-	void *buffer_list_addr;
+	void *buffer_list_addr[IBMVETH_MAX_RX_QUEUES];
 	void *filter_list_addr;
 	void *tx_ltb_ptr[IBMVETH_MAX_QUEUES];
 	unsigned int tx_ltb_size;
 	dma_addr_t tx_ltb_dma[IBMVETH_MAX_QUEUES];
-	dma_addr_t buffer_list_dma;
+	dma_addr_t buffer_list_dma[IBMVETH_MAX_RX_QUEUES];
 	dma_addr_t filter_list_dma;
-	struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS];
-	struct ibmveth_rx_q rx_queue;
+	struct ibmveth_buff_pool
+		rx_buff_pool[IBMVETH_MAX_RX_QUEUES][IBMVETH_NUM_BUFF_POOLS];
+	struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES];
+	u64 queue_handle[IBMVETH_MAX_RX_QUEUES];
+	unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES];
+	int multi_queue;
+	unsigned int num_rx_queues;
 	int rx_csum;
 	int large_send;
 	bool is_active_trunk;
@@ -325,6 +332,7 @@ struct ibmveth_adapter {
 	u64 tx_send_failed;
 	u64 tx_large_packets;
 	u64 rx_large_packets;
+
 	/* Ethtool settings */
 	u8 duplex;
 	u32 speed;
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 03/14] ibmveth: Refactor RX resource allocation for MQ RX bring-up

[Mingming Cao]

ibmveth_open() allocates the filter list and every RX queue inline.
That is already a long sequence and would get uglier once we loop over
num_rx_queues, especially on error unwind.

Pull the RX bits into helpers and wire them into open()/close() in the
same patch:

  ibmveth_alloc_filter_list() / ibmveth_free_filter_list()
    - shared multicast filter list (one per adapter, not per queue)

  ibmveth_alloc_rx_queues() / ibmveth_cleanup_rx_resources()
    - per-queue buffer lists and RX rings, looping [0, num_rx_queues)

alloc_rx_queues() rolls back on failure so open() does not need nested
goto chains for every queue index. open-failure and close release the
same resources through the same helpers.

Runtime behavior stays single-queue (num_rx_queues is still 1). Buffer
pools, IRQ, TX LTB, and PHYP registration remain inline for later
helper patches.

Also set rc = -ENOMEM before the TX LTB allocation loop so a failed
ibmveth_allocate_tx_ltb() still returns a useful errno after the RX
allocation blocks move into helpers.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Introduce RX/filter allocation helpers in the same patch that wires
  their first open/close callers; v3 left unused statics ahead of the
  old open/close pipeline patch.
- Preserve correct -ENOMEM return on TX LTB allocation failure after
  the RX helper extract.
- Drop reliance on v3's separate "open/close pipeline" patch for this
  wiring.

 drivers/net/ethernet/ibm/ibmveth.c | 268 ++++++++++++++++++++---------
 1 file changed, 190 insertions(+), 78 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 8e758362cb26..1007dd95cde0 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -151,6 +151,184 @@ static unsigned int ibmveth_real_max_tx_queues(void)
 	return min(n_cpu, IBMVETH_MAX_QUEUES);
 }
 
+/**
+ * ibmveth_alloc_filter_list - Allocate and map filter list
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
+{
+	struct device *dev = &adapter->vdev->dev;
+	struct net_device *netdev = adapter->netdev;
+
+	adapter->filter_list_addr = (void *)get_zeroed_page(GFP_KERNEL);
+	if (!adapter->filter_list_addr) {
+		netdev_err(netdev, "unable to allocate filter pages\n");
+		return -ENOMEM;
+	}
+
+	adapter->filter_list_dma = dma_map_single(dev,
+						  adapter->filter_list_addr,
+						  4096, DMA_BIDIRECTIONAL);
+	if (dma_mapping_error(dev, adapter->filter_list_dma)) {
+		netdev_err(netdev, "unable to map filter list pages\n");
+		free_page((unsigned long)adapter->filter_list_addr);
+		adapter->filter_list_addr = NULL;
+		return -ENOMEM;
+	}
+
+	netdev_dbg(netdev, "filter list @ 0x%p (DMA: 0x%llx)\n",
+		   adapter->filter_list_addr,
+		   (unsigned long long)adapter->filter_list_dma);
+
+	return 0;
+}
+
+/**
+ * ibmveth_free_filter_list - Free filter list resources
+ * @adapter: ibmveth adapter structure
+ */
+static void
+ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
+{
+	struct device *dev = &adapter->vdev->dev;
+
+	if (adapter->filter_list_dma) {
+		dma_unmap_single(dev, adapter->filter_list_dma, 4096,
+				 DMA_BIDIRECTIONAL);
+		adapter->filter_list_dma = 0;
+	}
+
+	if (adapter->filter_list_addr) {
+		free_page((unsigned long)adapter->filter_list_addr);
+		adapter->filter_list_addr = NULL;
+	}
+}
+
+/**
+ * ibmveth_alloc_rx_queues - Allocate per-queue RX resources
+ * @adapter: ibmveth adapter structure
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
+{
+	struct device *dev = &adapter->vdev->dev;
+	struct net_device *netdev = adapter->netdev;
+	int i;
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		adapter->buffer_list_addr[i] =
+			(void *)get_zeroed_page(GFP_KERNEL);
+		if (!adapter->buffer_list_addr[i]) {
+			netdev_err(netdev,
+				   "unable to allocate buffer list for queue %d\n",
+				   i);
+			goto err_cleanup;
+		}
+
+		adapter->rx_queue[i].queue_len =
+			sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+		adapter->rx_queue[i].queue_addr =
+			dma_alloc_coherent(dev, adapter->rx_queue[i].queue_len,
+					   &adapter->rx_queue[i].queue_dma,
+					   GFP_KERNEL);
+		if (!adapter->rx_queue[i].queue_addr) {
+			netdev_err(netdev,
+				   "unable to allocate RX queue for queue %d\n",
+				   i);
+			goto err_cleanup;
+		}
+
+		adapter->buffer_list_dma[i] =
+			dma_map_single(dev, adapter->buffer_list_addr[i],
+				       4096, DMA_BIDIRECTIONAL);
+		if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
+			netdev_err(netdev,
+				   "unable to map buffer list for queue %d\n",
+				   i);
+			adapter->buffer_list_dma[i] = 0;
+			goto err_cleanup;
+		}
+
+		adapter->rx_queue[i].index = 0;
+		adapter->rx_queue[i].num_slots = rxq_entries;
+		adapter->rx_queue[i].toggle = 1;
+
+		netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
+			   i, adapter->buffer_list_addr[i],
+			   (unsigned long long)adapter->buffer_list_dma[i],
+			   adapter->rx_queue[i].queue_addr,
+			   (unsigned long long)adapter->rx_queue[i].queue_dma,
+			   (unsigned long long)rxq_entries);
+	}
+
+	netdev_dbg(netdev, "allocated %d RX queue(s) with %d entries each\n",
+		   adapter->num_rx_queues, rxq_entries);
+
+	return 0;
+
+err_cleanup:
+	/* Clean up previously allocated queues */
+	for (; i >= 0; i--) {
+		if (adapter->buffer_list_dma[i]) {
+			dma_unmap_single(dev, adapter->buffer_list_dma[i],
+					 4096, DMA_BIDIRECTIONAL);
+			adapter->buffer_list_dma[i] = 0;
+		}
+		if (adapter->rx_queue[i].queue_addr) {
+			dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+					  adapter->rx_queue[i].queue_addr,
+					  adapter->rx_queue[i].queue_dma);
+			adapter->rx_queue[i].queue_addr = NULL;
+		}
+		if (adapter->buffer_list_addr[i]) {
+			free_page((unsigned long)adapter->buffer_list_addr[i]);
+			adapter->buffer_list_addr[i] = NULL;
+		}
+	}
+
+	return -ENOMEM;
+}
+
+/**
+ * ibmveth_cleanup_rx_resources - Free all RX queue resources
+ * @adapter: ibmveth adapter structure
+ */
+static void
+ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
+{
+	struct device *dev = &adapter->vdev->dev;
+	int i;
+
+	netdev_dbg(adapter->netdev, "cleaning up %d RX queue(s)\n",
+		   adapter->num_rx_queues);
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		if (adapter->buffer_list_dma[i]) {
+			dma_unmap_single(dev, adapter->buffer_list_dma[i],
+					 4096, DMA_BIDIRECTIONAL);
+			adapter->buffer_list_dma[i] = 0;
+		}
+
+		if (adapter->rx_queue[i].queue_addr) {
+			dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+					  adapter->rx_queue[i].queue_addr,
+					  adapter->rx_queue[i].queue_dma);
+			adapter->rx_queue[i].queue_addr = NULL;
+		}
+
+		if (adapter->buffer_list_addr[i]) {
+			free_page((unsigned long)adapter->buffer_list_addr[i]);
+			adapter->buffer_list_addr[i] = NULL;
+		}
+	}
+}
+
 /* setup the initial settings for a buffer pool */
 static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
 				     u32 pool_index, u32 pool_size,
@@ -627,74 +805,34 @@ static int ibmveth_open(struct net_device *netdev)
 	int rc;
 	union ibmveth_buf_desc rxq_desc;
 	int i;
-	struct device *dev;
 
 	netdev_dbg(netdev, "open starting\n");
 
 	napi_enable(&adapter->napi[0]);
 
-	for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
+	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
 		rxq_entries += adapter->rx_buff_pool[0][i].size;
 
-	rc = -ENOMEM;
-	adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
-	if (!adapter->buffer_list_addr[0]) {
-		netdev_err(netdev, "unable to allocate list pages\n");
+	rc = ibmveth_alloc_filter_list(adapter);
+	if (rc)
 		goto out;
-	}
 
-	adapter->filter_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
-	if (!adapter->filter_list_addr) {
-		netdev_err(netdev, "unable to allocate filter pages\n");
-		goto out_free_buffer_list;
-	}
-
-	dev = &adapter->vdev->dev;
-
-	adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
-						rxq_entries;
-	adapter->rx_queue[0].queue_addr =
-		dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
-				   &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
-	if (!adapter->rx_queue[0].queue_addr)
+	rc = ibmveth_alloc_rx_queues(adapter, rxq_entries);
+	if (rc)
 		goto out_free_filter_list;
 
-	adapter->buffer_list_dma[0] =
-		dma_map_single(dev, adapter->buffer_list_addr[0],
-			       4096, DMA_BIDIRECTIONAL);
-	if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
-		netdev_err(netdev, "unable to map buffer list pages\n");
-		goto out_free_queue_mem;
-	}
-
-	adapter->filter_list_dma = dma_map_single(dev,
-			adapter->filter_list_addr, 4096, DMA_BIDIRECTIONAL);
-	if (dma_mapping_error(dev, adapter->filter_list_dma)) {
-		netdev_err(netdev, "unable to map filter list pages\n");
-		goto out_unmap_buffer_list;
-	}
-
+	rc = -ENOMEM;
 	for (i = 0; i < netdev->real_num_tx_queues; i++) {
 		if (ibmveth_allocate_tx_ltb(adapter, i))
 			goto out_free_tx_ltb;
 	}
 
-	adapter->rx_queue[0].index = 0;
-	adapter->rx_queue[0].num_slots = rxq_entries;
-	adapter->rx_queue[0].toggle = 1;
-
 	mac_address = ether_addr_to_u64(netdev->dev_addr);
 
 	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
 					adapter->rx_queue[0].queue_len;
 	rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
 
-	netdev_dbg(netdev, "buffer list @ 0x%p\n",
-		   adapter->buffer_list_addr[0]);
-	netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
-	netdev_dbg(netdev, "receive q   @ 0x%p\n",
-		   adapter->rx_queue[0].queue_addr);
-
 	h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
 
 	lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
@@ -709,7 +847,7 @@ static int ibmveth_open(struct net_device *netdev)
 				     rxq_desc.desc,
 				     mac_address);
 		rc = -ENONET;
-		goto out_unmap_filter_list;
+		goto out_free_tx_ltb;
 	}
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
@@ -736,8 +874,6 @@ static int ibmveth_open(struct net_device *netdev)
 		goto out_free_buffer_pools;
 	}
 
-	rc = -ENOMEM;
-
 	netdev_dbg(netdev, "initial replenish cycle\n");
 	ibmveth_interrupt(netdev->irq, netdev);
 
@@ -753,26 +889,12 @@ static int ibmveth_open(struct net_device *netdev)
 			ibmveth_free_buffer_pool(adapter,
 						 &adapter->rx_buff_pool[0][i]);
 	}
-out_unmap_filter_list:
-	dma_unmap_single(dev, adapter->filter_list_dma, 4096,
-			 DMA_BIDIRECTIONAL);
-
 out_free_tx_ltb:
-	while (--i >= 0) {
+	while (--i >= 0)
 		ibmveth_free_tx_ltb(adapter, i);
-	}
-
-out_unmap_buffer_list:
-	dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
-			 DMA_BIDIRECTIONAL);
-out_free_queue_mem:
-	dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
-			  adapter->rx_queue[0].queue_addr,
-			  adapter->rx_queue[0].queue_dma);
+	ibmveth_cleanup_rx_resources(adapter);
 out_free_filter_list:
-	free_page((unsigned long)adapter->filter_list_addr);
-out_free_buffer_list:
-	free_page((unsigned long)adapter->buffer_list_addr[0]);
+	ibmveth_free_filter_list(adapter);
 out:
 	napi_disable(&adapter->napi[0]);
 	return rc;
@@ -781,7 +903,6 @@ static int ibmveth_open(struct net_device *netdev)
 static int ibmveth_close(struct net_device *netdev)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
-	struct device *dev = &adapter->vdev->dev;
 	long lpar_rc;
 	int i;
 
@@ -806,17 +927,8 @@ static int ibmveth_close(struct net_device *netdev)
 
 	ibmveth_update_rx_no_buffer(adapter);
 
-	dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
-			 DMA_BIDIRECTIONAL);
-	free_page((unsigned long)adapter->buffer_list_addr[0]);
-
-	dma_unmap_single(dev, adapter->filter_list_dma, 4096,
-			 DMA_BIDIRECTIONAL);
-	free_page((unsigned long)adapter->filter_list_addr);
-
-	dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
-			  adapter->rx_queue[0].queue_addr,
-			  adapter->rx_queue[0].queue_dma);
+	ibmveth_cleanup_rx_resources(adapter);
+	ibmveth_free_filter_list(adapter);
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
 		if (adapter->rx_buff_pool[0][i].active)
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 04/14] ibmveth: Refactor buffer pool management for per-queue MQ RX

[Mingming Cao]

Legacy ibmveth uses five adapter-level RX buffer pools (512 B through
64 KiB). pool_active[] enables the standard-MTU pools by default;
larger pools activate when MTU requires them. With single-queue RX
that set is shared on one completion path.

MQ requires the same pool model per queue: buffers post with
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against a queue handle and completions
return on that queue. Sharing pools across queues would mix ownership
and break queue-local replenish/drain/teardown.

Refactor around queue-local pools:

  rx_buff_pool[queue][pool]
  ibmveth_alloc_queue_buffer_pools()
  ibmveth_free_queue_buffer_pools()
  ibmveth_alloc_buffer_pools() / ibmveth_free_buffer_pools()

Queue 0 remains the template for pool geometry (size, buff_size,
threshold, index, active). For queues 1..N we copy metadata from
queue 0, then allocate backing arrays/skbs per queue.

Wire the helpers into open()/close() in the same patch. Runtime
remains single-queue (num_rx_queues is still 1).

Error handling is queue-safe:

  - allocation failure unwinds only what that queue allocated, then
    prior queues in the caller
  - free paths release by real allocations (free_map/dma_addr/skbuff),
    not only pool->active

That allocation-based free check is intentional: later resize and
failure paths can leave memory allocated after active was cleared.

Also disable the 64 KiB pool by default at standard MTU. MTU changes
activate it when required; leaving it enabled would pin about 16 MiB
per RX queue in MQ mode.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Introduce the pool helpers in the same patch that wires their first
  open/close callers, instead of leaving unused statics.
- Copy pool->index when cloning queue-0 geometry to later queues
  (needed for correlators; also required by incremental resize).

 drivers/net/ethernet/ibm/ibmveth.c | 164 +++++++++++++++++++++++++----
 drivers/net/ethernet/ibm/ibmveth.h |   2 +-
 2 files changed, 145 insertions(+), 21 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 1007dd95cde0..7a2ed49cad4f 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -625,6 +625,145 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
 	}
 }
 
+/**
+ * ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Allocates all active buffer pools for the specified queue.
+ * Pool metadata must be initialized before calling this function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int ibmveth_alloc_queue_buffer_pools(struct ibmveth_adapter *adapter,
+					    int queue)
+{
+	struct net_device *netdev = adapter->netdev;
+	int i;
+
+	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+		struct ibmveth_buff_pool *bpool =
+			&adapter->rx_buff_pool[queue][i];
+
+		if (!bpool->active)
+			continue;
+
+		if (ibmveth_alloc_buffer_pool(bpool)) {
+			netdev_err(netdev,
+				   "pool %d/%d alloc failed (size=%u count=%u)\n",
+				   i, queue,
+				   bpool->buff_size,
+				   bpool->size);
+			bpool->active = 0;
+
+			/* Free pools allocated so far for this queue */
+			while (--i >= 0) {
+				struct ibmveth_buff_pool *fpool =
+					&adapter->rx_buff_pool[queue][i];
+
+				if (fpool->active)
+					ibmveth_free_buffer_pool(adapter,
+								 fpool);
+			}
+			return -ENOMEM;
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Frees all active buffer pools for the specified queue.
+ */
+static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
+					    int queue)
+{
+	int i;
+
+	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+		struct ibmveth_buff_pool *pool =
+			&adapter->rx_buff_pool[queue][i];
+
+		/* Free pool if it has allocated memory, regardless of
+		 * active flag. Pools may have memory allocated but not
+		 * marked active during queue scale-up, so we must check
+		 * for actual allocations.
+		 */
+		if (pool->free_map || pool->dma_addr || pool->skbuff)
+			ibmveth_free_buffer_pool(adapter, pool);
+	}
+}
+
+/**
+ * ibmveth_alloc_buffer_pools - Allocate buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Initializes pool metadata for queues 1-N from queue 0 settings,
+ * then allocates buffer pools for all queues using the helper function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
+{
+	struct net_device *netdev = adapter->netdev;
+	int i, q, rc;
+
+	/* Initialize pool metadata for queues 1-15 from queue 0 settings */
+	for (q = 1; q < adapter->num_rx_queues; q++) {
+		for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+			struct ibmveth_buff_pool *src =
+				&adapter->rx_buff_pool[0][i];
+			struct ibmveth_buff_pool *dst =
+				&adapter->rx_buff_pool[q][i];
+
+			dst->size = src->size;
+			dst->index = src->index;
+			dst->buff_size = src->buff_size;
+			dst->threshold = src->threshold;
+			dst->active = src->active;
+		}
+	}
+
+	/* Allocate actual buffers for all queues */
+	for (q = 0; q < adapter->num_rx_queues; q++) {
+		rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
+		if (rc) {
+			/* Free pools for all previous queues */
+			while (--q >= 0)
+				ibmveth_free_queue_buffer_pools(adapter, q);
+			return rc;
+		}
+	}
+
+	netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
+		   adapter->num_rx_queues);
+	return 0;
+}
+
+/**
+ * ibmveth_free_buffer_pools - Free buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees buffer pools for all queues using the helper function.
+ */
+static void
+ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
+{
+	int q;
+
+	/* Free buffer pools for all queues */
+	for (q = 0; q < adapter->num_rx_queues; q++)
+		ibmveth_free_queue_buffer_pools(adapter, q);
+
+	netdev_dbg(adapter->netdev, "freed buffer pools for %d queue(s)\n",
+		   adapter->num_rx_queues);
+}
+
 /**
  * ibmveth_remove_buffer_from_pool - remove a buffer from a pool
  * @adapter: adapter instance
@@ -850,16 +989,9 @@ static int ibmveth_open(struct net_device *netdev)
 		goto out_free_tx_ltb;
 	}
 
-	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
-		if (!adapter->rx_buff_pool[0][i].active)
-			continue;
-		if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
-			netdev_err(netdev, "unable to alloc pool\n");
-			adapter->rx_buff_pool[0][i].active = 0;
-			rc = -ENOMEM;
-			goto out_free_buffer_pools;
-		}
-	}
+	rc = ibmveth_alloc_buffer_pools(adapter);
+	if (rc)
+		goto out_free_tx_ltb;
 
 	netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
 	rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
@@ -884,11 +1016,7 @@ static int ibmveth_open(struct net_device *netdev)
 	return 0;
 
 out_free_buffer_pools:
-	while (--i >= 0) {
-		if (adapter->rx_buff_pool[0][i].active)
-			ibmveth_free_buffer_pool(adapter,
-						 &adapter->rx_buff_pool[0][i]);
-	}
+	ibmveth_free_buffer_pools(adapter);
 out_free_tx_ltb:
 	while (--i >= 0)
 		ibmveth_free_tx_ltb(adapter, i);
@@ -927,14 +1055,10 @@ static int ibmveth_close(struct net_device *netdev)
 
 	ibmveth_update_rx_no_buffer(adapter);
 
+	ibmveth_free_buffer_pools(adapter);
 	ibmveth_cleanup_rx_resources(adapter);
 	ibmveth_free_filter_list(adapter);
 
-	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
-		if (adapter->rx_buff_pool[0][i].active)
-			ibmveth_free_buffer_pool(adapter,
-						 &adapter->rx_buff_pool[0][i]);
-
 	for (i = 0; i < netdev->real_num_tx_queues; i++)
 		ibmveth_free_tx_ltb(adapter, i);
 
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 8eaa3942fc40..abcd4054c67e 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -265,7 +265,7 @@ static inline long h_illan_attributes(unsigned long unit_address,
 static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
 static int pool_count[] = { 256, 512, 256, 256, 256 };
 static int pool_count_cmo[] = { 256, 512, 256, 256, 64 };
-static int pool_active[] = { 1, 1, 0, 0, 1};
+static int pool_active[] = { 1, 1, 0, 0, 0};
 
 #define IBM_VETH_INVALID_MAP ((u16)0xffff)
 
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 05/14] ibmveth: Refactor RX interrupt control for MQ RX queues

[Mingming Cao]

Queue 0 and subordinate RX queues use different interrupt control
interfaces in PHYP:

  - queue 0: h_vio_signal() after h_register_logical_lan()
  - queue N: H_VIOCTL against the queue handle/hwirq mapping

The current code is single-queue oriented and cannot safely scale to
multiple RX queues in poll completion and open/close IRQ setup.

Introduce queue-indexed interrupt helpers and wire them into
open()/close()/poll()/interrupt in the same patch:

  ibmveth_toggle_irq() / enable_irq() / disable_irq()
  ibmveth_setup_rx_interrupts() / ibmveth_cleanup_rx_interrupts()
  ibmveth_schedule_rx_queue()

These helpers centralize queue0-vs-subordinate dispatch.
request_irq() uses &adapter->napi[i] as the per-queue cookie so the
handler can resolve the queue index.

Move napi_enable() into setup_rx_interrupts() (after LAN registration
and buffer-pool allocation): request_irq -> napi_enable. In this
single-queue tree, setup does not yet unmask PHYP; the first poll/kick
still enables queue 0 via schedule_rx_queue().

Factor process-context RX kicks (open, resume, pool sysfs, netpoll)
into ibmveth_schedule_rx_queue(); keep ibmveth_interrupt() as a thin
IRQ-only wrapper.

cleanup_rx_interrupts() masks PHYP and synchronizes IRQs before
napi_disable (intentional storm-safety vs classic close ordering).

Runtime remains single-queue (num_rx_queues is still 1).

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Include irq.h / irqdomain.h with first irq_dispose_mapping() use.
- Introduce IRQ helpers in the same patch that wires open/close/poll
  callers, instead of leaving unused statics.
- Factor process-context RX kicks into ibmveth_schedule_rx_queue();
  keep ibmveth_interrupt() as the IRQ-only wrapper.
- On cleanup, mask PHYP and synchronize_irq before napi_disable
  (storm-safety; not fully behavior-preserving vs classic close).
- Leave queue_irq[0] set after cleanup (queue 0 uses netdev->irq;
  next open reuses it). Only subordinate virqs are disposed.

 drivers/net/ethernet/ibm/ibmveth.c | 258 +++++++++++++++++++++++++----
 1 file changed, 225 insertions(+), 33 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 7a2ed49cad4f..664169c4d27a 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -21,6 +21,8 @@
 #include <linux/skbuff.h>
 #include <linux/init.h>
 #include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
 #include <linux/mm.h>
 #include <linux/pm.h>
 #include <linux/ethtool.h>
@@ -329,6 +331,203 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
 	}
 }
 
+/**
+ * ibmveth_toggle_irq - Common helper to enable/disable queue interrupts
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ * @enable: true to enable, false to disable
+ *
+ * For queue 0 (primary), uses h_vio_signal() as it's registered via
+ * h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL
+ * with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control.
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_toggle_irq(struct ibmveth_adapter *adapter, int queue_index,
+		   bool enable)
+{
+	unsigned long rc;
+	unsigned long irq = adapter->queue_irq[queue_index];
+	const char *action = enable ? "enable" : "disable";
+
+	if (queue_index == 0) {
+		/* Primary queue: use h_vio_signal() */
+		rc = h_vio_signal(adapter->vdev->unit_address,
+				  enable ? VIO_IRQ_ENABLE : VIO_IRQ_DISABLE);
+	} else {
+		/* Subordinate queues: use H_VIOCTL with hardware IRQ */
+		struct irq_data *irq_data = irq_get_irq_data(irq);
+		irq_hw_number_t hwirq;
+		u64 vioctl_cmd = enable ? H_ENABLE_VIO_INTERRUPT :
+			H_DISABLE_VIO_INTERRUPT;
+
+		if (!irq_data) {
+			netdev_err(adapter->netdev,
+				   "Failed to get IRQ data for queue %d (virq=%lu)\n",
+				   queue_index, irq);
+			return -EINVAL;
+		}
+
+		hwirq = irqd_to_hwirq(irq_data);
+		rc = plpar_hcall_norets(H_VIOCTL,
+					adapter->vdev->unit_address,
+					vioctl_cmd,
+					hwirq, 0, 0);
+
+		if (rc == H_PARAMETER) {
+			/* H_PARAMETER is non-fatal when IRQ is already in
+			 * the requested state.
+			 */
+			netdev_warn_once(adapter->netdev,
+					 "H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n",
+					 action, queue_index, hwirq);
+			return 0;
+		}
+	}
+
+	if (rc)
+		netdev_err(adapter->netdev,
+			   "Failed to %s IRQ for queue %d, rc=%ld\n",
+			   action, queue_index, rc);
+	return rc;
+}
+
+/**
+ * ibmveth_disable_irq - Disable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_disable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+	return ibmveth_toggle_irq(adapter, queue_index, false);
+}
+
+/**
+ * ibmveth_enable_irq - Enable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int
+ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+	return ibmveth_toggle_irq(adapter, queue_index, true);
+}
+
+/**
+ * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
+ * @adapter: ibmveth adapter structure
+ *
+ * Registers interrupt handlers for all RX queues and enables NAPI polling.
+ * On error, cleans up any successfully registered IRQs before returning.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+	struct net_device *netdev = adapter->netdev;
+	int i, rc;
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		if (!adapter->queue_irq[i]) {
+			netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
+			rc = -EINVAL;
+			goto err_free_irqs;
+		}
+
+		rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt,
+				 0, netdev->name, &adapter->napi[i]);
+		if (rc) {
+			netdev_err(netdev,
+				   "request_irq() failed for irq 0x%x queue %d: %d\n",
+				   adapter->queue_irq[i], i, rc);
+			goto err_free_irqs;
+		}
+	}
+
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		napi_enable(&adapter->napi[i]);
+
+	return 0;
+
+err_free_irqs:
+	while (--i >= 0)
+		free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+	return rc;
+}
+
+/**
+ * ibmveth_cleanup_rx_interrupts - Mask PHYP, disable NAPI, free IRQs
+ * @adapter: ibmveth adapter structure
+ *
+ * Tears down RX interrupt delivery for all queues. Mask PHYP before
+ * napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without
+ * masking (same storm window as scale-down). Safe for close and for
+ * open failure after setup_rx_interrupts() already unmasked PHYP.
+ */
+static void
+ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+	int i;
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		if (adapter->queue_irq[i]) {
+			ibmveth_disable_irq(adapter, i);
+			synchronize_irq(adapter->queue_irq[i]);
+		}
+	}
+
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		napi_disable(&adapter->napi[i]);
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		if (adapter->queue_irq[i])
+			free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+	}
+
+	/* Dispose IRQ mappings for subordinate queues (1-15).
+	 * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
+	 */
+	for (i = 1; i < adapter->num_rx_queues; i++) {
+		if (adapter->queue_irq[i]) {
+			irq_dispose_mapping(adapter->queue_irq[i]);
+			adapter->queue_irq[i] = 0;
+		}
+	}
+
+	/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */
+}
+
+/**
+ * ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue
+ * @adapter: ibmveth adapter structure
+ * @qindex: RX queue index
+ *
+ * Shared by the IRQ handler and process-context kick paths (open, resume,
+ * pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper.
+ */
+static void ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
+				      int qindex)
+{
+	struct napi_struct *napi = &adapter->napi[qindex];
+	unsigned long lpar_rc;
+
+	if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
+		return;
+
+	if (napi_schedule_prep(napi)) {
+		lpar_rc = ibmveth_disable_irq(adapter, qindex);
+		WARN_ON(lpar_rc != H_SUCCESS);
+		__napi_schedule(napi);
+	}
+}
+
 /* setup the initial settings for a buffer pool */
 static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
 				     u32 pool_index, u32 pool_size,
@@ -947,8 +1146,6 @@ static int ibmveth_open(struct net_device *netdev)
 
 	netdev_dbg(netdev, "open starting\n");
 
-	napi_enable(&adapter->napi[0]);
-
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
 		rxq_entries += adapter->rx_buff_pool[0][i].size;
 
@@ -972,7 +1169,8 @@ static int ibmveth_open(struct net_device *netdev)
 					adapter->rx_queue[0].queue_len;
 	rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
 
-	h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
+	adapter->queue_irq[0] = netdev->irq;
+	ibmveth_disable_irq(adapter, 0);
 
 	lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
 
@@ -993,21 +1191,16 @@ static int ibmveth_open(struct net_device *netdev)
 	if (rc)
 		goto out_free_tx_ltb;
 
-	netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
-	rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
-			 netdev);
-	if (rc != 0) {
-		netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
-			   netdev->irq, rc);
+	rc = ibmveth_setup_rx_interrupts(adapter);
+	if (rc) {
 		do {
 			lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
 		} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
 		goto out_free_buffer_pools;
 	}
 
 	netdev_dbg(netdev, "initial replenish cycle\n");
-	ibmveth_interrupt(netdev->irq, netdev);
+	ibmveth_schedule_rx_queue(adapter, 0);
 
 	netif_tx_start_all_queues(netdev);
 
@@ -1024,7 +1217,6 @@ static int ibmveth_open(struct net_device *netdev)
 out_free_filter_list:
 	ibmveth_free_filter_list(adapter);
 out:
-	napi_disable(&adapter->napi[0]);
 	return rc;
 }
 
@@ -1036,11 +1228,10 @@ static int ibmveth_close(struct net_device *netdev)
 
 	netdev_dbg(netdev, "close starting\n");
 
-	napi_disable(&adapter->napi[0]);
-
 	netif_tx_stop_all_queues(netdev);
 
-	h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
+	/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */
+	ibmveth_cleanup_rx_interrupts(adapter);
 
 	do {
 		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
@@ -1051,8 +1242,6 @@ static int ibmveth_close(struct net_device *netdev)
 			   "continuing with close\n", lpar_rc);
 	}
 
-	free_irq(netdev->irq, netdev);
-
 	ibmveth_update_rx_no_buffer(adapter);
 
 	ibmveth_free_buffer_pools(adapter);
@@ -1798,15 +1987,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 	/* We think we are done - reenable interrupts,
 	 * then check once more to make sure we are done.
 	 */
-	lpar_rc = h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_ENABLE);
+	lpar_rc = ibmveth_enable_irq(adapter, 0);
 	if (WARN_ON(lpar_rc != H_SUCCESS)) {
 		schedule_work(&adapter->work);
 		goto out;
 	}
 
 	if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
-		lpar_rc = h_vio_signal(adapter->vdev->unit_address,
-				       VIO_IRQ_DISABLE);
+		lpar_rc = ibmveth_disable_irq(adapter, 0);
 		goto restart_poll;
 	}
 
@@ -1816,16 +2004,16 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 
 static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
 {
-	struct net_device *netdev = dev_instance;
+	struct napi_struct *napi = dev_instance;
+	struct net_device *netdev = napi->dev;
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
-	unsigned long lpar_rc;
+	int qindex;
 
-	if (napi_schedule_prep(&adapter->napi[0])) {
-		lpar_rc = h_vio_signal(adapter->vdev->unit_address,
-				       VIO_IRQ_DISABLE);
-		WARN_ON(lpar_rc != H_SUCCESS);
-		__napi_schedule(&adapter->napi[0]);
-	}
+	qindex = napi - adapter->napi;
+	if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
+		return IRQ_NONE;
+
+	ibmveth_schedule_rx_queue(adapter, qindex);
 	return IRQ_HANDLED;
 }
 
@@ -1930,8 +2118,10 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
 #ifdef CONFIG_NET_POLL_CONTROLLER
 static void ibmveth_poll_controller(struct net_device *dev)
 {
-	ibmveth_replenish_task(netdev_priv(dev));
-	ibmveth_interrupt(dev->irq, dev);
+	struct ibmveth_adapter *adapter = netdev_priv(dev);
+
+	ibmveth_replenish_task(adapter);
+	ibmveth_schedule_rx_queue(adapter, 0);
 }
 #endif
 
@@ -2343,8 +2533,8 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
 	}
 	rtnl_unlock();
 
-	/* kick the interrupt handler to allocate/deallocate pools */
-	ibmveth_interrupt(netdev->irq, netdev);
+	/* kick RX processing to allocate/deallocate pools */
+	ibmveth_schedule_rx_queue(adapter, 0);
 	return count;
 
 unlock_err:
@@ -2384,7 +2574,9 @@ static struct kobj_type ktype_veth_pool = {
 static int ibmveth_resume(struct device *dev)
 {
 	struct net_device *netdev = dev_get_drvdata(dev);
-	ibmveth_interrupt(netdev->irq, netdev);
+	struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
+	ibmveth_schedule_rx_queue(adapter, 0);
 	return 0;
 }
 
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 06/14] ibmveth: Refactor TX resource allocation in open/close paths

[Mingming Cao]

Same story as the RX refactor: pull TX LTB alloc/free out of open/close
into helpers and wire them in this patch.

  ibmveth_alloc_tx_resources()
  ibmveth_free_tx_resources()

They wrap the existing per-queue allocate_tx_ltb() / free_tx_ltb()
primitives. alloc_tx_resources() allocates every TX queue and unwinds
partial failure itself; free_tx_resources() walks real_num_tx_queues.
TX was already multi-queue capable via ethtool -L, so there is no MQ
RX behaviour change here.

Also tighten TX LTB lifetime: free_tx_ltb() is safe on an unallocated
slot and clears tx_ltb_dma[] after unmap; allocate_tx_ltb() clears
tx_ltb_dma[] after DMA-map failure.

Move TX LTB allocation to the end of open(), after LAN registration,
RX pools, RX interrupt setup, and the initial replenish kick. A late
alloc_tx_resources() failure jumps to out_cleanup_rx_interrupts and
must not call free_tx_resources() again: alloc already freed any
partial TX LTBs. That also retires a mid-open TX leak where the old
index-based while (--i) free could skip successfully allocated LTBs.

close() uses free_tx_resources() and cleanup_rx_interrupts().

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Introduce the TX resource helpers in the same patch that wires their
  first open/close callers.
- Do not free TX LTBs again after a failed alloc_tx_resources();
  harden free_tx_ltb() against unset slots.
- Move TX allocation after RX IRQ setup / replenish kick so open()
  failure unwind no longer depends on a shared loop index (also fixes
  a mid-open TX LTB leak).

 drivers/net/ethernet/ibm/ibmveth.c | 90 ++++++++++++++++++++++--------
 1 file changed, 66 insertions(+), 24 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 664169c4d27a..131c1a61d9e7 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1077,8 +1077,14 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
 
 static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
 {
-	dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
-			 adapter->tx_ltb_size, DMA_TO_DEVICE);
+	if (!adapter->tx_ltb_ptr[idx])
+		return;
+
+	if (adapter->tx_ltb_dma[idx]) {
+		dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
+				 adapter->tx_ltb_size, DMA_TO_DEVICE);
+		adapter->tx_ltb_dma[idx] = 0;
+	}
 	kfree(adapter->tx_ltb_ptr[idx]);
 	adapter->tx_ltb_ptr[idx] = NULL;
 }
@@ -1101,12 +1107,54 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
 			   "unable to DMA map tx long term buffer\n");
 		kfree(adapter->tx_ltb_ptr[idx]);
 		adapter->tx_ltb_ptr[idx] = NULL;
+		adapter->tx_ltb_dma[idx] = 0;
 		return -ENOMEM;
 	}
 
 	return 0;
 }
 
+/**
+ * ibmveth_alloc_tx_resources - Allocate TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Allocates TX Long Term Buffers (LTBs) for all TX queues.
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
+{
+	struct net_device *netdev = adapter->netdev;
+	int i;
+
+	for (i = 0; i < netdev->real_num_tx_queues; i++) {
+		if (ibmveth_allocate_tx_ltb(adapter, i))
+			goto err_free_ltbs;
+	}
+
+	return 0;
+
+err_free_ltbs:
+	while (--i >= 0)
+		ibmveth_free_tx_ltb(adapter, i);
+	return -ENOMEM;
+}
+
+/**
+ * ibmveth_free_tx_resources - Free TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees TX Long Term Buffers (LTBs) for all TX queues.
+ */
+static void ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
+{
+	struct net_device *netdev = adapter->netdev;
+	int i;
+
+	for (i = 0; i < netdev->real_num_tx_queues; i++)
+		ibmveth_free_tx_ltb(adapter, i);
+}
+
 static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
         union ibmveth_buf_desc rxq_desc, u64 mac_address)
 {
@@ -1157,12 +1205,6 @@ static int ibmveth_open(struct net_device *netdev)
 	if (rc)
 		goto out_free_filter_list;
 
-	rc = -ENOMEM;
-	for (i = 0; i < netdev->real_num_tx_queues; i++) {
-		if (ibmveth_allocate_tx_ltb(adapter, i))
-			goto out_free_tx_ltb;
-	}
-
 	mac_address = ether_addr_to_u64(netdev->dev_addr);
 
 	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
@@ -1184,35 +1226,38 @@ static int ibmveth_open(struct net_device *netdev)
 				     rxq_desc.desc,
 				     mac_address);
 		rc = -ENONET;
-		goto out_free_tx_ltb;
+		goto out_free_queue_mem;
 	}
 
 	rc = ibmveth_alloc_buffer_pools(adapter);
 	if (rc)
-		goto out_free_tx_ltb;
+		goto out_unregister_lan;
 
 	rc = ibmveth_setup_rx_interrupts(adapter);
-	if (rc) {
-		do {
-			lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
-		} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-		goto out_free_buffer_pools;
-	}
+	if (rc)
+		goto out_unregister_lan;
 
 	netdev_dbg(netdev, "initial replenish cycle\n");
 	ibmveth_schedule_rx_queue(adapter, 0);
 
+	rc = ibmveth_alloc_tx_resources(adapter);
+	if (rc)
+		goto out_cleanup_rx_interrupts;
+
 	netif_tx_start_all_queues(netdev);
 
 	netdev_dbg(netdev, "open complete\n");
 
 	return 0;
 
-out_free_buffer_pools:
+out_cleanup_rx_interrupts:
+	ibmveth_cleanup_rx_interrupts(adapter);
+out_unregister_lan:
+	do {
+		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
+	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
 	ibmveth_free_buffer_pools(adapter);
-out_free_tx_ltb:
-	while (--i >= 0)
-		ibmveth_free_tx_ltb(adapter, i);
+out_free_queue_mem:
 	ibmveth_cleanup_rx_resources(adapter);
 out_free_filter_list:
 	ibmveth_free_filter_list(adapter);
@@ -1224,13 +1269,13 @@ static int ibmveth_close(struct net_device *netdev)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
 	long lpar_rc;
-	int i;
 
 	netdev_dbg(netdev, "close starting\n");
 
 	netif_tx_stop_all_queues(netdev);
 
 	/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */
+	ibmveth_free_tx_resources(adapter);
 	ibmveth_cleanup_rx_interrupts(adapter);
 
 	do {
@@ -1248,9 +1293,6 @@ static int ibmveth_close(struct net_device *netdev)
 	ibmveth_cleanup_rx_resources(adapter);
 	ibmveth_free_filter_list(adapter);
 
-	for (i = 0; i < netdev->real_num_tx_queues; i++)
-		ibmveth_free_tx_ltb(adapter, i);
-
 	netdev_dbg(netdev, "close complete\n");
 
 	return 0;
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 07/14] ibmveth: Add RX queue register/deregister helpers for MQ

[Mingming Cao]

MQ RX changes queue lifecycle from one adapter-level register/free pair
to a mixed model:

  - queue 0: registered via h_register_logical_lan*()
  - queues 1..N: registered via H_REG_LOGICAL_LAN_QUEUE

This patch extracts the queue-0 control-plane helpers used by open/close
today and wires them in the same commit:

  ibmveth_register_rx_queues()
  ibmveth_free_all_queues()

Also update ibmveth_register_logical_lan() so that when multi_queue is
later enabled, queue 0 uses h_register_logical_lan_with_handle() and
stores queue_handle[0]. Runtime remains single-queue: multi_queue is
still false, so queue 0 keeps the legacy h_register_logical_lan() flow
and no subordinate queue is registered. Subordinate
register/deregister helpers arrive with the MQ enablement patch that
first calls them.

Introduce adapter->hcall_stats here for register/free path accounting
(first use).

Open/close unwind:
  - failures after successful LAN registration enter
    out_unregister_queues, then fall through to out_free_buffer_pools,
    so RX pools are not leaked;
  - free_all_queues() runs before free_buffer_pools() on open failure
    and close, so PHYP cannot retain a registered LAN while its
    DMA-backed RX pools are released.

Failures before registration go directly to out_free_buffer_pools.
RX interrupt masking before napi_disable is already handled by
cleanup_rx_interrupts() from the IRQ-helper patch.

With the helper patches through this one, open/close follow the
MQ-ready bring-up/teardown pipeline (still single-queue until MQ
enablement):

ibmveth_open():

  1. ibmveth_alloc_filter_list()
  2. ibmveth_alloc_rx_queues()       - buffer lists + RX rings
  3. ibmveth_alloc_buffer_pools()   - guest RX memory before PHYP
  4. ibmveth_register_rx_queues()   - PHYP registration (no IRQ enable)
  5. netif_set_real_num_rx_queues()
  6. ibmveth_setup_rx_interrupts()  - request_irq + napi_enable
  7. initial replenish kick         - schedule_rx_queue(0)
  8. ibmveth_alloc_tx_resources()

ibmveth_close():

  1. netif_tx_stop_all_queues()
  2. ibmveth_free_tx_resources()
  3. ibmveth_cleanup_rx_interrupts() - mask PHYP, napi_disable, free_irq
  4. ibmveth_free_all_queues()       - H_FREE_LOGICAL_LAN
  5. ibmveth_free_buffer_pools()
  6. ibmveth_cleanup_rx_resources()
  7. ibmveth_free_filter_list()

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Introduce register/free helpers in the same patch that wires their
  first open/close callers; keep subordinate-only helpers deferred
  until MQ enablement.
- Introduce adapter->hcall_stats here (first use); not in patch 2.
- Correct open/close unwind so free_all_queues() precedes
  free_buffer_pools().
- Drop the orphaned big-bang "open/close pipeline" patch from v3; that
  wiring is incremental across helper patches 3-7 instead.

 drivers/net/ethernet/ibm/ibmveth.c | 170 ++++++++++++++++++++---------
 drivers/net/ethernet/ibm/ibmveth.h |  14 +++
 2 files changed, 133 insertions(+), 51 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 131c1a61d9e7..9d1242dbcb19 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -434,6 +434,9 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
 	struct net_device *netdev = adapter->netdev;
 	int i, rc;
 
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		napi_enable(&adapter->napi[i]);
+
 	for (i = 0; i < adapter->num_rx_queues; i++) {
 		if (!adapter->queue_irq[i]) {
 			netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
@@ -451,14 +454,13 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
 		}
 	}
 
-	for (i = 0; i < adapter->num_rx_queues; i++)
-		napi_enable(&adapter->napi[i]);
-
 	return 0;
 
 err_free_irqs:
 	while (--i >= 0)
 		free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		napi_disable(&adapter->napi[i]);
 	return rc;
 }
 
@@ -1156,9 +1158,14 @@ static void ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
 }
 
 static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
-        union ibmveth_buf_desc rxq_desc, u64 mac_address)
+					union ibmveth_buf_desc rxq_desc,
+					u64 mac_address)
 {
 	int rc, try_again = 1;
+	unsigned long ua = adapter->vdev->unit_address;
+	unsigned long buf_dma = adapter->buffer_list_dma[0];
+	unsigned long filter_dma = adapter->filter_list_dma;
+	u64 *qh0 = &adapter->queue_handle[0];
 
 	/*
 	 * After a kexec the adapter will still be open, so our attempt to
@@ -1166,13 +1173,25 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
 	 * try again, but only once.
 	 */
 retry:
-	rc = h_register_logical_lan(adapter->vdev->unit_address,
-				    adapter->buffer_list_dma[0], rxq_desc.desc,
-				    adapter->filter_list_dma, mac_address);
+	/* In multi-queue mode, obtain a queue handle for queue 0 so all RX
+	 * queues can use the same per-queue buffer hypercalls.
+	 */
+	if (adapter->multi_queue) {
+		rc = h_register_logical_lan_with_handle(ua, buf_dma,
+							rxq_desc.desc,
+							filter_dma,
+							mac_address,
+							qh0);
+	} else {
+		rc = h_register_logical_lan(ua, buf_dma, rxq_desc.desc,
+					    filter_dma, mac_address);
+	}
+	adapter->hcall_stats.reg_lan++;
 
 	if (rc != H_SUCCESS && try_again) {
 		do {
 			rc = h_free_logical_lan(adapter->vdev->unit_address);
+			adapter->hcall_stats.free_lan++;
 		} while (H_IS_LONG_BUSY(rc) || (rc == H_BUSY));
 
 		try_again = 0;
@@ -1182,14 +1201,89 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
 	return rc;
 }
 
+/**
+ * ibmveth_free_all_queues - Free all RX queues at once
+ * @adapter: ibmveth adapter structure
+ *
+ * Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall.
+ * Used during interface close and registration error cleanup.
+ *
+ * Clears queue handles only; queue_irq[] is released by
+ * ibmveth_cleanup_rx_interrupts().
+ */
+static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
+{
+	unsigned long lpar_rc;
+	int i;
+
+	netdev_dbg(adapter->netdev, "freeing all RX queues at once\n");
+
+	do {
+		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
+		adapter->hcall_stats.free_lan++;
+	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+	if (lpar_rc != H_SUCCESS) {
+		netdev_err(adapter->netdev,
+			   "h_free_logical_lan failed: %ld\n", lpar_rc);
+	}
+
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		adapter->queue_handle[i] = 0;
+}
+
+/**
+ * ibmveth_register_rx_queues - Register RX queues with hypervisor
+ * @adapter: ibmveth adapter structure
+ * @mac_address: MAC address for device registration
+ *
+ * Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue
+ * registration is added when multi-queue RX is enabled.
+ *
+ * Return: 0 on success, -ENONET if queue 0 registration fails
+ */
+static int
+ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
+{
+	struct net_device *netdev = adapter->netdev;
+	union ibmveth_buf_desc rxq_desc;
+	unsigned long lpar_rc;
+	int rc;
+
+	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+				    adapter->rx_queue[0].queue_len;
+	rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
+	adapter->queue_irq[0] = netdev->irq;
+
+	rc = ibmveth_disable_irq(adapter, 0);
+	if (rc != H_SUCCESS)
+		netdev_dbg(netdev,
+			   "Failed to disable IRQ for queue 0 before registration, rc=%d\n",
+			   rc);
+
+	lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
+	if (lpar_rc != H_SUCCESS) {
+		netdev_err(netdev,
+			   "h_register_logical_lan failed: %ld\n", lpar_rc);
+		netdev_err(netdev,
+			   "buffer TCE:0x%llx filter TCE:0x%llx rxq desc:0x%llx MAC:0x%llx\n",
+			   adapter->buffer_list_dma[0],
+			   adapter->filter_list_dma,
+			   rxq_desc.desc, mac_address);
+		return -ENONET;
+	}
+
+	netdev_dbg(netdev,
+		   "registered 1 RX queue with hypervisor (single-queue mode)\n");
+	return 0;
+}
+
 static int ibmveth_open(struct net_device *netdev)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
-	u64 mac_address;
+	u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
 	int rxq_entries = 1;
-	unsigned long lpar_rc;
 	int rc;
-	union ibmveth_buf_desc rxq_desc;
 	int i;
 
 	netdev_dbg(netdev, "open starting\n");
@@ -1205,37 +1299,23 @@ static int ibmveth_open(struct net_device *netdev)
 	if (rc)
 		goto out_free_filter_list;
 
-	mac_address = ether_addr_to_u64(netdev->dev_addr);
-
-	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
-					adapter->rx_queue[0].queue_len;
-	rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
-
-	adapter->queue_irq[0] = netdev->irq;
-	ibmveth_disable_irq(adapter, 0);
-
-	lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
-
-	if (lpar_rc != H_SUCCESS) {
-		netdev_err(netdev, "h_register_logical_lan failed with %ld\n",
-			   lpar_rc);
-		netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
-			   "desc:0x%llx MAC:0x%llx\n",
-				     adapter->buffer_list_dma[0],
-				     adapter->filter_list_dma,
-				     rxq_desc.desc,
-				     mac_address);
-		rc = -ENONET;
+	rc = ibmveth_alloc_buffer_pools(adapter);
+	if (rc)
 		goto out_free_queue_mem;
-	}
 
-	rc = ibmveth_alloc_buffer_pools(adapter);
+	rc = ibmveth_register_rx_queues(adapter, mac_address);
 	if (rc)
-		goto out_unregister_lan;
+		goto out_free_buffer_pools;
+
+	rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
+	if (rc) {
+		netdev_err(netdev, "failed to set number of rx queues\n");
+		goto out_unregister_queues;
+	}
 
 	rc = ibmveth_setup_rx_interrupts(adapter);
 	if (rc)
-		goto out_unregister_lan;
+		goto out_unregister_queues;
 
 	netdev_dbg(netdev, "initial replenish cycle\n");
 	ibmveth_schedule_rx_queue(adapter, 0);
@@ -1252,10 +1332,9 @@ static int ibmveth_open(struct net_device *netdev)
 
 out_cleanup_rx_interrupts:
 	ibmveth_cleanup_rx_interrupts(adapter);
-out_unregister_lan:
-	do {
-		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
-	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+out_unregister_queues:
+	ibmveth_free_all_queues(adapter);
+out_free_buffer_pools:
 	ibmveth_free_buffer_pools(adapter);
 out_free_queue_mem:
 	ibmveth_cleanup_rx_resources(adapter);
@@ -1268,7 +1347,6 @@ static int ibmveth_open(struct net_device *netdev)
 static int ibmveth_close(struct net_device *netdev)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
-	long lpar_rc;
 
 	netdev_dbg(netdev, "close starting\n");
 
@@ -1277,18 +1355,8 @@ static int ibmveth_close(struct net_device *netdev)
 	/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */
 	ibmveth_free_tx_resources(adapter);
 	ibmveth_cleanup_rx_interrupts(adapter);
-
-	do {
-		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
-	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
-	if (lpar_rc != H_SUCCESS) {
-		netdev_err(netdev, "h_free_logical_lan failed with %lx, "
-			   "continuing with close\n", lpar_rc);
-	}
-
 	ibmveth_update_rx_no_buffer(adapter);
-
+	ibmveth_free_all_queues(adapter);
 	ibmveth_free_buffer_pools(adapter);
 	ibmveth_cleanup_rx_resources(adapter);
 	ibmveth_free_filter_list(adapter);
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index abcd4054c67e..2cfabddeee3d 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -269,6 +269,17 @@ static int pool_active[] = { 1, 1, 0, 0, 0};
 
 #define IBM_VETH_INVALID_MAP ((u16)0xffff)
 
+struct ibmveth_hcall_stats {
+	u64 reg_lan_queue;	/* H_REG_LOGICAL_LAN_QUEUE */
+	u64 reg_lan;		/* H_REGISTER_LOGICAL_LAN */
+	u64 add_bufs_queue;	/* H_ADD_LOGICAL_LAN_BUFFERS_QUEUE */
+	u64 add_bufs;		/* H_ADD_LOGICAL_LAN_BUFFERS */
+	u64 add_buf;		/* H_ADD_LOGICAL_LAN_BUFFER */
+	u64 free_lan_queue;	/* H_FREE_LOGICAL_LAN_QUEUE */
+	u64 free_lan;		/* H_FREE_LOGICAL_LAN */
+	u64 send_lan;		/* H_SEND_LOGICAL_LAN */
+};
+
 struct ibmveth_buff_pool {
     u32 size;
     u32 index;
@@ -333,6 +344,9 @@ struct ibmveth_adapter {
 	u64 tx_large_packets;
 	u64 rx_large_packets;
 
+	/* Multi-queue statistics */
+	struct ibmveth_hcall_stats hcall_stats;
+
 	/* Ethtool settings */
 	u8 duplex;
 	u32 speed;
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 08/14] ibmveth: Add queue-aware RX buffer submit helper for MQ

[Mingming Cao]

Replenish is the last open-path hypervisor call that still needs
per-queue awareness before MQ is turned on. Today
ibmveth_replenish_buffer_pool() calls h_add_logical_lan_buffer() or
h_add_logical_lan_buffers() directly; MQ posts via
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[].

Add ibmveth_add_logical_lan_buffers() to pick the hcall:
multi_queue uses h_add_logical_lan_buffers_queue() (up to 12 buffers,
IOBAs packed with odd counts in the upper 32 bits); legacy uses the
existing single- and multi-buffer hcalls. Count add_buf/add_bufs/
add_bufs_queue in hcall_stats.

Thread queue_index through the RX helpers used by poll and replenish:

  ibmveth_rxq_* accessors
  ibmveth_remove_buffer_from_pool()
  ibmveth_rxq_get_buffer() / ibmveth_rxq_harvest_buffer()
  ibmveth_replenish_buffer_pool() / ibmveth_replenish_task()

and update open/poll/netpoll callers in the same patch so arity stays
consistent. Also add per-queue replenish_lock so later concurrent
NAPI/resize paths can serialize buffer posting.

Until MQ enablement, callers still pass queue 0 and legacy hcalls
remain the live path.

H_FUNCTION handling is split:
  - multi_queue: hard-fail and break
  - legacy multi-buffer LPM fallback: set rx_buffers_per_hcall = 1 and
    break so the next replenish_buffer_pool() re-samples batch as 1
    (do not continue with a stale local batch)

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Introduce queue-aware replenish/poll helpers with their first callers
  in the same patch; do not leave a 2-arg replenish call ahead of the
  signature change.
- Restore the pre-MQ LPM H_FUNCTION break instead of continue; do not
  loop forever on a stale local batch size.
- Fold per-queue replenish_lock into this patch.
- Update kdoc for MQ parameters on remove_buffer_from_pool /
  rxq_harvest_buffer.

 drivers/net/ethernet/ibm/ibmveth.c | 341 +++++++++++++++++++----------
 drivers/net/ethernet/ibm/ibmveth.h |   7 +-
 2 files changed, 237 insertions(+), 111 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 9d1242dbcb19..cb93659fc057 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -101,49 +101,58 @@ static struct ibmveth_stat ibmveth_stats[] = {
 };
 
 /* simple methods of getting data from the current rxq entry */
-static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
+static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter,
+				    int queue_index)
 {
-	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
 
 	return be32_to_cpu(rxq->queue_addr[rxq->index].flags_off);
 }
 
-static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter,
+				     int queue_index)
 {
-	return (ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_TOGGLE) >>
-			IBMVETH_RXQ_TOGGLE_SHIFT;
+	return (ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_TOGGLE) >>
+		IBMVETH_RXQ_TOGGLE_SHIFT;
 }
 
-static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter,
+					     int queue_index)
 {
-	return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
+	return ibmveth_rxq_toggle(adapter, queue_index) ==
+		adapter->rx_queue[queue_index].toggle;
 }
 
-static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter,
+					   int queue_index)
 {
-	return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_VALID;
+	return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_VALID;
 }
 
-static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter,
+					   int queue_index)
 {
-	return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_OFF_MASK;
+	return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_OFF_MASK;
 }
 
-static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter,
+					   int queue_index)
 {
-	return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_LRG_PKT;
+	return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_LRG_PKT;
 }
 
-static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter,
+					   int queue_index)
 {
-	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
 
 	return be32_to_cpu(rxq->queue_addr[rxq->index].length);
 }
 
-static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
+					int queue_index)
 {
-	return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_CSUM_GOOD;
+	return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_CSUM_GOOD;
 }
 
 static unsigned int ibmveth_real_max_tx_queues(void)
@@ -260,6 +269,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
 		adapter->rx_queue[i].index = 0;
 		adapter->rx_queue[i].num_slots = rxq_entries;
 		adapter->rx_queue[i].toggle = 1;
+		spin_lock_init(&adapter->rx_queue[i].replenish_lock);
 
 		netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
 			   i, adapter->buffer_list_addr[i],
@@ -588,11 +598,75 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length)
 		asm("dcbf %0,%1,1" :: "b" (addr), "r" (offset));
 }
 
+/**
+ * ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor
+ * @adapter: ibmveth adapter structure
+ * @descs: array of buffer descriptors to add
+ * @filled: number of valid descriptors in the array
+ * @buff_size: size of each buffer (multi-queue mode only)
+ * @queue_index: RX queue index
+ *
+ * Return: hypervisor return code
+ */
+static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
+					    union ibmveth_buf_desc *descs,
+					    int filled,
+					    unsigned long buff_size,
+					    int queue_index)
+{
+	struct vio_dev *vdev = adapter->vdev;
+	unsigned long rc;
+
+	if (adapter->multi_queue) {
+		unsigned long buffersznum = (buff_size << 32) | filled;
+		unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL / 2] = {0};
+		unsigned long handle = adapter->queue_handle[queue_index];
+		int i;
+
+		/* Pack descriptor addresses into ioba pairs.
+		 * Each ioba holds two 32-bit addresses packed into 64 bits:
+		 * - Even descriptors (0,2,4...) go in high 32 bits
+		 * - Odd descriptors (1,3,5...) go in low 32 bits
+		 */
+		for (i = 0; i < filled && i < IBMVETH_MAX_RX_PER_HCALL; i++) {
+			int pair_idx = i / 2;
+			int is_high = (i % 2 == 0);
+
+			if (is_high)
+				ioba[pair_idx] = (unsigned long)
+					descs[i].fields.address << 32;
+			else
+				ioba[pair_idx] |= descs[i].fields.address;
+		}
+
+		rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
+						     handle,
+						     buffersznum,
+						     ioba[0], ioba[1], ioba[2],
+						     ioba[3], ioba[4], ioba[5]);
+		adapter->hcall_stats.add_bufs_queue++;
+	} else if (filled == 1) {
+		rc = h_add_logical_lan_buffer(vdev->unit_address,
+					      descs[0].desc);
+		adapter->hcall_stats.add_buf++;
+	} else {
+		rc = h_add_logical_lan_buffers(vdev->unit_address,
+					       descs[0].desc, descs[1].desc,
+					       descs[2].desc, descs[3].desc,
+					       descs[4].desc, descs[5].desc,
+					       descs[6].desc, descs[7].desc);
+		adapter->hcall_stats.add_bufs++;
+	}
+
+	return rc;
+}
+
 /* replenish the buffers for a pool.  note that we don't need to
  * skb_reserve these since they are used for incoming...
  */
 static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
-					  struct ibmveth_buff_pool *pool)
+					  struct ibmveth_buff_pool *pool,
+					  int queue_index)
 {
 	union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL] = {0};
 	u32 remaining = pool->size - atomic_read(&pool->available);
@@ -678,24 +752,16 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
 		if (!filled)
 			break;
 
-		/* single buffer case*/
-		if (filled == 1)
-			lpar_rc = h_add_logical_lan_buffer(vdev->unit_address,
-							   descs[0].desc);
-		else
-			/* Multi-buffer hcall */
-			lpar_rc = h_add_logical_lan_buffers(vdev->unit_address,
-							    descs[0].desc,
-							    descs[1].desc,
-							    descs[2].desc,
-							    descs[3].desc,
-							    descs[4].desc,
-							    descs[5].desc,
-							    descs[6].desc,
-							    descs[7].desc);
+		lpar_rc = ibmveth_add_logical_lan_buffers(adapter, descs,
+							  filled,
+							  pool->buff_size,
+							  queue_index);
+
 		if (lpar_rc != H_SUCCESS) {
 			dev_warn_ratelimited(dev,
-					     "RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n",
+					     "RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n",
+					     adapter->multi_queue ?
+					     "_queue" : "",
 					     filled, lpar_rc, batch);
 			goto hcall_failure;
 		}
@@ -736,24 +802,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
 		}
 		adapter->replenish_add_buff_failure += filled;
 
-		/*
-		 * If multi rx buffers hcall is no longer supported by FW
-		 * e.g. in the case of Live Partition Migration
-		 */
-		if (batch > 1 && lpar_rc == H_FUNCTION) {
-			/*
-			 * Instead of retry submit single buffer individually
-			 * here just set the max rx buffer per hcall to 1
-			 * buffers will be respleshed next time
-			 * when ibmveth_replenish_buffer_pool() is called again
-			 * with single-buffer case
-			 */
-			netdev_info(adapter->netdev,
-				    "RX Multi buffers not supported by FW, rc=%lu\n",
-				    lpar_rc);
-			adapter->rx_buffers_per_hcall = 1;
-			netdev_info(adapter->netdev,
-				    "Next rx replesh will fall back to single-buffer hcall\n");
+		if (lpar_rc == H_FUNCTION) {
+			if (adapter->multi_queue) {
+				netdev_err(adapter->netdev,
+					   "MQ buffer add H_FUNCTION (q=%d, batch=%d)\n",
+					   queue_index, batch);
+			} else if (batch > 1) {
+				/*
+				 * Live Partition Migration may drop multi-
+				 * buffer support. Fall back to single-buffer
+				 * on the next replenish; do not continue with
+				 * a stale local batch size (infinite loop).
+				 */
+				netdev_warn(adapter->netdev,
+					    "Legacy batch add H_FUNCTION (batch=%d), fallback\n",
+					    batch);
+				adapter->rx_buffers_per_hcall = 1;
+			}
 		}
 		break;
 	}
@@ -775,21 +840,33 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
 }
 
 /* replenish routine */
-static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
+static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
+				   int queue_index)
 {
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+	unsigned long flags;
 	int i;
 
+	if (queue_index >= adapter->num_rx_queues)
+		return;
+
 	adapter->replenish_task_cycles++;
 
+	spin_lock_irqsave(&rxq->replenish_lock, flags);
+
 	for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
-		struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
+		struct ibmveth_buff_pool *pool =
+			&adapter->rx_buff_pool[queue_index][i];
 
 		if (pool->active &&
 		    (atomic_read(&pool->available) < pool->threshold))
-			ibmveth_replenish_buffer_pool(adapter, pool);
+			ibmveth_replenish_buffer_pool(adapter, pool,
+						      queue_index);
 	}
 
 	ibmveth_update_rx_no_buffer(adapter);
+
+	spin_unlock_irqrestore(&rxq->replenish_lock, flags);
 }
 
 /* empty and free ana buffer pool - also used to do cleanup in error paths */
@@ -969,6 +1046,7 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
  * ibmveth_remove_buffer_from_pool - remove a buffer from a pool
  * @adapter: adapter instance
  * @correlator: identifies pool and index
+ * @queue_index: RX queue index (0..num_rx_queues-1)
  * @reuse: whether to reuse buffer
  *
  * Return:
@@ -977,7 +1055,8 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
  * * %-EFAULT - pool and index map to null skb
  */
 static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
-					   u64 correlator, bool reuse)
+					   u64 correlator, int queue_index,
+					   bool reuse)
 {
 	unsigned int pool  = correlator >> 32;
 	unsigned int index = correlator & 0xffffffffUL;
@@ -985,12 +1064,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
 	struct sk_buff *skb;
 
 	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
-	    WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+	    WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
 		schedule_work(&adapter->work);
 		return -EINVAL;
 	}
 
-	skb = adapter->rx_buff_pool[0][pool].skbuff[index];
+	skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
 	if (WARN_ON(!skb)) {
 		schedule_work(&adapter->work);
 		return -EFAULT;
@@ -1004,50 +1083,55 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
 		/* remove the skb pointer to mark free. actual freeing is done
 		 * by upper level networking after gro_receive
 		 */
-		adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
+		struct ibmveth_buff_pool *bpool =
+			&adapter->rx_buff_pool[queue_index][pool];
+
+		bpool->skbuff[index] = NULL;
 
 		dma_unmap_single(&adapter->vdev->dev,
-				 adapter->rx_buff_pool[0][pool].dma_addr[index],
-				 adapter->rx_buff_pool[0][pool].buff_size,
+				 bpool->dma_addr[index],
+				 bpool->buff_size,
 				 DMA_FROM_DEVICE);
 	}
 
-	free_index = adapter->rx_buff_pool[0][pool].producer_index;
-	adapter->rx_buff_pool[0][pool].producer_index++;
-	if (adapter->rx_buff_pool[0][pool].producer_index >=
-	    adapter->rx_buff_pool[0][pool].size)
-		adapter->rx_buff_pool[0][pool].producer_index = 0;
-	adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
+	free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;
+	adapter->rx_buff_pool[queue_index][pool].producer_index++;
+	if (adapter->rx_buff_pool[queue_index][pool].producer_index >=
+	    adapter->rx_buff_pool[queue_index][pool].size)
+		adapter->rx_buff_pool[queue_index][pool].producer_index = 0;
+	adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;
 
 	mb();
 
-	atomic_dec(&adapter->rx_buff_pool[0][pool].available);
+	atomic_dec(&adapter->rx_buff_pool[queue_index][pool].available);
 
 	return 0;
 }
 
 /* get the current buffer on the rx queue */
 static inline struct sk_buff *
-ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
+ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
+		       int queue_index)
 {
-	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
 	u64 correlator = rxq->queue_addr[rxq->index].correlator;
 	unsigned int pool = correlator >> 32;
 	unsigned int index = correlator & 0xffffffffUL;
 
 	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
-	    WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+	    WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
 		schedule_work(&adapter->work);
 		return NULL;
 	}
 
-	return adapter->rx_buff_pool[0][pool].skbuff[index];
+	return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
 }
 
 /**
  * ibmveth_rxq_harvest_buffer - Harvest buffer from pool
  *
  * @adapter: pointer to adapter
+ * @queue_index: RX queue index to harvest from
  * @reuse:   whether to reuse buffer
  *
  * Context: called from ibmveth_poll
@@ -1057,21 +1141,20 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
  * * other - non-zero return from ibmveth_remove_buffer_from_pool
  */
 static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
-				      bool reuse)
+				      int queue_index, bool reuse)
 {
+	struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
 	u64 cor;
 	int rc;
 
-	struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
-
 	cor = rxq->queue_addr[rxq->index].correlator;
-	rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
+	rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
 	if (unlikely(rc))
 		return rc;
 
-	if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
-		adapter->rx_queue[0].index = 0;
-		adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
+	if (++rxq->index == rxq->num_slots) {
+		rxq->index = 0;
+		rxq->toggle = !rxq->toggle;
 	}
 
 	return 0;
@@ -1993,34 +2076,45 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
 
 static int ibmveth_poll(struct napi_struct *napi, int budget)
 {
-	struct ibmveth_adapter *adapter =
-			container_of(napi, struct ibmveth_adapter, napi[0]);
-	struct net_device *netdev = adapter->netdev;
+	struct net_device *netdev = napi->dev;
+	struct ibmveth_adapter *adapter = netdev_priv(netdev);
 	int frames_processed = 0;
 	unsigned long lpar_rc;
+	int queue_index, rc;
 	u16 mss = 0;
 
+	queue_index = napi - adapter->napi;
+
+	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
+		return 0;
+
 restart_poll:
 	while (frames_processed < budget) {
-		if (!ibmveth_rxq_pending_buffer(adapter))
+		if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
 			break;
 
 		smp_rmb();
-		if (!ibmveth_rxq_buffer_valid(adapter)) {
+		if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
 			wmb(); /* suggested by larson1 */
 			adapter->rx_invalid_buffer++;
 			netdev_dbg(netdev, "recycling invalid buffer\n");
-			if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+			rc = ibmveth_rxq_harvest_buffer(adapter,
+							queue_index, true);
+			if (unlikely(rc))
 				break;
 		} else {
 			struct sk_buff *skb, *new_skb;
-			int length = ibmveth_rxq_frame_length(adapter);
-			int offset = ibmveth_rxq_frame_offset(adapter);
-			int csum_good = ibmveth_rxq_csum_good(adapter);
-			int lrg_pkt = ibmveth_rxq_large_packet(adapter);
+			int length = ibmveth_rxq_frame_length(adapter,
+							      queue_index);
+			int offset = ibmveth_rxq_frame_offset(adapter,
+							      queue_index);
+			int csum_good = ibmveth_rxq_csum_good(adapter,
+							      queue_index);
+			int lrg_pkt = ibmveth_rxq_large_packet(adapter,
+							       queue_index);
 			__sum16 iph_check = 0;
 
-			skb = ibmveth_rxq_get_buffer(adapter);
+			skb = ibmveth_rxq_get_buffer(adapter, queue_index);
 			if (unlikely(!skb))
 				break;
 
@@ -2045,12 +2139,18 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 							length);
 				if (rx_flush)
 					ibmveth_flush_buffer(skb->data,
-						length + offset);
-				if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+							     length + offset);
+				rc = ibmveth_rxq_harvest_buffer(adapter,
+								queue_index,
+								true);
+				if (unlikely(rc))
 					break;
 				skb = new_skb;
 			} else {
-				if (unlikely(ibmveth_rxq_harvest_buffer(adapter, false)))
+				rc = ibmveth_rxq_harvest_buffer(adapter,
+								queue_index,
+								false);
+				if (unlikely(rc))
 					break;
 				skb_reserve(skb, offset);
 			}
@@ -2086,7 +2186,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 		}
 	}
 
-	ibmveth_replenish_task(adapter);
+	ibmveth_replenish_task(adapter, queue_index);
 
 	if (frames_processed == budget)
 		goto out;
@@ -2097,14 +2197,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 	/* We think we are done - reenable interrupts,
 	 * then check once more to make sure we are done.
 	 */
-	lpar_rc = ibmveth_enable_irq(adapter, 0);
-	if (WARN_ON(lpar_rc != H_SUCCESS)) {
+	lpar_rc = ibmveth_enable_irq(adapter, queue_index);
+	if (lpar_rc != H_SUCCESS) {
+		netdev_err(netdev,
+			   "Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n",
+			   queue_index, lpar_rc);
 		schedule_work(&adapter->work);
 		goto out;
 	}
 
-	if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
-		lpar_rc = ibmveth_disable_irq(adapter, 0);
+	if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+	    napi_schedule(napi)) {
+		lpar_rc = ibmveth_disable_irq(adapter, queue_index);
+		WARN_ON(lpar_rc != H_SUCCESS);
 		goto restart_poll;
 	}
 
@@ -2229,9 +2334,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
 static void ibmveth_poll_controller(struct net_device *dev)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(dev);
+	int i;
 
-	ibmveth_replenish_task(adapter);
-	ibmveth_schedule_rx_queue(adapter, 0);
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		ibmveth_replenish_task(adapter, i);
+
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		ibmveth_schedule_rx_queue(adapter, i);
 }
 #endif
 
@@ -2428,7 +2537,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 
 	if (ret == H_SUCCESS &&
 	    (ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
-		adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
+		adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
 		netdev_dbg(netdev,
 			   "RX Multi-buffer hcall supported by FW, batch set to %u\n",
 			    adapter->rx_buffers_per_hcall);
@@ -2775,17 +2884,29 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
 
 	correlator = ((u64)IBMVETH_NUM_BUFF_POOLS << 32) | 0;
-	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
-	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+	KUNIT_EXPECT_EQ(test, -EINVAL,
+			ibmveth_remove_buffer_from_pool(adapter,
+							correlator, 0, false));
+	KUNIT_EXPECT_EQ(test, -EINVAL,
+			ibmveth_remove_buffer_from_pool(adapter,
+							correlator, 0, true));
 
 	correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
-	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
-	KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+	KUNIT_EXPECT_EQ(test, -EINVAL,
+			ibmveth_remove_buffer_from_pool(adapter,
+							correlator, 0, false));
+	KUNIT_EXPECT_EQ(test, -EINVAL,
+			ibmveth_remove_buffer_from_pool(adapter,
+							correlator, 0, true));
 
 	correlator = (u64)0 | 0;
 	pool->skbuff[0] = NULL;
-	KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
-	KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+	KUNIT_EXPECT_EQ(test, -EFAULT,
+			ibmveth_remove_buffer_from_pool(adapter,
+							correlator, 0, false));
+	KUNIT_EXPECT_EQ(test, -EFAULT,
+			ibmveth_remove_buffer_from_pool(adapter,
+							correlator, 0, true));
 
 	flush_work(&adapter->work);
 }
@@ -2830,15 +2951,15 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
 
 	adapter->rx_queue[0].queue_addr[0].correlator =
 		(u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
-	KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+	KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
 
 	adapter->rx_queue[0].queue_addr[0].correlator =
 		(u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
-	KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+	KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
 
 	pool->skbuff[0] = skb;
 	adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
-	KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
+	KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter, 0));
 
 	flush_work(&adapter->work);
 }
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 2cfabddeee3d..f0b2d470d012 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -14,6 +14,8 @@
 #ifndef _IBMVETH_H
 #define _IBMVETH_H
 
+#include <linux/spinlock_types.h>
+
 /* constants for H_MULTICAST_CTRL */
 #define IbmVethMcastReceptionModifyBit     0x80000UL
 #define IbmVethMcastReceptionEnableBit     0x20000UL
@@ -260,7 +262,9 @@ static inline long h_illan_attributes(unsigned long unit_address,
 #define IBMVETH_DEFAULT_QUEUES 8U
 #define IBMVETH_MAX_RX_QUEUES 1U
 #define IBMVETH_DEFAULT_RX_QUEUES 1U
-#define IBMVETH_MAX_RX_PER_HCALL 8U
+#define IBMVETH_MAX_RX_REGULAR 8U
+#define IBMVETH_MAX_RX_QUEUE 12U
+#define IBMVETH_MAX_RX_PER_HCALL 12U
 
 static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
 static int pool_count[] = { 256, 512, 256, 256, 256 };
@@ -302,6 +306,7 @@ struct ibmveth_rx_q {
     dma_addr_t queue_dma;
     u32        queue_len;
     struct ibmveth_rx_q_entry *queue_addr;
+	spinlock_t	replenish_lock;	/* per-queue buffer replenish */
 };
 
 struct ibmveth_adapter {
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 09/14] ibmveth: Enable multi-queue RX receive path

[Mingming Cao]

This is where MQ actually receives packets. If firmware sets
IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT in H_ILLAN_ATTRIBUTES, probe sets
multi_queue and num_rx_queues to min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES), matching the existing TX default (cap 8).
Up to IBMVETH_MAX_RX_QUEUES (16) remains available via ethtool -L.
Otherwise we stay at one queue like today.

Enable the live multi-queue RX path using helpers introduced earlier:

  - Probe: MQ capability bit, multi_queue/num_rx_queues,
    IBMVETH_MAX_RX_QUEUES=16, alloc_etherdev_mqs RX count, NAPI for
    all queues, MQ vs regular rx_buffers_per_hcall
  - ibmveth_register_logical_lan_queue() /
    ibmveth_register_single_rx_queue() for subordinate queues via
    H_REG_LOGICAL_LAN_QUEUE (irq_create_mapping for subordinate virqs)
  - Wire subordinates into ibmveth_register_rx_queues()
  - ibmveth_dispose_subordinate_irq_mappings() on partial failure
  - setup_rx_interrupts(): after request_irq + napi_enable, PHYP
    enable_irq for all queues when multi_queue && num_rx_queues > 1
  - MQ open replenishes every RX queue before setup_rx_interrupts()
    unmasks PHYP; SQ keeps the classic setup-then-schedule kick
  - ibmveth_dispose_subordinate_irq_mappings() on setup failure
    (queue 0 uses netdev->irq and is never disposed)

Subordinate register error handling:
  - H_FUNCTION is a hard open failure: honest netdev_err, then the
    generic failure/params logs, without clearing multi_queue or
    claiming single-queue fallback
  - Register each subordinate once; no blind try_again retry
  - No caller-side "Invalid hypervisor return" log; the callee
    already reports the hcall rc

setup_rx_interrupts() failure paths dispose subordinate virq mappings
from both err_free_irqs and err_disable_napi so request_irq failure
after successful subordinate registration cannot leak Linux mappings.

Hot-path netdev->stats TX/RX accounting moves to the next patch
(per-queue qstats).

Legacy firmware without the MQ bit is unchanged.

On probe failure after pool kobjects were created, put them before
free_netdev() so earlier pools cannot leak.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Fold subordinate register helpers and their review fixes into the MQ
  enablement patch that first uses them.
- Prefer request_irq -> napi_enable -> PHYP enable on MQ open.
- Preserve open unwind so set_real_num_rx / IRQ failures free LAN
  before buffer pools.
- MQ open replenishes every queue before setup_rx_interrupts() unmasks
  PHYP (drop avoidance during open; PHYP only interrupts after a
  successful enqueue). SQ keeps classic setup-then-schedule kick.
- Dispose subordinate virq mappings on setup_rx_interrupts()
  request_irq failure (err_free_irqs), matching err_disable_napi.
- Open unwind: setup/cleanup own subordinate dispose; skip duplicate
  dispose on those paths.
- H_FUNCTION on subordinate register is a hard open failure (no blind
  retry / no fake single-queue fallback).
- Note: hot-path netdev->stats accounting moves to the next patch (qstats).
- Put already-created pool kobjects on probe kobject_init_and_add /
  set_real_num_tx_queues / register_netdev failure (bisect-safe).

 drivers/net/ethernet/ibm/ibmveth.c | 456 ++++++++++++++++++++++++-----
 drivers/net/ethernet/ibm/ibmveth.h |   3 +-
 2 files changed, 380 insertions(+), 79 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index cb93659fc057..4ad7ced3c608 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -30,6 +30,7 @@
 #include <linux/ip.h>
 #include <linux/ipv6.h>
 #include <linux/slab.h>
+#include <linux/spinlock.h>
 #include <asm/hvcall.h>
 #include <linux/atomic.h>
 #include <asm/vio.h>
@@ -45,7 +46,6 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev);
 
 static struct kobj_type ktype_veth_pool;
 
-
 static const char ibmveth_driver_name[] = "ibmveth";
 static const char ibmveth_driver_string[] = "IBM Power Virtual Ethernet Driver";
 #define ibmveth_driver_version "1.06"
@@ -97,7 +97,17 @@ static struct ibmveth_stat ibmveth_stats[] = {
 	{ "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) },
 	{ "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) },
 	{ "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) },
-	{ "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }
+	{ "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) },
+	{ "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
+	{ "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) },
+	{ "hcall_add_bufs_queue",
+	  IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) },
+	{ "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) },
+	{ "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) },
+	{ "hcall_free_lan_queue",
+	  IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) },
+	{ "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) },
+	{ "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) },
 };
 
 /* simple methods of getting data from the current rxq entry */
@@ -429,12 +439,64 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
 	return ibmveth_toggle_irq(adapter, queue_index, true);
 }
 
+/**
+ * ibmveth_dispose_subordinate_irq_mapping - Drop one subordinate virq mapping
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: RX queue index (1..N)
+ *
+ * Subordinate queues get mappings from irq_create_mapping() during PHYP
+ * registration. Queue 0 uses netdev->irq from device tree and is left alone.
+ *
+ * Bound against IBMVETH_MAX_RX_QUEUES, not num_rx_queues: scale-down and
+ * scale-up fail paths dispose queues that are no longer in the published
+ * live set but still own a virq in queue_irq[]. The bulk helper still
+ * iterates only 1..num_rx_queues-1 for close/open-fail cleanup.
+ *
+ * Linux virq lifetime is owned by interrupt cleanup helpers. Call this only
+ * after free_irq() when a handler was installed, or from registration failure
+ * cleanup before request_irq().
+ */
+static void
+ibmveth_dispose_subordinate_irq_mapping(struct ibmveth_adapter *adapter,
+					int queue_idx)
+{
+	if (queue_idx <= 0 || queue_idx >= IBMVETH_MAX_RX_QUEUES)
+		return;
+
+	if (adapter->queue_irq[queue_idx]) {
+		irq_dispose_mapping(adapter->queue_irq[queue_idx]);
+		adapter->queue_irq[queue_idx] = 0;
+	}
+}
+
+/**
+ * ibmveth_dispose_subordinate_irq_mappings - Drop virq mappings for queues 1..N
+ * @adapter: ibmveth adapter structure
+ *
+ * Bulk helper for paths that registered subordinate queues but never installed
+ * Linux IRQ handlers.
+ */
+static void
+ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
+{
+	int i;
+
+	for (i = 1; i < adapter->num_rx_queues; i++)
+		ibmveth_dispose_subordinate_irq_mapping(adapter, i);
+}
+
 /**
  * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
  * @adapter: ibmveth adapter structure
  *
- * Registers interrupt handlers for all RX queues and enables NAPI polling.
- * On error, cleans up any successfully registered IRQs before returning.
+ * Registers interrupt handlers for all RX queues, enables NAPI, then
+ * enables hypervisor interrupt delivery for multi-queue mode after
+ * every queue has a Linux handler installed. For multi-queue open the
+ * caller should replenish RX buffers before this helper so traffic
+ * during open is not dropped (PHYP only interrupts after a successful
+ * enqueue, which needs buffers). Single-queue open leaves PHYP masked
+ * here and kicks NAPI afterward (classic path: first poll posts then
+ * enables).
  *
  * Return: 0 on success, negative error code on failure
  */
@@ -442,12 +504,9 @@ static int
 ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
 {
 	struct net_device *netdev = adapter->netdev;
-	int i, rc;
+	int i, rc, num = adapter->num_rx_queues;
 
-	for (i = 0; i < adapter->num_rx_queues; i++)
-		napi_enable(&adapter->napi[i]);
-
-	for (i = 0; i < adapter->num_rx_queues; i++) {
+	for (i = 0; i < num; i++) {
 		if (!adapter->queue_irq[i]) {
 			netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
 			rc = -EINVAL;
@@ -464,13 +523,42 @@ ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
 		}
 	}
 
+	for (i = 0; i < num; i++)
+		napi_enable(&adapter->napi[i]);
+
+	if (adapter->multi_queue && num > 1) {
+		for (i = 0; i < num; i++) {
+			rc = ibmveth_enable_irq(adapter, i);
+			if (rc) {
+				netdev_err(netdev,
+					   "Failed to enable IRQ for queue %d, rc=%d\n",
+					   i, rc);
+				while (--i >= 0)
+					ibmveth_disable_irq(adapter, i);
+				rc = -EIO;
+				goto err_disable_napi;
+			}
+		}
+	}
+
 	return 0;
 
+err_disable_napi:
+	/* PHYP unmask was rolled back above; disable NAPI before free_irq */
+	for (i = 0; i < num; i++)
+		napi_disable(&adapter->napi[i]);
+	for (i = 0; i < num; i++) {
+		if (adapter->queue_irq[i])
+			free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+	}
+	goto err_dispose_mappings;
+
 err_free_irqs:
 	while (--i >= 0)
 		free_irq(adapter->queue_irq[i], &adapter->napi[i]);
-	for (i = 0; i < adapter->num_rx_queues; i++)
-		napi_disable(&adapter->napi[i]);
+err_dispose_mappings:
+	/* Both setup failure paths own subordinate virq disposal. */
+	ibmveth_dispose_subordinate_irq_mappings(adapter);
 	return rc;
 }
 
@@ -503,15 +591,7 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
 			free_irq(adapter->queue_irq[i], &adapter->napi[i]);
 	}
 
-	/* Dispose IRQ mappings for subordinate queues (1-15).
-	 * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
-	 */
-	for (i = 1; i < adapter->num_rx_queues; i++) {
-		if (adapter->queue_irq[i]) {
-			irq_dispose_mapping(adapter->queue_irq[i]);
-			adapter->queue_irq[i] = 0;
-		}
-	}
+	ibmveth_dispose_subordinate_irq_mappings(adapter);
 
 	/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */
 }
@@ -521,8 +601,8 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
  * @adapter: ibmveth adapter structure
  * @qindex: RX queue index
  *
- * Shared by the IRQ handler and process-context kick paths (open, resume,
- * pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper.
+ * Shared by the IRQ handler and process-context kick sites (open, resume,
+ * pool sysfs, poll_controller).
  */
 static void ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
 				      int qindex)
@@ -834,9 +914,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
  */
 static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
 {
-	__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
+	int i;
 
-	adapter->rx_no_buffer = be64_to_cpup(p);
+	adapter->rx_no_buffer = 0;
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
+		u64 drops = be64_to_cpup(p);
+
+		adapter->rx_no_buffer += drops;
+	}
 }
 
 /* replenish routine */
@@ -847,8 +933,12 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
 	unsigned long flags;
 	int i;
 
-	if (queue_index >= adapter->num_rx_queues)
+	if (queue_index >= adapter->num_rx_queues) {
+		netdev_dbg(adapter->netdev,
+			   "Skipping replenish for freed queue %d (num_queues=%d)\n",
+			   queue_index, adapter->num_rx_queues);
 		return;
+	}
 
 	adapter->replenish_task_cycles++;
 
@@ -858,7 +948,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
 		struct ibmveth_buff_pool *pool =
 			&adapter->rx_buff_pool[queue_index][i];
 
-		if (pool->active &&
+		if (pool->active && pool->free_map &&
 		    (atomic_read(&pool->available) < pool->threshold))
 			ibmveth_replenish_buffer_pool(adapter, pool,
 						      queue_index);
@@ -1284,6 +1374,137 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
 	return rc;
 }
 
+/**
+ * ibmveth_register_logical_lan_queue - Register subordinate queue with
+ * hypervisor
+ * @adapter: ibmveth adapter structure
+ * @rxq_desc: Receive queue descriptor
+ * @queue_index: RX queue index (1..N for subordinate queues)
+ *
+ * Registers a subordinate receive queue using H_REG_LOGICAL_LAN_QUEUE.
+ * On success, stores the queue handle and virtual IRQ in the adapter.
+ * If IRQ mapping fails after a successful hypervisor registration, the
+ * queue is freed before returning.
+ *
+ * Return: H_SUCCESS on success, negative errno on IRQ mapping failure,
+ *         hypervisor error code otherwise
+ */
+static int
+ibmveth_register_logical_lan_queue(struct ibmveth_adapter *adapter,
+				   union ibmveth_buf_desc rxq_desc,
+				   int queue_index)
+{
+	unsigned long handle, hwirq;
+	unsigned int virq;
+	long lpar_rc;
+
+	netdev_dbg(adapter->netdev,
+		   "Attempting to register queue %d: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+		   queue_index, adapter->vdev->unit_address,
+		   (unsigned long long)adapter->buffer_list_dma[queue_index],
+		   (unsigned long long)rxq_desc.desc);
+
+	lpar_rc = h_reg_logical_lan_queue(adapter->vdev->unit_address,
+					  adapter->buffer_list_dma[queue_index],
+					  rxq_desc.desc, &handle, &hwirq);
+	adapter->hcall_stats.reg_lan_queue++;
+
+	if (lpar_rc == H_SUCCESS) {
+		virq = irq_create_mapping(NULL, hwirq);
+		if (!virq) {
+			unsigned long free_rc;
+			unsigned long ua = adapter->vdev->unit_address;
+
+			netdev_err(adapter->netdev,
+				   "Failed to map IRQ for queue %d (hwirq=%lu)\n",
+				   queue_index, hwirq);
+			do {
+				free_rc = h_free_logical_lan_queue(ua, handle);
+			} while (H_IS_LONG_BUSY(free_rc) ||
+				  (free_rc == H_BUSY));
+			adapter->hcall_stats.free_lan_queue++;
+			if (free_rc != H_SUCCESS)
+				netdev_err(adapter->netdev,
+					   "h_free_logical_lan_queue failed for queue %d after IRQ map failure: rc=0x%lx\n",
+					   queue_index, free_rc);
+			return -EINVAL;
+		}
+
+		adapter->queue_handle[queue_index] = handle;
+		adapter->queue_irq[queue_index] = virq;
+
+		netdev_dbg(adapter->netdev,
+			   "queue %d registered: handle=0x%llx irq=%u\n",
+			   queue_index, adapter->queue_handle[queue_index],
+			   adapter->queue_irq[queue_index]);
+		return H_SUCCESS;
+	}
+
+	/*
+	 * H_FUNCTION means firmware rejected this subordinate register
+	 * (MQ unsupported). That is a hard open failure: do not clear
+	 * multi_queue or claim single-queue fallback. Keep a specific
+	 * log, then the generic failure lines below (no early return).
+	 */
+	if (lpar_rc == H_FUNCTION)
+		netdev_err(adapter->netdev,
+			   "h_reg_logical_lan_queue H_FUNCTION for queue %d (firmware MQ unsupported)\n",
+			   queue_index);
+
+	netdev_err(adapter->netdev,
+		   "h_reg_logical_lan_queue failed for queue %d with %ld\n",
+		   queue_index, lpar_rc);
+	netdev_err(adapter->netdev,
+		   "queue %d params: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+		   queue_index, adapter->vdev->unit_address,
+		   (unsigned long long)adapter->buffer_list_dma[queue_index],
+		   (unsigned long long)rxq_desc.desc);
+
+	return lpar_rc;
+}
+
+/**
+ * ibmveth_register_single_rx_queue - Register one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to register (1..N)
+ * @mac_address: MAC address (unused; reserved for API symmetry)
+ *
+ * Builds the queue descriptor and registers with the hypervisor via
+ * ibmveth_register_logical_lan_queue().
+ *
+ * Return: 0 on success, -EINVAL if @queue_idx is invalid, -EIO on failure
+ */
+static int
+ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
+				 int queue_idx, u64 mac_address)
+{
+	struct net_device *netdev = adapter->netdev;
+	union ibmveth_buf_desc rxq_desc;
+	long lpar_rc;
+
+	(void)mac_address;
+
+	if (WARN_ON(queue_idx < 1 || queue_idx >= IBMVETH_MAX_RX_QUEUES))
+		return -EINVAL;
+
+	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+				    adapter->rx_queue[queue_idx].queue_len;
+	rxq_desc.fields.address = adapter->rx_queue[queue_idx].queue_dma;
+
+	lpar_rc = ibmveth_register_logical_lan_queue(adapter, rxq_desc,
+						     queue_idx);
+	if (lpar_rc != H_SUCCESS) {
+		netdev_err(netdev, "Failed to register queue %d: rc=0x%lx\n",
+			   queue_idx, lpar_rc);
+		return -EIO;
+	}
+
+	netdev_dbg(netdev, "Registered queue %d with handle 0x%llx\n",
+		   queue_idx, adapter->queue_handle[queue_idx]);
+
+	return 0;
+}
+
 /**
  * ibmveth_free_all_queues - Free all RX queues at once
  * @adapter: ibmveth adapter structure
@@ -1292,7 +1513,8 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
  * Used during interface close and registration error cleanup.
  *
  * Clears queue handles only; queue_irq[] is released by
- * ibmveth_cleanup_rx_interrupts().
+ * ibmveth_cleanup_rx_interrupts() on close, or by
+ * ibmveth_dispose_subordinate_irq_mappings() on partial register failure.
  */
 static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
 {
@@ -1320,10 +1542,11 @@ static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
  * @adapter: ibmveth adapter structure
  * @mac_address: MAC address for device registration
  *
- * Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue
- * registration is added when multi-queue RX is enabled.
+ * Registers queue 0 via ibmveth_register_logical_lan(), then subordinate
+ * queues 1..N when multi-queue mode is enabled.
  *
- * Return: 0 on success, -ENONET if queue 0 registration fails
+ * Return: 0 on success, -ENONET if queue 0 registration fails, -EIO on
+ *         subordinate queue registration failure
  */
 static int
 ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
@@ -1331,7 +1554,7 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
 	struct net_device *netdev = adapter->netdev;
 	union ibmveth_buf_desc rxq_desc;
 	unsigned long lpar_rc;
-	int rc;
+	int i, rc;
 
 	rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
 				    adapter->rx_queue[0].queue_len;
@@ -1356,9 +1579,31 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
 		return -ENONET;
 	}
 
+	if (adapter->num_rx_queues == 1 || !adapter->multi_queue) {
+		netdev_dbg(netdev,
+			   "registered 1 RX queue with hypervisor (single-queue mode)\n");
+		return 0;
+	}
+
+	netdev_dbg(netdev, "Registering %d subordinate queues (1-%d)\n",
+		   adapter->num_rx_queues - 1, adapter->num_rx_queues - 1);
+
+	for (i = 1; i < adapter->num_rx_queues; i++) {
+		rc = ibmveth_register_single_rx_queue(adapter, i, mac_address);
+		if (rc)
+			goto err_unregister;
+	}
+
 	netdev_dbg(netdev,
-		   "registered 1 RX queue with hypervisor (single-queue mode)\n");
+		   "registered %d RX queues with hypervisor (multi-queue mode)\n",
+		   adapter->num_rx_queues);
+
 	return 0;
+
+err_unregister:
+	ibmveth_dispose_subordinate_irq_mappings(adapter);
+	ibmveth_free_all_queues(adapter);
+	return rc;
 }
 
 static int ibmveth_open(struct net_device *netdev)
@@ -1396,12 +1641,29 @@ static int ibmveth_open(struct net_device *netdev)
 		goto out_unregister_queues;
 	}
 
+	/*
+	 * MQ: post buffers before setup_rx_interrupts() unmasks PHYP
+	 * (avoids drops if traffic arrives during open; PHYP allows
+	 * either order). Single-queue keeps the classic kick: setup
+	 * (no unmask) then schedule_rx_queue() so the first poll
+	 * replenishes and enables.
+	 */
+	if (adapter->multi_queue && adapter->num_rx_queues > 1) {
+		for (i = 0; i < adapter->num_rx_queues; i++) {
+			netdev_dbg(netdev,
+				   "initial replenish cycle for queue %d\n", i);
+			ibmveth_replenish_task(adapter, i);
+		}
+	}
+
 	rc = ibmveth_setup_rx_interrupts(adapter);
 	if (rc)
-		goto out_unregister_queues;
+		goto out_free_all_queues; /* setup already disposed IRQs */
 
-	netdev_dbg(netdev, "initial replenish cycle\n");
-	ibmveth_schedule_rx_queue(adapter, 0);
+	if (!(adapter->multi_queue && adapter->num_rx_queues > 1)) {
+		netdev_dbg(netdev, "initial replenish cycle\n");
+		ibmveth_schedule_rx_queue(adapter, 0);
+	}
 
 	rc = ibmveth_alloc_tx_resources(adapter);
 	if (rc)
@@ -1415,7 +1677,10 @@ static int ibmveth_open(struct net_device *netdev)
 
 out_cleanup_rx_interrupts:
 	ibmveth_cleanup_rx_interrupts(adapter);
+	goto out_free_all_queues; /* cleanup already disposed IRQs */
 out_unregister_queues:
+	ibmveth_dispose_subordinate_irq_mappings(adapter);
+out_free_all_queues:
 	ibmveth_free_all_queues(adapter);
 out_free_buffer_pools:
 	ibmveth_free_buffer_pools(adapter);
@@ -1702,6 +1967,11 @@ static int ibmveth_set_features(struct net_device *dev,
 	return rc1 ? rc1 : rc2;
 }
 
+/*
+ * Sum per-queue counters for rare ethtool reads. Do not write adapter
+ * globals on the hot path (ibmvnic-style); with qstats allocated for the
+ * adapter lifetime, these sums remain meaningful across ifdown/up.
+ */
 static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
 {
 	int i;
@@ -1838,12 +2108,15 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,
 		return 1;
 	}
 
+	adapter->hcall_stats.send_lan++;
 	return 0;
 }
 
 static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
-					 struct net_device *netdev)
+					 struct ibmveth_adapter *adapter,
+					 int queue_num)
 {
+	struct net_device *netdev = adapter->netdev;
 	struct ethhdr *ether_header;
 	int ret = 0;
 
@@ -1851,7 +2124,6 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
 
 	if (ether_addr_equal(ether_header->h_dest, netdev->dev_addr)) {
 		netdev_dbg(netdev, "veth doesn't support loopback packets, dropping packet.\n");
-		netdev->stats.tx_dropped++;
 		ret = -EOPNOTSUPP;
 	}
 
@@ -1867,7 +2139,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	int i, queue_num = skb_get_queue_mapping(skb);
 	unsigned long mss = 0;
 
-	if (ibmveth_is_packet_unsupported(skb, netdev))
+	if (ibmveth_is_packet_unsupported(skb, adapter, queue_num))
 		goto out;
 	/* veth can't checksum offload UDP */
 	if (skb->ip_summed == CHECKSUM_PARTIAL &&
@@ -1878,7 +2150,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	    skb_checksum_help(skb)) {
 
 		netdev_err(netdev, "tx: failed to checksum packet\n");
-		netdev->stats.tx_dropped++;
 		goto out;
 	}
 
@@ -1901,7 +2172,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	if (skb->ip_summed == CHECKSUM_PARTIAL && skb_is_gso(skb)) {
 		if (adapter->fw_large_send_support) {
 			mss = (unsigned long)skb_shinfo(skb)->gso_size;
-			adapter->tx_large_packets++;
 		} else if (!skb_is_gso_v6(skb)) {
 			/* Put -1 in the IP checksum to tell phyp it
 			 * is a largesend packet. Put the mss in
@@ -1910,7 +2180,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 			ip_hdr(skb)->check = 0xffff;
 			tcp_hdr(skb)->check =
 				cpu_to_be16(skb_shinfo(skb)->gso_size);
-			adapter->tx_large_packets++;
 		}
 	}
 
@@ -1918,7 +2187,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	if (unlikely(skb->len > adapter->tx_ltb_size)) {
 		netdev_err(adapter->netdev, "tx: packet size (%u) exceeds ltb (%u)\n",
 			   skb->len, adapter->tx_ltb_size);
-		netdev->stats.tx_dropped++;
 		goto out;
 	}
 	memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
@@ -1935,7 +2203,6 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	if (unlikely(total_bytes != skb->len)) {
 		netdev_err(adapter->netdev, "tx: incorrect packet len copied into ltb (%u != %u)\n",
 			   skb->len, total_bytes);
-		netdev->stats.tx_dropped++;
 		goto out;
 	}
 	desc.fields.flags_len = desc_flags | skb->len;
@@ -1944,18 +2211,13 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	dma_wmb();
 
 	if (ibmveth_send(adapter, desc.desc, mss)) {
-		adapter->tx_send_failed++;
-		netdev->stats.tx_dropped++;
 	} else {
-		netdev->stats.tx_packets++;
-		netdev->stats.tx_bytes += skb->len;
 	}
 
 out:
 	dev_consume_skb_any(skb);
 	return NETDEV_TX_OK;
 
-
 }
 
 static void ibmveth_rx_mss_helper(struct sk_buff *skb, u16 mss, int lrg_pkt)
@@ -2180,8 +2442,6 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 
 			napi_gro_receive(napi, skb);	/* send it up */
 
-			netdev->stats.rx_packets++;
-			netdev->stats.rx_bytes += length;
 			frames_processed++;
 		}
 	}
@@ -2357,8 +2617,7 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
 	struct ibmveth_adapter *adapter;
 	struct iommu_table *tbl;
 	unsigned long ret;
-	int i;
-	int rxqentries = 1;
+	int i, q;
 
 	tbl = get_iommu_table_base(&vdev->dev);
 
@@ -2373,18 +2632,25 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
 	/* add size of mapped tx buffers */
 	ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
 
-	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
-		/* add the size of the active receive buffers */
-		if (adapter->rx_buff_pool[0][i].active)
-			ret +=
-			    adapter->rx_buff_pool[0][i].size *
-			    IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
-					     buff_size, tbl);
-		rxqentries += adapter->rx_buff_pool[0][i].size;
-	}
-	/* add the size of the receive queue entries */
-	ret += IOMMU_PAGE_ALIGN(
-		rxqentries * sizeof(struct ibmveth_rx_q_entry), tbl);
+	for (q = 0; q < adapter->num_rx_queues; q++) {
+		int rxqentries = 1;
+
+		for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+			/* add the size of the active receive buffers */
+			struct ibmveth_buff_pool *bpool =
+				&adapter->rx_buff_pool[q][i];
+
+			/* add the size of the active receive buffers */
+			if (bpool->active)
+				ret += bpool->size *
+					IOMMU_PAGE_ALIGN(bpool->buff_size, tbl);
+			rxqentries += bpool->size;
+		}
+
+		/* add the size of the receive queue entries */
+		ret += IOMMU_PAGE_ALIGN(rxqentries *
+					sizeof(struct ibmveth_rx_q_entry), tbl);
+	}
 
 	return ret;
 }
@@ -2449,9 +2715,18 @@ static const struct net_device_ops ibmveth_netdev_ops = {
 #endif
 };
 
+static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
+				   int pools_ready)
+{
+	int i;
+
+	for (i = 0; i < pools_ready; i++)
+		kobject_put(&adapter->rx_buff_pool[0][i].kobj);
+}
+
 static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 {
-	int rc, i, mac_len;
+	int rc, i, mac_len, pools_ready = 0;
 	struct net_device *netdev;
 	struct ibmveth_adapter *adapter;
 	unsigned char *mac_addr_p;
@@ -2486,7 +2761,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 		return -EINVAL;
 	}
 
-	netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter), IBMVETH_MAX_QUEUES, 1);
+	netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter),
+				    IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES);
 	if (!netdev)
 		return -ENOMEM;
 
@@ -2499,7 +2775,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 	adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
 	ibmveth_init_link_settings(netdev);
 
-	netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
+	for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+		netif_napi_add_weight(netdev, &adapter->napi[i],
+				      ibmveth_poll, 16);
+
 
 	netdev->irq = dev->irq;
 	netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2531,16 +2810,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 		netdev->features |= NETIF_F_FRAGLIST;
 	}
 
-	/* Initialize queue count - always 1 for now */
-	adapter->multi_queue = 0;
-	adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
+	if (ret == H_SUCCESS &&
+	    (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) {
+		adapter->multi_queue = 1;
+		adapter->num_rx_queues = min(num_online_cpus(),
+					     IBMVETH_DEFAULT_QUEUES);
+		netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n",
+			   adapter->num_rx_queues);
+	} else {
+		adapter->multi_queue = 0;
+		adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
+	}
 
 	if (ret == H_SUCCESS &&
 	    (ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
-		adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
+		if (adapter->multi_queue)
+			adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_QUEUE;
+		else
+			adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
 		netdev_dbg(netdev,
 			   "RX Multi-buffer hcall supported by FW, batch set to %u\n",
-			    adapter->rx_buffers_per_hcall);
+			   adapter->rx_buffers_per_hcall);
 	} else {
 		adapter->rx_buffers_per_hcall = 1;
 		netdev_dbg(netdev,
@@ -2558,15 +2848,24 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
 		struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
-		int error;
 
 		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
 					 pool_count[i], pool_size[i],
 					 pool_active[i]);
-		error = kobject_init_and_add(kobj, &ktype_veth_pool,
-					     &dev->dev.kobj, "pool%d", i);
-		if (!error)
-			kobject_uevent(kobj, KOBJ_ADD);
+		rc = kobject_init_and_add(kobj, &ktype_veth_pool,
+					  &dev->dev.kobj, "pool%d", i);
+		if (rc) {
+			dev_err(&dev->dev,
+				"failed to create pool%d kobject: %d\n", i, rc);
+			/* init_and_add takes a ref even on failure */
+			kobject_put(kobj);
+			ibmveth_put_pool_kobjs(adapter, pools_ready);
+			free_netdev(netdev);
+			return rc;
+		}
+
+		pools_ready++;
+		kobject_uevent(kobj, KOBJ_ADD);
 	}
 
 	rc = netif_set_real_num_tx_queues(netdev, min(num_online_cpus(),
@@ -2574,6 +2873,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 	if (rc) {
 		netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n",
 			   rc);
+		ibmveth_put_pool_kobjs(adapter, pools_ready);
 		free_netdev(netdev);
 		return rc;
 	}
@@ -2590,6 +2890,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 
 	if (rc) {
 		netdev_dbg(netdev, "failed to register netdev rc=%d\n", rc);
+		ibmveth_put_pool_kobjs(adapter, pools_ready);
 		free_netdev(netdev);
 		return rc;
 	}
@@ -2761,7 +3062,6 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
 	return rc;
 }
 
-
 #define ATTR(_name, _mode)				\
 	struct attribute veth_##_name##_attr = {	\
 	.name = __stringify(_name), .mode = _mode,	\
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index f0b2d470d012..6e1a964df42a 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -30,6 +30,7 @@
 #define IbmVethMcastRemoveFilter     0x2UL
 #define IbmVethMcastClearFilterTable 0x3UL
 
+#define IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT	0x0000000000080000UL
 #define IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT	0x0000000000040000UL
 #define IBMVETH_ILLAN_LRG_SR_ENABLED	0x0000000000010000UL
 #define IBMVETH_ILLAN_LRG_SND_SUPPORT	0x0000000000008000UL
@@ -260,7 +261,7 @@ static inline long h_illan_attributes(unsigned long unit_address,
 #define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
 #define IBMVETH_MAX_QUEUES 16U
 #define IBMVETH_DEFAULT_QUEUES 8U
-#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_MAX_RX_QUEUES 16U
 #define IBMVETH_DEFAULT_RX_QUEUES 1U
 #define IBMVETH_MAX_RX_REGULAR 8U
 #define IBMVETH_MAX_RX_QUEUE 12U
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 10/14] ibmveth: Add per-queue RX and TX statistics collection

[Mingming Cao]

Count per-queue RX and TX statistics and expose them via ethtool -S
and ndo_get_stats64().

RX hot path (poll / IRQ / replenish) updates rx_qstats[] for packets,
bytes, polls, interrupts, large_packets, invalid_buffers, and
no_buffer_drops. TX hot path updates tx_qstats[] for packets, bytes,
large_packets, dropped_packets, send_failures, and checksum_offload.

Allocate qstat arrays for the adapter lifetime (probe/remove) and free
them from ibmveth_remove() / ibmveth_probe_cleanup(). Adapter-level
ethtool strings sum per-queue counters on read.
ibmveth_update_rx_no_buffer() fills rx_qstats[i].no_buffer_drops and
sums into adapter->rx_no_buffer.

Also fix get_channels() reporting: max_rx is IBMVETH_MAX_RX_QUEUES only
when MQ firmware is enabled; rx_count tracks adapter->num_rx_queues.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Merge v3's separate RX and TX stats commits into one patch.
- Introduce rx_queue_stats / tx_qstats / NUM macros here (first use).
- Allocate/free qstats at probe/remove instead of open/close.
- Report adapter-level ethtool strings by summing per-queue counters on
  read; drop aggregate_* helpers.
- Sum global rx_no_buffer across MQ queues into this statistics patch.
- Cacheline-align per-queue stats; derive field counts with offsetof so
  alignment padding is not counted as a statistic.
- probe_cleanup() cancels reset work, puts pool kobjects via helper from
  the prior patch, and frees qstats on probe failure paths.
- Keep plain u64 qstats like existing ibmveth / ibmvnic (PPC_PSERIES).

 drivers/net/ethernet/ibm/ibmveth.c | 320 +++++++++++++++++++++++++++--
 drivers/net/ethernet/ibm/ibmveth.h |  34 +++
 2 files changed, 338 insertions(+), 16 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 4ad7ced3c608..dbb47a42b4cb 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -228,6 +228,60 @@ ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
 	}
 }
 
+/**
+ * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
+{
+	adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
+				     sizeof(*adapter->rx_qstats),
+				     GFP_KERNEL);
+	if (!adapter->rx_qstats)
+		return -ENOMEM;
+
+	return 0;
+}
+
+/**
+ * ibmveth_free_rx_qstats - Free per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
+{
+	kfree(adapter->rx_qstats);
+	adapter->rx_qstats = NULL;
+}
+
+/**
+ * ibmveth_alloc_tx_qstats - Allocate per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_tx_qstats(struct ibmveth_adapter *adapter)
+{
+	adapter->tx_qstats = kcalloc(IBMVETH_MAX_QUEUES,
+				     sizeof(*adapter->tx_qstats),
+				     GFP_KERNEL);
+	if (!adapter->tx_qstats)
+		return -ENOMEM;
+
+	return 0;
+}
+
+/**
+ * ibmveth_free_tx_qstats - Free per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_tx_qstats(struct ibmveth_adapter *adapter)
+{
+	kfree(adapter->tx_qstats);
+	adapter->tx_qstats = NULL;
+}
+
 /**
  * ibmveth_alloc_rx_queues - Allocate per-queue RX resources
  * @adapter: ibmveth adapter structure
@@ -921,6 +975,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
 		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
 		u64 drops = be64_to_cpup(p);
 
+		if (adapter->rx_qstats)
+			adapter->rx_qstats[i].no_buffer_drops = drops;
 		adapter->rx_no_buffer += drops;
 	}
 }
@@ -1972,22 +2028,131 @@ static int ibmveth_set_features(struct net_device *dev,
  * globals on the hot path (ibmvnic-style); with qstats allocated for the
  * adapter lifetime, these sums remain meaningful across ifdown/up.
  */
+static u64 ibmveth_sum_rx_invalid_buffers(struct ibmveth_adapter *adapter)
+{
+	u64 total = 0;
+	int i;
+
+	if (!adapter->rx_qstats)
+		return adapter->rx_invalid_buffer;
+
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		total += adapter->rx_qstats[i].invalid_buffers;
+
+	return total;
+}
+
+static u64 ibmveth_sum_rx_large_packets(struct ibmveth_adapter *adapter)
+{
+	u64 total = 0;
+	int i;
+
+	if (!adapter->rx_qstats)
+		return adapter->rx_large_packets;
+
+	for (i = 0; i < adapter->num_rx_queues; i++)
+		total += adapter->rx_qstats[i].large_packets;
+
+	return total;
+}
+
+static u64 ibmveth_sum_tx_large_packets(struct ibmveth_adapter *adapter)
+{
+	struct net_device *netdev = adapter->netdev;
+	u64 total = 0;
+	int i;
+
+	if (!adapter->tx_qstats)
+		return adapter->tx_large_packets;
+
+	for (i = 0; i < netdev->real_num_tx_queues; i++)
+		total += adapter->tx_qstats[i].large_packets;
+
+	return total;
+}
+
+static u64 ibmveth_sum_tx_send_failed(struct ibmveth_adapter *adapter)
+{
+	struct net_device *netdev = adapter->netdev;
+	u64 total = 0;
+	int i;
+
+	if (!adapter->tx_qstats)
+		return adapter->tx_send_failed;
+
+	for (i = 0; i < netdev->real_num_tx_queues; i++)
+		total += adapter->tx_qstats[i].send_failures;
+
+	return total;
+}
+
+static u64 ibmveth_ethtool_adapter_stat(struct ibmveth_adapter *adapter,
+					int index)
+{
+	unsigned long offset = ibmveth_stats[index].offset;
+
+	if (offset == IBMVETH_STAT_OFF(rx_invalid_buffer))
+		return ibmveth_sum_rx_invalid_buffers(adapter);
+	if (offset == IBMVETH_STAT_OFF(rx_large_packets))
+		return ibmveth_sum_rx_large_packets(adapter);
+	if (offset == IBMVETH_STAT_OFF(tx_large_packets))
+		return ibmveth_sum_tx_large_packets(adapter);
+	if (offset == IBMVETH_STAT_OFF(tx_send_failed))
+		return ibmveth_sum_tx_send_failed(adapter);
+
+	return IBMVETH_GET_STAT(adapter, offset);
+}
+
 static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
 {
+	struct ibmveth_adapter *adapter = netdev_priv(dev);
+	u8 *p = data;
 	int i;
 
 	if (stringset != ETH_SS_STATS)
 		return;
 
-	for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++, data += ETH_GSTRING_LEN)
-		memcpy(data, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+	for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++) {
+		memcpy(p, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+		p += ETH_GSTRING_LEN;
+	}
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		ethtool_sprintf(&p, "rx%d_packets", i);
+		ethtool_sprintf(&p, "rx%d_bytes", i);
+		ethtool_sprintf(&p, "rx%d_interrupts", i);
+		ethtool_sprintf(&p, "rx%d_polls", i);
+		ethtool_sprintf(&p, "rx%d_large_packets", i);
+		ethtool_sprintf(&p, "rx%d_invalid_buffers", i);
+		ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
+	}
+
+	for (i = 0; i < dev->real_num_tx_queues; i++) {
+		ethtool_sprintf(&p, "tx%d_packets", i);
+		ethtool_sprintf(&p, "tx%d_bytes", i);
+		ethtool_sprintf(&p, "tx%d_large_packets", i);
+		ethtool_sprintf(&p, "tx%d_dropped_packets", i);
+		ethtool_sprintf(&p, "tx%d_send_failures", i);
+		ethtool_sprintf(&p, "tx%d_checksum_offload", i);
+	}
+
+	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+		ethtool_sprintf(&p, "pool%d_size", i);
+		ethtool_sprintf(&p, "pool%d_active", i);
+		ethtool_sprintf(&p, "pool%d_available", i);
+	}
 }
 
 static int ibmveth_get_sset_count(struct net_device *dev, int sset)
 {
+	struct ibmveth_adapter *adapter = netdev_priv(dev);
+
 	switch (sset) {
 	case ETH_SS_STATS:
-		return ARRAY_SIZE(ibmveth_stats);
+		return ARRAY_SIZE(ibmveth_stats) +
+		       adapter->num_rx_queues * IBMVETH_NUM_RX_QSTATS +
+		       dev->real_num_tx_queues * IBMVETH_NUM_TX_QSTATS +
+		       IBMVETH_NUM_BUFF_POOLS * 3;
 	default:
 		return -EOPNOTSUPP;
 	}
@@ -1996,21 +2161,59 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
 static void ibmveth_get_ethtool_stats(struct net_device *dev,
 				      struct ethtool_stats *stats, u64 *data)
 {
-	int i;
 	struct ibmveth_adapter *adapter = netdev_priv(dev);
+	int i, j;
 
 	for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++)
-		data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset);
+		data[i] = ibmveth_ethtool_adapter_stat(adapter, i);
+
+	for (j = 0; j < adapter->num_rx_queues; j++) {
+		if (adapter->rx_qstats) {
+			data[i++] = adapter->rx_qstats[j].packets;
+			data[i++] = adapter->rx_qstats[j].bytes;
+			data[i++] = adapter->rx_qstats[j].interrupts;
+			data[i++] = adapter->rx_qstats[j].polls;
+			data[i++] = adapter->rx_qstats[j].large_packets;
+			data[i++] = adapter->rx_qstats[j].invalid_buffers;
+			data[i++] = adapter->rx_qstats[j].no_buffer_drops;
+		} else {
+			i += IBMVETH_NUM_RX_QSTATS;
+		}
+	}
+
+	for (j = 0; j < dev->real_num_tx_queues; j++) {
+		if (adapter->tx_qstats) {
+			data[i++] = adapter->tx_qstats[j].packets;
+			data[i++] = adapter->tx_qstats[j].bytes;
+			data[i++] = adapter->tx_qstats[j].large_packets;
+			data[i++] = adapter->tx_qstats[j].dropped_packets;
+			data[i++] = adapter->tx_qstats[j].send_failures;
+			data[i++] = adapter->tx_qstats[j].checksum_offload;
+		} else {
+			i += IBMVETH_NUM_TX_QSTATS;
+		}
+	}
+
+	for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+		data[i++] = adapter->rx_buff_pool[0][j].size;
+		data[i++] = adapter->rx_buff_pool[0][j].active;
+		data[i++] = atomic_read(&adapter->rx_buff_pool[0][j].available);
+	}
 }
 
 static void ibmveth_get_channels(struct net_device *netdev,
 				 struct ethtool_channels *channels)
 {
+	struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
 	channels->max_tx = ibmveth_real_max_tx_queues();
 	channels->tx_count = netdev->real_num_tx_queues;
 
-	channels->max_rx = netdev->real_num_rx_queues;
-	channels->rx_count = netdev->real_num_rx_queues;
+	if (adapter->multi_queue)
+		channels->max_rx = IBMVETH_MAX_RX_QUEUES;
+	else
+		channels->max_rx = 1;
+	channels->rx_count = adapter->num_rx_queues;
 }
 
 static int ibmveth_set_channels(struct net_device *netdev,
@@ -2124,6 +2327,8 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
 
 	if (ether_addr_equal(ether_header->h_dest, netdev->dev_addr)) {
 		netdev_dbg(netdev, "veth doesn't support loopback packets, dropping packet.\n");
+		if (adapter->tx_qstats)
+			adapter->tx_qstats[queue_num].dropped_packets++;
 		ret = -EOPNOTSUPP;
 	}
 
@@ -2150,6 +2355,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	    skb_checksum_help(skb)) {
 
 		netdev_err(netdev, "tx: failed to checksum packet\n");
+		adapter->tx_qstats[queue_num].dropped_packets++;
 		goto out;
 	}
 
@@ -2161,6 +2367,8 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 
 		desc_flags |= (IBMVETH_BUF_NO_CSUM | IBMVETH_BUF_CSUM_GOOD);
 
+		adapter->tx_qstats[queue_num].checksum_offload++;
+
 		/* Need to zero out the checksum */
 		buf[0] = 0;
 		buf[1] = 0;
@@ -2172,6 +2380,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	if (skb->ip_summed == CHECKSUM_PARTIAL && skb_is_gso(skb)) {
 		if (adapter->fw_large_send_support) {
 			mss = (unsigned long)skb_shinfo(skb)->gso_size;
+			adapter->tx_qstats[queue_num].large_packets++;
 		} else if (!skb_is_gso_v6(skb)) {
 			/* Put -1 in the IP checksum to tell phyp it
 			 * is a largesend packet. Put the mss in
@@ -2180,6 +2389,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 			ip_hdr(skb)->check = 0xffff;
 			tcp_hdr(skb)->check =
 				cpu_to_be16(skb_shinfo(skb)->gso_size);
+			adapter->tx_qstats[queue_num].large_packets++;
 		}
 	}
 
@@ -2187,6 +2397,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	if (unlikely(skb->len > adapter->tx_ltb_size)) {
 		netdev_err(adapter->netdev, "tx: packet size (%u) exceeds ltb (%u)\n",
 			   skb->len, adapter->tx_ltb_size);
+		adapter->tx_qstats[queue_num].dropped_packets++;
 		goto out;
 	}
 	memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
@@ -2203,6 +2414,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	if (unlikely(total_bytes != skb->len)) {
 		netdev_err(adapter->netdev, "tx: incorrect packet len copied into ltb (%u != %u)\n",
 			   skb->len, total_bytes);
+		adapter->tx_qstats[queue_num].dropped_packets++;
 		goto out;
 	}
 	desc.fields.flags_len = desc_flags | skb->len;
@@ -2211,7 +2423,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
 	dma_wmb();
 
 	if (ibmveth_send(adapter, desc.desc, mss)) {
+		adapter->tx_qstats[queue_num].send_failures++;
+		adapter->tx_qstats[queue_num].dropped_packets++;
 	} else {
+		adapter->tx_qstats[queue_num].packets++;
+		adapter->tx_qstats[queue_num].bytes += skb->len;
 	}
 
 out:
@@ -2350,6 +2566,9 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
 		return 0;
 
+	if (adapter->rx_qstats)
+		adapter->rx_qstats[queue_index].polls++;
+
 restart_poll:
 	while (frames_processed < budget) {
 		if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
@@ -2358,7 +2577,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 		smp_rmb();
 		if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
 			wmb(); /* suggested by larson1 */
-			adapter->rx_invalid_buffer++;
+			if (adapter->rx_qstats)
+				adapter->rx_qstats[queue_index]
+					.invalid_buffers++;
+			else
+				adapter->rx_invalid_buffer++;
 			netdev_dbg(netdev, "recycling invalid buffer\n");
 			rc = ibmveth_rxq_harvest_buffer(adapter,
 							queue_index, true);
@@ -2432,7 +2655,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 			if ((length > netdev->mtu + ETH_HLEN) ||
 			    lrg_pkt || iph_check == 0xffff) {
 				ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
-				adapter->rx_large_packets++;
+				if (adapter->rx_qstats)
+					adapter->rx_qstats[queue_index]
+						.large_packets++;
+				else
+					adapter->rx_large_packets++;
 			}
 
 			if (csum_good) {
@@ -2442,6 +2669,11 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 
 			napi_gro_receive(napi, skb);	/* send it up */
 
+			if (adapter->rx_qstats) {
+				adapter->rx_qstats[queue_index].packets++;
+				adapter->rx_qstats[queue_index].bytes += length;
+			}
+
 			frames_processed++;
 		}
 	}
@@ -2488,6 +2720,9 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
 	if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
 		return IRQ_NONE;
 
+	if (adapter->rx_qstats)
+		adapter->rx_qstats[qindex].interrupts++;
+
 	ibmveth_schedule_rx_queue(adapter, qindex);
 	return IRQ_HANDLED;
 }
@@ -2698,6 +2933,40 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,
 	return vlan_features_check(skb, features);
 }
 
+/**
+ * ibmveth_get_stats64 - Return aggregated per-queue statistics
+ * @dev: network device
+ * @stats: rtnl link statistics storage
+ *
+ * Sums per-queue rx_qstats and tx_qstats into the rtnl counters.
+ * Callers use ndo_get_stats64(); avoid updating netdev->stats on the
+ * xmit/poll paths to keep per-queue counters off the hot cache line.
+ */
+static void ibmveth_get_stats64(struct net_device *dev,
+				struct rtnl_link_stats64 *stats)
+{
+	struct ibmveth_adapter *adapter = netdev_priv(dev);
+	int i;
+
+	if (adapter->rx_qstats) {
+		for (i = 0; i < adapter->num_rx_queues; i++) {
+			stats->rx_packets += adapter->rx_qstats[i].packets;
+			stats->rx_bytes += adapter->rx_qstats[i].bytes;
+		}
+	}
+
+	if (adapter->tx_qstats) {
+		for (i = 0; i < dev->real_num_tx_queues; i++) {
+			stats->tx_packets += adapter->tx_qstats[i].packets;
+			stats->tx_bytes += adapter->tx_qstats[i].bytes;
+			stats->tx_dropped +=
+				adapter->tx_qstats[i].dropped_packets;
+		}
+	}
+
+	stats->tx_errors = dev->stats.tx_errors;
+}
+
 static const struct net_device_ops ibmveth_netdev_ops = {
 	.ndo_open		= ibmveth_open,
 	.ndo_stop		= ibmveth_close,
@@ -2710,13 +2979,14 @@ static const struct net_device_ops ibmveth_netdev_ops = {
 	.ndo_validate_addr	= eth_validate_addr,
 	.ndo_set_mac_address    = ibmveth_set_mac_addr,
 	.ndo_features_check	= ibmveth_features_check,
+	.ndo_get_stats64	= ibmveth_get_stats64,
 #ifdef CONFIG_NET_POLL_CONTROLLER
 	.ndo_poll_controller	= ibmveth_poll_controller,
 #endif
 };
 
 static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
-				   int pools_ready)
+				  int pools_ready)
 {
 	int i;
 
@@ -2724,6 +2994,19 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
 		kobject_put(&adapter->rx_buff_pool[0][i].kobj);
 }
 
+static void ibmveth_probe_cleanup(struct ibmveth_adapter *adapter,
+				  int pools_ready)
+{
+	struct net_device *netdev = adapter->netdev;
+
+	cancel_work_sync(&adapter->work);
+	ibmveth_put_pool_kobjs(adapter, pools_ready);
+
+	ibmveth_free_tx_qstats(adapter);
+	ibmveth_free_rx_qstats(adapter);
+	free_netdev(netdev);
+}
+
 static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 {
 	int rc, i, mac_len, pools_ready = 0;
@@ -2779,6 +3062,11 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 		netif_napi_add_weight(netdev, &adapter->napi[i],
 				      ibmveth_poll, 16);
 
+	if (ibmveth_alloc_rx_qstats(adapter) ||
+	    ibmveth_alloc_tx_qstats(adapter)) {
+		ibmveth_probe_cleanup(adapter, 0);
+		return -ENOMEM;
+	}
 
 	netdev->irq = dev->irq;
 	netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2859,8 +3147,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 				"failed to create pool%d kobject: %d\n", i, rc);
 			/* init_and_add takes a ref even on failure */
 			kobject_put(kobj);
-			ibmveth_put_pool_kobjs(adapter, pools_ready);
-			free_netdev(netdev);
+			ibmveth_probe_cleanup(adapter, pools_ready);
 			return rc;
 		}
 
@@ -2873,8 +3160,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 	if (rc) {
 		netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n",
 			   rc);
-		ibmveth_put_pool_kobjs(adapter, pools_ready);
-		free_netdev(netdev);
+		ibmveth_probe_cleanup(adapter, pools_ready);
 		return rc;
 	}
 	adapter->tx_ltb_size = PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE);
@@ -2890,8 +3176,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 
 	if (rc) {
 		netdev_dbg(netdev, "failed to register netdev rc=%d\n", rc);
-		ibmveth_put_pool_kobjs(adapter, pools_ready);
-		free_netdev(netdev);
+		ibmveth_probe_cleanup(adapter, pools_ready);
 		return rc;
 	}
 
@@ -2913,6 +3198,9 @@ static void ibmveth_remove(struct vio_dev *dev)
 
 	unregister_netdev(netdev);
 
+	ibmveth_free_tx_qstats(adapter);
+	ibmveth_free_rx_qstats(adapter);
+
 	free_netdev(netdev);
 	dev_set_drvdata(&dev->dev, NULL);
 }
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 6e1a964df42a..8e20ccd4a1d5 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -285,6 +285,38 @@ struct ibmveth_hcall_stats {
 	u64 send_lan;		/* H_SEND_LOGICAL_LAN */
 };
 
+struct ibmveth_rx_queue_stats {
+	u64 packets;
+	u64 bytes;
+	u64 interrupts;
+	u64 polls;
+	u64 large_packets;
+	u64 invalid_buffers;
+	u64 no_buffer_drops;
+} ____cacheline_aligned_in_smp;
+
+struct ibmveth_tx_queue_stats {
+	u64 packets;
+	u64 bytes;
+	u64 large_packets;
+	u64 dropped_packets;
+	u64 send_failures;
+	u64 checksum_offload;
+} ____cacheline_aligned_in_smp;
+
+/*
+ * ethtool string count: use offsetof of the last counter so alignment
+ * padding from ____cacheline_aligned_in_smp is not counted. When adding
+ * a new counter at the end, point these at the new last field (same idea
+ * as sizeof(struct)/sizeof(u64) before alignment was added).
+ */
+#define IBMVETH_NUM_RX_QSTATS \
+	(offsetof(struct ibmveth_rx_queue_stats, no_buffer_drops) / \
+		sizeof(u64) + 1)
+#define IBMVETH_NUM_TX_QSTATS \
+	(offsetof(struct ibmveth_tx_queue_stats, checksum_offload) / \
+		sizeof(u64) + 1)
+
 struct ibmveth_buff_pool {
     u32 size;
     u32 index;
@@ -352,6 +384,8 @@ struct ibmveth_adapter {
 
 	/* Multi-queue statistics */
 	struct ibmveth_hcall_stats hcall_stats;
+	struct ibmveth_rx_queue_stats *rx_qstats;
+	struct ibmveth_tx_queue_stats *tx_qstats;
 
 	/* Ethtool settings */
 	u8 duplex;
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 11/14] ibmveth: Expose per-queue buffer pool details via debugfs

[Mingming Cao]

Add a read-only buffer_pools debugfs file that lists size, buff_size,
active, and available for every RX queue and buffer pool. That is the
MQ diagnostic view (multi-line table), which does not belong in sysfs.

Keep the historical queue-0 poolN sysfs ABI unchanged:

  .../poolN/{active,num,size}

Those are one-value-per-file *configuration* knobs for buffer-pool
classes on queue 0, not one directory per RX queue. With MQ, queue 0
pool geometry remains the shared template: open copies it to queues
1..N. Per-queue runtime pressure is what debugfs shows; this series
does not add per-queue pool sysfs dirs.

Unlike ibmvnic (one RX pool per RX queue, sized from firmware, almost
no pool sysfs), ibmveth historically has multiple size-class pools and
already exported poolN via sysfs. Moving only the new all-queue dump
to debugfs matches sysfs "one value per file" review feedback without
breaking that config ABI.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Move the all-queue buffer_pools diagnostic from sysfs to debugfs;
  subject updated to match.
- Keep historical queue-0 poolN/{active,num,size} sysfs as one-value
  config (template for MQ); do not add per-queue pool sysfs dirs.

 drivers/net/ethernet/ibm/ibmveth.c | 43 ++++++++++++++++++++++++++++++
 drivers/net/ethernet/ibm/ibmveth.h |  2 ++
 2 files changed, 45 insertions(+)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index dbb47a42b4cb..db1374746602 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -31,6 +31,7 @@
 #include <linux/ipv6.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/debugfs.h>
 #include <asm/hvcall.h>
 #include <linux/atomic.h>
 #include <asm/vio.h>
@@ -2985,6 +2986,44 @@ static const struct net_device_ops ibmveth_netdev_ops = {
 #endif
 };
 
+static int ibmveth_buffer_pools_show(struct seq_file *m, void *v)
+{
+	struct ibmveth_adapter *adapter = m->private;
+	int i, j;
+
+	seq_puts(m, "Queue  Pool  Size  BuffSize  Active  Available\n");
+	seq_puts(m, "-----  ----  ----  --------  ------  ---------\n");
+
+	for (i = 0; i < adapter->num_rx_queues; i++) {
+		for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+			struct ibmveth_buff_pool *pool =
+				&adapter->rx_buff_pool[i][j];
+
+			seq_printf(m, "%5d  %4d  %4u  %8u  %6d  %9d\n",
+				   i, j, pool->size, pool->buff_size,
+				   pool->active,
+				   atomic_read(&pool->available));
+		}
+	}
+
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(ibmveth_buffer_pools);
+
+static void ibmveth_debugfs_init(struct ibmveth_adapter *adapter)
+{
+	adapter->debugfs_dir = debugfs_create_dir(adapter->netdev->name,
+						  NULL);
+	debugfs_create_file("buffer_pools", 0400, adapter->debugfs_dir,
+			    adapter, &ibmveth_buffer_pools_fops);
+}
+
+static void ibmveth_debugfs_exit(struct ibmveth_adapter *adapter)
+{
+	debugfs_remove_recursive(adapter->debugfs_dir);
+	adapter->debugfs_dir = NULL;
+}
+
 static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
 				  int pools_ready)
 {
@@ -3182,6 +3221,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
 
 	netdev_dbg(netdev, "registered\n");
 
+	ibmveth_debugfs_init(adapter);
+
 	return 0;
 }
 
@@ -3193,6 +3234,8 @@ static void ibmveth_remove(struct vio_dev *dev)
 
 	cancel_work_sync(&adapter->work);
 
+	ibmveth_debugfs_exit(adapter);
+
 	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
 		kobject_put(&adapter->rx_buff_pool[0][i].kobj);
 
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 8e20ccd4a1d5..17faaa4c2d33 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -387,6 +387,8 @@ struct ibmveth_adapter {
 	struct ibmveth_rx_queue_stats *rx_qstats;
 	struct ibmveth_tx_queue_stats *tx_qstats;
 
+	struct dentry *debugfs_dir;
+
 	/* Ethtool settings */
 	u8 duplex;
 	u32 speed;
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 12/14] ibmveth: Implement incremental MQ RX queue resize

[Mingming Cao]

Add helpers to allocate, register, interrupt-setup, and free individual
RX queues, plus ibmveth_resize_rx_queues_incremental() to grow or shrink
the live RX queue count without tearing down surviving queues.
Surviving queues keep PHYP handles, buffer pools, and IRQ state across
resize.

Scale-up for each new queue:
  alloc -> register -> request_irq -> publish num_rx_queues ->
  replenish -> napi_enable -> enable_irq

so the queue is visible and buffered before PHYP delivery is unmasked.
On enable_irq failure after napi_enable, disable NAPI before teardown.
Scale-down:
  disable_irq+sync -> napi_disable -> drain -> deregister -> free

with PHYP mask before napi_disable/drain (same storm rule as close).

IRQ vs PHYP ownership for subordinates:
  - deregister_single is PHYP-only (H_FREE_LOGICAL_LAN_QUEUE + clear
    queue_handle)
  - Linux virq disposal lives in cleanup_single / dispose helpers
  - single-slot dispose is bounded by IBMVETH_MAX_RX_QUEUES (not
    num_rx_queues) so scale-down / fail paths can dispose queues that
    left the published live set but still own a mapping
  - dispose on request_irq failure where deregister no longer releases
    the mapping

Also add drain-path smp_rmb() before harvesting RX buffers so drain
matches poll's descriptor-ordering assumptions, and replenish before
re-enabling IRQ/NAPI on set_real_num_rx scale-down rollback.

Introduce ibmveth_resize_rx_channels() (validate + apply) and call it
from set_channels() when the interface is up so the resize helper is
not left unused. Full !IFF_UP RX stash ordering relative
to TX is completed in the next patch.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Copy pool->index when cloning buffer pools for incrementally added
  queues.
- Scale-up order: publish -> replenish -> napi_enable -> enable_irq
  (vs older enable-before-publish drafts); mirror NAPI-before-unmask on
  set_real_num_rx rollback.
- Linux-owned subordinate virq disposal; deregister is PHYP-only;
  dispose bound to MAX_RX_QUEUES; dispose on request_irq failure.
- Drain-path smp_rmb() before harvest.
- Mask PHYP before napi_disable/drain on scale-down and scale-up fail
  cleanup.
- Replenish before re-enabling IRQ/NAPI on set_real_num_rx scale-down
  rollback.
- Keep set_channels wiring as the following patch (same split as v3)
  but call resize_rx_channels() here when IFF_UP so the helper is not
  an unused static.

 drivers/net/ethernet/ibm/ibmveth.c | 512 ++++++++++++++++++++++++++++-
 1 file changed, 510 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index db1374746602..b57c7df92853 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -651,6 +651,58 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
 	/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */
 }
 
+/**
+ * ibmveth_setup_single_rx_interrupt - Setup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to setup
+ *
+ * Registers the IRQ handler for one queue. Used during incremental
+ * scale-up when adding new RX queues. The caller publishes the queue,
+ * replenishes buffers, enables NAPI, then unmasks PHYP delivery.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_setup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+				  int queue_idx)
+{
+	struct net_device *netdev = adapter->netdev;
+	int rc;
+
+	rc = request_irq(adapter->queue_irq[queue_idx], ibmveth_interrupt,
+			 0, netdev->name, &adapter->napi[queue_idx]);
+	if (rc) {
+		netdev_err(netdev, "request_irq() failed for queue %d: %d\n",
+			   queue_idx, rc);
+		return rc;
+	}
+
+	netdev_dbg(netdev, "Setup IRQ %d for queue %d\n",
+		   adapter->queue_irq[queue_idx], queue_idx);
+	return 0;
+}
+
+/**
+ * ibmveth_cleanup_single_rx_interrupt - Cleanup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to cleanup
+ *
+ * Frees the IRQ handler for one queue and releases the subordinate virq
+ * mapping. Used during incremental scale-down.
+ */
+static void
+ibmveth_cleanup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+				    int queue_idx)
+{
+	if (adapter->queue_irq[queue_idx]) {
+		free_irq(adapter->queue_irq[queue_idx],
+			 &adapter->napi[queue_idx]);
+		ibmveth_dispose_subordinate_irq_mapping(adapter, queue_idx);
+		netdev_dbg(adapter->netdev,
+			   "Freed IRQ for queue %d\n", queue_idx);
+	}
+}
+
 /**
  * ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue
  * @adapter: ibmveth adapter structure
@@ -1189,6 +1241,141 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
 		   adapter->num_rx_queues);
 }
 
+/**
+ * ibmveth_alloc_single_rx_queue - Allocate resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to allocate
+ * @rxq_entries: Number of RX queue entries
+ *
+ * Allocates buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-up without affecting existing queues.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx,
+			      int rxq_entries)
+{
+	struct device *dev = &adapter->vdev->dev;
+	struct net_device *netdev = adapter->netdev;
+	int i, rc = -ENOMEM;
+
+	adapter->buffer_list_addr[queue_idx] =
+		(void *)get_zeroed_page(GFP_KERNEL);
+	if (!adapter->buffer_list_addr[queue_idx]) {
+		netdev_err(netdev, "unable to allocate buffer list for queue %d\n",
+			   queue_idx);
+		return -ENOMEM;
+	}
+
+	adapter->rx_queue[queue_idx].queue_len =
+		sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+	adapter->rx_queue[queue_idx].queue_addr =
+		dma_alloc_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+				   &adapter->rx_queue[queue_idx].queue_dma,
+				   GFP_KERNEL);
+	if (!adapter->rx_queue[queue_idx].queue_addr) {
+		netdev_err(netdev, "unable to allocate RX queue for queue %d\n",
+			   queue_idx);
+		goto out_free_buflist;
+	}
+
+	adapter->buffer_list_dma[queue_idx] =
+		dma_map_single(dev, adapter->buffer_list_addr[queue_idx],
+			       4096, DMA_BIDIRECTIONAL);
+	if (dma_mapping_error(dev, adapter->buffer_list_dma[queue_idx])) {
+		netdev_err(netdev, "unable to map buffer list for queue %d\n",
+			   queue_idx);
+		goto out_free_rxq;
+	}
+
+	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+		struct ibmveth_buff_pool *src =
+			&adapter->rx_buff_pool[0][i];
+		struct ibmveth_buff_pool *dst =
+			&adapter->rx_buff_pool[queue_idx][i];
+
+		dst->size = src->size;
+		dst->index = src->index;
+		dst->buff_size = src->buff_size;
+		dst->threshold = src->threshold;
+		dst->active = src->active;
+	}
+
+	rc = ibmveth_alloc_queue_buffer_pools(adapter, queue_idx);
+	if (rc) {
+		netdev_err(netdev,
+			   "Failed to allocate buffer pools for queue %d\n",
+			   queue_idx);
+		goto out_unmap_buflist;
+	}
+
+	adapter->rx_queue[queue_idx].index = 0;
+	adapter->rx_queue[queue_idx].num_slots = rxq_entries;
+	adapter->rx_queue[queue_idx].toggle = 1;
+	spin_lock_init(&adapter->rx_queue[queue_idx].replenish_lock);
+
+	netdev_dbg(netdev,
+		   "Allocated queue %d: buffer_list @ %p (DMA: 0x%llx), rx_queue @ %p (DMA: 0x%llx), %d entries\n",
+		   queue_idx, adapter->buffer_list_addr[queue_idx],
+		   (unsigned long long)adapter->buffer_list_dma[queue_idx],
+		   adapter->rx_queue[queue_idx].queue_addr,
+		   (unsigned long long)adapter->rx_queue[queue_idx].queue_dma,
+		   rxq_entries);
+
+	return 0;
+
+out_unmap_buflist:
+	dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+			 4096, DMA_BIDIRECTIONAL);
+	adapter->buffer_list_dma[queue_idx] = 0;
+out_free_rxq:
+	dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+			  adapter->rx_queue[queue_idx].queue_addr,
+			  adapter->rx_queue[queue_idx].queue_dma);
+	adapter->rx_queue[queue_idx].queue_addr = NULL;
+out_free_buflist:
+	free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+	adapter->buffer_list_addr[queue_idx] = NULL;
+	return rc;
+}
+
+/**
+ * ibmveth_free_single_rx_queue - Free resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to free
+ *
+ * Frees buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-down without affecting remaining queues.
+ */
+static void
+ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
+{
+	struct device *dev = &adapter->vdev->dev;
+
+	ibmveth_free_queue_buffer_pools(adapter, queue_idx);
+
+	if (adapter->buffer_list_dma[queue_idx]) {
+		dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+				 4096, DMA_BIDIRECTIONAL);
+		adapter->buffer_list_dma[queue_idx] = 0;
+	}
+
+	if (adapter->rx_queue[queue_idx].queue_addr) {
+		dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+				  adapter->rx_queue[queue_idx].queue_addr,
+				  adapter->rx_queue[queue_idx].queue_dma);
+		adapter->rx_queue[queue_idx].queue_addr = NULL;
+	}
+
+	if (adapter->buffer_list_addr[queue_idx]) {
+		free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+		adapter->buffer_list_addr[queue_idx] = NULL;
+	}
+
+	netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
+}
+
 /**
  * ibmveth_remove_buffer_from_pool - remove a buffer from a pool
  * @adapter: adapter instance
@@ -1307,6 +1494,51 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
 	return 0;
 }
 
+/**
+ * ibmveth_drain_rx_queue - Drain pending buffers from an RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Queue index to drain
+ *
+ * Recycles all pending buffers back to the per-queue buffer pools.
+ * Must be called with NAPI disabled for this queue.
+ *
+ * Return: Number of buffers drained
+ */
+static int
+ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
+{
+	struct net_device *netdev = adapter->netdev;
+	int drained = 0;
+	int limit = adapter->rx_queue[queue_index].num_slots;
+	int rc;
+
+	netdev_dbg(netdev, "Draining RX queue %d (limit: %d slots)\n",
+		   queue_index, limit);
+
+	while (drained < limit &&
+	       ibmveth_rxq_pending_buffer(adapter, queue_index)) {
+		/* Match poll-side order before harvesting completion state. */
+		smp_rmb();
+		rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+		if (rc) {
+			netdev_err(netdev,
+				   "Failed to harvest buffer from queue %d during drain: %d\n",
+				   queue_index, rc);
+			break;
+		}
+		drained++;
+	}
+
+	if (drained > 0)
+		netdev_dbg(netdev, "Drained %d buffer(s) from RX queue %d\n",
+			   drained, queue_index);
+	else
+		netdev_dbg(netdev, "No buffers to drain from RX queue %d\n",
+			   queue_index);
+
+	return drained;
+}
+
 static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
 {
 	if (!adapter->tx_ltb_ptr[idx])
@@ -1562,6 +1794,227 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
 	return 0;
 }
 
+/**
+ * ibmveth_deregister_single_rx_queue - Deregister one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to deregister (1..N)
+ *
+ * Deregisters a single queue via H_FREE_LOGICAL_LAN_QUEUE. Linux IRQ handler
+ * teardown and subordinate virq mapping disposal are owned by interrupt
+ * cleanup helpers; queue 0 is freed only through ibmveth_free_all_queues()
+ * (H_FREE_LOGICAL_LAN).
+ */
+static void
+ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
+				   int queue_idx)
+{
+	unsigned long lpar_rc;
+	unsigned long ua = adapter->vdev->unit_address;
+	unsigned long qh = adapter->queue_handle[queue_idx];
+
+	if (!qh)
+		return;
+
+	do {
+		lpar_rc = h_free_logical_lan_queue(ua, qh);
+	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+	adapter->hcall_stats.free_lan_queue++;
+
+	if (lpar_rc != H_SUCCESS) {
+		netdev_err(adapter->netdev,
+			   "h_free_logical_lan_queue failed for queue %d: rc=0x%lx\n",
+			   queue_idx, lpar_rc);
+	}
+
+	adapter->queue_handle[queue_idx] = 0;
+
+	netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
+}
+
+/**
+ * ibmveth_resize_rx_queues_incremental - Resize RX queue count incrementally
+ * @adapter: ibmveth adapter structure
+ * @new_count: Target number of RX queues
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Adds or removes RX queues without tearing down the entire adapter.
+ * Active queues continue receiving during scale-up; scale-down drains
+ * excess queues before deregistering them with the hypervisor.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
+				     int new_count, int rxq_entries)
+{
+	struct net_device *netdev = adapter->netdev;
+	u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
+	int old_count = adapter->num_rx_queues;
+	int failed_queue;
+	int rc, i;
+
+	if (old_count == new_count) {
+		netdev_dbg(netdev, "RX queue count unchanged (%d), nothing to do\n",
+			   old_count);
+		return 0;
+	}
+
+	if (new_count < 1 || new_count > IBMVETH_MAX_RX_QUEUES) {
+		netdev_err(netdev, "Invalid RX queue count %d (must be 1-%d)\n",
+			   new_count, IBMVETH_MAX_RX_QUEUES);
+		return -EINVAL;
+	}
+
+	netdev_info(netdev, "Incrementally resizing RX queues: %d to %d\n",
+		    old_count, new_count);
+
+	if (new_count > old_count) {
+		netdev_dbg(netdev, "Scale-up: adding queues %d-%d\n",
+			   old_count, new_count - 1);
+
+		for (i = old_count; i < new_count; i++) {
+			rc = ibmveth_alloc_single_rx_queue(adapter, i,
+							   rxq_entries);
+			if (rc) {
+				netdev_err(netdev, "Failed to allocate queue %d: %d\n",
+					   i, rc);
+				goto cleanup_new_queues;
+			}
+
+			rc = ibmveth_register_single_rx_queue(adapter, i,
+							      mac_address);
+			if (rc) {
+				netdev_err(netdev, "Failed to register queue %d: %d\n",
+					   i, rc);
+				ibmveth_free_single_rx_queue(adapter, i);
+				goto cleanup_new_queues;
+			}
+
+			rc = ibmveth_setup_single_rx_interrupt(adapter, i);
+			if (rc) {
+				netdev_err(netdev,
+					   "Failed to setup IRQ for queue %d: %d\n",
+					   i, rc);
+				/* request_irq failed: mapped but no handler */
+				ibmveth_dispose_subordinate_irq_mapping(adapter,
+									i);
+				ibmveth_deregister_single_rx_queue(adapter, i);
+				ibmveth_free_single_rx_queue(adapter, i);
+				goto cleanup_new_queues;
+			}
+
+			/*
+			 * Fully ready before PHYP delivery, matching open():
+			 * publish -> replenish -> napi_enable -> enable_irq.
+			 * That way ibmveth_interrupt() cannot run on an
+			 * unpublished, empty, or NAPI-disabled queue.
+			 */
+			adapter->num_rx_queues = i + 1;
+			ibmveth_replenish_task(adapter, i);
+			napi_enable(&adapter->napi[i]);
+
+			rc = ibmveth_enable_irq(adapter, i);
+			if (rc) {
+				netdev_err(netdev,
+					   "Failed to enable IRQ for queue %d: %d\n",
+					   i, rc);
+				adapter->num_rx_queues = i;
+				napi_disable(&adapter->napi[i]);
+				ibmveth_cleanup_single_rx_interrupt(adapter, i);
+				ibmveth_deregister_single_rx_queue(adapter, i);
+				ibmveth_free_single_rx_queue(adapter, i);
+				goto cleanup_new_queues;
+			}
+		}
+
+		rc = netif_set_real_num_rx_queues(netdev, new_count);
+		if (rc) {
+			netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+				   new_count, rc);
+			goto cleanup_new_queues;
+		}
+	} else {
+		netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
+			   new_count, old_count - 1);
+
+		/*
+		 * Mask PHYP delivery before napi_disable/drain. Otherwise
+		 * ibmveth_interrupt returns IRQ_HANDLED without masking when
+		 * NAPI is disabled, and the HV can storm during drain.
+		 */
+		for (i = new_count; i < old_count; i++) {
+			ibmveth_disable_irq(adapter, i);
+			synchronize_irq(adapter->queue_irq[i]);
+		}
+
+		for (i = new_count; i < old_count; i++)
+			napi_disable(&adapter->napi[i]);
+
+		for (i = new_count; i < old_count; i++)
+			ibmveth_drain_rx_queue(adapter, i);
+
+		synchronize_net();
+
+		rc = netif_set_real_num_rx_queues(netdev, new_count);
+		if (rc) {
+			netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+				   new_count, rc);
+			for (i = new_count; i < old_count; i++) {
+				ibmveth_replenish_task(adapter, i);
+				napi_enable(&adapter->napi[i]);
+				ibmveth_enable_irq(adapter, i);
+			}
+			return rc;
+		}
+
+		adapter->num_rx_queues = new_count;
+
+		for (i = new_count; i < old_count; i++) {
+			ibmveth_cleanup_single_rx_interrupt(adapter, i);
+			ibmveth_deregister_single_rx_queue(adapter, i);
+			ibmveth_free_single_rx_queue(adapter, i);
+		}
+	}
+
+	netdev_info(netdev, "Successfully resized to %d RX queues (incremental)\n",
+		    adapter->num_rx_queues);
+
+	if (firmware_has_feature(FW_FEATURE_CMO))
+		vio_cmo_set_dev_desired(adapter->vdev,
+					ibmveth_get_desired_dma(adapter->vdev));
+
+	return 0;
+
+cleanup_new_queues:
+	failed_queue = i;
+	netdev_err(netdev,
+		   "Scale-up failed at queue %d, cleaning up queues %d-%d\n",
+		   failed_queue, old_count, failed_queue - 1);
+	for (i = old_count; i < failed_queue; i++) {
+		ibmveth_disable_irq(adapter, i);
+		synchronize_irq(adapter->queue_irq[i]);
+	}
+
+	for (i = old_count; i < failed_queue; i++)
+		napi_disable(&adapter->napi[i]);
+
+	for (i = old_count; i < failed_queue; i++)
+		ibmveth_drain_rx_queue(adapter, i);
+
+	synchronize_net();
+
+	for (i = old_count; i < failed_queue; i++) {
+		ibmveth_cleanup_single_rx_interrupt(adapter, i);
+		ibmveth_deregister_single_rx_queue(adapter, i);
+		ibmveth_free_single_rx_queue(adapter, i);
+	}
+	adapter->num_rx_queues = old_count;
+	netdev_warn(netdev, "Keeping %d queues after scale-up failure\n",
+		    old_count);
+	return rc;
+}
+
 /**
  * ibmveth_free_all_queues - Free all RX queues at once
  * @adapter: ibmveth adapter structure
@@ -2217,12 +2670,62 @@ static void ibmveth_get_channels(struct net_device *netdev,
 	channels->rx_count = adapter->num_rx_queues;
 }
 
+/**
+ * ibmveth_resize_rx_channels - Validate and apply a new RX queue count
+ * @adapter: ibmveth adapter structure
+ * @goal_rx: desired RX queue count
+ *
+ * When the interface is up, resize live queues via
+ * ibmveth_resize_rx_queues_incremental(). When down, only stash
+ * adapter->num_rx_queues for the next open().
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int ibmveth_resize_rx_channels(struct ibmveth_adapter *adapter,
+				      unsigned int goal_rx)
+{
+	struct net_device *netdev = adapter->netdev;
+	unsigned int old_rx = adapter->num_rx_queues;
+	int rxq_entries;
+	int rc;
+
+	if (goal_rx > 1 && !adapter->multi_queue) {
+		netdev_err(netdev,
+			   "Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n",
+			   goal_rx);
+		return -EOPNOTSUPP;
+	}
+
+	if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
+		netdev_err(netdev,
+			   "Invalid RX queue count %u (must be 1-%d)\n",
+			   goal_rx, IBMVETH_MAX_RX_QUEUES);
+		return -EINVAL;
+	}
+
+	if (goal_rx == old_rx)
+		return 0;
+
+	if (!(netdev->flags & IFF_UP)) {
+		adapter->num_rx_queues = goal_rx;
+		return 0;
+	}
+
+	rxq_entries = adapter->rx_queue[0].num_slots;
+	rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
+						  rxq_entries);
+	if (rc)
+		netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
+	return rc;
+}
+
 static int ibmveth_set_channels(struct net_device *netdev,
 				struct ethtool_channels *channels)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
 	unsigned int old = netdev->real_num_tx_queues,
 		     goal = channels->tx_count;
+	unsigned int goal_rx = channels->rx_count;
 	int rc, i;
 
 	/* If ndo_open has not been called yet then don't allocate, just set
@@ -2231,6 +2734,13 @@ static int ibmveth_set_channels(struct net_device *netdev,
 	if (!(netdev->flags & IFF_UP))
 		return netif_set_real_num_tx_queues(netdev, goal);
 
+	/* Resize RX first while UP so ibmveth_resize_rx_channels() is used
+	 * in this patch. !IFF_UP RX stash ordering lands next.
+	 */
+	rc = ibmveth_resize_rx_channels(adapter, goal_rx);
+	if (rc)
+		return rc;
+
 	/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
 	 * but we may need to alloc/free the ltb's.
 	 */
@@ -2245,7 +2755,6 @@ static int ibmveth_set_channels(struct net_device *netdev,
 		if (!rc)
 			continue;
 
-		/* if something goes wrong, free everything we just allocated */
 		netdev_err(netdev, "Failed to allocate more tx queues, returning to %d queues\n",
 			   old);
 		goal = old;
@@ -2259,7 +2768,6 @@ static int ibmveth_set_channels(struct net_device *netdev,
 		goal = old;
 		old = i;
 	}
-	/* Free any that are no longer needed */
 	for (i = old; i > goal; i--) {
 		if (adapter->tx_ltb_ptr[i - 1])
 			ibmveth_free_tx_ltb(adapter, i - 1);
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 13/14] ibmveth: Wire ethtool set_channels to MQ RX queue resize

[Mingming Cao]

Complete set_channels() RX handling around ibmveth_resize_rx_channels():

  - When the interface is down: set TX queues first, then stash the
    desired RX count in adapter->num_rx_queues for the next open()
    (open publishes via netif_set_real_num_rx_queues). While down
    there are no RX queue mappings, buffers, or IRQs to grow or
    shrink, so do not allocate immediately.
  - When up: resize RX via ibmveth_resize_rx_channels(), then adjust
    TX LTBs with the existing stop/alloc/set_real_num_tx/free/wake
    path.
  - Non-MQ firmware returns -EOPNOTSUPP for rx > 1.
  - Validate rx_count within 1..IBMVETH_MAX_RX_QUEUES.

TX path hardening:
  - Initialize i = old_tx so a scale-down path that never enters the
    alloc loop still has defined bounds if set_real_num_tx_queues()
    fails.
  - Always return rc from set_channels().

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- On !IFF_UP, stash num_rx_queues only after TX set succeeds; do not
  allocate live subordinate IRQs/buffers while down.
- Initialize i = old_tx on the TX adjust path.
- Always return rc from set_channels().
- Split from the resize-helper patch (same split as v3) while keeping
  a live caller of resize_rx_channels() in the previous patch.

 drivers/net/ethernet/ibm/ibmveth.c | 51 ++++++++++++++++++++----------
 1 file changed, 34 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b57c7df92853..c99d8e8be7b3 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -2723,20 +2723,28 @@ static int ibmveth_set_channels(struct net_device *netdev,
 				struct ethtool_channels *channels)
 {
 	struct ibmveth_adapter *adapter = netdev_priv(netdev);
-	unsigned int old = netdev->real_num_tx_queues,
-		     goal = channels->tx_count;
 	unsigned int goal_rx = channels->rx_count;
+	unsigned int old_tx = netdev->real_num_tx_queues;
+	unsigned int goal_tx = channels->tx_count;
 	int rc, i;
 
 	/* If ndo_open has not been called yet then don't allocate, just set
 	 * desired netdev_queue's and return
 	 */
-	if (!(netdev->flags & IFF_UP))
-		return netif_set_real_num_tx_queues(netdev, goal);
+	if (!(netdev->flags & IFF_UP)) {
+		if (goal_tx != old_tx) {
+			rc = netif_set_real_num_tx_queues(netdev, goal_tx);
+			if (rc)
+				return rc;
+		}
+
+		/* Stash desired RX count only after TX succeeds (or was
+		 * already correct); open() publishes it via
+		 * netif_set_real_num_rx_queues().
+		 */
+		return ibmveth_resize_rx_channels(adapter, goal_rx);
+	}
 
-	/* Resize RX first while UP so ibmveth_resize_rx_channels() is used
-	 * in this patch. !IFF_UP RX stash ordering lands next.
-	 */
 	rc = ibmveth_resize_rx_channels(adapter, goal_rx);
 	if (rc)
 		return rc;
@@ -2744,10 +2752,17 @@ static int ibmveth_set_channels(struct net_device *netdev,
 	/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
 	 * but we may need to alloc/free the ltb's.
 	 */
+	if (goal_tx == old_tx)
+		return 0;
+
 	netif_tx_stop_all_queues(netdev);
 
-	/* Allocate any queue that we need */
-	for (i = old; i < goal; i++) {
+	/* Allocate any queue that we need. Initialize i to old_tx so a
+	 * scale-down path that never enters the loop still has defined
+	 * bounds if set_real_num_tx_queues() fails.
+	 */
+	i = old_tx;
+	for (; i < goal_tx; i++) {
 		if (adapter->tx_ltb_ptr[i])
 			continue;
 
@@ -2755,20 +2770,22 @@ static int ibmveth_set_channels(struct net_device *netdev,
 		if (!rc)
 			continue;
 
+		/* if something goes wrong, free everything we just allocated */
 		netdev_err(netdev, "Failed to allocate more tx queues, returning to %d queues\n",
-			   old);
-		goal = old;
-		old = i;
+			   old_tx);
+		goal_tx = old_tx;
+		old_tx = i;
 		break;
 	}
-	rc = netif_set_real_num_tx_queues(netdev, goal);
+	rc = netif_set_real_num_tx_queues(netdev, goal_tx);
 	if (rc) {
 		netdev_err(netdev, "Failed to set real tx queues, returning to %d queues\n",
-			   old);
-		goal = old;
-		old = i;
+			   old_tx);
+		goal_tx = old_tx;
+		old_tx = i;
 	}
-	for (i = old; i > goal; i--) {
+	/* Free any that are no longer needed */
+	for (i = old_tx; i > goal_tx; i--) {
 		if (adapter->tx_ltb_ptr[i - 1])
 			ibmveth_free_tx_ltb(adapter, i - 1);
 	}
-- 
2.50.1 (Apple Git-155)

[PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize

[Mingming Cao]

After aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor
with a correlator that no longer matches the per-queue buffer pools. Poll
treated this as fatal: ibmveth_rxq_get_buffer() WARNed and returned NULL
without advancing the ring, then restart_poll retried the same slot
forever.

Advance past bad correlators instead of spinning: validate correlators
without WARN_ON, skip invalid slots in poll (count as invalid_buffers),
and advance the RX ring when remove_buffer_from_pool cannot map the
correlator. Rate-limit the bad correlator message.

Complete NAPI when the interface is down or napi_disable is pending so
close/quiesce can finish. Do not restart_poll in that window. Close
keeps hypervisor IRQ disable before napi_disable (via
cleanup_rx_interrupts() / related cleanup helpers).

Also validate descriptor length against skb tailroom before skb_put(),
and after napi_complete_done() on the budget-exhausted shutdown path
return a value less than budget so NAPI does not immediately reschedule.

Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---

Changes in v4:
- Skip invalid correlators in poll instead of spinning (no WARN_ON;
  rate-limited message; advance ring when harvest cannot map).
- Validate descriptor length against skb tailroom before skb_put().
- After napi_complete_done() on the budget-exhausted shutdown path,
  return a value less than budget so NAPI does not immediately
  reschedule.
- Align KUnit comments with correlator validation (no WARN_ON).

 drivers/net/ethernet/ibm/ibmveth.c | 109 ++++++++++++++++++++++-------
 1 file changed, 85 insertions(+), 24 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index c99d8e8be7b3..09e06d86701a 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1376,6 +1376,24 @@ ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
 	netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
 }
 
+static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
+					 int queue_index, u64 correlator)
+{
+	unsigned int pool = correlator >> 32;
+	unsigned int index = correlator & 0xffffffffUL;
+
+	return pool < IBMVETH_NUM_BUFF_POOLS &&
+	       index < adapter->rx_buff_pool[queue_index][pool].size;
+}
+
+static void ibmveth_rxq_advance(struct ibmveth_rx_q *rxq)
+{
+	if (++rxq->index == rxq->num_slots) {
+		rxq->index = 0;
+		rxq->toggle = !rxq->toggle;
+	}
+}
+
 /**
  * ibmveth_remove_buffer_from_pool - remove a buffer from a pool
  * @adapter: adapter instance
@@ -1397,17 +1415,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
 	unsigned int free_index;
 	struct sk_buff *skb;
 
-	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
-	    WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
-		schedule_work(&adapter->work);
+	if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
 		return -EINVAL;
-	}
 
 	skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
-	if (WARN_ON(!skb)) {
-		schedule_work(&adapter->work);
+	if (!skb)
 		return -EFAULT;
-	}
 
 	/* if we are going to reuse the buffer then keep the pointers around
 	 * but mark index as available. replenish will see the skb pointer and
@@ -1452,11 +1465,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
 	unsigned int pool = correlator >> 32;
 	unsigned int index = correlator & 0xffffffffUL;
 
-	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
-	    WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
-		schedule_work(&adapter->work);
+	if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
 		return NULL;
-	}
 
 	return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
 }
@@ -1483,14 +1493,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
 
 	cor = rxq->queue_addr[rxq->index].correlator;
 	rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
-	if (unlikely(rc))
+	if (unlikely(rc)) {
+		if (rc == -EINVAL || rc == -EFAULT)
+			goto advance;
 		return rc;
-
-	if (++rxq->index == rxq->num_slots) {
-		rxq->index = 0;
-		rxq->toggle = !rxq->toggle;
 	}
 
+advance:
+	ibmveth_rxq_advance(rxq);
+
 	return 0;
 }
 
@@ -3092,11 +3103,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
 		return 0;
 
+	if (!netif_running(netdev) || napi_disable_pending(napi)) {
+		napi_complete_done(napi, 0);
+		return 0;
+	}
+
 	if (adapter->rx_qstats)
 		adapter->rx_qstats[queue_index].polls++;
 
 restart_poll:
 	while (frames_processed < budget) {
+		if (!netif_running(netdev) || napi_disable_pending(napi))
+			break;
+
 		if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
 			break;
 
@@ -3126,8 +3145,45 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 			__sum16 iph_check = 0;
 
 			skb = ibmveth_rxq_get_buffer(adapter, queue_index);
-			if (unlikely(!skb))
-				break;
+			if (unlikely(!skb)) {
+				if (net_ratelimit())
+					netdev_err(netdev,
+						   "bad correlator on queue %d, skipping slot\n",
+						   queue_index);
+				if (adapter->rx_qstats)
+					adapter->rx_qstats[queue_index]
+						.invalid_buffers++;
+				else
+					adapter->rx_invalid_buffer++;
+				rc = ibmveth_rxq_harvest_buffer(adapter,
+								queue_index,
+								true);
+				if (unlikely(rc))
+					break;
+				continue;
+			}
+
+			if (unlikely((unsigned int)offset +
+				     (unsigned int)length >
+				     skb_tailroom(skb))) {
+				if (net_ratelimit())
+					netdev_err(netdev,
+						   "RX frame %u+%u exceeds buffer %u on queue %d, dropping\n",
+						   offset, length,
+						   skb_tailroom(skb),
+						   queue_index);
+				if (adapter->rx_qstats)
+					adapter->rx_qstats[queue_index]
+						.invalid_buffers++;
+				else
+					adapter->rx_invalid_buffer++;
+				rc = ibmveth_rxq_harvest_buffer(adapter,
+								queue_index,
+								true);
+				if (unlikely(rc))
+					break;
+				continue;
+			}
 
 			/* if the large packet bit is set in the rx queue
 			 * descriptor, the mss will be written by PHYP eight
@@ -3206,8 +3262,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 
 	ibmveth_replenish_task(adapter, queue_index);
 
-	if (frames_processed == budget)
+	if (frames_processed == budget) {
+		if (!netif_running(netdev) || napi_disable_pending(napi)) {
+			napi_complete_done(napi, frames_processed);
+			/* After complete_done, must not return full budget. */
+			return frames_processed ? frames_processed - 1 : 0;
+		}
 		goto out;
+	}
 
 	if (!napi_complete_done(napi, frames_processed))
 		goto out;
@@ -3225,6 +3287,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
 	}
 
 	if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+	    netif_running(netdev) &&
+	    !napi_disable_pending(napi) &&
 	    napi_schedule(napi)) {
 		lpar_rc = ibmveth_disable_irq(adapter, queue_index);
 		WARN_ON(lpar_rc != H_SUCCESS);
@@ -4014,8 +4078,7 @@ static void ibmveth_reset_kunit(struct work_struct *w)
  * @test: pointer to kunit structure
  *
  * Tests the error returns from ibmveth_remove_buffer_from_pool.
- * ibmveth_remove_buffer_from_pool also calls WARN_ON, so dmesg should be
- * checked to see that these warnings happened.
+ * Bad correlators return -EINVAL/-EFAULT (no WARN_ON).
  *
  * Return: void
  */
@@ -4071,9 +4134,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
  * ibmveth_rxq_get_buffer_test - unit test for ibmveth_rxq_get_buffer
  * @test: pointer to kunit structure
  *
- * Tests ibmveth_rxq_get_buffer. ibmveth_rxq_get_buffer also calls WARN_ON for
- * the NULL returns, so dmesg should be checked to see that these warnings
- * happened.
+ * Tests ibmveth_rxq_get_buffer invalid correlator returns NULL without WARN.
  *
  * Return: void
  */
-- 
2.50.1 (Apple Git-155)

Re: [PATCH net-next v4 01/14] ibmveth: Add MQ RX hypercall wrappers and call definitions

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add MQ RX hypercall wrappers and call definitions

This adds three new PHYP call IDs for multi-queue receive support and four
static inline hcall wrapper helpers in ibmveth.h, bumping MAX_HCALL_OPCODE
to the highest new opcode. No callers are added by this patch.

A couple of questions below about the new wrappers and about the hvcall.h
hunk.

> diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
> index dff90a7d7f70b..d79baf2e38273 100644
> --- a/arch/powerpc/include/asm/hvcall.h
> +++ b/arch/powerpc/include/asm/hvcall.h
> @@ -362,7 +362,10 @@
>  #define H_GUEST_DELETE		0x488
>  #define H_PKS_WRAP_OBJECT	0x490
>  #define H_PKS_UNWRAP_OBJECT	0x494
> -#define MAX_HCALL_OPCODE	H_PKS_UNWRAP_OBJECT
> +#define H_REG_LOGICAL_LAN_QUEUE 0x49C
> +#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0
> +#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8
> +#define MAX_HCALL_OPCODE	H_FREE_LOGICAL_LAN_QUEUE

[Severity: Low]
Could the changelog cite the PAPR/PHYP document and revision that assigns
0x49C, 0x4A0 and 0x4A8?  These become architecture-wide constants shared
with the KVM hcall emulation, so it would help to have the provenance
recorded in the commit.

There are also holes left at 0x498 and at 0x4A4 (the latter from dropping
H_FREE_LOGICAL_LAN_BUFFER_QUEUE in v4).  Are those numbers reserved by the
architecture for something else, or are they simply unused?

Separately, MAX_HCALL_OPCODE is not driver-local.  It sizes
kvm_arch.enabled_hcalls:

  arch/powerpc/include/asm/kvm_host.h:
      DECLARE_BITMAP(enabled_hcalls, MAX_HCALL_OPCODE/4 + 1);

and it bounds the userspace-visible opcode range accepted by
KVM_CAP_PPC_ENABLE_HCALL:

  arch/powerpc/kvm/powerpc.c:kvm_vm_ioctl_enable_cap() {
      ...
      if (hcall > MAX_HCALL_OPCODE || (hcall & 3) ||
          cap->args[1] > 1)
      ...
  }

It also sizes the per-CPU hcall statistics array in
arch/powerpc/platforms/pseries/hvCall_inst.c and gates
kvmppc_pseries_do_hcall() and kvmppc_h_pr().  As far as I can tell the bump
is behaviourally inert for these new opcodes, since kvm_vm_ioctl_enable_cap()
still rejects them via kvmppc_book3s_hcall_implemented(), and the KVM
dispatch path ends up returning RESUME_HOST either way.  Would it be worth
mentioning the cross-subsystem effect in the changelog so the powerpc/KVM
side has something to ack against?

>  
>  /* Scope args for H_SCM_UNBIND_ALL */
>  #define H_UNBIND_SCOPE_ALL (0x1)
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index d87713668ed30..c88dbeb7cd967 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -66,6 +66,143 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address,
>  			    desc5, desc6, desc7, desc8);
>  }
>  
> +/**
> + * h_reg_logical_lan_queue - Register a subordinate receive queue

[Severity: Low]
This isn't a bug, but the naming here diverges from the existing wrappers in
this file, which all spell it out as h_register_logical_lan() and
h_free_logical_lan().  Would h_register_logical_lan_queue() read more
consistently?

[ ... ]

> +static inline long h_reg_logical_lan_queue(unsigned long unit_address,
> +					   unsigned long buffer_list,
> +					   unsigned long rec_queue,
> +					   unsigned long *queue_handle,
> +					   unsigned long *irq)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^

[Severity: Low]
This helper returns the queue handle as unsigned long *, which matches
retbuf's element type and matches the other output parameters in this header
(h_send_logical_lan()'s corellator_out, h_illan_attributes()'s
ret_attributes).

But h_register_logical_lan_with_handle() below declares the same conceptual
value, the PHYP queue handle read out of retbuf[0], as u64 *.  Can these two
be made to agree?  As it stands, callers have to pick a different storage
type depending on which of the two helpers they call, even though both are
returning the same thing.

The commit message says the new wrappers have "argument ordering and return
semantics matching the existing ibmveth hcall wrappers", which holds for
this one but not for the u64 * variant.

[ ... ]

> +static inline long h_free_logical_lan_queue(unsigned long unit_address,
> +					    unsigned long queue_handle)
> +{
> +	unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
> +
> +	return plpar_hcall9(H_FREE_LOGICAL_LAN_QUEUE,
> +			    retbuf, unit_address, queue_handle);
> +}

[Severity: Low]
The kerneldoc above documents no output data for this hcall, and retbuf is
never read.  Why not use plpar_hcall_norets() here, the way the existing
counterpart in this file does?

  #define h_free_logical_lan(ua) \
    plpar_hcall_norets(H_FREE_LOGICAL_LAN, ua)

That would drop the dead nine-entry stack buffer.

[ ... ]

> +static inline long
> +h_register_logical_lan_with_handle(unsigned long unit_address,
> +				   unsigned long buffer_list,
> +				   unsigned long rec_queue,
> +				   unsigned long filter_list,
> +				   unsigned long mac_address,
> +				   u64 *queue_handle)
                                    ^^^^^^^^^^^^^^^^^^

[Severity: Low]
This is the u64 * side of the inconsistency noted above.  Both types are
64-bit on ppc64, which is the only configuration that builds ibmveth, and
the callers added later in the series do compile against both prototypes, so
this looks like a readability issue rather than a functional one.  Could both
helpers use unsigned long * for the handle?

> +{
> +	unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
> +	long rc;
> +
> +	rc = plpar_hcall9(H_REGISTER_LOGICAL_LAN, retbuf,
> +			  unit_address, buffer_list, rec_queue,
> +			  filter_list, mac_address);
> +
> +	if (rc == H_SUCCESS && queue_handle)
> +		*queue_handle = retbuf[0];
> +
> +	return rc;
> +}

Thanks.
-- 
pw-bot: cr

Re: [PATCH net-next v4 02/14] ibmveth: Prepare MQ RX adapter data structures

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Prepare MQ RX adapter data structures

This converts the single-instance RX state in struct ibmveth_adapter
(napi, rx_queue, buffer_list_addr/dma, rx_buff_pool) into queue-indexed
arrays sized by the new IBMVETH_MAX_RX_QUEUES, adds multi_queue and
num_rx_queues, and mechanically rewrites all existing references to
index 0.  IBMVETH_MAX_RX_QUEUES stays at 1 in this patch.

A couple of observations below, neither of which changes runtime
behaviour today.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 73e051d26b9d8..8e758362cb263 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -482,42 +486,44 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
>  	mb();
>  
> -	atomic_dec(&(adapter->rx_buff_pool[pool].available));
> +	atomic_dec(&adapter->rx_buff_pool[0][pool].available);
>  
>  	return 0;
>  }
>  
>  /* get the current buffer on the rx queue */
> -static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
> +static inline struct sk_buff *
> +ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)

[Severity: Low]
This isn't a bug, but the patch carries a few edits that are not part of
the stated mechanical index-0 conversion:

  - dropping the redundant parentheses in the atomic_dec() call in
    ibmveth_remove_buffer_from_pool()
  - reflowing the ibmveth_rxq_get_buffer() prototype
  - rewriting the kernel-doc of ibmveth_get_desired_dma() from
    "Return value:" to "Return:"
  - inserting a blank line in struct ibmveth_adapter after
    rx_large_packets

The commit message says:

    This patch keeps behavior unchanged by mechanically switching existing
    references to index 0

Could these cosmetic edits move to a separate cleanup patch, or be
mentioned in the changelog?  As it stands each hunk has to first be
classified as conversion or cleanup before the "behavior unchanged" claim
can be confirmed.

[ ... ]

> @@ -1693,8 +1704,7 @@ static void ibmveth_poll_controller(struct net_device *dev)
>   *
>   * @vdev: struct vio_dev for the device whose desired IO mem is to be returned
>   *
> - * Return value:
> - *	Number of bytes of IO data the driver will need to perform well.
> + * Return: Number of bytes of IO data the driver will need to perform well.
>   */
>  static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
>  {

[ ... ]

> @@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  		memcpy(pool_count, pool_count_cmo, sizeof(pool_count));
>  
>  	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> -		struct kobject *kobj = &adapter->rx_buff_pool[i].kobj;
> +		struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
>  		int error;
>  
> -		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
> +		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
>  					 pool_count[i], pool_size[i],
>  					 pool_active[i]);
>  		error = kobject_init_and_add(kobj, &ktype_veth_pool,

[ ... ]

> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index c88dbeb7cd967..8eaa3942fc40b 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h

[ ... ]

> @@ -294,18 +296,23 @@ struct ibmveth_rx_q {
> -	struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS];
> -	struct ibmveth_rx_q rx_queue;
> +	struct ibmveth_buff_pool
> +		rx_buff_pool[IBMVETH_MAX_RX_QUEUES][IBMVETH_NUM_BUFF_POOLS];
> +	struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES];

[Severity: Low]
Is replicating the whole struct ibmveth_buff_pool per queue the layout you
want here?

struct ibmveth_buff_pool embeds a struct kobject, which is the per-pool
sysfs tuning object.  Only row 0's kobjects are ever initialized and
registered, in ibmveth_probe():

	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
		struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
		...
		error = kobject_init_and_add(kobj, &ktype_veth_pool,
					     &dev->dev.kobj, "pool%d", i);

and only row 0's are dropped, in ibmveth_remove():

	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
		kobject_put(&adapter->rx_buff_pool[0][i].kobj);

So every row above 0 carries a kobject that is never initialized and never
used.

Later in the series IBMVETH_MAX_RX_QUEUES is raised to 16U, at which point
the netdev private area unconditionally holds 16 x 5 pool structs
regardless of num_rx_queues, of which 75 embedded kobjects are dead
weight.

The follow-on code also shows that only part of the struct is really
per-queue: ibmveth_alloc_single_rx_queue() copies size, index, buff_size,
threshold and active from row 0 into each new row, so the pool
configuration is shared while free_map/dma_addr/skbuff/producer_index/
consumer_index/available are the genuinely per-queue state.

Would it be cleaner to split the struct into one shared, sysfs-visible
configuration object plus a small per-queue state array, given this patch
is the one that fixes the layout for the rest of the series?

Is it also intentional that the per-pool sysfs tuning interface now
implicitly means "queue 0 configures all queues"?  If so, could that be
stated in the changelog?

> @@ -325,6 +332,7 @@ struct ibmveth_adapter {
>  	u64 tx_send_failed;
>  	u64 tx_large_packets;
>  	u64 rx_large_packets;
> +
>  	/* Ethtool settings */
>  	u8 duplex;
>  	u32 speed;

Re: [PATCH net-next v4 03/14] ibmveth: Refactor RX resource allocation for MQ RX bring-up

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX resource allocation for MQ RX bring-up

This moves the filter list and per-queue RX ring allocation out of
ibmveth_open() into four new helpers, and wires them into both the
open error unwind and ibmveth_close().  Runtime behaviour is meant to
stay single-queue since num_rx_queues is still 1.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 8e758362cb263..1007dd95cde02 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -151,6 +151,184 @@ static unsigned int ibmveth_real_max_tx_queues(void)
>  	return min(n_cpu, IBMVETH_MAX_QUEUES);
>  }
>  
> +/**
> + * ibmveth_alloc_filter_list - Allocate and map filter list
> + * @adapter: ibmveth adapter structure
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
> +{
> +	struct device *dev = &adapter->vdev->dev;
> +	struct net_device *netdev = adapter->netdev;
> +
> +	adapter->filter_list_addr = (void *)get_zeroed_page(GFP_KERNEL);
> +	if (!adapter->filter_list_addr) {
> +		netdev_err(netdev, "unable to allocate filter pages\n");
> +		return -ENOMEM;
> +	}
> +
> +	adapter->filter_list_dma = dma_map_single(dev,
> +						  adapter->filter_list_addr,
> +						  4096, DMA_BIDIRECTIONAL);
> +	if (dma_mapping_error(dev, adapter->filter_list_dma)) {
> +		netdev_err(netdev, "unable to map filter list pages\n");
> +		free_page((unsigned long)adapter->filter_list_addr);
> +		adapter->filter_list_addr = NULL;
> +		return -ENOMEM;
> +	}

[Severity: Medium]

Should this error path also clear adapter->filter_list_dma?  On failure
dma_map_single() has stored DMA_MAPPING_ERROR, which is ~(dma_addr_t)0 and
therefore non-zero, and only filter_list_addr is reset here.

The sibling helper added in the same patch does zero the cookie in the
same situation:

	if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
		...
		adapter->buffer_list_dma[i] = 0;
		goto err_cleanup;
	}

so the two new helpers appear to disagree on the convention.

> +
> +	netdev_dbg(netdev, "filter list @ 0x%p (DMA: 0x%llx)\n",
> +		   adapter->filter_list_addr,
> +		   (unsigned long long)adapter->filter_list_dma);
> +
> +	return 0;
> +}
> +
> +/**
> + * ibmveth_free_filter_list - Free filter list resources
> + * @adapter: ibmveth adapter structure
> + */
> +static void
> +ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
> +{
> +	struct device *dev = &adapter->vdev->dev;
> +
> +	if (adapter->filter_list_dma) {
> +		dma_unmap_single(dev, adapter->filter_list_dma, 4096,
> +				 DMA_BIDIRECTIONAL);
> +		adapter->filter_list_dma = 0;
> +	}

[Severity: Medium]

Can this unmap the DMA_MAPPING_ERROR sentinel?  ibmveth_free_filter_list()
treats any non-zero filter_list_dma as a live mapping, and the mapping-error
branch of ibmveth_alloc_filter_list() leaves the sentinel behind.

The state looks reachable without dev_close().  ibmveth_set_csum_offload(),
ibmveth_set_tso(), ibmveth_change_mtu() and veth_pool_store() all call
ibmveth_close() and then ibmveth_open() directly while netif_running() is
still true:

	if (netif_running(dev)) {
		restart = 1;
		ibmveth_close(dev);
	}
	...
	if (restart)
		rc2 = ibmveth_open(dev);

If that re-open fails at the dma_mapping_error() check in
ibmveth_alloc_filter_list(), open() returns through the out: label without
calling ibmveth_free_filter_list(), so the adapter is left with
filter_list_addr == NULL and filter_list_dma == DMA_MAPPING_ERROR while the
netdev is still marked running.  A later "ip link set dev ethX down" then
reaches ndo_stop == ibmveth_close() -> ibmveth_free_filter_list() and unmaps
that invalid address.

On pseries the dma_iommu path catches this in iommu_free_check(), which emits
a ratelimited "iommu_free: invalid entry" message and a WARN_ON(1) (a panic
with panic_on_warn set), so no TCE state is changed, but the backtrace is
still user triggerable with CAP_NET_ADMIN plus a mapping failure.

Setting adapter->filter_list_dma = 0 in that error branch, matching
ibmveth_alloc_rx_queues(), would make the guard here reliable.

This is still present unchanged at the end of the series (a3781f4ae789), so
a later patch does not appear to address it.

> +
> +	if (adapter->filter_list_addr) {
> +		free_page((unsigned long)adapter->filter_list_addr);
> +		adapter->filter_list_addr = NULL;
> +	}
> +}
> +

[ ... ]

> +		adapter->buffer_list_dma[i] =
> +			dma_map_single(dev, adapter->buffer_list_addr[i],
> +				       4096, DMA_BIDIRECTIONAL);
> +		if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
> +			netdev_err(netdev,
> +				   "unable to map buffer list for queue %d\n",
> +				   i);
> +			adapter->buffer_list_dma[i] = 0;
> +			goto err_cleanup;
> +		}
> +

[ ... ]

> @@ -627,74 +805,34 @@ static int ibmveth_open(struct net_device *netdev)
>  
> -	rc = -ENOMEM;
> -	adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
> -	if (!adapter->buffer_list_addr[0]) {
> -		netdev_err(netdev, "unable to allocate list pages\n");
> +	rc = ibmveth_alloc_filter_list(adapter);
> +	if (rc)
>  		goto out;
> -	}

[ ... ]

> @@ -753,26 +889,12 @@ static int ibmveth_open(struct net_device *netdev)
> +	ibmveth_cleanup_rx_resources(adapter);
>  out_free_filter_list:
> -	free_page((unsigned long)adapter->filter_list_addr);
> -out_free_buffer_list:
> -	free_page((unsigned long)adapter->buffer_list_addr[0]);
> +	ibmveth_free_filter_list(adapter);
>  out:
>  	napi_disable(&adapter->napi[0]);
>  	return rc;

[ ... ]

Re: [PATCH net-next v4 04/14] ibmveth: Refactor buffer pool management for per-queue MQ RX

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor buffer pool management for per-queue MQ RX

This reworks the five RX buffer pools into per-queue arrays
(rx_buff_pool[queue][pool]) with new alloc/free helpers, wires those
helpers into ibmveth_open()/ibmveth_close(), and clones pool geometry from
queue 0 to queues 1..N.  It also flips the default active flag of the
64 KiB pool to 0.

A few questions below.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 1007dd95cde02..7a2ed49cad4f1 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -625,6 +625,145 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
>  	}
>  }
>  
> +/**
> + * ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue
> + * @adapter: ibmveth adapter structure
> + * @queue: queue index
> + *
> + * Allocates all active buffer pools for the specified queue.
> + * Pool metadata must be initialized before calling this function.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int ibmveth_alloc_queue_buffer_pools(struct ibmveth_adapter *adapter,
> +					    int queue)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +	int i;
> +
> +	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> +		struct ibmveth_buff_pool *bpool =
> +			&adapter->rx_buff_pool[queue][i];
> +
> +		if (!bpool->active)
> +			continue;
> +
> +		if (ibmveth_alloc_buffer_pool(bpool)) {
> +			netdev_err(netdev,
> +				   "pool %d/%d alloc failed (size=%u count=%u)\n",
> +				   i, queue,
> +				   bpool->buff_size,
> +				   bpool->size);
> +			bpool->active = 0;
> +
> +			/* Free pools allocated so far for this queue */
> +			while (--i >= 0) {
> +				struct ibmveth_buff_pool *fpool =
> +					&adapter->rx_buff_pool[queue][i];
> +
> +				if (fpool->active)
> +					ibmveth_free_buffer_pool(adapter,
> +								 fpool);
> +			}

[Severity: Low]

Is the free predicate here meant to differ from the one used by the sibling
helper added in the same patch?

This unwind path keys off fpool->active, while
ibmveth_free_queue_buffer_pools() deliberately frees by allocation
presence:

	if (pool->free_map || pool->dma_addr || pool->skbuff)
		ibmveth_free_buffer_pool(adapter, pool);

and the changelog states "free paths release by real allocations
(free_map/dma_addr/skbuff), not only pool->active".

So on the queue that fails, a pool holding allocations with active == 0 is
never released, which is exactly the case the allocation-based predicate
was added for.  Since a failing ndo_open means ibmveth_close() is never
called, would those free_map/dma_addr/skbuff arrays be leaked?

ibmveth_free_buffer_pool() already NULLs all three pointers and is
idempotent, so could the unwind just call ibmveth_free_queue_buffer_pools()
on the same queue (or use the same predicate)?  The same helper is reused
by the incremental resize path later in the series, where pools can hold
memory while active is clear.

> +			return -ENOMEM;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +/**
> + * ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue
> + * @adapter: ibmveth adapter structure
> + * @queue: queue index
> + *
> + * Frees all active buffer pools for the specified queue.
       ^^^^^^^^^^^^^^^^^^^^^^^^^^

[Severity: Low]

The kernel-doc summary says "all active buffer pools", but the body
intentionally ignores the active flag and frees on allocation presence, and
the in-body comment says the opposite of the summary.  Could the summary be
reworded to say pools that have allocations?  Later MQ patches reuse this
helper and depend on that distinction.

> + */
> +static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
> +					    int queue)
> +{
> +	int i;
> +
> +	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> +		struct ibmveth_buff_pool *pool =
> +			&adapter->rx_buff_pool[queue][i];
> +
> +		/* Free pool if it has allocated memory, regardless of
> +		 * active flag. Pools may have memory allocated but not
> +		 * marked active during queue scale-up, so we must check
> +		 * for actual allocations.
> +		 */
> +		if (pool->free_map || pool->dma_addr || pool->skbuff)
> +			ibmveth_free_buffer_pool(adapter, pool);
> +	}
> +}
> +

[ ... ]

> +static int
> +ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +	int i, q, rc;
> +
> +	/* Initialize pool metadata for queues 1-15 from queue 0 settings */
                                            ^^^^^^^^^^

[Severity: Low]

IBMVETH_MAX_RX_QUEUES is 1U at this commit, so queues 1-15 do not exist yet
and this loop body cannot run.  Should the comment say "queues 1..N" until
the bound is raised later in the series?

> +	for (q = 1; q < adapter->num_rx_queues; q++) {
> +		for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> +			struct ibmveth_buff_pool *src =
> +				&adapter->rx_buff_pool[0][i];
> +			struct ibmveth_buff_pool *dst =
> +				&adapter->rx_buff_pool[q][i];
> +
> +			dst->size = src->size;
> +			dst->index = src->index;
> +			dst->buff_size = src->buff_size;
> +			dst->threshold = src->threshold;
> +			dst->active = src->active;
> +		}
> +	}

[Severity: Low]

This turns the per-pool sysfs knobs into a per-queue template.  Is that
intended to stay undocumented?

Only queue 0 gets sysfs objects, in ibmveth_probe():

	struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
	...
	error = kobject_init_and_add(kobj, &ktype_veth_pool, &dev->dev.kobj,
				     "pool%d", i);

and veth_pool_show()/veth_pool_store() read and write that single instance.
After this change the queue-0 values are cloned above and each queue
allocates its own copy, so writing num = 4096 to one pool multiplies the
real allocation by adapter->num_rx_queues while sysfs still reports the
single-queue value.

The multiplier is 1 everywhere in this series (IBMVETH_DEFAULT_RX_QUEUES is
1), so nothing changes today.  Should these knobs stay adapter-wide (counts
divided across queues), gain per-queue nodes, or should the multiplier be
documented?

> +
> +	/* Allocate actual buffers for all queues */
> +	for (q = 0; q < adapter->num_rx_queues; q++) {
> +		rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
> +		if (rc) {
> +			/* Free pools for all previous queues */
> +			while (--q >= 0)
> +				ibmveth_free_queue_buffer_pools(adapter, q);
> +			return rc;
> +		}
> +	}
> +
> +	netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
> +		   adapter->num_rx_queues);
> +	return 0;
> +}
> +

[ ... ]

> @@ -927,14 +1055,10 @@ static int ibmveth_close(struct net_device *netdev)
>  
>  	ibmveth_update_rx_no_buffer(adapter);
>  
> +	ibmveth_free_buffer_pools(adapter);
>  	ibmveth_cleanup_rx_resources(adapter);
>  	ibmveth_free_filter_list(adapter);
>  
> -	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
> -		if (adapter->rx_buff_pool[0][i].active)
> -			ibmveth_free_buffer_pool(adapter,
> -						 &adapter->rx_buff_pool[0][i]);
> -

[Severity: Low]

The teardown order changed here, which the changelog does not mention (it
only says the helpers are wired into open()/close()).

Baseline:

	ibmveth_update_rx_no_buffer()
	ibmveth_cleanup_rx_resources()
	ibmveth_free_filter_list()
	free active pools

New:

	ibmveth_update_rx_no_buffer()
	ibmveth_free_buffer_pools()
	ibmveth_cleanup_rx_resources()
	ibmveth_free_filter_list()

h_free_logical_lan(), napi_disable() and free_irq() have all run by this
point, so neither PHYP nor NAPI still references the pool buffers or the RX
queue, and the swap looks safe.  Could the reorder be called out in the
changelog, since a future per-queue drain may need the RX completion queue
still present while pools are released?

>  	for (i = 0; i < netdev->real_num_tx_queues; i++)
>  		ibmveth_free_tx_ltb(adapter, i);
>  
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index 8eaa3942fc40b..abcd4054c67e2 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -265,7 +265,7 @@ static inline long h_illan_attributes(unsigned long unit_address,
>  static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
>  static int pool_count[] = { 256, 512, 256, 256, 256 };
>  static int pool_count_cmo[] = { 256, 512, 256, 256, 64 };
> -static int pool_active[] = { 1, 1, 0, 0, 1};
> +static int pool_active[] = { 1, 1, 0, 0, 0};
                                          ^^^

[Severity: High]

Does this drop large-receive support for existing single-queue users?

Pools 2 (16 KiB) and 3 (32 KiB) are already inactive by default, so with
the 64 KiB pool off the largest buffer ever posted at MTU 1500 becomes
2048 bytes.  ibmveth_probe() seeds each pool from this array:

	ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
				 pool_count[i], pool_size[i], pool_active[i]);

and the new helper skips inactive pools, so nothing larger is handed to
PHYP:

	if (!bpool->active)
		continue;

ibmveth_poll() does receive hypervisor-aggregated frames longer than the
MTU:

	if ((length > netdev->mtu + ETH_HLEN) || lrg_pkt ||
	    iph_check == 0xffff) {
		ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
		adapter->rx_large_packets++;
	}

Those arrive as single non-scatter buffers, and TSO/large-send is enabled
by default in ibmveth_probe() when firmware reports
IBMVETH_ILLAN_LRG_SND_SUPPORT.  With no buffer larger than 2 KiB posted,
are such frames simply dropped by PHYP and counted in the no-buffer counter
read by ibmveth_update_rx_no_buffer()?

The changelog says:

    "MTU changes activate it when required; leaving it enabled would pin
    about 16 MiB per RX queue in MQ mode."

Is the first half accurate?  ibmveth_change_mtu() activates pools only up
to the first pool whose buff_size covers the new MTU:

	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
		adapter->rx_buff_pool[0][i].active = 1;
		if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size) {
			...
			return 0;
		}
	}

At MTU 1500 that stops at the 2 KiB pool, so the 64 KiB pool is never
re-enabled; ibmveth_set_tso() does not touch pool->active either.  Recovery
then requires a manual write to pool4/active in sysfs.

The second half describes MQ, but IBMVETH_DEFAULT_RX_QUEUES is 1 both at
this commit and at the end of the series (a3781f4ae789 still has
pool_active[] = { 1, 1, 0, 0, 0}), so the only configurations affected are
today's single-queue ones.  Would it be better to keep the default at 1 and
scale the 64 KiB count (or deactivate it) when RX queues are actually
scaled up?

Note also that commit cd7c7ec3687986 ("ibmveth: change rx buffer default
allocation for CMO") enabled this pool on purpose and added
pool_count_cmo[] = { 256, 512, 256, 256, 64 } specifically to reduce the
64 KiB count under CMO, which suggests the default was deliberate.

As a side effect, ibmveth_get_desired_dma() sums only active pools:

	if (adapter->rx_buff_pool[0][i].active)
		ret += adapter->rx_buff_pool[0][i].size *
		       IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].buff_size,
					tbl);

so does the CMO entitlement the driver requests shrink silently as well?

>  
>  #define IBM_VETH_INVALID_MAP ((u16)0xffff)
>

Re: [PATCH net-next v4 05/14] ibmveth: Refactor RX interrupt control for MQ RX queues

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX interrupt control for MQ RX queues

This patch introduces queue-indexed RX interrupt helpers (toggle/enable/
disable, setup/cleanup, schedule) and wires them into open(), close(),
poll(), the IRQ handler and the process-context RX kick paths. request_irq()
now passes &adapter->napi[i] as the cookie so the handler can derive the
queue index, and napi_enable()/napi_disable() move into the new setup and
cleanup helpers. Runtime behaviour stays single-queue.

A few questions below.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 7a2ed49cad4f1..664169c4d27a4 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -329,6 +331,203 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
>  	}
>  }
>  
> +/**
> + * ibmveth_toggle_irq - Common helper to enable/disable queue interrupts
> + * @adapter: ibmveth adapter structure
> + * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
> + * @enable: true to enable, false to disable
> + *
> + * For queue 0 (primary), uses h_vio_signal() as it's registered via
> + * h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL
> + * with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control.
> + *
> + * Return: 0 on success, error code otherwise
> + */
> +static int
> +ibmveth_toggle_irq(struct ibmveth_adapter *adapter, int queue_index,
> +		   bool enable)
> +{
> +	unsigned long rc;
> +	unsigned long irq = adapter->queue_irq[queue_index];
> +	const char *action = enable ? "enable" : "disable";
> +
> +	if (queue_index == 0) {
> +		/* Primary queue: use h_vio_signal() */
> +		rc = h_vio_signal(adapter->vdev->unit_address,
> +				  enable ? VIO_IRQ_ENABLE : VIO_IRQ_DISABLE);
> +	} else {
> +		/* Subordinate queues: use H_VIOCTL with hardware IRQ */
> +		struct irq_data *irq_data = irq_get_irq_data(irq);
> +		irq_hw_number_t hwirq;
> +		u64 vioctl_cmd = enable ? H_ENABLE_VIO_INTERRUPT :
> +			H_DISABLE_VIO_INTERRUPT;
> +
> +		if (!irq_data) {
> +			netdev_err(adapter->netdev,
> +				   "Failed to get IRQ data for queue %d (virq=%lu)\n",
> +				   queue_index, irq);
> +			return -EINVAL;
> +		}
> +
> +		hwirq = irqd_to_hwirq(irq_data);
> +		rc = plpar_hcall_norets(H_VIOCTL,
> +					adapter->vdev->unit_address,
> +					vioctl_cmd,
> +					hwirq, 0, 0);
> +
> +		if (rc == H_PARAMETER) {
> +			/* H_PARAMETER is non-fatal when IRQ is already in
> +			 * the requested state.
> +			 */
> +			netdev_warn_once(adapter->netdev,
> +					 "H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n",
> +					 action, queue_index, hwirq);
> +			return 0;
> +		}

[Severity: Medium]
Is folding H_PARAMETER into success correct for both directions here?

hvcall.h documents H_PARAMETER as "Parameter invalid, out-of-range or
conflicting", and ibmveth_toggle_irq() cannot distinguish "already in the
requested state" from a bad hwirq, wrong unit address, or a queue that is no
longer registered.

In the enable direction the swallowed status bypasses the recovery in
ibmveth_poll():

	lpar_rc = ibmveth_enable_irq(adapter, 0);
	if (WARN_ON(lpar_rc != H_SUCCESS)) {
		schedule_work(&adapter->work);
		goto out;
	}

so NAPI can complete with the queue still masked at the hypervisor and
nothing left to reschedule it, i.e. that queue stops receiving, with a
single warn_once as the only trace.

In the disable direction, ibmveth_schedule_rx_queue() goes on to
__napi_schedule() and ibmveth_cleanup_rx_interrupts() goes on to
synchronize_irq()/free_irq() while believing delivery is masked.

For comparison, ibmvnic treats any non-zero rc from the same hypercalls as
an error in both directions, in disable_scrq_irq() and enable_scrq_irq().

The branch is dead in this commit (num_rx_queues is 1), but it becomes live
once the MQ patches land, where setup, resize and poll all depend on this
return value. Would restricting the folding to the disable direction, plus a
rate-limited per-queue message instead of netdev_warn_once(), be closer to
the intent?

> +	}
> +
> +	if (rc)
> +		netdev_err(adapter->netdev,
> +			   "Failed to %s IRQ for queue %d, rc=%ld\n",
> +			   action, queue_index, rc);
> +	return rc;
> +}

[Severity: Medium]
Does this mix two error domains in one int return?

rc holds an unsigned long hypervisor status from h_vio_signal() or
plpar_hcall_norets() (H_BUSY 1, H_HARDWARE -1, H_FUNCTION -2, H_PARAMETER
-4), truncated to int on return, while the irq_get_irq_data() failure path
returns -EINVAL.

The callers then disagree about what they got. ibmveth_schedule_rx_queue()
and ibmveth_poll() do:

	unsigned long lpar_rc;
	lpar_rc = ibmveth_disable_irq(adapter, qindex);
	WARN_ON(lpar_rc != H_SUCCESS);

while ibmveth_setup_rx_interrupts()/ibmveth_open() treat the same value as
an errno. One printk formats it with %ld, the other site logs it with %d.

By the end of the series this reaches userspace: the scale-up loop in
ibmveth_resize_rx_queues_incremental() does

	rc = ibmveth_enable_irq(adapter, i);
	if (rc) { ... goto cleanup_new_queues; }

and cleanup_new_queues ends with return rc, which flows through
ibmveth_resize_rx_channels() -> ibmveth_set_channels() to ethtool. H_PARAMETER
(-4) then surfaces as -EINTR, and H_BUSY (1) or H_CLOSED (2) as a positive
return that ethtool reads as success even though the queue count was rolled
back.

Since this commit establishes the helper API for the whole series, would it
make sense to settle the convention now, either normalizing to errno or
keeping the raw hcall status with a matching type, name, kernel-doc and
format specifier?

[ ... ]

> +/**
> + * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
> + * @adapter: ibmveth adapter structure
> + *
> + * Registers interrupt handlers for all RX queues and enables NAPI polling.
> + * On error, cleans up any successfully registered IRQs before returning.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +	int i, rc;
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++) {
> +		if (!adapter->queue_irq[i]) {
> +			netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
> +			rc = -EINVAL;
> +			goto err_free_irqs;
> +		}
> +
> +		rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt,
> +				 0, netdev->name, &adapter->napi[i]);
> +		if (rc) {
> +			netdev_err(netdev,
> +				   "request_irq() failed for irq 0x%x queue %d: %d\n",
> +				   adapter->queue_irq[i], i, rc);
> +			goto err_free_irqs;
> +		}
> +	}
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++)
> +		napi_enable(&adapter->napi[i]);

[Severity: Medium]
Can an interrupt arrive in the window between these two loops?

request_irq() unmasks the virq via irq_startup(), and the LAN is already
registered with the hypervisor by the time ibmveth_setup_rx_interrupts()
runs, yet NAPI is only enabled in the second loop. Previously
napi_enable(&adapter->napi[0]) was the first statement of ibmveth_open(), so
this window did not exist.

If an interrupt (including one latched at the controller while the line was
shut down by the previous close's free_irq()) is delivered inside the
window, ibmveth_interrupt() -> ibmveth_schedule_rx_queue() finds
napi_schedule_prep() failing:

	if (napi_schedule_prep(napi)) {
		lpar_rc = ibmveth_disable_irq(adapter, qindex);
		WARN_ON(lpar_rc != H_SUCCESS);
		__napi_schedule(napi);
	}

so nothing is scheduled and nothing is masked, while the handler still
returns IRQ_HANDLED. For a level-asserted hypervisor source that repeats
until napi_enable() runs.

This is also the inverse of the invariant the later scale-up path in this
series states, "publish -> replenish -> napi_enable -> enable_irq ... so
ibmveth_interrupt() cannot run on an unpublished, empty, or NAPI-disabled
queue". Would enabling a queue's NAPI before installing its handler restore
that ordering?

> +
> +	return 0;
> +
> +err_free_irqs:
> +	while (--i >= 0)
> +		free_irq(adapter->queue_irq[i], &adapter->napi[i]);
> +	return rc;
> +}
> +
> +/**
> + * ibmveth_cleanup_rx_interrupts - Mask PHYP, disable NAPI, free IRQs
> + * @adapter: ibmveth adapter structure
> + *
> + * Tears down RX interrupt delivery for all queues. Mask PHYP before
> + * napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without
> + * masking (same storm window as scale-down). Safe for close and for
> + * open failure after setup_rx_interrupts() already unmasked PHYP.
> + */

[Severity: Low]
Do these comments describe what the commit actually does?

"Safe for close and for open failure after setup_rx_interrupts() already
unmasked PHYP" - ibmveth_setup_rx_interrupts() only does request_irq() and
napi_enable(), and the changelog agrees ("setup does not yet unmask PHYP").
No ibmveth_open() failure path calls this helper either; setup rolls back
through its own err_free_irqs label and open goes to
out_free_buffer_pools/out_free_tx_ltb/out.

"Mask PHYP before napi_disable so ibmveth_interrupt cannot return
IRQ_HANDLED without masking" - see the question on the mask/napi_disable
ordering below; an in-flight ibmveth_poll() can undo the mask.

"Dispose IRQ mappings for subordinate queues (1-15)" sits above a loop that
starts at 1, but ibmveth.h defines IBMVETH_MAX_RX_QUEUES as 1U and
num_rx_queues is set to IBMVETH_DEFAULT_RX_QUEUES, so the loop cannot
iterate in this commit.

And ibmveth_toggle_irq()'s "Return: 0 on success, error code otherwise"
conflicts with returning raw H_* codes that callers compare against
H_SUCCESS.

The first and third become accurate later in the series; the second and
fourth remain as they are. Could the comments be trimmed to what this
commit provides?

> +static void
> +ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
> +{
> +	int i;
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++) {
> +		if (adapter->queue_irq[i]) {
> +			ibmveth_disable_irq(adapter, i);
> +			synchronize_irq(adapter->queue_irq[i]);
> +		}
> +	}

[Severity: Medium]
Can a running poll undo this mask before free_irq()?

synchronize_irq() waits for the hardirq handler only, not for an
ibmveth_poll() already running in softirq context, and that poll
unconditionally re-arms delivery on its completion path:

	lpar_rc = ibmveth_enable_irq(adapter, 0);

Sequence with RX traffic flowing during "ip link set ethN down":

CPU0 ibmveth_close() -> ibmveth_cleanup_rx_interrupts()
	ibmveth_disable_irq(adapter, 0);	/* PHYP masked */
	synchronize_irq(queue_irq[0]);		/* no hardirq running */
	napi_disable(&adapter->napi[0]);	/* blocks on the poll */

CPU1 softirq ibmveth_poll()
	lpar_rc = ibmveth_enable_irq(adapter, 0);	/* PHYP unmasked */
	napi_complete_done(...)

CPU0 then resumes and calls free_irq() with the source unmasked, and nothing
re-checks or re-masks after napi_disable(). If an interrupt lands in that
window, ibmveth_interrupt() -> ibmveth_schedule_rx_queue() takes the
napi_schedule_prep() failure path, so no mask is applied while the handler
still returns IRQ_HANDLED - the storm window the new kernel-doc says this
ordering avoids. The window is bounded by the following free_irq(), but the
teardown no longer reliably ends with the source masked.

Should ibmveth_poll() avoid re-enabling while teardown is in progress, or
should cleanup re-mask after napi_disable()?

> +
> +	for (i = 0; i < adapter->num_rx_queues; i++)
> +		napi_disable(&adapter->napi[i]);
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++) {
> +		if (adapter->queue_irq[i])
> +			free_irq(adapter->queue_irq[i], &adapter->napi[i]);
> +	}

[Severity: High]
This isn't a bug introduced by this patch, but the refactor makes the
unbalanced teardown harder to see and hands the same invariant to the rest
of the series, so it seems worth raising here.

napi_disable() runs unconditionally for every queue, and free_irq() is gated
only on queue_irq[i]. After a failed ibmveth_open() that leaves the
interface administratively up, NAPI was never enabled and no handler was
installed, yet both run on the next ndo_stop.

napi_disable_locked() does:

	while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC))
		usleep_range(20, 200);

and sets SCHED|NPSVC when it completes, so a second napi_disable() with no
intervening napi_enable() loops forever, under rtnl_lock() plus the netdev
lock. free_irq() on a never-requested IRQ additionally warns with "Trying to
free already-free IRQ".

Trigger in this tree: veth_pool_store() calls ibmveth_close() and then
ibmveth_open() directly; if open fails (-ENOMEM from
ibmveth_alloc_rx_queues(), -ENONET from ibmveth_register_logical_lan(), or
request_irq() failure inside ibmveth_setup_rx_interrupts()) it restores the
pool values and returns the error with the netdev still up. The same
close()/open() pattern exists in ibmveth_set_csum_offload() and
ibmveth_set_tso(). A later "ip link set dev ethN down" then reaches
ibmveth_cleanup_rx_interrupts() and hangs.

In the baseline, open() did napi_enable() at entry and napi_disable() at
out:, and close() called free_irq(netdev->irq, netdev) unconditionally, so
the same hang existed. What changes here is that queue_irq[0] is published
in open() before request_irq() can succeed and is deliberately never
cleared, so the new "if (adapter->queue_irq[i])" guard means "we have a virq
number", not "a handler is installed". Could the helper track whether
napi_enable()/request_irq() actually ran for each queue?

> +
> +	/* Dispose IRQ mappings for subordinate queues (1-15).
> +	 * Queue 0 uses netdev->irq from device tree, not irq_create_mapping().
> +	 */
> +	for (i = 1; i < adapter->num_rx_queues; i++) {
> +		if (adapter->queue_irq[i]) {
> +			irq_dispose_mapping(adapter->queue_irq[i]);
> +			adapter->queue_irq[i] = 0;
> +		}
> +	}
> +
> +	/* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */
> +}
> +
> +/**
> + * ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue
> + * @adapter: ibmveth adapter structure
> + * @qindex: RX queue index
> + *
> + * Shared by the IRQ handler and process-context kick paths (open, resume,
> + * pool sysfs, netpoll). Keep ibmveth_interrupt() as the IRQ-only wrapper.
> + */
> +static void ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
> +				      int qindex)
> +{
> +	struct napi_struct *napi = &adapter->napi[qindex];
> +	unsigned long lpar_rc;
> +
> +	if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
> +		return;
> +
> +	if (napi_schedule_prep(napi)) {
> +		lpar_rc = ibmveth_disable_irq(adapter, qindex);
> +		WARN_ON(lpar_rc != H_SUCCESS);
> +		__napi_schedule(napi);
> +	}
> +}
> +
>  /* setup the initial settings for a buffer pool */
>  static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
>  				     u32 pool_index, u32 pool_size,

> @@ -947,8 +1146,6 @@ static int ibmveth_open(struct net_device *netdev)
>  
>  	netdev_dbg(netdev, "open starting\n");
>  
> -	napi_enable(&adapter->napi[0]);
> -
>  	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
>  		rxq_entries += adapter->rx_buff_pool[0][i].size;
>  
> @@ -972,7 +1169,8 @@ static int ibmveth_open(struct net_device *netdev)
>  					adapter->rx_queue[0].queue_len;
>  	rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
>  
> -	h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
> +	adapter->queue_irq[0] = netdev->irq;
> +	ibmveth_disable_irq(adapter, 0);

Related to the teardown question above: queue_irq[0] is published here,
before h_register_logical_lan(), the buffer pool allocation and
request_irq(), and it is never cleared on any failure path.

>  
>  	lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
>  
> @@ -993,21 +1191,16 @@ static int ibmveth_open(struct net_device *netdev)
>  	if (rc)
>  		goto out_free_tx_ltb;
>  
> -	netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
> -	rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
> -			 netdev);
> -	if (rc != 0) {
> -		netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
> -			   netdev->irq, rc);
> +	rc = ibmveth_setup_rx_interrupts(adapter);
> +	if (rc) {
>  		do {
>  			lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
>  		} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> -
>  		goto out_free_buffer_pools;
>  	}
>  
>  	netdev_dbg(netdev, "initial replenish cycle\n");
> -	ibmveth_interrupt(netdev->irq, netdev);
> +	ibmveth_schedule_rx_queue(adapter, 0);
>  
>  	netif_tx_start_all_queues(netdev);
>  
> @@ -1024,7 +1217,6 @@ static int ibmveth_open(struct net_device *netdev)
>  out_free_filter_list:
>  	ibmveth_free_filter_list(adapter);
>  out:
> -	napi_disable(&adapter->napi[0]);
>  	return rc;
>  }
>  
> @@ -1036,11 +1228,10 @@ static int ibmveth_close(struct net_device *netdev)
>  
>  	netdev_dbg(netdev, "close starting\n");
>  
> -	napi_disable(&adapter->napi[0]);
> -
>  	netif_tx_stop_all_queues(netdev);
>  
> -	h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
> +	/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */
> +	ibmveth_cleanup_rx_interrupts(adapter);
>  
>  	do {
>  		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> @@ -1051,8 +1242,6 @@ static int ibmveth_close(struct net_device *netdev)
>  			   "continuing with close\n", lpar_rc);
>  	}
>  
> -	free_irq(netdev->irq, netdev);
> -

Also part of the teardown question: free_irq() now happens inside
ibmveth_cleanup_rx_interrupts(), i.e. before h_free_logical_lan(), where the
baseline order was napi_disable -> tx stop -> h_vio_signal(DISABLE) ->
h_free_logical_lan -> free_irq. The changelog mentions the mask/napi_disable
reordering but not this one. Was the free_irq/h_free_logical_lan inversion
intended?

>  	ibmveth_update_rx_no_buffer(adapter);
>  
>  	ibmveth_free_buffer_pools(adapter);
> @@ -1798,15 +1987,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>  	/* We think we are done - reenable interrupts,
>  	 * then check once more to make sure we are done.
>  	 */
> -	lpar_rc = h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_ENABLE);
> +	lpar_rc = ibmveth_enable_irq(adapter, 0);
>  	if (WARN_ON(lpar_rc != H_SUCCESS)) {
>  		schedule_work(&adapter->work);
>  		goto out;
>  	}
>  
>  	if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
> -		lpar_rc = h_vio_signal(adapter->vdev->unit_address,
> -				       VIO_IRQ_DISABLE);
> +		lpar_rc = ibmveth_disable_irq(adapter, 0);
>  		goto restart_poll;
>  	}
>  

[ ... ]

Cross-instance finding from sashiko-gemini (e3f4ca1fbf9d3f3114fde9e903e88e76f2f763bdede83ba454f20f9e157db105):
[Severity: Medium]
Incomplete Refactoring / Future Out-of-Bounds Access in ibmveth_poll

Re: [PATCH net-next v4 06/14] ibmveth: Refactor TX resource allocation in open/close paths

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor TX resource allocation in open/close paths

This moves the per-queue TX long term buffer alloc/free loops out of
ibmveth_open() and ibmveth_close() into ibmveth_alloc_tx_resources() and
ibmveth_free_tx_resources(), relocates the TX allocation to the end of
open(), and reworks the open() error unwind labels.

A few questions on the changelog and on the new ordering below.

[Severity: Medium]
The changelog says:

    That also retires a mid-open TX leak where the old index-based
    while (--i) free could skip successfully allocated LTBs.

Is that leak real?  In the parent commit the TX loop re-initializes the
shared index before allocating anything:

	rc = -ENOMEM;
	for (i = 0; i < netdev->real_num_tx_queues; i++) {
		if (ibmveth_allocate_tx_ltb(adapter, i))
			goto out_free_tx_ltb;
	}

and nothing between that loop and the later gotos (mac_address/rxq_desc
setup, ibmveth_register_logical_lan(), ibmveth_alloc_buffer_pools(),
ibmveth_setup_rx_interrupts()) reassigns i.  On every path that reaches
out_free_tx_ltb, i is either the failing index or real_num_tx_queues, so
while (--i >= 0) frees exactly what was allocated.  Paths taken before the
TX loop jump to out_free_filter_list, which is below the TX label.

The new helper also uses the same index-based unwind that the changelog
calls broken:

err_free_ltbs:
	while (--i >= 0)
		ibmveth_free_tx_ltb(adapter, i);

Could this paragraph be dropped or reworded to describe the actual
motivation (removing the dependence on a shared loop index and matching
the RX helpers)?  As written it reads like a leak fix that a stable
maintainer might want to backport, and there is no Fixes: tag.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 664169c4d27a4..131c1a61d9e7e 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -1077,8 +1077,14 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
>  
>  static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
>  {
> -	dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
> -			 adapter->tx_ltb_size, DMA_TO_DEVICE);
> +	if (!adapter->tx_ltb_ptr[idx])
> +		return;
> +
> +	if (adapter->tx_ltb_dma[idx]) {
> +		dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
> +				 adapter->tx_ltb_size, DMA_TO_DEVICE);
> +		adapter->tx_ltb_dma[idx] = 0;
> +	}

[Severity: Low]
Is the inner test on tx_ltb_dma[idx] wanted here?  It treats DMA address 0
as a "not mapped" sentinel, and the only architecturally valid check for a
bad handle is dma_mapping_error() at map time.  If a mapping ever
legitimately returned IOVA 0, the TCE entry would leak while kfree() below
still released the buffer.

On pseries this cannot happen today, since
arch/powerpc/kernel/iommu.c:iommu_table_reserve_pages() reserves entry 0:

	if (tbl->it_offset == 0)
		set_bit(0, tbl->it_map);

The test also looks redundant: ibmveth_allocate_tx_ltb() already kfree()s
the buffer and clears tx_ltb_ptr[idx] on dma_mapping_error(), so
tx_ltb_ptr[idx] != NULL already implies a valid mapping and the new early
return covers the stated "safe on an unallocated slot" goal.  Would
dropping the zero test and relying on the pointer check be simpler?

>  	kfree(adapter->tx_ltb_ptr[idx]);
>  	adapter->tx_ltb_ptr[idx] = NULL;
>  }

[ ... ]

> @@ -1184,35 +1226,38 @@ static int ibmveth_open(struct net_device *netdev)
>  				     rxq_desc.desc,
>  				     mac_address);
>  		rc = -ENONET;
> -		goto out_free_tx_ltb;
> +		goto out_free_queue_mem;
>  	}
>  
>  	rc = ibmveth_alloc_buffer_pools(adapter);
>  	if (rc)
> -		goto out_free_tx_ltb;
> +		goto out_unregister_lan;

[Severity: Low]
This changes hypervisor-visible behaviour on an RX error path, which the
changelog does not mention while stating "there is no MQ RX behaviour
change here".

In the parent commit an ibmveth_alloc_buffer_pools() failure went to
out_free_tx_ltb and then to ibmveth_cleanup_rx_resources(), leaving the
logical LAN registered with PHYP while the RX ring was freed and the
buffer list unmapped.  With the new label the h_free_logical_lan() loop
runs first, which is the safer order.

Is this an intentional fix folded into a TX-only refactor?  If so, could
it be split out with its own changelog text (the next patch in the series
reorders open() so this path disappears entirely, so the fix becomes hard
to find later).

>  
>  	rc = ibmveth_setup_rx_interrupts(adapter);
> -	if (rc) {
> -		do {
> -			lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> -		} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> -		goto out_free_buffer_pools;
> -	}
> +	if (rc)
> +		goto out_unregister_lan;
>  
>  	netdev_dbg(netdev, "initial replenish cycle\n");
>  	ibmveth_schedule_rx_queue(adapter, 0);
>  
> +	rc = ibmveth_alloc_tx_resources(adapter);
> +	if (rc)
> +		goto out_cleanup_rx_interrupts;
> +
>  	netif_tx_start_all_queues(netdev);

[ ... ]

> @@ -1224,13 +1269,13 @@ static int ibmveth_close(struct net_device *netdev)
>  {
>  	struct ibmveth_adapter *adapter = netdev_priv(netdev);
>  	long lpar_rc;
> -	int i;
>  
>  	netdev_dbg(netdev, "close starting\n");
>  
>  	netif_tx_stop_all_queues(netdev);
>  
>  	/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */
> +	ibmveth_free_tx_resources(adapter);
>  	ibmveth_cleanup_rx_interrupts(adapter);

[Severity: High]
Can a transmit already in flight write into the TX LTB after it has been
unmapped and freed here?

netif_tx_stop_all_queues() only sets __QUEUE_STATE_DRV_XOFF.  It does not
take __netif_tx_lock and does not wait for an ndo_start_xmit that is
already running on another CPU (netif_tx_disable() would).  ibmveth_close()
is also called directly, bypassing dev_close() -> dev_deactivate_many()
and its synchronize_net() / some_qdisc_is_busy() wait, from:

	ibmveth_set_csum_offload()
	ibmveth_set_tso()
	ibmveth_change_mtu()
	veth_pool_store()

all of which do:

	if (netif_running(dev)) {
		restart = 1;
		ibmveth_close(dev);
	}

So a transmitter that already passed netif_xmit_stopped() can be inside
ibmveth_start_xmit(), which copies into the LTB with no NULL check and no
lock:

	memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
	...
		memcpy(adapter->tx_ltb_ptr[queue_num] + total_bytes,
		       skb_frag_address_safe(frag), skb_frag_size(frag));
	...
	desc.fields.address = adapter->tx_ltb_dma[queue_num];

while ibmveth_free_tx_ltb() concurrently does:

	dma_unmap_single(...);
	adapter->tx_ltb_dma[idx] = 0;
	kfree(adapter->tx_ltb_ptr[idx]);
	adapter->tx_ltb_ptr[idx] = NULL;

That looks like either a write of up to tx_ltb_size bytes into freed heap
memory, or a NULL dereference on the cleared pointer.

A second question on the same move: the DMA unmap now happens before the
h_free_logical_lan() loop below, so a racing ibmveth_send() can hand PHYP
a descriptor whose address is 0 or an already unmapped TCE while the
logical LAN is still registered.  Previously the TX LTBs were freed at the
very end of close(), after h_free_logical_lan(), free_irq()/napi_disable()
and the buffer pool teardown, which incidentally left a large gap between
the XOFF and the free.  Was the inversion intended?

Would keeping ibmveth_free_tx_resources() after the h_free_logical_lan()
loop, switching to netif_tx_disable(), and/or adding an

	if (unlikely(!adapter->tx_ltb_ptr[queue_num]))

bail-out in ibmveth_start_xmit() be preferable?  The last one would also
cover the mirror window that the new open() ordering creates, where the
LAN is registered and RX interrupts/NAPI are live while tx_ltb_ptr[] is
still NULL during the sleeping GFP_KERNEL allocation in
ibmveth_alloc_tx_resources():

	ibmveth_schedule_rx_queue(adapter, 0);

	rc = ibmveth_alloc_tx_resources(adapter);

[Severity: Low]
The comment about PHYP mask, napi_disable and free_irq now sits above
ibmveth_free_tx_resources() instead of above the
ibmveth_cleanup_rx_interrupts() call it describes, so it reads as
documentation of the TX free.  Could it move below the TX free?

>  
>  	do {

[ ... ]

> -	for (i = 0; i < netdev->real_num_tx_queues; i++)
> -		ibmveth_free_tx_ltb(adapter, i);
> -
>  	netdev_dbg(netdev, "close complete\n");
>  
>  	return 0;

Re: [PATCH net-next v4 07/14] ibmveth: Add RX queue register/deregister helpers for MQ

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add RX queue register/deregister helpers for MQ

This patch factors the queue-0 control-plane work out of
ibmveth_open()/ibmveth_close() into ibmveth_register_rx_queues() and
ibmveth_free_all_queues(), teaches ibmveth_register_logical_lan() to use
h_register_logical_lan_with_handle() once multi_queue is set, and adds
adapter->hcall_stats. Runtime behaviour stays single-queue.

A few questions about the new open/close teardown pipeline below.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 131c1a61d9e7e..9d1242dbcb19d 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -1182,14 +1201,89 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
>  	return rc;
>  }
>  
> +/**
> + * ibmveth_free_all_queues - Free all RX queues at once
> + * @adapter: ibmveth adapter structure
> + *
> + * Uses H_FREE_LOGICAL_LAN to free all queues in one hypercall.
> + * Used during interface close and registration error cleanup.
> + *
> + * Clears queue handles only; queue_irq[] is released by
> + * ibmveth_cleanup_rx_interrupts().
> + */
> +static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
> +{
> +	unsigned long lpar_rc;
> +	int i;
> +
> +	netdev_dbg(adapter->netdev, "freeing all RX queues at once\n");
> +
> +	do {
> +		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> +		adapter->hcall_stats.free_lan++;
> +	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> +
> +	if (lpar_rc != H_SUCCESS) {
> +		netdev_err(adapter->netdev,
> +			   "h_free_logical_lan failed: %ld\n", lpar_rc);
> +	}
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++)
> +		adapter->queue_handle[i] = 0;
> +}

[Severity: Low]
Where is the firmware contract for this documented?  The helper issues one
adapter-level h_free_logical_lan() and then clears queue_handle[] for all
num_rx_queues entries, which assumes that the single hcall also
deregisters the subordinate queues that later patches register with
H_REG_LOGICAL_LAN_QUEUE.

The header already provides a per-queue wrapper that this path never uses:

  static inline long h_free_logical_lan_queue(unsigned long unit_address,
					      unsigned long queue_handle)

The assumption cannot be exercised at this commit because multi_queue is
still 0, and the helper is unchanged at the end of the series, so nothing
in the series validates it.  The changelog justifies the new ordering with
"PHYP cannot retain a registered LAN while its DMA-backed RX pools are
released"; if H_FREE_LOGICAL_LAN does not release subordinate queue
registrations, then ibmveth_free_buffer_pools() and
ibmveth_cleanup_rx_resources() would unmap memory PHYP still owns.

Could the PAPR reference for the subordinate-queue semantics be stated in
the kerneldoc or the changelog before the MQ enablement patch relies on it?

[ ... ]

> @@ -1252,10 +1332,9 @@ static int ibmveth_open(struct net_device *netdev)
>  
>  out_cleanup_rx_interrupts:
>  	ibmveth_cleanup_rx_interrupts(adapter);
> -out_unregister_lan:
> -	do {
> -		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> -	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> +out_unregister_queues:
> +	ibmveth_free_all_queues(adapter);
> +out_free_buffer_pools:
>  	ibmveth_free_buffer_pools(adapter);
>  out_free_queue_mem:
>  	ibmveth_cleanup_rx_resources(adapter);

[Severity: High]
This isn't a bug introduced by this patch, but should the unwind here leave
state that a later ibmveth_close() can walk again?

ibmveth_set_csum_offload(), ibmveth_set_tso() and ibmveth_change_mtu() call
ibmveth_close()/ibmveth_open() directly instead of dev_close()/dev_open(),
so when the re-open fails the error is only returned as an errno while
IFF_UP and netif_running() stay set:

	if (restart)
		rc2 = ibmveth_open(dev);

	return rc1 ? rc1 : rc2;

After that failed open, every RX resource is freed (buffer_list_addr[] is
NULLed by ibmveth_cleanup_rx_resources()), NAPI is left disabled, and
queue_irq[0] is deliberately kept populated.  The next ndo_stop, for
example retrying the same ethtool command or an "ip link set dev X down",
re-enters ibmveth_close() -> ibmveth_cleanup_rx_interrupts(), which does:

	for (i = 0; i < adapter->num_rx_queues; i++)
		napi_disable(&adapter->napi[i]);

	for (i = 0; i < adapter->num_rx_queues; i++) {
		if (adapter->queue_irq[i])
			free_irq(adapter->queue_irq[i], &adapter->napi[i]);
	}

Can the second napi_disable() ever return?  napi_disable_locked() leaves
NAPIF_STATE_SCHED|NAPIF_STATE_NPSVC set, and the second call waits in
usleep_range() for bits that no poll will clear, with rtnl held by the
ndo_stop caller.  The free_irq() that follows also refers to an IRQ that is
no longer requested.

If it did get past those, ibmveth_update_rx_no_buffer() would dereference
the now-NULL buffer_list_addr[0]:

	__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;

and ibmveth_free_all_queues() would issue H_FREE_LOGICAL_LAN for a LAN that
is not registered.

The baseline close() already had an unconditional napi_disable(), so the
hang predates this patch, but this patch centralizes and extends the
unconditional teardown (now also free_all_queues() plus the NULL-able
buffer_list_addr[0] read) without an "already torn down" guard, and the
same structure survives to the end of the series.  Would a state check at
the top of ibmveth_close(), or switching those callers to
dev_close()/dev_open(), be preferable?

> @@ -1268,7 +1347,6 @@ static int ibmveth_open(struct net_device *netdev)
>  static int ibmveth_close(struct net_device *netdev)
>  {
>  	struct ibmveth_adapter *adapter = netdev_priv(netdev);
> -	long lpar_rc;
>  
>  	netdev_dbg(netdev, "close starting\n");
>  
> @@ -1277,18 +1355,8 @@ static int ibmveth_close(struct net_device *netdev)
>  	/* PHYP mask + napi_disable + free_irq live in cleanup_rx_interrupts */
>  	ibmveth_free_tx_resources(adapter);
>  	ibmveth_cleanup_rx_interrupts(adapter);

[Severity: Medium]
Does the PHYP mask taken by ibmveth_cleanup_rx_interrupts() actually stay
in effect until free_irq()?

The helper masks queue 0 and calls synchronize_irq() before napi_disable():

	ibmveth_disable_irq(adapter, i);
	synchronize_irq(adapter->queue_irq[i]);
	...
	for (i = 0; i < adapter->num_rx_queues; i++)
		napi_disable(&adapter->napi[i]);

But a poll that is already in flight unmasks it again on completion, with
no coordination against teardown:

ibmveth_poll()
	if (!napi_complete_done(napi, frames_processed))
		goto out;

	/* We think we are done - reenable interrupts, ... */
	lpar_rc = ibmveth_enable_irq(adapter, 0);

So napi_disable() can return with the source unmasked.  In the window
before free_irq(), an RX interrupt is then delivered and
ibmveth_interrupt() -> ibmveth_schedule_rx_queue() sees napi_schedule_prep()
fail and returns IRQ_HANDLED without masking, since masking only happens on
the napi_schedule_prep() success branch.  Can a VIO source with pending RX
re-assert repeatedly until free_irq() shuts the line down, and can
ibmveth_free_all_queues() then issue H_FREE_LOGICAL_LAN while the source is
still unmasked?

The kerneldoc in cleanup_rx_interrupts() states "Mask PHYP before
napi_disable so ibmveth_interrupt cannot return IRQ_HANDLED without
masking", which does not seem to hold when the poll itself is the entity
that unmasks.  The pre-series code did napi_disable() first, then
h_vio_signal(VIO_IRQ_DISABLE), then free_irq(); this patch propagates the
new ordering into the open-failure unwind as well and places
ibmveth_free_all_queues() behind it, and ibmveth_poll() still unmasks
unconditionally at the end of the series.

> -
> -	do {
> -		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> -	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> -
> -	if (lpar_rc != H_SUCCESS) {
> -		netdev_err(netdev, "h_free_logical_lan failed with %lx, "
> -			   "continuing with close\n", lpar_rc);
> -	}
> -
>  	ibmveth_update_rx_no_buffer(adapter);
> -
> +	ibmveth_free_all_queues(adapter);

[Severity: Low]
Was moving ibmveth_update_rx_no_buffer() ahead of the H_FREE_LOGICAL_LAN
intentional?

Previously the counter was read after h_free_logical_lan() returned, so the
hypervisor was no longer writing it.  ibmveth_update_rx_no_buffer() reads
the last 8 bytes of the DMA-mapped buffer_list page that PHYP updates:

	__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;

	adapter->rx_no_buffer = be64_to_cpup(p);

At the new call site the LAN is still registered, so frames can still be
dropped and counted after the snapshot, and those drops are lost from the
rx_no_buffer value reported by ethtool -S.

The 7-step close pipeline in the changelog does not mention
ibmveth_update_rx_no_buffer() at all:

  ibmveth_close():

    1. netif_tx_stop_all_queues()
    2. ibmveth_free_tx_resources()
    3. ibmveth_cleanup_rx_interrupts() - mask PHYP, napi_disable, free_irq
    4. ibmveth_free_all_queues()       - H_FREE_LOGICAL_LAN
    5. ibmveth_free_buffer_pools()
    6. ibmveth_cleanup_rx_resources()
    7. ibmveth_free_filter_list()

Could the step list be updated to include it and to say where it belongs
relative to H_FREE_LOGICAL_LAN?  The ordering is unchanged through the end
of the series.

>  	ibmveth_free_buffer_pools(adapter);
>  	ibmveth_cleanup_rx_resources(adapter);
>  	ibmveth_free_filter_list(adapter);

[ ... ]

Re: [PATCH net-next v4 08/14] ibmveth: Add queue-aware RX buffer submit helper for MQ

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add queue-aware RX buffer submit helper for MQ

This adds ibmveth_add_logical_lan_buffers() to choose between the legacy
single/multi-buffer hcalls and the new per-queue buffer hcall, threads a
queue_index argument through the RX accessors, harvest and replenish
helpers, and adds a per-queue replenish_lock plus reworked H_FUNCTION
handling.

A few questions below about the new failure handling, the new lock and the
logging changes.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 9d1242dbcb19d..cb93659fc0576 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -588,11 +598,75 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length)
>  		asm("dcbf %0,%1,1" :: "b" (addr), "r" (offset));
>  }
>  
> +/**
> + * ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor
> + * @adapter: ibmveth adapter structure
> + * @descs: array of buffer descriptors to add
> + * @filled: number of valid descriptors in the array
> + * @buff_size: size of each buffer (multi-queue mode only)
> + * @queue_index: RX queue index
> + *
> + * Return: hypervisor return code
> + */
> +static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
> +					    union ibmveth_buf_desc *descs,
> +					    int filled,
> +					    unsigned long buff_size,
> +					    int queue_index)
> +{
> +	struct vio_dev *vdev = adapter->vdev;
> +	unsigned long rc;
> +
> +	if (adapter->multi_queue) {

[ ... ]

> +		rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
> +						     handle,
> +						     buffersznum,
> +						     ioba[0], ioba[1], ioba[2],
> +						     ioba[3], ioba[4], ioba[5]);
> +		adapter->hcall_stats.add_bufs_queue++;
> +	} else if (filled == 1) {
> +		rc = h_add_logical_lan_buffer(vdev->unit_address,
> +					      descs[0].desc);
> +		adapter->hcall_stats.add_buf++;
> +	} else {
> +		rc = h_add_logical_lan_buffers(vdev->unit_address,
> +					       descs[0].desc, descs[1].desc,
> +					       descs[2].desc, descs[3].desc,
> +					       descs[4].desc, descs[5].desc,
> +					       descs[6].desc, descs[7].desc);
> +		adapter->hcall_stats.add_bufs++;
> +	}

[Severity: Medium]

These hcall_stats fields are plain non-atomic u64 counters on the
adapter, but the only serialization this patch adds is the per-queue
rx_queue[i].replenish_lock taken in ibmveth_replenish_task().

Two NAPI instances replenishing different queues hold different locks, so
can these increments lose counts once MQ is enabled later in the series and
num_rx_queues becomes larger than 1?

The same question applies to the other adapter-wide counters reached from
this path:

	adapter->replenish_task_cycles++;
	adapter->replenish_add_buff_success += filled;
	adapter->replenish_add_buff_failure += filled;

and to ibmveth_update_rx_no_buffer(), which ibmveth_replenish_task() calls
while holding queue N's lock only, although it rewrites adapter-global
state for every queue:

	adapter->rx_no_buffer = 0;
	for (i = 0; i < adapter->num_rx_queues; i++) {
		...
		adapter->rx_qstats[i].no_buffer_drops = drops;
		adapter->rx_no_buffer += drops;
	}

Would rx_no_buffer and rx_qstats[].no_buffer_drops be clobbered when two
queues replenish concurrently?  ibmveth_close() calls the same helper with
no lock held at all.

>  /* replenish the buffers for a pool.  note that we don't need to
>   * skb_reserve these since they are used for incoming...
>   */
>  static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
> -					  struct ibmveth_buff_pool *pool)
> +					  struct ibmveth_buff_pool *pool,
> +					  int queue_index)
>  {
>  	union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL] = {0};
>  	u32 remaining = pool->size - atomic_read(&pool->available);
> @@ -678,24 +752,16 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,

[ ... ]

> +		lpar_rc = ibmveth_add_logical_lan_buffers(adapter, descs,
> +							  filled,
> +							  pool->buff_size,
> +							  queue_index);
> +
>  		if (lpar_rc != H_SUCCESS) {
>  			dev_warn_ratelimited(dev,
> -					     "RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n",
> +					     "RX h_add_logical_lan %s failed: filled=%u, rc=%lu, batch=%u\n",
> +					     adapter->multi_queue ?
> +					     "_queue" : "",
>  					     filled, lpar_rc, batch);
>  			goto hcall_failure;
>  		}

[Severity: Low]

The interpolated hcall name never matches an actual symbol.  In MQ mode
this prints

	RX h_add_logical_lan _queue failed: ...

with a stray space, and in legacy mode

	RX h_add_logical_lan  failed: ...

with a double space.  Would a full literal per branch read better?

The two new H_FUNCTION messages added below also print batch with %d,
although batch is declared u32 in this function ("u32 i, filled, batch;"),
while this warning uses %u for the same variable.

> @@ -736,24 +802,23 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
>  		}
>  		adapter->replenish_add_buff_failure += filled;
>  
> -		/*
> -		 * If multi rx buffers hcall is no longer supported by FW
> -		 * e.g. in the case of Live Partition Migration
> -		 */
> -		if (batch > 1 && lpar_rc == H_FUNCTION) {
> -			/*
> -			 * Instead of retry submit single buffer individually
> -			 * here just set the max rx buffer per hcall to 1
> -			 * buffers will be respleshed next time
> -			 * when ibmveth_replenish_buffer_pool() is called again
> -			 * with single-buffer case
> -			 */
> -			netdev_info(adapter->netdev,
> -				    "RX Multi buffers not supported by FW, rc=%lu\n",
> -				    lpar_rc);
> -			adapter->rx_buffers_per_hcall = 1;
> -			netdev_info(adapter->netdev,
> -				    "Next rx replesh will fall back to single-buffer hcall\n");
> +		if (lpar_rc == H_FUNCTION) {
> +			if (adapter->multi_queue) {
> +				netdev_err(adapter->netdev,
> +					   "MQ buffer add H_FUNCTION (q=%d, batch=%d)\n",
> +					   queue_index, batch);
> +			} else if (batch > 1) {

[Severity: High]

The multi_queue arm logs and breaks without changing any state:
adapter->multi_queue stays set, rx_buffers_per_hcall is untouched, there is
no fallback to h_add_logical_lan_buffer()/h_add_logical_lan_buffers(), and
no schedule_work(&adapter->work).

ibmveth_add_logical_lan_buffers() picks the hcall purely on
adapter->multi_queue, and that field is only written in ibmveth_probe(),
which is not re-run on resume (ibmveth_resume() only calls
ibmveth_schedule_rx_queue()).

So once the hypervisor stops implementing H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
- exactly the Live Partition Migration case the legacy arm below exists for
- can this queue ever post another RX buffer again?  pool->available stays
at 0, PHYP drops every inbound frame, and the interface still reports the
link up, so recovery would need an administrative down/up.

Every other unrecoverable condition in this driver schedules the reset
work (ibmveth_remove_buffer_from_pool(), ibmveth_rxq_get_buffer(), and the
new enable_irq failure path in ibmveth_poll()).  Should this arm do the
same, or fall back to the legacy hcalls the way the legacy arm downgrades
batch?

Separately, netdev_err() here is not rate limited, while the warning for
the same failure a few lines above uses dev_warn_ratelimited(), so a
dried-out pool can emit several console lines per poll cycle per pool.

This arm appears unchanged through the end of the series.

> +				/*
> +				 * Live Partition Migration may drop multi-
> +				 * buffer support. Fall back to single-buffer
> +				 * on the next replenish; do not continue with
> +				 * a stale local batch size (infinite loop).
> +				 */
> +				netdev_warn(adapter->netdev,
> +					    "Legacy batch add H_FUNCTION (batch=%d), fallback\n",
> +					    batch);
> +				adapter->rx_buffers_per_hcall = 1;
> +			}
>  		}
>  		break;
>  	}
> @@ -775,21 +840,33 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
>  }
>  
>  /* replenish routine */
> -static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
> +static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
> +				   int queue_index)
>  {
> +	struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
> +	unsigned long flags;
>  	int i;
>  
> +	if (queue_index >= adapter->num_rx_queues)
> +		return;
> +
>  	adapter->replenish_task_cycles++;
>  
> +	spin_lock_irqsave(&rxq->replenish_lock, flags);
> +
>  	for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
> -		struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
> +		struct ibmveth_buff_pool *pool =
> +			&adapter->rx_buff_pool[queue_index][i];
>  
>  		if (pool->active &&
>  		    (atomic_read(&pool->available) < pool->threshold))
> -			ibmveth_replenish_buffer_pool(adapter, pool);
> +			ibmveth_replenish_buffer_pool(adapter, pool,
> +						      queue_index);
>  	}
>  
>  	ibmveth_update_rx_no_buffer(adapter);
> +
> +	spin_unlock_irqrestore(&rxq->replenish_lock, flags);
>  }

[Severity: High]

Can this lock deadlock against itself when netconsole runs over this
ibmveth device?

ibmveth_replenish_buffer_pool() printk()s from inside the locked region:
netdev_info() on an invalid map index, dev_warn_ratelimited() on hcall
failure, and the two new H_FUNCTION messages.  With netconsole on this
interface a printk from there re-enters the driver:

netconsole write_msg()
  netpoll_send_udp()
    netpoll_send_skb()
      __netpoll_send_skb()
        netpoll_poll_dev()
          ops->ndo_poll_controller()  ->  ibmveth_poll_controller()
            ibmveth_replenish_task(adapter, i)

__netpoll_send_skb() calls netpoll_poll_dev() whenever the xmit does not
go through on the first try, and netpoll_poll_dev() only declines when
down_trylock(&ni->dev_lock) fails, when !netif_running(dev), or when
netif_local_xmit_active(dev):

	if (!ni || down_trylock(&ni->dev_lock))
		return;

	if (!netif_running(dev) || netif_local_xmit_active(dev)) {
		up(&ni->dev_lock);
		return;
	}
	...
	if (ops->ndo_poll_controller)
		ops->ndo_poll_controller(dev);

None of those covers the replenish path, so ibmveth_poll_controller() loops
ibmveth_replenish_task() over every queue and re-takes the same
replenish_lock on the same CPU, with interrupts already disabled.

Before this patch ibmveth_poll_controller() also called
ibmveth_replenish_task(), but with no lock the recursion was harmless.

This code appears unchanged through the end of the series.

[Severity: Medium]

Does this need to be spin_lock_irqsave()?  The lock now brackets the whole
multi-pool loop, so every netdev_alloc_skb(), every dma_map_single() and
every H_ADD_LOGICAL_LAN_BUFFER[S][_QUEUE] hypervisor call for all five
pools runs with local interrupts hard-disabled.

On a cold fill from ibmveth_open() or the first NAPI poll that is roughly
768 allocations and DMA maps and around 96 hcalls per queue at the default
MTU (pool_count[] = { 256, 512, 256, 256, 256 }), and more when the large
pools are active.

None of the protected state is written from hardirq context, and the commit
message says the lock exists only "so later concurrent NAPI/resize paths
can serialize buffer posting", so would spin_lock_bh(), dropping and
re-acquiring per batch, or preparing buffers outside the critical section
be enough?

>  
>  /* empty and free ana buffer pool - also used to do cleanup in error paths */

[ ... ]

> @@ -1004,50 +1083,55 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,

[ ... ]

> -	free_index = adapter->rx_buff_pool[0][pool].producer_index;
> -	adapter->rx_buff_pool[0][pool].producer_index++;
> -	if (adapter->rx_buff_pool[0][pool].producer_index >=
> -	    adapter->rx_buff_pool[0][pool].size)
> -		adapter->rx_buff_pool[0][pool].producer_index = 0;
> -	adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
> +	free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;
> +	adapter->rx_buff_pool[queue_index][pool].producer_index++;
> +	if (adapter->rx_buff_pool[queue_index][pool].producer_index >=
> +	    adapter->rx_buff_pool[queue_index][pool].size)
> +		adapter->rx_buff_pool[queue_index][pool].producer_index = 0;
> +	adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;
>  
>  	mb();
>  
> -	atomic_dec(&adapter->rx_buff_pool[0][pool].available);
> +	atomic_dec(&adapter->rx_buff_pool[queue_index][pool].available);

[Severity: High]

This isn't a bug introduced by this patch, but the new replenish_lock only
covers the producer half of this bookkeeping, so it is worth asking whether
the consumer half is still unprotected.

ibmveth_remove_buffer_from_pool() mutates free_map[], producer_index,
skbuff[], dma_addr[] and available for the same pool that
ibmveth_replenish_buffer_pool() mutates under replenish_lock, but takes no
lock:

producer: ibmveth_poll_controller()
            ibmveth_replenish_task()        /* holds replenish_lock */
              ibmveth_replenish_buffer_pool()

consumer: ibmveth_poll()                   /* holds nothing */
            ibmveth_rxq_harvest_buffer()
              ibmveth_remove_buffer_from_pool()

netpoll_poll_dev() serializes netpoll users against each other via
ni->dev_lock, and __napi_poll()'s netpoll_poll_lock() only stops netpoll
from re-polling a NAPI instance that is already running; neither prevents
ndo_poll_controller from running on one CPU while the same queue's NAPI
poll runs on another.

Can free_map[]/producer_index/consumer_index then hand the same pool index
out twice, so a buffer is posted to the hypervisor while its skb has
already gone up the stack, or lose a free index permanently?

Since the future work described in the commit message ("so later concurrent
NAPI/resize paths can serialize buffer posting") builds on this lock, should
the consumer side take it as well?

[ ... ]

> @@ -1993,34 +2076,45 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
>  
>  static int ibmveth_poll(struct napi_struct *napi, int budget)
>  {
> -	struct ibmveth_adapter *adapter =
> -			container_of(napi, struct ibmveth_adapter, napi[0]);
> -	struct net_device *netdev = adapter->netdev;
> +	struct net_device *netdev = napi->dev;
> +	struct ibmveth_adapter *adapter = netdev_priv(netdev);
>  	int frames_processed = 0;
>  	unsigned long lpar_rc;
> +	int queue_index, rc;
>  	u16 mss = 0;
>  
> +	queue_index = napi - adapter->napi;
> +
> +	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
> +		return 0;
> +

[ ... ]

> @@ -2097,14 +2197,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>  	/* We think we are done - reenable interrupts,
>  	 * then check once more to make sure we are done.
>  	 */
> -	lpar_rc = ibmveth_enable_irq(adapter, 0);
> -	if (WARN_ON(lpar_rc != H_SUCCESS)) {
> +	lpar_rc = ibmveth_enable_irq(adapter, queue_index);
> +	if (lpar_rc != H_SUCCESS) {
> +		netdev_err(netdev,
> +			   "Failed to enable IRQ for queue %d (rc=0x%lx), scheduling reset\n",
> +			   queue_index, lpar_rc);
>  		schedule_work(&adapter->work);
>  		goto out;
>  	}
>  
> -	if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
> -		lpar_rc = ibmveth_disable_irq(adapter, 0);
> +	if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
> +	    napi_schedule(napi)) {
> +		lpar_rc = ibmveth_disable_irq(adapter, queue_index);
> +		WARN_ON(lpar_rc != H_SUCCESS);
>  		goto restart_poll;
>  	}

[Severity: Low]

Two diagnostic changes here are not described in the commit message, which
only says callers were updated "so arity stays consistent": the WARN_ON()
around the ibmveth_enable_irq() failure is replaced by an unconditional
netdev_err(), which drops the taint and backtrace that test harnesses key
on, and a new WARN_ON() is added on the ibmveth_disable_irq() result.
Would it help to mention or split out that change?

Also, the new guard added at the top of ibmveth_poll()

	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
		return 0;

returns less than budget from a NAPI poll callback without calling
napi_complete_done(), which leaves the instance scheduled with the
hypervisor IRQ still masked.  The guard is unreachable at this commit,
since IBMVETH_MAX_RX_QUEUES is 1 and queue_index is always 0, but the same
early return is still there at the end of the series.  Should it complete
NAPI before returning?

[ ... ]

> @@ -2229,9 +2334,13 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
>  static void ibmveth_poll_controller(struct net_device *dev)
>  {
>  	struct ibmveth_adapter *adapter = netdev_priv(dev);
> +	int i;
>  
> -	ibmveth_replenish_task(adapter);
> -	ibmveth_schedule_rx_queue(adapter, 0);
> +	for (i = 0; i < adapter->num_rx_queues; i++)
> +		ibmveth_replenish_task(adapter, i);
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++)
> +		ibmveth_schedule_rx_queue(adapter, i);
>  }
>  #endif

[ ... ]

Re: [PATCH net-next v4 09/14] ibmveth: Enable multi-queue RX receive path

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Enable multi-queue RX receive path

This patch turns on the multi-queue RX path: probe latches multi_queue and
num_rx_queues from the H_ILLAN_ATTRIBUTES MQ bit, subordinate queues are
registered with H_REG_LOGICAL_LAN_QUEUE, and open replenishes every queue
before setup_rx_interrupts() unmasks PHYP. A few questions below on the
new ordering, the CMO entitlement accounting and the probe unwind.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index cb93659fc0576..4ad7ced3c6083 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -97,7 +97,17 @@ static struct ibmveth_stat ibmveth_stats[] = {
>  	{ "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) },
>  	{ "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) },
>  	{ "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) },
> -	{ "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }
> +	{ "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) },
> +	{ "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
> +	{ "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) },
> +	{ "hcall_add_bufs_queue",
> +	  IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) },
> +	{ "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) },
> +	{ "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) },
> +	{ "hcall_free_lan_queue",
> +	  IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) },
> +	{ "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) },
> +	{ "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) },
>  };

[Severity: Low]
Should hcall_send_lan be exported through ethtool -S?  It is incremented
once per successfully transmitted packet on the ibmveth_send() success
path:

drivers/net/ethernet/ibm/ibmveth.c:ibmveth_send() {
	...
	adapter->hcall_stats.send_lan++;
	return 0;
}

That makes it the same value as the standard rtnl_link_stats64 tx_packets
counter reported by ndo_get_stats64 / ip -s link, and
Documentation/networking/statistics.rst asks that counters with a standard
uAPI not be duplicated as driver-private ethtool -S strings.  These
strings become uAPI once merged.

The commit message subject is "Enable multi-queue RX receive path" and
does not mention that eight new statistics strings are being added.  Could
the stats additions be split out or at least described?

> @@ -429,12 +439,64 @@ ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
>  	return ibmveth_toggle_irq(adapter, queue_index, true);
>  }
>  
> +/**
> + * ibmveth_dispose_subordinate_irq_mapping - Drop one subordinate virq mapping
> + * @adapter: ibmveth adapter structure
> + * @queue_idx: RX queue index (1..N)
> + *
> + * Subordinate queues get mappings from irq_create_mapping() during PHYP
> + * registration. Queue 0 uses netdev->irq from device tree and is left alone.
> + *
> + * Bound against IBMVETH_MAX_RX_QUEUES, not num_rx_queues: scale-down and
> + * scale-up fail paths dispose queues that are no longer in the published
> + * live set but still own a virq in queue_irq[]. The bulk helper still
> + * iterates only 1..num_rx_queues-1 for close/open-fail cleanup.

[Severity: Low]
This isn't a bug, but do the "scale-down and scale-up fail paths" exist at
this commit?  ibmveth_resize_rx_queues_incremental() and
ibmveth_setup_single_rx_interrupt() arrive in a later patch, and the only
caller here is the bulk helper bounded by num_rx_queues.

The same applies to the comment added above ibmveth_get_strings() in this
patch:

	/*
	 * Sum per-queue counters for rare ethtool reads. Do not write adapter
	 * globals on the hot path (ibmvnic-style); with qstats allocated for the
	 * adapter lifetime, these sums remain meaningful across ifdown/up.
	 */

ibmveth_get_strings() only memcpy()s string tables here, and no
rx_qstats/tx_qstats members exist in struct ibmveth_adapter until the
following patch.  Could both comments move to the patches that add the
code they describe, so the intermediate commits read correctly?

> +	for (i = 0; i < num; i++)
> +		napi_enable(&adapter->napi[i]);
> +
> +	if (adapter->multi_queue && num > 1) {
> +		for (i = 0; i < num; i++) {
> +			rc = ibmveth_enable_irq(adapter, i);
> +			if (rc) {
> +				netdev_err(netdev,
> +					   "Failed to enable IRQ for queue %d, rc=%d\n",
> +					   i, rc);
> +				while (--i >= 0)
> +					ibmveth_disable_irq(adapter, i);
> +				rc = -EIO;
> +				goto err_disable_napi;
> +			}
> +		}
> +	}
> +
>  	return 0;
>  
> +err_disable_napi:
> +	/* PHYP unmask was rolled back above; disable NAPI before free_irq */
> +	for (i = 0; i < num; i++)
> +		napi_disable(&adapter->napi[i]);
> +	for (i = 0; i < num; i++) {
> +		if (adapter->queue_irq[i])
> +			free_irq(adapter->queue_irq[i], &adapter->napi[i]);
> +	}
> +	goto err_dispose_mappings;

[Severity: Low]
Can the rollback loop above be undone by an in-flight poll?  Queues 0..i-1
were already unmasked, so a poll may be running when
ibmveth_enable_irq() fails for queue i:

ibmveth_poll() {
	...
	if (!napi_complete_done(napi, frames_processed))
		goto out;
	...
	lpar_rc = ibmveth_enable_irq(adapter, queue_index);
}

There is no synchronize_irq() between the while (--i >= 0)
ibmveth_disable_irq() loop and napi_disable(), so a poll that completes
after the mask re-enables PHYP delivery.  napi_disable() waits for the
poll but does not undo that re-enable, and the following free_irq() plus
ibmveth_dispose_subordinate_irq_mappings() then leave the hypervisor
source enabled for a hwirq whose Linux mapping is gone.

The scale-down path added later does disable_irq() followed by
synchronize_irq() for exactly this reason.  Should this path do the same?

[ ... ]

> @@ -834,9 +914,15 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
>   */
>  static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
>  {
> -	__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
> +	int i;
>  
> -	adapter->rx_no_buffer = be64_to_cpup(p);
> +	adapter->rx_no_buffer = 0;
> +	for (i = 0; i < adapter->num_rx_queues; i++) {
> +		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
> +		u64 drops = be64_to_cpup(p);
> +
> +		adapter->rx_no_buffer += drops;
> +	}
>  }

[Severity: High]
Is this cross-queue aggregation safe against the RX queue resize path?
The only lock held here is the caller's own per-queue lock:

ibmveth_replenish_task(adapter, queue_index) {
	spin_lock_irqsave(&rxq->replenish_lock, flags);
	...
	ibmveth_update_rx_no_buffer(adapter);
	spin_unlock_irqrestore(&rxq->replenish_lock, flags);
}

That lock does not cover other queues' buffer_list_addr[] entries, nor
num_rx_queues.  The ethtool -L scale-down path added later in the series
publishes the new count and then frees the pages:

	adapter->num_rx_queues = new_count;

	for (i = new_count; i < old_count; i++) {
		...
		ibmveth_free_single_rx_queue(adapter, i);   /* free_page() + NULL */
	}

If a surviving queue's poll already loaded the older, larger
num_rx_queues, can this loop dereference NULL + 4096 - 8, or read a page
that was just freed, from softirq context?  A READ_ONCE() of the count
plus a NULL check, or serializing the aggregation against the resize
path, would make this explicit.

Separately, two concurrent polls on different queues both execute
adapter->rx_no_buffer = 0 followed by += drops on the same field.  Can
that lose increments and expose a transient 0 through
rx_no_buffer / ethtool -S?

> @@ -847,8 +933,12 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
>  	unsigned long flags;
>  	int i;
>  
> -	if (queue_index >= adapter->num_rx_queues)
> +	if (queue_index >= adapter->num_rx_queues) {
> +		netdev_dbg(adapter->netdev,
> +			   "Skipping replenish for freed queue %d (num_queues=%d)\n",
> +			   queue_index, adapter->num_rx_queues);
>  		return;
> +	}
>  
>  	adapter->replenish_task_cycles++;

[Severity: Low]
With several RX queues now polling concurrently, do the adapter-wide plain
u64 counters need serialization?  replenish_task_cycles is incremented
before the per-queue lock is taken, and replenish_no_mem,
replenish_add_buff_success/failure and the new hcall_stats fields are
touched under different per-queue locks, so the lockset intersection for
these adapter-global fields is empty.  hcall_stats.send_lan is also
incremented from ibmveth_start_xmit() on up to 16 TX queues in parallel.
The visible effect is lost increments in the ethtool -S values.

[ ... ]

> @@ -1284,6 +1374,137 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
> +	lpar_rc = h_reg_logical_lan_queue(adapter->vdev->unit_address,
> +					  adapter->buffer_list_dma[queue_index],
> +					  rxq_desc.desc, &handle, &hwirq);
> +	adapter->hcall_stats.reg_lan_queue++;

[Severity: Low]
Should this hcall retry on a busy return?  Every other long-running hcall
in the driver loops:

	do {
		lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
	} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));

including h_free_logical_lan_queue() in the IRQ-mapping failure path a few
lines below this call, and h_send_logical_lan() in ibmveth_send().  As
written, a transient H_BUSY / H_LONG_BUSY_ORDER_* here becomes -EIO from
ibmveth_register_single_rx_queue() and a failed ifup.

> +	/*
> +	 * H_FUNCTION means firmware rejected this subordinate register
> +	 * (MQ unsupported). That is a hard open failure: do not clear
> +	 * multi_queue or claim single-queue fallback. Keep a specific
> +	 * log, then the generic failure lines below (no early return).
> +	 */
> +	if (lpar_rc == H_FUNCTION)
> +		netdev_err(adapter->netdev,
> +			   "h_reg_logical_lan_queue H_FUNCTION for queue %d (firmware MQ unsupported)\n",
> +			   queue_index);

[Severity: High]
Can the adapter recover if firmware stops honouring the MQ register after
probe?  multi_queue and num_rx_queues are latched once in ibmveth_probe()
from the H_ILLAN_ATTRIBUTES bit and are never re-read or reduced, so a
H_FUNCTION here fails ibmveth_open() and every subsequent open the same
way.

The reset work item only closes and reopens, it never re-reads the
attributes:

ibmveth_reset() {
	rtnl_lock();
	dev_close(adapter->netdev);
	dev_open(adapter->netdev, NULL);
	rtnl_unlock();
}

ethtool -L ethX rx 1 while down does not restore RX either: multi_queue
stays 1, so rx_buffers_per_hcall remains IBMVETH_MAX_RX_QUEUE and
ibmveth_add_logical_lan_buffers() keeps calling
h_add_logical_lan_buffers_queue(), whose H_FUNCTION path has no fallback
when multi_queue is set.

The driver already assumes PHYP capabilities can disappear across LPM:

	/*
	 * Live Partition Migration may drop multi-
	 * buffer support. Fall back to single-buffer
	 * on the next replenish; ...
	 */

Would clearing multi_queue and falling back to one queue on H_FUNCTION be
preferable to leaving the interface permanently unable to open?

> +static int
> +ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
> +				 int queue_idx, u64 mac_address)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +	union ibmveth_buf_desc rxq_desc;
> +	long lpar_rc;
> +
> +	(void)mac_address;

[Severity: Low]
This isn't a bug, but is the mac_address parameter worth keeping?
H_REG_LOGICAL_LAN_QUEUE has no MAC operand and
ibmveth_register_logical_lan_queue() has no MAC parameter, so callers
compute ether_addr_to_u64(netdev->dev_addr) only to feed a parameter that
is voided here.  The (void)x; suppression cast is also not needed under
the kernel's warning set.

[ ... ]

> @@ -1356,9 +1579,31 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
> +	for (i = 1; i < adapter->num_rx_queues; i++) {
> +		rc = ibmveth_register_single_rx_queue(adapter, i, mac_address);
> +		if (rc)
> +			goto err_unregister;
> +	}
> +
>  	netdev_dbg(netdev,
> -		   "registered 1 RX queue with hypervisor (single-queue mode)\n");
> +		   "registered %d RX queues with hypervisor (multi-queue mode)\n",
> +		   adapter->num_rx_queues);
> +
>  	return 0;
> +
> +err_unregister:
> +	ibmveth_dispose_subordinate_irq_mappings(adapter);
> +	ibmveth_free_all_queues(adapter);
> +	return rc;
>  }

> @@ -1396,12 +1641,29 @@ static int ibmveth_open(struct net_device *netdev)
> +	/*
> +	 * MQ: post buffers before setup_rx_interrupts() unmasks PHYP
> +	 * (avoids drops if traffic arrives during open; PHYP allows
> +	 * either order). Single-queue keeps the classic kick: setup
> +	 * (no unmask) then schedule_rx_queue() so the first poll
> +	 * replenishes and enables.
> +	 */
> +	if (adapter->multi_queue && adapter->num_rx_queues > 1) {
> +		for (i = 0; i < adapter->num_rx_queues; i++) {
> +			netdev_dbg(netdev,
> +				   "initial replenish cycle for queue %d\n", i);
> +			ibmveth_replenish_task(adapter, i);
> +		}
> +	}
> +
>  	rc = ibmveth_setup_rx_interrupts(adapter);

[Severity: Medium]
Is PHYP delivery for subordinate queues guaranteed to be masked in this
window?  Queue 0 is explicitly masked before registration:

ibmveth_register_rx_queues() {
	adapter->queue_irq[0] = netdev->irq;
	rc = ibmveth_disable_irq(adapter, 0);
	...
}

but queues 1..N get no equivalent H_VIOCTL/H_DISABLE_VIO_INTERRUPT after
H_REG_LOGICAL_LAN_QUEUE plus irq_create_mapping().  Buffers are now
posted for every queue here, before any handler exists, and
ibmveth_setup_rx_interrupts() was also reordered so request_irq() for all
queues completes before any napi_enable():

	for (i = 0; i < num; i++) {
		...
		rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt, ...);
	}

	for (i = 0; i < num; i++)
		napi_enable(&adapter->napi[i]);

If an interrupt arrives in that window, ibmveth_schedule_rx_queue() finds
NAPI still disabled:

	if (napi_schedule_prep(napi)) {
		lpar_rc = ibmveth_disable_irq(adapter, qindex);
		WARN_ON(lpar_rc != H_SUCCESS);
		__napi_schedule(napi);
	}

so the handler returns IRQ_HANDLED without masking the hypervisor source
and without scheduling a poll.  That is the same storm window described in
ibmveth_cleanup_rx_interrupts() ("Mask PHYP before napi_disable so
ibmveth_interrupt cannot return IRQ_HANDLED without masking").  The
err_free_irqs path has the same gap: it frees handlers and disposes
subordinate mappings without masking already-registered subordinate
queues.

The later scale-up path uses register -> request_irq -> publish ->
replenish -> napi_enable -> enable_irq and documents why.  Would enabling
NAPI before request_irq (as the baseline did), or explicitly masking each
subordinate queue right after registration, close this window?

[ ... ]

> @@ -2373,18 +2632,25 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
>  	/* add size of mapped tx buffers */
>  	ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
>  
> -	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> -		/* add the size of the active receive buffers */
> -		if (adapter->rx_buff_pool[0][i].active)
> -			ret +=
> -			    adapter->rx_buff_pool[0][i].size *
> -			    IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
> -					     buff_size, tbl);
> -		rxqentries += adapter->rx_buff_pool[0][i].size;
> -	}
> -	/* add the size of the receive queue entries */
> -	ret += IOMMU_PAGE_ALIGN(
> -		rxqentries * sizeof(struct ibmveth_rx_q_entry), tbl);
> +	for (q = 0; q < adapter->num_rx_queues; q++) {
> +		int rxqentries = 1;

[Severity: Medium]
Does the buffer-list term still need scaling here?  The unchanged line
above this loop counts it once:

	ret = IBMVETH_BUFF_LIST_SIZE + IBMVETH_FILT_LIST_SIZE;

while multi-queue allocates and dma_map_single()s a distinct 4096-byte
buffer-list page per queue, which ibmveth_cleanup_rx_resources() unmaps
per index.  That leaves the desired value short by (num_rx_queues - 1)
buffer-list pages.

Also, is anything updating the CMO entitlement for the new default queue
count?  The VIO core samples it before the driver's probe runs:

arch/powerpc/platforms/pseries/vio.c:vio_cmo_bus_probe() {
	viodev->cmo.desired =
		IOMMU_PAGE_ALIGN(viodrv->get_desired_dma(viodev), tbl);
}

with dev_get_drvdata() still NULL, so the single-queue sized
IBMVETH_IO_ENTITLEMENT_DEFAULT is used for an adapter that will open with
up to 8 RX queues, and neither ibmveth_probe() nor ibmveth_open() calls
vio_cmo_set_dev_desired().  On a CMO partition, can this make
dma_map_single()/dma_alloc_coherent() fail during open on adapters that
worked in single-queue mode?

> +
> +		for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> +			/* add the size of the active receive buffers */
> +			struct ibmveth_buff_pool *bpool =
> +				&adapter->rx_buff_pool[q][i];

[Severity: Medium]
Is rx_buff_pool[q][i] populated for q >= 1 when this runs?  Pool metadata
for queues 1..N is only filled in by ibmveth_alloc_buffer_pools(), which
runs from ndo_open; ibmveth_probe() initializes only rx_buff_pool[0][i],
and the private area is zeroed, so before the first open these entries
have size = active = buff_size = 0.

ibmveth_change_mtu() pushes the result straight into firmware accounting:

	vio_cmo_set_dev_desired(viodev, ibmveth_get_desired_dma(viodev));

Can an MTU change while the interface is down therefore lower the
entitlement to roughly the single-queue value, right before an open that
allocates buffers for up to 8 queues?

> @@ -2449,9 +2715,18 @@ static const struct net_device_ops ibmveth_netdev_ops = {
>  #endif
>  };
>  
> +static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
> +				   int pools_ready)
> +{
> +	int i;
> +
> +	for (i = 0; i < pools_ready; i++)
> +		kobject_put(&adapter->rx_buff_pool[0][i].kobj);
> +}
> +

[ ... ]

> @@ -2531,16 +2810,27 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  		netdev->features |= NETIF_F_FRAGLIST;
>  	}
>  
> -	/* Initialize queue count - always 1 for now */
> -	adapter->multi_queue = 0;
> -	adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
> +	if (ret == H_SUCCESS &&
> +	    (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) {
> +		adapter->multi_queue = 1;
> +		adapter->num_rx_queues = min(num_online_cpus(),
> +					     IBMVETH_DEFAULT_QUEUES);
> +		netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n",
> +			   adapter->num_rx_queues);
> +	} else {
> +		adapter->multi_queue = 0;
> +		adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
> +	}

[Severity: Medium]
Now that num_rx_queues can be greater than one here, does
ibmveth_resume() need updating too?  It still kicks queue 0 only:

ibmveth_resume() {
	...
	ibmveth_schedule_rx_queue(adapter, 0);
	return 0;
}

The kick exists because the poll routine is what replenishes buffers and
re-arms the PHYP interrupt via napi_complete_done() ->
ibmveth_enable_irq().  After a suspend/resume or partition migration,
queues 1..N-1 would then have no buffers posted and no interrupt
re-armed until an ifdown/ifup.

> @@ -2558,15 +2848,24 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  
>  	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
>  		struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
> -		int error;
>  
>  		ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
>  					 pool_count[i], pool_size[i],
>  					 pool_active[i]);
> -		error = kobject_init_and_add(kobj, &ktype_veth_pool,
> -					     &dev->dev.kobj, "pool%d", i);
> -		if (!error)
> -			kobject_uevent(kobj, KOBJ_ADD);
> +		rc = kobject_init_and_add(kobj, &ktype_veth_pool,
> +					  &dev->dev.kobj, "pool%d", i);
> +		if (rc) {
> +			dev_err(&dev->dev,
> +				"failed to create pool%d kobject: %d\n", i, rc);
> +			/* init_and_add takes a ref even on failure */
> +			kobject_put(kobj);
> +			ibmveth_put_pool_kobjs(adapter, pools_ready);
> +			free_netdev(netdev);
> +			return rc;
> +		}
> +
> +		pools_ready++;
> +		kobject_uevent(kobj, KOBJ_ADD);
>  	}

[Severity: Medium]
Is it safe to free_netdev() immediately after ibmveth_put_pool_kobjs()?
The kobjects are embedded in netdev_priv(netdev), and ktype_veth_pool has
no release callback:

static struct kobj_type ktype_veth_pool = {
	.release        = NULL,
	...
};

With CONFIG_DEBUG_KOBJECT_RELEASE=y the final put defers cleanup using a
work item stored inside the kobject itself:

lib/kobject.c:kobject_release() {
#ifdef CONFIG_DEBUG_KOBJECT_RELEASE
	unsigned long delay = HZ + HZ * get_random_u32_below(4);
	...
	INIT_DELAYED_WORK(&kobj->release, kobject_delayed_cleanup);
	schedule_delayed_work(&kobj->release, delay);
#else
	kobject_cleanup(kobj);
#endif
}

free_netdev() then releases that memory with no window, so
kobject_delayed_cleanup() would read kobj->state_in_sysfs, kobj->parent
and kobj->name from freed storage.  The same shape applies to the two
other new put_pool_kobjs() call sites below.

> @@ -2574,6 +2873,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  	if (rc) {
>  		netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n",
>  			   rc);
> +		ibmveth_put_pool_kobjs(adapter, pools_ready);
>  		free_netdev(netdev);
>  		return rc;
>  	}
> @@ -2590,6 +2890,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  
>  	if (rc) {
>  		netdev_dbg(netdev, "failed to register netdev rc=%d\n", rc);
> +		ibmveth_put_pool_kobjs(adapter, pools_ready);
>  		free_netdev(netdev);
>  		return rc;
>  	}

[ ... ]

Re: [PATCH net-next v4 10/14] ibmveth: Add per-queue RX and TX statistics collection

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add per-queue RX and TX statistics collection

This patch adds per-queue RX and TX counter arrays that live for the
adapter lifetime, exports them through ethtool -S, and adds an
ndo_get_stats64 implementation that sums them. It also changes what
ibmveth_get_channels() reports for max_rx and rx_count.

A few questions below.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 4ad7ced3c6083..dbb47a42b4cb5 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -921,6 +975,8 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
>  		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
>  		u64 drops = be64_to_cpup(p);
>  
> +		if (adapter->rx_qstats)
> +			adapter->rx_qstats[i].no_buffer_drops = drops;
>  		adapter->rx_no_buffer += drops;
>  	}
>  }

[Severity: Medium]

Can concurrent polls on different queues corrupt these counters?

ibmveth_update_rx_no_buffer() writes rx_qstats[i] for every queue, but
its caller ibmveth_replenish_task(adapter, queue_index) holds only that
one queue's lock:

	spin_lock_irqsave(&rxq->replenish_lock, flags);
	...
	ibmveth_update_rx_no_buffer(adapter);
	spin_unlock_irqrestore(&rxq->replenish_lock, flags);

So two NAPI polls on different queues run the same all-queue loop under
disjoint locks (replenish_lock[0] vs replenish_lock[1]):

CPU A: reads queue 1 hypervisor value 100
CPU B: reads queue 1 hypervisor value 105, stores 105
CPU A: stores 100

Does that make rxN_no_buffer_drops move backwards as seen by ethtool?
The adapter->rx_no_buffer = 0 / += drops sequence around it is also an
unsynchronized read-modify-write visible to a concurrent ethtool reader.

This also seems to work against the ____cacheline_aligned_in_smp
annotation added to struct ibmveth_rx_queue_stats, since every replenish
cycle now dirties every queue's stats cache line from a foreign CPU.

Would passing queue_index into the helper and touching only
rx_qstats[queue_index] and buffer_list_addr[queue_index] work, deriving
the adapter-level rx_no_buffer by summing on read the way the patch
already does for rx_large_packets and rx_invalid_buffer?

Related: at the end of the series the RX scale-down path in
ibmveth_resize_rx_queues_incremental() lowers adapter->num_rx_queues and
then frees a queue's buffer_list page while polls on surviving queues keep
running. Can a poll that already loaded the old bound dereference
buffer_list_addr[i] for a freed page here?

[Severity: Medium]

Should no_buffer_drops be accumulated rather than assigned?

The value PHYP writes into the last 8 bytes of the buffer-list page is an
absolute count for the life of that page, and the page is re-obtained with
get_zeroed_page(GFP_KERNEL) by ibmveth_alloc_rx_queues() on every
ibmveth_open() and released by ibmveth_cleanup_rx_resources() on every
ibmveth_close(), including the close/open pairs done by the reset work,
ibmveth_set_csum_offload() and ibmveth_set_tso().

Since rx_qstats[i].no_buffer_drops is assigned with "= drops", does
rxN_no_buffer_drops (and the recomputed adapter->rx_no_buffer) jump
backwards after an ifdown/ifup, a driver reset, or an ethtool -K tso
change?

That appears to contradict the comment this patch adds above the sum
helpers:

	 * globals on the hot path (ibmvnic-style); with qstats allocated for the
	 * adapter lifetime, these sums remain meaningful across ifdown/up.

> @@ -1972,22 +2028,131 @@ static int ibmveth_set_features(struct net_device *dev,
>   * globals on the hot path (ibmvnic-style); with qstats allocated for the
>   * adapter lifetime, these sums remain meaningful across ifdown/up.
>   */
> +static u64 ibmveth_sum_rx_invalid_buffers(struct ibmveth_adapter *adapter)
> +{
> +	u64 total = 0;
> +	int i;
> +
> +	if (!adapter->rx_qstats)
> +		return adapter->rx_invalid_buffer;
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++)
> +		total += adapter->rx_qstats[i].invalid_buffers;
> +
> +	return total;
> +}

[ ... ]

> +static u64 ibmveth_sum_tx_send_failed(struct ibmveth_adapter *adapter)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +	u64 total = 0;
> +	int i;
> +
> +	if (!adapter->tx_qstats)
> +		return adapter->tx_send_failed;
> +
> +	for (i = 0; i < netdev->real_num_tx_queues; i++)
> +		total += adapter->tx_qstats[i].send_failures;
> +
> +	return total;
> +}

[Severity: Medium]

Do these sums go backwards when the queue count is reduced?

The qstats arrays persist for the adapter lifetime, but the sums are
bounded by the currently configured queue count. ibmveth_set_channels()
lowers netdev->real_num_tx_queues:

	rc = netif_set_real_num_tx_queues(netdev, goal);

Nothing folds or clears the counters of the queues that disappear, so
after "ethtool -L ethX tx N" with a smaller N, do the adapter-level
ethtool strings tx_large_packets and tx_send_failed, plus the
rtnl_link_stats64 tx_packets/tx_bytes/tx_dropped summed in
ibmveth_get_stats64(), all drop by the traffic accounted to the removed
queues, and then jump forward again when the count is raised?

Since this patch removes the hot-path netdev->stats accounting,
ibmveth_get_stats64() is now the only source for the rtnl counters, so a
decrease is directly user visible; tools computing deltas (ip -s link,
ifstat) will read it as a counter wrap. The same applies on the RX side
with adapter->num_rx_queues once the incremental RX resize commit later in
this series changes it at runtime.

> +static u64 ibmveth_ethtool_adapter_stat(struct ibmveth_adapter *adapter,
> +					int index)
> +{
> +	unsigned long offset = ibmveth_stats[index].offset;
> +
> +	if (offset == IBMVETH_STAT_OFF(rx_invalid_buffer))
> +		return ibmveth_sum_rx_invalid_buffers(adapter);
> +	if (offset == IBMVETH_STAT_OFF(rx_large_packets))
> +		return ibmveth_sum_rx_large_packets(adapter);
> +	if (offset == IBMVETH_STAT_OFF(tx_large_packets))
> +		return ibmveth_sum_tx_large_packets(adapter);
> +	if (offset == IBMVETH_STAT_OFF(tx_send_failed))
> +		return ibmveth_sum_tx_send_failed(adapter);
> +
> +	return IBMVETH_GET_STAT(adapter, offset);
> +}
> +
>  static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
>  {
> +	struct ibmveth_adapter *adapter = netdev_priv(dev);
> +	u8 *p = data;
>  	int i;
>  
>  	if (stringset != ETH_SS_STATS)
>  		return;
>  
> -	for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++, data += ETH_GSTRING_LEN)
> -		memcpy(data, ibmveth_stats[i].name, ETH_GSTRING_LEN);
> +	for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++) {
> +		memcpy(p, ibmveth_stats[i].name, ETH_GSTRING_LEN);
> +		p += ETH_GSTRING_LEN;
> +	}
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++) {
> +		ethtool_sprintf(&p, "rx%d_packets", i);
> +		ethtool_sprintf(&p, "rx%d_bytes", i);
> +		ethtool_sprintf(&p, "rx%d_interrupts", i);
> +		ethtool_sprintf(&p, "rx%d_polls", i);
> +		ethtool_sprintf(&p, "rx%d_large_packets", i);
> +		ethtool_sprintf(&p, "rx%d_invalid_buffers", i);
> +		ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
> +	}
> +
> +	for (i = 0; i < dev->real_num_tx_queues; i++) {
> +		ethtool_sprintf(&p, "tx%d_packets", i);
> +		ethtool_sprintf(&p, "tx%d_bytes", i);
> +		ethtool_sprintf(&p, "tx%d_large_packets", i);
> +		ethtool_sprintf(&p, "tx%d_dropped_packets", i);
> +		ethtool_sprintf(&p, "tx%d_send_failures", i);
> +		ethtool_sprintf(&p, "tx%d_checksum_offload", i);
> +	}

[Severity: Medium]

Should the per-queue packet and byte counters use the standard per-queue
statistics interface instead of driver-private ethtool strings?

rx%d_packets, rx%d_bytes, tx%d_packets and tx%d_bytes map one to one onto
struct netdev_queue_stats_rx and struct netdev_queue_stats_tx, which are
exported through netlink via struct netdev_stat_ops. No netdev_stat_ops
is added here, so generic tooling (ynl) cannot consume these values, and
Documentation/networking/statistics.rst asks that counters with a standard
uAPI not be duplicated in ethtool -S.

Also, the new ibmveth_get_stats64() populates only rx/tx packets, bytes
and tx_dropped:

	stats->tx_errors = dev->stats.tx_errors;

After this patch, is there any remaining writer of dev->stats in this
driver? The only match in the file is this read, so tx_errors appears to
be permanently 0. The loss counters this patch does collect
(no_buffer_drops from the hypervisor, invalid_buffers) are not mapped
into rx_dropped / rx_missed_errors / rx_errors either, so ip -s link
reports zero RX drops while ethtool -S shows a non-zero rx_no_buffer.

> +
> +	for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> +		ethtool_sprintf(&p, "pool%d_size", i);
> +		ethtool_sprintf(&p, "pool%d_active", i);
> +		ethtool_sprintf(&p, "pool%d_available", i);
> +	}
>  }
>  
>  static int ibmveth_get_sset_count(struct net_device *dev, int sset)
>  {
> +	struct ibmveth_adapter *adapter = netdev_priv(dev);
> +
>  	switch (sset) {
>  	case ETH_SS_STATS:
> -		return ARRAY_SIZE(ibmveth_stats);
> +		return ARRAY_SIZE(ibmveth_stats) +
> +		       adapter->num_rx_queues * IBMVETH_NUM_RX_QSTATS +
> +		       dev->real_num_tx_queues * IBMVETH_NUM_TX_QSTATS +
> +		       IBMVETH_NUM_BUFF_POOLS * 3;
>  	default:
>  		return -EOPNOTSUPP;
>  	}

[ ... ]

> @@ -1996,21 +2161,59 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
>  static void ibmveth_get_ethtool_stats(struct net_device *dev,
>  				      struct ethtool_stats *stats, u64 *data)
>  {
> -	int i;
>  	struct ibmveth_adapter *adapter = netdev_priv(dev);
> +	int i, j;
>  
>  	for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++)
> -		data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset);
> +		data[i] = ibmveth_ethtool_adapter_stat(adapter, i);
> +
> +	for (j = 0; j < adapter->num_rx_queues; j++) {
> +		if (adapter->rx_qstats) {
> +			data[i++] = adapter->rx_qstats[j].packets;
> +			data[i++] = adapter->rx_qstats[j].bytes;
> +			data[i++] = adapter->rx_qstats[j].interrupts;
> +			data[i++] = adapter->rx_qstats[j].polls;
> +			data[i++] = adapter->rx_qstats[j].large_packets;
> +			data[i++] = adapter->rx_qstats[j].invalid_buffers;
> +			data[i++] = adapter->rx_qstats[j].no_buffer_drops;
> +		} else {
> +			i += IBMVETH_NUM_RX_QSTATS;
> +		}
> +	}

[Severity: Low]

Can adapter->rx_qstats or adapter->tx_qstats ever be NULL here?

The allocations happen in ibmveth_probe() before register_netdev(), and a
failure aborts probe with ibmveth_probe_cleanup() + return -ENOMEM.
ibmveth_remove() frees them only after unregister_netdev(). So on a
registered netdev the NULL case seems unreachable, which would make these
else arms, the else arms in ibmveth_poll(), and the
"if (!adapter->*_qstats) return adapter-><legacy>;" early returns in the
four sum helpers dead code.

The two styles also disagree: ibmveth_is_packet_unsupported() guards with
"if (adapter->tx_qstats)" while ibmveth_start_xmit() dereferences
adapter->tx_qstats[queue_num] unconditionally at six new sites on the same
call chain. Which of the two is intended?

A side effect is that adapter->rx_invalid_buffer, rx_large_packets,
tx_large_packets and tx_send_failed are never incremented again while
still being listed in ibmveth_stats[].

Separately, ibmveth_ethtool_adapter_stat() dispatches by comparing
ibmveth_stats[index].offset against IBMVETH_STAT_OFF() values. Would
adding or reordering a field in struct ibmveth_adapter silently redirect a
statistic to the wrong source here?

> +
> +	for (j = 0; j < dev->real_num_tx_queues; j++) {
> +		if (adapter->tx_qstats) {
> +			data[i++] = adapter->tx_qstats[j].packets;
> +			data[i++] = adapter->tx_qstats[j].bytes;
> +			data[i++] = adapter->tx_qstats[j].large_packets;
> +			data[i++] = adapter->tx_qstats[j].dropped_packets;
> +			data[i++] = adapter->tx_qstats[j].send_failures;
> +			data[i++] = adapter->tx_qstats[j].checksum_offload;
> +		} else {
> +			i += IBMVETH_NUM_TX_QSTATS;
> +		}
> +	}
> +
> +	for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
> +		data[i++] = adapter->rx_buff_pool[0][j].size;
> +		data[i++] = adapter->rx_buff_pool[0][j].active;
> +		data[i++] = atomic_read(&adapter->rx_buff_pool[0][j].available);
> +	}
>  }

[Severity: Medium]

Should the pool%d_* strings be queue-indexed?

The driver now keeps an independent pool set per RX queue
(rx_buff_pool[queue][pool], populated for all q < adapter->num_rx_queues
by ibmveth_alloc_buffer_pools() and updated per queue by
ibmveth_replenish_task()), but these three entries read only
rx_buff_pool[0][j] under queue-agnostic names, so the state of queues
1..N-1 is not visible. Is that intended for someone debugging drops on a
non-zero queue?

Also, size and active are configuration values already exposed through the
per-pool sysfs kobjects rather than statistics, and this new permanent
ethtool string set is not mentioned in the commit message.

>  static void ibmveth_get_channels(struct net_device *netdev,
>  				 struct ethtool_channels *channels)
>  {
> +	struct ibmveth_adapter *adapter = netdev_priv(netdev);
> +
>  	channels->max_tx = ibmveth_real_max_tx_queues();
>  	channels->tx_count = netdev->real_num_tx_queues;
>  
> -	channels->max_rx = netdev->real_num_rx_queues;
> -	channels->rx_count = netdev->real_num_rx_queues;
> +	if (adapter->multi_queue)
> +		channels->max_rx = IBMVETH_MAX_RX_QUEUES;
> +	else
> +		channels->max_rx = 1;
> +	channels->rx_count = adapter->num_rx_queues;
>  }

[Severity: Low]

This isn't a bug, but would this ABI-visible get_channels() reporting
change be easier to review as its own patch? It is independent of the
statistics work, and if it is a fix it would want its own Fixes: tag.

There is also a mismatch it introduces: the new RX loops in
ibmveth_get_strings() / ibmveth_get_sset_count() / ibmveth_get_stats64()
use adapter->num_rx_queues while the TX loops use
dev->real_num_tx_queues. netdev->real_num_rx_queues is only synced to
adapter->num_rx_queues in ibmveth_open(), so on a never-opened interface
ethtool -l reports a count the stack does not have yet.

[ ... ]

> @@ -2150,6 +2355,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
>  	    skb_checksum_help(skb)) {
>  
>  		netdev_err(netdev, "tx: failed to checksum packet\n");
> +		adapter->tx_qstats[queue_num].dropped_packets++;
>  		goto out;
>  	}

[ ... ]

> @@ -2211,7 +2423,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
>  	dma_wmb();
>  
>  	if (ibmveth_send(adapter, desc.desc, mss)) {
> +		adapter->tx_qstats[queue_num].send_failures++;
> +		adapter->tx_qstats[queue_num].dropped_packets++;
>  	} else {
> +		adapter->tx_qstats[queue_num].packets++;
> +		adapter->tx_qstats[queue_num].bytes += skb->len;
>  	}

[Severity: Medium]

These empty if/else bodies in the pre-image show that the previous commit
in the series ("ibmveth: Enable multi-queue RX receive path") deleted the
hot-path netdev->stats accounting, and the replacement only arrives here.
ibmveth_poll() likewise increments no packet or byte counter at that
commit, and no ndo_get_stats64 exists yet.

Does that leave the tree at the parent commit reporting zero
rx_packets/rx_bytes/tx_packets/tx_bytes/tx_dropped through ip -s link,
where the pre-series baseline reported them? Would squashing the removal
into this patch, or deferring the removal until this replacement lands,
keep every commit in the series bisectable for counter-related issues?

[ ... ]

> @@ -2698,6 +2933,40 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,
>  	return vlan_features_check(skb, features);
>  }
>  

[ ... ]

>  static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
> -				   int pools_ready)
> +				  int pools_ready)
>  {
>  	int i;

[Severity: Low]

This isn't a bug, but this whitespace-only reflow breaks the previously
correct open-parenthesis alignment of the second parameter (checkpatch:
"Alignment should match open parenthesis") in a function this patch does
not otherwise change. Could it be dropped?

> @@ -2724,6 +2994,19 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
>  		kobject_put(&adapter->rx_buff_pool[0][i].kobj);
>  }
>  
> +static void ibmveth_probe_cleanup(struct ibmveth_adapter *adapter,
> +				  int pools_ready)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +
> +	cancel_work_sync(&adapter->work);
> +	ibmveth_put_pool_kobjs(adapter, pools_ready);
> +
> +	ibmveth_free_tx_qstats(adapter);
> +	ibmveth_free_rx_qstats(adapter);
> +	free_netdev(netdev);
> +}
> +

[Severity: High]

This isn't a problem introduced by this patch, since the old open-coded
error paths had the same omission, but the new helper is now the single
place four probe failure paths funnel through, so it looks like the
natural spot to fix it.

ibmveth_probe() stores the netdev in the VIO drvdata right after
alloc_etherdev_mqs():

	dev_set_drvdata(&dev->dev, netdev);

ibmveth_probe_cleanup() ends with free_netdev(netdev) but never clears it,
and ibmveth_remove() (the only caller of dev_set_drvdata(&dev->dev, NULL))
does not run when probe fails. So after any probe failure the vio_dev
keeps a pointer to freed memory.

On FW_FEATURE_CMO systems the stale pointer is consumed on the next bind
attempt, before the driver probe re-initializes it:

vio_bus_probe()
  vio_cmo_bus_probe()
    viodev->cmo.desired = IOMMU_PAGE_ALIGN(viodrv->get_desired_dma(viodev), tbl);
      ibmveth_get_desired_dma()

ibmveth_get_desired_dma() only checks for NULL:

	netdev = dev_get_drvdata(&vdev->dev);
	if (netdev == NULL)
		return ...;

and then dereferences netdev->mtu and adapter->num_rx_queues, using the
latter as the loop bound over adapter->rx_buff_pool[q][i]. Since that
bound is read from freed memory, can this read past the end of the freed
allocation as well? Would adding dev_set_drvdata(&dev->dev, NULL) to
ibmveth_probe_cleanup() close this?

>  static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  {
>  	int rc, i, mac_len, pools_ready = 0;
> @@ -2779,6 +3062,11 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  		netif_napi_add_weight(netdev, &adapter->napi[i],
>  				      ibmveth_poll, 16);
>  
> +	if (ibmveth_alloc_rx_qstats(adapter) ||
> +	    ibmveth_alloc_tx_qstats(adapter)) {
> +		ibmveth_probe_cleanup(adapter, 0);
> +		return -ENOMEM;
> +	}
>  
>  	netdev->irq = dev->irq;
>  	netdev->netdev_ops = &ibmveth_netdev_ops;

[ ... ]

> @@ -2913,6 +3198,9 @@ static void ibmveth_remove(struct vio_dev *dev)
>  
>  	unregister_netdev(netdev);
>  
> +	ibmveth_free_tx_qstats(adapter);
> +	ibmveth_free_rx_qstats(adapter);
> +
>  	free_netdev(netdev);
>  	dev_set_drvdata(&dev->dev, NULL);
>  }

[Severity: High]

This is a pre-existing issue, since the cancel_work_sync() placement in
ibmveth_remove() predates this patch, but the patch adds two new frees
into exactly that window.

ibmveth_remove() calls cancel_work_sync(&adapter->work) at the top,
before unregister_netdev(), so the device is still open when the cancel
returns and softirq context can re-arm the work:

ibmveth_poll()                     /* on ibmveth_enable_irq() failure */
  schedule_work(&adapter->work);

ibmveth_replenish_buffer_pool()    /* on an invalid free_map index */
  schedule_work(&adapter->work);

Timeline:

CPU A: cancel_work_sync() returns, queue empty
CPU B: ibmveth_poll() -> schedule_work(&adapter->work)
CPU A: unregister_netdev()
CPU A: ibmveth_free_tx_qstats() / ibmveth_free_rx_qstats()
CPU A: free_netdev(netdev)
CPU B: ibmveth_reset() runs container_of(w, struct ibmveth_adapter, work)

Can the worker then dereference the freed adapter and netdev in
rtnl_lock(); dev_close(adapter->netdev); dev_open(...)?

If it reaches dev_open() -> netif_tx_start_all_queues(), TX restarts with
tx_qstats == NULL, and the unguarded adapter->tx_qstats[queue_num]
dereferences added to ibmveth_start_xmit() by this patch would then be a
NULL dereference. The new "if (adapter->rx_qstats)" hot-path guards do not
help, since the adapter holding the pointer is itself inside the freed
netdev.

Would moving the cancel (or disable_work_sync()) to after
unregister_netdev() address this?

Re: [PATCH net-next v4 11/14] ibmveth: Expose per-queue buffer pool details via debugfs

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Expose per-queue buffer pool details via debugfs

This adds a read-only buffer_pools debugfs file that dumps size,
buff_size, active and available for every RX queue and buffer pool, and
wires creation into ibmveth_probe() and teardown into ibmveth_remove().
The existing queue-0 poolN sysfs files are left unchanged.

A few questions about the new code below.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index dbb47a42b4cb5..db13747466025 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -2985,6 +2986,44 @@ static const struct net_device_ops ibmveth_netdev_ops = {
>  #endif
>  };
>  
> +static int ibmveth_buffer_pools_show(struct seq_file *m, void *v)
> +{
> +	struct ibmveth_adapter *adapter = m->private;
> +	int i, j;
> +
> +	seq_puts(m, "Queue  Pool  Size  BuffSize  Active  Available\n");
> +	seq_puts(m, "-----  ----  ----  --------  ------  ---------\n");
> +
> +	for (i = 0; i < adapter->num_rx_queues; i++) {
> +		for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
> +			struct ibmveth_buff_pool *pool =
> +				&adapter->rx_buff_pool[i][j];

[Severity: Low]
Should this dump be taken under rtnl_lock()?

ibmveth_buffer_pools_show() reads adapter->num_rx_queues plus
pool->size, pool->buff_size, pool->active and pool->available with no
lock held, but every writer of those fields runs under RTNL:

veth_pool_store() {
	...
	rtnl_lock();
	...
	pool->active = newactive;
	pool->buff_size = newbuff_size;
	pool->size = newsize;
	...
	/* on ibmveth_open() failure, the same three fields are
	 * written back with the old values */
			pool->active = oldactive;
			pool->buff_size = oldbuff_size;
			pool->size = oldsize;
	...
}

ibmveth_alloc_buffer_pools() likewise copies the queue-0 geometry into
queues 1..N-1 field by field under RTNL from ibmveth_open(), and the
reset path goes through dev_close()/dev_open() under rtnl_lock().

Can a concurrent read of buffer_pools therefore print a torn snapshot,
for example active=1 paired with the previous size/buff_size, or a
half-copied per-queue template, or the transient rollback state that was
never an actual configuration?

There does not appear to be a memory-safety consequence here, since
rx_buff_pool[] is a fixed-size array embedded in struct ibmveth_adapter
and only scalar members are read, so the concern is limited to the
values printed.

> +
> +			seq_printf(m, "%5d  %4d  %4u  %8u  %6d  %9d\n",
> +				   i, j, pool->size, pool->buff_size,
> +				   pool->active,
> +				   atomic_read(&pool->available));

[Severity: Low]
Are Active and Available meaningful while the interface is down?

ibmveth_free_buffer_pool() frees free_map, unmaps and frees all
skbuffs, and frees dma_addr/skbuff, but leaves pool->active set and
never decrements pool->available:

	kfree(pool->free_map);
	pool->free_map = NULL;
	...
	if (pool->skbuff) {
		kfree(pool->skbuff);
		pool->skbuff = NULL;
	}
	/* pool->active and pool->available untouched */

available is only cleared later, in ibmveth_alloc_buffer_pool() via
atomic_set(&pool->available, 0) on the next open.

So a read after ibmveth_close() -> ibmveth_free_buffer_pools() ->
ibmveth_free_queue_buffer_pools() -> ibmveth_free_buffer_pool() reports
every configured pool as Active=1 with the last non-zero Available,
even though the skbuffs and DMA mappings are gone.

Related: for queues 1..N-1 the table prints all-zero rows before the
first open, because the queue-0 template is only copied into those
queues inside ibmveth_alloc_buffer_pools() during open.

The commit message says "Per-queue runtime pressure is what debugfs
shows" - could the down/uninitialized cases be either reflected in the
output or documented?

> +		}
> +	}
> +
> +	return 0;
> +}
> +DEFINE_SHOW_ATTRIBUTE(ibmveth_buffer_pools);
> +
> +static void ibmveth_debugfs_init(struct ibmveth_adapter *adapter)
> +{
> +	adapter->debugfs_dir = debugfs_create_dir(adapter->netdev->name,
> +						  NULL);
> +	debugfs_create_file("buffer_pools", 0400, adapter->debugfs_dir,
> +			    adapter, &ibmveth_buffer_pools_fops);
> +}

[Severity: Medium]
Is the mutable interface name a safe key for a directory created
directly in the debugfs root?

Three separate things follow from passing NULL as the parent and
netdev->name as the name.

First, the name goes stale. The directory is created once, as the last
statement of ibmveth_probe(), and the driver installs no netdev
notifier and never calls debugfs_rename(). After the udev/systemd
predictable-name rename that happens right after register_netdev(), or
after any later ip link set dev ethN name foo, does the debugfs
directory keep the obsolete name for the lifetime of the device?

Second, with the debugfs root as parent the name has to be globally
unique there. Consider the sequence: first adapter probes as eth0, udev
renames it to enP1p1s0, a second ibmveth adapter probes and
alloc_etherdev_mqs() names it eth0 again because eth0 is free in the
netdev namespace. debugfs_create_dir("eth0", NULL) then collides with
the first adapter's stale directory:

fs/debugfs/inode.c:debugfs_start_creating() {
	...
	dentry = simple_start_creating(parent, name);
	if (IS_ERR(dentry)) {
		if (dentry == ERR_PTR(-EEXIST))
			pr_err("'%s' already exists in '%pd'\n", name, parent);
	...
}

The ERR_PTR is stored in adapter->debugfs_dir and handed to
debugfs_create_file() as the parent, which returns immediately:

fs/debugfs/inode.c:debugfs_start_creating() {
	...
	if (IS_ERR(parent))
		return parent;
	...
}

Does the second and every subsequent adapter then end up with no
buffer_pools file at all, with only that debugfs pr_err as an
indication?

Third, netdev->name is read here after register_netdev() has published
the device, without rtnl_lock(), while dev_change_name() rewrites
net_device->name under RTNL. Can the string handed to
debugfs_create_dir() be a partially updated name?

Would a driver-owned parent directory plus a stable identifier such as
dev_name(&dev->dev) avoid all three?

> +
> +static void ibmveth_debugfs_exit(struct ibmveth_adapter *adapter)
> +{
> +	debugfs_remove_recursive(adapter->debugfs_dir);
> +	adapter->debugfs_dir = NULL;
> +}
> +
>  static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
>  				  int pools_ready)
>  {
> @@ -3182,6 +3221,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>  
>  	netdev_dbg(netdev, "registered\n");
>  
> +	ibmveth_debugfs_init(adapter);
> +
>  	return 0;
>  }

[ ... ]

Cross-instance finding from sashiko-gemini (b6bfc50872782421f7178a42615ea39fe861918e607d995ed830a5a2db8a4460):
[Severity: Medium]
Addition of driver-private ethtool -S strings for standard queue statistics.

Re: [PATCH net-next v4 12/14] ibmveth: Implement incremental MQ RX queue resize

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Implement incremental MQ RX queue resize

Adds per-queue alloc/register/irq-setup/free helpers plus
ibmveth_resize_rx_queues_incremental() so the live RX queue count can grow
or shrink without tearing down surviving queues, and wires
ibmveth_resize_rx_channels() into set_channels() when the interface is up.

A few questions on the new resize paths below.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index db13747466025..b57c7df92853f 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c

[ ... ]

> @@ -1307,6 +1494,51 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
>  	return 0;
>  }
>  
> +/**
> + * ibmveth_drain_rx_queue - Drain pending buffers from an RX queue
> + * @adapter: ibmveth adapter structure
> + * @queue_index: Queue index to drain
> + *
> + * Recycles all pending buffers back to the per-queue buffer pools.
> + * Must be called with NAPI disabled for this queue.
> + *
> + * Return: Number of buffers drained
> + */
> +static int
> +ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
> +{
> +	struct net_device *netdev = adapter->netdev;
> +	int drained = 0;
> +	int limit = adapter->rx_queue[queue_index].num_slots;
> +	int rc;
> +
> +	netdev_dbg(netdev, "Draining RX queue %d (limit: %d slots)\n",
> +		   queue_index, limit);
> +
> +	while (drained < limit &&
> +	       ibmveth_rxq_pending_buffer(adapter, queue_index)) {
> +		/* Match poll-side order before harvesting completion state. */
> +		smp_rmb();
> +		rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
> +		if (rc) {
> +			netdev_err(netdev,
> +				   "Failed to harvest buffer from queue %d during drain: %d\n",
> +				   queue_index, rc);
> +			break;
> +		}

[Severity: Low]
Can this error arm ever run?  ibmveth_rxq_harvest_buffer() normalizes
every failure to a 0 return:

	rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
	if (unlikely(rc)) {
		if (rc == -EINVAL || rc == -EFAULT)
			goto advance;
		return rc;
	}

advance:
	ibmveth_rxq_advance(rxq);

	return 0;

and ibmveth_remove_buffer_from_pool() only ever returns 0, -EINVAL or
-EFAULT, so both non-zero values take the goto advance path.  The
netdev_err()/break looks like dead code.

> +		drained++;
> +	}
> +
> +	if (drained > 0)
> +		netdev_dbg(netdev, "Drained %d buffer(s) from RX queue %d\n",
> +			   drained, queue_index);
> +	else
> +		netdev_dbg(netdev, "No buffers to drain from RX queue %d\n",
> +			   queue_index);
> +
> +	return drained;
> +}
> +

[ ... ]

> @@ -1562,6 +1794,227 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
>  	return 0;
>  }
>  

[ ... ]

> +static int
> +ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
> +				     int new_count, int rxq_entries)
> +{

[ ... ]

> +		for (i = old_count; i < new_count; i++) {
> +			rc = ibmveth_alloc_single_rx_queue(adapter, i,
> +							   rxq_entries);

[ ... ]

> +			/*
> +			 * Fully ready before PHYP delivery, matching open():
> +			 * publish -> replenish -> napi_enable -> enable_irq.
> +			 * That way ibmveth_interrupt() cannot run on an
> +			 * unpublished, empty, or NAPI-disabled queue.
> +			 */
> +			adapter->num_rx_queues = i + 1;

[Severity: Medium]
Should this publish store be ordered?  adapter->num_rx_queues is the only
gate that makes rx_queue[i].queue_addr/num_slots/toggle/replenish_lock,
rx_buff_pool[i][*] and buffer_list_addr[i] visible to lockless readers,
and it is written with a plain store here while the queues added in
earlier iterations are already running NAPI with PHYP unmasked.

The consumers also use plain loads from hardirq and softirq context:

ibmveth_interrupt():
	if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
		return IRQ_NONE;

ibmveth_poll():
	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
		return 0;

and ibmveth_update_rx_no_buffer() dereferences buffer_list_addr[i] for
every i below the count.  On powerpc, can a reader see the incremented
count while still observing stale contents of that slot (for example a
buffer_list_addr[i] left NULL by an earlier scale-down), or the converse
stale count for the queue whose interrupt was just unmasked?  Would
WRITE_ONCE()/READ_ONCE() plus a release barrier before the publish be
appropriate here?

> +			ibmveth_replenish_task(adapter, i);
> +			napi_enable(&adapter->napi[i]);
> +
> +			rc = ibmveth_enable_irq(adapter, i);

[Severity: Medium]
After the unmask, should the queue be re-checked for descriptors that
PHYP enqueued while delivery was masked?  Buffers are posted by
ibmveth_replenish_task() before ibmveth_enable_irq(), so PHYP can write
descriptors during the masked window.  ibmveth_poll() does exactly this
compensation:

	lpar_rc = ibmveth_enable_irq(adapter, queue_index);
	...
	if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
	    napi_schedule(napi)) {
		ibmveth_disable_irq(adapter, queue_index);
		goto restart_poll;
	}

Without an equivalent pending check plus napi_schedule() here (and in the
scale-down rollback loop further down, which is even more exposed since
PHYP may have enqueued during the disable/drain window), can the new
queue sit with unharvested descriptors until unrelated traffic raises the
next interrupt?

> +			if (rc) {
> +				netdev_err(netdev,
> +					   "Failed to enable IRQ for queue %d: %d\n",
> +					   i, rc);
> +				adapter->num_rx_queues = i;
> +				napi_disable(&adapter->napi[i]);
> +				ibmveth_cleanup_single_rx_interrupt(adapter, i);
> +				ibmveth_deregister_single_rx_queue(adapter, i);
> +				ibmveth_free_single_rx_queue(adapter, i);
> +				goto cleanup_new_queues;
> +			}

[Severity: Medium]
Is rc an errno at this point?  ibmveth_enable_irq() -> ibmveth_toggle_irq()
returns the raw plpar_hcall_norets() result and only normalizes
H_PARAMETER to 0, so rc can be H_HARDWARE (-1), H_FUNCTION (-2),
H_PRIVILEGE (-3) or the positive H_BUSY (1).  That value is returned
unchanged through ibmveth_resize_rx_channels() -> ibmveth_set_channels()
into the ethtool ioctl, so userspace sees EPERM/ENOENT/ESRCH, or for a
positive code a positive ioctl return that ethtool reads as success even
though the resize failed and unwound.

The pre-existing ibmveth_setup_rx_interrupts() converts the same failure
with rc = -EIO; should this path do the same?

> +		}
> +
> +		rc = netif_set_real_num_rx_queues(netdev, new_count);
> +		if (rc) {
> +			netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
> +				   new_count, rc);
> +			goto cleanup_new_queues;
> +		}
> +	} else {
> +		netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
> +			   new_count, old_count - 1);
> +
> +		/*
> +		 * Mask PHYP delivery before napi_disable/drain. Otherwise
> +		 * ibmveth_interrupt returns IRQ_HANDLED without masking when
> +		 * NAPI is disabled, and the HV can storm during drain.
> +		 */
> +		for (i = new_count; i < old_count; i++) {
> +			ibmveth_disable_irq(adapter, i);
> +			synchronize_irq(adapter->queue_irq[i]);
> +		}
> +
> +		for (i = new_count; i < old_count; i++)
> +			napi_disable(&adapter->napi[i]);

[Severity: Medium]
Does the mask actually hold across the drain?  synchronize_irq() waits
for the hardirq handler, not for an ibmveth_poll() already in flight on
another CPU, and that poll re-arms delivery unconditionally:

CPU0 (resize)
	ibmveth_disable_irq(adapter, i);
	synchronize_irq(adapter->queue_irq[i]);   /* handler idle, returns */
	napi_disable(&adapter->napi[i]);          /* returns after
						     napi_complete_done() */

CPU1 (ibmveth_poll for queue i, scheduled before the mask)
	if (!napi_complete_done(napi, frames_processed))
		goto out;

	lpar_rc = ibmveth_enable_irq(adapter, queue_index);  /* unmasked */

so the drain can run with PHYP delivery re-enabled.  With NAPI disabled,
ibmveth_schedule_rx_queue() fails napi_schedule_prep() and returns
without masking, and once adapter->num_rx_queues is lowered each further
delivery hits WARN_ON(qindex >= adapter->num_rx_queues) in
ibmveth_interrupt() and returns IRQ_NONE.  Is the storm rule stated in
the comment still guaranteed?  The same ordering is repeated in the
cleanup_new_queues label below.

> +
> +		for (i = new_count; i < old_count; i++)
> +			ibmveth_drain_rx_queue(adapter, i);
> +
> +		synchronize_net();
> +
> +		rc = netif_set_real_num_rx_queues(netdev, new_count);
> +		if (rc) {
> +			netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
> +				   new_count, rc);
> +			for (i = new_count; i < old_count; i++) {
> +				ibmveth_replenish_task(adapter, i);
> +				napi_enable(&adapter->napi[i]);
> +				ibmveth_enable_irq(adapter, i);
> +			}
> +			return rc;
> +		}

[Severity: Medium]
Should the ibmveth_enable_irq() return code be checked in this rollback
loop?  This is the only new call site that discards it.

If the H_VIOCTL enable fails for one of these queues, the queue stays in
the live set (adapter->num_rx_queues is still old_count) with NAPI
enabled and buffers posted, but with PHYP delivery masked, so that RX
queue is silently stalled until a close/open or another resize.

Elsewhere the driver treats this as actionable, e.g. ibmveth_poll():

	lpar_rc = ibmveth_enable_irq(adapter, queue_index);
	if (lpar_rc != H_SUCCESS) {
		...
		schedule_work(&adapter->work);
	}

> +
> +		adapter->num_rx_queues = new_count;

[Severity: Medium]
Do the interface statistics go backwards here?  ibmveth_get_stats64(),
ibmveth_get_ethtool_stats(), ibmveth_sum_rx_invalid_buffers() and
ibmveth_sum_rx_large_packets() all sum rx_qstats[0..num_rx_queues-1]:

	if (adapter->rx_qstats) {
		for (i = 0; i < adapter->num_rx_queues; i++) {
			stats->rx_packets += adapter->rx_qstats[i].packets;

so lowering the count drops the packets/bytes already accumulated by the
removed queues from the reported totals.

In the other direction, ibmveth_alloc_single_rx_queue() resets
rx_queue[queue_idx].index/num_slots/toggle and copies the pool geometry
but never clears adapter->rx_qstats[queue_idx], so re-adding a previously
removed index resurrects its old counters and the totals jump up.

Should the retired queues' counts be folded into an accumulator and the
slot cleared on (re-)allocation, so rtnl_link_stats64 stays monotonic?

> +
> +		for (i = new_count; i < old_count; i++) {
> +			ibmveth_cleanup_single_rx_interrupt(adapter, i);
> +			ibmveth_deregister_single_rx_queue(adapter, i);
> +			ibmveth_free_single_rx_queue(adapter, i);
> +		}
> +	}
> +
> +	netdev_info(netdev, "Successfully resized to %d RX queues (incremental)\n",
> +		    adapter->num_rx_queues);
> +
> +	if (firmware_has_feature(FW_FEATURE_CMO))
> +		vio_cmo_set_dev_desired(adapter->vdev,
> +					ibmveth_get_desired_dma(adapter->vdev));

[Severity: Medium]
Is the entitlement raised too late on CMO LPARs?  All the CMO-charged DMA
work for the new queues has already happened by the time this runs:
dma_map_single() of the buffer-list page and dma_alloc_coherent() of the
descriptor ring in ibmveth_alloc_single_rx_queue(), plus every RX skb
mapping done by ibmveth_replenish_task().  Those run while the device's
desired entitlement still reflects the old, smaller queue count, and
vio_dma_iommu_alloc_coherent() refuses the allocation when
vio_cmo_alloc() cannot cover it.

It is also skipped on every failure and rollback return, so a resize that
fails leaves the entitlement untouched.

The driver's own convention is the opposite order, in
ibmveth_change_mtu():

	vio_cmo_set_dev_desired(viodev, ibmveth_get_desired_dma(viodev));
	...
	ibmveth_open(...)

Note that simply moving the call is not enough, since
ibmveth_get_desired_dma() itself loops over adapter->num_rx_queues, so
the target count has to be accounted before the allocations and rolled
back on failure.

> +
> +	return 0;
> +
> +cleanup_new_queues:
> +	failed_queue = i;
> +	netdev_err(netdev,
> +		   "Scale-up failed at queue %d, cleaning up queues %d-%d\n",
> +		   failed_queue, old_count, failed_queue - 1);

[Severity: Low]
When the first newly added queue is the one that fails, failed_queue ==
old_count, so this prints an inverted empty range such as "cleaning up
queues 4-3" while the three loops below correctly iterate zero times.

> +	for (i = old_count; i < failed_queue; i++) {
> +		ibmveth_disable_irq(adapter, i);
> +		synchronize_irq(adapter->queue_irq[i]);
> +	}
> +
> +	for (i = old_count; i < failed_queue; i++)
> +		napi_disable(&adapter->napi[i]);
> +
> +	for (i = old_count; i < failed_queue; i++)
> +		ibmveth_drain_rx_queue(adapter, i);
> +
> +	synchronize_net();
> +
> +	for (i = old_count; i < failed_queue; i++) {
> +		ibmveth_cleanup_single_rx_interrupt(adapter, i);
> +		ibmveth_deregister_single_rx_queue(adapter, i);
> +		ibmveth_free_single_rx_queue(adapter, i);
> +	}
> +	adapter->num_rx_queues = old_count;

[Severity: High]
Can a concurrent NAPI poll dereference the pages this loop just freed?
adapter->num_rx_queues is restored only after the free loop, so for the
duration of that loop the indices [old_count, failed_queue) are freed but
still inside the published live range.

ibmveth_free_single_rx_queue() unmaps buffer_list_dma[i], frees the
coherent ring and then does:

	free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
	adapter->buffer_list_addr[queue_idx] = NULL;

Meanwhile every surviving queue's poll reaches
ibmveth_replenish_task() -> ibmveth_update_rx_no_buffer(), which walks the
published range with no NULL or liveness check:

	for (i = 0; i < adapter->num_rx_queues; i++) {
		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
		u64 drops = be64_to_cpup(p);

So for a 2->4 scale-up where queue 2 came up and queue 3 fails, a poll on
queue 0 or 1 reads buffer_list_addr[2] either as a just-freed page or as
NULL + 0xff8, in softirq context.  ibmveth_poll_controller() reaches the
same range from netpoll without rtnl, and additionally posts buffers into
pools that are being freed.

The guard in ibmveth_replenish_task():

	if (queue_index >= adapter->num_rx_queues) {

does not help while the count is still the higher value.  The inline
enable_irq failure handler above and the successful scale-down path both
lower adapter->num_rx_queues before freeing; should this label do the
same, i.e. set adapter->num_rx_queues = old_count before the free loop?

> +	netdev_warn(netdev, "Keeping %d queues after scale-up failure\n",
> +		    old_count);
> +	return rc;
> +}
> +

[ ... ]

> @@ -2217,12 +2670,62 @@ static void ibmveth_get_channels(struct net_device *netdev,
>  	channels->rx_count = adapter->num_rx_queues;
>  }
>  

[ ... ]

> +static int ibmveth_resize_rx_channels(struct ibmveth_adapter *adapter,
> +				      unsigned int goal_rx)
> +{

[ ... ]

> +	if (goal_rx == old_rx)
> +		return 0;
> +
> +	if (!(netdev->flags & IFF_UP)) {
> +		adapter->num_rx_queues = goal_rx;
> +		return 0;
> +	}
> +

[Severity: Low]
This isn't a bug in the final series, but at this commit the stash branch
is unreachable: the only caller, ibmveth_set_channels(), already returns
early on the same !IFF_UP condition before reaching
ibmveth_resize_rx_channels(), so ethtool -L <dev> rx N on a down
interface returns 0 while the requested count is discarded.  The next
patch in the series ("ibmveth: Wire ethtool set_channels to MQ RX queue
resize") replaces that early return and makes this branch live, so this
only shows up at this bisect point.

> +	rxq_entries = adapter->rx_queue[0].num_slots;
> +	rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
> +						  rxq_entries);
> +	if (rc)
> +		netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
> +	return rc;
> +}
> +
>  static int ibmveth_set_channels(struct net_device *netdev,
>  				struct ethtool_channels *channels)
>  {
>  	struct ibmveth_adapter *adapter = netdev_priv(netdev);
>  	unsigned int old = netdev->real_num_tx_queues,
>  		     goal = channels->tx_count;
> +	unsigned int goal_rx = channels->rx_count;
>  	int rc, i;
>  
>  	/* If ndo_open has not been called yet then don't allocate, just set
> @@ -2231,6 +2734,13 @@ static int ibmveth_set_channels(struct net_device *netdev,
>  	if (!(netdev->flags & IFF_UP))
>  		return netif_set_real_num_tx_queues(netdev, goal);
>  
> +	/* Resize RX first while UP so ibmveth_resize_rx_channels() is used
> +	 * in this patch. !IFF_UP RX stash ordering lands next.
> +	 */
> +	rc = ibmveth_resize_rx_channels(adapter, goal_rx);
> +	if (rc)
> +		return rc;
> +

[Severity: Medium]
Should the RX side be unwound when the TX phase below fails?  The RX
resize is committed here first, and then ibmveth_allocate_tx_ltb() can
return -ENOMEM or netif_set_real_num_tx_queues() can fail, in which case
the function ends with return rc and no compensating
ibmveth_resize_rx_channels(adapter, old_rx).

ETHTOOL_SCHANNELS then reports failure while half of the requested
configuration is in effect: on scale-down the excess RX queues have
already been deregistered from PHYP and freed, on scale-up new queues are
registered with IRQs installed and DMA memory pinned, and ethtool -l
afterwards reports the new rx_count.

>  	/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
>  	 * but we may need to alloc/free the ltb's.
>  	 */

[ ... ]

Re: [PATCH net-next v4 13/14] ibmveth: Wire ethtool set_channels to MQ RX queue resize

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Wire ethtool set_channels to MQ RX queue resize

This completes the RX side of ibmveth_set_channels(): when the interface is
down the TX queue count is published and the desired RX count is stashed for
the next open(), and when it is up the RX resize runs through
ibmveth_resize_rx_channels() before the existing TX LTB alloc/free path. It
also renames the TX bookkeeping variables to old_tx/goal_tx and adds an early
return when the TX count is unchanged.

A few questions below.

>       - Non-MQ firmware returns -EOPNOTSUPP for rx > 1.
>       - Validate rx_count within 1..IBMVETH_MAX_RX_QUEUES.

[Severity: Low]
Do these two bullets describe changes that are actually in this patch?

Both checks live in ibmveth_resize_rx_channels() in
drivers/net/ethernet/ibm/ibmveth.c:

	if (goal_rx > 1 && !adapter->multi_queue) {
		...
		return -EOPNOTSUPP;
	}

	if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
		...
		return -EINVAL;
	}

and they were added by the preceding patch "ibmveth: Implement incremental MQ
RX queue resize". This patch neither adds nor changes them.

Related to the split: the parent commit carried a comment in
ibmveth_set_channels() that referred to the patch series itself, which this
patch now deletes:

	/* Resize RX first while UP so ibmveth_resize_rx_channels() is used
	 * in this patch. !IFF_UP RX stash ordering lands next.
	 */

Could that comment be dropped from the parent instead, so no committed state
of the tree describes an unfinished series?

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index b57c7df92853f..c99d8e8be7b37 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -2723,20 +2723,28 @@ static int ibmveth_set_channels(struct net_device *netdev,
>  				struct ethtool_channels *channels)
>  {
>  	struct ibmveth_adapter *adapter = netdev_priv(netdev);
> -	unsigned int old = netdev->real_num_tx_queues,
> -		     goal = channels->tx_count;
>  	unsigned int goal_rx = channels->rx_count;
> +	unsigned int old_tx = netdev->real_num_tx_queues;
> +	unsigned int goal_tx = channels->tx_count;
>  	int rc, i;
>  
>  	/* If ndo_open has not been called yet then don't allocate, just set
>  	 * desired netdev_queue's and return
>  	 */
> -	if (!(netdev->flags & IFF_UP))
> -		return netif_set_real_num_tx_queues(netdev, goal);
> +	if (!(netdev->flags & IFF_UP)) {

[Severity: High]
Is netdev->flags & IFF_UP a reliable proxy here for "the driver holds RX
resources"?

ibmveth tears down and re-creates its resources internally while IFF_UP stays
set. veth_pool_store() does:

	ibmveth_close(netdev);
	...
	if (netif_running(netdev)) {
		rc = ibmveth_open(netdev);
		if (rc) {
			pool->active = oldactive;
			...
			goto unlock_err;

and ibmveth_change_mtu() does the same close/open dance. If the reopen fails,
for example a GFP_KERNEL failure in ibmveth_alloc_rx_queues() or
ibmveth_alloc_buffer_pools(), the error is returned without clearing
dev->flags, so IFF_UP remains set while ibmveth_close() ->
ibmveth_cleanup_rx_interrupts() has already run:

	for (i = 0; i < adapter->num_rx_queues; i++)
		napi_disable(&adapter->napi[i]);

and ibmveth_cleanup_rx_resources() has set buffer_list_addr[i] and
rx_queue[i].queue_addr to NULL, with adapter->num_rx_queues left unchanged.

A later "ethtool -L ethX rx <smaller>" then takes the UP branch and the
scale-down loop in ibmveth_resize_rx_queues_incremental() calls
napi_disable(&adapter->napi[i]) a second time on an already-disabled NAPI.
Does napi_disable_locked() ever return in that case?

net/core/dev.c:napi_disable_locked() {
	...
	while (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) {
		usleep_range(20, 200);
		val = READ_ONCE(n->state);
	}
	...
}

That would be an unkillable spin with rtnl_lock held.

The scale-up direction on the same stale state reaches
ibmveth_replenish_task() -> ibmveth_update_rx_no_buffer(), which does:

	for (i = 0; i < adapter->num_rx_queues; i++) {
		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
		u64 drops = be64_to_cpup(p);

with buffer_list_addr[i] already NULL.

This dispatch on IFF_UP came from the preceding patch, but this patch
finalizes it as the user-visible interface. Would keying the decision on an
explicit "RX resources allocated" state in the adapter avoid this?

> +		if (goal_tx != old_tx) {
> +			rc = netif_set_real_num_tx_queues(netdev, goal_tx);
> +			if (rc)
> +				return rc;
> +		}
> +
> +		/* Stash desired RX count only after TX succeeds (or was
> +		 * already correct); open() publishes it via
> +		 * netif_set_real_num_rx_queues().
> +		 */
> +		return ibmveth_resize_rx_channels(adapter, goal_rx);
> +	}

[Severity: Medium]
Does this path need to refresh the VIO CMO DMA entitlement?

The stash branch of ibmveth_resize_rx_channels() just does:

	if (!(netdev->flags & IFF_UP)) {
		adapter->num_rx_queues = goal_rx;
		return 0;
	}

and ibmveth_get_desired_dma() sizes the request per RX queue:

	for (q = 0; q < adapter->num_rx_queues; q++) {

The only two vio_cmo_set_dev_desired() call sites in the driver are the live
resize path in ibmveth_resize_rx_queues_incremental():

	if (firmware_has_feature(FW_FEATURE_CMO))
		vio_cmo_set_dev_desired(adapter->vdev,
					ibmveth_get_desired_dma(adapter->vdev));

and ibmveth_change_mtu(), which refreshes it even while the interface is
closed. ibmveth_open() does not call it.

So on a CMO LPAR, after

	ip link set dev ethX down
	ethtool -L ethX rx 16
	ip link set dev ethX up

the entitlement is still sized for the old, smaller RX count while open()
DMA-maps the larger footprint (per-queue buffer-list page, RX ring, buffer
pools). Can vio_dma_iommu_map_page() -> vio_cmo_alloc() then fail and make
the link fail to come up with -ENOMEM? Shrinking while down has the mirror
effect of leaving entitlement reserved until the next MTU change or live
resize.

[Severity: Low]
Should this branch also publish the RX count to the netdev?

TX is published immediately with netif_set_real_num_tx_queues(), but RX only
reaches the stash in ibmveth_resize_rx_channels(), so
netdev->real_num_rx_queues keeps its old value until the next
ibmveth_open(). Meanwhile ibmveth_get_channels() reports the private counter:

	channels->rx_count = adapter->num_rx_queues;

After "ip link set dev ethX down; ethtool -L ethX rx N", ethtool -l reports
N while /sys/class/net/ethX/queues/rx-*, the RPS/RFS maps and netdev-genl
queue enumeration still show the old count.

netif_set_real_num_rx_queues() only touches kobjects and RPS maps and
allocates no driver resources, and probe already sized the netdev with
alloc_etherdev_mqs(..., IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES), so is
there a reason RX cannot be published here the way TX is?

>  
> -	/* Resize RX first while UP so ibmveth_resize_rx_channels() is used
> -	 * in this patch. !IFF_UP RX stash ordering lands next.
> -	 */
>  	rc = ibmveth_resize_rx_channels(adapter, goal_rx);
>  	if (rc)
>  		return rc;

[Severity: High]
Can this live resize race with a surviving queue's NAPI poll?

The scale-down branch of ibmveth_resize_rx_queues_incremental() takes its
grace period before lowering the count, and the frees come after the store:

	synchronize_net();

	rc = netif_set_real_num_rx_queues(netdev, new_count);
	...
	adapter->num_rx_queues = new_count;

	for (i = new_count; i < old_count; i++) {
		ibmveth_cleanup_single_rx_interrupt(adapter, i);
		ibmveth_deregister_single_rx_queue(adapter, i);
		ibmveth_free_single_rx_queue(adapter, i);
	}

ibmveth_free_single_rx_queue() then does:

	if (adapter->buffer_list_addr[queue_idx]) {
		free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
		adapter->buffer_list_addr[queue_idx] = NULL;
	}

A surviving queue's poll -> ibmveth_replenish_task() ->
ibmveth_update_rx_no_buffer() uses the count as a lockless bound and
dereferences the pointer without a NULL check:

	for (i = 0; i < adapter->num_rx_queues; i++) {
		__be64 *p = adapter->buffer_list_addr[i] + 4096 - 8;
		u64 drops = be64_to_cpup(p);

The reader holds only rx_queue[k].replenish_lock, the writer holds rtnl, so
the locksets do not intersect and disabling the removed queues' NAPIs does
not quiesce the surviving pollers. Can this read a freed page, or
dereference NULL + 4088 if the pointer load lands after the NULL store?

The scale-up side publishes the count with a plain store before the reader
can see the pointer stores:

	adapter->num_rx_queues = i + 1;
	ibmveth_replenish_task(adapter, i);

On PowerPC, can a reader observe the larger count with a stale NULL
buffer_list_addr[i]?

The resize helper and its UP-path caller came from the preceding patch
"ibmveth: Implement incremental MQ RX queue resize", so this is not
introduced here, but this patch completes the user-facing entry point that
drives it. The later patch "ibmveth: Fix MQ RX poll and shutdown hangs after
queue resize" only touches correlator validation and poll shutdown and leaves
ibmveth_update_rx_no_buffer() and this ordering unchanged.

[Severity: Medium]
Can the adapter-wide rx_no_buffer counter lose updates once more than one RX
queue is configured through this path?

ibmveth_update_rx_no_buffer() resets and re-accumulates a shared counter and
writes every queue's qstats entry:

	adapter->rx_no_buffer = 0;
	for (i = 0; i < adapter->num_rx_queues; i++) {
		...
		adapter->rx_no_buffer += drops;
	}

but it is called from ibmveth_replenish_task() while only that caller's
per-queue lock is held:

	spin_lock_irqsave(&rxq->replenish_lock, flags);
	...
	ibmveth_update_rx_no_buffer(adapter);

	spin_unlock_irqrestore(&rxq->replenish_lock, flags);

With polls for queue 0 and queue 1 running on different CPUs, the locksets
are disjoint, so the "= 0" and the "+=" from both CPUs interleave and
ethtool -S / ndo_get_stats64() can report a value near zero or well below
the real one. This came in with the earlier per-queue aggregation and MQ RX
patches rather than this one, and the later "ibmveth: Fix MQ RX poll and
shutdown hangs after queue resize" patch does not touch this function.

[Severity: Low]
Is a partially applied configuration possible here?

In the UP path the RX resize is committed first, including hypervisor
registration, IRQ setup/teardown, buffer alloc/free and the update of
adapter->num_rx_queues. If the later netif_set_real_num_tx_queues() fails,
for example -ENOMEM out of netdev_queue_update_kobjects(), set_channels()
returns that error with the RX count already changed, so the resulting
configuration matches neither the old nor the requested one and nothing
rolls RX back.

The !IFF_UP branch above has the mirror ordering: TX is committed with
netif_set_real_num_tx_queues() before any RX validation runs, since the
-EOPNOTSUPP and -EINVAL checks live inside ibmveth_resize_rx_channels(),
which is called last. A rejected RX value would leave the TX count changed
with no rollback. Today the ethtool core clamps rx_count against max_rx
before the driver op runs, so this half is latent, but should the RX
validation happen before TX is touched?

> @@ -2744,10 +2752,17 @@ static int ibmveth_set_channels(struct net_device *netdev,
>  	/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
>  	 * but we may need to alloc/free the ltb's.
>  	 */
> +	if (goal_tx == old_tx)
> +		return 0;
> +
>  	netif_tx_stop_all_queues(netdev);
>  
> -	/* Allocate any queue that we need */
> -	for (i = old; i < goal; i++) {
> +	/* Allocate any queue that we need. Initialize i to old_tx so a
> +	 * scale-down path that never enters the loop still has defined
> +	 * bounds if set_real_num_tx_queues() fails.
> +	 */
> +	i = old_tx;
> +	for (; i < goal_tx; i++) {

[Severity: Low]
Is there a case where i was undefined before this change?

The previous form was:

	for (i = old; i < goal; i++) {

The init clause runs unconditionally before the first condition test, so i
was already assigned even when the body never executes. Splitting it into
"i = old_tx; for (; i < goal_tx; i++)" looks semantically identical, and the
new comment suggests the idiomatic form was unsafe. Could the comment be
dropped or reworded?

The changelog bullet "Always return rc from set_channels()" also looks like a
non-change, since the UP path already ended with "return rc;" and the down
path already returned the netif_set_real_num_tx_queues() value.

In the one case where an error is actually lost, the claim does not hold. On
ibmveth_allocate_tx_ltb() failure the rewind runs:

		goal_tx = old_tx;
		old_tx = i;
		break;
	}
	rc = netif_set_real_num_tx_queues(netdev, goal_tx);

so the -ENOMEM is overwritten by a call that cannot fail because goal_tx now
equals dev->real_num_tx_queues, and "ethtool -L ethX tx N" reports success
after an allocation failure. That clobber is identical at the baseline and is
not introduced here, it just means the hardening bullet does not describe it.

Separately, the genuinely new TX behaviour in this hunk, "if (goal_tx ==
old_tx) return 0;", which now skips netif_tx_stop_all_queues() and
netif_tx_wake_all_queues() when only RX is resized, is not mentioned in the
commit message. Could it be added there?

[ ... ]

Re: [PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize

[Jakub Kicinski]

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Fix MQ RX poll and shutdown hangs after queue resize

This changes ibmveth_poll() to skip RX slots whose correlator cannot be
mapped to a buffer pool instead of retrying them forever, completes NAPI
when the interface is going down, and adds a descriptor length check
before skb_put().  The WARN_ON plus adapter reset escalation in the
correlator validation paths is replaced by a rate-limited message.

No Link: tags present in the commit header.

A few questions below.

[Severity: Low]
Would it make sense to split this into separate patches with individual
Fixes: tags?  There appear to be four independent changes here:

  - skipping bad RX correlators and dropping the WARN_ON/reset escalation
  - completing NAPI when the interface is down or napi_disable is pending
  - validating the descriptor length against skb tailroom before skb_put()
  - changing the return value on the budget-exhausted shutdown path

There is no Fixes: tag, so stable maintainers cannot pick up the hang fix
without also picking up the policy change that removes the reset based
recovery, and the skb_put() hardening is not visible as a fix on its own.

> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index c99d8e8be7b37..09e06d86701a5 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -1376,6 +1376,24 @@ ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
>  	netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
>  }
>  
> +static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
> +					 int queue_index, u64 correlator)
> +{
> +	unsigned int pool = correlator >> 32;
> +	unsigned int index = correlator & 0xffffffffUL;
> +
> +	return pool < IBMVETH_NUM_BUFF_POOLS &&
> +	       index < adapter->rx_buff_pool[queue_index][pool].size;
> +}

[Severity: High]
This isn't a bug introduced by this patch, since the pre-existing WARN_ON
checks were also bounds-only, but can a correlator naming an inactive or
already freed pool still pass ibmveth_rxq_correlator_valid() and then
dereference a NULL array?

ibmveth_init_buffer_pool() sets pool->size for all pools regardless of
active state, while ibmveth_alloc_queue_buffer_pools() skips inactive
pools, and ibmveth_free_buffer_pool() clears the arrays without clearing
size or active:

	kfree(pool->free_map);
	pool->free_map = NULL;
	...
	if (pool->skbuff) {
		kfree(pool->skbuff);
		pool->skbuff = NULL;
	}

With the defaults in ibmveth.h:

	static int pool_count[] = { 256, 512, 256, 256, 256 };
	static int pool_active[] = { 1, 1, 0, 0, 0};

a correlator of pool=2, index=0 satisfies both conditions of the new
helper, and both users then index a NULL pointer before the !skb test can
help:

	return adapter->rx_buff_pool[queue_index][pool].skbuff[index];

and

	skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];

Should the helper also require pool->active and non-NULL skbuff/dma_addr/
free_map, or should ibmveth_free_buffer_pool() clear pool->size?  The same
window seems to apply to pools torn down by ethtool -L scale-down, an MTU
change, or sysfs deactivation.

[Severity: Low]
This is a pre-existing weakness rather than something new here, but the
helper only bounds-checks, so a stale correlator that happens to land in
range after an ethtool -L resize is treated as a live frame.  With
reuse=false the buffer can be handed to the stack while the hypervisor may
still own it, and with reuse=true the index is pushed onto free_map a
second time.  Is there any generation counter or per-queue tag available in
the correlator that could distinguish a stale descriptor from a current
one?

> +
> +static void ibmveth_rxq_advance(struct ibmveth_rx_q *rxq)
> +{
> +	if (++rxq->index == rxq->num_slots) {
> +		rxq->index = 0;
> +		rxq->toggle = !rxq->toggle;
> +	}
> +}
> +
>  /**
>   * ibmveth_remove_buffer_from_pool - remove a buffer from a pool
>   * @adapter: adapter instance
> @@ -1397,17 +1415,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
>  	unsigned int free_index;
>  	struct sk_buff *skb;
>  
> -	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
> -	    WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
> -		schedule_work(&adapter->work);
> +	if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
>  		return -EINVAL;
> -	}
>  
>  	skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
> -	if (WARN_ON(!skb)) {
> -		schedule_work(&adapter->work);
> +	if (!skb)
>  		return -EFAULT;
> -	}

[Severity: Medium]
The schedule_work(&adapter->work) escalation is dropped here and in
ibmveth_rxq_get_buffer(), so nothing bounds the condition any more.  The
commit message describes the new skip policy but does not mention that all
recovery escalation is gone.

The previous reset performed a close/open cycle, which issued
h_free_logical_lan and re-registered the logical LAN, flushing every buffer
registration the hypervisor still held.  If the bad correlator exists
because PHYP still holds buffers from a pool that
ibmveth_free_buffer_pool() already unmapped and freed during an
ethtool -L resize:

	dma_unmap_single(&adapter->vdev->dev, pool->dma_addr[i],
			 pool->buff_size, DMA_FROM_DEVICE);
	dev_kfree_skb_any(skb);

can the hypervisor keep writing into those freed pages indefinitely now
that the driver only logs and advances?

Separately, this also folds together two different classes: -EINVAL for an
out-of-range correlator, and -EFAULT where pool and index are in range but
skbuff[index] is NULL, which indicates driver/hypervisor state desync.  Is
silently skipping the -EFAULT case intended?

And if the descriptor's correlator belongs to a different queue's pool, the
skip never reclaims that queue's slot, so that pool's available count stays
inflated and ibmveth_replenish_task() stops replenishing it:

	if (pool->active && pool->free_map &&
	    (atomic_read(&pool->available) < pool->threshold))

>  
>  	/* if we are going to reuse the buffer then keep the pointers around
>  	 * but mark index as available. replenish will see the skb pointer and
> @@ -1452,11 +1465,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
>  	unsigned int pool = correlator >> 32;
>  	unsigned int index = correlator & 0xffffffffUL;
>  
> -	if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
> -	    WARN_ON(index >= adapter->rx_buff_pool[queue_index][pool].size)) {
> -		schedule_work(&adapter->work);
> +	if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
>  		return NULL;
> -	}
>  
>  	return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
>  }
> @@ -1483,14 +1493,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
>  
>  	cor = rxq->queue_addr[rxq->index].correlator;
>  	rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
> -	if (unlikely(rc))
> +	if (unlikely(rc)) {
> +		if (rc == -EINVAL || rc == -EFAULT)
> +			goto advance;
>  		return rc;
> -
> -	if (++rxq->index == rxq->num_slots) {
> -		rxq->index = 0;
> -		rxq->toggle = !rxq->toggle;
>  	}
>  
> +advance:
> +	ibmveth_rxq_advance(rxq);
> +
>  	return 0;
>  }

[Severity: Low]
Can ibmveth_rxq_harvest_buffer() still return non-zero after this change?
ibmveth_remove_buffer_from_pool() has exactly three returns: -EINVAL,
-EFAULT and 0, and both error codes now go to the advance label, so
"return rc;" looks unreachable.

That would make all five "if (unlikely(rc)) break;" checks in
ibmveth_poll(), including the two added by this patch, dead code, along
with the harvest failure path in ibmveth_drain_rx_queue():

	rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
	if (rc) {
		netdev_err(netdev,
			   "Failed to harvest buffer from queue %d during drain: %d\n",
			   queue_index, rc);
		break;
	}
	drained++;

Should unmappable slots encountered during drain still be counted as
drained++ with no diagnostic?  The kernel-doc above harvest still documents
"other - non-zero return from ibmveth_remove_buffer_from_pool", which no
longer matches the implementation.

>  
> @@ -3092,11 +3103,19 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>  	if (WARN_ON(queue_index < 0 || queue_index >= adapter->num_rx_queues))
>  		return 0;
>  
> +	if (!netif_running(netdev) || napi_disable_pending(napi)) {
> +		napi_complete_done(napi, 0);
> +		return 0;
> +	}
> +
>  	if (adapter->rx_qstats)
>  		adapter->rx_qstats[queue_index].polls++;
>  
>  restart_poll:
>  	while (frames_processed < budget) {
> +		if (!netif_running(netdev) || napi_disable_pending(napi))
> +			break;
> +
>  		if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
>  			break;
>  

[ ... ]

> @@ -3126,8 +3145,45 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>  			__sum16 iph_check = 0;
>  
>  			skb = ibmveth_rxq_get_buffer(adapter, queue_index);
> -			if (unlikely(!skb))
> -				break;
> +			if (unlikely(!skb)) {
> +				if (net_ratelimit())
> +					netdev_err(netdev,
> +						   "bad correlator on queue %d, skipping slot\n",
> +						   queue_index);

[ ... ]

> +
> +			if (unlikely((unsigned int)offset +
> +				     (unsigned int)length >
> +				     skb_tailroom(skb))) {

[Severity: High]
This is not a regression from this patch, since there was no length check
at all before, but can this check be bypassed by 32-bit wraparound?

Both operands are cast to unsigned int, so the sum is modulo 2^32.  Unlike
offset, length is not masked; ibmveth_rxq_frame_length() returns the whole
descriptor field:

	return be32_to_cpu(rxq->queue_addr[rxq->index].length);

and in ibmveth.h that field is a full __be32 written by the hypervisor,
while only offset gets IBMVETH_RXQ_OFF_MASK (0x0000FFFF).

For a corrupted length like 0xFFFFFFF0 with a small offset, the addition
wraps to a small value and the guard does not fire.  The copybreak test
compares against "static unsigned int rx_copybreak", so 0xFFFFFFF0 is not
less than rx_copybreak and no copy skb is allocated, and execution reaches:

	skb_put(skb, length);

which trips skb_over_panic() -> BUG().  Would an overflow-safe form work
better here, for example:

	if (offset > tailroom || length > tailroom - offset)

or validating length against pool->buff_size?

[ ... ]

> @@ -3206,8 +3262,14 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>  
>  	ibmveth_replenish_task(adapter, queue_index);
>  
> -	if (frames_processed == budget)
> +	if (frames_processed == budget) {
> +		if (!netif_running(netdev) || napi_disable_pending(napi)) {
> +			napi_complete_done(napi, frames_processed);
> +			/* After complete_done, must not return full budget. */
> +			return frames_processed ? frames_processed - 1 : 0;
> +		}
>  		goto out;
> +	}
>  
>  	if (!napi_complete_done(napi, frames_processed))
>  		goto out;
> @@ -3225,6 +3287,8 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
>  	}
>  
>  	if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
> +	    netif_running(netdev) &&
> +	    !napi_disable_pending(napi) &&
>  	    napi_schedule(napi)) {
>  		lpar_rc = ibmveth_disable_irq(adapter, queue_index);
>  		WARN_ON(lpar_rc != H_SUCCESS);

[Severity: High]
This isn't a bug introduced by this patch, as the
napi_complete_done() -> ibmveth_enable_irq() -> ibmveth_rxq_pending_buffer()
sequence is unchanged, but do the new guards actually close the shutdown
window the commit message describes?

The new mid-loop break also lands here with frames_processed < budget, so
the quiesce case still reaches:

	if (!napi_complete_done(napi, frames_processed))
		goto out;

Clearing NAPI_STATE_SCHED there is what napi_disable() waits for, so
ibmveth_close() can proceed past ibmveth_cleanup_rx_interrupts() into
ibmveth_cleanup_rx_resources():

	if (adapter->rx_queue[i].queue_addr) {
		dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
				  adapter->rx_queue[i].queue_addr,
				  adapter->rx_queue[i].queue_dma);
		adapter->rx_queue[i].queue_addr = NULL;

while the still running poll executes:

	lpar_rc = ibmveth_enable_irq(adapter, queue_index);

re-arming the PHYP interrupt that close just masked, and then evaluates
ibmveth_rxq_pending_buffer(), which reads
rxq->queue_addr[rxq->index].flags_off - a NULL dereference or a read of
freed coherent memory.  ibmveth_close() has no synchronize_net() between
napi_disable()/free_irq() and the frees, unlike the scale-down path.

Note the new netif_running()/napi_disable_pending() terms are evaluated
after ibmveth_rxq_pending_buffer() in the same condition, so they cannot
prevent that access; they only suppress the following
ibmveth_disable_irq().

Related question on that suppression: in the window where
__LINK_STATE_START is already cleared by __dev_close_many() but
NAPI_STATE_DISABLE is not yet set, the pre-patch code re-masked PHYP via
the napi_schedule() branch.  With the new guards, poll now returns leaving
delivery unmasked into the napi_disable()/free_irq() window, and the
interrupt handler does not mask either when napi_schedule_prep() fails:

	if (napi_schedule_prep(napi)) {
		lpar_rc = ibmveth_disable_irq(adapter, qindex);
		WARN_ON(lpar_rc != H_SUCCESS);
		__napi_schedule(napi);
	}

Can that leave the queue interrupt storming until free_irq()?

[ ... ]

Re: [PATCH net-next v4 14/14] ibmveth: Fix MQ RX poll and shutdown hangs after queue resize

[Jakub Kicinski]

On Thu, 30 Jul 2026 17:47:17 -0700 Mingming Cao wrote:
> After aggressive ethtool -L cycling, PHYP can leave a VALID RX descriptor
> with a correlator that no longer matches the per-queue buffer pools. Poll
> treated this as fatal: ibmveth_rxq_get_buffer() WARNed and returned NULL
> without advancing the ring, then restart_poll retried the same slot
> forever.
> 
> Advance past bad correlators instead of spinning: validate correlators
> without WARN_ON, skip invalid slots in poll (count as invalid_buffers),
> and advance the RX ring when remove_buffer_from_pool cannot map the
> correlator. Rate-limit the bad correlator message.
> 
> Complete NAPI when the interface is down or napi_disable is pending so
> close/quiesce can finish. Do not restart_poll in that window. Close
> keeps hypervisor IRQ disable before napi_disable (via
> cleanup_rx_interrupts() / related cleanup helpers).
> 
> Also validate descriptor length against skb tailroom before skb_put(),
> and after napi_complete_done() on the budget-exhausted shutdown path
> return a value less than budget so NAPI does not immediately reschedule.

This patch looks very odd, like there's a bug somewhere and instead of
fixing it we're making the rx path more resilient. BTW minor coding
note - you're adding a bunch of heavily indented code to an already
long function. Please consider factoring things out. Similar comment
can be applied to other patches.