* [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support
@ 2026-08-14 7:36 Mingming Cao
2026-08-14 7:36 ` [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
` (14 more replies)
0 siblings, 15 replies; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Hi,
Power11 PHYP adds Virtual Ethernet multi-queue (MQ) RX: multiple
logical-LAN RX queues, per-queue buffer posting, and completion
delivery. Guest Linux did not use that; ibmveth still registered one
RX queue even when PHYP was MQ-capable.
This series adds the ibmveth MQ client for net-next. When PHYP
advertises IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT via H_ILLAN_ATTRIBUTES,
probe enables MQ with a default RX count of min(num_online_cpus(), 8)
(same cap as TX today); ethtool -L can raise RX up to 16. Packets are
received on per-queue NAPI. Older firmware without the bit is unchanged.
Queue selection remains firmware-defined (PHYP hash). Ethtool RSS hash
get/set for that algorithm is deferred to a follow-up series so this
one stays MQ datapath only.
User-visible bits: ethtool -l/-L (channels), ethtool -S and
ndo_get_stats64 (per-queue + aggregate), and a read-only debugfs
buffer_pools dump (v3's multi-line sysfs dump moved to debugfs; the
historical queue-0 poolN/ sysfs ABI is unchanged).
Background:
ibmveth today uses one logical LAN, one set of buffer pools, and one
NAPI context. PHYP MQ mode gives each RX queue its own handle (post via
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, subordinate register via
H_REG_LOGICAL_LAN_QUEUE); traffic can land on any active queue. The
driver needs per-queue pools, IRQs, and NAPI to match. Legacy firmware
keeps the original hcall path.
Series layout (15 patches):
1-2 Hypercall wrappers; MQ adapter layout (MAX_RX_QUEUES stays 1)
3-9 Queue-aware helpers (still SQ runtime): RX, per-queue pools,
IRQ, TX, PHYP, buffer submit (open/close 3-8); poll harden (9)
10 Enable MQ datapath at probe/open (subordinate register helpers
land here with first use)
11-13 Per-queue RX/TX stats; get_channels MQ counts; debugfs buffer_pools
14-15 Incremental RX resize (+ resize_rx_channels); wire ethtool
set_channels
- Helper patches (3-8) reshape ibmveth_open()/close() into
queue-aware helpers. Patch 9 hardens the SQ poll path with the same
queue-index helpers; it does not change open/close. Runtime
behaviour is unchanged through 3-9: num_rx_queues stays 1 and
multi_queue is false until patch 10.
- Patch 10 is the switch: probe sets multi_queue from firmware, raises
num_rx_queues, registers subordinates, and replenishes every active
queue.
- Statistics types land with first use (hcall_stats in 7, rx/tx qstats
in 11).
Testing:
ppc64le PowerVM LPAR, MQ-capable firmware:
* ethtool -L cycling (16/1/8/11/1/3/16/8/1) with ping - no hangs
* ethtool -L under iperf3; link down/up during traffic
* ifdown/ifup under iperf3 RX+TX (MQ and ethtool -L rx 1)
* Legacy firmware (no MQ bit): open/close/stress on helper path
* Bisect-friendly restack; W=1 clean at tip
Changes in v5:
* Restack mailed v4 (14 patches) to v5 (15):
v4 1-8 helpers -> v5 1-8
(new) SQ poll harden -> v5 9 (before MQ enable)
v4 9 MQ enable -> v5 10
v4 10 stats -> v5 11
(new) get_channels -> v5 12 (peeled from stats)
v4 11 debugfs -> v5 13
v4 12 resize -> v5 14
v4 13 set_channels -> v5 15
v4 14 trailing poll/shutdown -> folded into v5 5/9/10/14
(mailed "P14" was that trailer, not v5 14)
* Teardown-first resize after aggressive ethtool -L; thin defensive
poll skip remains; no correlator generation field this series
* opened / rx_irq_setup; set_channels keys on opened (not IFF_UP)
* filter_list_dma=0 on map error; restore default-active 64 KiB pool;
unwind pools by allocation presence; probe_cleanup clears vio
drvdata; remove: unregister then cancel_work
* TX quiesce before freeing bounce buffers; guard start_xmit if LTB gone
* MQ H_FUNCTION recovery (reset + SQ fallback); no printk under
replenish_lock; lock harvest with replenish; resume kicks all queues
* Per-queue update_rx_no_buffer; publish-before-free on resize;
CMO refresh; IRQ helpers return errno
* Harvest abort (no fake GRO / UAF); poll refuses PHYP re-arm on close;
wrap-safe skb_put; atomic set_channels; monotonic stats across shrink
* Keep mask -> sync -> napi_disable on teardown; open stays
request_irq -> napi_enable while PHYP masked; scale-up/recovery keep
napi_enable before enable_irq
* Pool geometry kept on free; restart_rx_queue after open/scale-down;
remask after napi_disable; schedule_rx_queue masks only when
napi_schedule_prep succeeds (STOP + poll no-rearm for storms)
Changes in v4:
Addresses Simon's v3 review and related fixes:
* First-use helpers/includes (irqdomain.h with first dispose); no
unused statics; dropped orphan open/close pipeline patch
* Open/close unwind (free LAN before RX pools); no double TX teardown
* MQ open: replenish all queues before PHYP unmask; H_FUNCTION on
subordinate register is a hard open failure
* Resize/set_channels hardenings; stats probe-lifetime + sum-on-read;
buffer_pools diagnostic on debugfs
* Patch 9: put already-created pool kobjects on probe failure paths
* Patch 14: correlator skip, skb tailroom check, napi_complete_done
shutdown return < budget
* Bisect-friendly restack (helpers with first use)
Changes in v3:
* Dropped RFC; addressed style / DMA feedback from earlier revisions
* Early MQ enablement iterations (see lore links below)
Comments welcome.
---
Previous versions
v4: https://lore.kernel.org/r/cover.1785457143.git.mmc@linux.ibm.com
v3: https://lore.kernel.org/r/20260706193603.8039-1-mmc@linux.ibm.com
v2: https://lore.kernel.org/r/20260701222327.61325-1-mmc@linux.ibm.com
v1: https://lore.kernel.org/r/cover.1782758799.git.mmc@linux.ibm.com
v4 review (Jakub Kicinski):
https://lore.kernel.org/r/20260806183614.3171785-1-kuba@kernel.org
v3 review (Simon Horman):
https://lore.kernel.org/r/20260714124327.GJ1364329@horms.kernel.org
Mingming Cao (15):
ibmveth: Add MQ RX hypercall wrappers and call definitions
ibmveth: Prepare MQ RX adapter data structures
ibmveth: Refactor RX resource allocation for MQ RX bring-up
ibmveth: Refactor buffer pool management for per-queue MQ RX
ibmveth: Refactor RX interrupt control for MQ RX queues
ibmveth: Refactor TX resource allocation in open/close paths
ibmveth: Add RX queue register helpers for MQ
ibmveth: Add queue-aware RX buffer submit helper for MQ
ibmveth: Harden RX poll path with helpers
ibmveth: Enable multi-queue RX receive path
ibmveth: Add per-queue RX and TX statistics collection
ibmveth: Report MQ-aware RX counts in ethtool get_channels
ibmveth: Expose per-queue buffer pool details via debugfs
ibmveth: Implement incremental MQ RX queue resize
ibmveth: Wire ethtool set_channels to MQ RX queue resize
arch/powerpc/include/asm/hvcall.h | 6 +-
drivers/net/ethernet/ibm/ibmveth.c | 3950 ++++++++++++++++++++++------
drivers/net/ethernet/ibm/ibmveth.h | 218 +-
3 files changed, 3432 insertions(+), 742 deletions(-)
--
2.50.1 (Apple Git-155)
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 02/15] ibmveth: Prepare MQ RX adapter data structures Mingming Cao
` (13 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Single-queue ibmveth only needs h_register_logical_lan() plus legacy
buffer add/free calls. MQ RX uses per-queue handles, so the driver must
also be able to register/deregister subordinate queues and post
buffers against a specific queue handle.
Add the PHYP call IDs for:
H_REG_LOGICAL_LAN_QUEUE (0x49C)
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE (0x4A0)
H_FREE_LOGICAL_LAN_QUEUE (0x4A8)
as defined in PAPR 11.20.00. The gaps at 0x498 (reserved) and 0x4A4
(reserved for H_FREE_LOGICAL_LAN_BUFFER_QUEUE) are intentional; this
series tears queues down with H_FREE_LOGICAL_LAN_QUEUE and does not add
the buffer-queue free hcall.
Raising MAX_HCALL_OPCODE for these IDs also widens KVM's
kvm_arch.enabled_hcalls bitmap and the range accepted by
KVM_CAP_PPC_ENABLE_HCALL.
Add ibmveth.h wrapper helpers (h_register_logical_lan_queue(),
h_add_logical_lan_buffers_queue(), h_free_logical_lan_queue()) with
argument ordering and return semantics matching the existing ibmveth
hcall wrappers. h_free_logical_lan_queue() uses plpar_hcall_norets()
like h_free_logical_lan(). Also add h_register_logical_lan_with_handle()
so queue 0 can capture the PHYP queue handle in MQ mode; both handle
out-params use unsigned long *.
This patch is intentionally plumbing only: no runtime behavior change
yet. Legacy firmware keeps H_REGISTER_LOGICAL_LAN and the existing
buffer hcalls. The new wrappers are used only when a later commit sets
multi_queue from H_ILLAN_ATTRIBUTES.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Cite PAPR 11.20.00 for MQ hcall IDs 0x49C / 0x4A0 / 0x4A8 (v4 named
the opcodes only)
- Call out reserved gaps at 0x498 and 0x4A4 (0x4A4 =
H_FREE_LOGICAL_LAN_BUFFER_QUEUE); decision: do not add that hcall -
queue teardown uses H_FREE_LOGICAL_LAN_QUEUE
- Note MAX_HCALL_OPCODE raise also widens KVM enabled_hcalls /
KVM_CAP_PPC_ENABLE_HCALL accepted range (cross-subsystem)
- Rename h_reg_logical_lan_queue() -> h_register_logical_lan_queue() to
match h_register_logical_lan() / h_free_logical_lan()
- Unify queue_handle out-params as unsigned long * on both
h_register_logical_lan_queue() and h_register_logical_lan_with_handle()
(v4 used u64 * on with_handle)
- h_free_logical_lan_queue() uses plpar_hcall_norets() like
h_free_logical_lan() (v4 used plpar_hcall9 with unused retbuf)
Changes in v4:
- Document @queue_handle and @irq in h_reg_logical_lan_queue() kdoc.
- Wrap h_register_logical_lan_with_handle() prototype for readability /
checkpatch.
- Drop unused H_FREE_LOGICAL_LAN_BUFFER_QUEUE wrapper/opcode (no caller;
buffer return is local harvest + queue free).
arch/powerpc/include/asm/hvcall.h | 6 +-
drivers/net/ethernet/ibm/ibmveth.h | 136 +++++++++++++++++++++++++++++
2 files changed, 141 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
index dff90a7d7f70..cb0ea53491e6 100644
--- a/arch/powerpc/include/asm/hvcall.h
+++ b/arch/powerpc/include/asm/hvcall.h
@@ -362,7 +362,11 @@
#define H_GUEST_DELETE 0x488
#define H_PKS_WRAP_OBJECT 0x490
#define H_PKS_UNWRAP_OBJECT 0x494
-#define MAX_HCALL_OPCODE H_PKS_UNWRAP_OBJECT
+/* 0x498 reserved; 0x4A4 = H_FREE_LOGICAL_LAN_BUFFER_QUEUE (unused here) */
+#define H_REG_LOGICAL_LAN_QUEUE 0x49C
+#define H_ADD_LOGICAL_LAN_BUFFERS_QUEUE 0x4A0
+#define H_FREE_LOGICAL_LAN_QUEUE 0x4A8
+#define MAX_HCALL_OPCODE H_FREE_LOGICAL_LAN_QUEUE
/* Scope args for H_SCM_UNBIND_ALL */
#define H_UNBIND_SCOPE_ALL (0x1)
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index d87713668ed3..c0ef03cad9b9 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -66,6 +66,142 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address,
desc5, desc6, desc7, desc8);
}
+/**
+ * h_register_logical_lan_queue - Register a subordinate receive queue
+ * @unit_address: Device unit address
+ * @buffer_list: DMA address of 4KB page for tracking registered buffers
+ * @rec_queue: Buffer descriptor of receive queue
+ * @queue_handle: Output queue handle on success (may be NULL)
+ * @irq: Output hypervisor IRQ number on success (may be NULL)
+ *
+ * Registers a subordinate receive queue with the hypervisor.
+ *
+ * Return:
+ * H_SUCCESS (0) on success
+ * H_PARAMETER if parameters are invalid
+ *
+ * On success, hypervisor returns:
+ * R3: H_SUCCESS
+ * R4: Queue handle
+ * R5: IRQ number for this queue
+ */
+static inline long
+h_register_logical_lan_queue(unsigned long unit_address,
+ unsigned long buffer_list,
+ unsigned long rec_queue,
+ unsigned long *queue_handle,
+ unsigned long *irq)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+ long rc;
+
+ rc = plpar_hcall9(H_REG_LOGICAL_LAN_QUEUE,
+ retbuf, unit_address,
+ buffer_list, rec_queue);
+
+ if (rc == H_SUCCESS) {
+ if (queue_handle)
+ *queue_handle = retbuf[0];
+ if (irq)
+ *irq = retbuf[1];
+ }
+
+ return rc;
+}
+
+/**
+ * h_add_logical_lan_buffers_queue - Add buffers to subordinate queue
+ * @unit_address: Device unit address
+ * @queue_handle: Queue handle from h_register_logical_lan_queue()
+ * @buffersznum: Buffer size (upper 32 bits) | count (lower 32 bits)
+ * @ioba12: Buffer addresses 1 and 2 packed ((addr1 << 32) | addr2)
+ * @ioba34: Buffer addresses 3 and 4 packed
+ * @ioba56: Buffer addresses 5 and 6 packed
+ * @ioba78: Buffer addresses 7 and 8 packed
+ * @ioba910: Buffer addresses 9 and 10 packed
+ * @ioba1112: Buffer addresses 11 and 12 packed
+ *
+ * Return:
+ * H_SUCCESS - All buffers added successfully
+ * H_PARAMETER - Invalid parameters
+ * H_HARDWARE - Hardware error
+ */
+static inline long h_add_logical_lan_buffers_queue(unsigned long unit_address,
+ unsigned long queue_handle,
+ unsigned long buffersznum,
+ unsigned long ioba12,
+ unsigned long ioba34,
+ unsigned long ioba56,
+ unsigned long ioba78,
+ unsigned long ioba910,
+ unsigned long ioba1112)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+
+ return plpar_hcall9(H_ADD_LOGICAL_LAN_BUFFERS_QUEUE,
+ retbuf, unit_address,
+ queue_handle, buffersznum,
+ ioba12, ioba34, ioba56,
+ ioba78, ioba910, ioba1112);
+}
+
+/**
+ * h_free_logical_lan_queue - Deregister subordinate receive queue
+ * @unit_address: Device unit address
+ * @queue_handle: Queue handle from h_register_logical_lan_queue()
+ *
+ * Deregisters and frees all structures associated with the subordinate queue.
+ *
+ * Return:
+ * H_SUCCESS - Queue freed successfully
+ * H_PARAMETER - Invalid parameters
+ * H_HARDWARE - Hardware error
+ * H_STATE - VIOA not in valid state
+ * H_BUSY / H_LONG_BUSY_* - Resource busy, retry
+ */
+static inline long h_free_logical_lan_queue(unsigned long unit_address,
+ unsigned long queue_handle)
+{
+ return plpar_hcall_norets(H_FREE_LOGICAL_LAN_QUEUE,
+ unit_address, queue_handle);
+}
+
+/**
+ * h_register_logical_lan_with_handle - Register primary queue and get handle
+ * @unit_address: Device unit address
+ * @buffer_list: DMA address of buffer list
+ * @rec_queue: Buffer descriptor of receive queue
+ * @filter_list: DMA address of filter list
+ * @mac_address: MAC address
+ * @queue_handle: Output parameter for queue handle (may be NULL)
+ *
+ * Registers the primary receive queue (queue 0) with the hypervisor and
+ * returns the queue handle. This is needed in multi-queue mode to use
+ * h_add_logical_lan_buffers_queue() for all queues including queue 0.
+ *
+ * Return: H_SUCCESS (0) on success, error code otherwise
+ */
+static inline long
+h_register_logical_lan_with_handle(unsigned long unit_address,
+ unsigned long buffer_list,
+ unsigned long rec_queue,
+ unsigned long filter_list,
+ unsigned long mac_address,
+ unsigned long *queue_handle)
+{
+ unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
+ long rc;
+
+ rc = plpar_hcall9(H_REGISTER_LOGICAL_LAN, retbuf,
+ unit_address, buffer_list, rec_queue,
+ filter_list, mac_address);
+
+ if (rc == H_SUCCESS && queue_handle)
+ *queue_handle = retbuf[0];
+
+ return rc;
+}
+
/* FW allows us to send 6 descriptors but we only use one so mark
* the other 5 as unused (0)
*/
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 02/15] ibmveth: Prepare MQ RX adapter data structures
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-08-14 7:36 ` [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
` (12 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
MQ RX needs per-queue state for NAPI, queue handles/IRQs, RX rings,
buffer-list DMA mappings, and buffer pools. The current driver stores
most of this as single instances tied to queue 0.
Convert those fields to queue-indexed layouts sized by
IBMVETH_MAX_RX_QUEUES:
rx_queue[]
napi[]
queue_handle[] / queue_irq[]
buffer_list_addr[] / buffer_list_dma[]
rx_buff_pool[queue][pool]
and add multi_queue / num_rx_queues to track MQ capability and how
many RX queues are active. Keep IBMVETH_MAX_RX_QUEUES at 1 for now so
this remains a structural preparation patch; later enablement raises
the limit when multi-queue RX is actually turned on.
This patch keeps behavior unchanged by mechanically switching existing
references to index 0 (for example rx_queue -> rx_queue[0],
rx_buff_pool[pool] -> rx_buff_pool[0][pool], napi -> napi[0]).
open/poll/close still drive a single RX queue only.
Queue-0 pool sysfs (poolN/) remains the shared geometry template for
all RX queues: later patches clone that metadata per queue. Per-queue
runtime visibility is added via debugfs later, not per-queue sysfs.
Per-queue statistics structs and hcall_stats are introduced later with
their first use (register helpers / statistics collection).
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Drop unrelated style nits that v4 mixed into the index-0 conversion
(extra atomic_dec parentheses, ibmveth_rxq_get_buffer() prototype
reflow, get_desired_dma "Return:" kdoc, blank line after
rx_large_packets) - this tip patch keeps only the mechanical [0]
retargeting
- Document queue-0 poolN/ sysfs as the shared geometry/template for all
RX queues (buff_size/size/active); queues 1..N get no separate pool
sysfs in this series
- Decision: keep full ibmveth_buff_pool per queue for this series
(unused embedded kobjects on rows 1..N); defer config-vs-runtime
layout split to a cover follow-up
Changes in v4:
- Keep IBMVETH_MAX_RX_QUEUES at 1 until MQ enablement (same idea as v3,
but v3 also planted unused stats types here).
- Layout-only: queue-indexed adapter fields only. Do not introduce
hcall_stats / rx_qstats here (first-use).
- Defer hcall_stats to patch 7 (register helpers) and rx/tx qstats +
NUM macros to patch 10 (statistics collection).
- Subject: "Prepare MQ RX adapter data structures" (was "...and
statistics structures" in earlier drafts).
drivers/net/ethernet/ibm/ibmveth.c | 206 ++++++++++++++++-------------
drivers/net/ethernet/ibm/ibmveth.h | 17 ++-
2 files changed, 124 insertions(+), 99 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 73e051d26b9d..7cb828b476c1 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -101,7 +101,9 @@ static struct ibmveth_stat ibmveth_stats[] = {
/* simple methods of getting data from the current rxq entry */
static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
{
- return be32_to_cpu(adapter->rx_queue.queue_addr[adapter->rx_queue.index].flags_off);
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+
+ return be32_to_cpu(rxq->queue_addr[rxq->index].flags_off);
}
static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
@@ -112,7 +114,7 @@ static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
{
- return ibmveth_rxq_toggle(adapter) == adapter->rx_queue.toggle;
+ return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
}
static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
@@ -132,7 +134,9 @@ static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
{
- return be32_to_cpu(adapter->rx_queue.queue_addr[adapter->rx_queue.index].length);
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+
+ return be32_to_cpu(rxq->queue_addr[rxq->index].length);
}
static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
@@ -386,7 +390,7 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
*/
static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
{
- __be64 *p = adapter->buffer_list_addr + 4096 - 8;
+ __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
adapter->rx_no_buffer = be64_to_cpup(p);
}
@@ -399,7 +403,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
adapter->replenish_task_cycles++;
for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
- struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[i];
+ struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
if (pool->active &&
(atomic_read(&pool->available) < pool->threshold))
@@ -463,12 +467,12 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
struct sk_buff *skb;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
schedule_work(&adapter->work);
return -EINVAL;
}
- skb = adapter->rx_buff_pool[pool].skbuff[index];
+ skb = adapter->rx_buff_pool[0][pool].skbuff[index];
if (WARN_ON(!skb)) {
schedule_work(&adapter->work);
return -EFAULT;
@@ -482,24 +486,24 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* remove the skb pointer to mark free. actual freeing is done
* by upper level networking after gro_receive
*/
- adapter->rx_buff_pool[pool].skbuff[index] = NULL;
+ adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
dma_unmap_single(&adapter->vdev->dev,
- adapter->rx_buff_pool[pool].dma_addr[index],
- adapter->rx_buff_pool[pool].buff_size,
+ adapter->rx_buff_pool[0][pool].dma_addr[index],
+ adapter->rx_buff_pool[0][pool].buff_size,
DMA_FROM_DEVICE);
}
- free_index = adapter->rx_buff_pool[pool].producer_index;
- adapter->rx_buff_pool[pool].producer_index++;
- if (adapter->rx_buff_pool[pool].producer_index >=
- adapter->rx_buff_pool[pool].size)
- adapter->rx_buff_pool[pool].producer_index = 0;
- adapter->rx_buff_pool[pool].free_map[free_index] = index;
+ free_index = adapter->rx_buff_pool[0][pool].producer_index;
+ adapter->rx_buff_pool[0][pool].producer_index++;
+ if (adapter->rx_buff_pool[0][pool].producer_index >=
+ adapter->rx_buff_pool[0][pool].size)
+ adapter->rx_buff_pool[0][pool].producer_index = 0;
+ adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
mb();
- atomic_dec(&(adapter->rx_buff_pool[pool].available));
+ atomic_dec(&adapter->rx_buff_pool[0][pool].available);
return 0;
}
@@ -507,17 +511,18 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* get the current buffer on the rx queue */
static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
{
- u64 correlator = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator;
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+ u64 correlator = rxq->queue_addr[rxq->index].correlator;
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[pool].size)) {
+ WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
schedule_work(&adapter->work);
return NULL;
}
- return adapter->rx_buff_pool[pool].skbuff[index];
+ return adapter->rx_buff_pool[0][pool].skbuff[index];
}
/**
@@ -538,14 +543,16 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
u64 cor;
int rc;
- cor = adapter->rx_queue.queue_addr[adapter->rx_queue.index].correlator;
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+
+ cor = rxq->queue_addr[rxq->index].correlator;
rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
if (unlikely(rc))
return rc;
- if (++adapter->rx_queue.index == adapter->rx_queue.num_slots) {
- adapter->rx_queue.index = 0;
- adapter->rx_queue.toggle = !adapter->rx_queue.toggle;
+ if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
+ adapter->rx_queue[0].index = 0;
+ adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
}
return 0;
@@ -595,7 +602,7 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
*/
retry:
rc = h_register_logical_lan(adapter->vdev->unit_address,
- adapter->buffer_list_dma, rxq_desc.desc,
+ adapter->buffer_list_dma[0], rxq_desc.desc,
adapter->filter_list_dma, mac_address);
if (rc != H_SUCCESS && try_again) {
@@ -623,14 +630,14 @@ static int ibmveth_open(struct net_device *netdev)
netdev_dbg(netdev, "open starting\n");
- napi_enable(&adapter->napi);
+ napi_enable(&adapter->napi[0]);
for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- rxq_entries += adapter->rx_buff_pool[i].size;
+ rxq_entries += adapter->rx_buff_pool[0][i].size;
rc = -ENOMEM;
- adapter->buffer_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
- if (!adapter->buffer_list_addr) {
+ adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[0]) {
netdev_err(netdev, "unable to allocate list pages\n");
goto out;
}
@@ -643,17 +650,18 @@ static int ibmveth_open(struct net_device *netdev)
dev = &adapter->vdev->dev;
- adapter->rx_queue.queue_len = sizeof(struct ibmveth_rx_q_entry) *
+ adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
rxq_entries;
- adapter->rx_queue.queue_addr =
- dma_alloc_coherent(dev, adapter->rx_queue.queue_len,
- &adapter->rx_queue.queue_dma, GFP_KERNEL);
- if (!adapter->rx_queue.queue_addr)
+ adapter->rx_queue[0].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
+ &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
+ if (!adapter->rx_queue[0].queue_addr)
goto out_free_filter_list;
- adapter->buffer_list_dma = dma_map_single(dev,
- adapter->buffer_list_addr, 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->buffer_list_dma)) {
+ adapter->buffer_list_dma[0] =
+ dma_map_single(dev, adapter->buffer_list_addr[0],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
netdev_err(netdev, "unable to map buffer list pages\n");
goto out_free_queue_mem;
}
@@ -670,19 +678,21 @@ static int ibmveth_open(struct net_device *netdev)
goto out_free_tx_ltb;
}
- adapter->rx_queue.index = 0;
- adapter->rx_queue.num_slots = rxq_entries;
- adapter->rx_queue.toggle = 1;
+ adapter->rx_queue[0].index = 0;
+ adapter->rx_queue[0].num_slots = rxq_entries;
+ adapter->rx_queue[0].toggle = 1;
mac_address = ether_addr_to_u64(netdev->dev_addr);
rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
- adapter->rx_queue.queue_len;
- rxq_desc.fields.address = adapter->rx_queue.queue_dma;
+ adapter->rx_queue[0].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
- netdev_dbg(netdev, "buffer list @ 0x%p\n", adapter->buffer_list_addr);
+ netdev_dbg(netdev, "buffer list @ 0x%p\n",
+ adapter->buffer_list_addr[0]);
netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
- netdev_dbg(netdev, "receive q @ 0x%p\n", adapter->rx_queue.queue_addr);
+ netdev_dbg(netdev, "receive q @ 0x%p\n",
+ adapter->rx_queue[0].queue_addr);
h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
@@ -693,7 +703,7 @@ static int ibmveth_open(struct net_device *netdev)
lpar_rc);
netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
"desc:0x%llx MAC:0x%llx\n",
- adapter->buffer_list_dma,
+ adapter->buffer_list_dma[0],
adapter->filter_list_dma,
rxq_desc.desc,
mac_address);
@@ -702,11 +712,11 @@ static int ibmveth_open(struct net_device *netdev)
}
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (!adapter->rx_buff_pool[i].active)
+ if (!adapter->rx_buff_pool[0][i].active)
continue;
- if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[i])) {
+ if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
netdev_err(netdev, "unable to alloc pool\n");
- adapter->rx_buff_pool[i].active = 0;
+ adapter->rx_buff_pool[0][i].active = 0;
rc = -ENOMEM;
goto out_free_buffer_pools;
}
@@ -738,9 +748,9 @@ static int ibmveth_open(struct net_device *netdev)
out_free_buffer_pools:
while (--i >= 0) {
- if (adapter->rx_buff_pool[i].active)
+ if (adapter->rx_buff_pool[0][i].active)
ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[i]);
+ &adapter->rx_buff_pool[0][i]);
}
out_unmap_filter_list:
dma_unmap_single(dev, adapter->filter_list_dma, 4096,
@@ -752,18 +762,18 @@ static int ibmveth_open(struct net_device *netdev)
}
out_unmap_buffer_list:
- dma_unmap_single(dev, adapter->buffer_list_dma, 4096,
+ dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
DMA_BIDIRECTIONAL);
out_free_queue_mem:
- dma_free_coherent(dev, adapter->rx_queue.queue_len,
- adapter->rx_queue.queue_addr,
- adapter->rx_queue.queue_dma);
+ dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
+ adapter->rx_queue[0].queue_addr,
+ adapter->rx_queue[0].queue_dma);
out_free_filter_list:
free_page((unsigned long)adapter->filter_list_addr);
out_free_buffer_list:
- free_page((unsigned long)adapter->buffer_list_addr);
+ free_page((unsigned long)adapter->buffer_list_addr[0]);
out:
- napi_disable(&adapter->napi);
+ napi_disable(&adapter->napi[0]);
return rc;
}
@@ -776,7 +786,7 @@ static int ibmveth_close(struct net_device *netdev)
netdev_dbg(netdev, "close starting\n");
- napi_disable(&adapter->napi);
+ napi_disable(&adapter->napi[0]);
netif_tx_stop_all_queues(netdev);
@@ -795,22 +805,22 @@ static int ibmveth_close(struct net_device *netdev)
ibmveth_update_rx_no_buffer(adapter);
- dma_unmap_single(dev, adapter->buffer_list_dma, 4096,
+ dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->buffer_list_addr);
+ free_page((unsigned long)adapter->buffer_list_addr[0]);
dma_unmap_single(dev, adapter->filter_list_dma, 4096,
DMA_BIDIRECTIONAL);
free_page((unsigned long)adapter->filter_list_addr);
- dma_free_coherent(dev, adapter->rx_queue.queue_len,
- adapter->rx_queue.queue_addr,
- adapter->rx_queue.queue_dma);
+ dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
+ adapter->rx_queue[0].queue_addr,
+ adapter->rx_queue[0].queue_dma);
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (adapter->rx_buff_pool[i].active)
+ if (adapter->rx_buff_pool[0][i].active)
ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[i]);
+ &adapter->rx_buff_pool[0][i]);
for (i = 0; i < netdev->real_num_tx_queues; i++)
ibmveth_free_tx_ltb(adapter, i);
@@ -1448,7 +1458,7 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
static int ibmveth_poll(struct napi_struct *napi, int budget)
{
struct ibmveth_adapter *adapter =
- container_of(napi, struct ibmveth_adapter, napi);
+ container_of(napi, struct ibmveth_adapter, napi[0]);
struct net_device *netdev = adapter->netdev;
int frames_processed = 0;
unsigned long lpar_rc;
@@ -1573,11 +1583,11 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
struct ibmveth_adapter *adapter = netdev_priv(netdev);
unsigned long lpar_rc;
- if (napi_schedule_prep(&adapter->napi)) {
+ if (napi_schedule_prep(&adapter->napi[0])) {
lpar_rc = h_vio_signal(adapter->vdev->unit_address,
VIO_IRQ_DISABLE);
WARN_ON(lpar_rc != H_SUCCESS);
- __napi_schedule(&adapter->napi);
+ __napi_schedule(&adapter->napi[0]);
}
return IRQ_HANDLED;
}
@@ -1645,7 +1655,7 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
int need_restart = 0;
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size)
+ if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size)
break;
if (i == IBMVETH_NUM_BUFF_POOLS)
@@ -1660,9 +1670,9 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
/* Look for an active buffer pool that can hold the new MTU */
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- adapter->rx_buff_pool[i].active = 1;
+ adapter->rx_buff_pool[0][i].active = 1;
- if (new_mtu_oh <= adapter->rx_buff_pool[i].buff_size) {
+ if (new_mtu_oh <= adapter->rx_buff_pool[0][i].buff_size) {
WRITE_ONCE(dev->mtu, new_mtu);
vio_cmo_set_dev_desired(viodev,
ibmveth_get_desired_dma
@@ -1720,12 +1730,12 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
/* add the size of the active receive buffers */
- if (adapter->rx_buff_pool[i].active)
+ if (adapter->rx_buff_pool[0][i].active)
ret +=
- adapter->rx_buff_pool[i].size *
- IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[i].
+ adapter->rx_buff_pool[0][i].size *
+ IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
buff_size, tbl);
- rxqentries += adapter->rx_buff_pool[i].size;
+ rxqentries += adapter->rx_buff_pool[0][i].size;
}
/* add the size of the receive queue entries */
ret += IOMMU_PAGE_ALIGN(
@@ -1844,7 +1854,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
ibmveth_init_link_settings(netdev);
- netif_napi_add_weight(netdev, &adapter->napi, ibmveth_poll, 16);
+ netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
netdev->irq = dev->irq;
netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -1876,6 +1886,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev->features |= NETIF_F_FRAGLIST;
}
+ /* Initialize queue count - always 1 for now */
+ adapter->multi_queue = 0;
+ adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
+
if (ret == H_SUCCESS &&
(ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
@@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
memcpy(pool_count, pool_count_cmo, sizeof(pool_count));
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- struct kobject *kobj = &adapter->rx_buff_pool[i].kobj;
+ struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
int error;
- ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+ ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
error = kobject_init_and_add(kobj, &ktype_veth_pool,
@@ -1949,7 +1963,7 @@ static void ibmveth_remove(struct vio_dev *dev)
cancel_work_sync(&adapter->work);
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- kobject_put(&adapter->rx_buff_pool[i].kobj);
+ kobject_put(&adapter->rx_buff_pool[0][i].kobj);
unregister_netdev(netdev);
@@ -2035,11 +2049,12 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
/* Make sure there is a buffer pool with buffers that
can hold a packet of the size of the MTU */
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (pool == &adapter->rx_buff_pool[i])
+ if (pool == &adapter->rx_buff_pool[0][i])
continue;
- if (!adapter->rx_buff_pool[i].active)
+ if (!adapter->rx_buff_pool[0][i].active)
continue;
- if (mtu <= adapter->rx_buff_pool[i].buff_size)
+ if (mtu <=
+ adapter->rx_buff_pool[0][i].buff_size)
break;
}
@@ -2213,11 +2228,11 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
/* Set sane values for buffer pools */
for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+ ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
- pool = &adapter->rx_buff_pool[0];
+ pool = &adapter->rx_buff_pool[0][0];
pool->skbuff = kunit_kcalloc(test, pool->size, sizeof(void *), GFP_KERNEL);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
@@ -2225,7 +2240,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
- correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0].size;
+ correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
@@ -2258,30 +2273,33 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
INIT_WORK(&adapter->work, ibmveth_reset_kunit);
- adapter->rx_queue.queue_len = 1;
- adapter->rx_queue.index = 0;
- adapter->rx_queue.queue_addr = kunit_kzalloc(test, sizeof(struct ibmveth_rx_q_entry),
- GFP_KERNEL);
- KUNIT_ASSERT_NOT_ERR_OR_NULL(test, adapter->rx_queue.queue_addr);
+ adapter->rx_queue[0].queue_len = 1;
+ adapter->rx_queue[0].index = 0;
+ adapter->rx_queue[0].queue_addr =
+ kunit_kzalloc(test, sizeof(struct ibmveth_rx_q_entry),
+ GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, adapter->rx_queue[0].queue_addr);
/* Set sane values for buffer pools */
for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
+ ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
- pool = &adapter->rx_buff_pool[0];
+ pool = &adapter->rx_buff_pool[0][0];
pool->skbuff = kunit_kcalloc(test, pool->size, sizeof(void *), GFP_KERNEL);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
- adapter->rx_queue.queue_addr[0].correlator = (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
+ adapter->rx_queue[0].queue_addr[0].correlator =
+ (u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
- adapter->rx_queue.queue_addr[0].correlator = (u64)0 << 32 | adapter->rx_buff_pool[0].size;
+ adapter->rx_queue[0].queue_addr[0].correlator =
+ (u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
pool->skbuff[0] = skb;
- adapter->rx_queue.queue_addr[0].correlator = (u64)0 << 32 | 0;
+ adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
flush_work(&adapter->work);
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index c0ef03cad9b9..fc6a01e810c6 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -257,6 +257,8 @@ static inline long h_illan_attributes(unsigned long unit_address,
#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
#define IBMVETH_MAX_QUEUES 16U
#define IBMVETH_DEFAULT_QUEUES 8U
+#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_DEFAULT_RX_QUEUES 1U
#define IBMVETH_MAX_RX_PER_HCALL 8U
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
@@ -293,18 +295,23 @@ struct ibmveth_rx_q {
struct ibmveth_adapter {
struct vio_dev *vdev;
struct net_device *netdev;
- struct napi_struct napi;
+ struct napi_struct napi[IBMVETH_MAX_RX_QUEUES];
struct work_struct work;
unsigned int mcastFilterSize;
- void *buffer_list_addr;
+ void *buffer_list_addr[IBMVETH_MAX_RX_QUEUES];
void *filter_list_addr;
void *tx_ltb_ptr[IBMVETH_MAX_QUEUES];
unsigned int tx_ltb_size;
dma_addr_t tx_ltb_dma[IBMVETH_MAX_QUEUES];
- dma_addr_t buffer_list_dma;
+ dma_addr_t buffer_list_dma[IBMVETH_MAX_RX_QUEUES];
dma_addr_t filter_list_dma;
- struct ibmveth_buff_pool rx_buff_pool[IBMVETH_NUM_BUFF_POOLS];
- struct ibmveth_rx_q rx_queue;
+ struct ibmveth_buff_pool
+ rx_buff_pool[IBMVETH_MAX_RX_QUEUES][IBMVETH_NUM_BUFF_POOLS];
+ struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES];
+ u64 queue_handle[IBMVETH_MAX_RX_QUEUES];
+ unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES];
+ int multi_queue;
+ unsigned int num_rx_queues;
int rx_csum;
int large_send;
bool is_active_trunk;
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-08-14 7:36 ` [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
2026-08-14 7:36 ` [PATCH net-next v5 02/15] ibmveth: Prepare MQ RX adapter data structures Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
` (11 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
ibmveth_open() allocates the filter list and every RX queue inline.
That is already a long sequence and would get uglier once we loop over
num_rx_queues, especially on error unwind.
Pull the RX bits into helpers and wire them into open()/close() in the
same patch:
ibmveth_alloc_filter_list() / ibmveth_free_filter_list()
- shared multicast filter list (one per adapter, not per queue)
ibmveth_alloc_rx_queues() / ibmveth_cleanup_rx_resources()
- per-queue buffer lists and RX rings, looping [0, num_rx_queues)
alloc_rx_queues() rolls back on failure so open() does not need nested
goto chains for every queue index. open-failure and close release the
same resources through the same helpers.
Runtime behavior stays single-queue (num_rx_queues is still 1). Buffer
pools, IRQ, TX LTB, and PHYP registration remain inline for later
helper patches.
Also set rc = -ENOMEM before the TX LTB allocation loop so a failed
ibmveth_allocate_tx_ltb() still returns a useful errno after the RX
allocation blocks move into helpers.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- On filter_list DMA map failure: free_page and zero filter_list_dma so
a later free path cannot dma_unmap the DMA_MAPPING_ERROR sentinel
(match buffer_list_dma convention). Closes reopen->ifdown WARN path
after set_csum/set_tso/change_mtu/pool_store close+open while running
- v4 left the sentinel
- Call out rc = -ENOMEM before the TX LTB loop after RX helper extract
(already in v4; still required so allocate_tx_ltb failure returns a
useful errno)
Changes in v4:
- Introduce RX/filter allocation helpers in the same patch that wires
their first open/close callers; v3 left unused statics ahead of the
old open/close pipeline patch.
- Preserve correct -ENOMEM return on TX LTB allocation failure after
the RX helper extract.
- Drop reliance on v3's separate "open/close pipeline" patch for this
wiring.
drivers/net/ethernet/ibm/ibmveth.c | 270 ++++++++++++++++++++---------
1 file changed, 192 insertions(+), 78 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 7cb828b476c1..168940bfa109 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -151,6 +151,186 @@ static unsigned int ibmveth_real_max_tx_queues(void)
return min(n_cpu, IBMVETH_MAX_QUEUES);
}
+/**
+ * ibmveth_alloc_filter_list - Allocate and map filter list
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+
+ adapter->filter_list_addr = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->filter_list_addr) {
+ netdev_err(netdev, "unable to allocate filter pages\n");
+ return -ENOMEM;
+ }
+
+ adapter->filter_list_dma = dma_map_single(dev,
+ adapter->filter_list_addr,
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->filter_list_dma)) {
+ netdev_err(netdev, "unable to map filter list pages\n");
+ free_page((unsigned long)adapter->filter_list_addr);
+ adapter->filter_list_addr = NULL;
+ /* Do not leave DMA_MAPPING_ERROR for free_filter_list(). */
+ adapter->filter_list_dma = 0;
+ return -ENOMEM;
+ }
+
+ netdev_dbg(netdev, "filter list @ 0x%p (DMA: 0x%llx)\n",
+ adapter->filter_list_addr,
+ (unsigned long long)adapter->filter_list_dma);
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_filter_list - Free filter list resources
+ * @adapter: ibmveth adapter structure
+ */
+static void
+ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+
+ if (adapter->filter_list_dma) {
+ dma_unmap_single(dev, adapter->filter_list_dma, 4096,
+ DMA_BIDIRECTIONAL);
+ adapter->filter_list_dma = 0;
+ }
+
+ if (adapter->filter_list_addr) {
+ free_page((unsigned long)adapter->filter_list_addr);
+ adapter->filter_list_addr = NULL;
+ }
+}
+
+/**
+ * ibmveth_alloc_rx_queues - Allocate per-queue RX resources
+ * @adapter: ibmveth adapter structure
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ adapter->buffer_list_addr[i] =
+ (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[i]) {
+ netdev_err(netdev,
+ "unable to allocate buffer list for queue %d\n",
+ i);
+ goto err_cleanup;
+ }
+
+ adapter->rx_queue[i].queue_len =
+ sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+ adapter->rx_queue[i].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[i].queue_len,
+ &adapter->rx_queue[i].queue_dma,
+ GFP_KERNEL);
+ if (!adapter->rx_queue[i].queue_addr) {
+ netdev_err(netdev,
+ "unable to allocate RX queue for queue %d\n",
+ i);
+ goto err_cleanup;
+ }
+
+ adapter->buffer_list_dma[i] =
+ dma_map_single(dev, adapter->buffer_list_addr[i],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[i])) {
+ netdev_err(netdev,
+ "unable to map buffer list for queue %d\n",
+ i);
+ adapter->buffer_list_dma[i] = 0;
+ goto err_cleanup;
+ }
+
+ adapter->rx_queue[i].index = 0;
+ adapter->rx_queue[i].num_slots = rxq_entries;
+ adapter->rx_queue[i].toggle = 1;
+
+ netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
+ i, adapter->buffer_list_addr[i],
+ (unsigned long long)adapter->buffer_list_dma[i],
+ adapter->rx_queue[i].queue_addr,
+ (unsigned long long)adapter->rx_queue[i].queue_dma,
+ (unsigned long long)rxq_entries);
+ }
+
+ netdev_dbg(netdev, "allocated %d RX queue(s) with %d entries each\n",
+ adapter->num_rx_queues, rxq_entries);
+
+ return 0;
+
+err_cleanup:
+ /* Clean up previously allocated queues */
+ for (; i >= 0; i--) {
+ if (adapter->buffer_list_dma[i]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[i],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[i] = 0;
+ }
+ if (adapter->rx_queue[i].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+ adapter->rx_queue[i].queue_addr,
+ adapter->rx_queue[i].queue_dma);
+ adapter->rx_queue[i].queue_addr = NULL;
+ }
+ if (adapter->buffer_list_addr[i]) {
+ free_page((unsigned long)adapter->buffer_list_addr[i]);
+ adapter->buffer_list_addr[i] = NULL;
+ }
+ }
+
+ return -ENOMEM;
+}
+
+/**
+ * ibmveth_cleanup_rx_resources - Free all RX queue resources
+ * @adapter: ibmveth adapter structure
+ */
+static void
+ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
+{
+ struct device *dev = &adapter->vdev->dev;
+ int i;
+
+ netdev_dbg(adapter->netdev, "cleaning up %d RX queue(s)\n",
+ adapter->num_rx_queues);
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->buffer_list_dma[i]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[i],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[i] = 0;
+ }
+
+ if (adapter->rx_queue[i].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
+ adapter->rx_queue[i].queue_addr,
+ adapter->rx_queue[i].queue_dma);
+ adapter->rx_queue[i].queue_addr = NULL;
+ }
+
+ if (adapter->buffer_list_addr[i]) {
+ free_page((unsigned long)adapter->buffer_list_addr[i]);
+ adapter->buffer_list_addr[i] = NULL;
+ }
+ }
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
@@ -626,74 +806,34 @@ static int ibmveth_open(struct net_device *netdev)
int rc;
union ibmveth_buf_desc rxq_desc;
int i;
- struct device *dev;
netdev_dbg(netdev, "open starting\n");
napi_enable(&adapter->napi[0]);
- for(i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
rxq_entries += adapter->rx_buff_pool[0][i].size;
- rc = -ENOMEM;
- adapter->buffer_list_addr[0] = (void *)get_zeroed_page(GFP_KERNEL);
- if (!adapter->buffer_list_addr[0]) {
- netdev_err(netdev, "unable to allocate list pages\n");
+ rc = ibmveth_alloc_filter_list(adapter);
+ if (rc)
goto out;
- }
- adapter->filter_list_addr = (void*) get_zeroed_page(GFP_KERNEL);
- if (!adapter->filter_list_addr) {
- netdev_err(netdev, "unable to allocate filter pages\n");
- goto out_free_buffer_list;
- }
-
- dev = &adapter->vdev->dev;
-
- adapter->rx_queue[0].queue_len = sizeof(struct ibmveth_rx_q_entry) *
- rxq_entries;
- adapter->rx_queue[0].queue_addr =
- dma_alloc_coherent(dev, adapter->rx_queue[0].queue_len,
- &adapter->rx_queue[0].queue_dma, GFP_KERNEL);
- if (!adapter->rx_queue[0].queue_addr)
+ rc = ibmveth_alloc_rx_queues(adapter, rxq_entries);
+ if (rc)
goto out_free_filter_list;
- adapter->buffer_list_dma[0] =
- dma_map_single(dev, adapter->buffer_list_addr[0],
- 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->buffer_list_dma[0])) {
- netdev_err(netdev, "unable to map buffer list pages\n");
- goto out_free_queue_mem;
- }
-
- adapter->filter_list_dma = dma_map_single(dev,
- adapter->filter_list_addr, 4096, DMA_BIDIRECTIONAL);
- if (dma_mapping_error(dev, adapter->filter_list_dma)) {
- netdev_err(netdev, "unable to map filter list pages\n");
- goto out_unmap_buffer_list;
- }
-
+ rc = -ENOMEM;
for (i = 0; i < netdev->real_num_tx_queues; i++) {
if (ibmveth_allocate_tx_ltb(adapter, i))
goto out_free_tx_ltb;
}
- adapter->rx_queue[0].index = 0;
- adapter->rx_queue[0].num_slots = rxq_entries;
- adapter->rx_queue[0].toggle = 1;
-
mac_address = ether_addr_to_u64(netdev->dev_addr);
rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
adapter->rx_queue[0].queue_len;
rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
- netdev_dbg(netdev, "buffer list @ 0x%p\n",
- adapter->buffer_list_addr[0]);
- netdev_dbg(netdev, "filter list @ 0x%p\n", adapter->filter_list_addr);
- netdev_dbg(netdev, "receive q @ 0x%p\n",
- adapter->rx_queue[0].queue_addr);
-
h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
@@ -708,7 +848,7 @@ static int ibmveth_open(struct net_device *netdev)
rxq_desc.desc,
mac_address);
rc = -ENONET;
- goto out_unmap_filter_list;
+ goto out_free_tx_ltb;
}
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
@@ -735,8 +875,6 @@ static int ibmveth_open(struct net_device *netdev)
goto out_free_buffer_pools;
}
- rc = -ENOMEM;
-
netdev_dbg(netdev, "initial replenish cycle\n");
ibmveth_interrupt(netdev->irq, netdev);
@@ -752,26 +890,12 @@ static int ibmveth_open(struct net_device *netdev)
ibmveth_free_buffer_pool(adapter,
&adapter->rx_buff_pool[0][i]);
}
-out_unmap_filter_list:
- dma_unmap_single(dev, adapter->filter_list_dma, 4096,
- DMA_BIDIRECTIONAL);
-
out_free_tx_ltb:
- while (--i >= 0) {
+ while (--i >= 0)
ibmveth_free_tx_ltb(adapter, i);
- }
-
-out_unmap_buffer_list:
- dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
- DMA_BIDIRECTIONAL);
-out_free_queue_mem:
- dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
- adapter->rx_queue[0].queue_addr,
- adapter->rx_queue[0].queue_dma);
+ ibmveth_cleanup_rx_resources(adapter);
out_free_filter_list:
- free_page((unsigned long)adapter->filter_list_addr);
-out_free_buffer_list:
- free_page((unsigned long)adapter->buffer_list_addr[0]);
+ ibmveth_free_filter_list(adapter);
out:
napi_disable(&adapter->napi[0]);
return rc;
@@ -780,7 +904,6 @@ static int ibmveth_open(struct net_device *netdev)
static int ibmveth_close(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- struct device *dev = &adapter->vdev->dev;
long lpar_rc;
int i;
@@ -805,17 +928,8 @@ static int ibmveth_close(struct net_device *netdev)
ibmveth_update_rx_no_buffer(adapter);
- dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
- DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->buffer_list_addr[0]);
-
- dma_unmap_single(dev, adapter->filter_list_dma, 4096,
- DMA_BIDIRECTIONAL);
- free_page((unsigned long)adapter->filter_list_addr);
-
- dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
- adapter->rx_queue[0].queue_addr,
- adapter->rx_queue[0].queue_dma);
+ ibmveth_cleanup_rx_resources(adapter);
+ ibmveth_free_filter_list(adapter);
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
if (adapter->rx_buff_pool[0][i].active)
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (2 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
` (10 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=true, Size: 9264 bytes --]
Legacy ibmveth uses five adapter-level RX buffer pools (512 B through
64 KiB). pool_active[] enables the standard-MTU pools by default;
larger pools activate when MTU requires them. With single-queue RX
that set is shared on one completion path.
MQ requires the same pool model per queue: buffers post with
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against a queue handle and completions
return on that queue. Sharing pools across queues would mix ownership
and break queue-local replenish/drain/teardown.
Refactor around queue-local pools:
rx_buff_pool[queue][pool]
ibmveth_alloc_queue_buffer_pools()
ibmveth_free_queue_buffer_pools()
ibmveth_alloc_buffer_pools() / ibmveth_free_buffer_pools()
Queue 0 remains the template for pool geometry and activation policy
(size, buff_size, threshold, index, active). Queues 1..N copy that
metadata from queue 0, then allocate backing arrays/skbs per queue.
The existing poolN/ sysfs knobs stay adapter-wide on queue 0: writing
a pool size multiplies real memory by num_rx_queues (still 1 here).
Per-queue runtime state is exposed later via debugfs, not separate
per-queue pool sysfs nodes.
Wire the helpers into open()/close() in the same patch. Runtime
remains single-queue (num_rx_queues is still 1).
Error handling is queue-safe: allocation failure unwinds only what
that queue allocated (then prior queues in the caller), and free paths
release by real allocations (free_map/dma_addr/skbuff), not only
pool->active. Later resize/failure paths can leave memory allocated
after active was cleared.
close() also reorders pool teardown ahead of cleanup_rx_resources()
and filter-list free. That is safe here because h_free_logical_lan(),
napi_disable(), and free_irq() have already run, so neither PHYP nor
NAPI still reference the pool buffers or RX completion queue.
Keep the legacy 64 KiB pool enabled by default at standard MTU (same
as single-queue policy).
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- vs mailed v4: keep pool_active[] = {1,1,0,0,1} (v4 set the 64 KiB
pool inactive). No code delta vs prior patches here; SQ large-receive /
CMO stay at the historical baseline. change_mtu does not re-enable
pool4 at MTU 1500; MQ memory pressure belongs at scale-up, not SQ
default
- Unwind / free pools by real allocations (free_map/dma_addr/skbuff),
not only pool->active, so open-fail cannot leak partially allocated
pools (v4 fail path freed by active and skipped the failing pool)
- Clarify kdoc: free paths use allocation presence (not "all active");
queue 1..N metadata copy from queue 0 (v4 said "queues 1-15" while
MAX is still 1 here)
- Document poolN sysfs size multiplies by num_rx_queues (queue 0 is the
shared template; no per-queue pool sysfs in this series)
- Call out close() pool-free before cleanup_rx_resources, after LAN/
NAPI/IRQ teardown (order already in v4; spell it for later drain)
Changes in v4:
- Introduce the pool helpers in the same patch that wires their first
open/close callers, instead of leaving unused statics.
- Copy pool->index when cloning queue-0 geometry to later queues
(needed for correlators; also required by incremental resize).
drivers/net/ethernet/ibm/ibmveth.c | 161 +++++++++++++++++++++++++----
1 file changed, 141 insertions(+), 20 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 168940bfa109..1a428ee60d58 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -627,6 +627,142 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
}
}
+/**
+ * ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Frees buffer pools that still hold allocations for the specified
+ * queue (by free_map / dma_addr / skbuff presence), regardless of the
+ * active flag.
+ */
+static void ibmveth_free_queue_buffer_pools(struct ibmveth_adapter *adapter,
+ int queue)
+{
+ int i;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[queue][i];
+
+ /* Free pool if it has allocated memory, regardless of
+ * active flag. Allocation and active can diverge on failure
+ * paths, so check for actual allocations.
+ */
+ if (pool->free_map || pool->dma_addr || pool->skbuff)
+ ibmveth_free_buffer_pool(adapter, pool);
+ }
+}
+
+/**
+ * ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue
+ * @adapter: ibmveth adapter structure
+ * @queue: queue index
+ *
+ * Allocates backing storage for each active pool on @queue.
+ * Inactive pools (!active) are skipped. Pool metadata must be
+ * initialized before calling this function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int ibmveth_alloc_queue_buffer_pools(struct ibmveth_adapter *adapter,
+ int queue)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *bpool =
+ &adapter->rx_buff_pool[queue][i];
+
+ if (!bpool->active)
+ continue;
+
+ if (ibmveth_alloc_buffer_pool(bpool)) {
+ netdev_err(netdev,
+ "queue %d pool %d alloc failed (size=%u count=%u)\n",
+ queue, i,
+ bpool->buff_size,
+ bpool->size);
+ bpool->active = 0;
+ /* Free by allocation presence, not active — the
+ * failing pool cleared active first and would be
+ * skipped by an active-only unwind.
+ */
+ ibmveth_free_queue_buffer_pools(adapter, queue);
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ibmveth_alloc_buffer_pools - Allocate buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Initializes pool metadata for queues 1-N from queue 0 settings,
+ * then allocates buffer pools for all queues using the helper function.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i, q, rc;
+
+ /* Initialize pool metadata for queues 1..N from queue 0 settings */
+ for (q = 1; q < adapter->num_rx_queues; q++) {
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *src =
+ &adapter->rx_buff_pool[0][i];
+ struct ibmveth_buff_pool *dst =
+ &adapter->rx_buff_pool[q][i];
+
+ dst->size = src->size;
+ dst->index = src->index;
+ dst->buff_size = src->buff_size;
+ dst->threshold = src->threshold;
+ dst->active = src->active;
+ }
+ }
+
+ /* Allocate actual buffers for all queues */
+ for (q = 0; q < adapter->num_rx_queues; q++) {
+ rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
+ if (rc) {
+ /* Free pools for all previous queues */
+ while (--q >= 0)
+ ibmveth_free_queue_buffer_pools(adapter, q);
+ return rc;
+ }
+ }
+
+ netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
+ adapter->num_rx_queues);
+ return 0;
+}
+
+/**
+ * ibmveth_free_buffer_pools - Free buffer pools for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees buffer pools for all queues using the helper function.
+ */
+static void
+ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
+{
+ int q;
+
+ /* Free buffer pools for all queues */
+ for (q = 0; q < adapter->num_rx_queues; q++)
+ ibmveth_free_queue_buffer_pools(adapter, q);
+
+ netdev_dbg(adapter->netdev, "freed buffer pools for %d queue(s)\n",
+ adapter->num_rx_queues);
+}
+
/**
* ibmveth_remove_buffer_from_pool - remove a buffer from a pool
* @adapter: adapter instance
@@ -851,16 +987,9 @@ static int ibmveth_open(struct net_device *netdev)
goto out_free_tx_ltb;
}
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- if (!adapter->rx_buff_pool[0][i].active)
- continue;
- if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
- netdev_err(netdev, "unable to alloc pool\n");
- adapter->rx_buff_pool[0][i].active = 0;
- rc = -ENOMEM;
- goto out_free_buffer_pools;
- }
- }
+ rc = ibmveth_alloc_buffer_pools(adapter);
+ if (rc)
+ goto out_free_tx_ltb;
netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
@@ -885,11 +1014,7 @@ static int ibmveth_open(struct net_device *netdev)
return 0;
out_free_buffer_pools:
- while (--i >= 0) {
- if (adapter->rx_buff_pool[0][i].active)
- ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[0][i]);
- }
+ ibmveth_free_buffer_pools(adapter);
out_free_tx_ltb:
while (--i >= 0)
ibmveth_free_tx_ltb(adapter, i);
@@ -928,14 +1053,10 @@ static int ibmveth_close(struct net_device *netdev)
ibmveth_update_rx_no_buffer(adapter);
+ ibmveth_free_buffer_pools(adapter);
ibmveth_cleanup_rx_resources(adapter);
ibmveth_free_filter_list(adapter);
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
- if (adapter->rx_buff_pool[0][i].active)
- ibmveth_free_buffer_pool(adapter,
- &adapter->rx_buff_pool[0][i]);
-
for (i = 0; i < netdev->real_num_tx_queues; i++)
ibmveth_free_tx_ltb(adapter, i);
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (3 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
` (9 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Queue 0 and subordinate RX queues use different interrupt control
interfaces in PHYP:
- queue 0: h_vio_signal() after h_register_logical_lan()
- queue N: H_VIOCTL against the queue handle/hwirq mapping
The current code is single-queue oriented and cannot safely scale to
multiple RX queues in poll completion and open/close IRQ setup.
Introduce queue-indexed interrupt helpers and wire them into
open()/close()/poll()/interrupt in the same patch:
ibmveth_toggle_irq() / enable_irq() / disable_irq()
ibmveth_setup_rx_interrupts() / ibmveth_cleanup_rx_interrupts()
ibmveth_schedule_rx_queue()
These helpers centralize queue0-vs-subordinate dispatch.
request_irq() uses &adapter->napi[i] as the per-queue cookie so the
handler can resolve the queue index.
Move napi_enable() into setup_rx_interrupts() (after LAN registration
and buffer-pool allocation): request_irq -> napi_enable. In this
single-queue tree, setup does not yet unmask PHYP; the first poll/kick
still enables queue 0 via schedule_rx_queue(). That order matches the
later scale-up rule (NAPI live before PHYP unmask), not an inverted
window relative to it.
Factor process-context RX kicks (open, resume, pool sysfs, netpoll)
into ibmveth_schedule_rx_queue(); keep ibmveth_interrupt() as a thin
IRQ-only wrapper.
cleanup_rx_interrupts() masks PHYP and synchronizes IRQs before
napi_disable, then free_irq. Close then proceeds to h_free_logical_lan():
free_irq before free_lan is intentional once PHYP delivery is masked.
opened / rx_irq_setup gate whether cleanup walks IRQ/NAPI state.
schedule_rx_queue() masks PHYP only when napi_schedule_prep() succeeds.
Masking on prep failure can race a completing poll that already
re-enabled PHYP and leave NAPI idle with the queue masked (TX OK, RX
stalled until reload). Teardown storm control stays on STOP
(disable_irq + synchronize_irq before napi_disable) and the later poll
re-arm guard, not on the schedule helper failure path.
IRQ helpers return 0 or negative errno only (never raw H_* to
ethtool/resize). H_PARAMETER is folded to success only on disable
(idempotent mask). On enable it remains an error so a stuck-masked
queue stays visible to poll/resize recovery.
Runtime remains single-queue (num_rx_queues is still 1).
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Remask+sync after napi_disable in cleanup (in-flight poll can re-arm)
- Interrupt: quiet IRQ_NONE on out-of-range qindex (no WARN storm)
- Opened / rx_irq_setup gate cleanup so close after a failed open
cannot napi_disable / free_irq without a prior enable/request; set
opened on successful open; rx_irq_setup only on full setup success
- H_PARAMETER fold disable-only; enable stays error (stuck-masked visible)
- IRQ helpers return 0 / negative errno only (never raw H_* to ethtool)
- Decision: keep open IRQ order request_irq -> napi_enable while PHYP
stays masked until schedule/enable (coherent with later scale-up)
- schedule_rx_queue returns bool (napi_schedule_prep success)
- Keep mask-only-on-prep-success (no else-mask; avoids idle+masked race)
- Call out free_irq-before-free_lan as intentional once PHYP is masked
- Poll re-arm during teardown lands with SQ poll-harden (not claimed here)
- synchronize_net() after RX IRQ/NAPI teardown in close
- Add subordinate IRQ dispose helpers (per-queue + bulk 1..N; bound to MAX)
Changes in v4:
- Include irq.h / irqdomain.h with first irq_dispose_mapping() use.
- Introduce IRQ helpers in the same patch that wires open/close/poll
callers, instead of leaving unused statics.
- Factor process-context RX kicks into ibmveth_schedule_rx_queue();
keep ibmveth_interrupt() as the IRQ-only wrapper.
- On cleanup, mask PHYP and synchronize_irq before napi_disable
(storm-safety; not fully behavior-preserving vs classic close).
- Leave queue_irq[0] set after cleanup (queue 0 uses netdev->irq;
next open reuses it). Only subordinate virqs are disposed.
drivers/net/ethernet/ibm/ibmveth.c | 396 ++++++++++++++++++++++++++---
drivers/net/ethernet/ibm/ibmveth.h | 4 +
2 files changed, 359 insertions(+), 41 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 1a428ee60d58..99eeb6ef51bf 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -21,6 +21,8 @@
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
#include <linux/mm.h>
#include <linux/pm.h>
#include <linux/ethtool.h>
@@ -331,6 +333,311 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
}
}
+/**
+ * ibmveth_toggle_irq - Common helper to enable/disable queue interrupts
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ * @enable: true to enable, false to disable
+ *
+ * For queue 0 (primary), uses h_vio_signal() as it's registered via
+ * h_register_logical_lan(). For subordinate queues (1+), uses H_VIOCTL
+ * with H_ENABLE/DISABLE_VIO_INTERRUPT for per-queue interrupt control.
+ *
+ * Return: 0 on success, negative errno on failure (never raw H_*).
+ */
+static int
+ibmveth_toggle_irq(struct ibmveth_adapter *adapter, int queue_index,
+ bool enable)
+{
+ unsigned long h_rc;
+ unsigned long irq = adapter->queue_irq[queue_index];
+ const char *action = enable ? "enable" : "disable";
+
+ if (queue_index == 0) {
+ /* Primary queue: use h_vio_signal() */
+ h_rc = h_vio_signal(adapter->vdev->unit_address,
+ enable ? VIO_IRQ_ENABLE : VIO_IRQ_DISABLE);
+ } else {
+ /* Subordinate queues: use H_VIOCTL with hardware IRQ */
+ struct irq_data *irq_data = irq_get_irq_data(irq);
+ irq_hw_number_t hwirq;
+ u64 vioctl_cmd = enable ? H_ENABLE_VIO_INTERRUPT :
+ H_DISABLE_VIO_INTERRUPT;
+
+ if (!irq_data) {
+ netdev_err(adapter->netdev,
+ "Failed to get IRQ data for queue %d (virq=%lu)\n",
+ queue_index, irq);
+ return -EINVAL;
+ }
+
+ hwirq = irqd_to_hwirq(irq_data);
+ h_rc = plpar_hcall_norets(H_VIOCTL,
+ adapter->vdev->unit_address,
+ vioctl_cmd,
+ hwirq, 0, 0);
+
+ /*
+ * H_PARAMETER is ambiguous (already in requested state vs bad
+ * args). Fold only on disable as an idempotent mask. On enable
+ * keep it an error so a stuck-masked queue stays visible to
+ * poll/resize recovery.
+ */
+ if (h_rc == H_PARAMETER && !enable) {
+ dev_warn_ratelimited(&adapter->netdev->dev,
+ "H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n",
+ action, queue_index, hwirq);
+ return 0;
+ }
+ }
+
+ if (h_rc) {
+ netdev_err(adapter->netdev,
+ "Failed to %s IRQ for queue %d, rc=0x%lx\n",
+ action, queue_index, h_rc);
+ return -EIO;
+ }
+ return 0;
+}
+
+/**
+ * ibmveth_disable_irq - Disable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, negative errno on failure
+ */
+static int
+ibmveth_disable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+ return ibmveth_toggle_irq(adapter, queue_index, false);
+}
+
+/**
+ * ibmveth_enable_irq - Enable interrupt for a specific queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Index of the queue (0 for primary, 1+ for subordinate)
+ *
+ * Return: 0 on success, negative errno on failure
+ */
+static int
+ibmveth_enable_irq(struct ibmveth_adapter *adapter, int queue_index)
+{
+ return ibmveth_toggle_irq(adapter, queue_index, true);
+}
+
+/**
+ * ibmveth_dispose_subordinate_irq_mapping - Drop one subordinate virq mapping
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: RX queue index (1..N)
+ *
+ * Subordinate queues get mappings from irq_create_mapping() during PHYP
+ * registration. Queue 0 uses netdev->irq from device tree and is left alone.
+ *
+ * Bound against IBMVETH_MAX_RX_QUEUES, not num_rx_queues: a caller may
+ * dispose a queue that is no longer in the published live set but still
+ * owns a virq in queue_irq[]. Contrast with the bulk helper, which only
+ * walks 1..num_rx_queues-1 (close / open-fail cleanup of the live set).
+ *
+ * Linux virq lifetime is owned by interrupt cleanup helpers. Call this only
+ * after free_irq() when a handler was installed, or from registration failure
+ * cleanup before request_irq().
+ */
+static void
+ibmveth_dispose_subordinate_irq_mapping(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ if (queue_idx <= 0 || queue_idx >= IBMVETH_MAX_RX_QUEUES)
+ return;
+
+ if (adapter->queue_irq[queue_idx]) {
+ irq_dispose_mapping(adapter->queue_irq[queue_idx]);
+ adapter->queue_irq[queue_idx] = 0;
+ }
+}
+
+/**
+ * ibmveth_dispose_subordinate_irq_mappings - Drop virq mappings for queues 1..N
+ * @adapter: ibmveth adapter structure
+ *
+ * Bulk helper for close / open-fail cleanup of the published live set
+ * (queues 1..num_rx_queues-1). Paths that need a retired or not-yet-published
+ * queue must call ibmveth_dispose_subordinate_irq_mapping() directly.
+ */
+static void
+ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
+{
+ int i;
+
+ for (i = 1; i < adapter->num_rx_queues; i++)
+ ibmveth_dispose_subordinate_irq_mapping(adapter, i);
+}
+
+/**
+ * ibmveth_setup_rx_interrupts - Register IRQs and enable NAPI
+ * @adapter: ibmveth adapter structure
+ *
+ * Registers interrupt handlers for all RX queues, enables NAPI, then
+ * enables hypervisor interrupt delivery for multi-queue mode after
+ * every queue has a Linux handler installed. For multi-queue open the
+ * caller should replenish RX buffers before this helper so traffic
+ * during open is not dropped (PHYP only interrupts after a successful
+ * enqueue, which needs buffers). Single-queue open leaves PHYP masked
+ * here and kicks NAPI afterward (classic path: first poll posts then
+ * enables).
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i, rc, num = adapter->num_rx_queues;
+
+ for (i = 0; i < num; i++) {
+ if (!adapter->queue_irq[i]) {
+ netdev_err(netdev, "queue %d has invalid IRQ (0)\n", i);
+ rc = -EINVAL;
+ goto err_free_irqs;
+ }
+
+ rc = request_irq(adapter->queue_irq[i], ibmveth_interrupt,
+ 0, netdev->name, &adapter->napi[i]);
+ if (rc) {
+ netdev_err(netdev,
+ "request_irq() failed for irq 0x%x queue %d: %d\n",
+ adapter->queue_irq[i], i, rc);
+ goto err_free_irqs;
+ }
+ }
+
+ for (i = 0; i < num; i++)
+ napi_enable(&adapter->napi[i]);
+
+ if (adapter->multi_queue && num > 1) {
+ for (i = 0; i < num; i++) {
+ rc = ibmveth_enable_irq(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d, rc=%d\n",
+ i, rc);
+ while (--i >= 0) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+ rc = -EIO;
+ goto err_disable_napi;
+ }
+ }
+ }
+
+ /* Set only on full success; fail paths leave this false so a later
+ * close() / cleanup is a no-op.
+ */
+ adapter->rx_irq_setup = true;
+ return 0;
+
+err_disable_napi:
+ /* PHYP unmask was rolled back above; disable NAPI before free_irq */
+ for (i = 0; i < num; i++)
+ napi_disable(&adapter->napi[i]);
+ for (i = 0; i < num; i++) {
+ if (adapter->queue_irq[i])
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ }
+ goto err_dispose_mappings;
+
+err_free_irqs:
+ while (--i >= 0)
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+err_dispose_mappings:
+ /* Both setup failure paths own subordinate virq disposal. */
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+ return rc;
+}
+
+/**
+ * ibmveth_cleanup_rx_interrupts - Mask PHYP IRQs, stop NAPI, and free IRQs
+ * @adapter: ibmveth adapter structure
+ *
+ * Mask and synchronize each queue IRQ before napi_disable() so the handler
+ * cannot miss a PHYP mask while NAPI is already dead. Remask after
+ * napi_disable() in case an in-flight poll re-armed PHYP while we waited.
+ * free_irq() runs only after that. Safe for close and for open failure after
+ * setup_rx_interrupts() already unmasked PHYP. No-op if setup never
+ * succeeded (avoids double napi_disable / free_irq after a failed close+open
+ * while IFF_UP remains set).
+ */
+static void
+ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
+{
+ int i;
+
+ if (!adapter->rx_irq_setup)
+ return;
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (!adapter->queue_irq[i])
+ continue;
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (!adapter->queue_irq[i])
+ continue;
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ if (adapter->queue_irq[i])
+ free_irq(adapter->queue_irq[i], &adapter->napi[i]);
+ }
+
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+
+ /* Queue 0 uses netdev->irq; leave queue_irq[0] for next open. */
+ adapter->rx_irq_setup = false;
+}
+
+/**
+ * ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue
+ * @adapter: ibmveth adapter structure
+ * @qindex: RX queue index
+ *
+ * Shared by the IRQ handler and process-context kick sites (open, resume,
+ * pool sysfs, poll_controller).
+ *
+ * Return: true if NAPI was scheduled (and PHYP masked), false if prep failed
+ */
+static bool ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
+ int qindex)
+{
+ struct napi_struct *napi = &adapter->napi[qindex];
+ int rc;
+
+ if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
+ return false;
+
+ /*
+ * Only mask PHYP when NAPI will run. Masking on prep failure can
+ * race a completing poll that already re-enabled the queue, leaving
+ * NAPI idle with the IRQ masked (TX works, RX stalls) until reload.
+ * Storm prevention on teardown remains in cleanup/disable paths.
+ */
+ if (napi_schedule_prep(napi)) {
+ rc = ibmveth_disable_irq(adapter, qindex);
+ WARN_ON(rc);
+ __napi_schedule(napi);
+ return true;
+ }
+ return false;
+}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
@@ -945,8 +1252,6 @@ static int ibmveth_open(struct net_device *netdev)
netdev_dbg(netdev, "open starting\n");
- napi_enable(&adapter->napi[0]);
-
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
rxq_entries += adapter->rx_buff_pool[0][i].size;
@@ -970,7 +1275,8 @@ static int ibmveth_open(struct net_device *netdev)
adapter->rx_queue[0].queue_len;
rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
- h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
+ adapter->queue_irq[0] = netdev->irq;
+ ibmveth_disable_irq(adapter, 0);
lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
@@ -991,24 +1297,20 @@ static int ibmveth_open(struct net_device *netdev)
if (rc)
goto out_free_tx_ltb;
- netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
- rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
- netdev);
- if (rc != 0) {
- netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
- netdev->irq, rc);
+ rc = ibmveth_setup_rx_interrupts(adapter);
+ if (rc) {
do {
lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
goto out_free_buffer_pools;
}
netdev_dbg(netdev, "initial replenish cycle\n");
- ibmveth_interrupt(netdev->irq, netdev);
+ ibmveth_schedule_rx_queue(adapter, 0);
netif_tx_start_all_queues(netdev);
+ adapter->opened = true;
netdev_dbg(netdev, "open complete\n");
return 0;
@@ -1022,7 +1324,6 @@ static int ibmveth_open(struct net_device *netdev)
out_free_filter_list:
ibmveth_free_filter_list(adapter);
out:
- napi_disable(&adapter->napi[0]);
return rc;
}
@@ -1032,27 +1333,32 @@ static int ibmveth_close(struct net_device *netdev)
long lpar_rc;
int i;
- netdev_dbg(netdev, "close starting\n");
+ /* Gate on opened, not IFF_UP: pool_store/change_mtu close+open can
+ * leave IFF_UP set after a failed reopen.
+ */
+ if (!adapter->opened)
+ return 0;
- napi_disable(&adapter->napi[0]);
+ adapter->opened = false;
+
+ netdev_dbg(netdev, "close starting\n");
netif_tx_stop_all_queues(netdev);
- h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_DISABLE);
+ ibmveth_cleanup_rx_interrupts(adapter);
+ /* Wait for softirq/poll that already passed shutdown checks. */
+ synchronize_net();
+ ibmveth_update_rx_no_buffer(adapter);
+ /* Full LAN teardown (subordinates arrive with register helpers). */
do {
lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
-
if (lpar_rc != H_SUCCESS) {
- netdev_err(netdev, "h_free_logical_lan failed with %lx, "
- "continuing with close\n", lpar_rc);
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan failed with %lx, continuing\n",
+ lpar_rc);
}
-
- free_irq(netdev->irq, netdev);
-
- ibmveth_update_rx_no_buffer(adapter);
-
ibmveth_free_buffer_pools(adapter);
ibmveth_cleanup_rx_resources(adapter);
ibmveth_free_filter_list(adapter);
@@ -1696,7 +2002,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
container_of(napi, struct ibmveth_adapter, napi[0]);
struct net_device *netdev = adapter->netdev;
int frames_processed = 0;
- unsigned long lpar_rc;
+ int rc;
u16 mss = 0;
restart_poll:
@@ -1796,15 +2102,15 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
/* We think we are done - reenable interrupts,
* then check once more to make sure we are done.
*/
- lpar_rc = h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_ENABLE);
- if (WARN_ON(lpar_rc != H_SUCCESS)) {
+ rc = ibmveth_enable_irq(adapter, 0);
+ if (WARN_ON(rc)) {
schedule_work(&adapter->work);
goto out;
}
if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
- lpar_rc = h_vio_signal(adapter->vdev->unit_address,
- VIO_IRQ_DISABLE);
+ rc = ibmveth_disable_irq(adapter, 0);
+ WARN_ON(rc);
goto restart_poll;
}
@@ -1814,16 +2120,20 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
{
- struct net_device *netdev = dev_instance;
+ struct napi_struct *napi = dev_instance;
+ struct net_device *netdev = napi->dev;
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- unsigned long lpar_rc;
+ int qindex;
- if (napi_schedule_prep(&adapter->napi[0])) {
- lpar_rc = h_vio_signal(adapter->vdev->unit_address,
- VIO_IRQ_DISABLE);
- WARN_ON(lpar_rc != H_SUCCESS);
- __napi_schedule(&adapter->napi[0]);
- }
+ qindex = napi - adapter->napi;
+ /*
+ * Quiet on out-of-range: teardown can leave a residual IRQ after the
+ * live count drops. Do not WARN-storm; return IRQ_NONE until free_irq.
+ */
+ if (qindex < 0 || qindex >= adapter->num_rx_queues)
+ return IRQ_NONE;
+
+ ibmveth_schedule_rx_queue(adapter, qindex);
return IRQ_HANDLED;
}
@@ -1928,8 +2238,10 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
#ifdef CONFIG_NET_POLL_CONTROLLER
static void ibmveth_poll_controller(struct net_device *dev)
{
- ibmveth_replenish_task(netdev_priv(dev));
- ibmveth_interrupt(dev->irq, dev);
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+
+ ibmveth_replenish_task(adapter);
+ ibmveth_schedule_rx_queue(adapter, 0);
}
#endif
@@ -2342,8 +2654,8 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
}
rtnl_unlock();
- /* kick the interrupt handler to allocate/deallocate pools */
- ibmveth_interrupt(netdev->irq, netdev);
+ /* kick RX processing to allocate/deallocate pools */
+ ibmveth_schedule_rx_queue(adapter, 0);
return count;
unlock_err:
@@ -2383,7 +2695,9 @@ static struct kobj_type ktype_veth_pool = {
static int ibmveth_resume(struct device *dev)
{
struct net_device *netdev = dev_get_drvdata(dev);
- ibmveth_interrupt(netdev->irq, netdev);
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
+ ibmveth_schedule_rx_queue(adapter, 0);
return 0;
}
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index fc6a01e810c6..bf5dd5703773 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -312,6 +312,10 @@ struct ibmveth_adapter {
unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES];
int multi_queue;
unsigned int num_rx_queues;
+ /* Lifetime: true after successful ndo_open until close clears it. */
+ bool opened;
+ /* Lifetime: true while RX IRQ handlers / NAPI are installed. */
+ bool rx_irq_setup;
int rx_csum;
int large_send;
bool is_active_trunk;
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 06/15] ibmveth: Refactor TX resource allocation in open/close paths
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (4 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 07/15] ibmveth: Add RX queue register helpers for MQ Mingming Cao
` (8 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Same story as the RX refactor: pull TX LTB alloc/free out of open/close
into helpers and wire them in this patch.
ibmveth_alloc_tx_resources()
ibmveth_free_tx_resources()
They wrap the existing per-queue allocate_tx_ltb() / free_tx_ltb()
primitives. alloc_tx_resources() allocates every TX queue and unwinds
partial failure itself; free_tx_resources() walks real_num_tx_queues.
The helpers remove dependence on shared open/close loop indices and
match the RX helper structure. TX was already multi-queue capable via
ethtool -L.
Also tighten TX LTB lifetime: free_tx_ltb() keys off tx_ltb_ptr[]
presence (not a dma==0 sentinel) and clears tx_ltb_dma[] after unmap;
allocate_tx_ltb() clears tx_ltb_dma[] after DMA-map failure.
Move TX LTB allocation to the end of open(), after LAN registration,
RX pools, RX interrupt setup, and the initial replenish kick. A late
alloc_tx_resources() failure jumps to out_cleanup_rx_interrupts and
must not call free_tx_resources() again: alloc already freed any
partial TX LTBs. start_xmit() bails if tx_ltb_ptr[] is gone so RX can
be live while TX LTB alloc still runs (and so close/failed-reopen with
IFF_UP set cannot UAF).
After LAN registration, open-fail teardown frees the logical LAN before
tearing down RX pool DMA (intentional safer order than leaving the LAN
registered while unmapping RX memory).
close() quiesces TX with netif_tx_disable() (stop_all_queues does not
wait for in-flight ndo_start_xmit), then frees LTBs after
h_free_logical_lan() via free_tx_resources() - required because direct
close() callers bypass synchronize_net().
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Quiesce TX with netif_tx_disable before free (stop_all_queues does not
wait for in-flight xmit); free LTBs after h_free_logical_lan - direct
close() callers bypass synchronize_net()
- Guard start_xmit if tx_ltb_ptr gone so open can leave RX live while TX
LTB alloc still runs (also covers close/failed-reopen with IFF_UP set)
- Drop fake mid-open TX-leak / Fixes: motivation; reword as helper
extraction matching RX (shared loop-index independence)
- Free TX LTB by pointer presence (drop dma==0 sentinel; dma_mapping_error
already cleared the slot on map failure)
- Document intentional open-fail LAN-first unwind (free_lan before RX
pool/DMA teardown) rather than leaving it silent in a TX-only refactor
- Drop drive-by blank-line cosmetics (header / start_xmit)
Changes in v4:
- Introduce the TX resource helpers in the same patch that wires their
first open/close callers.
- Do not free TX LTBs again after a failed alloc_tx_resources();
harden free_tx_ltb() against unset slots.
- Move TX allocation after RX IRQ setup / replenish kick so open()
failure unwind no longer depends on a shared loop index (also fixes
a mid-open TX LTB leak).
drivers/net/ethernet/ibm/ibmveth.c | 98 +++++++++++++++++++++++-------
1 file changed, 75 insertions(+), 23 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 99eeb6ef51bf..b39e8c53cbfd 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1183,8 +1183,12 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
{
+ if (!adapter->tx_ltb_ptr[idx])
+ return;
+
dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
adapter->tx_ltb_size, DMA_TO_DEVICE);
+ adapter->tx_ltb_dma[idx] = 0;
kfree(adapter->tx_ltb_ptr[idx]);
adapter->tx_ltb_ptr[idx] = NULL;
}
@@ -1207,12 +1211,54 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
"unable to DMA map tx long term buffer\n");
kfree(adapter->tx_ltb_ptr[idx]);
adapter->tx_ltb_ptr[idx] = NULL;
+ adapter->tx_ltb_dma[idx] = 0;
return -ENOMEM;
}
return 0;
}
+/**
+ * ibmveth_alloc_tx_resources - Allocate TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Allocates TX Long Term Buffers (LTBs) for all TX queues.
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++) {
+ if (ibmveth_allocate_tx_ltb(adapter, i))
+ goto err_free_ltbs;
+ }
+
+ return 0;
+
+err_free_ltbs:
+ while (--i >= 0)
+ ibmveth_free_tx_ltb(adapter, i);
+ return -ENOMEM;
+}
+
+/**
+ * ibmveth_free_tx_resources - Free TX resources for all queues
+ * @adapter: ibmveth adapter structure
+ *
+ * Frees TX Long Term Buffers (LTBs) for all TX queues.
+ */
+static void ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ int i;
+
+ for (i = 0; i < netdev->real_num_tx_queues; i++)
+ ibmveth_free_tx_ltb(adapter, i);
+}
+
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
union ibmveth_buf_desc rxq_desc, u64 mac_address)
{
@@ -1263,12 +1309,6 @@ static int ibmveth_open(struct net_device *netdev)
if (rc)
goto out_free_filter_list;
- rc = -ENOMEM;
- for (i = 0; i < netdev->real_num_tx_queues; i++) {
- if (ibmveth_allocate_tx_ltb(adapter, i))
- goto out_free_tx_ltb;
- }
-
mac_address = ether_addr_to_u64(netdev->dev_addr);
rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
@@ -1290,24 +1330,24 @@ static int ibmveth_open(struct net_device *netdev)
rxq_desc.desc,
mac_address);
rc = -ENONET;
- goto out_free_tx_ltb;
+ goto out_free_queue_mem;
}
rc = ibmveth_alloc_buffer_pools(adapter);
if (rc)
- goto out_free_tx_ltb;
+ goto out_unregister_lan;
rc = ibmveth_setup_rx_interrupts(adapter);
- if (rc) {
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
- goto out_free_buffer_pools;
- }
+ if (rc)
+ goto out_unregister_lan;
netdev_dbg(netdev, "initial replenish cycle\n");
ibmveth_schedule_rx_queue(adapter, 0);
+ rc = ibmveth_alloc_tx_resources(adapter);
+ if (rc)
+ goto out_cleanup_rx_interrupts;
+
netif_tx_start_all_queues(netdev);
adapter->opened = true;
@@ -1315,11 +1355,14 @@ static int ibmveth_open(struct net_device *netdev)
return 0;
-out_free_buffer_pools:
+out_cleanup_rx_interrupts:
+ ibmveth_cleanup_rx_interrupts(adapter);
+out_unregister_lan:
+ do {
+ lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
ibmveth_free_buffer_pools(adapter);
-out_free_tx_ltb:
- while (--i >= 0)
- ibmveth_free_tx_ltb(adapter, i);
+out_free_queue_mem:
ibmveth_cleanup_rx_resources(adapter);
out_free_filter_list:
ibmveth_free_filter_list(adapter);
@@ -1331,7 +1374,6 @@ static int ibmveth_close(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
long lpar_rc;
- int i;
/* Gate on opened, not IFF_UP: pool_store/change_mtu close+open can
* leave IFF_UP set after a failed reopen.
@@ -1343,7 +1385,10 @@ static int ibmveth_close(struct net_device *netdev)
netdev_dbg(netdev, "close starting\n");
- netif_tx_stop_all_queues(netdev);
+ /* Disable and wait for in-flight ndo_start_xmit (stop_all_queues
+ * alone does not). Direct close() callers bypass synchronize_net().
+ */
+ netif_tx_disable(netdev);
ibmveth_cleanup_rx_interrupts(adapter);
/* Wait for softirq/poll that already passed shutdown checks. */
@@ -1359,13 +1404,14 @@ static int ibmveth_close(struct net_device *netdev)
"h_free_logical_lan failed with %lx, continuing\n",
lpar_rc);
}
+ /* Free TX LTBs after quiesce and after H_FREE_LOGICAL_LAN so xmit
+ * cannot touch unmapped bounce buffers while the LAN is live.
+ */
+ ibmveth_free_tx_resources(adapter);
ibmveth_free_buffer_pools(adapter);
ibmveth_cleanup_rx_resources(adapter);
ibmveth_free_filter_list(adapter);
- for (i = 0; i < netdev->real_num_tx_queues; i++)
- ibmveth_free_tx_ltb(adapter, i);
-
netdev_dbg(netdev, "close complete\n");
return 0;
@@ -1789,6 +1835,12 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
int i, queue_num = skb_get_queue_mapping(skb);
unsigned long mss = 0;
+ /* Close / failed reopen can free LTBs while IFF_UP is still set. */
+ if (unlikely(!adapter->tx_ltb_ptr[queue_num])) {
+ dev_kfree_skb_any(skb);
+ return NETDEV_TX_OK;
+ }
+
if (ibmveth_is_packet_unsupported(skb, netdev))
goto out;
/* veth can't checksum offload UDP */
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 07/15] ibmveth: Add RX queue register helpers for MQ
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (5 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 08/15] ibmveth: Add queue-aware RX buffer submit helper " Mingming Cao
` (7 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
MQ RX changes queue lifecycle from one adapter-level register/free pair
to a mixed model:
- queue 0: registered via h_register_logical_lan*()
- queues 1..N: registered via H_REG_LOGICAL_LAN_QUEUE
This patch extracts the queue-0 control-plane helpers used by open/close
today and wires them in the same commit:
ibmveth_register_rx_queues()
ibmveth_free_all_queues()
H_FREE_LOGICAL_LAN tears down the primary LAN and any subordinate queues
registered under it (PAPR/PHYP full-teardown contract). Close and
open-fail use that one free_lan. Incremental scale-down uses per-queue
H_FREE_LOGICAL_LAN_QUEUE instead; that path arrives with resize.
Also update ibmveth_register_logical_lan() so that when multi_queue is
later enabled, queue 0 uses h_register_logical_lan_with_handle() and
stores queue_handle[0]. Runtime remains single-queue: multi_queue is
still false, so queue 0 keeps the legacy h_register_logical_lan() flow
and no subordinate queue is registered. Subordinate register helpers
arrive with MQ enablement; deregister arrives with resize at first use.
Introduce adapter->hcall_stats here for register/free path accounting
(first use).
Open/close unwind:
- failures after successful LAN registration enter
out_unregister_queues, then fall through to out_free_buffer_pools,
so RX pools are not leaked;
- free_all_queues() runs before free_buffer_pools() on open failure
and close, so PHYP cannot retain a registered LAN while its
DMA-backed RX pools are released.
Failures before registration go directly to out_free_buffer_pools.
RX interrupt masking before napi_disable is already handled by
cleanup_rx_interrupts() from the IRQ-helper patch.
As of this patch, open/close follow the MQ-ready pipeline below
(still single-queue). The MQ enablement patch changes open's kick
(replenish-all before setup_rx_interrupts, then restart_rx_queue per
queue for SQ and MQ) and should restate that open path there. Close
shape stays the same through enablement.
ibmveth_open() (this commit):
1. ibmveth_alloc_filter_list()
2. ibmveth_alloc_rx_queues() - buffer lists + RX rings
3. ibmveth_alloc_buffer_pools() - guest RX memory before PHYP
4. ibmveth_register_rx_queues() - PHYP registration (no IRQ enable)
5. netif_set_real_num_rx_queues()
6. ibmveth_setup_rx_interrupts() - request_irq + napi_enable
7. initial kick - schedule_rx_queue(0)
8. ibmveth_alloc_tx_resources()
9. netif_tx_start_all_queues(); adapter->opened = true
ibmveth_close() (this commit):
0. if (!opened) return; opened = false
1. netif_tx_disable()
2. ibmveth_cleanup_rx_interrupts() - mask PHYP, napi_disable, free_irq
3. synchronize_net()
4. ibmveth_update_rx_no_buffer() - last glimpse while LAN registered
5. ibmveth_free_all_queues() - H_FREE_LOGICAL_LAN (this patch)
6. ibmveth_free_tx_resources()
7. ibmveth_free_buffer_pools()
8. ibmveth_cleanup_rx_resources()
9. ibmveth_free_filter_list()
Per-queue NULL-safe update_rx_no_buffer() lands in the MQ enablement
patch. Idempotent close (opened / rx_irq_setup) is owned by the
IRQ-helper patch.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Document H_FREE_LOGICAL_LAN full-teardown contract in kdoc/changelog:
one free_lan drops primary + any subordinate queues
(H_FREE_LOGICAL_LAN_QUEUE is for incremental scale-down only) - v4
helper text did not spell the subordinate semantics
- Call out close() update_rx_no_buffer() before free_lan as intentional
last-glimpse accounting while the LAN is still registered (order
already in v4; v4 step list omitted it); NULL-safe per-queue form
lands with MQ enablement
- opened / rx_irq_setup idempotent close is owned by the IRQ-helper
patch (same failed-reopen / second napi_disable hang raised here)
- Poll re-arm during teardown not claimed here (same race as IRQ patch;
lands with poll harden)
Changes in v4:
- Introduce register/free helpers in the same patch that wires their
first open/close callers; keep subordinate-only helpers deferred
until MQ enablement.
- Introduce adapter->hcall_stats here (first use); not in patch 2.
- Correct open/close unwind so free_all_queues() precedes
free_buffer_pools().
- Drop the orphaned big-bang "open/close pipeline" patch from v3; that
wiring is incremental across helper patches 3-7 instead.
drivers/net/ethernet/ibm/ibmveth.c | 169 +++++++++++++++++++++--------
drivers/net/ethernet/ibm/ibmveth.h | 14 +++
2 files changed, 137 insertions(+), 46 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index b39e8c53cbfd..8519cad50322 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1260,9 +1260,14 @@ static void ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
}
static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
- union ibmveth_buf_desc rxq_desc, u64 mac_address)
+ union ibmveth_buf_desc rxq_desc,
+ u64 mac_address)
{
int rc, try_again = 1;
+ unsigned long ua = adapter->vdev->unit_address;
+ unsigned long buf_dma = adapter->buffer_list_dma[0];
+ unsigned long filter_dma = adapter->filter_list_dma;
+ unsigned long qh0;
/*
* After a kexec the adapter will still be open, so our attempt to
@@ -1270,13 +1275,27 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
* try again, but only once.
*/
retry:
- rc = h_register_logical_lan(adapter->vdev->unit_address,
- adapter->buffer_list_dma[0], rxq_desc.desc,
- adapter->filter_list_dma, mac_address);
+ /* In multi-queue mode, obtain a queue handle for queue 0 so all RX
+ * queues can use the same per-queue buffer hypercalls.
+ */
+ if (adapter->multi_queue) {
+ rc = h_register_logical_lan_with_handle(ua, buf_dma,
+ rxq_desc.desc,
+ filter_dma,
+ mac_address,
+ &qh0);
+ if (rc == H_SUCCESS)
+ adapter->queue_handle[0] = qh0;
+ } else {
+ rc = h_register_logical_lan(ua, buf_dma, rxq_desc.desc,
+ filter_dma, mac_address);
+ }
+ adapter->hcall_stats.reg_lan++;
if (rc != H_SUCCESS && try_again) {
do {
rc = h_free_logical_lan(adapter->vdev->unit_address);
+ adapter->hcall_stats.free_lan++;
} while (H_IS_LONG_BUSY(rc) || (rc == H_BUSY));
try_again = 0;
@@ -1286,14 +1305,93 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
return rc;
}
+/**
+ * ibmveth_free_all_queues - Free all RX queues at once
+ * @adapter: ibmveth adapter structure
+ *
+ * Issues one H_FREE_LOGICAL_LAN for full adapter teardown. Per PAPR/PHYP,
+ * that drops the primary LAN and any subordinate queues registered under
+ * it. Incremental scale-down uses H_FREE_LOGICAL_LAN_QUEUE per queue
+ * instead; do not use this helper for partial live-set shrink.
+ *
+ * Used during interface close and registration error cleanup.
+ *
+ * Clears queue handles only; queue_irq[] is released by
+ * ibmveth_cleanup_rx_interrupts().
+ */
+static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
+{
+ unsigned long lpar_rc;
+ int i;
+
+ netdev_dbg(adapter->netdev, "freeing all RX queues at once\n");
+
+ do {
+ lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
+ adapter->hcall_stats.free_lan++;
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan failed: %ld\n", lpar_rc);
+ }
+
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ adapter->queue_handle[i] = 0;
+}
+
+/**
+ * ibmveth_register_rx_queues - Register RX queues with hypervisor
+ * @adapter: ibmveth adapter structure
+ * @mac_address: MAC address for device registration
+ *
+ * Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue
+ * registration is added when multi-queue RX is enabled.
+ *
+ * Return: 0 on success, -ENONET if queue 0 registration fails
+ */
+static int
+ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
+{
+ struct net_device *netdev = adapter->netdev;
+ union ibmveth_buf_desc rxq_desc;
+ unsigned long lpar_rc;
+ int rc;
+
+ rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+ adapter->rx_queue[0].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
+ adapter->queue_irq[0] = netdev->irq;
+
+ rc = ibmveth_disable_irq(adapter, 0);
+ if (rc)
+ netdev_dbg(netdev,
+ "Failed to disable IRQ for queue 0 before registration, rc=%d\n",
+ rc);
+
+ lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev,
+ "h_register_logical_lan failed: %ld\n", lpar_rc);
+ netdev_err(netdev,
+ "buffer TCE:0x%llx filter TCE:0x%llx rxq desc:0x%llx MAC:0x%llx\n",
+ adapter->buffer_list_dma[0],
+ adapter->filter_list_dma,
+ rxq_desc.desc, mac_address);
+ return -ENONET;
+ }
+
+ netdev_dbg(netdev,
+ "registered 1 RX queue with hypervisor (single-queue mode)\n");
+ return 0;
+}
+
static int ibmveth_open(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- u64 mac_address;
+ u64 mac_address = ether_addr_to_u64(netdev->dev_addr);
int rxq_entries = 1;
- unsigned long lpar_rc;
int rc;
- union ibmveth_buf_desc rxq_desc;
int i;
netdev_dbg(netdev, "open starting\n");
@@ -1309,37 +1407,23 @@ static int ibmveth_open(struct net_device *netdev)
if (rc)
goto out_free_filter_list;
- mac_address = ether_addr_to_u64(netdev->dev_addr);
-
- rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
- adapter->rx_queue[0].queue_len;
- rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
-
- adapter->queue_irq[0] = netdev->irq;
- ibmveth_disable_irq(adapter, 0);
-
- lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
-
- if (lpar_rc != H_SUCCESS) {
- netdev_err(netdev, "h_register_logical_lan failed with %ld\n",
- lpar_rc);
- netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
- "desc:0x%llx MAC:0x%llx\n",
- adapter->buffer_list_dma[0],
- adapter->filter_list_dma,
- rxq_desc.desc,
- mac_address);
- rc = -ENONET;
+ rc = ibmveth_alloc_buffer_pools(adapter);
+ if (rc)
goto out_free_queue_mem;
- }
- rc = ibmveth_alloc_buffer_pools(adapter);
+ rc = ibmveth_register_rx_queues(adapter, mac_address);
if (rc)
- goto out_unregister_lan;
+ goto out_free_buffer_pools;
+
+ rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
+ if (rc) {
+ netdev_err(netdev, "failed to set number of rx queues\n");
+ goto out_unregister_queues;
+ }
rc = ibmveth_setup_rx_interrupts(adapter);
if (rc)
- goto out_unregister_lan;
+ goto out_free_all_queues; /* setup already disposed IRQs */
netdev_dbg(netdev, "initial replenish cycle\n");
ibmveth_schedule_rx_queue(adapter, 0);
@@ -1357,10 +1441,12 @@ static int ibmveth_open(struct net_device *netdev)
out_cleanup_rx_interrupts:
ibmveth_cleanup_rx_interrupts(adapter);
-out_unregister_lan:
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+ goto out_free_all_queues; /* cleanup already disposed IRQs */
+out_unregister_queues:
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+out_free_all_queues:
+ ibmveth_free_all_queues(adapter);
+out_free_buffer_pools:
ibmveth_free_buffer_pools(adapter);
out_free_queue_mem:
ibmveth_cleanup_rx_resources(adapter);
@@ -1373,7 +1459,6 @@ static int ibmveth_open(struct net_device *netdev)
static int ibmveth_close(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- long lpar_rc;
/* Gate on opened, not IFF_UP: pool_store/change_mtu close+open can
* leave IFF_UP set after a failed reopen.
@@ -1395,15 +1480,7 @@ static int ibmveth_close(struct net_device *netdev)
synchronize_net();
ibmveth_update_rx_no_buffer(adapter);
- /* Full LAN teardown (subordinates arrive with register helpers). */
- do {
- lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
- } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
- if (lpar_rc != H_SUCCESS) {
- netdev_err(adapter->netdev,
- "h_free_logical_lan failed with %lx, continuing\n",
- lpar_rc);
- }
+ ibmveth_free_all_queues(adapter);
/* Free TX LTBs after quiesce and after H_FREE_LOGICAL_LAN so xmit
* cannot touch unmapped bounce buffers while the LAN is live.
*/
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index bf5dd5703773..fae3473cc498 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -268,6 +268,17 @@ static int pool_active[] = { 1, 1, 0, 0, 1};
#define IBM_VETH_INVALID_MAP ((u16)0xffff)
+struct ibmveth_hcall_stats {
+ u64 reg_lan_queue; /* H_REG_LOGICAL_LAN_QUEUE */
+ u64 reg_lan; /* H_REGISTER_LOGICAL_LAN */
+ u64 add_bufs_queue; /* H_ADD_LOGICAL_LAN_BUFFERS_QUEUE */
+ u64 add_bufs; /* H_ADD_LOGICAL_LAN_BUFFERS */
+ u64 add_buf; /* H_ADD_LOGICAL_LAN_BUFFER */
+ u64 free_lan_queue; /* H_FREE_LOGICAL_LAN_QUEUE */
+ u64 free_lan; /* H_FREE_LOGICAL_LAN */
+ u64 send_lan; /* H_SEND_LOGICAL_LAN */
+};
+
struct ibmveth_buff_pool {
u32 size;
u32 index;
@@ -335,6 +346,9 @@ struct ibmveth_adapter {
u64 tx_send_failed;
u64 tx_large_packets;
u64 rx_large_packets;
+
+ /* Hypercall statistics */
+ struct ibmveth_hcall_stats hcall_stats;
/* Ethtool settings */
u8 duplex;
u32 speed;
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 08/15] ibmveth: Add queue-aware RX buffer submit helper for MQ
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (6 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 07/15] ibmveth: Add RX queue register helpers for MQ Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 09/15] ibmveth: Harden RX poll path with helpers Mingming Cao
` (6 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=true, Size: 33104 bytes --]
Replenish is the last open-path hypervisor call that still needs
per-queue awareness before MQ is turned on. Today
ibmveth_replenish_buffer_pool() calls h_add_logical_lan_buffer() or
h_add_logical_lan_buffers() directly; MQ posts via
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[].
Add ibmveth_add_logical_lan_buffers() to pick the hcall:
multi_queue uses h_add_logical_lan_buffers_queue() (up to 12 buffers;
IOBAs packed two per word: even indices in the high 32 bits, odd in
the low); legacy uses the existing single- and multi-buffer hcalls.
Count add_buf/add_bufs/add_bufs_queue in hcall_stats.
Thread queue_index through the RX helpers used by poll and replenish,
and update open/poll/netpoll callers so arity stays consistent. Add
per-queue replenish_lock so later concurrent NAPI/resize paths can
serialize buffer posting (producer and harvest/remove consumer).
Until MQ enablement, callers still pass queue 0 and legacy hcalls
remain the live path.
H_FUNCTION handling is split:
- multi_queue: schedule adapter reset after dropping replenish_lock
(do not printk under the lock - netconsole can re-enter replenish)
- legacy multi-buffer LPM fallback: set rx_buffers_per_hcall = 1 and
break so the next replenish re-samples batch as 1
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- On MQ buffer-add H_FUNCTION: schedule adapter reset after dropping
replenish_lock (v4 logged/broke with no recovery; can permanently dry
the pool)
- Move replenish fail logging / reset scheduling out from under
replenish_lock so netconsole cannot deadlock re-entering replenish
- Serialize harvest/remove with per-queue replenish_lock (netpoll
replenish vs NAPI consumer; v4 locked producer only)
- Fail logs use real wrapper names: h_add_logical_lan_buffers[_queue] /
h_add_logical_lan_buffer (v4 interpolated broken lan[_queue] strings)
- Document replenish_lock + no-printk-under-lock for netconsole (first
lock use); outcomes enum + ibmveth_replenish_fail defined here
- Defer adapter-global counter atomics and irqsave critical-section
shorten to cover follow-up
- Bad queue_index poll path: napi_complete before return lands with
poll harden (not claimed fully here)
- Keep pool active/size/threshold across free_buffer_pool (probe/sysfs
geometry); only clear runtime allocations + available (ifdown/up
reopen must still see active pools)
- get_buffer: use correlator_valid (drop WARN_ON; keep schedule_work
until poll skip owns reset)
- Introduce ibmveth_rxq_correlator_valid / ibmveth_rxq_advance at first
remove/harvest use; init replenish_lock in remove_buffer KUnit
- On RESET_MAP/RESET_MQ, stop remaining pool walks (goto unlock)
Changes in v4:
- Introduce queue-aware replenish/poll helpers with their first callers
in the same patch; do not leave a 2-arg replenish call ahead of the
signature change.
- Restore the pre-MQ LPM H_FUNCTION break instead of continue; do not
loop forever on a stale local batch size.
- Fold per-queue replenish_lock into this patch.
- Update kdoc for MQ parameters on remove_buffer_from_pool /
rxq_harvest_buffer.
drivers/net/ethernet/ibm/ibmveth.c | 523 +++++++++++++++++++++--------
drivers/net/ethernet/ibm/ibmveth.h | 6 +-
2 files changed, 387 insertions(+), 142 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 8519cad50322..58a639a962a6 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -30,6 +30,7 @@
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/slab.h>
+#include <linux/spinlock.h>
#include <asm/hvcall.h>
#include <linux/atomic.h>
#include <asm/vio.h>
@@ -101,49 +102,58 @@ static struct ibmveth_stat ibmveth_stats[] = {
};
/* simple methods of getting data from the current rxq entry */
-static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter)
+static inline u32 ibmveth_rxq_flags(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
return be32_to_cpu(rxq->queue_addr[rxq->index].flags_off);
}
-static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_toggle(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return (ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_TOGGLE) >>
- IBMVETH_RXQ_TOGGLE_SHIFT;
+ return (ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_TOGGLE) >>
+ IBMVETH_RXQ_TOGGLE_SHIFT;
}
-static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_pending_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_toggle(adapter) == adapter->rx_queue[0].toggle;
+ return ibmveth_rxq_toggle(adapter, queue_index) ==
+ adapter->rx_queue[queue_index].toggle;
}
-static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_buffer_valid(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_VALID;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_VALID;
}
-static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_offset(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_OFF_MASK;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_OFF_MASK;
}
-static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_large_packet(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_LRG_PKT;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_LRG_PKT;
}
-static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_frame_length(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
return be32_to_cpu(rxq->queue_addr[rxq->index].length);
}
-static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter)
+static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- return ibmveth_rxq_flags(adapter) & IBMVETH_RXQ_CSUM_GOOD;
+ return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_CSUM_GOOD;
}
static unsigned int ibmveth_real_max_tx_queues(void)
@@ -262,6 +272,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
adapter->rx_queue[i].index = 0;
adapter->rx_queue[i].num_slots = rxq_entries;
adapter->rx_queue[i].toggle = 1;
+ spin_lock_init(&adapter->rx_queue[i].replenish_lock);
netdev_dbg(netdev, "queue %d: buffer_list @ 0x%p (DMA: 0x%llx), rx_queue @ 0x%p (DMA: 0x%llx), %llu entries\n",
i, adapter->buffer_list_addr[i],
@@ -696,11 +707,92 @@ static inline void ibmveth_flush_buffer(void *addr, unsigned long length)
asm("dcbf %0,%1,1" :: "b" (addr), "r" (offset));
}
-/* replenish the buffers for a pool. note that we don't need to
- * skb_reserve these since they are used for incoming...
+/**
+ * ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor
+ * @adapter: ibmveth adapter structure
+ * @descs: array of buffer descriptors to add
+ * @filled: number of valid descriptors in the array
+ * @buff_size: size of each buffer (multi-queue mode only)
+ * @queue_index: RX queue index
+ *
+ * Return: hypervisor return code
*/
-static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
- struct ibmveth_buff_pool *pool)
+static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
+ union ibmveth_buf_desc *descs,
+ int filled,
+ unsigned long buff_size,
+ int queue_index)
+{
+ struct vio_dev *vdev = adapter->vdev;
+ unsigned long rc;
+
+ if (adapter->multi_queue) {
+ unsigned long buffersznum = (buff_size << 32) | filled;
+ unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL / 2] = {0};
+ unsigned long handle = adapter->queue_handle[queue_index];
+ int i;
+
+ /* Pack descriptor addresses into ioba pairs.
+ * Each ioba holds two 32-bit addresses packed into 64 bits:
+ * - Even descriptors (0,2,4...) go in high 32 bits
+ * - Odd descriptors (1,3,5...) go in low 32 bits
+ */
+ for (i = 0; i < filled && i < IBMVETH_MAX_RX_PER_HCALL; i++) {
+ int pair_idx = i / 2;
+ int is_high = (i % 2 == 0);
+
+ if (is_high)
+ ioba[pair_idx] = (unsigned long)
+ descs[i].fields.address << 32;
+ else
+ ioba[pair_idx] |= descs[i].fields.address;
+ }
+
+ rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
+ handle,
+ buffersznum,
+ ioba[0], ioba[1], ioba[2],
+ ioba[3], ioba[4], ioba[5]);
+ adapter->hcall_stats.add_bufs_queue++;
+ } else if (filled == 1) {
+ rc = h_add_logical_lan_buffer(vdev->unit_address,
+ descs[0].desc);
+ adapter->hcall_stats.add_buf++;
+ } else {
+ rc = h_add_logical_lan_buffers(vdev->unit_address,
+ descs[0].desc, descs[1].desc,
+ descs[2].desc, descs[3].desc,
+ descs[4].desc, descs[5].desc,
+ descs[6].desc, descs[7].desc);
+ adapter->hcall_stats.add_bufs++;
+ }
+
+ return rc;
+}
+
+/* Outcomes for ibmveth_replenish_buffer_pool(); logged after unlock. */
+enum {
+ IBMVETH_REPLENISH_OK = 0,
+ IBMVETH_REPLENISH_RESET_MAP,
+ IBMVETH_REPLENISH_RESET_MQ,
+ IBMVETH_REPLENISH_HCALL_FAIL,
+ IBMVETH_REPLENISH_BATCH_FALLBACK,
+};
+
+struct ibmveth_replenish_fail {
+ unsigned long lpar_rc;
+ u32 filled;
+ u32 batch;
+};
+
+/* Replenish the buffers for a pool.
+ * Caller must hold the per-queue replenish_lock. Do not printk here —
+ * netconsole on the same device can re-enter replenish_task.
+ */
+static int ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
+ struct ibmveth_buff_pool *pool,
+ int queue_index,
+ struct ibmveth_replenish_fail *fail)
{
union ibmveth_buf_desc descs[IBMVETH_MAX_RX_PER_HCALL] = {0};
u32 remaining = pool->size - atomic_read(&pool->available);
@@ -712,6 +804,7 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
dma_addr_t dma_addr;
struct device *dev;
u32 index;
+ int outcome = IBMVETH_REPLENISH_OK;
vdev = adapter->vdev;
dev = &vdev->dev;
@@ -726,12 +819,9 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
/* Fill a batch of descriptors */
for (filled = 0; filled < min(remaining, batch); filled++) {
index = pool->free_map[free_index];
- if (WARN_ON(index == IBM_VETH_INVALID_MAP)) {
+ if (index == IBM_VETH_INVALID_MAP) {
adapter->replenish_add_buff_failure++;
- netdev_info(adapter->netdev,
- "Invalid map index %u, reset\n",
- index);
- schedule_work(&adapter->work);
+ outcome = IBMVETH_REPLENISH_RESET_MAP;
break;
}
@@ -783,28 +873,21 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
free_index = 0;
}
+ if (outcome != IBMVETH_REPLENISH_OK)
+ break;
+
if (!filled)
break;
- /* single buffer case*/
- if (filled == 1)
- lpar_rc = h_add_logical_lan_buffer(vdev->unit_address,
- descs[0].desc);
- else
- /* Multi-buffer hcall */
- lpar_rc = h_add_logical_lan_buffers(vdev->unit_address,
- descs[0].desc,
- descs[1].desc,
- descs[2].desc,
- descs[3].desc,
- descs[4].desc,
- descs[5].desc,
- descs[6].desc,
- descs[7].desc);
+ lpar_rc = ibmveth_add_logical_lan_buffers(adapter, descs,
+ filled,
+ pool->buff_size,
+ queue_index);
+
if (lpar_rc != H_SUCCESS) {
- dev_warn_ratelimited(dev,
- "RX h_add_logical_lan failed: filled=%u, rc=%lu, batch=%u\n",
- filled, lpar_rc, batch);
+ fail->lpar_rc = lpar_rc;
+ fail->filled = filled;
+ fail->batch = batch;
goto hcall_failure;
}
@@ -844,30 +927,35 @@ static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
}
adapter->replenish_add_buff_failure += filled;
- /*
- * If multi rx buffers hcall is no longer supported by FW
- * e.g. in the case of Live Partition Migration
- */
- if (batch > 1 && lpar_rc == H_FUNCTION) {
- /*
- * Instead of retry submit single buffer individually
- * here just set the max rx buffer per hcall to 1
- * buffers will be respleshed next time
- * when ibmveth_replenish_buffer_pool() is called again
- * with single-buffer case
- */
- netdev_info(adapter->netdev,
- "RX Multi buffers not supported by FW, rc=%lu\n",
- lpar_rc);
- adapter->rx_buffers_per_hcall = 1;
- netdev_info(adapter->netdev,
- "Next rx replesh will fall back to single-buffer hcall\n");
+ if (lpar_rc == H_FUNCTION) {
+ if (adapter->multi_queue) {
+ /*
+ * LPM / firmware may drop MQ buffer hcalls.
+ * Schedule reset so we do not sit forever in
+ * no-buffer with the link still up.
+ */
+ outcome = IBMVETH_REPLENISH_RESET_MQ;
+ } else if (batch > 1) {
+ /*
+ * Live Partition Migration may drop multi-
+ * buffer support. Fall back to single-buffer
+ * on the next replenish; do not continue with
+ * a stale local batch size (infinite loop).
+ */
+ adapter->rx_buffers_per_hcall = 1;
+ outcome = IBMVETH_REPLENISH_BATCH_FALLBACK;
+ } else {
+ outcome = IBMVETH_REPLENISH_HCALL_FAIL;
+ }
+ } else {
+ outcome = IBMVETH_REPLENISH_HCALL_FAIL;
}
break;
}
mb();
atomic_add(buffers_added, &(pool->available));
+ return outcome;
}
/*
@@ -883,21 +971,85 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
}
/* replenish routine */
-static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
+static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- int i;
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
+ struct ibmveth_replenish_fail fail = {};
+ unsigned long flags;
+ int i, rc;
+ int need_reset = 0;
+ int batch_fallback = 0;
+ int hcall_fail = 0;
+
+ if (queue_index >= adapter->num_rx_queues) {
+ netdev_dbg(adapter->netdev,
+ "Skipping replenish for freed queue %d (num_queues=%d)\n",
+ queue_index, adapter->num_rx_queues);
+ return;
+ }
adapter->replenish_task_cycles++;
- for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
- struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
+ spin_lock_irqsave(&rxq->replenish_lock, flags);
- if (pool->active &&
- (atomic_read(&pool->available) < pool->threshold))
- ibmveth_replenish_buffer_pool(adapter, pool);
+ for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[queue_index][i];
+
+ if (pool->active && pool->free_map &&
+ (atomic_read(&pool->available) < pool->threshold)) {
+ rc = ibmveth_replenish_buffer_pool(adapter, pool,
+ queue_index, &fail);
+ switch (rc) {
+ case IBMVETH_REPLENISH_RESET_MAP:
+ case IBMVETH_REPLENISH_RESET_MQ:
+ need_reset = rc;
+ goto out_unlock;
+ case IBMVETH_REPLENISH_BATCH_FALLBACK:
+ batch_fallback = 1;
+ break;
+ case IBMVETH_REPLENISH_HCALL_FAIL:
+ hcall_fail = 1;
+ break;
+ default:
+ break;
+ }
+ }
}
+out_unlock:
ibmveth_update_rx_no_buffer(adapter);
+
+ spin_unlock_irqrestore(&rxq->replenish_lock, flags);
+
+ /* Log and schedule reset only after dropping replenish_lock. */
+ if (need_reset == IBMVETH_REPLENISH_RESET_MAP) {
+ netdev_info(adapter->netdev,
+ "Invalid RX free_map entry on queue %d, reset\n",
+ queue_index);
+ schedule_work(&adapter->work);
+ } else if (need_reset == IBMVETH_REPLENISH_RESET_MQ) {
+ dev_err_ratelimited(&adapter->netdev->dev,
+ "MQ buffer add H_FUNCTION (q=%d, batch=%u), reset\n",
+ queue_index, fail.batch);
+ schedule_work(&adapter->work);
+ }
+
+ if (batch_fallback)
+ dev_warn_ratelimited(&adapter->netdev->dev,
+ "Legacy batch add H_FUNCTION (batch=%u), fallback\n",
+ fail.batch);
+
+ if (hcall_fail)
+ dev_warn_ratelimited(&adapter->netdev->dev,
+ "RX %s failed: filled=%u, rc=%lu, batch=%u\n",
+ adapter->multi_queue ?
+ "h_add_logical_lan_buffers_queue" :
+ (fail.batch == 1 ?
+ "h_add_logical_lan_buffer" :
+ "h_add_logical_lan_buffers"),
+ fail.filled, fail.lpar_rc, fail.batch);
}
/* empty and free ana buffer pool - also used to do cleanup in error paths */
@@ -932,6 +1084,14 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
kfree(pool->skbuff);
pool->skbuff = NULL;
}
+
+ /*
+ * Keep probe/sysfs geometry (active, size, buff_size, threshold).
+ * Clearing active here was a v3 ifdown/up regression: open skips
+ * !active pools, so reopen posted no RX buffers (TX OK, ARP/RX
+ * dead) at any queue count, including RX=8 with no -L.
+ */
+ atomic_set(&pool->available, 0);
}
/**
@@ -1070,35 +1230,75 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
adapter->num_rx_queues);
}
+static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
+ int queue_index, u64 correlator)
+{
+ unsigned int pool = correlator >> 32;
+ unsigned int index = correlator & 0xffffffffUL;
+ struct ibmveth_buff_pool *bpool;
+
+ if (pool >= IBMVETH_NUM_BUFF_POOLS)
+ return false;
+
+ bpool = &adapter->rx_buff_pool[queue_index][pool];
+
+ /* init_buffer_pool() sets size for inactive pools; free_buffer_pool()
+ * clears skbuff but used to leave size/active set. Require a live
+ * pool with allocated arrays before indexing.
+ */
+ if (!bpool->active || !bpool->skbuff || !bpool->free_map)
+ return false;
+
+ return index < bpool->size;
+}
+
+static void ibmveth_rxq_advance(struct ibmveth_rx_q *rxq)
+{
+ if (++rxq->index == rxq->num_slots) {
+ rxq->index = 0;
+ rxq->toggle = !rxq->toggle;
+ }
+}
+
/**
* ibmveth_remove_buffer_from_pool - remove a buffer from a pool
* @adapter: adapter instance
* @correlator: identifies pool and index
+ * @queue_index: RX queue index (0..num_rx_queues-1)
* @reuse: whether to reuse buffer
*
+ * Context: may run concurrently with netpoll replenish_task on the same
+ * queue; takes per-queue replenish_lock to serialize free_map /
+ * producer_index / available against the producer.
+ *
* Return:
* * %0 - success
* * %-EINVAL - correlator maps to pool or index out of range
* * %-EFAULT - pool and index map to null skb
*/
static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
- u64 correlator, bool reuse)
+ u64 correlator, int queue_index,
+ bool reuse)
{
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
unsigned int free_index;
struct sk_buff *skb;
+ unsigned long flags;
+ int rc = 0;
- if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
- schedule_work(&adapter->work);
- return -EINVAL;
+ spin_lock_irqsave(&rxq->replenish_lock, flags);
+
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator)) {
+ rc = -EINVAL;
+ goto out_unlock;
}
- skb = adapter->rx_buff_pool[0][pool].skbuff[index];
- if (WARN_ON(!skb)) {
- schedule_work(&adapter->work);
- return -EFAULT;
+ skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
+ if (!skb) {
+ rc = -EFAULT;
+ goto out_unlock;
}
/* if we are going to reuse the buffer then keep the pointers around
@@ -1109,75 +1309,88 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
/* remove the skb pointer to mark free. actual freeing is done
* by upper level networking after gro_receive
*/
- adapter->rx_buff_pool[0][pool].skbuff[index] = NULL;
+ struct ibmveth_buff_pool *bpool =
+ &adapter->rx_buff_pool[queue_index][pool];
+
+ bpool->skbuff[index] = NULL;
dma_unmap_single(&adapter->vdev->dev,
- adapter->rx_buff_pool[0][pool].dma_addr[index],
- adapter->rx_buff_pool[0][pool].buff_size,
+ bpool->dma_addr[index],
+ bpool->buff_size,
DMA_FROM_DEVICE);
}
- free_index = adapter->rx_buff_pool[0][pool].producer_index;
- adapter->rx_buff_pool[0][pool].producer_index++;
- if (adapter->rx_buff_pool[0][pool].producer_index >=
- adapter->rx_buff_pool[0][pool].size)
- adapter->rx_buff_pool[0][pool].producer_index = 0;
- adapter->rx_buff_pool[0][pool].free_map[free_index] = index;
+ free_index = adapter->rx_buff_pool[queue_index][pool].producer_index;
+ adapter->rx_buff_pool[queue_index][pool].producer_index++;
+ if (adapter->rx_buff_pool[queue_index][pool].producer_index >=
+ adapter->rx_buff_pool[queue_index][pool].size)
+ adapter->rx_buff_pool[queue_index][pool].producer_index = 0;
+ adapter->rx_buff_pool[queue_index][pool].free_map[free_index] = index;
mb();
- atomic_dec(&adapter->rx_buff_pool[0][pool].available);
+ atomic_dec(&adapter->rx_buff_pool[queue_index][pool].available);
- return 0;
+out_unlock:
+ spin_unlock_irqrestore(&rxq->replenish_lock, flags);
+ return rc;
}
/* get the current buffer on the rx queue */
-static inline struct sk_buff *ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter)
+static inline struct sk_buff *
+ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
u64 correlator = rxq->queue_addr[rxq->index].correlator;
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
- if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
- WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator)) {
schedule_work(&adapter->work);
return NULL;
}
- return adapter->rx_buff_pool[0][pool].skbuff[index];
+ return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
}
/**
* ibmveth_rxq_harvest_buffer - Harvest buffer from pool
*
* @adapter: pointer to adapter
+ * @queue_index: RX queue index to harvest from
* @reuse: whether to reuse buffer
*
* Context: called from ibmveth_poll
*
+ * On a bad correlator (-EINVAL/-EFAULT) the ring is still advanced so poll
+ * cannot spin forever on one slot. The error is still returned: callers must
+ * not treat it as a successful take from the pool (especially reuse=false,
+ * which would hand the SKB to the stack while it remains pool-owned).
+ *
* Return:
- * * %0 - success
- * * other - non-zero return from ibmveth_remove_buffer_from_pool
+ * * %0 - buffer removed from pool (or marked for reuse) and ring advanced
+ * * other - non-zero return from ibmveth_remove_buffer_from_pool; ring has
+ * still been advanced for -EINVAL/-EFAULT
*/
static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
- bool reuse)
+ int queue_index, bool reuse)
{
+ struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
u64 cor;
int rc;
- struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
-
cor = rxq->queue_addr[rxq->index].correlator;
- rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
- if (unlikely(rc))
+ rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
+ if (unlikely(rc)) {
+ /* Skip a corrupt slot without claiming pool ownership. */
+ if (rc == -EINVAL || rc == -EFAULT)
+ ibmveth_rxq_advance(rxq);
return rc;
-
- if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
- adapter->rx_queue[0].index = 0;
- adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
}
+ ibmveth_rxq_advance(rxq);
+
return 0;
}
@@ -2127,34 +2340,41 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
static int ibmveth_poll(struct napi_struct *napi, int budget)
{
- struct ibmveth_adapter *adapter =
- container_of(napi, struct ibmveth_adapter, napi[0]);
- struct net_device *netdev = adapter->netdev;
+ struct net_device *netdev = napi->dev;
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
int frames_processed = 0;
- int rc;
+ int queue_index, rc;
u16 mss = 0;
+ queue_index = napi - adapter->napi;
+
restart_poll:
while (frames_processed < budget) {
- if (!ibmveth_rxq_pending_buffer(adapter))
+ if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
break;
smp_rmb();
- if (!ibmveth_rxq_buffer_valid(adapter)) {
+ if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
wmb(); /* suggested by larson1 */
adapter->rx_invalid_buffer++;
netdev_dbg(netdev, "recycling invalid buffer\n");
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+ rc = ibmveth_rxq_harvest_buffer(adapter,
+ queue_index, true);
+ if (unlikely(rc))
break;
} else {
struct sk_buff *skb, *new_skb;
- int length = ibmveth_rxq_frame_length(adapter);
- int offset = ibmveth_rxq_frame_offset(adapter);
- int csum_good = ibmveth_rxq_csum_good(adapter);
- int lrg_pkt = ibmveth_rxq_large_packet(adapter);
+ int length = ibmveth_rxq_frame_length(adapter,
+ queue_index);
+ int offset = ibmveth_rxq_frame_offset(adapter,
+ queue_index);
+ int csum_good = ibmveth_rxq_csum_good(adapter,
+ queue_index);
+ int lrg_pkt = ibmveth_rxq_large_packet(adapter,
+ queue_index);
__sum16 iph_check = 0;
- skb = ibmveth_rxq_get_buffer(adapter);
+ skb = ibmveth_rxq_get_buffer(adapter, queue_index);
if (unlikely(!skb))
break;
@@ -2179,12 +2399,18 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
length);
if (rx_flush)
ibmveth_flush_buffer(skb->data,
- length + offset);
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, true)))
+ length + offset);
+ rc = ibmveth_rxq_harvest_buffer(adapter,
+ queue_index,
+ true);
+ if (unlikely(rc))
break;
skb = new_skb;
} else {
- if (unlikely(ibmveth_rxq_harvest_buffer(adapter, false)))
+ rc = ibmveth_rxq_harvest_buffer(adapter,
+ queue_index,
+ false);
+ if (unlikely(rc))
break;
skb_reserve(skb, offset);
}
@@ -2220,7 +2446,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
}
}
- ibmveth_replenish_task(adapter);
+ ibmveth_replenish_task(adapter, queue_index);
if (frames_processed == budget)
goto out;
@@ -2231,14 +2457,18 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
/* We think we are done - reenable interrupts,
* then check once more to make sure we are done.
*/
- rc = ibmveth_enable_irq(adapter, 0);
- if (WARN_ON(rc)) {
+ rc = ibmveth_enable_irq(adapter, queue_index);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d (rc=%d), scheduling reset\n",
+ queue_index, rc);
schedule_work(&adapter->work);
goto out;
}
- if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
- rc = ibmveth_disable_irq(adapter, 0);
+ if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
+ napi_schedule(napi)) {
+ rc = ibmveth_disable_irq(adapter, queue_index);
WARN_ON(rc);
goto restart_poll;
}
@@ -2369,7 +2599,7 @@ static void ibmveth_poll_controller(struct net_device *dev)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
- ibmveth_replenish_task(adapter);
+ ibmveth_replenish_task(adapter, 0);
ibmveth_schedule_rx_queue(adapter, 0);
}
#endif
@@ -2568,7 +2798,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
if (ret == H_SUCCESS &&
(ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
- adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
netdev_dbg(netdev,
"RX Multi-buffer hcall supported by FW, batch set to %u\n",
adapter->rx_buffers_per_hcall);
@@ -2889,8 +3119,7 @@ static void ibmveth_reset_kunit(struct work_struct *w)
* @test: pointer to kunit structure
*
* Tests the error returns from ibmveth_remove_buffer_from_pool.
- * ibmveth_remove_buffer_from_pool also calls WARN_ON, so dmesg should be
- * checked to see that these warnings happened.
+ * Bad correlators return -EINVAL/-EFAULT (no WARN_ON).
*
* Return: void
*/
@@ -2904,6 +3133,8 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
INIT_WORK(&adapter->work, ibmveth_reset_kunit);
+ spin_lock_init(&adapter->rx_queue[0].replenish_lock);
+
/* Set sane values for buffer pools */
for (int i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
@@ -2915,17 +3146,29 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pool->skbuff);
correlator = ((u64)IBMVETH_NUM_BUFF_POOLS << 32) | 0;
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter,
+ correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter,
+ correlator, 0, true));
correlator = ((u64)0 << 32) | adapter->rx_buff_pool[0][0].size;
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EINVAL, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter,
+ correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EINVAL,
+ ibmveth_remove_buffer_from_pool(adapter,
+ correlator, 0, true));
correlator = (u64)0 | 0;
pool->skbuff[0] = NULL;
- KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, false));
- KUNIT_EXPECT_EQ(test, -EFAULT, ibmveth_remove_buffer_from_pool(adapter, correlator, true));
+ KUNIT_EXPECT_EQ(test, -EFAULT,
+ ibmveth_remove_buffer_from_pool(adapter,
+ correlator, 0, false));
+ KUNIT_EXPECT_EQ(test, -EFAULT,
+ ibmveth_remove_buffer_from_pool(adapter,
+ correlator, 0, true));
flush_work(&adapter->work);
}
@@ -2934,9 +3177,7 @@ static void ibmveth_remove_buffer_from_pool_test(struct kunit *test)
* ibmveth_rxq_get_buffer_test - unit test for ibmveth_rxq_get_buffer
* @test: pointer to kunit structure
*
- * Tests ibmveth_rxq_get_buffer. ibmveth_rxq_get_buffer also calls WARN_ON for
- * the NULL returns, so dmesg should be checked to see that these warnings
- * happened.
+ * Tests ibmveth_rxq_get_buffer invalid correlator returns NULL without WARN.
*
* Return: void
*/
@@ -2970,15 +3211,15 @@ static void ibmveth_rxq_get_buffer_test(struct kunit *test)
adapter->rx_queue[0].queue_addr[0].correlator =
(u64)IBMVETH_NUM_BUFF_POOLS << 32 | 0;
- KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
adapter->rx_queue[0].queue_addr[0].correlator =
(u64)0 << 32 | adapter->rx_buff_pool[0][0].size;
- KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, NULL, ibmveth_rxq_get_buffer(adapter, 0));
pool->skbuff[0] = skb;
adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
- KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter));
+ KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter, 0));
flush_work(&adapter->work);
}
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index fae3473cc498..d02444d5b3b8 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -14,6 +14,8 @@
#ifndef _IBMVETH_H
#define _IBMVETH_H
+#include <linux/spinlock_types.h>
+
/* constants for H_MULTICAST_CTRL */
#define IbmVethMcastReceptionModifyBit 0x80000UL
#define IbmVethMcastReceptionEnableBit 0x20000UL
@@ -259,7 +261,8 @@ static inline long h_illan_attributes(unsigned long unit_address,
#define IBMVETH_DEFAULT_QUEUES 8U
#define IBMVETH_MAX_RX_QUEUES 1U
#define IBMVETH_DEFAULT_RX_QUEUES 1U
-#define IBMVETH_MAX_RX_PER_HCALL 8U
+#define IBMVETH_MAX_RX_REGULAR 8U
+#define IBMVETH_MAX_RX_PER_HCALL 12U
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
static int pool_count[] = { 256, 512, 256, 256, 256 };
@@ -301,6 +304,7 @@ struct ibmveth_rx_q {
dma_addr_t queue_dma;
u32 queue_len;
struct ibmveth_rx_q_entry *queue_addr;
+ spinlock_t replenish_lock; /* per-queue buffer replenish */
};
struct ibmveth_adapter {
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 09/15] ibmveth: Harden RX poll path with helpers
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (7 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 08/15] ibmveth: Add queue-aware RX buffer submit helper " Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
` (5 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=true, Size: 11536 bytes --]
ibmveth_poll() must handle several distinct skip/fail outcomes on each
RX slot, not only the happy path:
- interface close / napi_disable must not re-arm PHYP delivery
- bad correlators and harvest errors must not look like successful GRO
- oversize or wrap-around offset+length must not skb_put() past the
buffer
- an out-of-range queue index must napi_complete_done rather than
fall through and keep polling
Doing all of that inline turns the NAPI callback into a deeply nested
switchyard. Split each outcome into a small helper so ibmveth_poll()
stays a thin budget loop:
ibmveth_poll_stopping()
ibmveth_poll_harvest_slot() / recycle_invalid / skip_bad_correlator
ibmveth_poll_drop_oversize()
ibmveth_poll_deliver_frame()
ibmveth_poll_bump_invalid()
Drop schedule_work from rxq_get_buffer() (skip_bad_correlator owns
reset escalation). ibmveth_poll_stopping() ensures close/napi_disable
does not re-arm PHYP. Runtime is still single-queue: helpers are defined
and called from the existing SQ poll path in this same patch (first
use). The next patch turns on MQ and reuses this loop; subordinate
register helpers stay there.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Wrap poll_stopping return after complete so it stays under 80 cols
- On poll_stopping after the budget loop: return < budget after
napi_complete_done (was frames_processed-1 even when under budget)
- New in v5: peel SQ poll harden + helpers before MQ enable so tip P10
stays bring-up focused (mailed v4 09/14 MQ enable -> tip P10; also
14->15). Kitchen-sink / enable-path poll fixes land here on the live
single-queue path first.
- After napi_complete_done, check poll_stopping again before enable_irq
(avoid re-arm while resize/close waits on napi_disable)
drivers/net/ethernet/ibm/ibmveth.c | 283 ++++++++++++++++++++---------
1 file changed, 193 insertions(+), 90 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 58a639a962a6..86299c62d4ec 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1346,10 +1346,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
unsigned int pool = correlator >> 32;
unsigned int index = correlator & 0xffffffffUL;
- if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator)) {
- schedule_work(&adapter->work);
+ if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
return NULL;
- }
return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
}
@@ -2338,125 +2336,230 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
}
}
+static void ibmveth_poll_bump_invalid(struct ibmveth_adapter *adapter,
+ int queue_index)
+{
+ adapter->rx_invalid_buffer++;
+}
+
+static bool ibmveth_poll_stopping(struct net_device *netdev,
+ struct napi_struct *napi)
+{
+ return !netif_running(netdev) || napi_disable_pending(napi);
+}
+
+static bool ibmveth_poll_harvest_slot(struct ibmveth_adapter *adapter,
+ int queue_index, bool reuse)
+{
+ int rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, reuse);
+
+ return !rc || rc == -EINVAL || rc == -EFAULT;
+}
+
+static bool ibmveth_poll_recycle_invalid(struct net_device *netdev,
+ struct ibmveth_adapter *adapter,
+ int queue_index)
+{
+ netdev_dbg(netdev, "recycling invalid buffer\n");
+ ibmveth_poll_bump_invalid(adapter, queue_index);
+ return ibmveth_poll_harvest_slot(adapter, queue_index, true);
+}
+
+static bool ibmveth_poll_skip_bad_correlator(struct net_device *netdev,
+ struct ibmveth_adapter *adapter,
+ int queue_index)
+{
+ if (net_ratelimit())
+ netdev_err(netdev,
+ "bad correlator on queue %d, skipping slot\n",
+ queue_index);
+ /* Residual stale slot after resize: recover via reset rather
+ * than spinning forever. Always escalate; only the log is
+ * rate-limited.
+ */
+ schedule_work(&adapter->work);
+ ibmveth_poll_bump_invalid(adapter, queue_index);
+ return ibmveth_poll_harvest_slot(adapter, queue_index, true);
+}
+
+static bool ibmveth_poll_drop_oversize(struct net_device *netdev,
+ struct ibmveth_adapter *adapter,
+ int queue_index, unsigned int off,
+ unsigned int len, unsigned int room)
+{
+ if (net_ratelimit())
+ netdev_err(netdev,
+ "RX frame %u+%u exceeds buffer %u on queue %d, dropping\n",
+ off, len, room, queue_index);
+ ibmveth_poll_bump_invalid(adapter, queue_index);
+ return ibmveth_poll_harvest_slot(adapter, queue_index, true);
+}
+
+/**
+ * ibmveth_poll_deliver_frame - Build SKB from one valid RX slot and GRO it
+ * @napi: NAPI context for this RX queue
+ * @adapter: ibmveth adapter
+ * @netdev: net_device for @adapter
+ * @queue_index: RX queue index
+ *
+ * Return: 1 frame delivered, 0 if the slot was skipped cleanly, -1 on error.
+ */
+static int ibmveth_poll_deliver_frame(struct napi_struct *napi,
+ struct ibmveth_adapter *adapter,
+ struct net_device *netdev,
+ int queue_index)
+{
+ struct sk_buff *skb, *new_skb;
+ unsigned int room, off, len;
+ int length, offset, csum_good, lrg_pkt;
+ __sum16 iph_check = 0;
+ u16 mss = 0;
+ int rc;
+
+ length = ibmveth_rxq_frame_length(adapter, queue_index);
+ offset = ibmveth_rxq_frame_offset(adapter, queue_index);
+ csum_good = ibmveth_rxq_csum_good(adapter, queue_index);
+ lrg_pkt = ibmveth_rxq_large_packet(adapter, queue_index);
+
+ skb = ibmveth_rxq_get_buffer(adapter, queue_index);
+ if (unlikely(!skb)) {
+ if (!ibmveth_poll_skip_bad_correlator(netdev, adapter,
+ queue_index))
+ return -1;
+ return 0;
+ }
+
+ room = skb_tailroom(skb);
+ off = offset;
+ len = length;
+ if (unlikely(off >= room || len > room - off)) {
+ if (!ibmveth_poll_drop_oversize(netdev, adapter, queue_index,
+ off, len, room))
+ return -1;
+ return 0;
+ }
+
+ if (lrg_pkt) {
+ __be64 *rxmss = (__be64 *)(skb->data + 8);
+
+ mss = (u16)be64_to_cpu(*rxmss);
+ }
+
+ new_skb = NULL;
+ if (length < rx_copybreak)
+ new_skb = netdev_alloc_skb(netdev, length);
+
+ if (new_skb) {
+ skb_copy_to_linear_data(new_skb, skb->data + offset, length);
+ if (rx_flush)
+ ibmveth_flush_buffer(skb->data, length + offset);
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (unlikely(rc)) {
+ kfree_skb(new_skb);
+ return -1;
+ }
+ skb = new_skb;
+ } else {
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, false);
+ if (unlikely(rc))
+ return -1;
+ skb_reserve(skb, offset);
+ }
+
+ skb_put(skb, length);
+ skb->protocol = eth_type_trans(skb, netdev);
+
+ if (skb->protocol == cpu_to_be16(ETH_P_IP))
+ iph_check = ip_hdr(skb)->check;
+
+ if ((length > netdev->mtu + ETH_HLEN) || lrg_pkt ||
+ iph_check == 0xffff) {
+ ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
+ adapter->rx_large_packets++;
+ }
+
+ if (csum_good) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ ibmveth_rx_csum_helper(skb, adapter);
+ }
+
+ napi_gro_receive(napi, skb);
+
+ netdev->stats.rx_packets++;
+ netdev->stats.rx_bytes += length;
+
+ return 1;
+}
+
static int ibmveth_poll(struct napi_struct *napi, int budget)
{
struct net_device *netdev = napi->dev;
struct ibmveth_adapter *adapter = netdev_priv(netdev);
int frames_processed = 0;
int queue_index, rc;
- u16 mss = 0;
queue_index = napi - adapter->napi;
+ if (WARN_ON(queue_index < 0 ||
+ queue_index >= adapter->num_rx_queues)) {
+ napi_complete_done(napi, 0);
+ return 0;
+ }
+
+ if (ibmveth_poll_stopping(netdev, napi)) {
+ napi_complete_done(napi, 0);
+ return 0;
+ }
+
restart_poll:
while (frames_processed < budget) {
+ if (ibmveth_poll_stopping(netdev, napi))
+ break;
+
if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
break;
smp_rmb();
if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
wmb(); /* suggested by larson1 */
- adapter->rx_invalid_buffer++;
- netdev_dbg(netdev, "recycling invalid buffer\n");
- rc = ibmveth_rxq_harvest_buffer(adapter,
- queue_index, true);
- if (unlikely(rc))
+ if (!ibmveth_poll_recycle_invalid(netdev, adapter,
+ queue_index))
break;
} else {
- struct sk_buff *skb, *new_skb;
- int length = ibmveth_rxq_frame_length(adapter,
- queue_index);
- int offset = ibmveth_rxq_frame_offset(adapter,
- queue_index);
- int csum_good = ibmveth_rxq_csum_good(adapter,
- queue_index);
- int lrg_pkt = ibmveth_rxq_large_packet(adapter,
- queue_index);
- __sum16 iph_check = 0;
-
- skb = ibmveth_rxq_get_buffer(adapter, queue_index);
- if (unlikely(!skb))
+ rc = ibmveth_poll_deliver_frame(napi, adapter, netdev,
+ queue_index);
+ if (rc < 0)
break;
-
- /* if the large packet bit is set in the rx queue
- * descriptor, the mss will be written by PHYP eight
- * bytes from the start of the rx buffer, which is
- * skb->data at this stage
- */
- if (lrg_pkt) {
- __be64 *rxmss = (__be64 *)(skb->data + 8);
-
- mss = (u16)be64_to_cpu(*rxmss);
- }
-
- new_skb = NULL;
- if (length < rx_copybreak)
- new_skb = netdev_alloc_skb(netdev, length);
-
- if (new_skb) {
- skb_copy_to_linear_data(new_skb,
- skb->data + offset,
- length);
- if (rx_flush)
- ibmveth_flush_buffer(skb->data,
- length + offset);
- rc = ibmveth_rxq_harvest_buffer(adapter,
- queue_index,
- true);
- if (unlikely(rc))
- break;
- skb = new_skb;
- } else {
- rc = ibmveth_rxq_harvest_buffer(adapter,
- queue_index,
- false);
- if (unlikely(rc))
- break;
- skb_reserve(skb, offset);
- }
-
- skb_put(skb, length);
- skb->protocol = eth_type_trans(skb, netdev);
-
- /* PHYP without PLSO support places a -1 in the ip
- * checksum for large send frames.
- */
- if (skb->protocol == cpu_to_be16(ETH_P_IP)) {
- struct iphdr *iph = (struct iphdr *)skb->data;
-
- iph_check = iph->check;
- }
-
- if ((length > netdev->mtu + ETH_HLEN) ||
- lrg_pkt || iph_check == 0xffff) {
- ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
- adapter->rx_large_packets++;
- }
-
- if (csum_good) {
- skb->ip_summed = CHECKSUM_UNNECESSARY;
- ibmveth_rx_csum_helper(skb, adapter);
- }
-
- napi_gro_receive(napi, skb); /* send it up */
-
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += length;
- frames_processed++;
+ if (rc > 0)
+ frames_processed++;
}
}
ibmveth_replenish_task(adapter, queue_index);
+ if (ibmveth_poll_stopping(netdev, napi)) {
+ napi_complete_done(napi, frames_processed);
+ /* After complete, must not return budget (NAPI resched). */
+ if (frames_processed < budget)
+ return frames_processed;
+ return budget - 1;
+ }
+
if (frames_processed == budget)
goto out;
if (!napi_complete_done(napi, frames_processed))
goto out;
- /* We think we are done - reenable interrupts,
- * then check once more to make sure we are done.
+ /*
+ * napi_disable() sets DISABLE then waits for this poll. Without a
+ * second stopping check here, enable_irq() can re-arm PHYP after
+ * resize already masked the queue — late IRQs then hit the handler
+ * after num_rx_queues was published lower (lab WARN at interrupt).
*/
+ if (ibmveth_poll_stopping(netdev, napi))
+ goto out;
+
rc = ibmveth_enable_irq(adapter, queue_index);
if (rc) {
netdev_err(netdev,
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 10/15] ibmveth: Enable multi-queue RX receive path
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (8 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 09/15] ibmveth: Harden RX poll path with helpers Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 11/15] ibmveth: Add per-queue RX and TX statistics collection Mingming Cao
` (4 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=true, Size: 35729 bytes --]
Turn on the live MQ RX path when firmware sets
IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT in H_ILLAN_ATTRIBUTES. Probe sets
multi_queue and num_rx_queues to min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES), matching the existing TX default (cap 8).
Up to IBMVETH_MAX_RX_QUEUES (16) remains available via ethtool -L.
Without the bit, behaviour stays single-queue.
Wire subordinate queues through H_REG_LOGICAL_LAN_QUEUE (with
irq_create_mapping for subordinate virqs), request_irq/napi_enable,
and PHYP enable_irq when multi_queue && num_rx_queues > 1. Queue 0
continues to use netdev->irq and is never disposed with the
subordinates.
Restate open() here (supersedes the register-helpers patch kick):
... register_rx_queues / set_real_num_rx_queues ...
replenish_task() for every live RX queue (before IRQ setup)
setup_rx_interrupts() (MQ also unmasks PHYP)
restart_rx_queue() for every live queue (schedule NAPI, or
enable_irq if prep fails)
... alloc_tx_resources / tx_start / opened ...
That path is the same for SQ and MQ. It replaces the prior
setup-then-schedule_rx_queue(0) kick and the mailed v4 MQ-only
replenish + kick_if_pending split. Close shape is unchanged.
H_FUNCTION on subordinate register is a hard open failure (no silent
single-queue fallback). Setup failure paths dispose subordinate virq
mappings so a request_irq failure after successful registration cannot
leak Linux mappings. Buffer-add H_FUNCTION latches mq_fallback so the
next open drops to single-queue (apply_mq_fallback at open entry).
On probe failure after pool kobjects were created, put them before
free_netdev(). Refresh VIO CMO entitlement after MQ probe / pool init,
sizing get_desired_dma() from queue-0 pool metadata across
num_rx_queues.
RX poll harden landed in the previous patch on the SQ poll path so this
enablement patch stays focused on MQ bring-up. Hot-path accounting
moves to the next patch (per-queue qstats).
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- RTNL/serialized readers use get_num_rx_queues() once the helper exists
(open/close/register/apply_mq_fallback/alloc/cleanup/IRQ setup)
- Interrupt: quiet IRQ_NONE on out-of-range qindex vs published live
count (scale-down residual IRQ; no WARN storm)
- Series renumber: mailed v4 09/14 MQ enable -> tip P10 (P09 peel; 14->15)
- Open: replenish all queues before setup; restart_rx_queue after setup
for SQ and MQ - replaces mailed MQ-only replenish + kick_if_pending vs
SQ schedule_rx_queue(0)
- update_rx_no_buffer(queue_index): NULL-safe, no all-queue walk under one
lock (fixes the resize/race class). Still stores adapter->rx_no_buffer
for now; per-queue slot + monotonic sum lands with qstats next
- H_FUNCTION: subordinate register fails open + stash mq_fallback;
buffer-add schedules reset + mq_fallback so next open drops to SQ
- Introduce get_num_rx_queues / publish_num_rx_queues (READ/WRITE_ONCE)
at first use; lockless IRQ/poll/replenish/interrupt bounds use the getter
- Introduce kick_rx_queue_if_pending() for pending-after-unmask; open uses
restart_rx_queue, resize calls the helper later
- resume() schedules every live RX queue (not only queue 0)
- Call out enable_irq-fail synchronize_irq and unused mac register-arg
cleanup (mailed enable-path review; code already earlier in tip)
Changes in v4:
- Fold subordinate register helpers and their review fixes into the MQ
enablement patch that first uses them.
- Prefer request_irq -> napi_enable -> PHYP enable on MQ open.
- Preserve open unwind so set_real_num_rx / IRQ failures free LAN
before buffer pools.
- MQ open replenishes every queue before setup_rx_interrupts() unmasks
PHYP (drop avoidance during open; PHYP only interrupts after a
successful enqueue). SQ keeps classic setup-then-schedule kick.
- Dispose subordinate virq mappings on setup_rx_interrupts()
request_irq failure (err_free_irqs), matching err_disable_napi.
- Open unwind: setup/cleanup own subordinate dispose; skip duplicate
dispose on those paths.
- H_FUNCTION on subordinate register is a hard open failure (no blind
retry / no fake single-queue fallback).
- Note: hot-path netdev->stats accounting moves to the next patch (qstats).
- Put already-created pool kobjects on probe kobject_init_and_add /
set_real_num_tx_queues / register_netdev failure (bisect-safe).
drivers/net/ethernet/ibm/ibmveth.c | 504 +++++++++++++++++++++++++----
drivers/net/ethernet/ibm/ibmveth.h | 5 +-
2 files changed, 439 insertions(+), 70 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 86299c62d4ec..36f0926711e6 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -156,6 +156,22 @@ static inline int ibmveth_rxq_csum_good(struct ibmveth_adapter *adapter,
return ibmveth_rxq_flags(adapter, queue_index) & IBMVETH_RXQ_CSUM_GOOD;
}
+/* Lockless IRQ/poll readers vs resize publishers. */
+static inline unsigned int
+ibmveth_get_num_rx_queues(const struct ibmveth_adapter *adapter)
+{
+ return READ_ONCE(adapter->num_rx_queues);
+}
+
+static inline void
+ibmveth_publish_num_rx_queues(struct ibmveth_adapter *adapter,
+ unsigned int num)
+{
+ /* Per-queue state must be visible before readers observe num. */
+ smp_wmb();
+ WRITE_ONCE(adapter->num_rx_queues, num);
+}
+
static unsigned int ibmveth_real_max_tx_queues(void)
{
unsigned int n_cpu = num_online_cpus();
@@ -235,7 +251,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
struct net_device *netdev = adapter->netdev;
int i;
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
adapter->buffer_list_addr[i] =
(void *)get_zeroed_page(GFP_KERNEL);
if (!adapter->buffer_list_addr[i]) {
@@ -283,7 +299,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
}
netdev_dbg(netdev, "allocated %d RX queue(s) with %d entries each\n",
- adapter->num_rx_queues, rxq_entries);
+ ibmveth_get_num_rx_queues(adapter), rxq_entries);
return 0;
@@ -321,9 +337,9 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
int i;
netdev_dbg(adapter->netdev, "cleaning up %d RX queue(s)\n",
- adapter->num_rx_queues);
+ ibmveth_get_num_rx_queues(adapter));
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
if (adapter->buffer_list_dma[i]) {
dma_unmap_single(dev, adapter->buffer_list_dma[i],
4096, DMA_BIDIRECTIONAL);
@@ -480,7 +496,7 @@ ibmveth_dispose_subordinate_irq_mappings(struct ibmveth_adapter *adapter)
{
int i;
- for (i = 1; i < adapter->num_rx_queues; i++)
+ for (i = 1; i < ibmveth_get_num_rx_queues(adapter); i++)
ibmveth_dispose_subordinate_irq_mapping(adapter, i);
}
@@ -503,7 +519,7 @@ static int
ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
- int i, rc, num = adapter->num_rx_queues;
+ int i, rc, num = ibmveth_get_num_rx_queues(adapter);
for (i = 0; i < num; i++) {
if (!adapter->queue_irq[i]) {
@@ -587,24 +603,24 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
if (!adapter->rx_irq_setup)
return;
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
if (!adapter->queue_irq[i])
continue;
ibmveth_disable_irq(adapter, i);
synchronize_irq(adapter->queue_irq[i]);
}
- for (i = 0; i < adapter->num_rx_queues; i++)
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++)
napi_disable(&adapter->napi[i]);
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
if (!adapter->queue_irq[i])
continue;
ibmveth_disable_irq(adapter, i);
synchronize_irq(adapter->queue_irq[i]);
}
- for (i = 0; i < adapter->num_rx_queues; i++) {
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
if (adapter->queue_irq[i])
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
}
@@ -631,7 +647,7 @@ static bool ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
struct napi_struct *napi = &adapter->napi[qindex];
int rc;
- if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
+ if (WARN_ON(qindex < 0 || qindex >= ibmveth_get_num_rx_queues(adapter)))
return false;
/*
@@ -649,6 +665,24 @@ static bool ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
return false;
}
+/**
+ * ibmveth_kick_rx_queue_if_pending - Schedule NAPI if PHYP posted while masked
+ * @adapter: ibmveth adapter
+ * @queue_index: RX queue index just unmasked
+ *
+ * After enable_irq() / MQ open unmask, descriptors may already be pending
+ * (buffers were posted while PHYP was masked). Use schedule_rx_queue() so
+ * PHYP is masked before NAPI runs — napi_schedule() then disable_irq() from
+ * process context can race a completing poll and leave the queue masked
+ * with NAPI idle.
+ */
+static void
+ibmveth_kick_rx_queue_if_pending(struct ibmveth_adapter *adapter,
+ int queue_index)
+{
+ if (ibmveth_rxq_pending_buffer(adapter, queue_index))
+ ibmveth_schedule_rx_queue(adapter, queue_index);
+}
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
@@ -963,11 +997,21 @@ static int ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
* because there was not a buffer in the buffer list capable of holding
* the frame.
*/
-static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
+static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter,
+ int queue_index)
{
- __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
+ __be64 *p;
+ u64 drops;
+
+ if (queue_index < 0 ||
+ queue_index >= ibmveth_get_num_rx_queues(adapter) ||
+ !adapter->buffer_list_addr[queue_index])
+ return;
+
+ p = adapter->buffer_list_addr[queue_index] + 4096 - 8;
+ drops = be64_to_cpup(p);
- adapter->rx_no_buffer = be64_to_cpup(p);
+ adapter->rx_no_buffer = drops;
}
/* replenish routine */
@@ -982,10 +1026,10 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
int batch_fallback = 0;
int hcall_fail = 0;
- if (queue_index >= adapter->num_rx_queues) {
+ if (queue_index >= ibmveth_get_num_rx_queues(adapter)) {
netdev_dbg(adapter->netdev,
"Skipping replenish for freed queue %d (num_queues=%d)\n",
- queue_index, adapter->num_rx_queues);
+ queue_index, ibmveth_get_num_rx_queues(adapter));
return;
}
@@ -1019,7 +1063,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
}
out_unlock:
- ibmveth_update_rx_no_buffer(adapter);
+ ibmveth_update_rx_no_buffer(adapter, queue_index);
spin_unlock_irqrestore(&rxq->replenish_lock, flags);
@@ -1033,6 +1077,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
dev_err_ratelimited(&adapter->netdev->dev,
"MQ buffer add H_FUNCTION (q=%d, batch=%u), reset\n",
queue_index, fail.batch);
+ adapter->mq_fallback = true;
schedule_work(&adapter->work);
}
@@ -1052,6 +1097,30 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
fail.filled, fail.lpar_rc, fail.batch);
}
+/**
+ * ibmveth_restart_rx_queue - Post buffers and ensure Q can take RX
+ * @adapter: ibmveth adapter
+ * @qindex: RX queue index
+ *
+ * SQ open leaves PHYP masked until the first poll. If schedule_prep fails,
+ * NAPI never runs and the queue stays masked (TX OK, RX/ARP dead) until
+ * reload. Replenish first so an enable_irq fallback can actually deliver.
+ * Also used after every open (SQ and MQ) and after scale-down so a
+ * queue is not left idle+masked.
+ */
+static void ibmveth_restart_rx_queue(struct ibmveth_adapter *adapter,
+ int qindex)
+{
+ int rc;
+
+ ibmveth_replenish_task(adapter, qindex);
+ if (ibmveth_schedule_rx_queue(adapter, qindex))
+ return;
+
+ rc = ibmveth_enable_irq(adapter, qindex);
+ WARN_ON(rc);
+}
+
/* empty and free ana buffer pool - also used to do cleanup in error paths */
static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
struct ibmveth_buff_pool *pool)
@@ -1180,7 +1249,7 @@ ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
int i, q, rc;
/* Initialize pool metadata for queues 1..N from queue 0 settings */
- for (q = 1; q < adapter->num_rx_queues; q++) {
+ for (q = 1; q < ibmveth_get_num_rx_queues(adapter); q++) {
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
struct ibmveth_buff_pool *src =
&adapter->rx_buff_pool[0][i];
@@ -1196,7 +1265,7 @@ ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
}
/* Allocate actual buffers for all queues */
- for (q = 0; q < adapter->num_rx_queues; q++) {
+ for (q = 0; q < ibmveth_get_num_rx_queues(adapter); q++) {
rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
if (rc) {
/* Free pools for all previous queues */
@@ -1207,7 +1276,7 @@ ibmveth_alloc_buffer_pools(struct ibmveth_adapter *adapter)
}
netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
- adapter->num_rx_queues);
+ ibmveth_get_num_rx_queues(adapter));
return 0;
}
@@ -1223,11 +1292,11 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
int q;
/* Free buffer pools for all queues */
- for (q = 0; q < adapter->num_rx_queues; q++)
+ for (q = 0; q < ibmveth_get_num_rx_queues(adapter); q++)
ibmveth_free_queue_buffer_pools(adapter, q);
netdev_dbg(adapter->netdev, "freed buffer pools for %d queue(s)\n",
- adapter->num_rx_queues);
+ ibmveth_get_num_rx_queues(adapter));
}
static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
@@ -1516,6 +1585,137 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
return rc;
}
+/**
+ * ibmveth_register_logical_lan_queue - Register subordinate queue with
+ * hypervisor
+ * @adapter: ibmveth adapter structure
+ * @rxq_desc: Receive queue descriptor
+ * @queue_index: RX queue index (1..N for subordinate queues)
+ *
+ * Registers a subordinate receive queue using H_REG_LOGICAL_LAN_QUEUE.
+ * On success, stores the queue handle and virtual IRQ in the adapter.
+ * If IRQ mapping fails after a successful hypervisor registration, the
+ * queue is freed before returning.
+ *
+ * Return: H_SUCCESS on success, negative errno on IRQ mapping failure,
+ * hypervisor error code otherwise
+ */
+static int
+ibmveth_register_logical_lan_queue(struct ibmveth_adapter *adapter,
+ union ibmveth_buf_desc rxq_desc,
+ int queue_index)
+{
+ unsigned long handle, hwirq;
+ unsigned int virq;
+ long lpar_rc;
+ unsigned long ua = adapter->vdev->unit_address;
+ unsigned long bl = adapter->buffer_list_dma[queue_index];
+
+ netdev_dbg(adapter->netdev,
+ "register queue %d: ua=0x%lx bl=0x%lx rxq=0x%llx\n",
+ queue_index, ua, bl, rxq_desc.desc);
+ do {
+ lpar_rc = h_register_logical_lan_queue(ua, bl,
+ rxq_desc.desc, &handle,
+ &hwirq);
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+ adapter->hcall_stats.reg_lan_queue++;
+
+ if (lpar_rc == H_SUCCESS) {
+ virq = irq_create_mapping(NULL, hwirq);
+ if (!virq) {
+ unsigned long free_rc;
+
+ netdev_err(adapter->netdev,
+ "Failed to map IRQ for queue %d (hwirq=%lu)\n",
+ queue_index, hwirq);
+ do {
+ free_rc = h_free_logical_lan_queue(ua, handle);
+ } while (H_IS_LONG_BUSY(free_rc) ||
+ (free_rc == H_BUSY));
+ adapter->hcall_stats.free_lan_queue++;
+ if (free_rc != H_SUCCESS)
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan_queue failed for queue %d after IRQ map failure: rc=0x%lx\n",
+ queue_index, free_rc);
+ return -EINVAL;
+ }
+
+ adapter->queue_handle[queue_index] = handle;
+ adapter->queue_irq[queue_index] = virq;
+
+ netdev_dbg(adapter->netdev,
+ "queue %d registered: handle=0x%llx irq=%u\n",
+ queue_index, adapter->queue_handle[queue_index],
+ adapter->queue_irq[queue_index]);
+ return H_SUCCESS;
+ }
+
+ /*
+ * H_FUNCTION means firmware rejected this subordinate register
+ * (MQ unsupported / dropped after LPM). Caller fails this open and
+ * latches mq_fallback so the next open applies SQ; keep a specific
+ * log then the generic failure lines below.
+ */
+ if (lpar_rc == H_FUNCTION)
+ netdev_err(adapter->netdev,
+ "h_register_logical_lan_queue H_FUNCTION for queue %d (firmware MQ unsupported)\n",
+ queue_index);
+
+ netdev_err(adapter->netdev,
+ "h_register_logical_lan_queue failed for queue %d with %ld\n",
+ queue_index, lpar_rc);
+ netdev_err(adapter->netdev,
+ "queue %d params: unit_addr=0x%x buffer_list_dma=0x%llx rxq_desc=0x%llx\n",
+ queue_index, adapter->vdev->unit_address,
+ adapter->buffer_list_dma[queue_index],
+ rxq_desc.desc);
+
+ return lpar_rc;
+}
+
+/**
+ * ibmveth_register_single_rx_queue - Register one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to register (1..N)
+ *
+ * Builds the queue descriptor and registers with the hypervisor via
+ * ibmveth_register_logical_lan_queue().
+ *
+ * Return: 0 on success, -EINVAL if @queue_idx is invalid, -EOPNOTSUPP if
+ * firmware rejects MQ (H_FUNCTION), -EIO on other failures
+ */
+static int
+ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ struct net_device *netdev = adapter->netdev;
+ union ibmveth_buf_desc rxq_desc;
+ long lpar_rc;
+
+ if (WARN_ON(queue_idx < 1 || queue_idx >= IBMVETH_MAX_RX_QUEUES))
+ return -EINVAL;
+
+ rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
+ adapter->rx_queue[queue_idx].queue_len;
+ rxq_desc.fields.address = adapter->rx_queue[queue_idx].queue_dma;
+
+ lpar_rc = ibmveth_register_logical_lan_queue(adapter, rxq_desc,
+ queue_idx);
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(netdev, "Failed to register queue %d: rc=0x%lx\n",
+ queue_idx, lpar_rc);
+ if (lpar_rc == H_FUNCTION)
+ return -EOPNOTSUPP;
+ return -EIO;
+ }
+
+ netdev_dbg(netdev, "Registered queue %d with handle 0x%llx\n",
+ queue_idx, adapter->queue_handle[queue_idx]);
+
+ return 0;
+}
+
/**
* ibmveth_free_all_queues - Free all RX queues at once
* @adapter: ibmveth adapter structure
@@ -1528,7 +1728,8 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
* Used during interface close and registration error cleanup.
*
* Clears queue handles only; queue_irq[] is released by
- * ibmveth_cleanup_rx_interrupts().
+ * ibmveth_cleanup_rx_interrupts() on close, or by
+ * ibmveth_dispose_subordinate_irq_mappings() on partial register failure.
*/
static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
{
@@ -1547,7 +1748,7 @@ static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
"h_free_logical_lan failed: %ld\n", lpar_rc);
}
- for (i = 0; i < adapter->num_rx_queues; i++)
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++)
adapter->queue_handle[i] = 0;
}
@@ -1556,10 +1757,12 @@ static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
* @adapter: ibmveth adapter structure
* @mac_address: MAC address for device registration
*
- * Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue
- * registration is added when multi-queue RX is enabled.
+ * Registers queue 0 via ibmveth_register_logical_lan(), then subordinate
+ * queues 1..N when multi-queue mode is enabled.
*
- * Return: 0 on success, -ENONET if queue 0 registration fails
+ * Return: 0 on success, -ENONET if queue 0 registration fails,
+ * -EOPNOTSUPP if firmware rejects a subordinate queue (H_FUNCTION;
+ * caller latches mq_fallback), -EIO on other subordinate failures
*/
static int
ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
@@ -1567,7 +1770,8 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
struct net_device *netdev = adapter->netdev;
union ibmveth_buf_desc rxq_desc;
unsigned long lpar_rc;
- int rc;
+ unsigned int num;
+ int i, rc;
rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
adapter->rx_queue[0].queue_len;
@@ -1592,9 +1796,67 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
return -ENONET;
}
+ num = ibmveth_get_num_rx_queues(adapter);
+ if (num == 1 || !adapter->multi_queue) {
+ netdev_dbg(netdev,
+ "registered 1 RX queue with hypervisor (single-queue mode)\n");
+ return 0;
+ }
+
+ netdev_dbg(netdev, "Registering %d subordinate queues (1-%d)\n",
+ num - 1, num - 1);
+
+ for (i = 1; i < num; i++) {
+ rc = ibmveth_register_single_rx_queue(adapter, i);
+ if (rc) {
+ /* Firmware MQ gone: fall back to SQ on next open. */
+ if (rc == -EOPNOTSUPP)
+ adapter->mq_fallback = true;
+ goto err_unregister;
+ }
+ }
+
netdev_dbg(netdev,
- "registered 1 RX queue with hypervisor (single-queue mode)\n");
+ "registered %d RX queues with hypervisor (multi-queue mode)\n",
+ num);
+
return 0;
+
+err_unregister:
+ ibmveth_dispose_subordinate_irq_mappings(adapter);
+ ibmveth_free_all_queues(adapter);
+ return rc;
+}
+
+/**
+ * ibmveth_apply_mq_fallback - Drop multi-queue mode after firmware rejection
+ * @adapter: ibmveth adapter
+ *
+ * mq_fallback is latched when firmware rejects MQ (subordinate register or
+ * buffer-add H_FUNCTION). Apply only at the start of open after teardown so
+ * num_rx_queues is not shrunk while IRQ/NAPI still reference higher queues.
+ * Clears multi_queue and forces a single RX queue so subsequent
+ * register/buffer hcalls use the classic single-queue path.
+ */
+static void ibmveth_apply_mq_fallback(struct ibmveth_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ if (!adapter->mq_fallback)
+ return;
+
+ adapter->mq_fallback = false;
+
+ if (!adapter->multi_queue && ibmveth_get_num_rx_queues(adapter) == 1)
+ return;
+
+ netdev_warn(netdev,
+ "Falling back to single RX queue (firmware MQ unavailable)\n");
+ adapter->multi_queue = 0;
+ ibmveth_publish_num_rx_queues(adapter, 1);
+ /* real_num_rx_queues is set later in open after resources exist. */
+ if (adapter->rx_buffers_per_hcall > IBMVETH_MAX_RX_REGULAR)
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
}
static int ibmveth_open(struct net_device *netdev)
@@ -1607,6 +1869,8 @@ static int ibmveth_open(struct net_device *netdev)
netdev_dbg(netdev, "open starting\n");
+ ibmveth_apply_mq_fallback(adapter);
+
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
rxq_entries += adapter->rx_buff_pool[0][i].size;
@@ -1626,18 +1890,33 @@ static int ibmveth_open(struct net_device *netdev)
if (rc)
goto out_free_buffer_pools;
- rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
+ rc = netif_set_real_num_rx_queues(netdev,
+ ibmveth_get_num_rx_queues(adapter));
+
if (rc) {
netdev_err(netdev, "failed to set number of rx queues\n");
goto out_unregister_queues;
}
+ /*
+ * Post buffers before setup_rx_interrupts(). MQ setup then unmasks
+ * PHYP; SQ setup leaves PHYP masked. kick_if_pending alone is not
+ * enough: after ifdown/up (RX=8, no -L) NAPI can be idle with no
+ * pending descriptor and the queue stays dead (TX OK, ARP/RX fail).
+ * restart_rx_queue() replenishes, schedules NAPI, and unmasks if
+ * prep fails.
+ */
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
+ netdev_dbg(netdev, "initial replenish cycle for queue %d\n", i);
+ ibmveth_replenish_task(adapter, i);
+ }
+
rc = ibmveth_setup_rx_interrupts(adapter);
if (rc)
goto out_free_all_queues; /* setup already disposed IRQs */
- netdev_dbg(netdev, "initial replenish cycle\n");
- ibmveth_schedule_rx_queue(adapter, 0);
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++)
+ ibmveth_restart_rx_queue(adapter, i);
rc = ibmveth_alloc_tx_resources(adapter);
if (rc)
@@ -1670,6 +1949,7 @@ static int ibmveth_open(struct net_device *netdev)
static int ibmveth_close(struct net_device *netdev)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
+ int i;
/* Gate on opened, not IFF_UP: pool_store/change_mtu close+open can
* leave IFF_UP set after a failed reopen.
@@ -1690,7 +1970,8 @@ static int ibmveth_close(struct net_device *netdev)
/* Wait for softirq/poll that already passed shutdown checks. */
synchronize_net();
- ibmveth_update_rx_no_buffer(adapter);
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++)
+ ibmveth_update_rx_no_buffer(adapter, i);
ibmveth_free_all_queues(adapter);
/* Free TX LTBs after quiesce and after H_FREE_LOGICAL_LAN so xmit
* cannot touch unmapped bounce buffers while the LAN is live.
@@ -2094,6 +2375,7 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,
return 1;
}
+ adapter->hcall_stats.send_lan++;
return 0;
}
@@ -2501,7 +2783,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
queue_index = napi - adapter->napi;
if (WARN_ON(queue_index < 0 ||
- queue_index >= adapter->num_rx_queues)) {
+ queue_index >= ibmveth_get_num_rx_queues(adapter))) {
napi_complete_done(napi, 0);
return 0;
}
@@ -2589,10 +2871,11 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
qindex = napi - adapter->napi;
/*
- * Quiet on out-of-range: teardown can leave a residual IRQ after the
- * live count drops. Do not WARN-storm; return IRQ_NONE until free_irq.
+ * Quiet on out-of-range: scale-down publishes a lower live count
+ * before free_irq(). A residual IRQ must not WARN-storm; return
+ * IRQ_NONE until the handler is removed.
*/
- if (qindex < 0 || qindex >= adapter->num_rx_queues)
+ if (qindex < 0 || qindex >= ibmveth_get_num_rx_queues(adapter))
return IRQ_NONE;
ibmveth_schedule_rx_queue(adapter, qindex);
@@ -2701,9 +2984,14 @@ static int ibmveth_change_mtu(struct net_device *dev, int new_mtu)
static void ibmveth_poll_controller(struct net_device *dev)
{
struct ibmveth_adapter *adapter = netdev_priv(dev);
+ unsigned int num = ibmveth_get_num_rx_queues(adapter);
+ int i;
- ibmveth_replenish_task(adapter, 0);
- ibmveth_schedule_rx_queue(adapter, 0);
+ for (i = 0; i < num; i++)
+ ibmveth_replenish_task(adapter, i);
+
+ for (i = 0; i < num; i++)
+ ibmveth_schedule_rx_queue(adapter, i);
}
#endif
@@ -2721,8 +3009,7 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
struct ibmveth_adapter *adapter;
struct iommu_table *tbl;
unsigned long ret;
- int i;
- int rxqentries = 1;
+ int i, q;
tbl = get_iommu_table_base(&vdev->dev);
@@ -2732,23 +3019,35 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
adapter = netdev_priv(netdev);
- ret = IBMVETH_BUFF_LIST_SIZE + IBMVETH_FILT_LIST_SIZE;
+ /* One buffer list page per RX queue; filter list is shared. */
+ ret = IBMVETH_BUFF_LIST_SIZE * ibmveth_get_num_rx_queues(adapter) +
+ IBMVETH_FILT_LIST_SIZE;
ret += IOMMU_PAGE_ALIGN(netdev->mtu, tbl);
/* add size of mapped tx buffers */
ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- /* add the size of the active receive buffers */
- if (adapter->rx_buff_pool[0][i].active)
- ret +=
- adapter->rx_buff_pool[0][i].size *
- IOMMU_PAGE_ALIGN(adapter->rx_buff_pool[0][i].
- buff_size, tbl);
- rxqentries += adapter->rx_buff_pool[0][i].size;
- }
- /* add the size of the receive queue entries */
- ret += IOMMU_PAGE_ALIGN(
- rxqentries * sizeof(struct ibmveth_rx_q_entry), tbl);
+ /*
+ * Pool metadata for queues 1+ is copied from queue 0 at open.
+ * Always size from pool 0 x num_rx_queues (valid at probe before that
+ * copy).
+ */
+ for (q = 0; q < ibmveth_get_num_rx_queues(adapter); q++) {
+ int rxqentries = 1;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *bpool =
+ &adapter->rx_buff_pool[0][i];
+
+ if (bpool->active)
+ ret += bpool->size *
+ IOMMU_PAGE_ALIGN(bpool->buff_size, tbl);
+ rxqentries += bpool->size;
+ }
+
+ /* add the size of the receive queue entries */
+ ret += IOMMU_PAGE_ALIGN(rxqentries *
+ sizeof(struct ibmveth_rx_q_entry), tbl);
+ }
return ret;
}
@@ -2813,9 +3112,18 @@ static const struct net_device_ops ibmveth_netdev_ops = {
#endif
};
+static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
+ int pools_ready)
+{
+ int i;
+
+ for (i = 0; i < pools_ready; i++)
+ kobject_put(&adapter->rx_buff_pool[0][i].kobj);
+}
+
static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
{
- int rc, i, mac_len;
+ int rc, i, mac_len, pools_ready = 0;
struct net_device *netdev;
struct ibmveth_adapter *adapter;
unsigned char *mac_addr_p;
@@ -2850,7 +3158,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
return -EINVAL;
}
- netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter), IBMVETH_MAX_QUEUES, 1);
+ netdev = alloc_etherdev_mqs(sizeof(struct ibmveth_adapter),
+ IBMVETH_MAX_QUEUES, IBMVETH_MAX_RX_QUEUES);
if (!netdev)
return -ENOMEM;
@@ -2863,7 +3172,9 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
adapter->mcastFilterSize = be32_to_cpu(*mcastFilterSize_p);
ibmveth_init_link_settings(netdev);
- netif_napi_add_weight(netdev, &adapter->napi[0], ibmveth_poll, 16);
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+ netif_napi_add_weight(netdev, &adapter->napi[i],
+ ibmveth_poll, 16);
netdev->irq = dev->irq;
netdev->netdev_ops = &ibmveth_netdev_ops;
@@ -2895,16 +3206,29 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev->features |= NETIF_F_FRAGLIST;
}
- /* Initialize queue count - always 1 for now */
- adapter->multi_queue = 0;
- adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
+ if (ret == H_SUCCESS &&
+ (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) {
+ adapter->multi_queue = 1;
+ ibmveth_publish_num_rx_queues(adapter,
+ min(num_online_cpus(),
+ IBMVETH_DEFAULT_QUEUES));
+ netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n",
+ ibmveth_get_num_rx_queues(adapter));
+ } else {
+ adapter->multi_queue = 0;
+ ibmveth_publish_num_rx_queues(adapter,
+ IBMVETH_DEFAULT_RX_QUEUES);
+ }
if (ret == H_SUCCESS &&
(ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
- adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
+ if (adapter->multi_queue)
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_QUEUE;
+ else
+ adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
netdev_dbg(netdev,
"RX Multi-buffer hcall supported by FW, batch set to %u\n",
- adapter->rx_buffers_per_hcall);
+ adapter->rx_buffers_per_hcall);
} else {
adapter->rx_buffers_per_hcall = 1;
netdev_dbg(netdev,
@@ -2922,25 +3246,62 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
- int error;
ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
pool_count[i], pool_size[i],
pool_active[i]);
- error = kobject_init_and_add(kobj, &ktype_veth_pool,
- &dev->dev.kobj, "pool%d", i);
- if (!error)
- kobject_uevent(kobj, KOBJ_ADD);
+ rc = kobject_init_and_add(kobj, &ktype_veth_pool,
+ &dev->dev.kobj, "pool%d", i);
+ if (rc) {
+ dev_err(&dev->dev,
+ "failed to create pool%d kobject: %d\n", i, rc);
+ /* init_and_add takes a ref even on failure */
+ kobject_put(kobj);
+ ibmveth_put_pool_kobjs(adapter, pools_ready);
+ dev_set_drvdata(&dev->dev, NULL);
+ free_netdev(netdev);
+ return rc;
+ }
+
+ pools_ready++;
+ kobject_uevent(kobj, KOBJ_ADD);
}
+ /*
+ * VIO CMO entitlement was set before probe (netdev NULL → default).
+ * Recompute now that num_rx_queues and pool 0 metadata are known.
+ */
+ if (firmware_has_feature(FW_FEATURE_CMO))
+ vio_cmo_set_dev_desired(dev, ibmveth_get_desired_dma(dev));
+
rc = netif_set_real_num_tx_queues(netdev, min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES));
if (rc) {
netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n",
rc);
+ ibmveth_put_pool_kobjs(adapter, pools_ready);
+ dev_set_drvdata(&dev->dev, NULL);
free_netdev(netdev);
return rc;
}
+
+ /*
+ * alloc_etherdev_mqs() sized real_num_rx_queues to MAX. Match the
+ * advertised default (or SQ 1) before register_netdev so down-state
+ * readers agree with adapter->num_rx_queues / ethtool -l.
+ */
+ rc = netif_set_real_num_rx_queues(netdev,
+ ibmveth_get_num_rx_queues(adapter));
+
+ if (rc) {
+ netdev_dbg(netdev, "failed to set number of rx queues rc=%d\n",
+ rc);
+ ibmveth_put_pool_kobjs(adapter, pools_ready);
+ dev_set_drvdata(&dev->dev, NULL);
+ free_netdev(netdev);
+ return rc;
+ }
+
adapter->tx_ltb_size = PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE);
for (i = 0; i < IBMVETH_MAX_QUEUES; i++)
adapter->tx_ltb_ptr[i] = NULL;
@@ -2954,6 +3315,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
if (rc) {
netdev_dbg(netdev, "failed to register netdev rc=%d\n", rc);
+ ibmveth_put_pool_kobjs(adapter, pools_ready);
+ dev_set_drvdata(&dev->dev, NULL);
free_netdev(netdev);
return rc;
}
@@ -3125,7 +3488,6 @@ static ssize_t veth_pool_store(struct kobject *kobj, struct attribute *attr,
return rc;
}
-
#define ATTR(_name, _mode) \
struct attribute veth_##_name##_attr = { \
.name = __stringify(_name), .mode = _mode, \
@@ -3158,8 +3520,12 @@ static int ibmveth_resume(struct device *dev)
{
struct net_device *netdev = dev_get_drvdata(dev);
struct ibmveth_adapter *adapter = netdev_priv(netdev);
+ unsigned int num = ibmveth_get_num_rx_queues(adapter);
+ int i;
+
+ for (i = 0; i < num; i++)
+ ibmveth_schedule_rx_queue(adapter, i);
- ibmveth_schedule_rx_queue(adapter, 0);
return 0;
}
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index d02444d5b3b8..1394ef97bd8d 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -30,6 +30,7 @@
#define IbmVethMcastRemoveFilter 0x2UL
#define IbmVethMcastClearFilterTable 0x3UL
+#define IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT 0x0000000000080000UL
#define IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT 0x0000000000040000UL
#define IBMVETH_ILLAN_LRG_SR_ENABLED 0x0000000000010000UL
#define IBMVETH_ILLAN_LRG_SND_SUPPORT 0x0000000000008000UL
@@ -259,9 +260,10 @@ static inline long h_illan_attributes(unsigned long unit_address,
#define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
#define IBMVETH_MAX_QUEUES 16U
#define IBMVETH_DEFAULT_QUEUES 8U
-#define IBMVETH_MAX_RX_QUEUES 1U
+#define IBMVETH_MAX_RX_QUEUES 16U
#define IBMVETH_DEFAULT_RX_QUEUES 1U
#define IBMVETH_MAX_RX_REGULAR 8U
+#define IBMVETH_MAX_RX_QUEUE 12U
#define IBMVETH_MAX_RX_PER_HCALL 12U
static int pool_size[] = { 512, 1024 * 2, 1024 * 16, 1024 * 32, 1024 * 64 };
@@ -327,6 +329,7 @@ struct ibmveth_adapter {
unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES];
int multi_queue;
unsigned int num_rx_queues;
+ bool mq_fallback;
/* Lifetime: true after successful ndo_open until close clears it. */
bool opened;
/* Lifetime: true while RX IRQ handlers / NAPI are installed. */
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 11/15] ibmveth: Add per-queue RX and TX statistics collection
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (9 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels Mingming Cao
` (3 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Count per-queue RX and TX statistics and expose them via ethtool -S
and ndo_get_stats64().
RX hot path (poll / IRQ / replenish) updates rx_qstats[] for packets,
bytes, polls, interrupts, large_packets, invalid_buffers, and
no_buffer_drops. TX hot path updates tx_qstats[] for packets, bytes,
large_packets, dropped_packets, send_failures, and checksum_offload.
Allocate qstat arrays for the adapter lifetime (probe/remove) and free
them from ibmveth_remove() / ibmveth_probe_cleanup(). Adapter-level
ethtool strings sum per-queue counters on read.
ibmveth_update_rx_no_buffer() assigns PHYP's page-absolute no_buffer
count into rx_qstats[i].no_buffer_drops (= not +=): that PHYP field is
absolute for the life of the buffer-list page (zeroed each open).
When the absolute decreases (reopen / queue reuse after -L), fold the
previous slot value into adapter->rx_no_buffer_retired so the adapter
ethtool sum stays monotonic. Per-queue ethtool still shows the live
page absolute.
Keep get_stats64 / adapter ethtool totals monotonic across channel
shrink by summing the full allocated qstat arrays.
Expose hcall_* counters via ethtool -S string table here (with the other
stats), not in earlier IRQ/MQ enable patches.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- Series renumber: mailed v4 10/14 stats -> tip P11 (P09 peel;
get_channels -> P12)
- rx_no_buffer_retired + sum MAX_* slots so adapter no-buffer / qstat
totals stay monotonic across reopen and channel shrink
- probe_cleanup: clear vio drvdata before free_netdev (CMO cannot see a
freed netdev on rebind)
- remove: unregister_netdev then cancel_work_sync (no UAF reset worker)
Changes in v4:
- Merge v3's separate RX and TX stats commits into one patch.
- Introduce rx_queue_stats / tx_qstats / NUM macros here (first use).
- Allocate/free qstats at probe/remove instead of open/close.
- Report adapter-level ethtool strings by summing per-queue counters on
read; drop aggregate_* helpers.
- Sum global rx_no_buffer across MQ queues into this statistics patch.
- Cacheline-align per-queue stats; derive field counts with offsetof so
alignment padding is not counted as a statistic.
- probe_cleanup() cancels reset work, puts pool kobjects via helper from
the prior patch, and frees qstats on probe failure paths.
- Keep plain u64 qstats like existing ibmveth / ibmvnic (PPC_PSERIES).
drivers/net/ethernet/ibm/ibmveth.c | 401 ++++++++++++++++++++++++++---
drivers/net/ethernet/ibm/ibmveth.h | 36 +++
2 files changed, 400 insertions(+), 37 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 36f0926711e6..3202b657c9b8 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -98,7 +98,17 @@ static struct ibmveth_stat ibmveth_stats[] = {
{ "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) },
{ "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) },
{ "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) },
- { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }
+ { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) },
+ { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
+ { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) },
+ { "hcall_add_bufs_queue",
+ IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) },
+ { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) },
+ { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) },
+ { "hcall_free_lan_queue",
+ IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) },
+ { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) },
+ { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) },
};
/* simple methods of getting data from the current rxq entry */
@@ -237,6 +247,60 @@ ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
}
}
+/**
+ * ibmveth_alloc_rx_qstats - Allocate per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->rx_qstats = kcalloc(IBMVETH_MAX_RX_QUEUES,
+ sizeof(*adapter->rx_qstats),
+ GFP_KERNEL);
+ if (!adapter->rx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_rx_qstats - Free per-queue RX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_rx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->rx_qstats);
+ adapter->rx_qstats = NULL;
+}
+
+/**
+ * ibmveth_alloc_tx_qstats - Allocate per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ *
+ * Return: 0 on success, -ENOMEM on failure
+ */
+static int ibmveth_alloc_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ adapter->tx_qstats = kcalloc(IBMVETH_MAX_QUEUES,
+ sizeof(*adapter->tx_qstats),
+ GFP_KERNEL);
+ if (!adapter->tx_qstats)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * ibmveth_free_tx_qstats - Free per-queue TX statistics
+ * @adapter: ibmveth adapter structure
+ */
+static void ibmveth_free_tx_qstats(struct ibmveth_adapter *adapter)
+{
+ kfree(adapter->tx_qstats);
+ adapter->tx_qstats = NULL;
+}
+
/**
* ibmveth_alloc_rx_queues - Allocate per-queue RX resources
* @adapter: ibmveth adapter structure
@@ -1011,7 +1075,23 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter,
p = adapter->buffer_list_addr[queue_index] + 4096 - 8;
drops = be64_to_cpup(p);
- adapter->rx_no_buffer = drops;
+ /*
+ * PHYP's buffer-list page counter is absolute for that page. A new
+ * page (reopen / queue reuse after -L) starts near zero; fold the
+ * previous absolute into retired so adapter-level sums stay
+ * monotonic.
+ */
+ if (adapter->rx_qstats) {
+ u64 *slot = &adapter->rx_qstats[queue_index].no_buffer_drops;
+
+ if (drops < *slot)
+ adapter->rx_no_buffer_retired += *slot;
+ *slot = drops;
+ } else {
+ if (drops < adapter->rx_no_buffer)
+ adapter->rx_no_buffer_retired += adapter->rx_no_buffer;
+ adapter->rx_no_buffer = drops;
+ }
}
/* replenish routine */
@@ -2239,22 +2319,158 @@ static int ibmveth_set_features(struct net_device *dev,
return rc1 ? rc1 : rc2;
}
+/*
+ * Sum per-queue counters for rare ethtool reads. Do not write adapter
+ * globals on the hot path (ibmvnic-style). Accumulated qstats remain
+ * meaningful across ifdown/up. no_buffer_drops slots stay PHYP-page
+ * absolute (=); ibmveth_update_rx_no_buffer() folds decreases into
+ * rx_no_buffer_retired so the adapter sum never goes backwards across
+ * reopen or queue reuse.
+ */
+static u64 ibmveth_sum_rx_invalid_buffers(struct ibmveth_adapter *adapter)
+{
+ u64 total = 0;
+ int i;
+
+ if (!adapter->rx_qstats)
+ return adapter->rx_invalid_buffer;
+
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+ total += adapter->rx_qstats[i].invalid_buffers;
+
+ return total;
+}
+
+static u64 ibmveth_sum_rx_large_packets(struct ibmveth_adapter *adapter)
+{
+ u64 total = 0;
+ int i;
+
+ if (!adapter->rx_qstats)
+ return adapter->rx_large_packets;
+
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+ total += adapter->rx_qstats[i].large_packets;
+
+ return total;
+}
+
+static u64 ibmveth_sum_rx_no_buffer(struct ibmveth_adapter *adapter)
+{
+ u64 total = adapter->rx_no_buffer_retired;
+ int i;
+
+ if (!adapter->rx_qstats)
+ return total + adapter->rx_no_buffer;
+
+ /* Sum-on-read: hot path only refreshes the local queue's qstat. */
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++)
+ total += adapter->rx_qstats[i].no_buffer_drops;
+
+ return total;
+}
+
+static u64 ibmveth_sum_tx_large_packets(struct ibmveth_adapter *adapter)
+{
+ u64 total = 0;
+ int i;
+
+ if (!adapter->tx_qstats)
+ return adapter->tx_large_packets;
+
+ /* Sum every allocated slot so shrinking TX channels cannot drop
+ * historical counters from retired queues.
+ */
+ for (i = 0; i < IBMVETH_MAX_QUEUES; i++)
+ total += adapter->tx_qstats[i].large_packets;
+
+ return total;
+}
+
+static u64 ibmveth_sum_tx_send_failed(struct ibmveth_adapter *adapter)
+{
+ u64 total = 0;
+ int i;
+
+ if (!adapter->tx_qstats)
+ return adapter->tx_send_failed;
+
+ for (i = 0; i < IBMVETH_MAX_QUEUES; i++)
+ total += adapter->tx_qstats[i].send_failures;
+
+ return total;
+}
+
+static u64 ibmveth_ethtool_adapter_stat(struct ibmveth_adapter *adapter,
+ int index)
+{
+ unsigned long offset = ibmveth_stats[index].offset;
+
+ if (offset == IBMVETH_STAT_OFF(rx_invalid_buffer))
+ return ibmveth_sum_rx_invalid_buffers(adapter);
+ if (offset == IBMVETH_STAT_OFF(rx_large_packets))
+ return ibmveth_sum_rx_large_packets(adapter);
+ if (offset == IBMVETH_STAT_OFF(rx_no_buffer))
+ return ibmveth_sum_rx_no_buffer(adapter);
+ if (offset == IBMVETH_STAT_OFF(tx_large_packets))
+ return ibmveth_sum_tx_large_packets(adapter);
+ if (offset == IBMVETH_STAT_OFF(tx_send_failed))
+ return ibmveth_sum_tx_send_failed(adapter);
+
+ return IBMVETH_GET_STAT(adapter, offset);
+}
+
static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ u8 *p = data;
int i;
if (stringset != ETH_SS_STATS)
return;
- for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++, data += ETH_GSTRING_LEN)
- memcpy(data, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+ for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++) {
+ memcpy(p, ibmveth_stats[i].name, ETH_GSTRING_LEN);
+ p += ETH_GSTRING_LEN;
+ }
+
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
+ ethtool_sprintf(&p, "rx%d_packets", i);
+ ethtool_sprintf(&p, "rx%d_bytes", i);
+ ethtool_sprintf(&p, "rx%d_interrupts", i);
+ ethtool_sprintf(&p, "rx%d_polls", i);
+ ethtool_sprintf(&p, "rx%d_large_packets", i);
+ ethtool_sprintf(&p, "rx%d_invalid_buffers", i);
+ ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
+ }
+
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ ethtool_sprintf(&p, "tx%d_packets", i);
+ ethtool_sprintf(&p, "tx%d_bytes", i);
+ ethtool_sprintf(&p, "tx%d_large_packets", i);
+ ethtool_sprintf(&p, "tx%d_dropped_packets", i);
+ ethtool_sprintf(&p, "tx%d_send_failures", i);
+ ethtool_sprintf(&p, "tx%d_checksum_offload", i);
+ }
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ ethtool_sprintf(&p, "pool%d_size", i);
+ ethtool_sprintf(&p, "pool%d_active", i);
+ ethtool_sprintf(&p, "pool%d_available", i);
+ }
}
static int ibmveth_get_sset_count(struct net_device *dev, int sset)
{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+
switch (sset) {
case ETH_SS_STATS:
- return ARRAY_SIZE(ibmveth_stats);
+ return ARRAY_SIZE(ibmveth_stats) +
+ ibmveth_get_num_rx_queues(adapter) *
+ IBMVETH_NUM_RX_QSTATS +
+ dev->real_num_tx_queues * IBMVETH_NUM_TX_QSTATS +
+ IBMVETH_NUM_BUFF_POOLS * 3;
default:
return -EOPNOTSUPP;
}
@@ -2263,11 +2479,44 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
static void ibmveth_get_ethtool_stats(struct net_device *dev,
struct ethtool_stats *stats, u64 *data)
{
- int i;
struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i, j;
for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++)
- data[i] = IBMVETH_GET_STAT(adapter, ibmveth_stats[i].offset);
+ data[i] = ibmveth_ethtool_adapter_stat(adapter, i);
+
+ for (j = 0; j < ibmveth_get_num_rx_queues(adapter); j++) {
+ if (adapter->rx_qstats) {
+ data[i++] = adapter->rx_qstats[j].packets;
+ data[i++] = adapter->rx_qstats[j].bytes;
+ data[i++] = adapter->rx_qstats[j].interrupts;
+ data[i++] = adapter->rx_qstats[j].polls;
+ data[i++] = adapter->rx_qstats[j].large_packets;
+ data[i++] = adapter->rx_qstats[j].invalid_buffers;
+ data[i++] = adapter->rx_qstats[j].no_buffer_drops;
+ } else {
+ i += IBMVETH_NUM_RX_QSTATS;
+ }
+ }
+
+ for (j = 0; j < dev->real_num_tx_queues; j++) {
+ if (adapter->tx_qstats) {
+ data[i++] = adapter->tx_qstats[j].packets;
+ data[i++] = adapter->tx_qstats[j].bytes;
+ data[i++] = adapter->tx_qstats[j].large_packets;
+ data[i++] = adapter->tx_qstats[j].dropped_packets;
+ data[i++] = adapter->tx_qstats[j].send_failures;
+ data[i++] = adapter->tx_qstats[j].checksum_offload;
+ } else {
+ i += IBMVETH_NUM_TX_QSTATS;
+ }
+ }
+
+ for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+ data[i++] = adapter->rx_buff_pool[0][j].size;
+ data[i++] = adapter->rx_buff_pool[0][j].active;
+ data[i++] = atomic_read(&adapter->rx_buff_pool[0][j].available);
+ }
}
static void ibmveth_get_channels(struct net_device *netdev,
@@ -2380,8 +2629,10 @@ static int ibmveth_send(struct ibmveth_adapter *adapter,
}
static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
- struct net_device *netdev)
+ struct ibmveth_adapter *adapter,
+ int queue_num)
{
+ struct net_device *netdev = adapter->netdev;
struct ethhdr *ether_header;
int ret = 0;
@@ -2389,7 +2640,7 @@ static int ibmveth_is_packet_unsupported(struct sk_buff *skb,
if (ether_addr_equal(ether_header->h_dest, netdev->dev_addr)) {
netdev_dbg(netdev, "veth doesn't support loopback packets, dropping packet.\n");
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
ret = -EOPNOTSUPP;
}
@@ -2411,7 +2662,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
return NETDEV_TX_OK;
}
- if (ibmveth_is_packet_unsupported(skb, netdev))
+ if (ibmveth_is_packet_unsupported(skb, adapter, queue_num))
goto out;
/* veth can't checksum offload UDP */
if (skb->ip_summed == CHECKSUM_PARTIAL &&
@@ -2422,7 +2673,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
skb_checksum_help(skb)) {
netdev_err(netdev, "tx: failed to checksum packet\n");
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
@@ -2434,6 +2685,8 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
desc_flags |= (IBMVETH_BUF_NO_CSUM | IBMVETH_BUF_CSUM_GOOD);
+ adapter->tx_qstats[queue_num].checksum_offload++;
+
/* Need to zero out the checksum */
buf[0] = 0;
buf[1] = 0;
@@ -2445,7 +2698,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (skb->ip_summed == CHECKSUM_PARTIAL && skb_is_gso(skb)) {
if (adapter->fw_large_send_support) {
mss = (unsigned long)skb_shinfo(skb)->gso_size;
- adapter->tx_large_packets++;
+ adapter->tx_qstats[queue_num].large_packets++;
} else if (!skb_is_gso_v6(skb)) {
/* Put -1 in the IP checksum to tell phyp it
* is a largesend packet. Put the mss in
@@ -2454,7 +2707,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
ip_hdr(skb)->check = 0xffff;
tcp_hdr(skb)->check =
cpu_to_be16(skb_shinfo(skb)->gso_size);
- adapter->tx_large_packets++;
+ adapter->tx_qstats[queue_num].large_packets++;
}
}
@@ -2462,7 +2715,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (unlikely(skb->len > adapter->tx_ltb_size)) {
netdev_err(adapter->netdev, "tx: packet size (%u) exceeds ltb (%u)\n",
skb->len, adapter->tx_ltb_size);
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
@@ -2479,7 +2732,7 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
if (unlikely(total_bytes != skb->len)) {
netdev_err(adapter->netdev, "tx: incorrect packet len copied into ltb (%u != %u)\n",
skb->len, total_bytes);
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
goto out;
}
desc.fields.flags_len = desc_flags | skb->len;
@@ -2488,11 +2741,11 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
dma_wmb();
if (ibmveth_send(adapter, desc.desc, mss)) {
- adapter->tx_send_failed++;
- netdev->stats.tx_dropped++;
+ adapter->tx_qstats[queue_num].send_failures++;
+ adapter->tx_qstats[queue_num].dropped_packets++;
} else {
- netdev->stats.tx_packets++;
- netdev->stats.tx_bytes += skb->len;
+ adapter->tx_qstats[queue_num].packets++;
+ adapter->tx_qstats[queue_num].bytes += skb->len;
}
out:
@@ -2621,7 +2874,10 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
static void ibmveth_poll_bump_invalid(struct ibmveth_adapter *adapter,
int queue_index)
{
- adapter->rx_invalid_buffer++;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].invalid_buffers++;
+ else
+ adapter->rx_invalid_buffer++;
}
static bool ibmveth_poll_stopping(struct net_device *netdev,
@@ -2757,7 +3013,10 @@ static int ibmveth_poll_deliver_frame(struct napi_struct *napi,
if ((length > netdev->mtu + ETH_HLEN) || lrg_pkt ||
iph_check == 0xffff) {
ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
- adapter->rx_large_packets++;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].large_packets++;
+ else
+ adapter->rx_large_packets++;
}
if (csum_good) {
@@ -2767,8 +3026,10 @@ static int ibmveth_poll_deliver_frame(struct napi_struct *napi,
napi_gro_receive(napi, skb);
- netdev->stats.rx_packets++;
- netdev->stats.rx_bytes += length;
+ if (adapter->rx_qstats) {
+ adapter->rx_qstats[queue_index].packets++;
+ adapter->rx_qstats[queue_index].bytes += length;
+ }
return 1;
}
@@ -2793,6 +3054,9 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
return 0;
}
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[queue_index].polls++;
+
restart_poll:
while (frames_processed < budget) {
if (ibmveth_poll_stopping(netdev, napi))
@@ -2878,6 +3142,9 @@ static irqreturn_t ibmveth_interrupt(int irq, void *dev_instance)
if (qindex < 0 || qindex >= ibmveth_get_num_rx_queues(adapter))
return IRQ_NONE;
+ if (adapter->rx_qstats)
+ adapter->rx_qstats[qindex].interrupts++;
+
ibmveth_schedule_rx_queue(adapter, qindex);
return IRQ_HANDLED;
}
@@ -3095,6 +3362,42 @@ static netdev_features_t ibmveth_features_check(struct sk_buff *skb,
return vlan_features_check(skb, features);
}
+/**
+ * ibmveth_get_stats64 - Return aggregated per-queue statistics
+ * @dev: network device
+ * @stats: rtnl link statistics storage
+ *
+ * Sums per-queue rx_qstats and tx_qstats into the rtnl counters.
+ * Walk the full allocated arrays (not the live queue count) so shrinking
+ * channels cannot make the totals go backwards.
+ * Callers use ndo_get_stats64(); avoid updating netdev->stats on the
+ * xmit/poll paths to keep per-queue counters off the hot cache line.
+ */
+static void ibmveth_get_stats64(struct net_device *dev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ int i;
+
+ if (adapter->rx_qstats) {
+ for (i = 0; i < IBMVETH_MAX_RX_QUEUES; i++) {
+ stats->rx_packets += adapter->rx_qstats[i].packets;
+ stats->rx_bytes += adapter->rx_qstats[i].bytes;
+ }
+ }
+
+ if (adapter->tx_qstats) {
+ for (i = 0; i < IBMVETH_MAX_QUEUES; i++) {
+ stats->tx_packets += adapter->tx_qstats[i].packets;
+ stats->tx_bytes += adapter->tx_qstats[i].bytes;
+ stats->tx_dropped +=
+ adapter->tx_qstats[i].dropped_packets;
+ }
+ }
+
+ stats->tx_errors = dev->stats.tx_errors;
+}
+
static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_open = ibmveth_open,
.ndo_stop = ibmveth_close,
@@ -3107,6 +3410,7 @@ static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_validate_addr = eth_validate_addr,
.ndo_set_mac_address = ibmveth_set_mac_addr,
.ndo_features_check = ibmveth_features_check,
+ .ndo_get_stats64 = ibmveth_get_stats64,
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = ibmveth_poll_controller,
#endif
@@ -3121,6 +3425,23 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
kobject_put(&adapter->rx_buff_pool[0][i].kobj);
}
+static void ibmveth_probe_cleanup(struct ibmveth_adapter *adapter,
+ int pools_ready)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ cancel_work_sync(&adapter->work);
+ ibmveth_put_pool_kobjs(adapter, pools_ready);
+
+ ibmveth_free_tx_qstats(adapter);
+ ibmveth_free_rx_qstats(adapter);
+ /* Probe failure never reaches ibmveth_remove(); clear before free so
+ * CMO get_desired_dma() cannot see a freed netdev on rebind.
+ */
+ dev_set_drvdata(&adapter->vdev->dev, NULL);
+ free_netdev(netdev);
+}
+
static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
{
int rc, i, mac_len, pools_ready = 0;
@@ -3176,6 +3497,12 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netif_napi_add_weight(netdev, &adapter->napi[i],
ibmveth_poll, 16);
+ if (ibmveth_alloc_rx_qstats(adapter) ||
+ ibmveth_alloc_tx_qstats(adapter)) {
+ ibmveth_probe_cleanup(adapter, 0);
+ return -ENOMEM;
+ }
+
netdev->irq = dev->irq;
netdev->netdev_ops = &ibmveth_netdev_ops;
netdev->ethtool_ops = &netdev_ethtool_ops;
@@ -3257,9 +3584,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
"failed to create pool%d kobject: %d\n", i, rc);
/* init_and_add takes a ref even on failure */
kobject_put(kobj);
- ibmveth_put_pool_kobjs(adapter, pools_ready);
- dev_set_drvdata(&dev->dev, NULL);
- free_netdev(netdev);
+ ibmveth_probe_cleanup(adapter, pools_ready);
return rc;
}
@@ -3279,9 +3604,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
if (rc) {
netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n",
rc);
- ibmveth_put_pool_kobjs(adapter, pools_ready);
- dev_set_drvdata(&dev->dev, NULL);
- free_netdev(netdev);
+ ibmveth_probe_cleanup(adapter, pools_ready);
return rc;
}
@@ -3296,9 +3619,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
if (rc) {
netdev_dbg(netdev, "failed to set number of rx queues rc=%d\n",
rc);
- ibmveth_put_pool_kobjs(adapter, pools_ready);
- dev_set_drvdata(&dev->dev, NULL);
- free_netdev(netdev);
+ ibmveth_probe_cleanup(adapter, pools_ready);
return rc;
}
@@ -3315,9 +3636,7 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
if (rc) {
netdev_dbg(netdev, "failed to register netdev rc=%d\n", rc);
- ibmveth_put_pool_kobjs(adapter, pools_ready);
- dev_set_drvdata(&dev->dev, NULL);
- free_netdev(netdev);
+ ibmveth_probe_cleanup(adapter, pools_ready);
return rc;
}
@@ -3332,12 +3651,20 @@ static void ibmveth_remove(struct vio_dev *dev)
struct ibmveth_adapter *adapter = netdev_priv(netdev);
int i;
- cancel_work_sync(&adapter->work);
-
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
kobject_put(&adapter->rx_buff_pool[0][i].kobj);
+ /*
+ * Unregister first so NAPI/xmit cannot re-arm reset work after we
+ * cancel it. cancel_work_sync() before unregister left a window
+ * where poll could schedule_work() and the worker ran after
+ * free_netdev().
+ */
unregister_netdev(netdev);
+ cancel_work_sync(&adapter->work);
+
+ ibmveth_free_tx_qstats(adapter);
+ ibmveth_free_rx_qstats(adapter);
free_netdev(netdev);
dev_set_drvdata(&dev->dev, NULL);
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 1394ef97bd8d..0960448f53ea 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -284,6 +284,38 @@ struct ibmveth_hcall_stats {
u64 send_lan; /* H_SEND_LOGICAL_LAN */
};
+struct ibmveth_rx_queue_stats {
+ u64 packets;
+ u64 bytes;
+ u64 interrupts;
+ u64 polls;
+ u64 large_packets;
+ u64 invalid_buffers;
+ u64 no_buffer_drops;
+} ____cacheline_aligned_in_smp;
+
+struct ibmveth_tx_queue_stats {
+ u64 packets;
+ u64 bytes;
+ u64 large_packets;
+ u64 dropped_packets;
+ u64 send_failures;
+ u64 checksum_offload;
+} ____cacheline_aligned_in_smp;
+
+/*
+ * ethtool string count: use offsetof of the last counter so alignment
+ * padding from ____cacheline_aligned_in_smp is not counted. When adding
+ * a new counter at the end, point these at the new last field (same idea
+ * as sizeof(struct)/sizeof(u64) before alignment was added).
+ */
+#define IBMVETH_NUM_RX_QSTATS \
+ (offsetof(struct ibmveth_rx_queue_stats, no_buffer_drops) / \
+ sizeof(u64) + 1)
+#define IBMVETH_NUM_TX_QSTATS \
+ (offsetof(struct ibmveth_tx_queue_stats, checksum_offload) / \
+ sizeof(u64) + 1)
+
struct ibmveth_buff_pool {
u32 size;
u32 index;
@@ -349,6 +381,7 @@ struct ibmveth_adapter {
u64 replenish_add_buff_success;
u64 rx_invalid_buffer;
u64 rx_no_buffer;
+ u64 rx_no_buffer_retired; /* PHYP page reset / queue-reuse carry */
u64 tx_map_failed;
u64 tx_send_failed;
u64 tx_large_packets;
@@ -356,6 +389,9 @@ struct ibmveth_adapter {
/* Hypercall statistics */
struct ibmveth_hcall_stats hcall_stats;
+ struct ibmveth_rx_queue_stats *rx_qstats;
+ struct ibmveth_tx_queue_stats *tx_qstats;
+
/* Ethtool settings */
u8 duplex;
u32 speed;
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (10 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 11/15] ibmveth: Add per-queue RX and TX statistics collection Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 13/15] ibmveth: Expose per-queue buffer pool details via debugfs Mingming Cao
` (2 subsequent siblings)
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Report MQ-aware channel limits and counts from get_channels():
- max_rx is IBMVETH_MAX_RX_QUEUES when MQ firmware is enabled, else 1
- rx_count tracks the driver's published num_rx_queues
Previously get_channels() mirrored netdev->real_num_rx_queues for both
fields, which under-reports max_rx on MQ firmware and can disagree with
the driver's private RX count before the next open.
Keep this out of the stats patch: channel reporting is ethtool -l ABI,
independent of per-queue counters, so it can be reviewed and blamed on
its own.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- get_channels rx_count via get_num_rx_queues() (READ_ONCE consistency)
- New peel: get_channels from v4 stats -> tip P12 (14->15)
- get_channels: max_rx = MAX_RX_QUEUES when MQ else 1; rx_count =
get_num_rx_queues() (was mirroring real_num_rx_queues for both)
drivers/net/ethernet/ibm/ibmveth.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 3202b657c9b8..5d4ca4b3d3d0 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -2522,11 +2522,16 @@ static void ibmveth_get_ethtool_stats(struct net_device *dev,
static void ibmveth_get_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
+ struct ibmveth_adapter *adapter = netdev_priv(netdev);
+
channels->max_tx = ibmveth_real_max_tx_queues();
channels->tx_count = netdev->real_num_tx_queues;
- channels->max_rx = netdev->real_num_rx_queues;
- channels->rx_count = netdev->real_num_rx_queues;
+ if (adapter->multi_queue)
+ channels->max_rx = IBMVETH_MAX_RX_QUEUES;
+ else
+ channels->max_rx = 1;
+ channels->rx_count = ibmveth_get_num_rx_queues(adapter);
}
static int ibmveth_set_channels(struct net_device *netdev,
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 13/15] ibmveth: Expose per-queue buffer pool details via debugfs
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (11 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 14/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
2026-08-14 7:36 ` [PATCH net-next v5 15/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=true, Size: 5712 bytes --]
Add a read-only buffer_pools debugfs file that lists size, buff_size,
active, and available for every RX queue and buffer pool. That is the
MQ diagnostic view (multi-line table), which does not belong in sysfs.
Keep the historical queue-0 poolN sysfs ABI unchanged:
.../poolN/{active,num,size}
Those are one-value-per-file *configuration* knobs for buffer-pool
classes on queue 0, not one directory per RX queue. With MQ, queue 0
pool geometry remains the shared template: open copies it to queues
1..N. Per-queue runtime pressure is what debugfs shows; this series
does not add per-queue pool sysfs dirs.
Unlike ibmvnic (one RX pool per RX queue, sized from firmware, almost
no pool sysfs), ibmveth historically has multiple size-class pools and
already exported poolN via sysfs. Moving only the new all-queue dump
to debugfs matches sysfs "one value per file" review feedback without
breaking that config ABI.
Path: /sys/kernel/debug/ibmveth/<dev_name>/buffer_pools
(e.g. /sys/kernel/debug/ibmveth/30000002/buffer_pools)
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- debugfs buffer_pools_show walks get_num_rx_queues()
- Series renumber: mailed v4 11/14 debugfs -> tip P13 (14->15)
- Path uses stable vio dev_name under a driver-owned root (not netdev
name - avoids rename/collide)
- rtnl_lock around dump (writers are under RTNL)
- Show Active/Available as 0 when pool !live (debugfs view; free-path
available clear already in the buffer-submit patch)
Changes in v4:
- Move the all-queue buffer_pools diagnostic from sysfs to debugfs;
subject updated to match.
- Keep historical queue-0 poolN/{active,num,size} sysfs as one-value
config (template for MQ); do not add per-queue pool sysfs dirs.
drivers/net/ethernet/ibm/ibmveth.c | 69 ++++++++++++++++++++++++++++++
drivers/net/ethernet/ibm/ibmveth.h | 2 +
2 files changed, 71 insertions(+)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 5d4ca4b3d3d0..1b58a3c6ce77 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -31,6 +31,7 @@
#include <linux/ipv6.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
+#include <linux/debugfs.h>
#include <asm/hvcall.h>
#include <linux/atomic.h>
#include <asm/vio.h>
@@ -3421,6 +3422,68 @@ static const struct net_device_ops ibmveth_netdev_ops = {
#endif
};
+static int ibmveth_buffer_pools_show(struct seq_file *m, void *v)
+{
+ struct ibmveth_adapter *adapter = m->private;
+ int i, j;
+
+ /*
+ * Writers (veth_pool_store, open template copy, reset close/open)
+ * update these fields under RTNL. Take the same lock so the dump
+ * is not a torn scalar snapshot. Not required for
+ * memory safety — embedded arrays only.
+ */
+ rtnl_lock();
+
+ seq_puts(m, "Queue Pool Size BuffSize Active Available\n");
+ seq_puts(m, "----- ---- ---- -------- ------ ---------\n");
+ if (!adapter->opened)
+ seq_puts(m, "# down: Active/Available 0 unless allocated\n");
+
+ for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
+ for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
+ struct ibmveth_buff_pool *pool =
+ &adapter->rx_buff_pool[i][j];
+ bool live = pool->skbuff && pool->free_map;
+ int active = live ? pool->active : 0;
+ int available = live ? atomic_read(&pool->available)
+ : 0;
+
+ seq_printf(m, "%5d %4d %4u %8u %6d %9d\n",
+ i, j, pool->size, pool->buff_size,
+ active, available);
+ }
+ }
+
+ rtnl_unlock();
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(ibmveth_buffer_pools);
+
+/* Driver-owned root so per-adapter dirs use a stable vio name, not the
+ * mutable netdev->name (avoids stale names / eth0 collisions after rename).
+ */
+static struct dentry *ibmveth_dbg_root;
+
+static void ibmveth_debugfs_init(struct ibmveth_adapter *adapter)
+{
+ if (!ibmveth_dbg_root)
+ ibmveth_dbg_root =
+ debugfs_create_dir(ibmveth_driver_name, NULL);
+
+ adapter->debugfs_dir =
+ debugfs_create_dir(dev_name(&adapter->vdev->dev),
+ ibmveth_dbg_root);
+ debugfs_create_file("buffer_pools", 0400, adapter->debugfs_dir,
+ adapter, &ibmveth_buffer_pools_fops);
+}
+
+static void ibmveth_debugfs_exit(struct ibmveth_adapter *adapter)
+{
+ debugfs_remove_recursive(adapter->debugfs_dir);
+ adapter->debugfs_dir = NULL;
+}
+
static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
int pools_ready)
{
@@ -3647,6 +3710,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
netdev_dbg(netdev, "registered\n");
+ ibmveth_debugfs_init(adapter);
+
return 0;
}
@@ -3656,6 +3721,8 @@ static void ibmveth_remove(struct vio_dev *dev)
struct ibmveth_adapter *adapter = netdev_priv(netdev);
int i;
+ ibmveth_debugfs_exit(adapter);
+
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
kobject_put(&adapter->rx_buff_pool[0][i].kobj);
@@ -3891,6 +3958,8 @@ static int __init ibmveth_module_init(void)
static void __exit ibmveth_module_exit(void)
{
vio_unregister_driver(&ibmveth_driver);
+ debugfs_remove_recursive(ibmveth_dbg_root);
+ ibmveth_dbg_root = NULL;
}
module_init(ibmveth_module_init);
diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
index 0960448f53ea..8c826d23ddf4 100644
--- a/drivers/net/ethernet/ibm/ibmveth.h
+++ b/drivers/net/ethernet/ibm/ibmveth.h
@@ -392,6 +392,8 @@ struct ibmveth_adapter {
struct ibmveth_rx_queue_stats *rx_qstats;
struct ibmveth_tx_queue_stats *tx_qstats;
+ struct dentry *debugfs_dir;
+
/* Ethtool settings */
u8 duplex;
u32 speed;
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 14/15] ibmveth: Implement incremental MQ RX queue resize
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (12 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 13/15] ibmveth: Expose per-queue buffer pool details via debugfs Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 15/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=true, Size: 26860 bytes --]
Teardown first on ethtool -L scale-down: drain and deregister/destroy
queues being removed before unmapping their buffer pools. Freeing pool
state while stale correlators can still complete was the hang root
cause; poll-path skip helpers alone cannot paper over that ownership
hole.
Strengthen correlator liveness (active pool + non-NULL backing arrays)
and clear freed pool metadata so torn-down pools no longer look
in-range. Restore schedule_work escalation for truly impossible
correlators.
Also implement incremental scale-up/down helpers, WRITE_ONCE
num_rx_queues publication, rollback/CMO updates, and related resize
error paths.
After enable_irq on scale-up (and scale-down rollback unmask), kick
NAPI if the RXQ already has pending work. After successful scale-down
destroy, restart remaining queues so Q0 cannot stay masked with NAPI
idle.
On scale-up, raise CMO desired for the target RX queue count via
ibmveth_desired_dma_for_rxqs() before dma_map / alloc / replenish.
ibmveth_resize_rx_channels() rejects rx > 1 without MQ firmware
(-EOPNOTSUPP) and validates rx_count within 1..IBMVETH_MAX_RX_QUEUES
(-EINVAL). The set_channels wiring patch consumes this helper.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- resize / get_desired_dma read live count via get_num_rx_queues()
- Scale-down / scale-up-fail: remask+sync after napi_disable (poll may
re-arm while disable waits)
- Series renumber: mailed v4 12/14 resize -> tip P14 (14->15)
- Absorb kitchen-sink resize ownership: teardown-first scale-down
(drain/deregister before unmap) + live-pool correlator_valid /
schedule_work (cover has peel map)
- Ordered num_rx_queues publish + publish-before-destroy
- After unmask: kick if pending; enable_irq errno + rollback check;
raise CMO before scale-up allocs
- Scale-up enable_irq-fail: publish-down / napi_disable / drain / destroy
- After scale-down destroy, restart remaining RX queues so Q0 cannot stay
masked with NAPI idle
- resize_rx_channels: reject rx>1 without MQ / validate 1..MAX; call
before !opened early return (stash still tip P15)
Changes in v4:
- Copy pool->index when cloning buffer pools for incrementally added
queues.
- Scale-up order: publish -> replenish -> napi_enable -> enable_irq
(vs older enable-before-publish drafts); mirror NAPI-before-unmask on
set_real_num_rx rollback.
- Linux-owned subordinate virq disposal; deregister is PHYP-only;
dispose bound to MAX_RX_QUEUES; dispose on request_irq failure.
- Drain-path smp_rmb() before harvest.
- Mask PHYP before napi_disable/drain on scale-down and scale-up fail
cleanup.
- Replenish before re-enabling IRQ/NAPI on set_real_num_rx scale-down
rollback.
- Keep set_channels wiring as the following patch (same split as v3)
but call resize_rx_channels() here when IFF_UP so the helper is not
an unused static.
drivers/net/ethernet/ibm/ibmveth.c | 682 +++++++++++++++++++++++++++--
1 file changed, 646 insertions(+), 36 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 1b58a3c6ce77..84f4a0deb0c5 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -696,6 +696,58 @@ ibmveth_cleanup_rx_interrupts(struct ibmveth_adapter *adapter)
adapter->rx_irq_setup = false;
}
+/**
+ * ibmveth_setup_single_rx_interrupt - Setup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to setup
+ *
+ * Registers the IRQ handler for one queue. Used during incremental
+ * scale-up when adding new RX queues. The caller publishes the queue,
+ * replenishes buffers, enables NAPI, then unmasks PHYP delivery.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_setup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ struct net_device *netdev = adapter->netdev;
+ int rc;
+
+ rc = request_irq(adapter->queue_irq[queue_idx], ibmveth_interrupt,
+ 0, netdev->name, &adapter->napi[queue_idx]);
+ if (rc) {
+ netdev_err(netdev, "request_irq() failed for queue %d: %d\n",
+ queue_idx, rc);
+ return rc;
+ }
+
+ netdev_dbg(netdev, "Setup IRQ %d for queue %d\n",
+ adapter->queue_irq[queue_idx], queue_idx);
+ return 0;
+}
+
+/**
+ * ibmveth_cleanup_single_rx_interrupt - Cleanup interrupt for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to cleanup
+ *
+ * Frees the IRQ handler for one queue and releases the subordinate virq
+ * mapping. Used during incremental scale-down.
+ */
+static void
+ibmveth_cleanup_single_rx_interrupt(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ if (adapter->queue_irq[queue_idx]) {
+ free_irq(adapter->queue_irq[queue_idx],
+ &adapter->napi[queue_idx]);
+ ibmveth_dispose_subordinate_irq_mapping(adapter, queue_idx);
+ netdev_dbg(adapter->netdev,
+ "Freed IRQ for queue %d\n", queue_idx);
+ }
+}
+
/**
* ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue
* @adapter: ibmveth adapter structure
@@ -1380,6 +1432,141 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
ibmveth_get_num_rx_queues(adapter));
}
+/**
+ * ibmveth_alloc_single_rx_queue - Allocate resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to allocate
+ * @rxq_entries: Number of RX queue entries
+ *
+ * Allocates buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-up without affecting existing queues.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_alloc_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx,
+ int rxq_entries)
+{
+ struct device *dev = &adapter->vdev->dev;
+ struct net_device *netdev = adapter->netdev;
+ int i, rc = -ENOMEM;
+
+ adapter->buffer_list_addr[queue_idx] =
+ (void *)get_zeroed_page(GFP_KERNEL);
+ if (!adapter->buffer_list_addr[queue_idx]) {
+ netdev_err(netdev, "unable to allocate buffer list for queue %d\n",
+ queue_idx);
+ return -ENOMEM;
+ }
+
+ adapter->rx_queue[queue_idx].queue_len =
+ sizeof(struct ibmveth_rx_q_entry) * rxq_entries;
+ adapter->rx_queue[queue_idx].queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ &adapter->rx_queue[queue_idx].queue_dma,
+ GFP_KERNEL);
+ if (!adapter->rx_queue[queue_idx].queue_addr) {
+ netdev_err(netdev, "unable to allocate RX queue for queue %d\n",
+ queue_idx);
+ goto out_free_buflist;
+ }
+
+ adapter->buffer_list_dma[queue_idx] =
+ dma_map_single(dev, adapter->buffer_list_addr[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, adapter->buffer_list_dma[queue_idx])) {
+ netdev_err(netdev, "unable to map buffer list for queue %d\n",
+ queue_idx);
+ goto out_free_rxq;
+ }
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *src =
+ &adapter->rx_buff_pool[0][i];
+ struct ibmveth_buff_pool *dst =
+ &adapter->rx_buff_pool[queue_idx][i];
+
+ dst->size = src->size;
+ dst->index = src->index;
+ dst->buff_size = src->buff_size;
+ dst->threshold = src->threshold;
+ dst->active = src->active;
+ }
+
+ rc = ibmveth_alloc_queue_buffer_pools(adapter, queue_idx);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to allocate buffer pools for queue %d\n",
+ queue_idx);
+ goto out_unmap_buflist;
+ }
+
+ adapter->rx_queue[queue_idx].index = 0;
+ adapter->rx_queue[queue_idx].num_slots = rxq_entries;
+ adapter->rx_queue[queue_idx].toggle = 1;
+ spin_lock_init(&adapter->rx_queue[queue_idx].replenish_lock);
+
+ netdev_dbg(netdev,
+ "Allocated queue %d: buffer_list @ %p (DMA: 0x%llx), rx_queue @ %p (DMA: 0x%llx), %d entries\n",
+ queue_idx, adapter->buffer_list_addr[queue_idx],
+ (unsigned long long)adapter->buffer_list_dma[queue_idx],
+ adapter->rx_queue[queue_idx].queue_addr,
+ (unsigned long long)adapter->rx_queue[queue_idx].queue_dma,
+ rxq_entries);
+
+ return 0;
+
+out_unmap_buflist:
+ dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[queue_idx] = 0;
+out_free_rxq:
+ dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ adapter->rx_queue[queue_idx].queue_addr,
+ adapter->rx_queue[queue_idx].queue_dma);
+ adapter->rx_queue[queue_idx].queue_addr = NULL;
+out_free_buflist:
+ free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+ adapter->buffer_list_addr[queue_idx] = NULL;
+ return rc;
+}
+
+/**
+ * ibmveth_free_single_rx_queue - Free resources for a single RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to free
+ *
+ * Frees buffer list, RX queue, and per-queue buffer pools for one queue.
+ * Used during incremental scale-down without affecting remaining queues.
+ */
+static void
+ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
+{
+ struct device *dev = &adapter->vdev->dev;
+
+ ibmveth_free_queue_buffer_pools(adapter, queue_idx);
+
+ if (adapter->buffer_list_dma[queue_idx]) {
+ dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
+ 4096, DMA_BIDIRECTIONAL);
+ adapter->buffer_list_dma[queue_idx] = 0;
+ }
+
+ if (adapter->rx_queue[queue_idx].queue_addr) {
+ dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
+ adapter->rx_queue[queue_idx].queue_addr,
+ adapter->rx_queue[queue_idx].queue_dma);
+ adapter->rx_queue[queue_idx].queue_addr = NULL;
+ }
+
+ if (adapter->buffer_list_addr[queue_idx]) {
+ free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
+ adapter->buffer_list_addr[queue_idx] = NULL;
+ }
+
+ netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
+}
+
static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
int queue_index, u64 correlator)
{
@@ -1542,6 +1729,56 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
return 0;
}
+/**
+ * ibmveth_drain_rx_queue - Drain pending buffers from an RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_index: Queue index to drain
+ *
+ * Recycles all pending buffers back to the per-queue buffer pools.
+ * Must be called with NAPI disabled for this queue.
+ *
+ * Return: Number of buffers drained
+ */
+static int
+ibmveth_drain_rx_queue(struct ibmveth_adapter *adapter, int queue_index)
+{
+ struct net_device *netdev = adapter->netdev;
+ int drained = 0;
+ int limit = adapter->rx_queue[queue_index].num_slots;
+ int rc;
+
+ netdev_dbg(netdev, "Draining RX queue %d (limit: %d slots)\n",
+ queue_index, limit);
+
+ while (drained < limit &&
+ ibmveth_rxq_pending_buffer(adapter, queue_index)) {
+ /* Match poll-side order before harvesting completion state. */
+ smp_rmb();
+ rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, true);
+ if (rc) {
+ /* -EINVAL/-EFAULT already advanced past the slot. */
+ if (rc == -EINVAL || rc == -EFAULT) {
+ drained++;
+ continue;
+ }
+ netdev_err(netdev,
+ "Failed to harvest buffer from queue %d during drain: %d\n",
+ queue_index, rc);
+ break;
+ }
+ drained++;
+ }
+
+ if (drained > 0)
+ netdev_dbg(netdev, "Drained %d buffer(s) from RX queue %d\n",
+ drained, queue_index);
+ else
+ netdev_dbg(netdev, "No buffers to drain from RX queue %d\n",
+ queue_index);
+
+ return drained;
+}
+
static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
{
if (!adapter->tx_ltb_ptr[idx])
@@ -1797,6 +2034,356 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
return 0;
}
+/**
+ * ibmveth_deregister_single_rx_queue - Deregister one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to deregister (1..N)
+ *
+ * Deregisters a single queue via H_FREE_LOGICAL_LAN_QUEUE. Linux IRQ handler
+ * teardown and subordinate virq mapping disposal are owned by interrupt
+ * cleanup helpers; queue 0 is freed only through ibmveth_free_all_queues()
+ * (H_FREE_LOGICAL_LAN).
+ */
+static void
+ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ unsigned long lpar_rc;
+ unsigned long ua = adapter->vdev->unit_address;
+ unsigned long qh = adapter->queue_handle[queue_idx];
+
+ if (!qh)
+ return;
+
+ do {
+ lpar_rc = h_free_logical_lan_queue(ua, qh);
+ } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
+
+ adapter->hcall_stats.free_lan_queue++;
+
+ if (lpar_rc != H_SUCCESS) {
+ netdev_err(adapter->netdev,
+ "h_free_logical_lan_queue failed for queue %d: rc=0x%lx\n",
+ queue_idx, lpar_rc);
+ }
+
+ adapter->queue_handle[queue_idx] = 0;
+
+ netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
+}
+
+/**
+ * ibmveth_destroy_subordinate_rx_queue - Tear down one subordinate RX queue
+ * @adapter: ibmveth adapter structure
+ * @queue_idx: Queue index to destroy (1..N)
+ *
+ * Deregister with PHYP before unmapping buffer pools so hypervisor buffer
+ * ownership is released while queue metadata is still valid.
+ */
+static void
+ibmveth_destroy_subordinate_rx_queue(struct ibmveth_adapter *adapter,
+ int queue_idx)
+{
+ ibmveth_deregister_single_rx_queue(adapter, queue_idx);
+ ibmveth_cleanup_single_rx_interrupt(adapter, queue_idx);
+ ibmveth_free_single_rx_queue(adapter, queue_idx);
+}
+
+/**
+ * ibmveth_desired_dma_for_rxqs - CMO entitlement for a given RX queue count
+ * @adapter: ibmveth adapter
+ * @rxqs: number of RX queues to size for
+ *
+ * Same math as ibmveth_get_desired_dma(), but uses @rxqs instead of the
+ * live adapter->num_rx_queues. Scale-up raises desired for the *target*
+ * count before allocating so vio_cmo_alloc cannot fail mid-resize.
+ *
+ * Return: bytes of IO memory desired for @rxqs RX queues
+ */
+static unsigned long
+ibmveth_desired_dma_for_rxqs(struct ibmveth_adapter *adapter,
+ unsigned int rxqs)
+{
+ struct net_device *netdev = adapter->netdev;
+ struct iommu_table *tbl;
+ unsigned long ret;
+ int i, q;
+
+ tbl = get_iommu_table_base(&adapter->vdev->dev);
+
+ ret = IBMVETH_BUFF_LIST_SIZE * rxqs + IBMVETH_FILT_LIST_SIZE;
+ ret += IOMMU_PAGE_ALIGN(netdev->mtu, tbl);
+ ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
+
+ /*
+ * Pool metadata for queues 1+ is copied from queue 0 at open.
+ * Always size from pool 0 x @rxqs (valid at probe before that copy).
+ */
+ for (q = 0; q < rxqs; q++) {
+ int rxqentries = 1;
+
+ for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
+ struct ibmveth_buff_pool *bpool =
+ &adapter->rx_buff_pool[0][i];
+
+ if (bpool->active)
+ ret += bpool->size *
+ IOMMU_PAGE_ALIGN(bpool->buff_size, tbl);
+ rxqentries += bpool->size;
+ }
+
+ ret += IOMMU_PAGE_ALIGN(rxqentries *
+ sizeof(struct ibmveth_rx_q_entry), tbl);
+ }
+
+ return ret;
+}
+
+/**
+ * ibmveth_resize_rx_queues_incremental - Resize RX queue count incrementally
+ * @adapter: ibmveth adapter structure
+ * @new_count: Target number of RX queues
+ * @rxq_entries: Number of entries per RX queue
+ *
+ * Adds or removes RX queues without tearing down the entire adapter.
+ * Active queues continue receiving during scale-up; scale-down drains
+ * excess queues before deregistering them with the hypervisor.
+ *
+ * Return: 0 on success, negative error code on failure
+ */
+static int
+ibmveth_resize_rx_queues_incremental(struct ibmveth_adapter *adapter,
+ int new_count, int rxq_entries)
+{
+ struct net_device *netdev = adapter->netdev;
+ int old_count = ibmveth_get_num_rx_queues(adapter);
+ int failed_queue;
+ int rc, i;
+
+ if (old_count == new_count) {
+ netdev_dbg(netdev, "RX queue count unchanged (%d), nothing to do\n",
+ old_count);
+ return 0;
+ }
+
+ if (new_count < 1 || new_count > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev, "Invalid RX queue count %d (must be 1-%d)\n",
+ new_count, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ netdev_info(netdev, "Incrementally resizing RX queues: %d to %d\n",
+ old_count, new_count);
+
+ if (new_count > old_count) {
+ netdev_dbg(netdev, "Scale-up: adding queues %d-%d\n",
+ old_count, new_count - 1);
+
+ /*
+ * Raise CMO desired for the target count before dma_map /
+ * dma_alloc_coherent / replenish (same order as change_mtu).
+ * Do not bump live num_rx_queues here — only entitlement.
+ */
+ if (firmware_has_feature(FW_FEATURE_CMO)) {
+ unsigned long dma;
+
+ dma = ibmveth_desired_dma_for_rxqs(adapter, new_count);
+ vio_cmo_set_dev_desired(adapter->vdev, dma);
+ }
+
+ for (i = old_count; i < new_count; i++) {
+ rc = ibmveth_alloc_single_rx_queue(adapter, i,
+ rxq_entries);
+ if (rc) {
+ netdev_err(netdev, "Failed to allocate queue %d: %d\n",
+ i, rc);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_register_single_rx_queue(adapter, i);
+ if (rc) {
+ netdev_err(netdev, "Failed to register queue %d: %d\n",
+ i, rc);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ rc = ibmveth_setup_single_rx_interrupt(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to setup IRQ for queue %d: %d\n",
+ i, rc);
+ /* request_irq failed: mapped but no handler */
+ ibmveth_dispose_subordinate_irq_mapping(adapter,
+ i);
+ ibmveth_deregister_single_rx_queue(adapter, i);
+ ibmveth_free_single_rx_queue(adapter, i);
+ goto cleanup_new_queues;
+ }
+
+ /*
+ * Fully ready before PHYP delivery, matching open():
+ * publish -> replenish -> napi_enable -> enable_irq.
+ * That way ibmveth_interrupt() cannot run on an
+ * unpublished, empty, or NAPI-disabled queue.
+ */
+ ibmveth_publish_num_rx_queues(adapter, i + 1);
+ ibmveth_replenish_task(adapter, i);
+ napi_enable(&adapter->napi[i]);
+
+ rc = ibmveth_enable_irq(adapter, i);
+ if (rc) {
+ netdev_err(netdev,
+ "Failed to enable IRQ for queue %d: %d\n",
+ i, rc);
+ /*
+ * Published, replenished, and NAPI-enabled,
+ * but PHYP never unmasked. Match scale-down /
+ * shared cleanup: drain posted buffers, then
+ * deregister before unmap via
+ * destroy_subordinate.
+ */
+ ibmveth_publish_num_rx_queues(adapter, i);
+ napi_disable(&adapter->napi[i]);
+ ibmveth_drain_rx_queue(adapter, i);
+ synchronize_net();
+ ibmveth_destroy_subordinate_rx_queue(adapter,
+ i);
+ /* enable_irq returns errno; keep -EIO. */
+ rc = -EIO;
+ goto cleanup_new_queues;
+ }
+ ibmveth_kick_rx_queue_if_pending(adapter, i);
+ }
+
+ rc = netif_set_real_num_rx_queues(netdev, new_count);
+ if (rc) {
+ netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+ new_count, rc);
+ goto cleanup_new_queues;
+ }
+ } else {
+ netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
+ new_count, old_count - 1);
+
+ /*
+ * Mask PHYP before napi_disable so the handler cannot miss
+ * a mask while NAPI is already dead. An in-flight poll can
+ * still re-arm PHYP while napi_disable() waits, so remask
+ * and sync again after NAPI is stopped. Then drain and drop
+ * the live count before freeing so netpoll cannot walk dying
+ * queues (handler may still be registered until destroy).
+ */
+ for (i = new_count; i < old_count; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = new_count; i < old_count; i++)
+ napi_disable(&adapter->napi[i]);
+
+ for (i = new_count; i < old_count; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = new_count; i < old_count; i++)
+ ibmveth_drain_rx_queue(adapter, i);
+
+ synchronize_net();
+
+ ibmveth_publish_num_rx_queues(adapter, new_count);
+
+ rc = netif_set_real_num_rx_queues(netdev, new_count);
+ if (rc) {
+ netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
+ new_count, rc);
+ ibmveth_publish_num_rx_queues(adapter, old_count);
+ for (i = new_count; i < old_count; i++) {
+ int irq_rc;
+
+ ibmveth_replenish_task(adapter, i);
+ /* START: NAPI before PHYP unmask. */
+ napi_enable(&adapter->napi[i]);
+ irq_rc = ibmveth_enable_irq(adapter, i);
+ if (irq_rc) {
+ netdev_err(netdev,
+ "Failed to re-enable IRQ for queue %d during scale-down rollback (rc=%d), scheduling reset\n",
+ i, irq_rc);
+ schedule_work(&adapter->work);
+ continue;
+ }
+ ibmveth_kick_rx_queue_if_pending(adapter, i);
+ }
+ return rc;
+ }
+
+ for (i = new_count; i < old_count; i++)
+ ibmveth_destroy_subordinate_rx_queue(adapter, i);
+
+ /* Q0 (and any still-live queues) were not in the teardown
+ * loop. Scale-down can leave Q0 masked with NAPI idle — kick
+ * so ARP/RX does not die until rmmod (lab: -L rx 1).
+ */
+ for (i = 0; i < new_count; i++)
+ ibmveth_restart_rx_queue(adapter, i);
+ }
+
+ netdev_info(netdev, "Successfully resized to %d RX queues (incremental)\n",
+ ibmveth_get_num_rx_queues(adapter));
+
+ if (firmware_has_feature(FW_FEATURE_CMO))
+ vio_cmo_set_dev_desired(adapter->vdev,
+ ibmveth_get_desired_dma(adapter->vdev));
+
+ return 0;
+
+cleanup_new_queues:
+ failed_queue = i;
+ if (failed_queue > old_count)
+ netdev_err(netdev,
+ "Scale-up failed at queue %d, cleaning up queues %d-%d\n",
+ failed_queue, old_count, failed_queue - 1);
+ else
+ netdev_err(netdev,
+ "Scale-up failed at queue %d, nothing to clean up\n",
+ failed_queue);
+
+ for (i = old_count; i < failed_queue; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = old_count; i < failed_queue; i++)
+ napi_disable(&adapter->napi[i]);
+
+ /* Same remask as scale-down: poll may have re-armed during disable. */
+ for (i = old_count; i < failed_queue; i++) {
+ ibmveth_disable_irq(adapter, i);
+ synchronize_irq(adapter->queue_irq[i]);
+ }
+
+ for (i = old_count; i < failed_queue; i++)
+ ibmveth_drain_rx_queue(adapter, i);
+
+ synchronize_net();
+
+ /* Drop the live count before freeing the half-added queues. */
+ ibmveth_publish_num_rx_queues(adapter, old_count);
+
+ for (i = old_count; i < failed_queue; i++)
+ ibmveth_destroy_subordinate_rx_queue(adapter, i);
+
+ /* Roll CMO desired back to the surviving queue count. */
+ if (firmware_has_feature(FW_FEATURE_CMO))
+ vio_cmo_set_dev_desired(adapter->vdev,
+ ibmveth_get_desired_dma(adapter->vdev));
+
+ netdev_warn(netdev, "Keeping %d queues after scale-up failure\n",
+ old_count);
+ return rc;
+}
+
/**
* ibmveth_free_all_queues - Free all RX queues at once
* @adapter: ibmveth adapter structure
@@ -2535,6 +3122,55 @@ static void ibmveth_get_channels(struct net_device *netdev,
channels->rx_count = ibmveth_get_num_rx_queues(adapter);
}
+/**
+ * ibmveth_resize_rx_channels - Validate and apply a new RX queue count
+ * @adapter: ibmveth adapter
+ * @goal_rx: requested RX queue count
+ *
+ * Rejects rx > 1 without MQ firmware (-EOPNOTSUPP) and rx outside
+ * 1..IBMVETH_MAX_RX_QUEUES (-EINVAL). When RX resources are not live
+ * (!opened), only validate — the caller stashes the desired count.
+ * When up, apply via ibmveth_resize_rx_queues_incremental().
+ *
+ * Return: 0 or negative errno
+ */
+static int ibmveth_resize_rx_channels(struct ibmveth_adapter *adapter,
+ unsigned int goal_rx)
+{
+ struct net_device *netdev = adapter->netdev;
+ unsigned int old_rx = ibmveth_get_num_rx_queues(adapter);
+ int rxq_entries;
+ int rc;
+
+ if (goal_rx > 1 && !adapter->multi_queue) {
+ netdev_err(netdev,
+ "Cannot resize to %u RX queues: multi-queue mode not supported by firmware\n",
+ goal_rx);
+ return -EOPNOTSUPP;
+ }
+
+ if (goal_rx < 1 || goal_rx > IBMVETH_MAX_RX_QUEUES) {
+ netdev_err(netdev,
+ "Invalid RX queue count %u (must be 1-%d)\n",
+ goal_rx, IBMVETH_MAX_RX_QUEUES);
+ return -EINVAL;
+ }
+
+ if (goal_rx == old_rx)
+ return 0;
+
+ /* Down / failed-open: caller stashes. Do not allocate here. */
+ if (!adapter->opened)
+ return 0;
+
+ rxq_entries = adapter->rx_queue[0].num_slots;
+ rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
+ rxq_entries);
+ if (rc)
+ netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
+ return rc;
+}
+
static int ibmveth_set_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
@@ -2543,10 +3179,15 @@ static int ibmveth_set_channels(struct net_device *netdev,
goal = channels->tx_count;
int rc, i;
- /* If ndo_open has not been called yet then don't allocate, just set
- * desired netdev_queue's and return
+ /* Validate RX (and resize when opened) before the down-path early
+ * return so MQ/range errors are not deferred to the wiring patch.
+ * RX stash + CMO while down still lands with that patch.
*/
- if (!(netdev->flags & IFF_UP))
+ rc = ibmveth_resize_rx_channels(adapter, channels->rx_count);
+ if (rc)
+ return rc;
+
+ if (!adapter->opened)
return netif_set_real_num_tx_queues(netdev, goal);
/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
@@ -3281,8 +3922,6 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
struct net_device *netdev = dev_get_drvdata(&vdev->dev);
struct ibmveth_adapter *adapter;
struct iommu_table *tbl;
- unsigned long ret;
- int i, q;
tbl = get_iommu_table_base(&vdev->dev);
@@ -3291,38 +3930,9 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
return IOMMU_PAGE_ALIGN(IBMVETH_IO_ENTITLEMENT_DEFAULT, tbl);
adapter = netdev_priv(netdev);
+ return ibmveth_desired_dma_for_rxqs(adapter,
+ ibmveth_get_num_rx_queues(adapter));
- /* One buffer list page per RX queue; filter list is shared. */
- ret = IBMVETH_BUFF_LIST_SIZE * ibmveth_get_num_rx_queues(adapter) +
- IBMVETH_FILT_LIST_SIZE;
- ret += IOMMU_PAGE_ALIGN(netdev->mtu, tbl);
- /* add size of mapped tx buffers */
- ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
-
- /*
- * Pool metadata for queues 1+ is copied from queue 0 at open.
- * Always size from pool 0 x num_rx_queues (valid at probe before that
- * copy).
- */
- for (q = 0; q < ibmveth_get_num_rx_queues(adapter); q++) {
- int rxqentries = 1;
-
- for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
- struct ibmveth_buff_pool *bpool =
- &adapter->rx_buff_pool[0][i];
-
- if (bpool->active)
- ret += bpool->size *
- IOMMU_PAGE_ALIGN(bpool->buff_size, tbl);
- rxqentries += bpool->size;
- }
-
- /* add the size of the receive queue entries */
- ret += IOMMU_PAGE_ALIGN(rxqentries *
- sizeof(struct ibmveth_rx_q_entry), tbl);
- }
-
- return ret;
}
static int ibmveth_set_mac_addr(struct net_device *dev, void *p)
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH net-next v5 15/15] ibmveth: Wire ethtool set_channels to MQ RX queue resize
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
` (13 preceding siblings ...)
2026-08-14 7:36 ` [PATCH net-next v5 14/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
@ 2026-08-14 7:36 ` Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
14 siblings, 1 reply; 31+ messages in thread
From: Mingming Cao @ 2026-08-14 7:36 UTC (permalink / raw)
To: netdev
Cc: davem, kuba, edumazet, pabeni, andrew+netdev, nnac123, maddy, mpe,
linuxppc-dev, haren, ricklind, davemarq, bjking1, shaik.abdulla1,
Mingming Cao
Complete set_channels() RX handling around ibmveth_resize_rx_channels():
- When down: set TX queues first, then stash the desired RX count
in adapter->num_rx_queues for the next open() (open publishes via
netif_set_real_num_rx_queues). There are no RX mappings/buffers/IRQs
to grow or shrink while down, so do not allocate immediately.
- When up: resize RX, then adjust TX LTBs with the existing
stop/alloc/set_real_num_tx/free/wake path.
- Skip TX stop/wake when the TX count is unchanged.
Make TX/RX updates fail atomically: validate both ranges up front;
apply TX before stashing RX while down; when up, roll RX back if TX
cannot reach the requested count. Initialize the TX alloc loop bound
so a scale-down path that never allocates still has defined cleanup
if set_real_num_tx_queues() fails.
Gate live RX resize on adapter->opened (not IFF_UP): RX resources
exist only after a successful open, matching the while-down stash path.
Signed-off-by: Mingming Cao <mmc@linux.ibm.com>
Reviewed-by: Dave Marquardt <davemarq@linux.ibm.com>
Tested-by: Shaik Abdulla <shaik.abdulla1@ibm.com>
---
Changes in v5:
- set_channels / resize_rx_channels read via get_num_rx_queues()
- Series renumber: mailed v4 13/14 set_channels -> tip P15 (end; 14->15)
- set_channels: gate on adapter->opened (not IFF_UP) for live RX resize
vs while-down stash
- Down-path RX stash also refreshes CMO (publish + set_real_num_rx)
- When up: resize RX then TX; roll RX back if TX cannot reach goal
Changes in v4:
- On !IFF_UP, stash num_rx_queues only after TX set succeeds; do not
allocate live subordinate IRQs/buffers while down.
- Initialize i = old_tx on the TX adjust path.
- Always return rc from set_channels().
- Split from the resize-helper patch (same split as v3) while keeping
a live caller of resize_rx_channels() in the previous patch.
drivers/net/ethernet/ibm/ibmveth.c | 107 +++++++++++++++++++++++------
1 file changed, 85 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 84f4a0deb0c5..35cbd2553e86 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -800,6 +800,7 @@ ibmveth_kick_rx_queue_if_pending(struct ibmveth_adapter *adapter,
if (ibmveth_rxq_pending_buffer(adapter, queue_index))
ibmveth_schedule_rx_queue(adapter, queue_index);
}
+
/* setup the initial settings for a buffer pool */
static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
u32 pool_index, u32 pool_size,
@@ -3175,28 +3176,75 @@ static int ibmveth_set_channels(struct net_device *netdev,
struct ethtool_channels *channels)
{
struct ibmveth_adapter *adapter = netdev_priv(netdev);
- unsigned int old = netdev->real_num_tx_queues,
- goal = channels->tx_count;
+ unsigned int old_rx = ibmveth_get_num_rx_queues(adapter);
+ unsigned int goal_rx = channels->rx_count;
+ unsigned int old_tx = netdev->real_num_tx_queues;
+ unsigned int goal_tx = channels->tx_count;
+ unsigned int want_tx = goal_tx;
+ int rxq_entries = adapter->rx_queue[0].num_slots;
+ bool rx_changed = false;
int rc, i;
- /* Validate RX (and resize when opened) before the down-path early
- * return so MQ/range errors are not deferred to the wiring patch.
- * RX stash + CMO while down still lands with that patch.
- */
- rc = ibmveth_resize_rx_channels(adapter, channels->rx_count);
+ if (goal_tx < 1 || goal_tx > ibmveth_real_max_tx_queues()) {
+ netdev_err(netdev,
+ "Invalid TX queue count %u (must be 1-%u)\n",
+ goal_tx, ibmveth_real_max_tx_queues());
+ return -EINVAL;
+ }
+
+ /* RX range / MQ checks live in ibmveth_resize_rx_channels(). */
+ rc = ibmveth_resize_rx_channels(adapter, goal_rx);
if (rc)
return rc;
- if (!adapter->opened)
- return netif_set_real_num_tx_queues(netdev, goal);
+ /* If RX resources are not live (never opened, or close+open failed
+ * while IFF_UP stayed set), only stash desired queue counts.
+ */
+ if (!adapter->opened) {
+ /* Apply TX first so a failure leaves RX stash unchanged. */
+ rc = netif_set_real_num_tx_queues(netdev, goal_tx);
+ if (rc)
+ return rc;
+
+ /* Stash desired RX count; open() publishes it via
+ * netif_set_real_num_rx_queues() after queue registration.
+ * Refresh CMO now so open() can map the larger footprint;
+ * open itself does not call vio_cmo_set_dev_desired.
+ */
+ if (goal_rx != ibmveth_get_num_rx_queues(adapter)) {
+ ibmveth_publish_num_rx_queues(adapter, goal_rx);
+ rc = netif_set_real_num_rx_queues(netdev, goal_rx);
+ if (rc) {
+ ibmveth_publish_num_rx_queues(adapter, old_rx);
+ return rc;
+ }
+ if (firmware_has_feature(FW_FEATURE_CMO)) {
+ unsigned long dma;
+
+ dma = ibmveth_get_desired_dma(adapter->vdev);
+ vio_cmo_set_dev_desired(adapter->vdev, dma);
+ }
+ }
+ return 0;
+ }
+
+ if (goal_rx != old_rx)
+ rx_changed = true;
/* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
* but we may need to alloc/free the ltb's.
*/
+ if (goal_tx == old_tx)
+ return 0;
+
netif_tx_stop_all_queues(netdev);
- /* Allocate any queue that we need */
- for (i = old; i < goal; i++) {
+ /* Allocate any new TX LTBs. i starts at old_tx for the free walk
+ * below when this loop body never runs (goal_tx == old_tx already
+ * returned; goal_tx < old_tx is scale-down).
+ */
+ i = old_tx;
+ for (; i < goal_tx; i++) {
if (adapter->tx_ltb_ptr[i])
continue;
@@ -3205,28 +3253,43 @@ static int ibmveth_set_channels(struct net_device *netdev,
continue;
/* if something goes wrong, free everything we just allocated */
- netdev_err(netdev, "Failed to allocate more tx queues, returning to %d queues\n",
- old);
- goal = old;
- old = i;
+ netdev_err(netdev, "Failed to allocate more tx queues, returning to %u queues\n",
+ old_tx);
+ goal_tx = old_tx;
+ old_tx = i;
break;
}
- rc = netif_set_real_num_tx_queues(netdev, goal);
+ rc = netif_set_real_num_tx_queues(netdev, goal_tx);
if (rc) {
- netdev_err(netdev, "Failed to set real tx queues, returning to %d queues\n",
- old);
- goal = old;
- old = i;
+ netdev_err(netdev, "Failed to set real tx queues, returning to %u queues\n",
+ old_tx);
+ goal_tx = old_tx;
+ old_tx = i;
}
/* Free any that are no longer needed */
- for (i = old; i > goal; i--) {
+ for (i = old_tx; i > goal_tx; i--) {
if (adapter->tx_ltb_ptr[i - 1])
ibmveth_free_tx_ltb(adapter, i - 1);
}
netif_tx_wake_all_queues(netdev);
- return rc;
+ if (netdev->real_num_tx_queues != want_tx) {
+ if (rx_changed) {
+ int rb;
+
+ rb = ibmveth_resize_rx_queues_incremental(adapter,
+ old_rx,
+ rxq_entries);
+ if (rb)
+ netdev_err(netdev,
+ "Failed to roll back RX queues to %u after TX failure: %d\n",
+ old_rx, rb);
+ }
+ return rc ? rc : -ENOMEM;
+ }
+
+ return 0;
}
static const struct ethtool_ops netdev_ethtool_ops = {
--
2.50.1 (Apple Git-155)
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions
2026-08-14 7:36 ` [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add MQ RX hypercall wrappers and call definitions
This adds the PHYP call IDs for H_REG_LOGICAL_LAN_QUEUE,
H_ADD_LOGICAL_LAN_BUFFERS_QUEUE and H_FREE_LOGICAL_LAN_QUEUE, raising
MAX_HCALL_OPCODE accordingly, and adds matching static inline wrappers in
ibmveth.h. It is plumbing only, with no runtime behaviour change until a
later commit enables multi_queue.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index d87713668ed30..c0ef03cad9b9a 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -66,6 +66,142 @@ static inline long h_add_logical_lan_buffers(unsigned long unit_address,
[ ... ]
> +/**
> + * h_add_logical_lan_buffers_queue - Add buffers to subordinate queue
> + * @unit_address: Device unit address
> + * @queue_handle: Queue handle from h_register_logical_lan_queue()
[Severity: Low]
This isn't a bug, but do these two kernel-doc blocks in the same header
contradict each other about which handles are valid here?
h_add_logical_lan_buffers_queue() is documented as "Add buffers to
subordinate queue" with "@queue_handle: Queue handle from
h_register_logical_lan_queue()", while h_register_logical_lan_with_handle()
below says its queue 0 handle:
* Registers the primary receive queue (queue 0) with the hypervisor and
* returns the queue handle. This is needed in multi-queue mode to use
* h_add_logical_lan_buffers_queue() for all queues including queue 0.
By the end of the series the second wording is the one that matches the
code: ibmveth_add_logical_lan_buffers() in
drivers/net/ethernet/ibm/ibmveth.c passes
adapter->queue_handle[queue_index] for every index, queue_index == 0
included, and that handle comes from h_register_logical_lan_with_handle().
Would it be clearer to widen the add-buffers kdoc to say the handle may
come from either h_register_logical_lan_queue() or
h_register_logical_lan_with_handle(), so a future caller doesn't conclude
queue 0 must use the legacy buffer hcalls?
> + * @buffersznum: Buffer size (upper 32 bits) | count (lower 32 bits)
> + * @ioba12: Buffer addresses 1 and 2 packed ((addr1 << 32) | addr2)
> + * @ioba34: Buffer addresses 3 and 4 packed
> + * @ioba56: Buffer addresses 5 and 6 packed
> + * @ioba78: Buffer addresses 7 and 8 packed
> + * @ioba910: Buffer addresses 9 and 10 packed
> + * @ioba1112: Buffer addresses 11 and 12 packed
> + *
> + * Return:
> + * H_SUCCESS - All buffers added successfully
> + * H_PARAMETER - Invalid parameters
> + * H_HARDWARE - Hardware error
> + */
> +static inline long h_add_logical_lan_buffers_queue(unsigned long unit_address,
> + unsigned long queue_handle,
[ ... ]
> +/**
> + * h_register_logical_lan_with_handle - Register primary queue and get handle
> + * @unit_address: Device unit address
> + * @buffer_list: DMA address of buffer list
> + * @rec_queue: Buffer descriptor of receive queue
> + * @filter_list: DMA address of filter list
> + * @mac_address: MAC address
> + * @queue_handle: Output parameter for queue handle (may be NULL)
> + *
> + * Registers the primary receive queue (queue 0) with the hypervisor and
> + * returns the queue handle. This is needed in multi-queue mode to use
> + * h_add_logical_lan_buffers_queue() for all queues including queue 0.
> + *
> + * Return: H_SUCCESS (0) on success, error code otherwise
> + */
[ ... ]
Cross-instance finding from sashiko-gemini (34d1289184ae332db8c101e251d3dec96e9969f2e8dee8d98c3efff05039c9d0):
[Severity: High]
Calling plpar_hcall9() with missing arguments causes an out-of-bounds stack read, leaking uninitialized kernel stack memory to the hypervisor.
--
pw-bot: cr
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 02/15] ibmveth: Prepare MQ RX adapter data structures
2026-08-14 7:36 ` [PATCH net-next v5 02/15] ibmveth: Prepare MQ RX adapter data structures Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Prepare MQ RX adapter data structures
This converts the single-instance RX state in struct ibmveth_adapter (napi,
buffer_list_addr/dma, rx_queue, rx_buff_pool) into queue-indexed arrays sized
by a new IBMVETH_MAX_RX_QUEUES, and adds queue_handle[]/queue_irq[] plus
multi_queue/num_rx_queues bookkeeping. All existing references are
mechanically retargeted to index 0, and IBMVETH_MAX_RX_QUEUES is kept at 1 so
behaviour is unchanged.
No Link: tags are present in the commit header.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 73e051d26b9d8..7cb828b476c1b 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -482,24 +486,24 @@ static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
>
> mb();
>
> - atomic_dec(&(adapter->rx_buff_pool[pool].available));
> + atomic_dec(&adapter->rx_buff_pool[0][pool].available);
>
> return 0;
> }
[Severity: Low]
This isn't a bug, but the v5 change notes below the --- separator say the
"extra atomic_dec parentheses" nit from v4 was dropped, and that "this tip
patch keeps only the mechanical [0] retargeting".
The atomic_dec() in ibmveth_remove_buffer_from_pool() still drops the
redundant inner parentheses that the baseline had:
- atomic_dec(&(adapter->rx_buff_pool[pool].available));
+ atomic_dec(&adapter->rx_buff_pool[0][pool].available);
Should the changelog be adjusted, or the parentheses kept as-is so the hunk
really is index-only?
[ ... ]
> @@ -1876,6 +1886,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
> netdev->features |= NETIF_F_FRAGLIST;
> }
>
> + /* Initialize queue count - always 1 for now */
> + adapter->multi_queue = 0;
> + adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
> +
[Severity: Low]
At this commit these two assignments in ibmveth_probe() are the only accesses
to multi_queue and num_rx_queues, and queue_handle[]/queue_irq[] have neither
a reader nor a writer anywhere in the driver.
Combined with IBMVETH_MAX_RX_QUEUES being 1U, every array conversion in this
patch is a pure layout change with no behavioural effect.
Is it worth mentioning in the commit message which later patch first consumes
each of these fields? Reading the rest of the series, queue_irq[] and
queue_handle[] get their first users in ibmveth_toggle_irq() and
ibmveth_register_logical_lan_queue(), num_rx_queues is read through
ibmveth_get_num_rx_queues(), and IBMVETH_MAX_RX_QUEUES becomes 16U in
"ibmveth: Enable multi-queue RX receive path", so nothing is broken here.
> if (ret == H_SUCCESS &&
> (ret_attr & IBMVETH_ILLAN_RX_MULTI_BUFF_SUPPORT)) {
> adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
> @@ -1898,10 +1912,10 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
> memcpy(pool_count, pool_count_cmo, sizeof(pool_count));
>
> for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> - struct kobject *kobj = &adapter->rx_buff_pool[i].kobj;
> + struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
> int error;
>
> - ibmveth_init_buffer_pool(&adapter->rx_buff_pool[i], i,
> + ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
> pool_count[i], pool_size[i],
> pool_active[i]);
> error = kobject_init_and_add(kobj, &ktype_veth_pool,
[Severity: Low]
This is a pre-existing issue and is not introduced by this patch, which only
rewrites rx_buff_pool[i] to rx_buff_pool[0][i] on these lines.
The kobjects registered here live inside netdev_priv() memory, and the probe
failure paths after this loop call free_netdev() without a matching
kobject_del()/kobject_put():
rc = register_netdev(netdev);
if (rc) {
free_netdev(netdev);
return rc;
}
The only teardown loop is in ibmveth_remove():
for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
kobject_put(&adapter->rx_buff_pool[0][i].kobj);
which a failed probe never reaches, so a netif_set_real_num_tx_queues() or
register_netdev() failure leaves live .../poolN sysfs directories backed by
freed storage.
Worth noting this is addressed later in the same series: "ibmveth: Enable
multi-queue RX receive path" adds ibmveth_probe_cleanup(), which calls
ibmveth_put_pool_kobjs() before free_netdev() on both probe failure paths.
[ ... ]
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index c0ef03cad9b9a..fc6a01e810c6e 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -257,6 +257,8 @@ static inline long h_illan_attributes(unsigned long unit_address,
> #define IBMVETH_MAX_TX_BUF_SIZE (1024 * 64)
> #define IBMVETH_MAX_QUEUES 16U
> #define IBMVETH_DEFAULT_QUEUES 8U
> +#define IBMVETH_MAX_RX_QUEUES 1U
> +#define IBMVETH_DEFAULT_RX_QUEUES 1U
> #define IBMVETH_MAX_RX_PER_HCALL 8U
[ ... ]
> + struct ibmveth_rx_q rx_queue[IBMVETH_MAX_RX_QUEUES];
> + u64 queue_handle[IBMVETH_MAX_RX_QUEUES];
> + unsigned int queue_irq[IBMVETH_MAX_RX_QUEUES];
> + int multi_queue;
> + unsigned int num_rx_queues;
[ ... ]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up
2026-08-14 7:36 ` [PATCH net-next v5 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX resource allocation for MQ RX bring-up
This moves the filter list and per-queue RX buffer list / RX ring
allocation out of ibmveth_open() into four new helpers, and wires them
into both the open failure unwind and ibmveth_close(). Runtime behaviour
is meant to stay single-queue, and rc = -ENOMEM is now set before the TX
LTB loop.
A few questions below about the new release helpers and the remaining
inline unwind in ibmveth_open().
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 7cb828b476c1b..168940bfa1094 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -151,6 +151,186 @@ static unsigned int ibmveth_real_max_tx_queues(void)
> return min(n_cpu, IBMVETH_MAX_QUEUES);
> }
>
> +/**
> + * ibmveth_alloc_filter_list - Allocate and map filter list
> + * @adapter: ibmveth adapter structure
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int
> +ibmveth_alloc_filter_list(struct ibmveth_adapter *adapter)
> +{
> + struct device *dev = &adapter->vdev->dev;
> + struct net_device *netdev = adapter->netdev;
> +
> + adapter->filter_list_addr = (void *)get_zeroed_page(GFP_KERNEL);
> + if (!adapter->filter_list_addr) {
> + netdev_err(netdev, "unable to allocate filter pages\n");
> + return -ENOMEM;
> + }
> +
> + adapter->filter_list_dma = dma_map_single(dev,
> + adapter->filter_list_addr,
> + 4096, DMA_BIDIRECTIONAL);
> + if (dma_mapping_error(dev, adapter->filter_list_dma)) {
> + netdev_err(netdev, "unable to map filter list pages\n");
> + free_page((unsigned long)adapter->filter_list_addr);
> + adapter->filter_list_addr = NULL;
> + /* Do not leave DMA_MAPPING_ERROR for free_filter_list(). */
> + adapter->filter_list_dma = 0;
> + return -ENOMEM;
> + }
> +
> + netdev_dbg(netdev, "filter list @ 0x%p (DMA: 0x%llx)\n",
> + adapter->filter_list_addr,
> + (unsigned long long)adapter->filter_list_dma);
> +
> + return 0;
> +}
> +
> +/**
> + * ibmveth_free_filter_list - Free filter list resources
> + * @adapter: ibmveth adapter structure
> + */
> +static void
> +ibmveth_free_filter_list(struct ibmveth_adapter *adapter)
> +{
> + struct device *dev = &adapter->vdev->dev;
> +
> + if (adapter->filter_list_dma) {
> + dma_unmap_single(dev, adapter->filter_list_dma, 4096,
> + DMA_BIDIRECTIONAL);
> + adapter->filter_list_dma = 0;
> + }
> +
> + if (adapter->filter_list_addr) {
> + free_page((unsigned long)adapter->filter_list_addr);
> + adapter->filter_list_addr = NULL;
> + }
> +}
[Severity: Medium]
Is this patch doing more than a refactor? The new pointer and DMA handle
guards in ibmveth_free_filter_list(), plus the same guards in
ibmveth_cleanup_rx_resources(), look like they also fix a double free that
exists before this patch.
In the baseline, the ibmveth_open() unwind released the pages without
clearing anything:
out_free_filter_list:
free_page((unsigned long)adapter->filter_list_addr);
out_free_buffer_list:
free_page((unsigned long)adapter->buffer_list_addr[0]);
and ibmveth_close() then repeated the same work unconditionally:
dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
DMA_BIDIRECTIONAL);
free_page((unsigned long)adapter->buffer_list_addr[0]);
dma_unmap_single(dev, adapter->filter_list_dma, 4096,
DMA_BIDIRECTIONAL);
free_page((unsigned long)adapter->filter_list_addr);
ibmveth_set_csum_offload(), ibmveth_set_tso(), ibmveth_change_mtu() and
veth_pool_store() all propagate an ibmveth_open() failure while leaving the
interface running, so the following ndo_stop re-ran those frees on pages
that were already returned, together with a dma_unmap of an already
unmapped address.
The applied commit message describes this as a pure refactor ("Runtime
behavior stays single-queue"), and the reasoning for the guards only
appears in the v5 changelog below the --- line, which is stripped when the
patch is applied. Could the commit message state that the double
free_page() and stale dma_unmap are fixed here, and could a Fixes: tag be
added so stable trees can pick it up?
Note the sibling release helper ibmveth_free_tx_ltb() is still not
idempotent at this point in the series; that is only addressed by the later
"ibmveth: Refactor TX resource allocation in open/close paths" patch.
[ ... ]
> + netdev_dbg(netdev, "allocated %d RX queue(s) with %d entries each\n",
> + adapter->num_rx_queues, rxq_entries);
[ ... ]
> +static void
> +ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
> +{
> + struct device *dev = &adapter->vdev->dev;
> + int i;
> +
> + netdev_dbg(adapter->netdev, "cleaning up %d RX queue(s)\n",
> + adapter->num_rx_queues);
[Severity: Low]
This isn't a bug, but adapter->num_rx_queues is declared as unsigned int in
struct ibmveth_adapter in ibmveth.h, so should these two new debug messages
in ibmveth_alloc_rx_queues() and ibmveth_cleanup_rx_resources() use %u
rather than %d? The same mismatch is still there at the end of the series
where the value comes from ibmveth_get_num_rx_queues(), which also returns
unsigned int.
> +
> + for (i = 0; i < adapter->num_rx_queues; i++) {
> + if (adapter->buffer_list_dma[i]) {
> + dma_unmap_single(dev, adapter->buffer_list_dma[i],
> + 4096, DMA_BIDIRECTIONAL);
> + adapter->buffer_list_dma[i] = 0;
> + }
> +
> + if (adapter->rx_queue[i].queue_addr) {
> + dma_free_coherent(dev, adapter->rx_queue[i].queue_len,
> + adapter->rx_queue[i].queue_addr,
> + adapter->rx_queue[i].queue_dma);
> + adapter->rx_queue[i].queue_addr = NULL;
> + }
> +
> + if (adapter->buffer_list_addr[i]) {
> + free_page((unsigned long)adapter->buffer_list_addr[i]);
> + adapter->buffer_list_addr[i] = NULL;
> + }
> + }
> +}
> +
> /* setup the initial settings for a buffer pool */
> static void ibmveth_init_buffer_pool(struct ibmveth_buff_pool *pool,
> u32 pool_index, u32 pool_size,
[ ... ]
> @@ -752,26 +890,12 @@ static int ibmveth_open(struct net_device *netdev)
> ibmveth_free_buffer_pool(adapter,
> &adapter->rx_buff_pool[0][i]);
> }
> -out_unmap_filter_list:
> - dma_unmap_single(dev, adapter->filter_list_dma, 4096,
> - DMA_BIDIRECTIONAL);
> -
> out_free_tx_ltb:
> - while (--i >= 0) {
> + while (--i >= 0)
> ibmveth_free_tx_ltb(adapter, i);
> - }
[Severity: Medium]
This isn't a bug introduced by this patch, but does the fall-through from
out_free_buffer_pools into out_free_tx_ltb leak the TX long term buffers?
Both labels share the loop counter i. On the buffer pool failure and the
request_irq() failure paths, out_free_buffer_pools already runs
while (--i >= 0) down to i == -1, so out_free_tx_ltb then evaluates
--i == -2 and runs zero iterations:
out_free_buffer_pools:
while (--i >= 0) {
...
}
out_free_tx_ltb:
while (--i >= 0)
ibmveth_free_tx_ltb(adapter, i);
Every tx_ltb_ptr[] allocation plus its dma_map_single(DMA_TO_DEVICE) made
by the earlier loop over real_num_tx_queues then stays around, and a later
successful open overwrites the pointers and handles.
The shared counter disappears at the end of the series, where TX LTB
allocation moves into ibmveth_alloc_tx_resources() with its own err_free_ltbs
unwind, so this only matters for this commit and its predecessors.
> -
> -out_unmap_buffer_list:
> - dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
> - DMA_BIDIRECTIONAL);
> -out_free_queue_mem:
> - dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
> - adapter->rx_queue[0].queue_addr,
> - adapter->rx_queue[0].queue_dma);
> + ibmveth_cleanup_rx_resources(adapter);
> out_free_filter_list:
> - free_page((unsigned long)adapter->filter_list_addr);
> -out_free_buffer_list:
> - free_page((unsigned long)adapter->buffer_list_addr[0]);
> + ibmveth_free_filter_list(adapter);
> out:
> napi_disable(&adapter->napi[0]);
> return rc;
[Severity: High]
This is a pre-existing issue, but can the hypervisor still be writing into
these pages here?
On the buffer pool allocation failure path in ibmveth_open():
if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
netdev_err(netdev, "unable to alloc pool\n");
adapter->rx_buff_pool[0][i].active = 0;
rc = -ENOMEM;
goto out_free_buffer_pools;
}
there is no h_free_logical_lan() loop, unlike the request_irq() failure
path just below it, so PHYP is still registered against the buffer list
page, the RX ring and the filter list when control reaches
ibmveth_cleanup_rx_resources() and ibmveth_free_filter_list(), which
dma_unmap and free them.
The consolidated helpers make that sequence easier to read but do not
change it. At the end of the series ibmveth_open() allocates the buffer
pools before ibmveth_register_rx_queues() and every post-registration exit
goes through ibmveth_free_all_queues() first, so is the ordering here worth
fixing earlier, or is it intentionally left to the later "ibmveth: Add RX
queue register helpers for MQ" patch?
> @@ -805,17 +928,8 @@ static int ibmveth_close(struct net_device *netdev)
>
> ibmveth_update_rx_no_buffer(adapter);
>
> - dma_unmap_single(dev, adapter->buffer_list_dma[0], 4096,
> - DMA_BIDIRECTIONAL);
> - free_page((unsigned long)adapter->buffer_list_addr[0]);
> -
> - dma_unmap_single(dev, adapter->filter_list_dma, 4096,
> - DMA_BIDIRECTIONAL);
> - free_page((unsigned long)adapter->filter_list_addr);
> -
> - dma_free_coherent(dev, adapter->rx_queue[0].queue_len,
> - adapter->rx_queue[0].queue_addr,
> - adapter->rx_queue[0].queue_dma);
> + ibmveth_cleanup_rx_resources(adapter);
> + ibmveth_free_filter_list(adapter);
[Severity: High]
Can ibmveth_close() dereference a NULL buffer_list_addr[0] now?
ibmveth_cleanup_rx_resources() clears the pointer after freeing it:
free_page((unsigned long)adapter->buffer_list_addr[i]);
adapter->buffer_list_addr[i] = NULL;
while ibmveth_update_rx_no_buffer(), called just above, dereferences it
unconditionally:
__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
adapter->rx_no_buffer = be64_to_cpup(p);
The baseline inline code in ibmveth_close() left the pointer stale, so the
read did not fault. With the helper clearing it, a failed re-open leaves
the field NULL while the interface is still up:
ibmveth_set_csum_offload() / ibmveth_set_tso() / ibmveth_change_mtu() /
veth_pool_store()
ibmveth_close()
ibmveth_open() <- fails, out_free_tx_ltb path runs
ibmveth_cleanup_rx_resources() buffer_list_addr[0] = NULL
and the next ndo_stop, or the queued ibmveth_reset() work reaching
dev_close(), enters ibmveth_close() and reads through NULL + 4096 - 8.
Later patches in the series appear to close this: ibmveth_close() becomes
gated on adapter->opened, and ibmveth_update_rx_no_buffer() takes a queue
index and returns early when buffer_list_addr[queue_index] is NULL. Would
it make sense to add that NULL check in this patch, since this is the commit
that starts clearing the pointer?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX
2026-08-14 7:36 ` [PATCH net-next v5 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor buffer pool management for per-queue MQ RX
This moves the five RX buffer pools to a per-queue array and introduces
ibmveth_alloc_buffer_pools() / ibmveth_free_buffer_pools() plus the
per-queue helpers, wiring them into open() and close(). Queue 0 stays the
template for pool geometry and activation, and runtime is still
single-queue.
A few questions below about the new unwind paths and about two of the
new comments.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 168940bfa1094..1a428ee60d58c 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -627,6 +627,142 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
> }
> }
>
> +/**
> + * ibmveth_free_queue_buffer_pools - Free buffer pools for a single queue
[ ... ]
> +/**
> + * ibmveth_alloc_queue_buffer_pools - Allocate buffer pools for a single queue
> + * @adapter: ibmveth adapter structure
> + * @queue: queue index
> + *
> + * Allocates backing storage for each active pool on @queue.
> + * Inactive pools (!active) are skipped. Pool metadata must be
> + * initialized before calling this function.
> + *
> + * Return: 0 on success, negative error code on failure
> + */
> +static int ibmveth_alloc_queue_buffer_pools(struct ibmveth_adapter *adapter,
> + int queue)
> +{
> + struct net_device *netdev = adapter->netdev;
> + int i;
> +
> + for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> + struct ibmveth_buff_pool *bpool =
> + &adapter->rx_buff_pool[queue][i];
> +
> + if (!bpool->active)
> + continue;
> +
> + if (ibmveth_alloc_buffer_pool(bpool)) {
> + netdev_err(netdev,
> + "queue %d pool %d alloc failed (size=%u count=%u)\n",
> + queue, i,
> + bpool->buff_size,
> + bpool->size);
> + bpool->active = 0;
> + /* Free by allocation presence, not active — the
> + * failing pool cleared active first and would be
> + * skipped by an active-only unwind.
> + */
[Severity: Low]
Is the stated rationale in this comment accurate? Freeing by allocation
presence is a fine policy for the later resize paths, but the specific
claim that an active-only unwind would skip the failing pool and leak it
does not seem to hold here, because ibmveth_alloc_buffer_pool() already
undoes its own partial allocations before returning -1:
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_alloc_buffer_pool() {
pool->dma_addr = kzalloc_objs(dma_addr_t, pool->size);
if (!pool->dma_addr) {
kfree(pool->free_map);
pool->free_map = NULL;
return -1;
}
pool->skbuff = kcalloc(pool->size, sizeof(void *), GFP_KERNEL);
if (!pool->skbuff) {
kfree(pool->dma_addr);
pool->dma_addr = NULL;
kfree(pool->free_map);
pool->free_map = NULL;
return -1;
}
...
}
By the time bpool->active = 0 runs, free_map/dma_addr/skbuff are all NULL
for the failing pool, so there is nothing for an active-only unwind to
miss. The same claim appears in the changelog ("v4 fail path freed by
active and skipped the failing pool", "open-fail cannot leak partially
allocated pools"). Could the comment and changelog instead justify
free-by-presence purely in terms of the later resize paths?
> + ibmveth_free_queue_buffer_pools(adapter, queue);
> + return -ENOMEM;
> + }
> + }
> +
> + return 0;
> +}
> +
[ ... ]
> + /* Allocate actual buffers for all queues */
> + for (q = 0; q < adapter->num_rx_queues; q++) {
> + rc = ibmveth_alloc_queue_buffer_pools(adapter, q);
> + if (rc) {
> + /* Free pools for all previous queues */
> + while (--q >= 0)
> + ibmveth_free_queue_buffer_pools(adapter, q);
> + return rc;
> + }
> + }
> +
> + netdev_dbg(netdev, "allocated buffer pools for %d queue(s)\n",
> + adapter->num_rx_queues);
[Severity: Low]
This isn't a bug, but adapter->num_rx_queues is declared as
"unsigned int num_rx_queues;" in drivers/net/ethernet/ibm/ibmveth.h, so
should these two new messages use %u rather than %d? The same applies to
the netdev_dbg() in ibmveth_free_buffer_pools() below.
> + return 0;
> +}
> +
> +/**
> + * ibmveth_free_buffer_pools - Free buffer pools for all queues
> + * @adapter: ibmveth adapter structure
> + *
> + * Frees buffer pools for all queues using the helper function.
> + */
> +static void
> +ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
> +{
> + int q;
> +
> + /* Free buffer pools for all queues */
> + for (q = 0; q < adapter->num_rx_queues; q++)
> + ibmveth_free_queue_buffer_pools(adapter, q);
> +
> + netdev_dbg(adapter->netdev, "freed buffer pools for %d queue(s)\n",
> + adapter->num_rx_queues);
> +}
> +
> /**
> * ibmveth_remove_buffer_from_pool - remove a buffer from a pool
> * @adapter: adapter instance
> @@ -851,16 +987,9 @@ static int ibmveth_open(struct net_device *netdev)
> goto out_free_tx_ltb;
> }
>
> - for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> - if (!adapter->rx_buff_pool[0][i].active)
> - continue;
> - if (ibmveth_alloc_buffer_pool(&adapter->rx_buff_pool[0][i])) {
> - netdev_err(netdev, "unable to alloc pool\n");
> - adapter->rx_buff_pool[0][i].active = 0;
> - rc = -ENOMEM;
> - goto out_free_buffer_pools;
> - }
> - }
> + rc = ibmveth_alloc_buffer_pools(adapter);
> + if (rc)
> + goto out_free_tx_ltb;
[Severity: Medium]
This isn't a gap introduced by this patch (the old inline pool loop fell
through to the same labels), but since the edge is being rewritten here:
this goto is taken after ibmveth_register_logical_lan() has already
succeeded, and the unwind it lands on never issues h_free_logical_lan():
out_free_tx_ltb:
while (--i >= 0)
ibmveth_free_tx_ltb(adapter, i);
ibmveth_cleanup_rx_resources(adapter);
out_free_filter_list:
ibmveth_free_filter_list(adapter);
ibmveth_cleanup_rx_resources() does the dma_unmap_single() of
buffer_list_dma[], the dma_free_coherent() of the RX completion queue and
the free_page() of the buffer-list page, and ibmveth_free_filter_list()
releases the filter list, all while PHYP still holds those addresses. The
adjacent request_irq() failure path does perform the handshake the driver
otherwise considers mandatory:
do {
lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
} while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
Can PHYP write the rx_no_buffer counter into the freed buffer-list page, or
post a completion into the freed RX queue, after this path returns? The
h_vio_signal(VIO_IRQ_DISABLE) issued just before registration also is not
rebalanced here.
Later patches in this series do repair this: "ibmveth: Refactor TX resource
allocation in open/close paths" retargets this edge to out_unregister_lan,
and by the end of the series ibmveth_open() allocates the pools before
ibmveth_register_rx_queues() and routes post-registration failures through
ibmveth_free_all_queues() -> h_free_logical_lan(). Would it be reasonable
to point this goto at a label that unregisters the logical LAN already in
this patch, so the intermediate commit is not bisect-visible?
>
> netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
> rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
> @@ -885,11 +1014,7 @@ static int ibmveth_open(struct net_device *netdev)
> return 0;
>
> out_free_buffer_pools:
> - while (--i >= 0) {
> - if (adapter->rx_buff_pool[0][i].active)
> - ibmveth_free_buffer_pool(adapter,
> - &adapter->rx_buff_pool[0][i]);
> - }
> + ibmveth_free_buffer_pools(adapter);
> out_free_tx_ltb:
> while (--i >= 0)
> ibmveth_free_tx_ltb(adapter, i);
[Severity: Medium]
This is a pre-existing issue rather than one introduced here, but does this
hunk silently fix a TX long-term-buffer leak?
In the parent commit the pool unwind consumed i:
out_free_buffer_pools:
while (--i >= 0) {
if (adapter->rx_buff_pool[0][i].active)
ibmveth_free_buffer_pool(adapter,
&adapter->rx_buff_pool[0][i]);
}
out_free_tx_ltb:
while (--i >= 0)
ibmveth_free_tx_ltb(adapter, i);
so on reaching out_free_tx_ltb by fall-through, i was already -1 and the
first test of --i evaluated to -2, meaning no tx_ltb_ptr[]/tx_ltb_dma[]
buffer was released. With the pool unwind now a helper call that does not
touch i, i still holds the TX queue count from the completed TX allocation
loop, so the TX LTBs are freed.
Every PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE) TX buffer plus its DMA mapping was
leaked on each failing open, and it repeats per attempt ("ip link set <dev>
up" under memory pressure, or the close+open pair inside veth_pool_store(),
ibmveth_change_mtu() and ibmveth_reset()).
Would it make sense to split this out as its own patch with a Fixes: tag so
stable trees pick it up, or at least describe it in the changelog?
>
> @@ -928,14 +1053,10 @@ static int ibmveth_close(struct net_device *netdev)
[ ... ]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues
2026-08-14 7:36 ` [PATCH net-next v5 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor RX interrupt control for MQ RX queues
This patch introduces queue-indexed RX interrupt helpers (toggle/enable/
disable, setup/cleanup, schedule) and wires them into open(), close(),
poll() and the IRQ handler, with request_irq() now using &adapter->napi[i]
as the per-queue cookie. Runtime behaviour stays single-queue.
A few questions below.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 1a428ee60d58c..99eeb6ef51bf3 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -331,6 +333,311 @@ ibmveth_cleanup_rx_resources(struct ibmveth_adapter *adapter)
[ ... ]
> + /*
> + * H_PARAMETER is ambiguous (already in requested state vs bad
> + * args). Fold only on disable as an idempotent mask. On enable
> + * keep it an error so a stuck-masked queue stays visible to
> + * poll/resize recovery.
> + */
> + if (h_rc == H_PARAMETER && !enable) {
> + dev_warn_ratelimited(&adapter->netdev->dev,
> + "H_VIOCTL %s IRQ returned H_PARAMETER for queue %d (hwirq=%lu)\n",
> + action, queue_index, hwirq);
> + return 0;
> + }
[ ... ]
> + if (adapter->multi_queue && num > 1) {
> + for (i = 0; i < num; i++) {
> + rc = ibmveth_enable_irq(adapter, i);
> + if (rc) {
> + netdev_err(netdev,
> + "Failed to enable IRQ for queue %d, rc=%d\n",
> + i, rc);
> + while (--i >= 0) {
> + ibmveth_disable_irq(adapter, i);
> + synchronize_irq(adapter->queue_irq[i]);
> + }
> + rc = -EIO;
> + goto err_disable_napi;
> + }
> + }
> + }
[ ... ]
> +err_disable_napi:
> + /* PHYP unmask was rolled back above; disable NAPI before free_irq */
> + for (i = 0; i < num; i++)
> + napi_disable(&adapter->napi[i]);
> + for (i = 0; i < num; i++) {
> + if (adapter->queue_irq[i])
> + free_irq(adapter->queue_irq[i], &adapter->napi[i]);
> + }
[Severity: Medium]
In ibmveth_setup_rx_interrupts(), when ibmveth_enable_irq() fails for queue
i the rollback loop begins with --i, so queue i itself is never masked, yet
err_disable_napi then napi_disable()s and free_irq()s that same queue.
Is queue i guaranteed to be masked at that point? ibmveth_toggle_irq()
folds H_PARAMETER to success only on disable, and its own comment says
H_PARAMETER may mean "already in requested state":
if (h_rc == H_PARAMETER && !enable) {
so an enable can report an error for a queue PHYP has already unmasked.
While napi_disable() sleeps, an interrupt on that still-unmasked queue
reaches ibmveth_interrupt() -> ibmveth_schedule_rx_queue(), where
napi_schedule_prep() fails and the helper deliberately does not mask:
if (napi_schedule_prep(napi)) {
rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(rc);
__napi_schedule(napi);
return true;
}
return false;
The handler still returns IRQ_HANDLED, so nothing quiets the source. Can
this leave the PHYP interrupt asserted and re-entering the handler for the
duration of the napi_disable() wait, which is the opposite of the "mask
PHYP and synchronize_irq before napi_disable" rule stated in the changelog?
The multi_queue && num > 1 branch cannot run at this commit because
multi_queue is hardcoded to 0, but it becomes live later in the series and
the rollback loop is unchanged at the end of the series.
[ ... ]
> +/**
> + * ibmveth_schedule_rx_queue - Mask PHYP IRQ and schedule NAPI for one RX queue
> + * @adapter: ibmveth adapter structure
> + * @qindex: RX queue index
> + *
> + * Shared by the IRQ handler and process-context kick sites (open, resume,
> + * pool sysfs, poll_controller).
> + *
> + * Return: true if NAPI was scheduled (and PHYP masked), false if prep failed
> + */
> +static bool ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
> + int qindex)
> +{
> + struct napi_struct *napi = &adapter->napi[qindex];
> + int rc;
> +
> + if (WARN_ON(qindex < 0 || qindex >= adapter->num_rx_queues))
> + return false;
[Severity: Low]
This isn't a bug, but the documented return contract does not match the
code in two ways.
The out-of-range WARN_ON branch also returns false, which is not a prep
failure, so callers cannot distinguish the two cases.
And "true ... (and PHYP masked)" is not established, since the return of
ibmveth_disable_irq() is only passed to WARN_ON() and then discarded:
if (napi_schedule_prep(napi)) {
rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(rc);
__napi_schedule(napi);
return true;
}
so when the mask hypercall fails the helper still reports true while the
queue remains unmasked. Should the kernel-doc be reworded, or should the
mask failure be reflected in the return value for callers like open(),
resume(), veth_pool_store() and the later ibmveth_restart_rx_queue()?
[ ... ]
> @@ -991,24 +1297,20 @@ static int ibmveth_open(struct net_device *netdev)
> if (rc)
> goto out_free_tx_ltb;
>
> - netdev_dbg(netdev, "registering irq 0x%x\n", netdev->irq);
> - rc = request_irq(netdev->irq, ibmveth_interrupt, 0, netdev->name,
> - netdev);
> - if (rc != 0) {
> - netdev_err(netdev, "unable to request irq 0x%x, rc %d\n",
> - netdev->irq, rc);
> + rc = ibmveth_setup_rx_interrupts(adapter);
> + if (rc) {
> do {
> lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> -
> goto out_free_buffer_pools;
> }
[Severity: High]
This isn't a bug introduced by this patch, but the ordering here is worth a
question now that close() gains an early return.
ibmveth_open() registers the logical LAN before allocating buffer pools, and
the pool allocation failure branch takes goto out_free_tx_ltb, which runs:
out_free_tx_ltb:
while (--i >= 0)
ibmveth_free_tx_ltb(adapter, i);
ibmveth_cleanup_rx_resources(adapter);
out_free_filter_list:
ibmveth_free_filter_list(adapter);
That dma_unmaps buffer_list_dma[0], dma_free_coherent()s the RX ring and
frees the filter list page without ever calling h_free_logical_lan(). Does
this leave PHYP registered against the buffer list, filter list and RX
descriptor ring after those pages have been unmapped and returned to the
page allocator?
The same ordering exists in the baseline at 4f93b12cf7b2, so the root
problem predates this patch. What changes here is that the new
"if (!adapter->opened) return 0;" gate in ibmveth_close() also removes the
accidental recovery a later ndo_stop used to provide by running the
h_free_logical_lan() loop.
For reference, this is resolved by the end of the series: open() there
allocates the buffer pools before ibmveth_register_rx_queues(), and every
post-registration error path funnels into ibmveth_free_all_queues(), which
issues h_free_logical_lan() in a retry loop.
> @@ -1032,27 +1333,32 @@ static int ibmveth_close(struct net_device *netdev)
> long lpar_rc;
> int i;
>
> - netdev_dbg(netdev, "close starting\n");
> + /* Gate on opened, not IFF_UP: pool_store/change_mtu close+open can
> + * leave IFF_UP set after a failed reopen.
> + */
> + if (!adapter->opened)
> + return 0;
>
> - napi_disable(&adapter->napi[0]);
> + adapter->opened = false;
[Severity: Low]
This is a pre-existing issue rather than something introduced here, but
should this gate carry a Fixes: tag and a note in the changelog?
In the baseline, ibmveth_close() unconditionally ran
napi_disable(&adapter->napi[0]) and free_irq(netdev->irq, netdev). After a
failed reopen from veth_pool_store() or ibmveth_change_mtu() (both call
close() then open() directly and leave IFF_UP set when open fails), a
subsequent "ip link set down" re-entered ndo_stop with NAPI already
disabled and no IRQ registered. napi_disable_locked() leaves
NAPIF_STATE_SCHED|NPSVC set, so the second napi_disable() spins in its
usleep_range() loop while holding rtnl, and free_irq() splats "Trying to
free already-free IRQ".
The changelog describes the mechanism ("opened / rx_irq_setup gate whether
cleanup walks IRQ/NAPI state") and the v5 notes below the --- describe the
scenario, but the commit message itself never says a hang is being fixed
and there is no Fixes: tag, so the fix is hard to identify for backports.
It also cannot be applied on its own since it depends on the new helpers,
flags and queue_irq[] array added here.
[Severity: Medium]
Separately, can this early return skip TX long-term-buffer teardown? At
this commit ibmveth_set_channels() keys its allocation decision off IFF_UP:
if (!(netdev->flags & IFF_UP))
return netif_set_real_num_tx_queues(netdev, goal);
so after a failed reopen (IFF_UP still set, adapter->opened false),
"ethtool -L eth0 tx N" takes the allocating branch while the adapter is
closed. The next ndo_stop then returns early and never reaches:
for (i = 0; i < netdev->real_num_tx_queues; i++)
ibmveth_free_tx_ltb(adapter, i);
and the next ibmveth_open() assigns tx_ltb_ptr[i]/tx_ltb_dma[i]
unconditionally. Does that leak the bounce buffer and its DMA mapping per
TX queue?
By the end of the series ibmveth_set_channels() gates on
"if (!adapter->opened)" and only stashes queue counts, so no path allocates
TX LTBs while the adapter is closed, which removes this window.
[ ... ]
> @@ -1696,7 +2002,7 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
> container_of(napi, struct ibmveth_adapter, napi[0]);
> struct net_device *netdev = adapter->netdev;
> int frames_processed = 0;
> - unsigned long lpar_rc;
> + int rc;
> u16 mss = 0;
>
> restart_poll:
[ ... ]
> @@ -1796,15 +2102,15 @@ static int ibmveth_poll(struct napi_struct *napi, int budget)
> /* We think we are done - reenable interrupts,
> * then check once more to make sure we are done.
> */
> - lpar_rc = h_vio_signal(adapter->vdev->unit_address, VIO_IRQ_ENABLE);
> - if (WARN_ON(lpar_rc != H_SUCCESS)) {
> + rc = ibmveth_enable_irq(adapter, 0);
> + if (WARN_ON(rc)) {
> schedule_work(&adapter->work);
> goto out;
> }
>
> if (ibmveth_rxq_pending_buffer(adapter) && napi_schedule(napi)) {
> - lpar_rc = h_vio_signal(adapter->vdev->unit_address,
> - VIO_IRQ_DISABLE);
> + rc = ibmveth_disable_irq(adapter, 0);
> + WARN_ON(rc);
> goto restart_poll;
> }
[Severity: Low]
This isn't a bug today, but the two halves of the new per-queue protocol
disagree. ibmveth_interrupt() now resolves the queue that fired:
qindex = napi - adapter->napi;
...
ibmveth_schedule_rx_queue(adapter, qindex);
while ibmveth_poll() still recovers the adapter through
container_of(napi, struct ibmveth_adapter, napi[0]) and hardcodes queue 0
in ibmveth_enable_irq(adapter, 0) / ibmveth_disable_irq(adapter, 0). For a
napi index above 0 that container_of would compute a wrong adapter pointer
and the unmask would target queue 0 rather than the queue that fired.
Since IBMVETH_MAX_RX_QUEUES is 1 and multi_queue is 0 at this commit only
napi[0] can be registered, and later commits in the series ("ibmveth:
Harden RX poll path with helpers" / "ibmveth: Enable multi-queue RX receive
path") derive queue_index = napi - adapter->napi in poll() with a range
check. Would it be clearer to convert poll() in the same patch that
introduces the queue-indexed handler, so the two sides never disagree?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 06/15] ibmveth: Refactor TX resource allocation in open/close paths
2026-08-14 7:36 ` [PATCH net-next v5 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Refactor TX resource allocation in open/close paths
This moves the per-queue TX long term buffer alloc/free loops out of
ibmveth_open()/ibmveth_close() into ibmveth_alloc_tx_resources() and
ibmveth_free_tx_resources(), moves the TX allocation to the end of open()
after RX is live, and adds a tx_ltb_ptr[] presence check to
ibmveth_start_xmit(). No Link: tags are present in the commit header.
A few questions on the LTB lifetime rules the patch establishes.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 99eeb6ef51bf3..b39e8c53cbfd1 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -1183,8 +1183,12 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
>
> static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx)
> {
> + if (!adapter->tx_ltb_ptr[idx])
> + return;
> +
> dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx],
> adapter->tx_ltb_size, DMA_TO_DEVICE);
> + adapter->tx_ltb_dma[idx] = 0;
> kfree(adapter->tx_ltb_ptr[idx]);
> adapter->tx_ltb_ptr[idx] = NULL;
> }
[Severity: Medium]
tx_ltb_ptr[] now doubles as the guard for tx_ltb_dma[], but the retire
order here clears the guarded value first:
dma_unmap_single(...);
adapter->tx_ltb_dma[idx] = 0;
kfree(adapter->tx_ltb_ptr[idx]);
adapter->tx_ltb_ptr[idx] = NULL;
Can an ibmveth_start_xmit() that already passed the new check
if (unlikely(!adapter->tx_ltb_ptr[queue_num])) {
then reach
desc.fields.address = adapter->tx_ltb_dma[queue_num];
and hand a zeroed DMA address to H_SEND_LOGICAL_LAN?
The publish side has the mirror-image order: ibmveth_allocate_tx_ltb()
stores tx_ltb_ptr[idx] from kzalloc() before tx_ltb_dma[idx] is filled in
by dma_map_single(), with no release/acquire pairing.
Both the check in ibmveth_start_xmit() and its later uses (the memcpy into
tx_ltb_ptr[queue_num] and the descriptor address read) are plain loads with
no READ_ONCE and no common lock, so nothing prevents a reload of
tx_ltb_ptr[queue_num] after it was set to NULL.
Against ibmveth_close() this appears harmless, since ibmveth does not set
lltx and netif_tx_disable() therefore takes every __netif_tx_lock. Against
ibmveth_set_channels() (more below) there is no such serialization. Would a
real lock, or gating on adapter->opened, express this better than a pointer
used as a flag?
> @@ -1207,12 +1211,54 @@ static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx)
> "unable to DMA map tx long term buffer\n");
> kfree(adapter->tx_ltb_ptr[idx]);
> adapter->tx_ltb_ptr[idx] = NULL;
> + adapter->tx_ltb_dma[idx] = 0;
> return -ENOMEM;
> }
>
> return 0;
> }
>
> +/**
> + * ibmveth_alloc_tx_resources - Allocate TX resources for all queues
> + * @adapter: ibmveth adapter structure
> + *
> + * Allocates TX Long Term Buffers (LTBs) for all TX queues.
> + *
> + * Return: 0 on success, -ENOMEM on failure
> + */
> +static int ibmveth_alloc_tx_resources(struct ibmveth_adapter *adapter)
> +{
> + struct net_device *netdev = adapter->netdev;
> + int i;
> +
> + for (i = 0; i < netdev->real_num_tx_queues; i++) {
> + if (ibmveth_allocate_tx_ltb(adapter, i))
> + goto err_free_ltbs;
> + }
[Severity: Low]
This isn't a bug introduced by this patch, and it looks unreachable by the
end of the series, but noting it for traceability: ibmveth_allocate_tx_ltb()
assigns the kzalloc() and dma_map_single() results straight into
tx_ltb_ptr[i] / tx_ltb_dma[i] with no check for an already-populated slot,
unlike ibmveth_set_channels() which does
if (adapter->tx_ltb_ptr[i])
continue;
so a caller reaching this helper with a live mapping in the slot would
strand the old TCE mapping. With the later set_channels rewrite in this
series the !adapter->opened branch allocates nothing and the opened branch
frees everything it allocated on failure, so no path leaves tx_ltb_ptr[]
populated while adapter->opened is false and open() cannot overwrite a live
mapping. Would a defensive check (or a comment stating the invariant) be
worth it here?
> +
> + return 0;
> +
> +err_free_ltbs:
> + while (--i >= 0)
> + ibmveth_free_tx_ltb(adapter, i);
> + return -ENOMEM;
> +}
> +
> +/**
> + * ibmveth_free_tx_resources - Free TX resources for all queues
> + * @adapter: ibmveth adapter structure
> + *
> + * Frees TX Long Term Buffers (LTBs) for all TX queues.
> + */
> +static void ibmveth_free_tx_resources(struct ibmveth_adapter *adapter)
> +{
> + struct net_device *netdev = adapter->netdev;
> + int i;
> +
> + for (i = 0; i < netdev->real_num_tx_queues; i++)
> + ibmveth_free_tx_ltb(adapter, i);
> +}
[Severity: Low]
Small wording mismatch: the kernel-doc says "Frees TX Long Term Buffers
(LTBs) for all TX queues", but the loop is bounded by real_num_tx_queues
while tx_ltb_ptr[] is IBMVETH_MAX_QUEUES wide. Since ibmveth_free_tx_ltb()
is now NULL-safe, either iterating IBMVETH_MAX_QUEUES or saying
"real_num_tx_queues" in the comment would make the two agree.
> @@ -1263,12 +1309,6 @@ static int ibmveth_open(struct net_device *netdev)
[ ... ]
> rc = ibmveth_alloc_buffer_pools(adapter);
> if (rc)
> - goto out_free_tx_ltb;
> + goto out_unregister_lan;
>
> rc = ibmveth_setup_rx_interrupts(adapter);
> - if (rc) {
> - do {
> - lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> - } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> - goto out_free_buffer_pools;
> - }
> + if (rc)
> + goto out_unregister_lan;
[ ... ]
> -out_free_buffer_pools:
> +out_cleanup_rx_interrupts:
> + ibmveth_cleanup_rx_interrupts(adapter);
> +out_unregister_lan:
> + do {
> + lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> + } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> ibmveth_free_buffer_pools(adapter);
> -out_free_tx_ltb:
> - while (--i >= 0)
> - ibmveth_free_tx_ltb(adapter, i);
> +out_free_queue_mem:
> ibmveth_cleanup_rx_resources(adapter);
[Severity: Medium]
This isn't a bug introduced by this patch, but is the changelog wording
accurate about what changed here? The commit message says:
"After LAN registration, open-fail teardown frees the logical LAN before
tearing down RX pool DMA (intentional safer order than leaving the LAN
registered while unmapping RX memory)."
At the parent commit, an ibmveth_alloc_buffer_pools() failure after a
successful ibmveth_register_logical_lan() went
rc = ibmveth_alloc_buffer_pools(adapter);
if (rc)
goto out_free_tx_ltb;
and out_free_tx_ltb fell through to ibmveth_cleanup_rx_resources() and
ibmveth_free_filter_list() with no h_free_logical_lan() anywhere on that
path, so the RX queue and buffer/filter lists were unmapped and freed while
PHYP still had the LAN registered. Post-patch that failure goes to
out_unregister_lan, which issues the deregistration hcall first.
So this looks less like a reordering and more like adding a hcall that was
never issued on that path. Should this be split out as its own patch with a
Fixes: tag so it can be picked up for stable, rather than described as an
ordering preference inside a TX-only refactor?
> @@ -1343,7 +1385,10 @@ static int ibmveth_close(struct net_device *netdev)
[ ... ]
> - netif_tx_stop_all_queues(netdev);
> + /* Disable and wait for in-flight ndo_start_xmit (stop_all_queues
> + * alone does not). Direct close() callers bypass synchronize_net().
> + */
> + netif_tx_disable(netdev);
[Severity: High]
This isn't a bug introduced by this patch, but the reasoning in this new
comment seems to apply equally to ibmveth_set_channels(), which still frees
the same LTBs after only netif_tx_stop_all_queues():
ibmveth_set_channels() {
...
netif_tx_stop_all_queues(netdev);
...
/* Free any that are no longer needed */
for (i = old; i > goal; i--) {
if (adapter->tx_ltb_ptr[i - 1])
ibmveth_free_tx_ltb(adapter, i - 1);
}
netif_tx_wake_all_queues(netdev);
}
netif_tx_stop_all_queues() only sets __QUEUE_STATE_DRV_XOFF; it takes no
__netif_tx_lock, and netif_set_real_num_tx_queues() only touches the qdisc
root lock. Since ibmveth does not declare lltx, an ndo_start_xmit already
running on another CPU holds the txq lock and keeps going:
CPU0 ibmveth_start_xmit()
/* already passed !adapter->tx_ltb_ptr[queue_num] */
memcpy(adapter->tx_ltb_ptr[queue_num], skb->data, skb_headlen(skb));
...
desc.fields.address = adapter->tx_ltb_dma[queue_num];
CPU1 ibmveth_set_channels()
ibmveth_free_tx_ltb(adapter, i - 1);
dma_unmap_single(...);
adapter->tx_ltb_dma[idx] = 0;
kfree(adapter->tx_ltb_ptr[idx]);
Can an ethtool -L shrink therefore memcpy into kfree()d memory and pass a
stale or zeroed DMA address to H_SEND_LOGICAL_LAN? The new one-shot
tx_ltb_ptr[] check in ibmveth_start_xmit() does not close this window, and
this stop-then-free sequence is unchanged at the end of the series. Should
ibmveth_set_channels() use netif_tx_disable() too?
> @@ -1359,13 +1404,14 @@ static int ibmveth_close(struct net_device *netdev)
[ ... ]
> @@ -1789,6 +1835,12 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb,
> int i, queue_num = skb_get_queue_mapping(skb);
> unsigned long mss = 0;
>
> + /* Close / failed reopen can free LTBs while IFF_UP is still set. */
> + if (unlikely(!adapter->tx_ltb_ptr[queue_num])) {
> + dev_kfree_skb_any(skb);
> + return NETDEV_TX_OK;
> + }
> +
[Severity: Medium]
Should this drop bump netdev->stats.tx_dropped like every other drop in this
function does?
if (unlikely(skb->len > adapter->tx_ltb_size)) {
...
netdev->stats.tx_dropped++;
goto out;
}
The other drop sites (ibmveth_is_packet_unsupported(), the checksum-help
failure, the oversize check, the total_bytes mismatch and the
ibmveth_send() failure) all increment tx_dropped, and returning
NETDEV_TX_OK tells the core the skb was consumed so the core cannot account
for it either. In exactly the abnormal states this guard targets, traffic
is then discarded with nothing visible in ip -s link or in
/sys/class/net/<if>/statistics/tx_dropped. Could this reuse the existing
out: label after the tx_dropped++ instead?
[Severity: Low]
This isn't a bug introduced by this patch, and it looks resolved later in
the series, but noting it for traceability against the claim in the commit
message that "close/failed-reopen with IFF_UP set cannot UAF".
The guard keys off tx_ltb_ptr[] rather than adapter->opened, while
ibmveth_set_channels() at this point in the series still selects its live
path from IFF_UP alone:
if (!(netdev->flags & IFF_UP))
return netif_set_real_num_tx_queues(netdev, goal);
In the state the driver documents in ibmveth_close() (IFF_UP set,
adapter->opened false after a failed reopen), ethtool -L would allocate LTBs
and finish with netif_tx_wake_all_queues() on an adapter whose logical LAN
was already released, so packets pass this pointer-only check and reach
ibmveth_send() with no registered LAN, and those LTBs are not freed by a
later close() because it early-returns on !adapter->opened. The later patch
"ibmveth: Wire ethtool set_channels to MQ RX queue resize" replaces the
IFF_UP gating with an adapter->opened test whose !opened branch allocates
nothing and wakes no queues, which removes this window.
> if (ibmveth_is_packet_unsupported(skb, netdev))
> goto out;
> /* veth can't checksum offload UDP */
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 07/15] ibmveth: Add RX queue register helpers for MQ
2026-08-14 7:36 ` [PATCH net-next v5 07/15] ibmveth: Add RX queue register helpers for MQ Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add RX queue register helpers for MQ
This extracts the queue-0 control-plane registration and teardown out of
ibmveth_open()/ibmveth_close() into ibmveth_register_rx_queues() and
ibmveth_free_all_queues(), and adds adapter->hcall_stats for register/free
path accounting. Runtime behaviour stays single-queue since multi_queue is
still false.
A few questions on the teardown error handling and on the changelog text.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index b39e8c53cbfd1..8519cad503225 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -1270,13 +1275,27 @@ static int ibmveth_register_logical_lan(struct ibmveth_adapter *adapter,
> * try again, but only once.
> */
> retry:
> - rc = h_register_logical_lan(adapter->vdev->unit_address,
> - adapter->buffer_list_dma[0], rxq_desc.desc,
> - adapter->filter_list_dma, mac_address);
> + /* In multi-queue mode, obtain a queue handle for queue 0 so all RX
> + * queues can use the same per-queue buffer hypercalls.
> + */
> + if (adapter->multi_queue) {
> + rc = h_register_logical_lan_with_handle(ua, buf_dma,
> + rxq_desc.desc,
> + filter_dma,
> + mac_address,
> + &qh0);
> + if (rc == H_SUCCESS)
> + adapter->queue_handle[0] = qh0;
> + } else {
> + rc = h_register_logical_lan(ua, buf_dma, rxq_desc.desc,
> + filter_dma, mac_address);
> + }
> + adapter->hcall_stats.reg_lan++;
>
> if (rc != H_SUCCESS && try_again) {
> do {
> rc = h_free_logical_lan(adapter->vdev->unit_address);
> + adapter->hcall_stats.free_lan++;
> } while (H_IS_LONG_BUSY(rc) || (rc == H_BUSY));
>
> try_again = 0;
[ ... ]
> +static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
> +{
> + unsigned long lpar_rc;
> + int i;
> +
> + netdev_dbg(adapter->netdev, "freeing all RX queues at once\n");
> +
> + do {
> + lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> + adapter->hcall_stats.free_lan++;
> + } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> +
> + if (lpar_rc != H_SUCCESS) {
> + netdev_err(adapter->netdev,
> + "h_free_logical_lan failed: %ld\n", lpar_rc);
> + }
> +
> + for (i = 0; i < adapter->num_rx_queues; i++)
> + adapter->queue_handle[i] = 0;
> +}
[Severity: High]
This isn't a bug introduced by this patch - the baseline ibmveth_close()
already had the same log-and-continue loop ("h_free_logical_lan failed with
%lx, continuing") followed by ibmveth_free_buffer_pools() - but the new
helper makes it harder to fix later, so a question about it.
The retry loop here only retries H_BUSY and H_IS_LONG_BUSY. Any other
status (H_PARAMETER, H_STATE, H_HARDWARE) leaves the loop, is logged, and
then the helper returns void, so neither ibmveth_close() nor the
ibmveth_open() unwind can see that the logical LAN may still be registered.
Right after the netdev_err(), queue_handle[] is cleared unconditionally:
for (i = 0; i < adapter->num_rx_queues; i++)
adapter->queue_handle[i] = 0;
Should that clearing be conditional on H_SUCCESS? Once multi_queue is
enabled later in the series, ibmveth_deregister_single_rx_queue() bails out
on "if (!qh)", so zeroing the handles after a failed free removes the only
reference a per-queue H_FREE_LOGICAL_LAN_QUEUE retry would need.
Would it make sense for this helper to return the hypercall status so the
callers can decide, rather than returning void?
[ ... ]
> @@ -1309,37 +1407,23 @@ static int ibmveth_open(struct net_device *netdev)
> if (rc)
> goto out_free_filter_list;
>
> - mac_address = ether_addr_to_u64(netdev->dev_addr);
> -
> - rxq_desc.fields.flags_len = IBMVETH_BUF_VALID |
> - adapter->rx_queue[0].queue_len;
> - rxq_desc.fields.address = adapter->rx_queue[0].queue_dma;
> -
> - adapter->queue_irq[0] = netdev->irq;
> - ibmveth_disable_irq(adapter, 0);
> -
> - lpar_rc = ibmveth_register_logical_lan(adapter, rxq_desc, mac_address);
> -
> - if (lpar_rc != H_SUCCESS) {
> - netdev_err(netdev, "h_register_logical_lan failed with %ld\n",
> - lpar_rc);
> - netdev_err(netdev, "buffer TCE:0x%llx filter TCE:0x%llx rxq "
> - "desc:0x%llx MAC:0x%llx\n",
> - adapter->buffer_list_dma[0],
> - adapter->filter_list_dma,
> - rxq_desc.desc,
> - mac_address);
> - rc = -ENONET;
> + rc = ibmveth_alloc_buffer_pools(adapter);
> + if (rc)
> goto out_free_queue_mem;
> - }
>
> - rc = ibmveth_alloc_buffer_pools(adapter);
> + rc = ibmveth_register_rx_queues(adapter, mac_address);
> if (rc)
> - goto out_unregister_lan;
> + goto out_free_buffer_pools;
> +
> + rc = netif_set_real_num_rx_queues(netdev, adapter->num_rx_queues);
> + if (rc) {
> + netdev_err(netdev, "failed to set number of rx queues\n");
> + goto out_unregister_queues;
> + }
>
> rc = ibmveth_setup_rx_interrupts(adapter);
> if (rc)
> - goto out_unregister_lan;
> + goto out_free_all_queues; /* setup already disposed IRQs */
[Severity: Low]
The changelog says:
"failures after successful LAN registration enter
out_unregister_queues, then fall through to out_free_buffer_pools"
Is that the label graph the code actually has? Only the
netif_set_real_num_rx_queues() failure jumps to out_unregister_queues. The
ibmveth_setup_rx_interrupts() failure goes straight to
out_free_all_queues, and the ibmveth_alloc_tx_resources() failure goes to
out_cleanup_rx_interrupts which then does "goto out_free_all_queues".
All three do reach ibmveth_free_all_queues() and
ibmveth_free_buffer_pools(), so the "RX pools are not leaked" part still
holds; only the description of which label the paths enter looks off.
Could the changelog be adjusted to match?
>
> netdev_dbg(netdev, "initial replenish cycle\n");
> ibmveth_schedule_rx_queue(adapter, 0);
> @@ -1357,10 +1441,12 @@ static int ibmveth_open(struct net_device *netdev)
>
> out_cleanup_rx_interrupts:
> ibmveth_cleanup_rx_interrupts(adapter);
> -out_unregister_lan:
> - do {
> - lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> - } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> + goto out_free_all_queues; /* cleanup already disposed IRQs */
> +out_unregister_queues:
> + ibmveth_dispose_subordinate_irq_mappings(adapter);
> +out_free_all_queues:
> + ibmveth_free_all_queues(adapter);
> +out_free_buffer_pools:
> ibmveth_free_buffer_pools(adapter);
> out_free_queue_mem:
> ibmveth_cleanup_rx_resources(adapter);
[Severity: High]
Same pre-existing concern as above, now on the open-failure unwind: since
ibmveth_free_all_queues() cannot report failure, this path unconditionally
continues into ibmveth_free_buffer_pools() (dma_unmap_single() plus
dev_kfree_skb_any() of every posted RX buffer) and
ibmveth_cleanup_rx_resources() (dma_unmap of the buffer-list page,
free_page(), dma_free_coherent() of the RX ring).
If H_FREE_LOGICAL_LAN returned something other than H_SUCCESS and the LAN
is still registered, can PHYP keep writing received frames and the
rx_no_buffer counter into those unmapped or freed pages?
> @@ -1395,15 +1480,7 @@ static int ibmveth_close(struct net_device *netdev)
> synchronize_net();
>
> ibmveth_update_rx_no_buffer(adapter);
> - /* Full LAN teardown (subordinates arrive with register helpers). */
> - do {
> - lpar_rc = h_free_logical_lan(adapter->vdev->unit_address);
> - } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> - if (lpar_rc != H_SUCCESS) {
> - netdev_err(adapter->netdev,
> - "h_free_logical_lan failed with %lx, continuing\n",
> - lpar_rc);
> - }
> + ibmveth_free_all_queues(adapter);
[Severity: High]
This is the pre-existing log-and-continue behaviour moved into the helper,
so the same question applies on the close path: after
ibmveth_free_all_queues() the function proceeds to
ibmveth_free_tx_resources(), ibmveth_free_buffer_pools() and
ibmveth_cleanup_rx_resources() regardless of the hypercall result.
The changelog states:
"free_all_queues() runs before free_buffer_pools() on open failure
and close, so PHYP cannot retain a registered LAN while its
DMA-backed RX pools are released."
Does that hold when the hypercall does not succeed? The ordering only
gives that guarantee on H_SUCCESS; on a non-busy failure the LAN may still
be registered when the pools go away. Could the changelog note that, or
could the helper propagate the status so close() can keep the DMA
mappings?
I also checked the end of the series (1f83fd44d867) and
ibmveth_free_all_queues() is unchanged there, so this is not addressed by
a later patch in the series.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index bf5dd5703773d..fae3473cc4985 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -268,6 +268,17 @@ static int pool_active[] = { 1, 1, 0, 0, 1};
>
> #define IBM_VETH_INVALID_MAP ((u16)0xffff)
>
> +struct ibmveth_hcall_stats {
> + u64 reg_lan_queue; /* H_REG_LOGICAL_LAN_QUEUE */
> + u64 reg_lan; /* H_REGISTER_LOGICAL_LAN */
> + u64 add_bufs_queue; /* H_ADD_LOGICAL_LAN_BUFFERS_QUEUE */
> + u64 add_bufs; /* H_ADD_LOGICAL_LAN_BUFFERS */
> + u64 add_buf; /* H_ADD_LOGICAL_LAN_BUFFER */
> + u64 free_lan_queue; /* H_FREE_LOGICAL_LAN_QUEUE */
> + u64 free_lan; /* H_FREE_LOGICAL_LAN */
> + u64 send_lan; /* H_SEND_LOGICAL_LAN */
> +};
[Severity: Low]
This isn't a bug, just a note on what was checked. At this commit only
reg_lan and free_lan have writers and none of the eight counters has a
reader, even though h_add_logical_lan_buffer(),
h_add_logical_lan_buffers() and h_send_logical_lan() already exist in the
driver.
Looking at the end of the series (1f83fd44d867), all eight counters get
both producers and a consumer through ethtool -S ("hcall_reg_lan_queue",
"hcall_reg_lan", "hcall_add_bufs_queue", "hcall_add_bufs", "hcall_add_buf",
"hcall_free_lan_queue", "hcall_free_lan", "hcall_send_lan"), so this
resolves within the series and needs no action.
For completeness: reg_lan is incremented even when registration fails, and
free_lan is incremented once per H_BUSY retry. Given the field comments
name the hypercalls and the struct is described as hypercall statistics,
invocation counts look like the intent, so no change is requested here
either.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 08/15] ibmveth: Add queue-aware RX buffer submit helper for MQ
2026-08-14 7:36 ` [PATCH net-next v5 08/15] ibmveth: Add queue-aware RX buffer submit helper " Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add queue-aware RX buffer submit helper for MQ
This adds ibmveth_add_logical_lan_buffers() to pick between the legacy
single/multi buffer hcalls and the new per-queue H_ADD_LOGICAL_LAN_BUFFERS_QUEUE
hcall, threads queue_index through the RX replenish/poll helpers, and
introduces a per-queue replenish_lock plus deferred (post-unlock) failure
logging and reset scheduling.
A few questions below about the new locking scope and about some of the
error paths that changed behaviour.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 8519cad503225..58a639a962a67 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -262,6 +272,7 @@ ibmveth_alloc_rx_queues(struct ibmveth_adapter *adapter, int rxq_entries)
> adapter->rx_queue[i].index = 0;
> adapter->rx_queue[i].num_slots = rxq_entries;
> adapter->rx_queue[i].toggle = 1;
> + spin_lock_init(&adapter->rx_queue[i].replenish_lock);
[Severity: Medium]
Is it safe to re-initialize replenish_lock here, given that
ibmveth_alloc_rx_queues() runs on every ndo_open?
The driver restarts itself directly in several places, guarded only by
netif_running(), for example ibmveth_set_csum_offload():
if (netif_running(dev)) {
restart = 1;
ibmveth_close(dev);
}
...
if (restart)
rc2 = ibmveth_open(dev);
veth_pool_store() and the MTU change path do the same. Because
__LINK_STATE_START is never cleared on these paths, netpoll_poll_disable()
is never called, and netpoll_poll_dev() only bails out on !npinfo or a held
dev_lock (dev_lock is taken solely by netpoll_poll_disable() from
__dev_open/__dev_close_many).
So can this sequence happen while netconsole is configured on this device?
CPU0 (printk -> netconsole)
ibmveth_poll_controller()
ibmveth_replenish_task(adapter, 0)
spin_lock_irqsave(&rxq->replenish_lock, flags);
CPU1 (ethtool -K restart)
ibmveth_open()
ibmveth_alloc_rx_queues()
spin_lock_init(&adapter->rx_queue[0].replenish_lock);
That resets a held spinlock to unlocked, and the later unlock on CPU0 runs
against re-initialized state.
[ ... ]
> -/* replenish the buffers for a pool. note that we don't need to
> - * skb_reserve these since they are used for incoming...
> +/**
> + * ibmveth_add_logical_lan_buffers - Add receive buffers to hypervisor
> + * @adapter: ibmveth adapter structure
> + * @descs: array of buffer descriptors to add
> + * @filled: number of valid descriptors in the array
> + * @buff_size: size of each buffer (multi-queue mode only)
> + * @queue_index: RX queue index
> + *
> + * Return: hypervisor return code
> */
> -static void ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
> - struct ibmveth_buff_pool *pool)
> +static long ibmveth_add_logical_lan_buffers(struct ibmveth_adapter *adapter,
> + union ibmveth_buf_desc *descs,
> + int filled,
> + unsigned long buff_size,
> + int queue_index)
> +{
> + struct vio_dev *vdev = adapter->vdev;
> + unsigned long rc;
> +
> + if (adapter->multi_queue) {
> + unsigned long buffersznum = (buff_size << 32) | filled;
> + unsigned long ioba[IBMVETH_MAX_RX_PER_HCALL / 2] = {0};
> + unsigned long handle = adapter->queue_handle[queue_index];
> + int i;
[Severity: Low]
This call site passes adapter->queue_handle[queue_index], which includes
queue 0, but the kernel-doc of h_add_logical_lan_buffers_queue() in
ibmveth.h describes it as "Add buffers to subordinate queue" with
"@queue_handle: Queue handle from h_register_logical_lan_queue()", and its
Return section lists only H_SUCCESS/H_PARAMETER/H_HARDWARE.
The queue 0 handle comes from h_register_logical_lan_with_handle(), whose
own doc says the handle is "needed in multi-queue mode to use
h_add_logical_lan_buffers_queue() for all queues including queue 0", and
ibmveth_replenish_buffer_pool() has dedicated recovery for H_FUNCTION from
this hcall.
Could the h_add_logical_lan_buffers_queue() kernel-doc be updated to cover
the primary queue handle and H_FUNCTION?
[ ... ]
> + rc = h_add_logical_lan_buffers_queue(vdev->unit_address,
> + handle,
> + buffersznum,
> + ioba[0], ioba[1], ioba[2],
> + ioba[3], ioba[4], ioba[5]);
> + adapter->hcall_stats.add_bufs_queue++;
> + } else if (filled == 1) {
> + rc = h_add_logical_lan_buffer(vdev->unit_address,
> + descs[0].desc);
> + adapter->hcall_stats.add_buf++;
> + } else {
> + rc = h_add_logical_lan_buffers(vdev->unit_address,
> + descs[0].desc, descs[1].desc,
> + descs[2].desc, descs[3].desc,
> + descs[4].desc, descs[5].desc,
> + descs[6].desc, descs[7].desc);
> + adapter->hcall_stats.add_bufs++;
> + }
[Severity: Low]
IBMVETH_MAX_RX_PER_HCALL changes from 8 to 12 in this patch, which resizes
descs[] and correlators[] in ibmveth_replenish_buffer_pool() to 12, but this
legacy branch still hands only descs[0..7] to the 8-descriptor
h_add_logical_lan_buffers(), and the caller then accounts all "filled"
descriptors as posted (consumer_index advanced, pool->available
incremented).
The combination needed to hit that (multi_queue == 0 with
rx_buffers_per_hcall > 8) does not appear reachable, since probe now sets
IBMVETH_MAX_RX_REGULAR (8) or 1, and later in the series
ibmveth_apply_mq_fallback() clamps to IBMVETH_MAX_RX_REGULAR.
Would a clamp on filled here, or a BUILD_BUG_ON tying the legacy branch to
IBMVETH_MAX_RX_REGULAR, be worth adding so the arity contract is enforced
rather than implied?
Also, the 12-address MQ packing (ioba[4], ioba[5]) cannot be exercised at
this commit, since probe caps the batch at 8:
> - adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_PER_HCALL;
> + adapter->rx_buffers_per_hcall = IBMVETH_MAX_RX_REGULAR;
[ ... ]
> +/* Replenish the buffers for a pool.
> + * Caller must hold the per-queue replenish_lock. Do not printk here —
> + * netconsole on the same device can re-enter replenish_task.
> + */
> +static int ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
> + struct ibmveth_buff_pool *pool,
> + int queue_index,
> + struct ibmveth_replenish_fail *fail)
> {
[Severity: High]
The "do not printk here" invariant is documented for this function, but can
code called from inside the critical section still printk indirectly?
dma_map_single() in the fill loop goes through iommu_map_page() on pseries,
and the driver passes attrs = 0:
arch/powerpc/kernel/iommu.c:iommu_map_page() {
...
if (!(attrs & DMA_ATTR_NO_WARN) && printk_ratelimit()) {
dev_info(dev, "iommu_alloc failed, tbl %p vaddr %p npages %d\n", ...);
...
}
That is exactly the condition replenish hits (TCE table or CMO entitlement
exhausted under pressure). With netconsole on this device, the printk
recurses:
printk -> write_msg -> netpoll_send_udp -> __netpoll_send_skb
-> netpoll_poll_dev -> ibmveth_poll_controller
-> ibmveth_replenish_task -> spin_lock_irqsave(&rxq->replenish_lock)
Since the lock is already held on this CPU with interrupts disabled, does
this self-deadlock? The same re-entry also reaches
ibmveth_rxq_harvest_buffer() -> ibmveth_remove_buffer_from_pool(), which now
takes the same lock.
[ ... ]
> @@ -883,21 +971,85 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
> }
>
> /* replenish routine */
> -static void ibmveth_replenish_task(struct ibmveth_adapter *adapter)
> +static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
> + int queue_index)
> {
> - int i;
> + struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
> + struct ibmveth_replenish_fail fail = {};
> + unsigned long flags;
> + int i, rc;
> + int need_reset = 0;
> + int batch_fallback = 0;
> + int hcall_fail = 0;
> +
> + if (queue_index >= adapter->num_rx_queues) {
> + netdev_dbg(adapter->netdev,
> + "Skipping replenish for freed queue %d (num_queues=%d)\n",
> + queue_index, adapter->num_rx_queues);
> + return;
> + }
>
> adapter->replenish_task_cycles++;
[Severity: Medium]
Several adapter-global fields are updated while holding only a per-queue
lock, or none at all. replenish_task_cycles++ here is outside the lock, and
inside ibmveth_replenish_buffer_pool() / ibmveth_add_logical_lan_buffers()
the following are adapter-wide:
batch = adapter->rx_buffers_per_hcall;
...
adapter->rx_buffers_per_hcall = 1;
adapter->hcall_stats.add_buf/add_bufs/add_bufs_queue++;
adapter->replenish_add_buff_success/failure, replenish_no_mem
Two queues replenishing on different CPUs hold different replenish_lock
instances, so the intersection of locks protecting these is empty. Can this
lose or tear the ethtool-visible hcall_stats and replenish_* counters? Today
it is already racy between ibmveth_poll_controller() and NAPI for
replenish_task_cycles++, which is incremented before the lock is taken.
>
> - for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
> - struct ibmveth_buff_pool *pool = &adapter->rx_buff_pool[0][i];
> + spin_lock_irqsave(&rxq->replenish_lock, flags);
>
> - if (pool->active &&
> - (atomic_read(&pool->available) < pool->threshold))
> - ibmveth_replenish_buffer_pool(adapter, pool);
> + for (i = (IBMVETH_NUM_BUFF_POOLS - 1); i >= 0; i--) {
> + struct ibmveth_buff_pool *pool =
> + &adapter->rx_buff_pool[queue_index][i];
> +
> + if (pool->active && pool->free_map &&
> + (atomic_read(&pool->available) < pool->threshold)) {
> + rc = ibmveth_replenish_buffer_pool(adapter, pool,
> + queue_index, &fail);
[Severity: High]
How long can interrupts stay disabled here?
The lock is taken with spin_lock_irqsave() around the whole five-pool walk,
and ibmveth_replenish_buffer_pool() loops "while (remaining > 0)" doing
netdev_alloc_skb() plus dma_map_single() per buffer, then a hypercall per
batch. With pool_active { 1, 1, 0, 0, 1 } and pool_count { 256, 512, 256,
256, 256 }, a cold start posts roughly 1024 buffers, including 256 of
buff_size 64 KB.
ibmveth_open() hits this deterministically:
for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++)
ibmveth_replenish_task(adapter, i);
So every ifup does on the order of a thousand GFP_ATOMIC allocations
(some high order), a thousand IOMMU mappings and ~128 hypercalls with local
interrupts off. Previously this ran with interrupts enabled and no lock.
Can that trip the hard lockup watchdog or produce RCU stalls, and does
disabling BH/IRQ progress here also make the GFP_ATOMIC allocations more
likely to fail exactly when memory is tight? Since
ibmveth_remove_buffer_from_pool() now takes the same lock per received
packet, the NAPI consumer is also blocked for the whole region.
The changelog notes "Defer adapter-global counter atomics and irqsave
critical-section shorten to cover follow-up" - would it be better to shorten
the region in this patch, since as merged the region exists?
[ ... ]
> +out_unlock:
> ibmveth_update_rx_no_buffer(adapter);
[Severity: Low]
This helper is now called per queue, but ibmveth_update_rx_no_buffer() still
reads queue 0's buffer-list page:
__be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
Each RX queue has its own buffer-list page whose last 8 bytes hold that
queue's no-buffer drop count, so rx_no_buffer would under-report queues
1..N. This is addressed later in the series, where the helper takes
queue_index and uses buffer_list_addr[queue_index], and MQ is only turned on
by a later patch, so no wrong-page read looks reachable at this commit.
Would it be cleaner to make the helper queue-aware in the same patch that
makes its caller queue-aware?
> +
> + spin_unlock_irqrestore(&rxq->replenish_lock, flags);
> +
> + /* Log and schedule reset only after dropping replenish_lock. */
> + if (need_reset == IBMVETH_REPLENISH_RESET_MAP) {
> + netdev_info(adapter->netdev,
> + "Invalid RX free_map entry on queue %d, reset\n",
> + queue_index);
> + schedule_work(&adapter->work);
[Severity: Low]
This isn't a bug introduced by this patch, but the new schedule_work() sites
add more ways to arm adapter->work from NAPI context, and ibmveth_remove()
still cancels before it stops the device:
cancel_work_sync(&adapter->work);
...
unregister_netdev(netdev);
free_netdev(netdev);
RX activity between cancel_work_sync() and the ndo_close() done by
unregister_netdev() can re-queue the work, which may then run after
free_netdev(). The ordering is swapped later in this series
(unregister_netdev() first, then cancel_work_sync()), with a comment
describing exactly this window.
> + } else if (need_reset == IBMVETH_REPLENISH_RESET_MQ) {
> + dev_err_ratelimited(&adapter->netdev->dev,
> + "MQ buffer add H_FUNCTION (q=%d, batch=%u), reset\n",
> + queue_index, fail.batch);
> + schedule_work(&adapter->work);
> + }
> +
> + if (batch_fallback)
> + dev_warn_ratelimited(&adapter->netdev->dev,
> + "Legacy batch add H_FUNCTION (batch=%u), fallback\n",
> + fail.batch);
> +
> + if (hcall_fail)
> + dev_warn_ratelimited(&adapter->netdev->dev,
> + "RX %s failed: filled=%u, rc=%lu, batch=%u\n",
> + adapter->multi_queue ?
> + "h_add_logical_lan_buffers_queue" :
> + (fail.batch == 1 ?
> + "h_add_logical_lan_buffer" :
> + "h_add_logical_lan_buffers"),
> + fail.filled, fail.lpar_rc, fail.batch);
> }
[Severity: Low]
Does this pick the wrong hcall name? The dispatch in
ibmveth_add_logical_lan_buffers() selects on filled, not batch:
} else if (filled == 1) {
rc = h_add_logical_lan_buffer(vdev->unit_address, descs[0].desc);
If batch is 8 and the fill loop breaks early after one buffer
(netdev_alloc_skb() or dma_mapping_error() failure), the failing call was
h_add_logical_lan_buffer(), but this prints h_add_logical_lan_buffers.
Should the selector be fail.filled == 1?
Separately, one "struct ibmveth_replenish_fail fail" and the
batch_fallback/hcall_fail flags are shared by all five pool iterations, so a
later pool's failure overwrites an earlier pool's rc/filled/batch, and one
log line can mix values from two pools.
>
> /* empty and free ana buffer pool - also used to do cleanup in error paths */
> @@ -932,6 +1084,14 @@ static void ibmveth_free_buffer_pool(struct ibmveth_adapter *adapter,
> kfree(pool->skbuff);
> pool->skbuff = NULL;
> }
> +
> + /*
> + * Keep probe/sysfs geometry (active, size, buff_size, threshold).
> + * Clearing active here was a v3 ifdown/up regression: open skips
> + * !active pools, so reopen posted no RX buffers (TX OK, ARP/RX
> + * dead) at any queue count, including RX=8 with no -L.
> + */
> + atomic_set(&pool->available, 0);
> }
[Severity: Low]
This comment describes a regression that does not exist in the tree: no
version of ibmveth_free_buffer_pool() ever cleared pool->active, and this
hunk is additive only. The same applies to the new comment in
ibmveth_rxq_correlator_valid() ("free_buffer_pool() clears skbuff but used
to leave size/active set").
References to "v3" and to lab options like "RX=8 with no -L" aren't
meaningful to a reader of mainline. Could the comments be rewritten in terms
of the code as it stands, and could the functional change here
(atomic_set(&pool->available, 0)) be mentioned in the commit message? Since
ibmveth_alloc_buffer_pool() already zeroes available on reallocation, it is
hard to tell from the patch whether this is a fix or a no-op.
[Severity: High]
This is a pre-existing race, but the new locking makes it look handled while
leaving it open. ibmveth_free_buffer_pool() kfrees pool->free_map,
pool->skbuff and pool->dma_addr without taking the per-queue
replenish_lock, while ibmveth_replenish_task() checks and uses those arrays
under that lock:
if (pool->active && pool->free_map &&
(atomic_read(&pool->available) < pool->threshold)) {
So the check is a TOCTOU: replenish touches the pool arrays with
{rxq->replenish_lock} held, the free path touches and frees them with no
lock, giving an empty lock intersection.
Reachable through a direct close, which does not run
netpoll_poll_disable():
CPU0 (printk -> netconsole)
ibmveth_poll_controller() -> ibmveth_replenish_task(adapter, 0)
passes "pool->free_map != NULL", then dereferences
pool->free_map / pool->skbuff / pool->dma_addr
CPU1 (ethtool -K / veth_pool_store / MTU change)
ibmveth_close() -> ibmveth_free_buffer_pools()
-> kfree(pool->free_map)
Can that produce a use-after-free of those arrays, and post freed buffers to
the hypervisor? Should ibmveth_free_buffer_pool() take replenish_lock (or
should the pool teardown otherwise be serialized against replenish)?
[ ... ]
> +static bool ibmveth_rxq_correlator_valid(struct ibmveth_adapter *adapter,
> + int queue_index, u64 correlator)
> +{
> + unsigned int pool = correlator >> 32;
> + unsigned int index = correlator & 0xffffffffUL;
> + struct ibmveth_buff_pool *bpool;
> +
> + if (pool >= IBMVETH_NUM_BUFF_POOLS)
> + return false;
> +
> + bpool = &adapter->rx_buff_pool[queue_index][pool];
> +
> + /* init_buffer_pool() sets size for inactive pools; free_buffer_pool()
> + * clears skbuff but used to leave size/active set. Require a live
> + * pool with allocated arrays before indexing.
> + */
> + if (!bpool->active || !bpool->skbuff || !bpool->free_map)
> + return false;
> +
> + return index < bpool->size;
> +}
[Severity: Medium]
Do the in-tree KUnit cases still pass with the added
"!bpool->free_map" precondition? Neither
ibmveth_remove_buffer_from_pool_test() nor ibmveth_rxq_get_buffer_test()
allocates free_map - both only kunit_kcalloc() pool->skbuff.
That makes validation fail before the skbuff[index] NULL check, so the two
-EFAULT expectations should now receive -EINVAL:
correlator = (u64)0 | 0;
pool->skbuff[0] = NULL;
KUNIT_EXPECT_EQ(test, -EFAULT,
ibmveth_remove_buffer_from_pool(adapter,
correlator, 0, false));
and the positive expectation in ibmveth_rxq_get_buffer_test() should receive
NULL:
pool->skbuff[0] = skb;
adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter, 0));
The index-out-of-range expectations also pass vacuously now, so the bound
check they were written for is no longer covered. The patch updates the
kernel-doc of both tests but not the fixtures - should the fixtures also
allocate free_map?
[ ... ]
> * Return:
> * * %0 - success
> * * %-EINVAL - correlator maps to pool or index out of range
> * * %-EFAULT - pool and index map to null skb
> */
> static int ibmveth_remove_buffer_from_pool(struct ibmveth_adapter *adapter,
> - u64 correlator, bool reuse)
> + u64 correlator, int queue_index,
> + bool reuse)
> {
[Severity: Low]
The documented meaning of -EINVAL is now incomplete: via
ibmveth_rxq_correlator_valid() this function also returns -EINVAL for an
inactive pool and for a pool whose skbuff/free_map arrays are not
allocated, which is a different condition from "out of range". The new
ibmveth_rxq_harvest_buffer() kernel-doc reinforces the narrow reading by
telling callers that -EINVAL/-EFAULT mean a "corrupt slot" safe to skip.
Could the return-code documentation be extended to cover the not-live-pool
case?
> + struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
> unsigned int pool = correlator >> 32;
> unsigned int index = correlator & 0xffffffffUL;
> unsigned int free_index;
> struct sk_buff *skb;
> + unsigned long flags;
> + int rc = 0;
>
> - if (WARN_ON(pool >= IBMVETH_NUM_BUFF_POOLS) ||
> - WARN_ON(index >= adapter->rx_buff_pool[0][pool].size)) {
> - schedule_work(&adapter->work);
> - return -EINVAL;
> + spin_lock_irqsave(&rxq->replenish_lock, flags);
> +
> + if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator)) {
> + rc = -EINVAL;
> + goto out_unlock;
> }
>
> - skb = adapter->rx_buff_pool[0][pool].skbuff[index];
> - if (WARN_ON(!skb)) {
> - schedule_work(&adapter->work);
> - return -EFAULT;
> + skb = adapter->rx_buff_pool[queue_index][pool].skbuff[index];
> + if (!skb) {
> + rc = -EFAULT;
> + goto out_unlock;
> }
[ ... ]
> static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter,
> - bool reuse)
> + int queue_index, bool reuse)
> {
> + struct ibmveth_rx_q *rxq = &adapter->rx_queue[queue_index];
> u64 cor;
> int rc;
>
> - struct ibmveth_rx_q *rxq = &adapter->rx_queue[0];
> -
> cor = rxq->queue_addr[rxq->index].correlator;
> - rc = ibmveth_remove_buffer_from_pool(adapter, cor, reuse);
> - if (unlikely(rc))
> + rc = ibmveth_remove_buffer_from_pool(adapter, cor, queue_index, reuse);
> + if (unlikely(rc)) {
> + /* Skip a corrupt slot without claiming pool ownership. */
> + if (rc == -EINVAL || rc == -EFAULT)
> + ibmveth_rxq_advance(rxq);
> return rc;
> -
> - if (++adapter->rx_queue[0].index == adapter->rx_queue[0].num_slots) {
> - adapter->rx_queue[0].index = 0;
> - adapter->rx_queue[0].toggle = !adapter->rx_queue[0].toggle;
> }
[Severity: Medium]
The reset that previously repaired this state is gone -
ibmveth_remove_buffer_from_pool() no longer calls
schedule_work(&adapter->work) for -EINVAL/-EFAULT, and here the ring is just
advanced. Does that permanently retire the buffer the hypervisor already
consumed?
On this path free_map does not regain the index, the
atomic_dec(&pool->available) is skipped, and pool->skbuff[index] plus its
DMA mapping stay owned by nobody.
Since replenish is gated on available < threshold (size * 7 / 8):
if (pool->active && pool->free_map &&
(atomic_read(&pool->available) < pool->threshold)) {
after roughly size/8 such events, does replenish stop for that pool
permanently, with the link still up and no recovery short of ifdown/up?
The poll "recycling invalid buffer" path and ibmveth_drain_rx_queue() both
swallow these errors without escalating:
/* -EINVAL/-EFAULT already advanced past the slot. */
if (rc == -EINVAL || rc == -EFAULT) {
drained++;
continue;
}
At the end of the series only the get_buffer path escalates via
ibmveth_poll_skip_bad_correlator(), and that code's own comment ("Residual
stale slot after resize") suggests stale correlators do occur. Should the
recycle and drain paths also reconcile available or schedule a reset?
[ ... ]
> @@ -2127,34 +2340,41 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
> - skb = ibmveth_rxq_get_buffer(adapter);
> + skb = ibmveth_rxq_get_buffer(adapter, queue_index);
> if (unlikely(!skb))
> break;
[Severity: Low]
This isn't a bug introduced by this patch, but breaking out here leaves
rxq->index and rxq->toggle untouched, so the descriptor stays pending. The
tail of ibmveth_poll() then completes NAPI, re-enables the IRQ, and:
if (ibmveth_rxq_pending_buffer(adapter, queue_index) &&
napi_schedule(napi)) {
rc = ibmveth_disable_irq(adapter, queue_index);
WARN_ON(rc);
goto restart_poll;
}
Can that spin on the same slot in softirq context when a VALID descriptor
has a correlator that fails validation, or whose pool->skbuff[index] is
NULL? The changelog says this lands with the poll harden patch ("Bad
queue_index poll path: napi_complete before return lands with poll
harden"), and later in the series
ibmveth_poll_skip_bad_correlator() schedules a reset and advances the ring.
[ ... ]
> diff --git a/drivers/net/ethernet/ibm/ibmveth.h b/drivers/net/ethernet/ibm/ibmveth.h
> index fae3473cc4985..d02444d5b3b89 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.h
> +++ b/drivers/net/ethernet/ibm/ibmveth.h
> @@ -259,7 +261,8 @@ static inline long h_illan_attributes(unsigned long unit_address,
> #define IBMVETH_DEFAULT_QUEUES 8U
> #define IBMVETH_MAX_RX_QUEUES 1U
> #define IBMVETH_DEFAULT_RX_QUEUES 1U
> -#define IBMVETH_MAX_RX_PER_HCALL 8U
> +#define IBMVETH_MAX_RX_REGULAR 8U
> +#define IBMVETH_MAX_RX_PER_HCALL 12U
The change of IBMVETH_MAX_RX_PER_HCALL from 8 to 12 is described in the
commit message only as "up to 12 buffers" for the MQ hcall - could the
commit message also state that the existing constant is being redefined, so
that the descs[]/correlators[] resize in ibmveth_replenish_buffer_pool() and
the new IBMVETH_MAX_RX_REGULAR use in probe are easy to follow?
> @@ -301,6 +304,7 @@ struct ibmveth_rx_q {
> dma_addr_t queue_dma;
> u32 queue_len;
> struct ibmveth_rx_q_entry *queue_addr;
> + spinlock_t replenish_lock; /* per-queue buffer replenish */
> };
This isn't a bug, but the new member uses tabs while the surrounding members
in this struct use the older space-based alignment - was the mixed style
intentional?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 09/15] ibmveth: Harden RX poll path with helpers
2026-08-14 7:36 ` [PATCH net-next v5 09/15] ibmveth: Harden RX poll path with helpers Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Harden RX poll path with helpers
This splits the inline RX slot handling in ibmveth_poll() into a set of
small helpers (stopping / harvest_slot / recycle_invalid /
skip_bad_correlator / drop_oversize / deliver_frame / bump_invalid) so the
NAPI callback stays a budget loop. It also moves the reset escalation out
of ibmveth_rxq_get_buffer() into ibmveth_poll_skip_bad_correlator() and
adds stopping checks so close/napi_disable does not re-arm PHYP.
A few questions below.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 58a639a962a67..86299c62d4ecf 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -1346,10 +1346,8 @@ ibmveth_rxq_get_buffer(struct ibmveth_adapter *adapter,
> unsigned int pool = correlator >> 32;
> unsigned int index = correlator & 0xffffffffUL;
>
> - if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator)) {
> - schedule_work(&adapter->work);
> + if (!ibmveth_rxq_correlator_valid(adapter, queue_index, correlator))
> return NULL;
> - }
>
> return adapter->rx_buff_pool[queue_index][pool].skbuff[index];
> }
> @@ -2338,125 +2336,230 @@ static void ibmveth_rx_csum_helper(struct sk_buff *skb,
> }
> }
>
> +static void ibmveth_poll_bump_invalid(struct ibmveth_adapter *adapter,
> + int queue_index)
> +{
> + adapter->rx_invalid_buffer++;
> +}
> +
> +static bool ibmveth_poll_stopping(struct net_device *netdev,
> + struct napi_struct *napi)
> +{
> + return !netif_running(netdev) || napi_disable_pending(napi);
> +}
> +
> +static bool ibmveth_poll_harvest_slot(struct ibmveth_adapter *adapter,
> + int queue_index, bool reuse)
> +{
> + int rc = ibmveth_rxq_harvest_buffer(adapter, queue_index, reuse);
> +
> + return !rc || rc == -EINVAL || rc == -EFAULT;
> +}
> +
> +static bool ibmveth_poll_recycle_invalid(struct net_device *netdev,
> + struct ibmveth_adapter *adapter,
> + int queue_index)
> +{
> + netdev_dbg(netdev, "recycling invalid buffer\n");
> + ibmveth_poll_bump_invalid(adapter, queue_index);
> + return ibmveth_poll_harvest_slot(adapter, queue_index, true);
> +}
> +
> +static bool ibmveth_poll_skip_bad_correlator(struct net_device *netdev,
> + struct ibmveth_adapter *adapter,
> + int queue_index)
> +{
> + if (net_ratelimit())
> + netdev_err(netdev,
> + "bad correlator on queue %d, skipping slot\n",
> + queue_index);
> + /* Residual stale slot after resize: recover via reset rather
> + * than spinning forever. Always escalate; only the log is
> + * rate-limited.
> + */
> + schedule_work(&adapter->work);
> + ibmveth_poll_bump_invalid(adapter, queue_index);
> + return ibmveth_poll_harvest_slot(adapter, queue_index, true);
> +}
> +
> +static bool ibmveth_poll_drop_oversize(struct net_device *netdev,
> + struct ibmveth_adapter *adapter,
> + int queue_index, unsigned int off,
> + unsigned int len, unsigned int room)
> +{
> + if (net_ratelimit())
> + netdev_err(netdev,
> + "RX frame %u+%u exceeds buffer %u on queue %d, dropping\n",
> + off, len, room, queue_index);
> + ibmveth_poll_bump_invalid(adapter, queue_index);
> + return ibmveth_poll_harvest_slot(adapter, queue_index, true);
> +}
[Severity: Medium]
The changelog describes this as splitting a nested switchyard into helpers,
but a few behaviour changes appear to ride along. Could they be called out
in the changelog, or split out?
First, ibmveth_poll_bump_invalid() is now called from both
ibmveth_poll_drop_oversize() and ibmveth_poll_skip_bad_correlator(), so the
exported rx_invalid_buffer counter also counts oversize frames and skipped
slots. Is that intended, and should it be mentioned?
Second, before this patch schedule_work() ran only when
ibmveth_rxq_correlator_valid() failed, and the poll loop simply broke out.
Now ibmveth_poll_skip_bad_correlator() escalates unconditionally, so it also
covers the valid-correlator-with-NULL-skbuff case, which returns -EFAULT
from ibmveth_remove_buffer_from_pool(). ibmveth_reset() does:
rtnl_lock();
dev_close(adapter->netdev);
dev_open(adapter->netdev, NULL);
so does this turn a condition that previously cost one aborted poll into a
full close/open link flap? The changelog says "skip_bad_correlator owns
reset escalation", which reads as parity, but the escalation scope looks
strictly larger.
Third, the new off/len test in ibmveth_poll_deliver_frame() below appears to
be the first bound check applied to the PHYP-supplied offset and length
before skb_reserve()/skb_put(). Should that carry a Fixes: tag? As it
stands it is mixed into roughly 190 lines of code motion, which makes it
hard to pick up on its own.
> +
> +/**
> + * ibmveth_poll_deliver_frame - Build SKB from one valid RX slot and GRO it
> + * @napi: NAPI context for this RX queue
> + * @adapter: ibmveth adapter
> + * @netdev: net_device for @adapter
> + * @queue_index: RX queue index
> + *
> + * Return: 1 frame delivered, 0 if the slot was skipped cleanly, -1 on error.
> + */
> +static int ibmveth_poll_deliver_frame(struct napi_struct *napi,
> + struct ibmveth_adapter *adapter,
> + struct net_device *netdev,
> + int queue_index)
> +{
> + struct sk_buff *skb, *new_skb;
> + unsigned int room, off, len;
> + int length, offset, csum_good, lrg_pkt;
> + __sum16 iph_check = 0;
> + u16 mss = 0;
> + int rc;
> +
> + length = ibmveth_rxq_frame_length(adapter, queue_index);
> + offset = ibmveth_rxq_frame_offset(adapter, queue_index);
> + csum_good = ibmveth_rxq_csum_good(adapter, queue_index);
> + lrg_pkt = ibmveth_rxq_large_packet(adapter, queue_index);
> +
> + skb = ibmveth_rxq_get_buffer(adapter, queue_index);
> + if (unlikely(!skb)) {
> + if (!ibmveth_poll_skip_bad_correlator(netdev, adapter,
> + queue_index))
> + return -1;
> + return 0;
> + }
> +
> + room = skb_tailroom(skb);
> + off = offset;
> + len = length;
> + if (unlikely(off >= room || len > room - off)) {
> + if (!ibmveth_poll_drop_oversize(netdev, adapter, queue_index,
> + off, len, room))
> + return -1;
> + return 0;
> + }
[ ... ]
> + skb_put(skb, length);
> + skb->protocol = eth_type_trans(skb, netdev);
> +
> + if (skb->protocol == cpu_to_be16(ETH_P_IP))
> + iph_check = ip_hdr(skb)->check;
> +
> + if ((length > netdev->mtu + ETH_HLEN) || lrg_pkt ||
> + iph_check == 0xffff) {
> + ibmveth_rx_mss_helper(skb, mss, lrg_pkt);
> + adapter->rx_large_packets++;
> + }
[Severity: High]
Is skb->network_header set for this skb at this point? The old code read
the checksum through skb->data:
if (skb->protocol == cpu_to_be16(ETH_P_IP)) {
struct iphdr *iph = (struct iphdr *)skb->data;
iph_check = iph->check;
}
ip_hdr(skb) resolves to skb_network_header(skb), which is
skb->head + skb->network_header, and there is no was-it-set validation.
eth_type_trans() only does skb_reset_mac_header() plus eth_skb_pull_mac(),
and the driver's single skb_reset_network_header() call sits inside
ibmveth_rx_csum_helper(), which runs after this read. For skbs from
netdev_alloc_skb(), __finalize_skb_around() poisons mac_header and
transport_header to ~0U but leaves network_header at 0.
If that reading is right, ip_hdr(skb) points at skb->head and iph_check
comes out of the uninitialized headroom rather than the received IP header.
Two effects follow:
A PHYP-without-PLSO large-send frame whose IP checksum is 0xffff, whose
length is within mtu + ETH_HLEN, and whose large packet bit is clear is no
longer recognised, so gso_size and gso_segs are never set.
When the stale headroom bytes happen to be 0xffff, ibmveth_rx_mss_helper()
runs on an ordinary frame with lrg_pkt == 0 and takes the offset branch:
} else if (offset) {
skb_shinfo(skb)->gso_size = ntohs(tcph->check);
tcph->check = 0;
}
which zeroes the real TCP checksum and marks the frame as GSO with a
gso_size taken from that checksum, before napi_gro_receive().
Would keeping (struct iphdr *)skb->data here be better, matching what
ibmveth_rx_mss_helper() and ibmveth_rx_csum_helper() do for the same skb at
the same stage? This same line is still present in the later patches of the
series.
> +
> + if (csum_good) {
> + skb->ip_summed = CHECKSUM_UNNECESSARY;
> + ibmveth_rx_csum_helper(skb, adapter);
> + }
> +
> + napi_gro_receive(napi, skb);
> +
> + netdev->stats.rx_packets++;
> + netdev->stats.rx_bytes += length;
> +
> + return 1;
> +}
> +
> static int ibmveth_poll(struct napi_struct *napi, int budget)
> {
> struct net_device *netdev = napi->dev;
> struct ibmveth_adapter *adapter = netdev_priv(netdev);
> int frames_processed = 0;
> int queue_index, rc;
> - u16 mss = 0;
>
> queue_index = napi - adapter->napi;
>
> + if (WARN_ON(queue_index < 0 ||
> + queue_index >= adapter->num_rx_queues)) {
> + napi_complete_done(napi, 0);
> + return 0;
> + }
> +
> + if (ibmveth_poll_stopping(netdev, napi)) {
> + napi_complete_done(napi, 0);
> + return 0;
> + }
> +
> restart_poll:
> while (frames_processed < budget) {
> + if (ibmveth_poll_stopping(netdev, napi))
> + break;
> +
> if (!ibmveth_rxq_pending_buffer(adapter, queue_index))
> break;
>
> smp_rmb();
> if (!ibmveth_rxq_buffer_valid(adapter, queue_index)) {
> wmb(); /* suggested by larson1 */
> - adapter->rx_invalid_buffer++;
> - netdev_dbg(netdev, "recycling invalid buffer\n");
> - rc = ibmveth_rxq_harvest_buffer(adapter,
> - queue_index, true);
> - if (unlikely(rc))
> + if (!ibmveth_poll_recycle_invalid(netdev, adapter,
> + queue_index))
> break;
> } else {
[ ... ]
> - skb = ibmveth_rxq_get_buffer(adapter, queue_index);
> - if (unlikely(!skb))
> + rc = ibmveth_poll_deliver_frame(napi, adapter, netdev,
> + queue_index);
> + if (rc < 0)
> break;
[ ... ]
> - netdev->stats.rx_packets++;
> - netdev->stats.rx_bytes += length;
> - frames_processed++;
> + if (rc > 0)
> + frames_processed++;
> }
> }
[Severity: Medium]
Since skipped slots return 0 and only rc > 0 bumps frames_processed, does
while (frames_processed < budget) still bound the work done in one poll
call?
Where the pre-patch code did:
skb = ibmveth_rxq_get_buffer(adapter, queue_index);
if (unlikely(!skb))
break;
a run of stale or oversize descriptors now walks every pending slot in the
ring while reporting zero work. rxq_entries is 1 plus the sum of all pool
sizes, so that is on the order of a thousand slots per invocation, with
schedule_work(&adapter->work) called once per skipped slot from
ibmveth_poll_skip_bad_correlator().
The recycle-invalid branch already had this shape before the patch, but this
extends it to the bad-correlator and oversize outcomes and adds the per-slot
schedule_work().
>
> ibmveth_replenish_task(adapter, queue_index);
>
> + if (ibmveth_poll_stopping(netdev, napi)) {
> + napi_complete_done(napi, frames_processed);
> + /* After complete, must not return budget (NAPI resched). */
> + if (frames_processed < budget)
> + return frames_processed;
> + return budget - 1;
> + }
[Severity: Medium]
This branch hands frames_processed to napi_complete_done() but returns a
different value to the core. With budget == 0, frames_processed is 0, so
0 < budget is false and the callback returns -1.
budget == 0 is a real caller value, and ibmveth registers
.ndo_poll_controller:
net/core/netpoll.c:poll_one_napi() {
work = napi->poll(napi, 0);
WARN_ONCE(work, "%pS exceeded budget in poll\n", napi->poll);
trace_napi_poll(napi, work, 0);
}
so a -1 return would trip that WARN_ONCE and record work = -1 in the
tracepoint.
Documentation/networking/napi.rst also states:
If the ``budget`` is 0 napi_complete_done() should never be called.
which this branch and the two new entry-time paths above (the WARN_ON
queue_index path and the first ibmveth_poll_stopping() path, both calling
napi_complete_done(napi, 0)) do without checking budget. Would the
documented form fit better here:
if (budget && napi_complete_done(napi, work_done))
return min(work_done, budget - 1);
Reaching the -1 case needs ibmveth_poll_stopping() to be false at the entry
check and true after the loop. netpoll_poll_dev() requires
netif_running(dev), so the flip would come from napi_disable_pending(), which
ibmveth_change_mtu() and veth_pool_store() produce via ibmveth_close() ->
ibmveth_cleanup_rx_interrupts() -> napi_disable() while IFF_UP is still set.
Separately, returning budget - 1 on a full-budget stopping exit under-counts
the work by one frame. The return budget - 1 is still present in the later
patches of the series.
>
> if (frames_processed == budget)
> goto out;
>
> if (!napi_complete_done(napi, frames_processed))
> goto out;
>
> - /* We think we are done - reenable interrupts,
> - * then check once more to make sure we are done.
> + /*
> + * napi_disable() sets DISABLE then waits for this poll. Without a
> + * second stopping check here, enable_irq() can re-arm PHYP after
> + * resize already masked the queue — late IRQs then hit the handler
> + * after num_rx_queues was published lower (lab WARN at interrupt).
> */
> + if (ibmveth_poll_stopping(netdev, napi))
> + goto out;
> +
> rc = ibmveth_enable_irq(adapter, queue_index);
[Severity: Medium]
This is a pre-existing pattern and not introduced by this patch, since the
baseline calls ibmveth_enable_irq() and ibmveth_rxq_pending_buffer() after
napi_complete_done() with no check at all. Still, the new check is a
check-then-act on NAPI_STATE_DISABLE and __LINK_STATE_START: once
napi_complete_done() has cleared NAPI_STATE_SCHED, napi_disable() can return
and the close path can proceed while this poll instance is still executing
its tail.
Looking at ibmveth_close(), the synchronize_net() that sits after
ibmveth_cleanup_rx_interrupts() and before ibmveth_free_all_queues():
ibmveth_cleanup_rx_interrupts(adapter);
/* Wait for softirq/poll that already passed shutdown checks. */
synchronize_net();
ibmveth_update_rx_no_buffer(adapter);
ibmveth_free_all_queues(adapter);
appears to order the freeing after the in-flight poll, since NAPI polls run
with BH disabled and a BH-disabled region blocks a grace period. On that
reading the residual effect is a PHYP re-arm on a queue about to be
released, whose interrupt is then discarded by napi_schedule_prep() or
free_irq(), rather than a use-after-free.
Does the comment's claim about the late-IRQ WARN hold as written, or does
the check only narrow the window?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 10/15] ibmveth: Enable multi-queue RX receive path
2026-08-14 7:36 ` [PATCH net-next v5 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Enable multi-queue RX receive path
This turns on the multi-queue RX path when firmware reports
IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT, registering subordinate queues via
H_REG_LOGICAL_LAN_QUEUE and wiring per-queue IRQs and NAPI instances. It
also reworks open()/close() to walk every live RX queue, adds an
mq_fallback latch, and refreshes the VIO CMO entitlement at probe.
A few questions below.
Regarding this part of the changelog:
Up to IBMVETH_MAX_RX_QUEUES (16) remains available via ethtool -L.
[Severity: Low]
Is that true at this commit? ibmveth_get_channels() still does:
channels->max_rx = netdev->real_num_rx_queues;
channels->rx_count = netdev->real_num_rx_queues;
and ibmveth_set_channels() never reads channels->rx_count, so an
"ethtool -L <dev> rx N" returns success without changing anything, and 16
RX queues cannot be requested until the later get_channels/set_channels
patches land. Could this sentence move to the patch that actually wires
rx_count up?
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 86299c62d4ecf..36f0926711e60 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -503,7 +519,7 @@ static int
> ibmveth_setup_rx_interrupts(struct ibmveth_adapter *adapter)
> {
> struct net_device *netdev = adapter->netdev;
> - int i, rc, num = adapter->num_rx_queues;
> + int i, rc, num = ibmveth_get_num_rx_queues(adapter);
>
> for (i = 0; i < num; i++) {
> if (!adapter->queue_irq[i]) {
[Severity: Low]
Now that this function can enable PHYP delivery on several queues, can the
err_disable_napi rollback leave a queue armed with no handler?
The rollback masks the already-enabled queues and calls synchronize_irq(),
but synchronize_irq() does not wait for NAPI. A poll already running for
one of those queues reaches:
if (ibmveth_poll_stopping(netdev, napi))
goto out;
rc = ibmveth_enable_irq(adapter, queue_index);
ibmveth_poll_stopping() is "!netif_running(netdev) || napi_disable_pending()",
and __dev_open() sets __LINK_STATE_START before calling ndo_open, so it is
false here and the poll re-arms PHYP. err_disable_napi then does:
for (i = 0; i < num; i++)
napi_disable(&adapter->napi[i]);
for (i = 0; i < num; i++) {
if (adapter->queue_irq[i])
free_irq(adapter->queue_irq[i], &adapter->napi[i]);
}
with no second mask pass, whereas ibmveth_cleanup_rx_interrupts()
deliberately masks again after napi_disable(). Should the rollback follow
the cleanup pattern and remask after napi_disable()?
> @@ -649,6 +665,24 @@ static bool ibmveth_schedule_rx_queue(struct ibmveth_adapter *adapter,
> return false;
> }
>
> +/**
> + * ibmveth_kick_rx_queue_if_pending - Schedule NAPI if PHYP posted while masked
> + * @adapter: ibmveth adapter
> + * @queue_index: RX queue index just unmasked
> + *
> + * After enable_irq() / MQ open unmask, descriptors may already be pending
> + * (buffers were posted while PHYP was masked). Use schedule_rx_queue() so
> + * PHYP is masked before NAPI runs — napi_schedule() then disable_irq() from
> + * process context can race a completing poll and leave the queue masked
> + * with NAPI idle.
> + */
> +static void
> +ibmveth_kick_rx_queue_if_pending(struct ibmveth_adapter *adapter,
> + int queue_index)
> +{
> + if (ibmveth_rxq_pending_buffer(adapter, queue_index))
> + ibmveth_schedule_rx_queue(adapter, queue_index);
> +}
[Severity: Low]
Does this commit build on its own? ibmveth_kick_rx_queue_if_pending() is a
non-inline static function with no caller anywhere in the tree at this
commit (open() uses ibmveth_restart_rx_queue() instead), so gcc/clang emit
"defined but not used", which is fatal with the default CONFIG_WERROR=y.
The first callers appear only in the later incremental resize patch. Could
the helper be introduced there instead, to keep this commit bisectable?
Also, the blank line before the following comment block was dropped:
+ ibmveth_schedule_rx_queue(adapter, queue_index);
+}
/* setup the initial settings for a buffer pool */
> @@ -963,11 +997,21 @@ static int ibmveth_replenish_buffer_pool(struct ibmveth_adapter *adapter,
> * because there was not a buffer in the buffer list capable of holding
> * the frame.
> */
> -static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter)
> +static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter,
> + int queue_index)
> {
> - __be64 *p = adapter->buffer_list_addr[0] + 4096 - 8;
> + __be64 *p;
> + u64 drops;
> +
> + if (queue_index < 0 ||
> + queue_index >= ibmveth_get_num_rx_queues(adapter) ||
> + !adapter->buffer_list_addr[queue_index])
> + return;
> +
> + p = adapter->buffer_list_addr[queue_index] + 4096 - 8;
> + drops = be64_to_cpup(p);
>
> - adapter->rx_no_buffer = be64_to_cpup(p);
> + adapter->rx_no_buffer = drops;
> }
[Severity: Low]
The function became queue-aware, but the store target is still the single
adapter-wide field exported by ethtool -S. With several RX queues, does
rx_no_buffer end up being "whichever queue ran last", and can it decrease?
Each queue reads its own absolute PHYP page counter and overwrites the same
scalar, from per-queue replenish paths that hold different replenish_locks,
plus the lock-free loop added in ibmveth_close():
for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++)
ibmveth_update_rx_no_buffer(adapter, i);
The next patch in the series ("ibmveth: Add per-queue RX and TX statistics
collection") moves this to adapter->rx_qstats[queue_index] with a summing
helper, so only this intermediate commit reports the wrong value. Could
the per-queue slot land together with the queue_index argument?
> @@ -1033,6 +1077,7 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
> dev_err_ratelimited(&adapter->netdev->dev,
> "MQ buffer add H_FUNCTION (q=%d, batch=%u), reset\n",
> queue_index, fail.batch);
> + adapter->mq_fallback = true;
> schedule_work(&adapter->work);
> }
[Severity: High]
This isn't a bug introduced by this patch, but it adds another producer of
adapter->work while ibmveth_remove() still cancels the work before the
interface is unregistered:
cancel_work_sync(&adapter->work);
...
unregister_netdev(netdev);
free_netdev(netdev);
Can this sequence use freed memory?
CPU0 (rmmod / DLPAR remove)
ibmveth_remove()
cancel_work_sync(&adapter->work); /* returns */
CPU1 (NAPI poll)
ibmveth_replenish_task()
adapter->mq_fallback = true;
schedule_work(&adapter->work); /* re-queued */
CPU0
unregister_netdev(netdev); /* holds RTNL */
free_netdev(netdev); /* frees adapter + work_struct */
The queued ibmveth_reset() blocks in rtnl_lock() while unregister_netdev()
runs, then derives adapter via container_of() on the freed work_struct and
touches adapter->netdev. Would unregistering/quiescing first and calling
cancel_work_sync() after be safer?
> @@ -1052,6 +1097,30 @@ static void ibmveth_replenish_task(struct ibmveth_adapter *adapter,
> fail.filled, fail.lpar_rc, fail.batch);
> }
>
> +/**
> + * ibmveth_restart_rx_queue - Post buffers and ensure Q can take RX
> + * @adapter: ibmveth adapter
> + * @qindex: RX queue index
> + *
> + * SQ open leaves PHYP masked until the first poll. If schedule_prep fails,
> + * NAPI never runs and the queue stays masked (TX OK, RX/ARP dead) until
> + * reload. Replenish first so an enable_irq fallback can actually deliver.
> + * Also used after every open (SQ and MQ) and after scale-down so a
> + * queue is not left idle+masked.
> + */
> +static void ibmveth_restart_rx_queue(struct ibmveth_adapter *adapter,
> + int qindex)
> +{
> + int rc;
> +
> + ibmveth_replenish_task(adapter, qindex);
> + if (ibmveth_schedule_rx_queue(adapter, qindex))
> + return;
> +
> + rc = ibmveth_enable_irq(adapter, qindex);
> + WARN_ON(rc);
> +}
[Severity: Medium]
The enable_irq() fallback here is reached exactly when
napi_schedule_prep() failed, i.e. when a poll for that queue is already
scheduled or running. In MQ mode ibmveth_setup_rx_interrupts() has already
unmasked PHYP before open() calls this helper, so a frame can arrive first:
ibmveth_interrupt(q)
ibmveth_schedule_rx_queue(q) /* masks PHYP, schedules NAPI */
ibmveth_open()
ibmveth_restart_rx_queue(adapter, q)
napi_schedule_prep() == false
ibmveth_enable_irq(adapter, q); /* re-arms PHYP mid-poll */
Since ibmveth_schedule_rx_queue() masks PHYP only inside the successful
prep branch:
if (napi_schedule_prep(napi)) {
rc = ibmveth_disable_irq(adapter, qindex);
WARN_ON(rc);
__napi_schedule(napi);
return true;
}
return false;
can the handler still quiesce the source afterwards? Each subsequent
interrupt returns IRQ_HANDLED without masking and sets NAPI_STATE_MISSED,
so napi_complete_done() keeps returning false and the queue stays unmasked
while traffic flows, giving an interrupt per frame on that queue.
Related: this WARN_ON(rc) treats a failed unmask hcall as advisory, while
ibmveth_poll() treats the same failure as a hardware-state problem:
rc = ibmveth_enable_irq(adapter, queue_index);
if (rc) {
netdev_err(...);
schedule_work(&adapter->work);
goto out;
}
Should the two paths handle the failure the same way?
Also, the comment says the helper is used "after scale-down", but no
scale-down caller exists at this commit; that path arrives with the later
resize patch.
[ ... ]
> @@ -1556,10 +1757,12 @@ static void ibmveth_free_all_queues(struct ibmveth_adapter *adapter)
> * @adapter: ibmveth adapter structure
> * @mac_address: MAC address for device registration
> *
> - * Registers queue 0 via ibmveth_register_logical_lan(). Subordinate queue
> - * registration is added when multi-queue RX is enabled.
> + * Registers queue 0 via ibmveth_register_logical_lan(), then subordinate
> + * queues 1..N when multi-queue mode is enabled.
> *
> - * Return: 0 on success, -ENONET if queue 0 registration fails
> + * Return: 0 on success, -ENONET if queue 0 registration fails,
> + * -EOPNOTSUPP if firmware rejects a subordinate queue (H_FUNCTION;
> + * caller latches mq_fallback), -EIO on other subordinate failures
[Severity: Low]
The kernel-doc says the caller latches mq_fallback, but the function body
does it itself and ibmveth_open() only propagates rc. Could the wording be
changed to say this function latches adapter->mq_fallback, so a future
change does not add a second latch or remove this one?
> @@ -1592,9 +1796,67 @@ ibmveth_register_rx_queues(struct ibmveth_adapter *adapter, u64 mac_address)
> return -ENONET;
> }
>
> + num = ibmveth_get_num_rx_queues(adapter);
> + if (num == 1 || !adapter->multi_queue) {
> + netdev_dbg(netdev,
> + "registered 1 RX queue with hypervisor (single-queue mode)\n");
> + return 0;
> + }
> +
> + netdev_dbg(netdev, "Registering %d subordinate queues (1-%d)\n",
> + num - 1, num - 1);
> +
> + for (i = 1; i < num; i++) {
> + rc = ibmveth_register_single_rx_queue(adapter, i);
> + if (rc) {
> + /* Firmware MQ gone: fall back to SQ on next open. */
> + if (rc == -EOPNOTSUPP)
> + adapter->mq_fallback = true;
> + goto err_unregister;
> + }
> + }
[Severity: High]
Who performs the "next open" that consumes the latched mq_fallback?
The only automatic recovery path is the reset work:
ibmveth_reset()
rtnl_lock();
dev_close(adapter->netdev);
dev_open(adapter->netdev, NULL); /* return value discarded */
rtnl_unlock();
If firmware stops honouring H_REG_LOGICAL_LAN_QUEUE (the LPM case the code
comment describes), that dev_open() fails with -EOPNOTSUPP, mq_fallback is
set, nothing looks at the error, and no further open is attempted. Does
the interface then stay administratively up but dead until an administrator
runs ifdown/ifup, even though the driver already knows the single-queue
retry would succeed?
Would retrying single-queue inside open(), or re-arming the reset work when
dev_open() fails, match the intent stated in the changelog?
[ ... ]
> @@ -2732,23 +3019,35 @@ static unsigned long ibmveth_get_desired_dma(struct vio_dev *vdev)
>
> adapter = netdev_priv(netdev);
>
> - ret = IBMVETH_BUFF_LIST_SIZE + IBMVETH_FILT_LIST_SIZE;
> + /* One buffer list page per RX queue; filter list is shared. */
> + ret = IBMVETH_BUFF_LIST_SIZE * ibmveth_get_num_rx_queues(adapter) +
> + IBMVETH_FILT_LIST_SIZE;
> ret += IOMMU_PAGE_ALIGN(netdev->mtu, tbl);
> /* add size of mapped tx buffers */
> ret += IOMMU_PAGE_ALIGN(IBMVETH_MAX_TX_BUF_SIZE, tbl);
[Severity: Medium]
This is a pre-existing shortfall rather than something this patch created,
but since the function is being rewritten and its result now becomes the
probe-time CMO request, should the TX term scale with the TX queue count?
ibmveth_alloc_tx_resources() maps one long-term buffer per TX queue:
for (i = 0; i < netdev->real_num_tx_queues; i++) {
if (ibmveth_allocate_tx_ltb(adapter, i))
goto err_free_ltbs;
}
with real_num_tx_queues defaulting to min(nr_cpus, 8), while the accounting
above adds exactly one IBMVETH_MAX_TX_BUF_SIZE. Can the requested
entitlement be short by up to 7 x 64KB, so that dma_map_single() in
ibmveth_allocate_tx_ltb() fails at open on an entitlement-constrained
partition? vio_cmo_set_dev_desired() returns void, so the shortfall is
silent.
[ ... ]
> @@ -2895,16 +3206,29 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
> netdev->features |= NETIF_F_FRAGLIST;
> }
>
> - /* Initialize queue count - always 1 for now */
> - adapter->multi_queue = 0;
> - adapter->num_rx_queues = IBMVETH_DEFAULT_RX_QUEUES;
> + if (ret == H_SUCCESS &&
> + (ret_attr & IBMVETH_ILLAN_RX_MULTI_QUEUE_SUPPORT)) {
> + adapter->multi_queue = 1;
> + ibmveth_publish_num_rx_queues(adapter,
> + min(num_online_cpus(),
> + IBMVETH_DEFAULT_QUEUES));
> + netdev_dbg(netdev, "RX multi queue mode enabled: %d queues\n",
> + ibmveth_get_num_rx_queues(adapter));
> + } else {
> + adapter->multi_queue = 0;
> + ibmveth_publish_num_rx_queues(adapter,
> + IBMVETH_DEFAULT_RX_QUEUES);
> + }
[Severity: Medium]
Activating several RX NAPI instances makes a number of adapter-wide
counters multi-writer. Can these lose increments?
ibmveth_replenish_task() bumps a shared counter outside any lock:
adapter->replenish_task_cycles++;
spin_lock_irqsave(&rxq->replenish_lock, flags);
and ibmveth_replenish_buffer_pool() updates shared counters under only the
queue-local lock:
adapter->replenish_add_buff_success += filled;
Per-queue replenish_locks give no mutual exclusion between different
queues, so plain read-modify-write on replenish_task_cycles,
replenish_add_buff_success/failure, replenish_no_mem and
hcall_stats.add_bufs_queue races. At this commit rx_invalid_buffer,
rx_large_packets and netdev->stats.rx_packets/rx_bytes are in the same
situation; the following qstats patch moves those to per-queue storage, but
the replenish counters remain shared through the end of the series. Should
they get per-queue slots as well?
> @@ -2922,25 +3246,62 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>
> for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> struct kobject *kobj = &adapter->rx_buff_pool[0][i].kobj;
> - int error;
>
> ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], i,
> pool_count[i], pool_size[i],
> pool_active[i]);
> - error = kobject_init_and_add(kobj, &ktype_veth_pool,
> - &dev->dev.kobj, "pool%d", i);
> - if (!error)
> - kobject_uevent(kobj, KOBJ_ADD);
> + rc = kobject_init_and_add(kobj, &ktype_veth_pool,
> + &dev->dev.kobj, "pool%d", i);
> + if (rc) {
> + dev_err(&dev->dev,
> + "failed to create pool%d kobject: %d\n", i, rc);
> + /* init_and_add takes a ref even on failure */
> + kobject_put(kobj);
> + ibmveth_put_pool_kobjs(adapter, pools_ready);
> + dev_set_drvdata(&dev->dev, NULL);
> + free_netdev(netdev);
> + return rc;
> + }
> +
> + pools_ready++;
> + kobject_uevent(kobj, KOBJ_ADD);
> }
[Severity: Low]
This is a pre-existing problem being repaired rather than a new one, but
propagating kobject_init_and_add() failure and putting already-created pool
kobjects before free_netdev() is a standalone fix for probe error handling
that predates the MQ work. Could it be split into its own patch with a
Fixes: tag so it can be backported independently of this feature?
[Severity: Medium]
This isn't a bug introduced by this patch either (ibmveth_remove() already
does kobject_put() followed by free_netdev()), but the pattern is now
replicated in four new probe error paths. Can these free memory that a
deferred kobject release still uses?
ktype_veth_pool has no release callback:
static const struct kobj_type ktype_veth_pool = {
.release = NULL,
.sysfs_ops = &veth_pool_ops,
.default_groups = veth_pool_groups,
};
and the kobjects live inside netdev_priv(). With
CONFIG_DEBUG_KOBJECT_RELEASE=y, kobject_release() in lib/kobject.c does not
clean up synchronously:
unsigned long delay = HZ + HZ * get_random_u32_below(4);
...
INIT_DELAYED_WORK(&kobj->release, kobject_delayed_cleanup);
schedule_delayed_work(&kobj->release, delay);
The delayed_work lives inside the kobject, so free_netdev() right after
kobject_put() leaves a queued work item in freed memory and
kobject_delayed_cleanup() dereferences a freed kobj 1-5 seconds later.
Would a no-op .release for ktype_veth_pool, or deferring free_netdev()
until the kobjects are really gone, avoid that?
> + /*
> + * VIO CMO entitlement was set before probe (netdev NULL → default).
> + * Recompute now that num_rx_queues and pool 0 metadata are known.
> + */
> + if (firmware_has_feature(FW_FEATURE_CMO))
> + vio_cmo_set_dev_desired(dev, ibmveth_get_desired_dma(dev));
> +
> rc = netif_set_real_num_tx_queues(netdev, min(num_online_cpus(),
> IBMVETH_DEFAULT_QUEUES));
> if (rc) {
> netdev_dbg(netdev, "failed to set number of tx queues rc=%d\n",
> rc);
> + ibmveth_put_pool_kobjs(adapter, pools_ready);
> + dev_set_drvdata(&dev->dev, NULL);
> free_netdev(netdev);
> return rc;
> }
[ ... ]
Cross-instance finding from sashiko-gemini (91dce8e1ae8c5dfd96a2db94571e030c1951674452013e31a94e2df21bfc801b):
[Severity: Medium]
Missing `smp_rmb()` in `ibmveth_get_num_rx_queues` allows out-of-order reads of uninitialized queue data.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 11/15] ibmveth: Add per-queue RX and TX statistics collection
2026-08-14 7:36 ` [PATCH net-next v5 11/15] ibmveth: Add per-queue RX and TX statistics collection Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Add per-queue RX and TX statistics collection
This adds per-queue RX and TX counter arrays that are allocated at probe
and freed at remove, updates them from the poll/IRQ/xmit paths, and
exposes the values through ethtool -S plus a new ndo_get_stats64
implementation. It also adds the hcall_* counters to the ethtool string
table and reworks the probe error paths and ibmveth_remove() teardown
order.
A few questions below.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 36f0926711e60..3202b657c9b8f 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -98,7 +98,17 @@ static struct ibmveth_stat ibmveth_stats[] = {
> { "fw_enabled_ipv6_csum", IBMVETH_STAT_OFF(fw_ipv6_csum_support) },
> { "tx_large_packets", IBMVETH_STAT_OFF(tx_large_packets) },
> { "rx_large_packets", IBMVETH_STAT_OFF(rx_large_packets) },
> - { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) }
> + { "fw_enabled_large_send", IBMVETH_STAT_OFF(fw_large_send_support) },
> + { "hcall_reg_lan_queue", IBMVETH_STAT_OFF(hcall_stats.reg_lan_queue) },
> + { "hcall_reg_lan", IBMVETH_STAT_OFF(hcall_stats.reg_lan) },
> + { "hcall_add_bufs_queue",
> + IBMVETH_STAT_OFF(hcall_stats.add_bufs_queue) },
> + { "hcall_add_bufs", IBMVETH_STAT_OFF(hcall_stats.add_bufs) },
> + { "hcall_add_buf", IBMVETH_STAT_OFF(hcall_stats.add_buf) },
> + { "hcall_free_lan_queue",
> + IBMVETH_STAT_OFF(hcall_stats.free_lan_queue) },
> + { "hcall_free_lan", IBMVETH_STAT_OFF(hcall_stats.free_lan) },
> + { "hcall_send_lan", IBMVETH_STAT_OFF(hcall_stats.send_lan) },
> };
[Severity: Low]
Can these hcall_* values lose increments once they become user visible?
The fields in adapter->hcall_stats are plain u64 bumped with an
unserialized read-modify-write. ibmveth_replenish_task() takes only the
queue-local lock:
spin_lock_irqsave(&rxq->replenish_lock, flags);
so two NAPI instances replenishing different RX queues both reach
adapter->hcall_stats.add_bufs_queue++;
with no shared serialization, and separate TX queues both reach
adapter->hcall_stats.send_lan++;
in ibmveth_send(). Two CPUs can read N and both store N+1.
Would per-queue counters summed on read (the same approach used for the
new rx_qstats/tx_qstats), or atomic64_t, be preferable here?
[ ... ]
> @@ -1011,7 +1075,23 @@ static void ibmveth_update_rx_no_buffer(struct ibmveth_adapter *adapter,
> p = adapter->buffer_list_addr[queue_index] + 4096 - 8;
> drops = be64_to_cpup(p);
>
> - adapter->rx_no_buffer = drops;
> + /*
> + * PHYP's buffer-list page counter is absolute for that page. A new
> + * page (reopen / queue reuse after -L) starts near zero; fold the
> + * previous absolute into retired so adapter-level sums stay
> + * monotonic.
> + */
> + if (adapter->rx_qstats) {
> + u64 *slot = &adapter->rx_qstats[queue_index].no_buffer_drops;
> +
> + if (drops < *slot)
> + adapter->rx_no_buffer_retired += *slot;
> + *slot = drops;
> + } else {
> + if (drops < adapter->rx_no_buffer)
> + adapter->rx_no_buffer_retired += adapter->rx_no_buffer;
> + adapter->rx_no_buffer = drops;
> + }
> }
[Severity: Low]
Does the no_buffer_drops slot lose the final PHYP value when a queue is
retired by a channel shrink?
This makes rx_qstats[i].no_buffer_drops a mirror of the page-absolute
counter, and folding into rx_no_buffer_retired only happens when a later
read observes a decrease. ibmveth_update_rx_no_buffer() has only two call
sites, the replenish path and ibmveth_close(), and it early-returns for
queue_index >= ibmveth_get_num_rx_queues(adapter)
Once the incremental resize patch later in this series lands, scale-down
does:
ibmveth_publish_num_rx_queues(adapter, new_count);
...
for (i = new_count; i < old_count; i++)
ibmveth_destroy_subordinate_rx_queue(adapter, i);
which reaches ibmveth_free_single_rx_queue() and dma_unmap_single() plus
free_page() on buffer_list_addr[i]. The live count is already lowered, so
a late harvest is rejected by the guard above, and everything PHYP
recorded in that page since the last replenish read is dropped from both
the per-queue slot and the adapter sum.
Would harvesting the absolute before lowering the published queue count
and freeing the page fix this?
> @@ -2239,22 +2319,158 @@ static int ibmveth_set_features(struct net_device *dev,
[ ... ]
> static void ibmveth_get_strings(struct net_device *dev, u32 stringset, u8 *data)
> {
> + struct ibmveth_adapter *adapter = netdev_priv(dev);
> + u8 *p = data;
> int i;
>
> if (stringset != ETH_SS_STATS)
> return;
>
> - for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++, data += ETH_GSTRING_LEN)
> - memcpy(data, ibmveth_stats[i].name, ETH_GSTRING_LEN);
> + for (i = 0; i < ARRAY_SIZE(ibmveth_stats); i++) {
> + memcpy(p, ibmveth_stats[i].name, ETH_GSTRING_LEN);
> + p += ETH_GSTRING_LEN;
> + }
> +
> + for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
> + ethtool_sprintf(&p, "rx%d_packets", i);
> + ethtool_sprintf(&p, "rx%d_bytes", i);
> + ethtool_sprintf(&p, "rx%d_interrupts", i);
> + ethtool_sprintf(&p, "rx%d_polls", i);
> + ethtool_sprintf(&p, "rx%d_large_packets", i);
> + ethtool_sprintf(&p, "rx%d_invalid_buffers", i);
> + ethtool_sprintf(&p, "rx%d_no_buffer_drops", i);
> + }
> +
> + for (i = 0; i < dev->real_num_tx_queues; i++) {
> + ethtool_sprintf(&p, "tx%d_packets", i);
> + ethtool_sprintf(&p, "tx%d_bytes", i);
> + ethtool_sprintf(&p, "tx%d_large_packets", i);
> + ethtool_sprintf(&p, "tx%d_dropped_packets", i);
> + ethtool_sprintf(&p, "tx%d_send_failures", i);
> + ethtool_sprintf(&p, "tx%d_checksum_offload", i);
> + }
[Severity: Medium]
Should the per-queue packet, byte and drop counters go through the
standard per-queue statistics interface instead of private ethtool
strings?
rx%d_packets, rx%d_bytes, tx%d_packets, tx%d_bytes and
tx%d_dropped_packets map directly onto existing fields:
include/net/netdev_queues.h
struct netdev_stat_ops {
void (*get_queue_stats_rx)(struct net_device *dev, int idx,
struct netdev_queue_stats_rx *stats);
...
The driver adds only .ndo_get_stats64 (device-wide) and never sets
netdev->stat_ops, so the newly collected per-queue values are reachable
only through the private ethtool blob, which cannot be removed once
shipped. The genuinely driver-specific counters (interrupts, polls,
invalid_buffers, no_buffer_drops, send_failures, checksum_offload) look
fine in ethtool -S.
Could the packets/bytes/dropped set be exposed via netdev_stat_ops
qstats instead?
> +
> + for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++) {
> + ethtool_sprintf(&p, "pool%d_size", i);
> + ethtool_sprintf(&p, "pool%d_active", i);
> + ethtool_sprintf(&p, "pool%d_available", i);
> + }
> }
>
> static int ibmveth_get_sset_count(struct net_device *dev, int sset)
> {
> + struct ibmveth_adapter *adapter = netdev_priv(dev);
> +
> switch (sset) {
> case ETH_SS_STATS:
> - return ARRAY_SIZE(ibmveth_stats);
> + return ARRAY_SIZE(ibmveth_stats) +
> + ibmveth_get_num_rx_queues(adapter) *
> + IBMVETH_NUM_RX_QSTATS +
> + dev->real_num_tx_queues * IBMVETH_NUM_TX_QSTATS +
> + IBMVETH_NUM_BUFF_POOLS * 3;
> default:
> return -EOPNOTSUPP;
> }
> @@ -2263,11 +2479,44 @@ static int ibmveth_get_sset_count(struct net_device *dev, int sset)
> static void ibmveth_get_ethtool_stats(struct net_device *dev,
> struct ethtool_stats *stats, u64 *data)
> {
[ ... ]
> + for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
> + data[i++] = adapter->rx_buff_pool[0][j].size;
> + data[i++] = adapter->rx_buff_pool[0][j].active;
> + data[i++] = atomic_read(&adapter->rx_buff_pool[0][j].available);
> + }
> }
[Severity: Medium]
This is a pre-existing issue rather than something this patch creates for
the lifetime part below, but the commit message describes only per-queue
RX/TX statistics and the hcall_* strings, while the patch also carries two
other changes.
First, three new ethtool -S entries per buffer pool are added:
ethtool_sprintf(&p, "pool%d_size", i);
ethtool_sprintf(&p, "pool%d_active", i);
ethtool_sprintf(&p, "pool%d_available", i);
and counted as IBMVETH_NUM_BUFF_POOLS * 3 in ibmveth_get_sset_count().
The values are read only from adapter->rx_buff_pool[0][j], but
rx_buff_pool is declared as
struct ibmveth_buff_pool
rx_buff_pool[IBMVETH_MAX_RX_QUEUES][IBMVETH_NUM_BUFF_POOLS];
so in multi-queue mode queues 1..N-1 are not represented and the string
names carry no queue index. pool size and active are also already
available through the per-pool sysfs attributes (veth_size_attr,
veth_num_attr, veth_active_attr). Should these strings be dropped, or at
least given a queue index and mentioned in the commit message?
Second, ibmveth_remove() reorders unregister_netdev() ahead of
cancel_work_sync(), and the new ibmveth_probe_cleanup() clears vio
drvdata before free_netdev(). The in-code comments describe both as
fixing existing problems (a worker running after free_netdev(), and CMO
get_desired_dma() seeing a freed netdev on rebind), neither of which the
commit message body mentions and neither of which has a Fixes: tag.
Could these be split into their own patches?
[ ... ]
> @@ -3107,6 +3410,7 @@ static const struct net_device_ops ibmveth_netdev_ops = {
> .ndo_validate_addr = eth_validate_addr,
> .ndo_set_mac_address = ibmveth_set_mac_addr,
> .ndo_features_check = ibmveth_features_check,
> + .ndo_get_stats64 = ibmveth_get_stats64,
> #ifdef CONFIG_NET_POLL_CONTROLLER
> .ndo_poll_controller = ibmveth_poll_controller,
> #endif
> @@ -3121,6 +3425,23 @@ static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
> kobject_put(&adapter->rx_buff_pool[0][i].kobj);
> }
>
> +static void ibmveth_probe_cleanup(struct ibmveth_adapter *adapter,
> + int pools_ready)
> +{
> + struct net_device *netdev = adapter->netdev;
> +
> + cancel_work_sync(&adapter->work);
> + ibmveth_put_pool_kobjs(adapter, pools_ready);
> +
> + ibmveth_free_tx_qstats(adapter);
> + ibmveth_free_rx_qstats(adapter);
> + /* Probe failure never reaches ibmveth_remove(); clear before free so
> + * CMO get_desired_dma() cannot see a freed netdev on rebind.
> + */
> + dev_set_drvdata(&adapter->vdev->dev, NULL);
> + free_netdev(netdev);
> +}
> +
[ ... ]
> @@ -3332,12 +3651,20 @@ static void ibmveth_remove(struct vio_dev *dev)
> struct ibmveth_adapter *adapter = netdev_priv(netdev);
> int i;
>
> - cancel_work_sync(&adapter->work);
> -
> for (i = 0; i < IBMVETH_NUM_BUFF_POOLS; i++)
> kobject_put(&adapter->rx_buff_pool[0][i].kobj);
>
> + /*
> + * Unregister first so NAPI/xmit cannot re-arm reset work after we
> + * cancel it. cancel_work_sync() before unregister left a window
> + * where poll could schedule_work() and the worker ran after
> + * free_netdev().
> + */
> unregister_netdev(netdev);
> + cancel_work_sync(&adapter->work);
> +
> + ibmveth_free_tx_qstats(adapter);
> + ibmveth_free_rx_qstats(adapter);
>
> free_netdev(netdev);
> dev_set_drvdata(&dev->dev, NULL);
[Severity: High]
With cancel_work_sync() now running after unregister_netdev(), can the
reset worker re-open an already unregistered netdev?
adapter->work is queued from several error paths, for example
ibmveth_replenish_task() on an invalid free_map entry or on MQ H_FUNCTION
after LPM, and from ibmveth_poll_skip_bad_correlator():
schedule_work(&adapter->work);
The handler has no registration or running check:
drivers/net/ethernet/ibm/ibmveth.c:ibmveth_reset() {
rtnl_lock();
dev_close(adapter->netdev);
dev_open(adapter->netdev, NULL);
rtnl_unlock();
}
A worker already queued (or already blocked in rtnl_lock() while
unregister_netdev() holds rtnl) proceeds once unregistration finishes.
dev_close() is then a no-op, and the re-open is not rejected:
net/core/dev.c:netif_open() {
if (dev->flags & IFF_UP)
return 0;
ret = __dev_open(dev, extack);
}
net/core/dev.c:__dev_open() {
ASSERT_RTNL();
dev_addr_check(dev);
if (!netif_device_present(dev)) {
...
if (!ret && ops->ndo_open)
ret = ops->ndo_open(dev);
}
IFF_UP was cleared by unregister and __LINK_STATE_PRESENT is never
cleared (the driver does not call netif_device_detach()), so
ibmveth_open() runs again on the unregistered device: it re-requests the
RX IRQs with dev_id = &adapter->napi[i], re-registers the logical LAN
with PHYP, and re-arms DMA into freshly allocated buffer lists.
cancel_work_sync() then returns and remove() continues into
ibmveth_free_tx_qstats() / ibmveth_free_rx_qstats() and free_netdev(),
so the installed IRQ handler and the hypervisor are left pointing at
freed memory. The next RX interrupt would execute
adapter->rx_qstats[qindex].interrupts++;
in ibmveth_interrupt() against a freed adapter and a freed qstat array.
Would a netif_running() or reg_state check inside ibmveth_reset(), or a
"removing" flag set before unregister_netdev(), close this?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels
2026-08-14 7:36 ` [PATCH net-next v5 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Report MQ-aware RX counts in ethtool get_channels
The patch changes ibmveth_get_channels() to report max_rx as
IBMVETH_MAX_RX_QUEUES when the adapter has multi-queue firmware support
(else 1), and to report rx_count from the driver's published
num_rx_queues instead of netdev->real_num_rx_queues.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 3202b657c9b8f..5d4ca4b3d3d07 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
> @@ -2522,11 +2522,16 @@ static void ibmveth_get_ethtool_stats(struct net_device *dev,
> static void ibmveth_get_channels(struct net_device *netdev,
> struct ethtool_channels *channels)
> {
> + struct ibmveth_adapter *adapter = netdev_priv(netdev);
> +
> channels->max_tx = ibmveth_real_max_tx_queues();
> channels->tx_count = netdev->real_num_tx_queues;
>
> - channels->max_rx = netdev->real_num_rx_queues;
> - channels->rx_count = netdev->real_num_rx_queues;
> + if (adapter->multi_queue)
> + channels->max_rx = IBMVETH_MAX_RX_QUEUES;
> + else
> + channels->max_rx = 1;
> + channels->rx_count = ibmveth_get_num_rx_queues(adapter);
> }
[Severity: Medium]
At this point in the series, does raising max_rx here make RX channel
requests silently succeed without changing anything?
ibmveth_set_channels() only derives its work from tx_count:
unsigned int old = netdev->real_num_tx_queues,
goal = channels->tx_count;
...
if (!(netdev->flags & IFF_UP))
return netif_set_real_num_tx_queues(netdev, goal);
channels->rx_count is never read there. The core validation in
ethtool_set_channels() only bounds the request by the driver-reported
maximum:
if (channels.rx_count > curr.max_rx ||
channels.tx_count > curr.max_tx ||
so with max_rx now 16, "ethtool -L ethX rx N" for any N in 1..16 that
differs from the current RX count passes validation, enters
ibmveth_set_channels() with tx_count unchanged, runs no LTB loop
iterations, and returns 0. The netlink path in ethnl_set_channels()
applies the same max_rx-only ceiling and maps the 0 return to success.
Before this change max_rx equalled real_num_rx_queues, so an increase was
rejected with -EINVAL by the core; afterwards it is accepted and ignored,
and "ethtool -l" still shows the old rx_count.
For the record, this appears to be resolved later in the same series: the
final patch "ibmveth: Wire ethtool set_channels to MQ RX queue resize"
makes ibmveth_set_channels() read channels->rx_count and apply it via
ibmveth_resize_rx_channels() with range validation of
1..IBMVETH_MAX_RX_QUEUES, so the get/set pair agrees at the end of the
series. Would it be worth ordering that patch before this one, or
mentioning the intermediate state in the changelog?
[Severity: Low]
A second, smaller point about the same lines: max_rx is derived only from
adapter->multi_queue and does not consult adapter->mq_fallback. After
firmware refuses multi-queue, ibmveth_apply_mq_fallback() defers the
downgrade to the next open:
if (!adapter->mq_fallback)
return;
adapter->mq_fallback = false;
...
adapter->multi_queue = 0;
ibmveth_publish_num_rx_queues(adapter, 1);
so between the firmware rejection and the next open, "ethtool -l" keeps
reporting max_rx = 16 and rx_count = 8 while the adapter is going to run
single-queue. Should get_channels() also take mq_fallback into account?
Also, at this patch the reported maximum of IBMVETH_MAX_RX_QUEUES (16) is
above anything the driver publishes, since ibmveth_probe() caps the RX
queue count:
adapter->multi_queue = 1;
ibmveth_publish_num_rx_queues(adapter,
min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES));
The end-of-series patch "ibmveth: Wire ethtool set_channels to MQ RX
queue resize" does make 1..IBMVETH_MAX_RX_QUEUES genuinely settable via
ibmveth_resize_rx_channels(), so this is only about the intermediate
state and the stale reporting while mq_fallback is latched.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 13/15] ibmveth: Expose per-queue buffer pool details via debugfs
2026-08-14 7:36 ` [PATCH net-next v5 13/15] ibmveth: Expose per-queue buffer pool details via debugfs Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Expose per-queue buffer pool details via debugfs
This adds a read-only buffer_pools file under
/sys/kernel/debug/ibmveth/<dev_name>/ that dumps size, buff_size, active
and available for every RX queue and buffer pool. The per-adapter
directory lives under a driver-owned root created lazily from
ibmveth_probe(), and the historical queue-0 poolN sysfs attributes are
left untouched.
A few questions below about the dumped values and about the lazy creation
of the driver root.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 5d4ca4b3d3d07..1b58a3c6ce77a 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -3421,6 +3422,68 @@ static const struct net_device_ops ibmveth_netdev_ops = {
> #endif
> };
>
> +static int ibmveth_buffer_pools_show(struct seq_file *m, void *v)
> +{
> + struct ibmveth_adapter *adapter = m->private;
> + int i, j;
> +
> + /*
> + * Writers (veth_pool_store, open template copy, reset close/open)
> + * update these fields under RTNL. Take the same lock so the dump
> + * is not a torn scalar snapshot. Not required for
> + * memory safety — embedded arrays only.
> + */
> + rtnl_lock();
[Severity: Low]
Is this comment accurate for all of the dumped fields? It says the
writers of "these fields" run under RTNL, but pool->available is also
printed here and is updated from the NAPI/softirq RX path:
ibmveth_replenish_buffer_pool()
atomic_add(buffers_added, &(pool->available));
ibmveth_remove_buffer_from_pool()
atomic_dec(&adapter->rx_buff_pool[queue_index][pool].available);
The second one runs under the per-queue replenish_lock, as its own
kernel-doc describes, and neither takes RTNL.
So rtnl_lock() does not serialize the Available column against the
geometry columns; only atomic_read() keeps that value from tearing.
Could the comment scope the RTNL claim to the geometry/active fields and
say that available is merely read atomically? As written, a later change
that adds a non-atomic read of a NAPI-updated pool field under this
rtnl_lock() would look safe when it is not.
> +
> + seq_puts(m, "Queue Pool Size BuffSize Active Available\n");
> + seq_puts(m, "----- ---- ---- -------- ------ ---------\n");
[Severity: Low]
This isn't a bug, but does the "Size" column name collide with the
existing poolN sysfs naming, with the opposite meaning?
The debugfs table prints pool->size (a buffer count) under "Size" and
pool->buff_size (bytes per buffer) under "BuffSize", while veth_pool_show()
maps the attributes the other way around:
else if (attr == &veth_num_attr)
return sprintf(buf, "%d\n", pool->size);
else if (attr == &veth_size_attr)
return sprintf(buf, "%d\n", pool->buff_size);
veth_pool_store() confirms this by range checking the size attribute
against IBMVETH_MAX_BUF_SIZE and num against IBMVETH_MAX_POOL_COUNT.
So "size" means a count in debugfs and a byte length in sysfs for the same
pool objects. Would naming the column "Count" or "Num" avoid that?
> + if (!adapter->opened)
> + seq_puts(m, "# down: Active/Available 0 unless allocated\n");
> +
> + for (i = 0; i < ibmveth_get_num_rx_queues(adapter); i++) {
> + for (j = 0; j < IBMVETH_NUM_BUFF_POOLS; j++) {
> + struct ibmveth_buff_pool *pool =
> + &adapter->rx_buff_pool[i][j];
[Severity: Low]
Can the Size/BuffSize columns print zero or stale values for queues
1..N here?
ibmveth_probe() publishes the queue count up front:
ibmveth_publish_num_rx_queues(adapter,
min(num_online_cpus(),
IBMVETH_DEFAULT_QUEUES));
but only initializes the queue-0 pools with
ibmveth_init_buffer_pool(&adapter->rx_buff_pool[0][i], ...). The copy to
the other queues happens later, in ibmveth_alloc_buffer_pools():
/* Initialize pool metadata for queues 1..N from queue 0 settings */
for (q = 1; q < ibmveth_get_num_rx_queues(adapter); q++) {
Reading buffer_pools before the first open therefore prints rows with
i > 0 showing Size 0 and BuffSize 0.
After an open/close cycle there is a second case: veth_pool_store() while
the device is down updates only the queue-0 pool and skips the reopen:
pool->active = newactive;
pool->buff_size = newbuff_size;
pool->size = newsize;
if (netif_running(netdev)) {
rc = ibmveth_open(netdev);
so queues 1..N keep the old geometry and the dump disagrees with queue 0.
The only hint emitted is "# down: Active/Available 0 unless allocated",
which covers just those two columns. Should the note also cover
Size/BuffSize for not-yet-templated queues, or should untemplated rows be
skipped?
> + bool live = pool->skbuff && pool->free_map;
> + int active = live ? pool->active : 0;
[Severity: Low]
Does the "Active" column here report something different from the
identically named sysfs attribute and ethtool statistic?
ibmveth_free_buffer_pool() deliberately keeps pool->active set when the
allocation goes away:
* Keep probe/sysfs geometry (active, size, buff_size, threshold).
* Clearing active here was a v3 ifdown/up regression: open skips
* !active pools, so reopen posted no RX buffers
while both existing surfaces print the raw flag:
veth_pool_show()
if (attr == &veth_active_attr)
return sprintf(buf, "%d\n", pool->active);
ibmveth_get_ethtool_stats()
data[i++] = adapter->rx_buff_pool[0][j].active;
With the interface down, .../pool0/active and poolN_active read 1 while
buffer_pools shows Active 0 for the same pool. Would a separate column
name such as "Alloc" or "Live" avoid overloading the existing meaning of
"active"?
The commit message says the file "lists size, buff_size, active, and
available for every RX queue and buffer pool" without mentioning that the
Active column is really an allocation indicator; only the stripped v5
changelog notes it. Could the commit message state that?
> + int available = live ? atomic_read(&pool->available)
> + : 0;
> +
> + seq_printf(m, "%5d %4d %4u %8u %6d %9d\n",
> + i, j, pool->size, pool->buff_size,
> + active, available);
> + }
> + }
> +
> + rtnl_unlock();
> + return 0;
> +}
> +DEFINE_SHOW_ATTRIBUTE(ibmveth_buffer_pools);
> +
> +/* Driver-owned root so per-adapter dirs use a stable vio name, not the
> + * mutable netdev->name (avoids stale names / eth0 collisions after rename).
> + */
> +static struct dentry *ibmveth_dbg_root;
> +
> +static void ibmveth_debugfs_init(struct ibmveth_adapter *adapter)
> +{
> + if (!ibmveth_dbg_root)
> + ibmveth_dbg_root =
> + debugfs_create_dir(ibmveth_driver_name, NULL);
> +
> + adapter->debugfs_dir =
> + debugfs_create_dir(dev_name(&adapter->vdev->dev),
> + ibmveth_dbg_root);
> + debugfs_create_file("buffer_pools", 0400, adapter->debugfs_dir,
> + adapter, &ibmveth_buffer_pools_fops);
> +}
[Severity: Medium]
Is this test-then-assign of ibmveth_dbg_root safe against concurrent
probes? ibmveth_debugfs_init() is called once per device from
ibmveth_probe(), and probe of two different vio devices is serialized only
by the per-device device_lock, so a DLPAR IO add racing with another add
(or with driver_attach during module load) can have both threads observe
NULL:
CPU0 CPU1
if (!ibmveth_dbg_root) if (!ibmveth_dbg_root)
debugfs_create_dir("ibmveth") debugfs_create_dir("ibmveth")
-> valid dentry -> ERR_PTR(-EEXIST)
ibmveth_dbg_root = dentry
ibmveth_dbg_root = ERR_PTR(-EEXIST)
fs/debugfs/inode.c returns ERR_PTR(-EEXIST) for the duplicate name
(debugfs_start_creating() -> simple_start_creating() with LOOKUP_EXCL,
then the dentry == ERR_PTR(-EEXIST) branch), and an ERR_PTR parent is
passed straight back out:
if (IS_ERR(parent))
return parent;
So after the race, every later per-adapter directory and buffer_pools file
is created with an ERR_PTR parent and silently fails, and the handle to the
real "ibmveth" directory is gone.
Would creating the root once in ibmveth_module_init(), before any probe can
run, avoid this?
> +
> +static void ibmveth_debugfs_exit(struct ibmveth_adapter *adapter)
> +{
> + debugfs_remove_recursive(adapter->debugfs_dir);
> + adapter->debugfs_dir = NULL;
> +}
> +
> static void ibmveth_put_pool_kobjs(struct ibmveth_adapter *adapter,
> int pools_ready)
> {
> @@ -3647,6 +3710,8 @@ static int ibmveth_probe(struct vio_dev *dev, const struct vio_device_id *id)
>
> netdev_dbg(netdev, "registered\n");
>
> + ibmveth_debugfs_init(adapter);
> +
> return 0;
> }
>
[ ... ]
> @@ -3891,6 +3958,8 @@ static int __init ibmveth_module_init(void)
> static void __exit ibmveth_module_exit(void)
> {
> vio_unregister_driver(&ibmveth_driver);
> + debugfs_remove_recursive(ibmveth_dbg_root);
> + ibmveth_dbg_root = NULL;
> }
[Severity: Medium]
Following on from the probe race above, does this leak the "ibmveth"
directory? If ibmveth_dbg_root holds ERR_PTR(-EEXIST),
debugfs_remove_recursive() returns immediately for IS_ERR_OR_NULL, so the
real directory and the simple_pin_fs reference it holds survive module
unload, and a later module load hits -EEXIST for the root every time.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 14/15] ibmveth: Implement incremental MQ RX queue resize
2026-08-14 7:36 ` [PATCH net-next v5 14/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Implement incremental MQ RX queue resize
This adds incremental RX queue scale-up/scale-down helpers so that
ethtool -L no longer needs a full adapter teardown, wires CMO
entitlement updates and rollback paths into the resize, and factors the
desired-DMA math into ibmveth_desired_dma_for_rxqs(). It also adds
ibmveth_resize_rx_channels() and calls it from ibmveth_set_channels().
A few questions on the changelog and on the new resize paths below.
[Severity: Medium]
Some of the work credited to this commit does not appear in its diff:
> Strengthen correlator liveness (active pool + non-NULL backing arrays)
> and clear freed pool metadata so torn-down pools no longer look
> in-range. Restore schedule_work escalation for truly impossible
> correlators.
> Also implement incremental scale-up/down helpers, WRITE_ONCE
> num_rx_queues publication, rollback/CMO updates, and related resize
> error paths.
git blame at this revision attributes those pieces to earlier commits in
the series:
- ibmveth_rxq_correlator_valid(), including the
"!bpool->skbuff || !bpool->free_map" liveness test, comes from
6246916e4cdd ("ibmveth: Add queue-aware RX buffer submit helper for
MQ")
- the schedule_work() escalation in
ibmveth_poll_skip_bad_correlator() comes from d613a86a7fa2
("ibmveth: Harden RX poll path with helpers")
- ibmveth_publish_num_rx_queues() with smp_wmb() + WRITE_ONCE() comes
from 4fbd8c64be1f ("ibmveth: Enable multi-queue RX receive path")
ibmveth_free_buffer_pool() is not touched here at all, and it explicitly
keeps active/size/buff_size/threshold, which reads as the opposite of
"clear freed pool metadata".
The changelog also describes a hang root cause being fixed, but the
parent commit has no live RX resize path in ibmveth_set_channels() -
this patch is what introduces it. Could the changelog be trimmed to the
contents of this diff?
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 1b58a3c6ce77a..84f4a0deb0c55 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -1380,6 +1432,141 @@ ibmveth_free_buffer_pools(struct ibmveth_adapter *adapter)
[ ... ]
> +/**
> + * ibmveth_free_single_rx_queue - Free resources for a single RX queue
> + * @adapter: ibmveth adapter structure
> + * @queue_idx: Queue index to free
> + *
> + * Frees buffer list, RX queue, and per-queue buffer pools for one queue.
> + * Used during incremental scale-down without affecting remaining queues.
> + */
> +static void
> +ibmveth_free_single_rx_queue(struct ibmveth_adapter *adapter, int queue_idx)
> +{
> + struct device *dev = &adapter->vdev->dev;
> +
> + ibmveth_free_queue_buffer_pools(adapter, queue_idx);
> +
> + if (adapter->buffer_list_dma[queue_idx]) {
> + dma_unmap_single(dev, adapter->buffer_list_dma[queue_idx],
> + 4096, DMA_BIDIRECTIONAL);
> + adapter->buffer_list_dma[queue_idx] = 0;
> + }
> +
> + if (adapter->rx_queue[queue_idx].queue_addr) {
> + dma_free_coherent(dev, adapter->rx_queue[queue_idx].queue_len,
> + adapter->rx_queue[queue_idx].queue_addr,
> + adapter->rx_queue[queue_idx].queue_dma);
> + adapter->rx_queue[queue_idx].queue_addr = NULL;
> + }
> +
> + if (adapter->buffer_list_addr[queue_idx]) {
> + free_page((unsigned long)adapter->buffer_list_addr[queue_idx]);
> + adapter->buffer_list_addr[queue_idx] = NULL;
> + }
> +
> + netdev_dbg(adapter->netdev, "Freed queue %d resources\n", queue_idx);
> +}
[ ... ]
> @@ -1797,6 +2034,356 @@ ibmveth_register_single_rx_queue(struct ibmveth_adapter *adapter,
> +static void
> +ibmveth_deregister_single_rx_queue(struct ibmveth_adapter *adapter,
> + int queue_idx)
> +{
> + unsigned long lpar_rc;
> + unsigned long ua = adapter->vdev->unit_address;
> + unsigned long qh = adapter->queue_handle[queue_idx];
> +
> + if (!qh)
> + return;
> +
> + do {
> + lpar_rc = h_free_logical_lan_queue(ua, qh);
> + } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY));
> +
> + adapter->hcall_stats.free_lan_queue++;
> +
> + if (lpar_rc != H_SUCCESS) {
> + netdev_err(adapter->netdev,
> + "h_free_logical_lan_queue failed for queue %d: rc=0x%lx\n",
> + queue_idx, lpar_rc);
> + }
> +
> + adapter->queue_handle[queue_idx] = 0;
> +
> + netdev_dbg(adapter->netdev, "Deregistered queue %d\n", queue_idx);
> +}
[Severity: Medium]
What happens if h_free_logical_lan_queue() returns a non-busy error such
as H_PARAMETER or H_HARDWARE here? The retry loop only covers
H_BUSY/H_IS_LONG_BUSY; other failures are logged, the handle is zeroed
and the function returns void, so the caller cannot tell.
ibmveth_destroy_subordinate_rx_queue() then goes on to release
everything PHYP may still own for that queue:
ibmveth_deregister_single_rx_queue(adapter, queue_idx);
ibmveth_cleanup_single_rx_interrupt(adapter, queue_idx);
ibmveth_free_single_rx_queue(adapter, queue_idx);
ibmveth_free_single_rx_queue() unmaps and frees the posted RX buffers via
ibmveth_free_queue_buffer_pools(), then dma_unmap_single() +
free_page() on buffer_list_addr[queue_idx] and dma_free_coherent() on
rx_queue[queue_idx].queue_addr.
If the hypervisor still owns the ring and the posted buffers, can this
let PHYP DMA into pages already returned to the allocator? With
queue_handle[queue_idx] cleared there is also no handle left to retry
the free with later. Should the deregister helper return an error and
the destroy path skip the unmap/free (or force a reset) in that case?
> +/**
> + * ibmveth_destroy_subordinate_rx_queue - Tear down one subordinate RX queue
> + * @adapter: ibmveth adapter structure
> + * @queue_idx: Queue index to destroy (1..N)
> + *
> + * Deregister with PHYP before unmapping buffer pools so hypervisor buffer
> + * ownership is released while queue metadata is still valid.
> + */
> +static void
> +ibmveth_destroy_subordinate_rx_queue(struct ibmveth_adapter *adapter,
> + int queue_idx)
> +{
> + ibmveth_deregister_single_rx_queue(adapter, queue_idx);
> + ibmveth_cleanup_single_rx_interrupt(adapter, queue_idx);
> + ibmveth_free_single_rx_queue(adapter, queue_idx);
> +}
[ ... ]
> + /*
> + * Fully ready before PHYP delivery, matching open():
> + * publish -> replenish -> napi_enable -> enable_irq.
> + * That way ibmveth_interrupt() cannot run on an
> + * unpublished, empty, or NAPI-disabled queue.
> + */
> + ibmveth_publish_num_rx_queues(adapter, i + 1);
[Severity: Medium]
Is the publication protocol complete on the reader side?
ibmveth_publish_num_rx_queues() pairs a write barrier with the store:
/* Per-queue state must be visible before readers observe num. */
smp_wmb();
WRITE_ONCE(adapter->num_rx_queues, num);
but the consumer has no matching barrier:
static inline unsigned int
ibmveth_get_num_rx_queues(const struct ibmveth_adapter *adapter)
{
return READ_ONCE(adapter->num_rx_queues);
}
Readers (ibmveth_interrupt(), ibmveth_poll(), ibmveth_poll_controller(),
ibmveth_replenish_task()) index per-queue state under only a control
dependency on that count, which powerpc does not order for loads. Can a
reader observe the bumped count while still seeing stale
queue_handle[i], queue_irq[i] or NULL rx_buff_pool[i][x] pointers for the
queue just added? ibmveth_drain_rx_queue() added by this patch does have
an explicit smp_rmb(); should ibmveth_get_num_rx_queues() carry the
acquire side instead?
> + ibmveth_replenish_task(adapter, i);
> + napi_enable(&adapter->napi[i]);
> +
> + rc = ibmveth_enable_irq(adapter, i);
> + if (rc) {
> + netdev_err(netdev,
> + "Failed to enable IRQ for queue %d: %d\n",
> + i, rc);
> + /*
> + * Published, replenished, and NAPI-enabled,
> + * but PHYP never unmasked. Match scale-down /
> + * shared cleanup: drain posted buffers, then
> + * deregister before unmap via
> + * destroy_subordinate.
> + */
> + ibmveth_publish_num_rx_queues(adapter, i);
> + napi_disable(&adapter->napi[i]);
[Severity: Medium]
Is the ordering here intentional? This path publishes the live count
down to i before napi_disable(&adapter->napi[i]), while the scale-down
loop and cleanup_new_queues below do napi_disable() first and publish
afterwards.
Queue i has already been published and napi_enable()d at this point, so a
poll can be scheduled for it (netpoll via ibmveth_poll_controller(), or a
PHYP interrupt if the unmask actually took effect and the hcall still
reported an error). ibmveth_poll() starts with:
if (WARN_ON(queue_index < 0 ||
queue_index >= ibmveth_get_num_rx_queues(adapter))) {
so can a poll scheduled for queue i now hit that WARN_ON?
> + ibmveth_drain_rx_queue(adapter, i);
> + synchronize_net();
> + ibmveth_destroy_subordinate_rx_queue(adapter,
> + i);
> + /* enable_irq returns errno; keep -EIO. */
> + rc = -EIO;
> + goto cleanup_new_queues;
> + }
> + ibmveth_kick_rx_queue_if_pending(adapter, i);
[Severity: Medium]
Can this leave a newly added queue permanently dead if replenishment
posted nothing?
ibmveth_replenish_task() is void and swallows both total SKB allocation
failure and ordinary post failures (IBMVETH_REPLENISH_HCALL_FAIL only
produces a rate-limited warning). The kick afterwards is conditional:
if (ibmveth_rxq_pending_buffer(adapter, queue_index))
ibmveth_schedule_rx_queue(adapter, queue_index);
With zero posted buffers PHYP has nothing to complete, so
ibmveth_rxq_pending_buffer() is false, no interrupt arrives, and poll
never runs to retry the replenish. There is no timer or delayed work in
ibmveth.c that would retry either. The queue is still counted in
real_num_rx_queues, so flows hashed to it are dropped until the admin
runs ethtool again or does ifdown/ifup.
The open path uses ibmveth_restart_rx_queue(), which always schedules one
poll and therefore gets a retry. Would the same be appropriate here, and
in the scale-down rollback below which repeats this sequence after
draining?
> + }
> +
> + rc = netif_set_real_num_rx_queues(netdev, new_count);
> + if (rc) {
> + netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
> + new_count, rc);
> + goto cleanup_new_queues;
> + }
> + } else {
> + netdev_dbg(netdev, "Scale-down: removing queues %d-%d\n",
> + new_count, old_count - 1);
[ ... ]
> + for (i = new_count; i < old_count; i++)
> + ibmveth_drain_rx_queue(adapter, i);
> +
> + synchronize_net();
> +
> + ibmveth_publish_num_rx_queues(adapter, new_count);
> +
> + rc = netif_set_real_num_rx_queues(netdev, new_count);
[Severity: High]
Is the ordering of synchronize_net() and the count publish inverted here?
The grace period runs before ibmveth_publish_num_rx_queues(adapter,
new_count), and the destroy loop that follows the publish has no further
grace period and takes no lock - in particular not the per-queue
rx_queue[i].replenish_lock that readers hold.
ibmveth_poll_controller() (ndo_poll_controller, so netpoll/netconsole,
outside RTNL and outside the NAPI state machine) samples the count once
and then walks it:
unsigned int num = ibmveth_get_num_rx_queues(adapter);
for (i = 0; i < num; i++)
ibmveth_replenish_task(adapter, i);
for (i = 0; i < num; i++)
ibmveth_schedule_rx_queue(adapter, i);
and ibmveth_replenish_task() re-checks the count before it takes the lock
and dereferences the pool arrays:
if (queue_index >= ibmveth_get_num_rx_queues(adapter)) { ... return; }
...
spin_lock_irqsave(&rxq->replenish_lock, flags);
... if (pool->active && pool->free_map && ...)
which looks like a pure time-of-check/time-of-use test against the
teardown path. Meanwhile ibmveth_free_buffer_pool() frees exactly those
pointers with no lock held:
kfree(pool->free_map);
pool->free_map = NULL;
... kfree(pool->dma_addr); ... kfree(pool->skbuff);
Can a reader that already passed the count check be inside
ibmveth_replenish_task() on queue i while free_map/skbuff/dma_addr are
kfree()d, buffer_list_addr[i] is free_page()d and rx_queue[i].queue_addr
is dma_free_coherent()d? The changelog says dropping the live count
means netpoll cannot walk dying queues, but with the grace period taken
before the publish that does not seem to hold. The same pattern appears
in cleanup_new_queues below (synchronize_net(), then publish, then
destroy).
> + if (rc) {
> + netdev_err(netdev, "Failed to set real RX queues to %d: %d\n",
> + new_count, rc);
> + ibmveth_publish_num_rx_queues(adapter, old_count);
> + for (i = new_count; i < old_count; i++) {
> + int irq_rc;
> +
> + ibmveth_replenish_task(adapter, i);
> + /* START: NAPI before PHYP unmask. */
> + napi_enable(&adapter->napi[i]);
> + irq_rc = ibmveth_enable_irq(adapter, i);
> + if (irq_rc) {
> + netdev_err(netdev,
> + "Failed to re-enable IRQ for queue %d during scale-down rollback (rc=%d), scheduling reset\n",
> + i, irq_rc);
> + schedule_work(&adapter->work);
> + continue;
> + }
> + ibmveth_kick_rx_queue_if_pending(adapter, i);
> + }
> + return rc;
> + }
> +
> + for (i = new_count; i < old_count; i++)
> + ibmveth_destroy_subordinate_rx_queue(adapter, i);
> +
> + /* Q0 (and any still-live queues) were not in the teardown
> + * loop. Scale-down can leave Q0 masked with NAPI idle — kick
> + * so ARP/RX does not die until rmmod (lab: -L rx 1).
> + */
> + for (i = 0; i < new_count; i++)
> + ibmveth_restart_rx_queue(adapter, i);
[Severity: Medium]
Is it safe to call ibmveth_restart_rx_queue() on queues that were never
quiesced by this function and are actively receiving?
ibmveth_restart_rx_queue() falls back to an unmask when the NAPI schedule
attempt fails:
ibmveth_replenish_task(adapter, qindex);
if (ibmveth_schedule_rx_queue(adapter, qindex))
return;
rc = ibmveth_enable_irq(adapter, qindex);
WARN_ON(rc);
napi_schedule_prep() failing is precisely the state "a poll is already in
flight and has masked PHYP", so the fallback unmasks PHYP behind that
running poll. The poll's own tail then calls ibmveth_enable_irq() on an
already-enabled subordinate queue, and ibmveth_toggle_irq() folds
H_PARAMETER to success only on disable:
if (h_rc == H_PARAMETER && !enable) { ... return 0; }
...
if (h_rc) { ... return -EIO; }
so the poll gets -EIO and escalates to schedule_work(&adapter->work),
i.e. a full close+open of the adapter. In the reverse interleaving the
resize-side enable is the one that returns -EIO and trips the WARN_ON.
With new_count >= 2 and traffic running on a surviving subordinate queue,
can ethtool -L rx <smaller> trigger a spurious adapter reset or that WARN
splat? Should the restart loop be limited to queue 0, or use a path that
does not unmask when a poll owns the queue?
> + }
> +
> + netdev_info(netdev, "Successfully resized to %d RX queues (incremental)\n",
> + ibmveth_get_num_rx_queues(adapter));
[ ... ]
> +cleanup_new_queues:
[ ... ]
> + synchronize_net();
> +
> + /* Drop the live count before freeing the half-added queues. */
> + ibmveth_publish_num_rx_queues(adapter, old_count);
> +
> + for (i = old_count; i < failed_queue; i++)
> + ibmveth_destroy_subordinate_rx_queue(adapter, i);
[ ... ]
> @@ -2535,6 +3122,55 @@ static void ibmveth_get_channels(struct net_device *netdev,
[ ... ]
> +static int ibmveth_resize_rx_channels(struct ibmveth_adapter *adapter,
> + unsigned int goal_rx)
> +{
[ ... ]
> + if (goal_rx == old_rx)
> + return 0;
> +
> + /* Down / failed-open: caller stashes. Do not allocate here. */
> + if (!adapter->opened)
> + return 0;
[Severity: Low]
The kernel-doc and this comment say the caller stashes the desired count
when RX resources are not live, but at this commit the only caller does:
rc = ibmveth_resize_rx_channels(adapter, channels->rx_count);
if (rc)
return rc;
if (!adapter->opened)
return netif_set_real_num_tx_queues(netdev, goal);
There is no adapter->num_rx_queues update, no
netif_set_real_num_rx_queues() and no CMO refresh on that path, so
ethtool -L ethX rx N on a down interface returns success while
ibmveth_get_channels() keeps reporting the old count.
The following patch in the series, "ibmveth: Wire ethtool set_channels to
MQ RX queue resize", does add the stash/publish/CMO block, so the final
state of the series matches the documentation - only this intermediate
commit does not.
> +
> + rxq_entries = adapter->rx_queue[0].num_slots;
> + rc = ibmveth_resize_rx_queues_incremental(adapter, goal_rx,
> + rxq_entries);
> + if (rc)
> + netdev_err(netdev, "Failed to resize RX queues: %d\n", rc);
> + return rc;
> +}
> +
> static int ibmveth_set_channels(struct net_device *netdev,
> struct ethtool_channels *channels)
> {
> @@ -2543,10 +3179,15 @@ static int ibmveth_set_channels(struct net_device *netdev,
> goal = channels->tx_count;
> int rc, i;
>
> - /* If ndo_open has not been called yet then don't allocate, just set
> - * desired netdev_queue's and return
> + /* Validate RX (and resize when opened) before the down-path early
> + * return so MQ/range errors are not deferred to the wiring patch.
> + * RX stash + CMO while down still lands with that patch.
> */
> - if (!(netdev->flags & IFF_UP))
> + rc = ibmveth_resize_rx_channels(adapter, channels->rx_count);
> + if (rc)
> + return rc;
[Severity: Low]
The live RX resize is committed here, ahead of all TX work, and at this
commit nothing unwinds it if the TX portion later fails - the tail of
ibmveth_set_channels() just returns rc after
netif_set_real_num_tx_queues() fails. So ethtool -L ethX rx N tx M can
return an error to userspace with the hypervisor-visible queue set,
adapter->num_rx_queues, netdev->real_num_rx_queues and the CMO
entitlement already changed.
The next patch in the series adds the rx_changed rollback that calls
ibmveth_resize_rx_queues_incremental(adapter, old_rx, rxq_entries) on TX
failure, so the series head does handle it; only this commit is missing
the unwind.
> +
> + if (!adapter->opened)
> return netif_set_real_num_tx_queues(netdev, goal);
[ ... ]
[Severity: Medium]
Not a problem introduced by this patch, but the in-tree KUnit cases in
ibmveth.c look broken as of this series and are still broken at the
series head.
ibmveth_rxq_correlator_valid() requires free_map to be non-NULL:
if (!bpool->active || !bpool->skbuff || !bpool->free_map)
return false;
Both fixtures only allocate pool->skbuff with kunit_kcalloc() after
ibmveth_init_buffer_pool(), which never sets free_map. So in
ibmveth_remove_buffer_from_pool_test() this case:
correlator = (u64)0 | 0;
pool->skbuff[0] = NULL;
KUNIT_EXPECT_EQ(test, -EFAULT,
ibmveth_remove_buffer_from_pool(adapter, correlator, 0, false));
gets -EINVAL from the correlator check instead of -EFAULT, and in
ibmveth_rxq_get_buffer_test():
pool->skbuff[0] = skb;
adapter->rx_queue[0].queue_addr[0].correlator = (u64)0 << 32 | 0;
KUNIT_EXPECT_PTR_EQ(test, skb, ibmveth_rxq_get_buffer(adapter, 0));
gets NULL. The liveness check came in with 6246916e4cdd ("ibmveth: Add
queue-aware RX buffer submit helper for MQ"). Should the fixtures also
allocate pool->free_map so CONFIG_IBMVETH_KUNIT_TEST builds pass?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH net-next v5 15/15] ibmveth: Wire ethtool set_channels to MQ RX queue resize
2026-08-14 7:36 ` [PATCH net-next v5 15/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
@ 2026-08-18 1:47 ` Jakub Kicinski
0 siblings, 0 replies; 31+ messages in thread
From: Jakub Kicinski @ 2026-08-18 1:47 UTC (permalink / raw)
To: mmc
Cc: Jakub Kicinski, netdev, davem, edumazet, pabeni, andrew+netdev,
nnac123, maddy, mpe, linuxppc-dev, haren, ricklind, davemarq,
bjking1, shaik.abdulla1
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
ibmveth: Wire ethtool set_channels to MQ RX queue resize
This completes the RX side of ibmveth_set_channels(): while the device is
down the desired RX count is stashed, and while it is up RX is resized
first and the TX LTB stop/alloc/set_real_num_tx/free/wake path runs
afterwards. It also adds up-front TX range validation and an RX rollback
when TX cannot reach the requested count.
A few questions on the new paths below.
> diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
> index 84f4a0deb0c55..35cbd2553e867 100644
> --- a/drivers/net/ethernet/ibm/ibmveth.c
> +++ b/drivers/net/ethernet/ibm/ibmveth.c
[ ... ]
> @@ -3175,28 +3176,75 @@ static int ibmveth_set_channels(struct net_device *netdev,
> struct ethtool_channels *channels)
> {
> struct ibmveth_adapter *adapter = netdev_priv(netdev);
> - unsigned int old = netdev->real_num_tx_queues,
> - goal = channels->tx_count;
> + unsigned int old_rx = ibmveth_get_num_rx_queues(adapter);
> + unsigned int goal_rx = channels->rx_count;
> + unsigned int old_tx = netdev->real_num_tx_queues;
> + unsigned int goal_tx = channels->tx_count;
> + unsigned int want_tx = goal_tx;
> + int rxq_entries = adapter->rx_queue[0].num_slots;
> + bool rx_changed = false;
> int rc, i;
>
> - /* Validate RX (and resize when opened) before the down-path early
> - * return so MQ/range errors are not deferred to the wiring patch.
> - * RX stash + CMO while down still lands with that patch.
> - */
> - rc = ibmveth_resize_rx_channels(adapter, channels->rx_count);
> + if (goal_tx < 1 || goal_tx > ibmveth_real_max_tx_queues()) {
> + netdev_err(netdev,
> + "Invalid TX queue count %u (must be 1-%u)\n",
> + goal_tx, ibmveth_real_max_tx_queues());
> + return -EINVAL;
> + }
[Severity: Medium]
This isn't a bug introduced by this patch, but the bound used here is
CPU-count dependent while the reported current value is not, so the two can
disagree.
ibmveth_get_channels() fills the reply from two different sources:
channels->max_tx = ibmveth_real_max_tx_queues();
channels->tx_count = netdev->real_num_tx_queues;
and ibmveth_real_max_tx_queues() is min(num_online_cpus(),
IBMVETH_MAX_QUEUES). After configuring N TX queues and then offlining CPUs
so that num_online_cpus() < N, the reply advertises tx_count > max_tx.
Both ethtool entry points reject a request whose tx_count exceeds the
driver-advertised max_tx:
net/ethtool/ioctl.c:ethtool_set_channels() {
...
if (channels.rx_count > curr.max_rx ||
channels.tx_count > curr.max_tx || ...)
return -EINVAL;
...
}
so even an RX-only change (which resubmits the current tx_count) fails with
-EINVAL until the CPUs come back online. Should max_tx be latched from the
configured count instead, or should the new range check be against
IBMVETH_MAX_QUEUES?
> +
> + /* RX range / MQ checks live in ibmveth_resize_rx_channels(). */
> + rc = ibmveth_resize_rx_channels(adapter, goal_rx);
> if (rc)
> return rc;
>
> - if (!adapter->opened)
> - return netif_set_real_num_tx_queues(netdev, goal);
> + /* If RX resources are not live (never opened, or close+open failed
> + * while IFF_UP stayed set), only stash desired queue counts.
> + */
> + if (!adapter->opened) {
> + /* Apply TX first so a failure leaves RX stash unchanged. */
> + rc = netif_set_real_num_tx_queues(netdev, goal_tx);
> + if (rc)
> + return rc;
> +
> + /* Stash desired RX count; open() publishes it via
> + * netif_set_real_num_rx_queues() after queue registration.
> + * Refresh CMO now so open() can map the larger footprint;
> + * open itself does not call vio_cmo_set_dev_desired.
> + */
[Severity: Low]
Is this comment describing what the code does? The block right below
publishes immediately rather than deferring to open():
ibmveth_publish_num_rx_queues() updates adapter->num_rx_queues and
netif_set_real_num_rx_queues() updates netdev->real_num_rx_queues, which
creates or removes the /sys/class/net/*/queues/rx-N entries while the
interface is down. ibmveth_open() only re-applies the same value later:
rc = netif_set_real_num_rx_queues(netdev,
ibmveth_get_num_rx_queues(adapter));
The commit message says the same thing:
- When down: set TX queues first, then stash the desired RX count
in adapter->num_rx_queues for the next open() (open publishes via
netif_set_real_num_rx_queues).
Could the comment and the changelog mention that real_num_rx_queues, the
rx-N sysfs entries and (below) the CMO entitlement all change immediately
on a down interface?
> + if (goal_rx != ibmveth_get_num_rx_queues(adapter)) {
> + ibmveth_publish_num_rx_queues(adapter, goal_rx);
[Severity: Medium]
Can publishing a count with no allocated queues behind it let a lockless
reader take an uninitialized spinlock?
adapter->num_rx_queues is the only validity bound the RX readers use, and
rx_queue[i].replenish_lock is only spin_lock_init()'ed for queues that are
actually allocated (ibmveth_alloc_rx_queues() and
ibmveth_alloc_single_rx_queue()).
ibmveth_replenish_task() checks nothing else before locking:
if (queue_index >= ibmveth_get_num_rx_queues(adapter)) {
...
return;
}
...
spin_lock_irqsave(&rxq->replenish_lock, flags);
and ibmveth_poll_controller() has no adapter->opened check:
unsigned int num = ibmveth_get_num_rx_queues(adapter);
for (i = 0; i < num; i++)
ibmveth_replenish_task(adapter, i);
That is reachable in the state this branch targets: pool_store and
change_mtu call ibmveth_close()+ibmveth_open() directly, so IFF_UP can stay
set (netif_running() true, netpoll still attached) with adapter->opened
false. An "ethtool -L ethX rx N" with N above any previously allocated
count then publishes N here, and the window stays open for as long as the
device is down.
ibmveth_publish_num_rx_queues() itself states the invariant:
/* Per-queue state must be visible before readers observe num. */
smp_wmb();
WRITE_ONCE(adapter->num_rx_queues, num);
Does this call site break it, given the per-queue state does not exist yet?
> + rc = netif_set_real_num_rx_queues(netdev, goal_rx);
> + if (rc) {
> + ibmveth_publish_num_rx_queues(adapter, old_rx);
> + return rc;
> + }
[Severity: Medium]
On this failure path, netdev->real_num_tx_queues was already moved to
goal_tx by the netif_set_real_num_tx_queues() call above, and only the RX
publish is undone.
netif_set_real_num_rx_queues() has a real failure path:
net/core/dev.c:netif_set_real_num_rx_queues() {
...
rc = net_rx_queue_update_kobjects(dev, dev->real_num_rx_queues, rxq);
if (rc)
return rc;
...
}
so ethtool can report failure while the TX count has permanently changed.
Should netif_set_real_num_tx_queues(netdev, old_tx) run here before
returning rc, and does the current code match the commit message claim
"Make TX/RX updates fail atomically"?
> + if (firmware_has_feature(FW_FEATURE_CMO)) {
> + unsigned long dma;
> +
> + dma = ibmveth_get_desired_dma(adapter->vdev);
> + vio_cmo_set_dev_desired(adapter->vdev, dma);
> + }
[Severity: Medium]
Is the entitlement raised here ever lowered again if the stashed count is
never realized? The desired value is sized for goal_rx RX queues, but no
buffer lists, queue DMA or pools are mapped while down.
If the next open() fails for the larger count, or if it applies the MQ
fallback and drops the count back to 1, neither ibmveth_open(),
ibmveth_close() nor ibmveth_apply_mq_fallback() recomputes desired DMA, so
the partition keeps CMO entitlement reserved for RX resources that do not
exist until a later successful set_channels or change_mtu.
> + }
> + return 0;
> + }
[Severity: Medium]
Can a request that returns 0 from this branch be silently discarded on the
next open()?
If firmware rejected MQ during a previous ibmveth_register_rx_queues(),
adapter->mq_fallback is latched while adapter->multi_queue is still
non-zero, so "ethtool -L ethX rx 8" passes the MQ check in
ibmveth_resize_rx_channels(), takes this !opened branch, publishes 8, sets
real_num_rx_queues = 8, raises CMO desired and returns success.
ibmveth_apply_mq_fallback() then runs first in ibmveth_open():
adapter->multi_queue = 0;
ibmveth_publish_num_rx_queues(adapter, 1);
with no error back to the user and no entitlement recompute, and
ibmveth_get_channels() afterwards reports:
channels->rx_count = ibmveth_get_num_rx_queues(adapter);
which is 1, contradicting the request that returned 0. Should the down
path reject (or at least warn about) goal_rx > 1 when mq_fallback is
latched?
> +
> + if (goal_rx != old_rx)
> + rx_changed = true;
>
> /* We have IBMVETH_MAX_QUEUES netdev_queue's allocated
> * but we may need to alloc/free the ltb's.
> */
> + if (goal_tx == old_tx)
> + return 0;
> +
> netif_tx_stop_all_queues(netdev);
>
> - /* Allocate any queue that we need */
> - for (i = old; i < goal; i++) {
> + /* Allocate any new TX LTBs. i starts at old_tx for the free walk
> + * below when this loop body never runs (goal_tx == old_tx already
> + * returned; goal_tx < old_tx is scale-down).
> + */
> + i = old_tx;
> + for (; i < goal_tx; i++) {
[Severity: Low]
The commit message says:
Initialize the TX alloc loop bound so a scale-down path that never
allocates still has defined cleanup if set_real_num_tx_queues() fails.
Was there anything undefined before? The removed line is:
for (i = old; i < goal; i++) {
whose initializer runs even when the body does not (old >= goal), so i was
already defined for the later "for (i = old; i > goal; i--)" free walk.
Could this be described as a readability change rather than a fix?
> if (adapter->tx_ltb_ptr[i])
> continue;
>
> @@ -3205,28 +3253,43 @@ static int ibmveth_set_channels(struct net_device *netdev,
[ ... ]
> netif_tx_wake_all_queues(netdev);
>
> - return rc;
> + if (netdev->real_num_tx_queues != want_tx) {
> + if (rx_changed) {
> + int rb;
> +
> + rb = ibmveth_resize_rx_queues_incremental(adapter,
> + old_rx,
> + rxq_entries);
> + if (rb)
> + netdev_err(netdev,
> + "Failed to roll back RX queues to %u after TX failure: %d\n",
> + old_rx, rb);
> + }
> + return rc ? rc : -ENOMEM;
> + }
[Severity: Medium]
This rollback is best effort only, so the same "fail atomically" question
applies to the up path. For "ethtool -L ethX rx <smaller> tx <larger>",
the RX scale-down has already destroyed queues; if
ibmveth_allocate_tx_ltb() then fails, the rollback here is a scale-up whose
own steps can fail too:
rc = ibmveth_alloc_single_rx_queue(adapter, i, rxq_entries);
if (rc) { ... goto cleanup_new_queues; }
and the same for ibmveth_register_single_rx_queue(),
ibmveth_setup_single_rx_interrupt(), ibmveth_enable_irq() and
netif_set_real_num_rx_queues(). Its cleanup path leaves RX at the reduced
count, and here that is only logged before returning an error.
Is there a way to order this so the destructive RX change happens only
after the TX LTB allocations have succeeded, so no partial state can be
left behind when the call reports failure?
> }
>
> static const struct ethtool_ops netdev_ethtool_ops = {
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2026-08-18 1:47 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-14 7:36 [PATCH net-next v5 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-08-14 7:36 ` [PATCH net-next v5 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 02/15] ibmveth: Prepare MQ RX adapter data structures Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 07/15] ibmveth: Add RX queue register helpers for MQ Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 08/15] ibmveth: Add queue-aware RX buffer submit helper " Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 09/15] ibmveth: Harden RX poll path with helpers Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 11/15] ibmveth: Add per-queue RX and TX statistics collection Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 13/15] ibmveth: Expose per-queue buffer pool details via debugfs Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 14/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
2026-08-14 7:36 ` [PATCH net-next v5 15/15] ibmveth: Wire ethtool set_channels to " Mingming Cao
2026-08-18 1:47 ` Jakub Kicinski
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.