[PATCH v1 00/18] ibmveth: Add multi-queue RX Support

[Mingming Cao <mmc@linux.ibm.com>]

Power11 PHYP firmware adds Virtual Ethernet multi-queue (MQ) RX for
the ibmveth device: multiple logical-LAN RX queues, per-queue buffer
posting, and completion delivery. Guest Linux did not use that
platform support; ibmveth still registered one RX queue even when
PHYP was MQ-capable.

This series adds the ibmveth MQ client. When PHYP advertises the
capability through H_ILLAN_ATTRIBUTES, the driver registers
multiple RX queues, receives on per-queue NAPI, and exposes queue
count through ethtool. Older firmware without the bit is unchanged.


ibmveth today registers one logical LAN, one set of buffer pools, and
one NAPI context. PHYP MQ mode gives each RX queue its own handle:
buffers are posted with H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, subordinate
queues register through H_REG_LOGICAL_LAN_QUEUE, and traffic can
land on any active queue. Queue selection is firmware-defined; v1
does not program RSS or hash tables. The driver needs per-queue
pools, IRQs, and poll state to match.

Queue-aware hcalls are selected only when probe sets multi_queue
from H_ILLAN_ATTRIBUTES; legacy firmware keeps the original hcall
path unchanged through the entire series.

This splits the work so review follows the actual bring-up sequence:

 1. Hypercall definitions and MQ data structures (patches 1-3)
 2. Refactor open/close into helpers - RX, per-queue pools,
    IRQ, TX, PHYP (4-10)
 3. Turn on the MQ datapath at probe/open (11)
 4. Per-queue RX/TX stats, get_stats64, and sysfs pool readout
    (12-14)
 5. Runtime RX queue resize via ethtool -L (15-17)
 6. Runtime stability fixes from LPAR testing (18)


 - Helper patches (4-10) reshape ibmveth_open()/close() into
queue-aware helpers. Runtime behaviour is unchanged through that
block: num_rx_queues stays 1 and multi_queue is false until patch 11.

- Patch 11 is the switch: probe sets multi_queue from firmware, raises
num_rx_queues, registers subordinates, and replenishes every active
queue.

- Patch 18 fixes poll hangs after aggressive ethtool -L cycling,
NAPI/close deadlocks on ip link down, and preserves probe-time
pool->active across close/open so RX works after link down/up.


Design notes
* Per-queue buffer pools (rx_buff_pool[queue][pool]) - PHYP ties
 posted buffers to a queue handle; a shared pool set does not work.
 Patch 5 also disables the 64 KiB pool at standard MTU to save
 per-queue memory in MQ.
* Legacy mode keeps queue 0 on h_register_logical_lan(); MQ uses
 handles for all queues (subordinates via H_REG_LOGICAL_LAN_QUEUE).
 Close uses H_FREE_LOGICAL_LAN for the whole adapter.
* ethtool -L resizes incrementally while the netdev stays up so
 surviving queues keep PHYP handles, pools, and IRQ state. A
 close/open cycle would drop traffic and force full LAN
 re-registration for every queue.


Tested on ppc64le PowerVM LPAR with MQ-capable firmware:
* MQ path: ethtool -L under iperf3 load, link down/up during traffic
* Legacy firmware (no MQ bit): full open/close/stress on the
 refactored helper path to confirm single-queue behaviour is
 unchanged
* ethtool -L resize while all RX queues are receiving traffic, not
 only a single-flow iperf session
* ip link down/up and ping after reopen (patch 18)


Future work
* IRQ affinity hints for subordinate queue IRQs returned by PHYP
* Summed global no_buffer drop counter across all RX queues in MQ mode
Comments and suggestions on patch split, design, and testing are
welcome.

Mingming Cao <mmc@linux.ibm.com>

Mingming Cao (18):
  ibmveth: Add MQ RX hypercall wrappers and call definitions
  ibmveth: Prepare adapter data structures for MQ RX
  ibmveth: Add MQ-ready RX statistics structures
  ibmveth: Refactor RX resource allocation for MQ RX bring-up
  ibmveth: Refactor buffer pool management for per-queue MQ RX
  ibmveth: Refactor RX interrupt control for MQ RX queues
  ibmveth: Refactor TX resource allocation in open/close paths
  ibmveth: Add RX queue register/deregister helpers for MQ
  ibmveth: Refactor open/close into MQ-ready resource pipeline
  ibmveth: Add queue-aware RX buffer submit helper for MQ
  ibmveth: Enable multi-queue RX receive path
  ibmveth: Add per-queue RX statistics collection and reporting
  ibmveth: Add per-queue TX statistics reporting
  ibmveth: Expose per-queue buffer pool details via sysfs
  ibmveth: Add helpers for incremental MQ RX queue resize
  ibmveth: Implement incremental MQ RX queue resize
  ibmveth: Wire ethtool set_channels to MQ RX queue resize
  ibmveth: Fix MQ RX poll and shutdown hangs after queue resize

 arch/powerpc/include/asm/hvcall.h  |    6 +-
 drivers/net/ethernet/ibm/ibmveth.c | 2451 +++++++++++++++++++++++-----
 drivers/net/ethernet/ibm/ibmveth.h |  226 ++-
 3 files changed, 2284 insertions(+), 399 deletions(-)

-- 
2.39.3 (Apple Git-146)