From: Jakub Kicinski <kuba@kernel.org>
To: Mingming Cao <mmc@linux.ibm.com>
Cc: netdev@vger.kernel.org, davem@davemloft.net, horms@kernel.org,
edumazet@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch,
nnac123@linux.ibm.com, maddy@linux.ibm.com, mpe@ellerman.id.au,
linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com,
chleroy@kernel.org, ritesh.list@gmail.com, sshegde@linux.ibm.com,
haren@linux.ibm.com, ricklind@linux.ibm.com,
davemarq@linux.ibm.com, bjking1@linux.ibm.com,
shaik.abdulla1@ibm.com
Subject: Re: [PATCH net-next v8 08/15] ibmveth: Add queue-aware RX buffer submit helper for MQ
Date: Mon, 16 Nov 2026 17:36:04 -0800 [thread overview]
Message-ID: <20261116173604.50e83f10@kernel.org> (raw)
In-Reply-To: <20261109235556.60604-9-mmc@linux.ibm.com>
On Mon, 9 Nov 2026 15:55:49 -0800 Mingming Cao wrote:
> This patch queue-indexes RX replenish and pool access ahead of MQ
> enable:
>
> - hcall: ibmveth_add_logical_lan_buffers() (MQ queue hcall vs legacy)
> - concurrency: per-queue replenish_lock; replenish outcomes logged
> after unlock
> - poll/pool: rxq_* take queue_index; INVALID_MAP and correlator
> errors refactored
> - limits: IBMVETH_MAX_RX_PER_HCALL 12, IBMVETH_MAX_RX_REGULAR 8
>
> Runtime stays single-queue until the MQ enable patch; correlator,
> lock and replenish return paths apply on queue 0 immediately.
>
> Replenish is the last open-path hypercall that still assumes queue 0.
> Registration is already queue-aware, and multi-queue posts buffers
> through H_ADD_LOGICAL_LAN_BUFFERS_QUEUE against adapter->queue_handle[],
> but ibmveth_replenish_buffer_pool() still calls
> h_add_logical_lan_buffer() or h_add_logical_lan_buffers() directly.
>
> Add ibmveth_add_logical_lan_buffers() to route RX buffer pool
> replenishment through H_ADD_LOGICAL_LAN_BUFFERS_QUEUE in multi-queue
> mode, falling back to the legacy 8-buffer and single-buffer hypercalls
> in single-queue mode.
>
> Supporting that helper means the RX side can no longer assume queue 0.
> The queue index is 0 everywhere until multi-queue RX is enabled, so
> the queue plumbing below does not change behaviour yet. The correlator
> and locking changes do take effect immediately, on the single queue.
>
> Parameterise the RX accessors by queue. The eight ibmveth_rxq_*
> helpers took only the adapter and hardcoded rx_queue[0]; they now take
> a queue_index, as do ibmveth_remove_buffer_from_pool() and
> ibmveth_rxq_get_buffer(). ibmveth_rxq_correlator() becomes
> ibmveth_rxq_read_correlator(); poll still reads the correlator once
> per slot, as net-next does, and now passes the queue index with it
> to rxq_get_buffer() and rxq_harvest_buffer().
>
> Add a per-queue replenish_lock to struct ibmveth_rx_q, initialised in
> probe. Replenish is not the only writer of a pool's free_map: the
> harvest and remove consumer runs from that queue's NAPI instance. At
> this commit replenish also runs only from NAPI (netpoll's budget-0
> poll skips it), but later patches (MQ open and the queue resize)
> replenish from process context while that queue's NAPI may run, so
> producer and consumer can run at once. Its users are NAPI and process
> context, so it is taken with spin_lock_bh().
>
> Give ibmveth_replenish_buffer_pool() a return value. It was void and
> logged from inside what is now a critical section. It now returns one
> of IBMVETH_REPLENISH_OK, _RESET_MAP, _RESET_MQ, _HCALL_FAIL or
> _BATCH_FALLBACK, and ibmveth_replenish_task() does the logging and any
> schedule_work() after dropping replenish_lock. Replenish map paths use
> DMA_ATTR_NO_WARN; failures are counted, and the next poll's replenish
> retries. As before this patch, a queue whose pools are left with
> nothing posted gets no further poll and waits for a reopen. The
> hcall-failure warning now logs against the netdev rather than the vio
> device, the two batch-fallback info messages become one ratelimited
> warning, and DMA_ATTR_NO_WARN silences the DMA layer's map-failure
> warning. In multi-queue mode an H_FUNCTION from the buffer-add hcall
> schedules a reset instead of the legacy single-buffer fallback;
> that arm is unreachable until MQ RX is enabled.
>
> An IBM_VETH_INVALID_MAP free_map entry no longer WARNs. It now drops
> the partly filled batch (those skbs stay in the pool for reuse),
> skips the remaining pools and schedules a reset, where the old code
> posted the partial batch and moved on to the next pool.
> replenish_task() also returns early for a queue index at or past the
> live count, and ibmveth_poll() logs an IRQ enable failure before it
> schedules the reset.
>
> Update replenish_task_cycles under replenish_lock; the statistics patch
> moves that counter per queue.
>
> Snapshot the first BATCH_FALLBACK and HCALL_FAIL into their own
> copies and log those after unlock, so a later pool cannot overwrite
> the values those two messages print.
>
> Make the correlator check per queue. ibmveth_rxq_correlator_valid()
> takes the queue index and also requires the pool to be active with a
> free_map. It no longer logs or schedules the reset, because
> remove_buffer_from_pool() now calls it under replenish_lock:
> ibmveth_rxq_bad_correlator() and ibmveth_rxq_no_skb() do the
> ratelimited log and the reset, called by rxq_harvest_buffer() after
> remove_buffer_from_pool() returns and by rxq_get_buffer() directly.
> The ring still advances on every error. The KUnit tests pass the queue
> index and set up free_map and replenish_lock.
>
> Redefine IBMVETH_MAX_RX_PER_HCALL from 8 to 12, the argument-list
> capacity of H_ADD_LOGICAL_LAN_BUFFERS_QUEUE, and add
> IBMVETH_MAX_RX_REGULAR (8) for the legacy hypercall. Probe sets
> rx_buffers_per_hcall to IBMVETH_MAX_RX_REGULAR, so the legacy batch
> stays at 8 until multi-queue RX is enabled.
>
> free_buffer_pool() keeps probe/sysfs geometry (active/size/threshold)
> and zeroes pool->available so teardown cannot leave a stale count.
The further you get in the series the sloppier the commits.
Please trim this text, tell your LLM we can read the code :/
next prev parent reply other threads:[~2026-11-17 1:36 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-09 23:55 [PATCH net-next v8 00/15] ibmveth: Add multi-queue RX support Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 01/15] ibmveth: Add MQ RX hypercall wrappers and call definitions Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 02/15] ibmveth: Prepare MQ RX adapter data structures Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 03/15] ibmveth: Refactor RX resource allocation for MQ RX bring-up Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 04/15] ibmveth: Refactor buffer pool management for per-queue MQ RX Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 05/15] ibmveth: Refactor RX interrupt control for MQ RX queues Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-13 18:46 ` mingming cao
2026-11-09 23:55 ` [PATCH net-next v8 06/15] ibmveth: Refactor TX resource allocation in open/close paths Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-13 18:54 ` mingming cao
2026-11-09 23:55 ` [PATCH net-next v8 07/15] ibmveth: Add RX queue register helpers for MQ Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-17 1:34 ` Jakub Kicinski
2026-11-09 23:55 ` [PATCH net-next v8 08/15] ibmveth: Add queue-aware RX buffer submit helper " Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-17 1:36 ` Jakub Kicinski [this message]
2026-11-09 23:55 ` [PATCH net-next v8 09/15] ibmveth: Split RX poll into per-slot helpers " Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-17 1:40 ` Jakub Kicinski
2026-11-09 23:55 ` [PATCH net-next v8 10/15] ibmveth: Enable multi-queue RX receive path Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-13 18:59 ` mingming cao
2026-11-09 23:55 ` [PATCH net-next v8 11/15] ibmveth: Add per-queue RX and TX statistics collection Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-13 19:00 ` mingming cao
2026-11-09 23:55 ` [PATCH net-next v8 12/15] ibmveth: Report MQ-aware RX counts in ethtool get_channels Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 13/15] ibmveth: Expose per-queue buffer pool details via debugfs Mingming Cao
2026-11-09 23:55 ` [PATCH net-next v8 14/15] ibmveth: Implement incremental MQ RX queue resize Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-13 19:07 ` mingming cao
2026-11-09 23:55 ` [PATCH net-next v8 15/15] ibmveth: Complete set_channels down-path Mingming Cao
2026-11-12 23:57 ` netdev-bot+sashiko
2026-11-17 1:42 ` [PATCH net-next v8 00/15] ibmveth: Add multi-queue RX support Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116173604.50e83f10@kernel.org \
--to=kuba@kernel.org \
--cc=andrew+netdev@lunn.ch \
--cc=bjking1@linux.ibm.com \
--cc=chleroy@kernel.org \
--cc=davem@davemloft.net \
--cc=davemarq@linux.ibm.com \
--cc=edumazet@kernel.org \
--cc=haren@linux.ibm.com \
--cc=horms@kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=maddy@linux.ibm.com \
--cc=mmc@linux.ibm.com \
--cc=mpe@ellerman.id.au \
--cc=netdev@vger.kernel.org \
--cc=nnac123@linux.ibm.com \
--cc=npiggin@gmail.com \
--cc=pabeni@redhat.com \
--cc=ricklind@linux.ibm.com \
--cc=ritesh.list@gmail.com \
--cc=shaik.abdulla1@ibm.com \
--cc=sshegde@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox