Re: [PATCH net-next] net: mana: Add CQE polling as first-level TX timeout recovery

[Breno Leitao <leitao@debian.org>]

On Fri, Nov 13, 2026 at 02:04:08PM +0530, Sahil Chandna wrote:
> On 12-11-2026 17:37, Breno Leitao wrote:
> > Hello Sahil,
> > 
> > On Thu, Nov 12, 2026 at 04:57:18AM -0600, Sahil Chandna wrote:
> >> Implement a graduated recovery approach: on the first TX timeout (or
> >> when the previous timeout was more than 2 minutes ago), schedule NAPI on
> >> the stalled queue's TX CQ via napi_schedule(). This triggers
> >> mana_poll_tx_cq() through the normal NAPI path, processing any pending
> >> completions without relying on the interrupt and recovering
> >> missed-interrupt scenarios without queue reset.
> > 
> > Which interrupt is being missed here?
> > 
> Hi Breno,
> 
> It is the TX CQ completion interrupt. After NAPI finishes, it re-arms
> the TX CQ. When a new CQE arrives, the hardware post EQE, which raises
> the MSI-X and schedules NAPI. If that notification for the TX CQ is lost
> , NAPI never runs for the TX CQ and pending_sends never drains and the
> queue stays stopped,  so napi_schedule() polls the CQ, reaps the CQEs
> and re-arms it.

Thanks.  Which one gets lost, the EQE or the MSI-X?

> >> If any subsequent TX timeout fires on any queue, within 2 minutes of
> >> the previous one, escalate to the existing port-level detach/attach
> >> reset, as the CQE poll did not recover the queue.
> > 
> > How was 2 minutes chosen? mana sets watchdog_timeo to 15 * HZ, no?
> The watchdog_timeo is TX stall timeout setting i.e if a queue remains
> stalled for this much time, it trigger watchdog timeout. Here, 2 minute
> is not timeout/wait. The 2 minutes here is for deciding whether a new
> queue stall event is a repeat of the previous incident or a new,
> unrelated one that should get the cheap poll first. For e.g. if a queue
> gets stalled, 15 * HZ sec later, timeout occurs and this patch trigger
> manual NAPI, if another stall occurs (it can be on any queue) within 2
> minutes of last one, the existing recovery mechanism of mana
> detach/attach will be tried.The patch is adding a first level less
> destructive recovery first.
> For selecting 2 minutes, I do not have any hardware basis for it afaik,
> but I can confirm and check with our host team as well.

So the goal is to tell an isolated stall from a wider problem.

A port-wide timestamp only approximates that. It cannot tell the same
queue from another one, and since dev_watchdog() reports one queue per
run, two queues that stall together show up ~15s apart: the first gets
a kick, the second gets the port reset anyway, no?

Could mana_tx_timeout() check this directly? Look at the other TX queues
the way dev_watchdog() does (stopped, with trans_start older than
watchdog_timeo). If more than one is stalled, reset right away.

Otherwise kick this queue, and reset if the same queue times out again
within a couple of watchdog periods, as gve_tx_timeout_try_q_kick()
does. That also gets rid of the 2 minutes.