From: Jesper Dangaard Brouer <hawk@kernel.org>
To: netdev@vger.kernel.org, "Toke Høiland-Jørgensen" <toke@toke.dk>
Cc: Jesper Dangaard Brouer <hawk@kernel.org>,
Eric Dumazet <eric.dumazet@gmail.com>,
"David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
ihor.solodrai@linux.dev, "Michael S. Tsirkin" <mst@redhat.com>,
makita.toshiaki@lab.ntt.co.jp, toshiaki.makita1@gmail.com,
bpf@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org, kernel-team@cloudflare.com
Subject: [PATCH net V2 0/2] veth: Fix TXQ stall race condition and add recovery
Date: Mon, 27 Oct 2025 21:05:25 +0100 [thread overview]
Message-ID: <176159549627.5396.15971398227283515867.stgit@firesoul> (raw)
This patchset addresses a race condition introduced in commit dc82a33297fc
("veth: apply qdisc backpressure on full ptr_ring to reduce TX drops"). In
production, this has been observed to cause a permanently stalled transmit
queue (TXQ) on ARM64 (Ampere Altra Max) systems, leading to a "lost wakeup"
scenario where the TXQ remains in the QUEUE_STATE_DRV_XOFF state and traffic
halts.
The root cause, which is fixed in patch 2, is a racy use of the
__ptr_ring_empty() API from the producer side (veth_xmit). The producer
stops the queue and then checks the ptr_ring consumer's head, but this is
not guaranteed to be correct, when observed from the producer side,
when the NAPI consumer on another CPU has just finished consuming.
This series fixes the bug and make the driver more resilient to recover.
The patches are ordered to first add recovery mechanisms, then fix the
underlying race.
V2:
- Drop patch that changed up/down NDOs
- For race fix add a smb_rmb and improve commit message reasoning for race cases
V1: https://lore.kernel.org/all/176123150256.2281302.7000617032469740443.stgit@firesoul/
---
Jesper Dangaard Brouer (2):
veth: enable dev_watchdog for detecting stalled TXQs
veth: more robust handing of race to avoid txq getting stuck
drivers/net/veth.c | 53 +++++++++++++++++++++++++++++-----------------
1 file changed, 34 insertions(+), 19 deletions(-)
--
next reply other threads:[~2025-10-27 20:05 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-27 20:05 Jesper Dangaard Brouer [this message]
2025-10-27 20:05 ` [PATCH net V2 1/2] veth: enable dev_watchdog for detecting stalled TXQs Jesper Dangaard Brouer
2025-10-28 9:10 ` Toke Høiland-Jørgensen
2025-10-27 20:05 ` [PATCH net V2 2/2] veth: more robust handing of race to avoid txq getting stuck Jesper Dangaard Brouer
2025-10-28 9:10 ` Toke Høiland-Jørgensen
2025-10-28 14:56 ` Toshiaki Makita
2025-10-29 10:33 ` Jesper Dangaard Brouer
2025-10-29 15:00 ` Toshiaki Makita
2025-10-30 19:06 ` Jesper Dangaard Brouer
2025-11-03 8:41 ` Toshiaki Makita
2025-10-30 12:28 ` Paolo Abeni
2025-11-05 15:54 ` Jesper Dangaard Brouer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=176159549627.5396.15971398227283515867.stgit@firesoul \
--to=hawk@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=davem@davemloft.net \
--cc=eric.dumazet@gmail.com \
--cc=ihor.solodrai@linux.dev \
--cc=kernel-team@cloudflare.com \
--cc=kuba@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=makita.toshiaki@lab.ntt.co.jp \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=toke@toke.dk \
--cc=toshiaki.makita1@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox