From: "Michael S. Tsirkin" <mst@redhat.com>
To: Congjie Zhou <zcjie0802@qq.com>
Cc: netdev <netdev@vger.kernel.org>,
jasowangio <jasowangio@gmail.com>, eperezma <eperezma@redhat.com>,
xuanzhuo <xuanzhuo@linux.alibaba.com>,
andrew+netdev <andrew+netdev@lunn.ch>,
davem <davem@davemloft.net>, edumazet <edumazet@kernel.org>,
kuba <kuba@kernel.org>, pabeni <pabeni@redhat.com>,
horms <horms@kernel.org>,
virtualization <virtualization@lists.linux.dev>,
linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [BUG] virtio_net: RX stall caused by masking callbacks after NAPI ownership release
Date: Thu, 19 Nov 2026 06:50:21 -0500 [thread overview]
Message-ID: <20261119064843-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <tencent_D0650F8CA2AAE4C258BFE5BCE805CDB4CA0A@qq.com>
On Tue, Nov 17, 2026 at 12:05:24AM +0800, Congjie Zhou wrote:
> > Maybe let's just check the poll list.
>
> Checking the poll list fixes the earlier race where the watchdog has
> already finished. However, the timer can still be consumed without
> triggering another poll if it fires after the old caller's check:
>
> old poll / watchdog successor busy-poll
> ------------------- -------------------
> napi_complete_done():
> clear SCHED, arm timer,
> return false
> acquire NAPI
> enable callbacks,
> then pause before completion
> disable callbacks
> napi_has_pending_poll():
> timer is still queued,
> so skip napi_schedule()
> old poll exits
> watchdog expires, sees SCHED=1:
> no scheduling, no MISSED
> watchdog returns
> napi_complete_done(..., 0):
> clear SCHED, return true
> queue is empty, exit
>
> At that point, the next used entry ends up with neither notifications
> enabled nor a pending timer or scheduled poll to process it.
>
> Stepping back, the fundamental ordering issue here is:
>
> old releases SCHED
> -> successor rearms callbacks
> -> old masks callbacks
>
> Right now, nothing guarantees that the old mask happens before the
> successor's rearm. Even if the successor still holds SCHED, it might
> have already done its final rearm. Similarly, having a queued timer
> doesn't guarantee another poll—the watchdog will simply bail out without
> scheduling because SCHED is already set. So all these state checks can
> look completely valid at the time, yet still fail to guarantee recovery.
>
> The root cause is that with the timer introduced by NAPI defer,
> napi_complete_done() can return false while still giving up SCHED
> ownership, leaving us with nothing else to synchronize the callback
> state. Before NAPI defer, returning false meant we still held onto
> SCHED, so everything worked as expected.
>
> To fix this properly, I think we either need to call
> virtqueue_disable_cb() before giving up SCHED ownership, or introduce a
> separate synchronization mechanism (like a lock) to ensure we can always
> recover NAPI execution. Simple flag checks alone don't seem enough to
> close this race completely.
Right. So I came up with this then. Pls note i could not repro the
problem so I am merely reasoning based on theoretical understanding
of it.
--->
virtio_net: mask callbacks before handing off NAPI ownership
Virtio is special in that enabling or disabling callbacks (interrupts)
accesses shared virtqueue state and must be serialized with polling. It
therefore enables callbacks before completing NAPI, and disables them
again if completion returns false.
With napi_complete_done(), that false return can come after ownership has
passed to another poll. The first poll can then disable callbacks after
its successor has enabled them, leaving RX stalled. Virtio therefore
needs to disable callbacks before handing off NAPI ownership.
Add a prepare/finish NAPI completion API so virtio-net can mask interrupts
before handing ownership off.
This preserves interrupt suppression without a callback or an additional
data-path barrier. Ordinary napi_complete_done() compiles identically in
the tested configuration.
Tested on x86-64 KVM with lock debugging, including IRQ deferral, concurrent
scheduling, busy polling, netpoll, and threaded NAPI. Split and packed
virtqueue TCP RX tests with IRQ deferral and busy polling also passed.
Reported-by: Congjie Zhou <zcjie0802@qq.com>
Closes: https://lore.kernel.org/netdev/tencent_D0650F8CA2AAE4C258BFE5BCE805CDB4CA0A@qq.com/
Fixes: 6f8b12d661d0 ("net: napi: add hard irqs deferral feature")
Assisted-by: LLM
---
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index dd2c41ef6fbe7..de370a39bc8fa 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -730,16 +730,18 @@ static void virtqueue_napi_schedule(struct napi_struct *napi,
static bool virtqueue_napi_complete(struct napi_struct *napi,
struct virtqueue *vq, int processed)
{
+ struct napi_completion completion;
int opaque;
opaque = virtqueue_enable_cb_prepare(vq);
- if (napi_complete_done(napi, processed)) {
+ if (napi_complete_done_prepare(napi, processed, &completion)) {
if (unlikely(virtqueue_poll(vq, opaque)))
virtqueue_napi_schedule(napi, vq);
else
return true;
} else {
virtqueue_disable_cb(vq);
+ napi_complete_done_finish(napi, &completion);
}
return false;
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index c0ae442c4344b..a182882624d6a 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -599,6 +599,15 @@ static inline void napi_schedule_irqoff(struct napi_struct *n)
*/
bool napi_complete_done(struct napi_struct *n, int work_done);
+struct napi_completion {
+ unsigned long timeout;
+};
+
+bool napi_complete_done_prepare(struct napi_struct *n, int work_done,
+ struct napi_completion *completion);
+void napi_complete_done_finish(struct napi_struct *n,
+ struct napi_completion *completion);
+
static inline bool napi_complete(struct napi_struct *n)
{
return napi_complete_done(n, 0);
diff --git a/net/core/dev.c b/net/core/dev.c
index 3ac700d37e596..fa9baa70e9d72 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -6999,9 +6999,17 @@ void __napi_schedule_irqoff(struct napi_struct *n)
}
EXPORT_SYMBOL(__napi_schedule_irqoff);
-bool napi_complete_done(struct napi_struct *n, int work_done)
+static __always_inline bool
+__napi_complete_done_finish(struct napi_struct *n, unsigned long timeout,
+ bool ret, bool allow_pending,
+ struct napi_completion *completion);
+
+static __always_inline bool
+__napi_complete_done_prepare(struct napi_struct *n, int work_done,
+ bool allow_pending,
+ struct napi_completion *completion)
{
- unsigned long flags, val, new, timeout = 0;
+ unsigned long flags, timeout = 0;
bool ret = true;
/*
@@ -7041,10 +7049,25 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
}
WRITE_ONCE(n->list_owner, -1);
- val = READ_ONCE(n->state);
+ return __napi_complete_done_finish(n, timeout, ret, allow_pending,
+ completion);
+}
+
+static __always_inline bool
+__napi_complete_done_finish(struct napi_struct *n, unsigned long timeout,
+ bool ret, bool allow_pending,
+ struct napi_completion *completion)
+{
+ unsigned long val = READ_ONCE(n->state), new;
+
do {
WARN_ON_ONCE(!(val & NAPIF_STATE_SCHED));
+ if (allow_pending && (!ret || (val & NAPIF_STATE_MISSED))) {
+ completion->timeout = timeout;
+ return false;
+ }
+
new = val & ~(NAPIF_STATE_MISSED | NAPIF_STATE_SCHED |
NAPIF_STATE_SCHED_THREADED |
NAPIF_STATE_PREFER_BUSY_POLL);
@@ -7067,8 +7090,48 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
HRTIMER_MODE_REL_PINNED);
return ret;
}
+
+bool napi_complete_done(struct napi_struct *n, int work_done)
+{
+ return __napi_complete_done_prepare(n, work_done, false, NULL);
+}
EXPORT_SYMBOL(napi_complete_done);
+/**
+ * napi_complete_done_prepare - prepare NAPI completion
+ * @n: NAPI context
+ * @work_done: number of packets processed
+ * @completion: opaque completion token
+ *
+ * Return: true if NAPI is complete and interrupts may remain enabled;
+ * false if ownership is retained. On false, mask interrupts and call
+ * napi_complete_done_finish().
+ */
+bool napi_complete_done_prepare(struct napi_struct *n, int work_done,
+ struct napi_completion *completion)
+{
+ return __napi_complete_done_prepare(n, work_done, true, completion);
+}
+EXPORT_SYMBOL_GPL(napi_complete_done_prepare);
+
+/**
+ * napi_complete_done_finish - finish NAPI completion after masking interrupts
+ * @n: NAPI context
+ * @completion: token passed to napi_complete_done_prepare()
+ *
+ * The caller must provide any device-specific interrupt ordering.
+ */
+void napi_complete_done_finish(struct napi_struct *n,
+ struct napi_completion *completion)
+{
+ if (unlikely(n->state & (NAPIF_STATE_NPSVC |
+ NAPIF_STATE_IN_BUSY_POLL)))
+ return;
+
+ __napi_complete_done_finish(n, completion->timeout, false, false, NULL);
+}
+EXPORT_SYMBOL_GPL(napi_complete_done_finish);
+
static void __skb_defer_free_flush(struct skb_defer_node *sdn, int budget)
{
struct llist_node *free_list;
prev parent reply other threads:[~2026-11-19 11:50 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 13:36 [BUG] virtio_net: RX stall caused by masking callbacks after NAPI ownership release Congjie Zhou
2026-11-11 13:50 ` [PATCH net 1/2] net: napi: add completion with an ordered IRQ mask callback Congjie Zhou
2026-11-11 13:56 ` netdev-bot+sinfo
2026-11-11 15:08 ` zcj
2026-11-12 13:52 ` sashiko-bot
2026-11-11 13:50 ` [PATCH net 2/2] virtio_net: mask callbacks before releasing NAPI ownership Congjie Zhou
2026-11-12 13:52 ` sashiko-bot
2026-11-13 10:38 ` zcj
2026-11-15 7:12 ` [PATCH net v2 1/2] net: napi: add completion with an ordered IRQ mask callback Congjie Zhou
2026-11-15 7:21 ` sashiko-bot
2026-11-15 18:26 ` Jakub Kicinski
2026-11-15 7:12 ` [PATCH net v2 2/2] virtio_net: mask callbacks before releasing NAPI ownership Congjie Zhou
2026-11-15 7:21 ` sashiko-bot
2026-11-15 20:09 ` [BUG] virtio_net: RX stall caused by masking callbacks after NAPI ownership release Michael S. Tsirkin
2026-11-15 23:07 ` Michael S. Tsirkin
2026-11-16 8:47 ` Congjie Zhou
2026-11-16 9:52 ` Michael S. Tsirkin
2026-11-16 16:05 ` Congjie Zhou
2026-11-19 11:50 ` Michael S. Tsirkin [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119064843-mutt-send-email-mst@kernel.org \
--to=mst@redhat.com \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=eperezma@redhat.com \
--cc=horms@kernel.org \
--cc=jasowangio@gmail.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=virtualization@lists.linux.dev \
--cc=xuanzhuo@linux.alibaba.com \
--cc=zcjie0802@qq.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox