From: "Arnd Bergmann" <arnd@arndb.de>
To: "Peng Yang" <peng.yang@oss.qualcomm.com>,
"Amit Shah" <amit@kernel.org>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>
Cc: kernel@quicinc.com, virtualization@lists.linux.dev,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH] virtio_console: add timeout to __send_to_port() spin loop
Date: Tue, 21 Apr 2026 09:38:30 +0200 [thread overview]
Message-ID: <37b513d6-770b-4442-b3b3-93b2801f3c71@app.fastmail.com> (raw)
In-Reply-To: <20260420-add_timeout_to___send_to_port-v1-1-6c32d33bf4f2@oss.qualcomm.com>
On Tue, Apr 21, 2026, at 08:57, Peng Yang wrote:
> __send_to_port() busy-waits in virtqueue_get_buf() while holding
> outvq_lock with IRQs disabled. If the host stops draining the TX
> virtqueue, this loop never terminates.
>
> This was observed during secondary VM boot: virtio_mem plugged memory
> in multiple iterations, each emitting dev_info() messages through the
> hvc console. A writev() on the hvc TTY entered __send_to_port() and
> stalled in the spin loop. When the watchdog bark ISR fired on another
> CPU, it attempted printk(), which tried to acquire outvq_lock through
> the same path and spun indefinitely. With all CPUs stuck, the watchdog
> could not be serviced and triggered a bite.
>
> Add a 200 ms deadline using ktime_get_mono_fast_ns() to bound the spin
> loop. ktime_get_mono_fast_ns() reads the hardware counter directly and
> is safe to call with IRQs disabled and spinlocks held.
>
> The 200 ms value is chosen to be far above normal host response latency
> (microseconds) to avoid spurious exits, yet well below the watchdog
> bark-to-bite window (typically 3 s) so that CPUs can escape the loop
> and complete the bark handler before a bite occurs.
Which host implementation do you use? The way the virtio_console
driver works really assumes that virtqueue_kick() consumes the
buffer synchronously. Even though that is not how virtio is
specified, this does tend to work. ;-)
> @@ -632,10 +634,18 @@ static ssize_t __send_to_port(struct port *port, struct scatterlist *sg,
> * buffer and relax the spinning requirement. The downside is
> * we need to kmalloc a GFP_ATOMIC buffer each time the
> * console driver writes something out.
> + *
> + * To avoid spinning forever if the host stops processing the
> + * TX virtqueue (e.g. during VM shutdown), a 200ms deadline is
> + * used to break out of the loop as a fallback.
*/
Did you by any chance mean to use microseconds instead of milliseconds?
Waiting this long with interrupts disabled likely breaks a lot
of assumptions, e.g. in the scheduler. If you have to deal with
a hypervisor that does not handle the console output synchronously,
the alternative suggested in the existing comment would likely
be more appropriate.
Arnd
next prev parent reply other threads:[~2026-04-21 7:39 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-21 6:57 [PATCH] virtio_console: add timeout to __send_to_port() spin loop Peng Yang
2026-04-21 7:38 ` Arnd Bergmann [this message]
2026-04-21 9:18 ` Peng Yang
2026-04-21 12:14 ` Arnd Bergmann
2026-04-21 14:11 ` Peng Yang
2026-04-21 14:23 ` Peng Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=37b513d6-770b-4442-b3b3-93b2801f3c71@app.fastmail.com \
--to=arnd@arndb.de \
--cc=amit@kernel.org \
--cc=gregkh@linuxfoundation.org \
--cc=kernel@quicinc.com \
--cc=linux-kernel@vger.kernel.org \
--cc=peng.yang@oss.qualcomm.com \
--cc=virtualization@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox