Linux virtualization list
 help / color / mirror / Atom feed
From: "Arnd Bergmann" <arnd@arndb.de>
To: "Peng Yang" <peng.yang@oss.qualcomm.com>,
	"Amit Shah" <amit@kernel.org>,
	"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>
Cc: kernel@quicinc.com, virtualization@lists.linux.dev,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] virtio_console: add timeout to __send_to_port() spin loop
Date: Tue, 21 Apr 2026 09:38:30 +0200	[thread overview]
Message-ID: <37b513d6-770b-4442-b3b3-93b2801f3c71@app.fastmail.com> (raw)
In-Reply-To: <20260420-add_timeout_to___send_to_port-v1-1-6c32d33bf4f2@oss.qualcomm.com>

On Tue, Apr 21, 2026, at 08:57, Peng Yang wrote:
> __send_to_port() busy-waits in virtqueue_get_buf() while holding
> outvq_lock with IRQs disabled. If the host stops draining the TX
> virtqueue, this loop never terminates.
>
> This was observed during secondary VM boot: virtio_mem plugged memory
> in multiple iterations, each emitting dev_info() messages through the
> hvc console. A writev() on the hvc TTY entered __send_to_port() and
> stalled in the spin loop. When the watchdog bark ISR fired on another
> CPU, it attempted printk(), which tried to acquire outvq_lock through
> the same path and spun indefinitely. With all CPUs stuck, the watchdog
> could not be serviced and triggered a bite.
>
> Add a 200 ms deadline using ktime_get_mono_fast_ns() to bound the spin
> loop. ktime_get_mono_fast_ns() reads the hardware counter directly and
> is safe to call with IRQs disabled and spinlocks held.
>
> The 200 ms value is chosen to be far above normal host response latency
> (microseconds) to avoid spurious exits, yet well below the watchdog
> bark-to-bite window (typically 3 s) so that CPUs can escape the loop
> and complete the bark handler before a bite occurs.

Which host implementation do you use? The way the virtio_console
driver works really assumes that virtqueue_kick() consumes the
buffer synchronously. Even though that is not how virtio is
specified, this does tend to work. ;-)

> @@ -632,10 +634,18 @@ static ssize_t __send_to_port(struct port *port, struct scatterlist *sg,
>  	 * buffer and relax the spinning requirement.  The downside is
> 	 * we need to kmalloc a GFP_ATOMIC buffer each time the
> 	 * console driver writes something out.
> +	 *
> +	 * To avoid spinning forever if the host stops processing the
> +	 * TX virtqueue (e.g. during VM shutdown), a 200ms deadline is
> +	 * used to break out of the loop as a fallback.
 	 */

Did you by any chance mean to use microseconds instead of milliseconds?
Waiting this long with interrupts disabled likely breaks a lot
of assumptions, e.g. in the scheduler. If you have to deal with
a hypervisor that does not handle the console output synchronously,
the alternative suggested in the existing comment would likely
be more appropriate.

      Arnd

  reply	other threads:[~2026-04-21  7:39 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-21  6:57 [PATCH] virtio_console: add timeout to __send_to_port() spin loop Peng Yang
2026-04-21  7:38 ` Arnd Bergmann [this message]
2026-04-21  9:18   ` Peng Yang
2026-04-21 12:14     ` Arnd Bergmann
2026-04-21 14:11       ` Peng Yang
2026-04-21 14:23       ` Peng Yang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=37b513d6-770b-4442-b3b3-93b2801f3c71@app.fastmail.com \
    --to=arnd@arndb.de \
    --cc=amit@kernel.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=kernel@quicinc.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peng.yang@oss.qualcomm.com \
    --cc=virtualization@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox