All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Nicholas Piggin" <npiggin@gmail.com>
To: "Marc-André Lureau" <marcandre.lureau@redhat.com>
Cc: qemu-devel@nongnu.org, "Paolo Bonzini" <pbonzini@redhat.com>,
	"Pavel Dovgalyuk" <pavel.dovgaluk@ispras.ru>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Ani Sinha" <anisinha@redhat.com>,
	"Daniel P . Berrangé" <berrange@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>, "John Snow" <jsnow@redhat.com>,
	qemu-ppc@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>
Subject: Re: [PATCH 2/3] chardev: report blocked write to chardev backend
Date: Mon, 20 Nov 2023 23:35:50 +1000	[thread overview]
Message-ID: <CX3OGGIB0IAN.1CDYNM9U7M5Y3@wheely> (raw)
In-Reply-To: <CAMxuvawXTrQ3Mu-aGbELnQyBRU4W9kuMQo-XM_zm4FbRymHkqA@mail.gmail.com>

On Mon Nov 20, 2023 at 10:06 PM AEST, Marc-André Lureau wrote:
> Hi
>
> On Thu, Nov 16, 2023 at 3:54 PM Nicholas Piggin <npiggin@gmail.com> wrote:
> >
> > If a chardev socket is not read, it will eventually fill and QEMU
> > can block attempting to write to it. A difficult bug in avocado
> > tests where the console socket was not being read from caused this
> > hang.
> >
> > warn if a chardev write is blocked for 100ms.
> >
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> > This is not necessary for the fix but it does trigger in the
> > failing avocado test without the previous patch applied. Maybe
> > it would be helpful?
> >
> > Thanks,
> > Nick
> >
> >  chardev/char.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/chardev/char.c b/chardev/char.c
> > index 996a024c7a..7c375e3cc4 100644
> > --- a/chardev/char.c
> > +++ b/chardev/char.c
> > @@ -114,6 +114,8 @@ static int qemu_chr_write_buffer(Chardev *s,
> >  {
> >      ChardevClass *cc = CHARDEV_GET_CLASS(s);
> >      int res = 0;
> > +    int nr_retries = 0;
> > +
> >      *offset = 0;
> >
> >      qemu_mutex_lock(&s->chr_write_lock);
> > @@ -126,6 +128,10 @@ static int qemu_chr_write_buffer(Chardev *s,
> >              } else {
> >                  g_usleep(100);
> >              }
> > +            if (++nr_retries == 1000) { /* 100ms */
> > +                warn_report("Chardev '%s' write blocked for > 100ms, "
> > +                            "socket buffer full?", s->label);
> > +            }
>
> That shouldn't happen, the frontend should poll and only write when it
> can. What is the qemu command being used here?

You can follow it through the thread here

https://lore.kernel.org/qemu-devel/ZVT-bY9YOr69QTPX@redhat.com/

In short, a console device is attached to a socket pair and nothing
ever reads from it. It eventually fills, and writing to it fails
indefinitely here.

It can be reproduced with:

make check-avocado
AVOCADO_TESTS=tests/avocado/reverse_debugging.py:test_ppc64_pseries


> I think this change can be worth for debugging though.
>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Thanks,
Nick


  reply	other threads:[~2023-11-20 13:36 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-16 11:53 [PATCH 1/3] tests/avocado: reverse_debugging drain console to prevent hang Nicholas Piggin
2023-11-16 11:53 ` [PATCH 2/3] chardev: report blocked write to chardev backend Nicholas Piggin
2023-11-20 12:06   ` Marc-André Lureau
2023-11-20 13:35     ` Nicholas Piggin [this message]
2023-11-21  9:39       ` Marc-André Lureau
2023-11-21  9:42         ` Daniel P. Berrangé
2023-11-21  9:44         ` Thomas Huth
2023-11-21 11:47           ` Marc-André Lureau
2023-11-22  9:55             ` Alex Bennée
2023-11-22 10:38             ` Thomas Huth
2023-11-22 10:42               ` Daniel P. Berrangé
2023-11-16 11:53 ` [PATCH 3/3] tests/avocado: Enable reverse_debugging.py tests in gitlab CI Nicholas Piggin
2023-11-16 12:33   ` Thomas Huth
2023-11-16 18:11   ` Thomas Huth
2023-11-17  7:35     ` Nicholas Piggin
2023-11-21  8:56       ` Thomas Huth
2023-11-21  9:14         ` Daniel P. Berrangé
2023-11-21  9:40           ` Thomas Huth
2023-11-16 13:26 ` [PATCH 1/3] tests/avocado: reverse_debugging drain console to prevent hang Ani Sinha
2023-11-16 13:31   ` Ani Sinha
2023-11-16 13:39     ` Ani Sinha

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CX3OGGIB0IAN.1CDYNM9U7M5Y3@wheely \
    --to=npiggin@gmail.com \
    --cc=alex.bennee@linaro.org \
    --cc=anisinha@redhat.com \
    --cc=berrange@redhat.com \
    --cc=bleal@redhat.com \
    --cc=crosa@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=marcandre.lureau@redhat.com \
    --cc=pavel.dovgaluk@ispras.ru \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.