qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Stefan Hajnoczi" <stefanha@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Daniel P. Berrangé" <berrange@redhat.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	"Jason Wang" <jasowang@redhat.com>,
	"Laurent Vivier" <lvivier@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Marc-André Lureau" <marcandre.lureau@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>
Subject: [PATCH 0/6] net: fix non-deterministic failures of the 'netdev-socket' qtest
Date: Thu,  4 Jan 2024 16:29:36 +0000	[thread overview]
Message-ID: <20240104162942.211458-1-berrange@redhat.com> (raw)

We've previously bumped up the timeouts in the netdev-socket qtest
to supposedly fix non-deterministic failures, however, the failures
are still hitting CI.

A simple 'listen()' and 'connect()' pairing across 2 QEMU processes
should be very quick to execute, even under high system load, so it
was never likely that the test was failing due to timeouts being
reached.

The actual root cause was a race condition in the test design. It
was spawning a QEMU with a 'server' netdev, and then spawning one
with the 'client' netdev. There was insufficient synchronization,
however, so it was possible for the 2nd QEMU process to attempt
to 'connect()' before the 'listen()' call was made by the 1st QEMU.

In the test scenarios that did not use the 'reconnect' flag, this
would result in the client QEMU never getting into the expected
state. The test code would thus loop on 'info network' until
hitting the maximum wait time.

This series reverts the increased timeouts, and fixes synchronization
in the test scenarios. It also improves reporting of errors in the
socket netdev backend so that 'info network' reports what actually
went wrong rather than a useless generic 'connection error' string.
This will help us diagnose any future CI problems, should they occurr.

Daniel P. Berrangé (6):
  Revert "netdev: set timeout depending on loadavg"
  Revert "osdep: add getloadavg"
  Revert "tests/qtest/netdev-socket: Raise connection timeout to 120
    seconds"
  net: add explicit info about connecting/listening state
  net: handle QIOTask completion to report useful error message
  qtest: ensure netdev-socket tests have non-overlapping names

 include/qemu/osdep.h        | 10 ---------
 meson.build                 |  1 -
 net/stream.c                | 18 +++++++++++-----
 tests/qtest/netdev-socket.c | 42 +++++++------------------------------
 4 files changed, 21 insertions(+), 50 deletions(-)

-- 
2.43.0



             reply	other threads:[~2024-01-04 16:30 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-04 16:29 Daniel P. Berrangé [this message]
2024-01-04 16:29 ` [PATCH 1/6] Revert "netdev: set timeout depending on loadavg" Daniel P. Berrangé
2024-01-04 16:29 ` [PATCH 2/6] Revert "osdep: add getloadavg" Daniel P. Berrangé
2024-01-04 16:29 ` [PATCH 3/6] Revert "tests/qtest/netdev-socket: Raise connection timeout to 120 seconds" Daniel P. Berrangé
2024-01-04 16:29 ` [PATCH 4/6] net: add explicit info about connecting/listening state Daniel P. Berrangé
2024-01-04 16:29 ` [PATCH 5/6] net: handle QIOTask completion to report useful error message Daniel P. Berrangé
2024-01-04 16:29 ` [PATCH 6/6] qtest: ensure netdev-socket tests have non-overlapping names Daniel P. Berrangé
2024-01-04 17:47   ` Philippe Mathieu-Daudé
2024-01-04 16:45 ` [PATCH 0/6] net: fix non-deterministic failures of the 'netdev-socket' qtest Stefan Hajnoczi
2024-01-15  2:36   ` Jason Wang
2024-01-15 10:19     ` Peter Maydell
2024-01-16  1:05       ` Jason Wang
2024-01-09 13:49 ` Daniel P. Berrangé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240104162942.211458-1-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=jasowang@redhat.com \
    --cc=lvivier@redhat.com \
    --cc=marcandre.lureau@redhat.com \
    --cc=mst@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@redhat.com \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).