From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Thomas Huth" <thuth@redhat.com>,
"Ani Sinha" <anisinha@redhat.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Peter Maydell" <peter.maydell@linaro.org>,
"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH 00/15] test/functional: improve functional test debugging & fix tuxrun
Date: Tue, 19 Nov 2024 15:05:04 +0000 [thread overview]
Message-ID: <20241119150519.1123365-1-berrange@redhat.com> (raw)
This started out as a series to get rid of the many GBs of temp
files the functional tests leave behind. Then it expanded into
improving the functional test debugging by ensuring we preserve
the QEMU stdout/stderr log file created by the QEMUMachine class.
In the course of doing that I encountered some other minor points
worth fixing, and then got side tracked into looking at the tuxrun
hangs with aarch64be. Investigating the latter exposed some further
holes in the debugging story prompting yet more patches, as well as
a final solution for tuxrun. So this series does:
* Purge all scratch files created by tests
* Preserve the stdout/stderr log file
* Capture debug log messages on QEMUMachine
* Provide a QMP backdoor for debugging stuck QEMUs
* Enhance console handling for partial line matches
* Fix the tuxrun tests by eliminating sleeps
There's quite alot of code here, but at the same time it feels like
the kind of stuff that'll be valuable either in the 9.2 release, or
in the soon to exist 9.2 stable branch.
NB, with this series applied Thomas' tuxrun conversion to functional
testing survives 200 iterations on my machine, whereas it would
reliably hang in < 20, and often in < 10, before.
Daniel P. Berrangé (15):
tests/functional: fix mips64el test to honour workdir
tests/functional: automatically clean up scratch files after tests
tests/functional: remove "AVOCADO" from env variable name
tests/functional: remove todo wrt avocado.utils.wait_for
tests/functional: remove leftover :avocado: tags
tests/functional: remove obsolete reference to avocado bug
tests/functional: remove comments talking about avocado
tests/functional: honour self.workdir in ACPI bits tests
tests/functional: put QEMUMachine logs in testcase log directory
tests/functional: honour requested test VM name in QEMUMachine
tests/functional: enable debug logging for QEMUMachine
tests/functional: logs details of console interaction operations
tests/functional: rewrite console handling to be bytewise
tests/functional: remove time.sleep usage from tuxrun tests
tests/functional: add a QMP backdoor for debugging stalled tests
docs/devel/testing/functional.rst | 16 ++++++
tests/functional/qemu_test/cmd.py | 65 ++++++++++++++++++------
tests/functional/qemu_test/testcase.py | 33 +++++++++---
tests/functional/qemu_test/tuxruntest.py | 17 +++----
tests/functional/test_acpi_bits.py | 56 +++++++-------------
tests/functional/test_arm_bpim2u.py | 20 --------
tests/functional/test_arm_orangepi.py | 27 ----------
tests/functional/test_m68k_nextcube.py | 3 +-
tests/functional/test_mips64el_malta.py | 4 +-
9 files changed, 119 insertions(+), 122 deletions(-)
--
2.46.0
next reply other threads:[~2024-11-19 15:06 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-19 15:05 Daniel P. Berrangé [this message]
2024-11-19 15:05 ` [PATCH 01/15] tests/functional: fix mips64el test to honour workdir Daniel P. Berrangé
2024-11-19 15:35 ` Alex Bennée
2024-11-19 17:08 ` Philippe Mathieu-Daudé
2024-11-19 15:05 ` [PATCH 02/15] tests/functional: automatically clean up scratch files after tests Daniel P. Berrangé
2024-11-19 16:21 ` Alex Bennée
2024-11-19 17:28 ` Thomas Huth
2024-11-19 15:05 ` [PATCH 03/15] tests/functional: remove "AVOCADO" from env variable name Daniel P. Berrangé
2024-11-19 16:22 ` Alex Bennée
2024-11-19 17:09 ` Philippe Mathieu-Daudé
2024-11-19 15:05 ` [PATCH 04/15] tests/functional: remove todo wrt avocado.utils.wait_for Daniel P. Berrangé
2024-11-19 16:37 ` Alex Bennée
2024-11-19 15:05 ` [PATCH 05/15] tests/functional: remove leftover :avocado: tags Daniel P. Berrangé
2024-11-19 16:37 ` Alex Bennée
2024-11-19 15:05 ` [PATCH 06/15] tests/functional: remove obsolete reference to avocado bug Daniel P. Berrangé
2024-11-19 16:39 ` Alex Bennée
2024-11-19 15:05 ` [PATCH 07/15] tests/functional: remove comments talking about avocado Daniel P. Berrangé
2024-11-19 17:32 ` Thomas Huth
2024-11-19 15:05 ` [PATCH 08/15] tests/functional: honour self.workdir in ACPI bits tests Daniel P. Berrangé
2024-11-19 17:09 ` Alex Bennée
2024-11-19 15:05 ` [PATCH 09/15] tests/functional: put QEMUMachine logs in testcase log directory Daniel P. Berrangé
2024-11-19 17:10 ` Alex Bennée
2024-11-19 15:05 ` [PATCH 10/15] tests/functional: honour requested test VM name in QEMUMachine Daniel P. Berrangé
2024-11-19 17:40 ` Thomas Huth
2024-11-19 15:05 ` [PATCH 11/15] tests/functional: enable debug logging for QEMUMachine Daniel P. Berrangé
2024-11-21 6:52 ` Thomas Huth
2024-11-19 15:05 ` [PATCH 12/15] tests/functional: logs details of console interaction operations Daniel P. Berrangé
2024-11-21 6:58 ` Thomas Huth
2024-11-19 15:05 ` [PATCH 13/15] tests/functional: rewrite console handling to be bytewise Daniel P. Berrangé
2024-11-19 17:11 ` Paolo Bonzini
2024-11-19 18:54 ` Daniel P. Berrangé
2024-11-19 19:26 ` Paolo Bonzini
2024-11-19 15:05 ` [PATCH 14/15] tests/functional: remove time.sleep usage from tuxrun tests Daniel P. Berrangé
2024-11-19 15:05 ` [PATCH 15/15] tests/functional: add a QMP backdoor for debugging stalled tests Daniel P. Berrangé
2024-11-21 7:01 ` Thomas Huth
2024-11-19 17:59 ` [PATCH 00/15] test/functional: improve functional test debugging & fix tuxrun Cédric Le Goater
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241119150519.1123365-1-berrange@redhat.com \
--to=berrange@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=anisinha@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).