qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/2] Replace custom test harness with "meson test"
@ 2021-10-28 14:28 Paolo Bonzini
  2021-10-28 14:28 ` [PATCH 1/2] meson: bump submodule to 0.59.3 Paolo Bonzini
  2021-10-28 14:28 ` [PATCH 2/2] build: use "meson test" as the test harness Paolo Bonzini
  0 siblings, 2 replies; 6+ messages in thread
From: Paolo Bonzini @ 2021-10-28 14:28 UTC (permalink / raw)
  To: qemu-devel

Hi all,

Starting with Meson 0.57, "meson test" has all features of QEMU's
makefile-based harness and more.  In particular, some features that
were added to reach feature parity are:

* print reproducer command line for each failed test right below the test

* keep the output of multiple (non-TAP) tests together in verbose mode,
  similar to "make --output-sync target"

* report on TAP subtests as they are encountered

It also includes nicer handling of test interruption, logging of the run
in the meson-logs/ subdirectory, and a progress report/spinner.  For
these reasons it would be nice to adopt it and remove the Perl scripts
that we have to present TAP output nicely.  It is also possible to
change qemu-iotests to TAP and describe it in tests/qemu-iotests/meson.build,
but this is left for later.

For people using "make check" there should be little or no change.
Otherwise:

* if you want to try "meson test", note that there could be timeouts on
  slower systems (and in particular OpenBSD is insanely slow on some
  tests).  Also if you use "meson test", note that CTRL+C will only
  interrupt the longest running test.  Pressing CTRL+C repeatedly three
  times (which you would likely do anyway, that's how humans work)
  interrupts the whole run.  This behavior does not apply to "make check".

* If you were using make check-report.tap and similar, they are replaced
  by targets like make check-report.junit.xml.  This is because Gitlab
  is able to parse the resulting XML and include on the website a report
  of which tests failed.

An example of non-verbose "make check" output is available at
https://gitlab.com/bonzini/qemu/-/jobs/1680980620.  A verbose run
instead is like https://asciinema.org/a/e5irnEszSnAheOHM30exbo3F6
(does not include check-block).

Paolo

RFC->v1:
- include submodule bump to 0.59.3
- do not include check-block part

Paolo Bonzini (2):
  meson: bump submodule to 0.59.3
  build: use "meson test" as the test harness

 Makefile                      |   3 +-
 configure                     |   2 +-
 meson                         |   2 +-
 meson.build                   |   5 +-
 scripts/meson-buildoptions.py |  16 --
 scripts/mtest2make.py         | 104 ++++--------
 scripts/tap-driver.pl         | 379 ------------------------------------------
 scripts/tap-merge.pl          | 111 -------------
 tests/fp/meson.build          |   2 +-
 9 files changed, 43 insertions(+), 581 deletions(-)
-- 
2.31.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-10-31 21:06 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-10-28 14:28 [PATCH 0/2] Replace custom test harness with "meson test" Paolo Bonzini
2021-10-28 14:28 ` [PATCH 1/2] meson: bump submodule to 0.59.3 Paolo Bonzini
2021-10-28 16:05   ` Thomas Huth
2021-10-28 14:28 ` [PATCH 2/2] build: use "meson test" as the test harness Paolo Bonzini
2021-10-28 18:03   ` Thomas Huth
2021-10-31 21:04     ` Paolo Bonzini

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).