qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Kevin Wolf" <kwolf@redhat.com>,
	qemu-block@nongnu.org, "Thomas Huth" <thuth@redhat.com>,
	"Hanna Reitz" <hreitz@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH v2 7/8] iotests: register each I/O test separately with meson
Date: Fri,  3 Mar 2023 16:07:26 +0000	[thread overview]
Message-ID: <20230303160727.3977246-8-berrange@redhat.com> (raw)
In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com>

Currently meson registers a single test that invokes an entire group of
I/O tests, hiding the test granularity from meson. There are various
downsides of doing this

 * You cannot ask 'meson test' to invoke a single I/O test
 * The meson test timeout can't be applied to the individual
   tests
 * Meson only gets a pass/fail for the overall I/O test group
   not individual tests
 * If a CI job gets killed by the GitLab timeout, we don't
   get visibility into how far through the I/O tests
   execution got.

This switches meson to perform test discovery by invoking 'check' in
dry-run mode. It then registers one meson test case for each I/O
test. Parallel execution remains disabled since the I/O tests do not
use self contained execution environments and thus conflict with
each other.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/meson.build | 35 ++++++++++++++++++++++++++++------
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 323a4acb6a..a162f683ef 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -32,16 +32,39 @@ foreach k, v : emulators
   endif
 endforeach
 
+qemu_iotests_check_cmd = files('check')
+
 foreach format, speed: qemu_iotests_formats
   if speed == 'quick'
     suites = 'block'
   else
     suites = ['block-' + speed, speed]
   endif
-  test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format],
-       depends: qemu_iotests_binaries, env: qemu_iotests_env,
-       protocol: 'tap',
-       suite: suites,
-       timeout: 0,
-       is_parallel: false)
+
+  args = ['-tap', '-' + format]
+  if speed == 'quick'
+      args += ['-g', 'auto']
+  endif
+
+  rc = run_command(
+      [qemu_iotests_check_cmd] + args + ['-n'],
+      check: true,
+  )
+
+  foreach item: rc.stdout().strip().split()
+      args = ['-tap', '-' + format, item,
+              '--source-dir', meson.current_source_dir(),
+              '--build-dir', meson.current_build_dir()]
+      # Some individual tests take as long as 45 seconds
+      # Bump the timeout to 3 minutes for some headroom
+      # on slow machines to minimize spurious failures
+      test('io-' + format + '-' + item,
+           qemu_iotests_check_cmd,
+           args: args,
+           depends: qemu_iotests_binaries,
+           env: qemu_iotests_env,
+           protocol: 'tap',
+           timeout: 180,
+           suite: suites)
+  endforeach
 endforeach
-- 
2.39.2



  parent reply	other threads:[~2023-03-03 16:09 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-03 16:07 [PATCH v2 0/8] iotests: make meson aware of individual I/O tests Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 1/8] iotests: explicitly pass source/build dir to 'check' command Daniel P. Berrangé
2023-03-03 16:53   ` Thomas Huth
2023-03-03 16:07 ` [PATCH v2 2/8] iotests: allow test discovery before building Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 3/8] iotests: strip subdir path when listing tests Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 4/8] iotests: print TAP protocol version when reporting tests Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 5/8] iotests: connect stdin to /dev/null when running tests Daniel P. Berrangé
2023-03-03 16:37   ` Thomas Huth
2023-03-03 16:07 ` [PATCH v2 6/8] iotests: always use a unique sub-directory per test Daniel P. Berrangé
2023-03-03 16:39   ` Thomas Huth
2023-05-19  2:20   ` Eric Blake
2023-03-03 16:07 ` Daniel P. Berrangé [this message]
2023-03-03 16:45   ` [PATCH v2 7/8] iotests: register each I/O test separately with meson Thomas Huth
2023-03-29 10:47   ` Thomas Huth
2023-03-29 11:18     ` Daniel P. Berrangé
2023-03-29 11:23       ` Thomas Huth
2023-03-03 16:07 ` [PATCH v2 8/8] iotests: remove the check-block.sh script Daniel P. Berrangé
2023-03-03 16:50   ` Thomas Huth
2023-03-03 17:10 ` [PATCH v2 0/8] iotests: make meson aware of individual I/O tests Thomas Huth
2023-03-10 16:17 ` Daniel P. Berrangé
2023-03-10 17:05   ` Hanna Czenczek
2023-03-14 13:54 ` Alex Bennée

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230303160727.3977246-8-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).