qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Kevin Wolf" <kwolf@redhat.com>,
	"Hanna Reitz" <hreitz@redhat.com>,
	qemu-block@nongnu.org, "Paolo Bonzini" <pbonzini@redhat.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH 5/5] iotests: register each I/O test separately with meson
Date: Thu,  2 Mar 2023 18:46:06 +0000	[thread overview]
Message-ID: <20230302184606.418541-6-berrange@redhat.com> (raw)
In-Reply-To: <20230302184606.418541-1-berrange@redhat.com>

Currently meson registers a single test that invokes an entire group of
I/O tests, hiding the test granularity from meson. There are various
downsides of doing this

 * You cannot ask 'meson test' to invoke a single I/O test
 * The meson test timeout can't be applied to the individual
   tests
 * Meson only gets a pass/fail for the overall I/O test group
   not individual tests
 * If a CI job gets killed by the GitLab timeout, we don't
   get visibility into how far through the I/O tests
   execution got.

This switches meson to perform test discovery by invoking 'check' in
dry-run mode. It then registers one meson test case for each I/O
test. Parallel execution remains disabled since the I/O tests do not
use self contained execution environments and thus conflict with
each other.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/meson.build | 33 +++++++++++++++++++++++++++------
 1 file changed, 27 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 323a4acb6a..48c82085af 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -32,16 +32,37 @@ foreach k, v : emulators
   endif
 endforeach
 
+qemu_iotests_check_cmd = files('check')
+
 foreach format, speed: qemu_iotests_formats
   if speed == 'quick'
     suites = 'block'
   else
     suites = ['block-' + speed, speed]
   endif
-  test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format],
-       depends: qemu_iotests_binaries, env: qemu_iotests_env,
-       protocol: 'tap',
-       suite: suites,
-       timeout: 0,
-       is_parallel: false)
+
+  args = ['-tap', '-' + format]
+  if speed == 'quick'
+      args += ['-g', 'auto']
+  endif
+
+  rc = run_command(
+      [qemu_iotests_check_cmd] + args + ['-n'],
+      check: true,
+  )
+
+  foreach item: rc.stdout().strip().split()
+      message('Adding test qemu-iotests-' + format + '-' + item)
+      args = ['-tap', '-' + format, item,
+              '--source-dir', meson.current_source_dir(),
+              '--build-dir', meson.current_build_dir()]
+      test('qemu-iotests-' + format + '-' + item,
+           qemu_iotests_check_cmd,
+           args: args,
+           is_parallel: false,
+           depends: qemu_iotests_binaries,
+           env: qemu_iotests_env,
+           protocol: 'tap',
+           suite: suites)
+  endforeach
 endforeach
-- 
2.39.2



  parent reply	other threads:[~2023-03-02 18:47 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-02 18:46 [PATCH 0/5] iotests: make meson aware of individual I/O tests Daniel P. Berrangé
2023-03-02 18:46 ` [PATCH 1/5] iotests: explicitly pass source/build dir to 'check' command Daniel P. Berrangé
2023-03-03 12:55   ` Alex Bennée
2023-03-03 13:01     ` Daniel P. Berrangé
2023-03-02 18:46 ` [PATCH 2/5] iotests: allow test discovery before building Daniel P. Berrangé
2023-03-03  8:14   ` Thomas Huth
2023-03-03 12:56   ` Alex Bennée
2023-03-02 18:46 ` [PATCH 3/5] iotests: strip subdir path when listing tests Daniel P. Berrangé
2023-03-03 12:58   ` Alex Bennée
2023-03-02 18:46 ` [PATCH 4/5] iotests: print TAP protocol version when reporting tests Daniel P. Berrangé
2023-03-03  8:17   ` Thomas Huth
2023-03-03 12:58   ` Alex Bennée
2023-03-02 18:46 ` Daniel P. Berrangé [this message]
2023-03-03  9:34   ` [PATCH 5/5] iotests: register each I/O test separately with meson Thomas Huth
2023-03-03 10:21     ` Daniel P. Berrangé
2023-03-02 18:54 ` [PATCH 0/5] iotests: make meson aware of individual I/O tests Daniel P. Berrangé
2023-03-03  8:30 ` Thomas Huth
2023-03-03  8:53   ` Daniel P. Berrangé
2023-03-03  9:39     ` Daniel P. Berrangé
2023-03-03 10:27     ` Thomas Huth
2023-03-03 10:45   ` Daniel P. Berrangé
2023-03-03 13:06     ` Daniel P. Berrangé
2023-03-03 15:49       ` Thomas Huth
2023-03-03 15:52         ` Daniel P. Berrangé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230302184606.418541-6-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).