All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Kevin Wolf" <kwolf@redhat.com>,
	qemu-block@nongnu.org, "Thomas Huth" <thuth@redhat.com>,
	"Hanna Reitz" <hreitz@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH v2 7/8] iotests: register each I/O test separately with meson
Date: Fri,  3 Mar 2023 16:07:26 +0000	[thread overview]
Message-ID: <20230303160727.3977246-8-berrange@redhat.com> (raw)
In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com>

Currently meson registers a single test that invokes an entire group of
I/O tests, hiding the test granularity from meson. There are various
downsides of doing this

 * You cannot ask 'meson test' to invoke a single I/O test
 * The meson test timeout can't be applied to the individual
   tests
 * Meson only gets a pass/fail for the overall I/O test group
   not individual tests
 * If a CI job gets killed by the GitLab timeout, we don't
   get visibility into how far through the I/O tests
   execution got.

This switches meson to perform test discovery by invoking 'check' in
dry-run mode. It then registers one meson test case for each I/O
test. Parallel execution remains disabled since the I/O tests do not
use self contained execution environments and thus conflict with
each other.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/meson.build | 35 ++++++++++++++++++++++++++++------
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 323a4acb6a..a162f683ef 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -32,16 +32,39 @@ foreach k, v : emulators
   endif
 endforeach
 
+qemu_iotests_check_cmd = files('check')
+
 foreach format, speed: qemu_iotests_formats
   if speed == 'quick'
     suites = 'block'
   else
     suites = ['block-' + speed, speed]
   endif
-  test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format],
-       depends: qemu_iotests_binaries, env: qemu_iotests_env,
-       protocol: 'tap',
-       suite: suites,
-       timeout: 0,
-       is_parallel: false)
+
+  args = ['-tap', '-' + format]
+  if speed == 'quick'
+      args += ['-g', 'auto']
+  endif
+
+  rc = run_command(
+      [qemu_iotests_check_cmd] + args + ['-n'],
+      check: true,
+  )
+
+  foreach item: rc.stdout().strip().split()
+      args = ['-tap', '-' + format, item,
+              '--source-dir', meson.current_source_dir(),
+              '--build-dir', meson.current_build_dir()]
+      # Some individual tests take as long as 45 seconds
+      # Bump the timeout to 3 minutes for some headroom
+      # on slow machines to minimize spurious failures
+      test('io-' + format + '-' + item,
+           qemu_iotests_check_cmd,
+           args: args,
+           depends: qemu_iotests_binaries,
+           env: qemu_iotests_env,
+           protocol: 'tap',
+           timeout: 180,
+           suite: suites)
+  endforeach
 endforeach
-- 
2.39.2



  parent reply	other threads:[~2023-03-03 16:09 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-03 16:07 [PATCH v2 0/8] iotests: make meson aware of individual I/O tests Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 1/8] iotests: explicitly pass source/build dir to 'check' command Daniel P. Berrangé
2023-03-03 16:53   ` Thomas Huth
2023-03-03 16:07 ` [PATCH v2 2/8] iotests: allow test discovery before building Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 3/8] iotests: strip subdir path when listing tests Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 4/8] iotests: print TAP protocol version when reporting tests Daniel P. Berrangé
2023-03-03 16:07 ` [PATCH v2 5/8] iotests: connect stdin to /dev/null when running tests Daniel P. Berrangé
2023-03-03 16:37   ` Thomas Huth
2023-03-03 16:07 ` [PATCH v2 6/8] iotests: always use a unique sub-directory per test Daniel P. Berrangé
2023-03-03 16:39   ` Thomas Huth
2023-05-19  2:20   ` Eric Blake
2023-03-03 16:07 ` Daniel P. Berrangé [this message]
2023-03-03 16:45   ` [PATCH v2 7/8] iotests: register each I/O test separately with meson Thomas Huth
2023-03-29 10:47   ` Thomas Huth
2023-03-29 11:18     ` Daniel P. Berrangé
2023-03-29 11:23       ` Thomas Huth
2023-03-03 16:07 ` [PATCH v2 8/8] iotests: remove the check-block.sh script Daniel P. Berrangé
2023-03-03 16:50   ` Thomas Huth
2023-03-03 17:10 ` [PATCH v2 0/8] iotests: make meson aware of individual I/O tests Thomas Huth
2023-03-10 16:17 ` Daniel P. Berrangé
2023-03-10 17:05   ` Hanna Czenczek
2023-03-14 13:54 ` Alex Bennée

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230303160727.3977246-8-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=hreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.