QEMU-Devel Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Pierrick Bouvier" <pierrick.bouvier@oss.qualcomm.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	qemu-block@nongnu.org, "Kevin Wolf" <kwolf@redhat.com>,
	"John Snow" <jsnow@redhat.com>,
	"Pierrick Bouvier" <pierrick.bouvier@linaro.org>,
	"Hanna Reitz" <hreitz@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH v6 07/16] scripts/mtest2make: support optional tests grouping
Date: Wed, 13 May 2026 12:59:54 +0100	[thread overview]
Message-ID: <20260513120003.226790-8-berrange@redhat.com> (raw)
In-Reply-To: <20260513120003.226790-1-berrange@redhat.com>

Currently tests can be classified into three speed groups depending on
whether the meson suite name ends in '-slow' or '-thorough' or neither.

This gets turned into make targets that match the name of the meson
suite, with the speed suffix stripped. e.g.

 * suite=block ->  'make check-block'
 * suite=block-slow -> 'make check-block SPEED=slow'
 * suite=block-thorough -> 'make check-block SPEED=thorough'

The set of tests under the "thorough" speed, however, can get rather
large and it would be useful to have a way to expose further make
targets for directly running a particular subset of tests.

This needs a way to run a target without requiring the SPEED variable,
while also not having them enabled by default as if they were 'quick'
tests.

This modifies mtest2make.py to support this idea by allowing for a new
suffix '-optional' on a suite. When this is present, a correspondingly
named make target will be created without the '-optional' suffix which
will never be run automatically.

This is intended to be combined with use of other suites. For example,
a single NBD test might be added to two suites, 'block-thorough' and
'block-nbd-optional'.

This would allow running it as part of all the block tests with
'make check-block SPEED=thorough', and as part of a standalone target
'make check-block-nbd'.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 scripts/mtest2make.py | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 915f02d600..383ea68b16 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -22,7 +22,7 @@ def names(self, base):
 print(r'''
 SPEED = quick
 
-.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
+.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
 .speed.slow = $(sort $(filter-out %-thorough, $1))
 .speed.thorough = $(sort $1)
 
@@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
             s = s[:-9]
             suites[s].speeds.add('thorough')
 
+def target_name(suite):
+    if suite.endswith('-optional'):
+        return suite[0:-9]
+    return suite
+
 def emit_prolog(suites, prefix):
-    all_targets = ' '.join((f'{prefix}-{k}'
+    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
                             for k in sorted(suites.keys())))
-    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
+    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
                         for k in sorted(suites.keys())))
     print()
     print(f'all-{prefix}-targets = {all_targets}')
@@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
     print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
 
 def emit_suite(name, suite, prefix):
+    tgtname = target_name(name)
     deps = ' '.join(sorted(suite.deps))
     print()
-    print(f'.{prefix}-{name}.deps = {deps}')
-    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
+    print(f'.{prefix}-{tgtname}.deps = {deps}')
+    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
 
     names = ' '.join(sorted(suite.names(name)))
-    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
-    if not name.endswith('-slow') and not name.endswith('-thorough'):
+    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
+    if not name.endswith('-slow') and \
+       not name.endswith('-thorough') and \
+       not name.endswith('-optional'):
         targets += f' {prefix} {prefix}-report.junit.xml'
     print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
     # for the "base" suite possibly add FOO-slow and FOO-thorough
-- 
2.54.0



  parent reply	other threads:[~2026-05-13 12:01 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-05-13 11:59 [PATCH v6 00/16] tests: do more testing of block drivers Daniel P. Berrangé
2026-05-13 11:59 ` [PATCH v6 01/16] python: bump qemu.qmp to v0.0.6 Daniel P. Berrangé
2026-05-13 11:59 ` [PATCH v6 02/16] gitlab: ensure all meson jobs capture build/meson-logs by default Daniel P. Berrangé
2026-05-13 11:59 ` [PATCH v6 03/16] tests: print reason when I/O test is skipped in TAP mode Daniel P. Berrangé
2026-05-15 14:53   ` Kevin Wolf
2026-05-13 11:59 ` [PATCH v6 04/16] tests: remove redundant meson suite for iotests Daniel P. Berrangé
2026-05-13 11:59 ` [PATCH v6 05/16] tests: ensure all qcow2 I/O tests are able to be run via make Daniel P. Berrangé
2026-05-15 14:54   ` Kevin Wolf
2026-05-13 11:59 ` [PATCH v6 06/16] scripts/mtest2make: ensure output has stable sorting Daniel P. Berrangé
2026-05-13 11:59 ` Daniel P. Berrangé [this message]
2026-05-13 11:59 ` [PATCH v6 08/16] tests: add a meson suite / make target per block I/O tests format Daniel P. Berrangé
2026-05-15 14:54   ` Kevin Wolf
2026-05-13 11:59 ` [PATCH v6 09/16] docs/devel/testing: expand documentation for 'make check-block' Daniel P. Berrangé
2026-05-15 14:55   ` Kevin Wolf
2026-05-13 11:59 ` [PATCH v6 10/16] tests: add nbd and luks to the I/O test suites Daniel P. Berrangé
2026-05-15 14:55   ` Kevin Wolf
2026-05-13 11:59 ` [PATCH v6 11/16] tests: use 'driver' as collective term for either format or protocol Daniel P. Berrangé
2026-05-15 14:48   ` Kevin Wolf
2026-05-13 11:59 ` [PATCH v6 12/16] tests: validate dmsetup result in test 128 Daniel P. Berrangé
2026-05-13 15:56   ` Pierrick Bouvier
2026-05-15 14:59   ` Kevin Wolf
2026-05-13 12:00 ` [PATCH v6 13/16] tests: fix check for sudo access in LUKS I/O test Daniel P. Berrangé
2026-05-15 15:01   ` Kevin Wolf
2026-05-13 12:00 ` [PATCH v6 14/16] tests/qemu-iotests: mark 185 as a flaky test Daniel P. Berrangé
2026-05-13 12:14   ` Thomas Huth
2026-05-13 15:59   ` Pierrick Bouvier
2026-05-15 15:05   ` Kevin Wolf
2026-05-13 12:00 ` [PATCH v6 15/16] gitlab: add jobs for thorough block tests Daniel P. Berrangé
2026-05-13 16:00   ` Pierrick Bouvier
2026-05-13 17:18     ` Daniel P. Berrangé
2026-05-13 17:21       ` Pierrick Bouvier
2026-05-13 12:00 ` [PATCH v6 16/16] gitlab: remove I/O tests from build-tcg-disabled job Daniel P. Berrangé
2026-05-15 15:11 ` [PATCH v6 00/16] tests: do more testing of block drivers Kevin Wolf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260513120003.226790-8-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=crosa@redhat.com \
    --cc=hreitz@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=pierrick.bouvier@linaro.org \
    --cc=pierrick.bouvier@oss.qualcomm.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox