[PATCH v2 00/13] tests: do more testing of block drivers

[PATCH v2 00/13] tests: do more testing of block drivers

[Daniel P. Berrangé]

Original (very broken) v1 posting:

  https://lists.nongnu.org/archive/html/qemu-devel/2025-10/msg01650.html

The recent set of regressions identified in the LUKS block driver
re-inforced that despite having a hugely useful set of I/O tests,
our CI coverage is still letting through too many bugs.

The core goals of this series were/are:

 * Add LUKS and NBD to the tested formats/protocols
   integrated into "make check-block SPEED=thorough"

 * Ensure that all qcow2 tests can be run with "SPEED=slow",
   not staying limited to only the 'auto' group used by the
   'make check-block' target in its default 'quick' mode

 * Add  'make check-block-$FORMAT' to expose a standalone
   target for running all tests for a given format (or
   equivalently a protocol)

 * Add GitLab CI jobs for exercising tests for all formats

A sample pipeline for this is

  https://gitlab.com/berrange/qemu/-/pipelines/2258731804/

Showing the 'block-FORMAT-centos' jobs which test the 10
block formats/protocols currently enabled.

This series fixes a couple of bugs

 * sendmsg generates a deprecation warning on centos

 * Test 128 check for device mapper usage doesn't work
   sufficiently well to detect gitlab's containe env
   can't do dynamic devfs

 * Test 185 is reliably failing on gitlab

I'm unclear if there are other non-deterministic failures that
would only appear if we start running the tests regularly, or
against QEMU's private runners instead of gitlab shared runners.

The ones I've fixed have been consistent across quite a few test
pipelines so far with the shared runners.

At this point it is a chicken & egg problem though.

If we don't bite the bullet and enable block I/O tests in GitLab
we'll never find out if they're stable enough to rely on.

As a mitigation against instability a patch in this series adds
a QEMU_TEST_IO_SKIP env variable that we are able to set as a
GitLab CI env variable. This lets us skip broken tests without
waiting for a git commit to disable them.

Changed in v2:

  * TL;DR: this time is actally works as described, rather
    unlike v1 which was majorly broken in its understanding
    of how mtest2make.py was working
  * Expand mtest2make.py to support a concept of an "optional"
    test suite, to create new standalone make check-xxx targets
    that don't get run by default
  * Fix sorting of Makefile.mtest to be stable
  * Fix or disable  some bugs in iotests exposed by gitlab
  * Fix the 'make check-block-$FORMAT' targets so they really
    don't get triggered by 'make chck' by default.
  * Avoid duplicating the same qcow2 tests in the 'quick'
    suite and 'slow' suite. The latter must only have the
    tests not in the 'auto' group
  * Add RST documentation of the new make targets
  * Add env variable to let us dynamically skip I/O tests
    without git commit changes


Daniel P. Berrangé (13):
  Revert "python/aqmp: fix send_fd_scm for python 3.6.x"
  tests: print reason when I/O test is skipped in TAP mode
  tests: remove redundant meson suite for iotests
  tests: ensure all qcow2 I/O tests are able to be run via make
  scripts/mtest2make: ensure output has stable sorting
  scripts/mtest2make: support optional tests grouping
  tests: add a meson suite / make target per block I/O tests format
  docs/devel/testing: expand documentation for 'make check-block'
  tests: add nbd and luks to the I/O test suites
  tests: skip I/O test 128 on GitLab due to broken dmsetup
  tests: fix check for sudo access in LUKS I/O test
  tests: add QEMU_TEST_IO_SKIP for skipping I/O tests
  gitlab: add jobs for thorough block tests

 .gitlab-ci.d/buildtest.yml       | 92 ++++++++++++++++++++++++++++++++
 docs/devel/testing/main.rst      | 34 ++++++++++--
 python/qemu/qmp/qmp_client.py    |  9 ++--
 scripts/mtest2make.py            | 30 +++++++----
 tests/Makefile.include           |  3 +-
 tests/qemu-iotests/128           |  2 +
 tests/qemu-iotests/149           | 13 +++--
 tests/qemu-iotests/meson.build   | 55 ++++++++++++++++---
 tests/qemu-iotests/testrunner.py | 18 ++++++-
 9 files changed, 224 insertions(+), 32 deletions(-)

-- 
2.52.0

[PATCH v2 01/13] Revert "python/aqmp: fix send_fd_scm for python 3.6.x"

[Daniel P. Berrangé]

This reverts commit a57cb3e23d5ac918a69d0aab918470ff0b429ff9.

The current code now only requires compatibility with Python
3.8 or later.

The conditional usage of 'sendmsg' on the async IO socket
wrapper will generate a deprecation warning on stderr
every time send_fd_scm is used with older Python versions.

This has the effect of breaking the QEMU I/O tests when run
on Python versions before the 'sendmsg' wrapper was removed.

Unconditionally accessing 'sock._sock' ensures we never use
the asyncio socket wrapper, and thus never risk triggering
deprecation warnings on any Python version

Most notably this fixes the QEMU block I/O tests on CentOS
Stream9 that use "sendmsg" for FD passing, which otherwise
generate deprecation messages breaking the expected output
comparison.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 python/qemu/qmp/qmp_client.py | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/python/qemu/qmp/qmp_client.py b/python/qemu/qmp/qmp_client.py
index 8beccfe29d..7a115b693b 100644
--- a/python/qemu/qmp/qmp_client.py
+++ b/python/qemu/qmp/qmp_client.py
@@ -720,12 +720,9 @@ def send_fd_scm(self, fd: int) -> None:
         if sock.family != socket.AF_UNIX:
             raise QMPError("Sending file descriptors requires a UNIX socket.")
 
-        if not hasattr(sock, 'sendmsg'):
-            # We need to void the warranty sticker.
-            # Access to sendmsg is scheduled for removal in Python 3.11.
-            # Find the real backing socket to use it anyway.
-            sock = sock._sock  # pylint: disable=protected-access
-
+        # Void the warranty sticker.
+        # Access to sendmsg in asyncio is scheduled for removal in Python 3.11.
+        sock = sock._sock  # pylint: disable=protected-access
         sock.sendmsg(
             [b' '],
             [(socket.SOL_SOCKET, socket.SCM_RIGHTS, struct.pack('@i', fd))]
-- 
2.52.0

Re: [PATCH v2 01/13] Revert "python/aqmp: fix send_fd_scm for python 3.6.x"

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> This reverts commit a57cb3e23d5ac918a69d0aab918470ff0b429ff9.
> 
> The current code now only requires compatibility with Python
> 3.8 or later.
> 
> The conditional usage of 'sendmsg' on the async IO socket
> wrapper will generate a deprecation warning on stderr
> every time send_fd_scm is used with older Python versions.
> 
> This has the effect of breaking the QEMU I/O tests when run
> on Python versions before the 'sendmsg' wrapper was removed.
> 
> Unconditionally accessing 'sock._sock' ensures we never use
> the asyncio socket wrapper, and thus never risk triggering
> deprecation warnings on any Python version
> 
> Most notably this fixes the QEMU block I/O tests on CentOS
> Stream9 that use "sendmsg" for FD passing, which otherwise
> generate deprecation messages breaking the expected output
> comparison.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   python/qemu/qmp/qmp_client.py | 9 +++------
>   1 file changed, 3 insertions(+), 6 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH v2 01/13] Revert "python/aqmp: fix send_fd_scm for python 3.6.x"

[John Snow]

On Tue, Jan 13, 2026 at 2:17 AM Thomas Huth <thuth@redhat.com> wrote:
>
> On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> > This reverts commit a57cb3e23d5ac918a69d0aab918470ff0b429ff9.
> >
> > The current code now only requires compatibility with Python
> > 3.8 or later.
> >
> > The conditional usage of 'sendmsg' on the async IO socket
> > wrapper will generate a deprecation warning on stderr
> > every time send_fd_scm is used with older Python versions.
> >
> > This has the effect of breaking the QEMU I/O tests when run
> > on Python versions before the 'sendmsg' wrapper was removed.
> >
> > Unconditionally accessing 'sock._sock' ensures we never use
> > the asyncio socket wrapper, and thus never risk triggering
> > deprecation warnings on any Python version
> >
> > Most notably this fixes the QEMU block I/O tests on CentOS
> > Stream9 that use "sendmsg" for FD passing, which otherwise
> > generate deprecation messages breaking the expected output
> > comparison.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >   python/qemu/qmp/qmp_client.py | 9 +++------
> >   1 file changed, 3 insertions(+), 6 deletions(-)
>
> Reviewed-by: Thomas Huth <thuth@redhat.com>

OK, please merge this with my blessing and I will worry about the
standalone repo afterwards.

Acked-by: John Snow <jsnow@redhat.com>

Re: [PATCH v2 01/13] Revert "python/aqmp: fix send_fd_scm for python 3.6.x"

[Daniel P. Berrangé]

On Wed, Jan 14, 2026 at 11:57:23AM -0500, John Snow wrote:
> On Tue, Jan 13, 2026 at 2:17 AM Thomas Huth <thuth@redhat.com> wrote:
> >
> > On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> > > This reverts commit a57cb3e23d5ac918a69d0aab918470ff0b429ff9.
> > >
> > > The current code now only requires compatibility with Python
> > > 3.8 or later.
> > >
> > > The conditional usage of 'sendmsg' on the async IO socket
> > > wrapper will generate a deprecation warning on stderr
> > > every time send_fd_scm is used with older Python versions.
> > >
> > > This has the effect of breaking the QEMU I/O tests when run
> > > on Python versions before the 'sendmsg' wrapper was removed.
> > >
> > > Unconditionally accessing 'sock._sock' ensures we never use
> > > the asyncio socket wrapper, and thus never risk triggering
> > > deprecation warnings on any Python version
> > >
> > > Most notably this fixes the QEMU block I/O tests on CentOS
> > > Stream9 that use "sendmsg" for FD passing, which otherwise
> > > generate deprecation messages breaking the expected output
> > > comparison.
> > >
> > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > > ---
> > >   python/qemu/qmp/qmp_client.py | 9 +++------
> > >   1 file changed, 3 insertions(+), 6 deletions(-)
> >
> > Reviewed-by: Thomas Huth <thuth@redhat.com>
> 
> OK, please merge this with my blessing and I will worry about the
> standalone repo afterwards.

We already merged it there with:

  https://gitlab.com/qemu-project/python-qemu-qmp/-/merge_requests/45

> 
> Acked-by: John Snow <jsnow@redhat.com>
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v2 02/13] tests: print reason when I/O test is skipped in TAP mode

[Daniel P. Berrangé]

The TAP output on a skipped test:

  ok raw 181 # SKIP

is not informative.

The test program included a reason, and that should be displayed
in TAP mode (it is already shown in non-TAP mode):

  ok raw 181 # SKIP Postcopy is not supported

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/testrunner.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index e2a3658994..dbe2dddc32 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -174,7 +174,7 @@ def test_print_one_line(self, test: str,
             elif status == 'fail':
                 print(f'not ok {self.env.imgfmt} {test}')
             elif status == 'not run':
-                print(f'ok {self.env.imgfmt} {test} # SKIP')
+                print(f'ok {self.env.imgfmt} {test} # SKIP {description}')
             return
 
         if lasttime:
-- 
2.52.0

Re: [PATCH v2 02/13] tests: print reason when I/O test is skipped in TAP mode

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> The TAP output on a skipped test:
> 
>    ok raw 181 # SKIP
> 
> is not informative.
> 
> The test program included a reason, and that should be displayed
> in TAP mode (it is already shown in non-TAP mode):
> 
>    ok raw 181 # SKIP Postcopy is not supported
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/qemu-iotests/testrunner.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

[PATCH v2 03/13] tests: remove redundant meson suite for iotests

[Daniel P. Berrangé]

If a test is in the 'block-slow' or 'block-thorough' suite, there is no
need to also add it to the 'slow' or 'thorough' suites.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index d7bae71ced..bf588cc2c9 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -33,7 +33,7 @@ foreach format, speed: qemu_iotests_formats
   if speed == 'quick'
     suites = 'block'
   else
-    suites = ['block-' + speed, speed]
+    suites = ['block-' + speed]
   endif
 
   args = ['-tap', '-' + format]
-- 
2.52.0

Re: [PATCH v2 03/13] tests: remove redundant meson suite for iotests

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> If a test is in the 'block-slow' or 'block-thorough' suite, there is no
> need to also add it to the 'slow' or 'thorough' suites.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/qemu-iotests/meson.build | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index d7bae71ced..bf588cc2c9 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -33,7 +33,7 @@ foreach format, speed: qemu_iotests_formats
>     if speed == 'quick'
>       suites = 'block'
>     else
> -    suites = ['block-' + speed, speed]
> +    suites = ['block-' + speed]
>     endif

Oh, weird, I was pretty sure that this was necessary at one point in time in 
the past ... but I just checked, and it now seems to work without this, 
indeed, so:

Tested-by: Thomas Huth <thuth@redhat.com>

By the way, we've got the same thing in tests/functional/meson.build, too...

  Thomas

[PATCH v2 04/13] tests: ensure all qcow2 I/O tests are able to be run via make

[Daniel P. Berrangé]

For block formats marked as 'quick', only tests in the 'auto' group are
added to the meson test suite.

The result of this is that qcow2 tests not in the 'auto' group cannot be
run at all, even if passing SPEED=slow or SPEED=thorough.

To fix this we need todo two passes over the I/O test list. First add
all tests from 'auto' group into the 'block' suite, so they are run by
default. Then on the second pass add any tests which were not in 'auto'
into the 'block-slow' suite, so they get run when SPEED=slow or
SPEED=thorough.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index bf588cc2c9..1a24d801a3 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -30,16 +30,48 @@ endforeach
 qemu_iotests_check_cmd = files('check')
 
 foreach format, speed: qemu_iotests_formats
+  # Formats tagged 'quick' get the subset of tests in the 'auto'
+  # group, run by default with 'make check' / 'make check-block'
+  seen = []
   if speed == 'quick'
-    suites = 'block'
+    args = ['-tap', '-' + format, '-g', 'auto']
+    suites = ['block']
+
+    rc = run_command(
+      [python, qemu_iotests_check_cmd] + args + ['-n'],
+      check: true,
+    )
+
+    foreach item: rc.stdout().strip().split()
+      seen += item
+      args = [qemu_iotests_check_cmd,
+              '-tap', '-' + format, item,
+              '--source-dir', meson.current_source_dir(),
+              '--build-dir', meson.current_build_dir()]
+      # Some individual tests take as long as 45 seconds
+      # Bump the timeout to 3 minutes for some headroom
+      # on slow machines to minimize spurious failures
+      test('io-' + format + '-' + item,
+           python,
+           args: args,
+           depends: qemu_iotests_binaries,
+           env: qemu_iotests_env,
+           protocol: 'tap',
+           timeout: 180,
+           suite: suites)
+    endforeach
+  endif
+
+  suites = []
+  # Any format tagged quick or slow also gets added to slow
+  # otherwise its tagged thorough
+  if speed != 'thorough'
+    suites += ['block-slow']
   else
-    suites = ['block-' + speed]
+    suites += ['block-thorough']
   endif
 
   args = ['-tap', '-' + format]
-  if speed == 'quick'
-      args += ['-g', 'auto']
-  endif
 
   rc = run_command(
       [python, qemu_iotests_check_cmd] + args + ['-n'],
@@ -47,6 +79,12 @@ foreach format, speed: qemu_iotests_formats
   )
 
   foreach item: rc.stdout().strip().split()
+      # Skip any tests already added from the 'auto' group
+      # as they're run in the 'quick' suite already
+      if item in seen
+          continue
+      endif
+
       args = [qemu_iotests_check_cmd,
               '-tap', '-' + format, item,
               '--source-dir', meson.current_source_dir(),
-- 
2.52.0

Re: [PATCH v2 04/13] tests: ensure all qcow2 I/O tests are able to be run via make

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> For block formats marked as 'quick', only tests in the 'auto' group are
> added to the meson test suite.
> 
> The result of this is that qcow2 tests not in the 'auto' group cannot be
> run at all, even if passing SPEED=slow or SPEED=thorough.
> 
> To fix this we need todo two passes over the I/O test list. First add
> all tests from 'auto' group into the 'block' suite, so they are run by
> default. Then on the second pass add any tests which were not in 'auto'
> into the 'block-slow' suite, so they get run when SPEED=slow or
> SPEED=thorough.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++++++----
>   1 file changed, 43 insertions(+), 5 deletions(-)
> 
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index bf588cc2c9..1a24d801a3 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -30,16 +30,48 @@ endforeach
>   qemu_iotests_check_cmd = files('check')
>   
>   foreach format, speed: qemu_iotests_formats
> +  # Formats tagged 'quick' get the subset of tests in the 'auto'
> +  # group, run by default with 'make check' / 'make check-block'
> +  seen = []
>     if speed == 'quick'
> -    suites = 'block'
> +    args = ['-tap', '-' + format, '-g', 'auto']
> +    suites = ['block']
> +
> +    rc = run_command(
> +      [python, qemu_iotests_check_cmd] + args + ['-n'],
> +      check: true,
> +    )
> +
> +    foreach item: rc.stdout().strip().split()
> +      seen += item
> +      args = [qemu_iotests_check_cmd,
> +              '-tap', '-' + format, item,
> +              '--source-dir', meson.current_source_dir(),
> +              '--build-dir', meson.current_build_dir()]
> +      # Some individual tests take as long as 45 seconds
> +      # Bump the timeout to 3 minutes for some headroom
> +      # on slow machines to minimize spurious failures
> +      test('io-' + format + '-' + item,
> +           python,
> +           args: args,
> +           depends: qemu_iotests_binaries,
> +           env: qemu_iotests_env,
> +           protocol: 'tap',
> +           timeout: 180,
> +           suite: suites)
> +    endforeach

It's a little bit unfortunate that we have this construct now twice in the 
meson.build file (with just the difference of the "-g auto") ... but I 
currently lack an idea how it could be done in a better way, so:

Reviewed-by: Thomas Huth <thuth@redhat.com>


> +  endif
> +
> +  suites = []
> +  # Any format tagged quick or slow also gets added to slow
> +  # otherwise its tagged thorough
> +  if speed != 'thorough'
> +    suites += ['block-slow']
>     else
> -    suites = ['block-' + speed]
> +    suites += ['block-thorough']
>     endif
>   
>     args = ['-tap', '-' + format]
> -  if speed == 'quick'
> -      args += ['-g', 'auto']
> -  endif
>   
>     rc = run_command(
>         [python, qemu_iotests_check_cmd] + args + ['-n'],
> @@ -47,6 +79,12 @@ foreach format, speed: qemu_iotests_formats
>     )
>   
>     foreach item: rc.stdout().strip().split()
> +      # Skip any tests already added from the 'auto' group
> +      # as they're run in the 'quick' suite already
> +      if item in seen
> +          continue
> +      endif
> +
>         args = [qemu_iotests_check_cmd,
>                 '-tap', '-' + format, item,
>                 '--source-dir', meson.current_source_dir(),

[PATCH v2 05/13] scripts/mtest2make: ensure output has stable sorting

[Daniel P. Berrangé]

When debugging mtest2make.py changes it is important to be able to
compare the old and new output. This requires that any lists in the
output have stable sort ordering.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 scripts/mtest2make.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 4b252defc3..915f02d600 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -67,8 +67,10 @@ def process_tests(test, targets, suites):
             suites[s].speeds.add('thorough')
 
 def emit_prolog(suites, prefix):
-    all_targets = ' '.join((f'{prefix}-{k}' for k in suites.keys()))
-    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml' for k in suites.keys()))
+    all_targets = ' '.join((f'{prefix}-{k}'
+                            for k in sorted(suites.keys())))
+    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
+                        for k in sorted(suites.keys())))
     print()
     print(f'all-{prefix}-targets = {all_targets}')
     print(f'all-{prefix}-xml = {all_xml}')
@@ -81,12 +83,12 @@ def emit_prolog(suites, prefix):
     print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
 
 def emit_suite(name, suite, prefix):
-    deps = ' '.join(suite.deps)
+    deps = ' '.join(sorted(suite.deps))
     print()
     print(f'.{prefix}-{name}.deps = {deps}')
     print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
 
-    names = ' '.join(suite.names(name))
+    names = ' '.join(sorted(suite.names(name)))
     targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
     if not name.endswith('-slow') and not name.endswith('-thorough'):
         targets += f' {prefix} {prefix}-report.junit.xml'
-- 
2.52.0

Re: [PATCH v2 05/13] scripts/mtest2make: ensure output has stable sorting

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> When debugging mtest2make.py changes it is important to be able to
> compare the old and new output. This requires that any lists in the
> output have stable sort ordering.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---

Reviewed-by: Thomas Huth <thuth@redhat.com>

[PATCH v2 06/13] scripts/mtest2make: support optional tests grouping

[Daniel P. Berrangé]

Currently tests can be classified into three speed groups depending on
whether the meson suite name ends in '-slow' or '-thorough' or neither.

This gets turned into make targets that match the name of the meson
suite, with the speed suffix stripped. e.g.

 * suite=block ->  'make check-block'
 * suite=block-slow -> 'make check-block SPEED=slow'
 * suite=block-thorough -> 'make check-block SPEED=thorough'

The set of tests under the "thorough" speed, however, can get rather
large and it would be useful to have a way to expose further make
targets for directly running a particular subset of tests.

This needs a way to run a target without requiring the SPEED varaible,
while also not having them enabled by default as if they were 'quick'
tests.

This modifies mtest2make.py to support this idea by allowing for a new
suffix '-optional' on a suite. When this is present, a correspondingly
named make target will be created without the '-optional' suffix which
will never be run automatically.

This is intended to be combined with use of other suites. For example,
a single NBD test might be added to two suites, 'block-thorough' and
'block-nbd-optional'.

This would allow running it as part of all the block tests with
'make check-block SPEED=thorough', and as part of a standalone target
'make check-block-nbd'.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 scripts/mtest2make.py | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 915f02d600..383ea68b16 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -22,7 +22,7 @@ def names(self, base):
 print(r'''
 SPEED = quick
 
-.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
+.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
 .speed.slow = $(sort $(filter-out %-thorough, $1))
 .speed.thorough = $(sort $1)
 
@@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
             s = s[:-9]
             suites[s].speeds.add('thorough')
 
+def target_name(suite):
+    if suite.endswith('-optional'):
+        return suite[0:-9]
+    return suite
+
 def emit_prolog(suites, prefix):
-    all_targets = ' '.join((f'{prefix}-{k}'
+    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
                             for k in sorted(suites.keys())))
-    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
+    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
                         for k in sorted(suites.keys())))
     print()
     print(f'all-{prefix}-targets = {all_targets}')
@@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
     print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
 
 def emit_suite(name, suite, prefix):
+    tgtname = target_name(name)
     deps = ' '.join(sorted(suite.deps))
     print()
-    print(f'.{prefix}-{name}.deps = {deps}')
-    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
+    print(f'.{prefix}-{tgtname}.deps = {deps}')
+    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
 
     names = ' '.join(sorted(suite.names(name)))
-    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
-    if not name.endswith('-slow') and not name.endswith('-thorough'):
+    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
+    if not name.endswith('-slow') and \
+       not name.endswith('-thorough') and \
+       not name.endswith('-optional'):
         targets += f' {prefix} {prefix}-report.junit.xml'
     print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
     # for the "base" suite possibly add FOO-slow and FOO-thorough
-- 
2.52.0

[PATCH v2 07/13] tests: add a meson suite / make target per block I/O tests format

[Daniel P. Berrangé]

Currently each block format is classified as either 'quick', 'slow' or
'thorough' and this controls whether its I/O tests are added to the meson
suites 'block-quick', 'block-slow' or 'block-thorough'.

This suites are exposed the 'check-block' make target, accepting the
optional SPEED variable.

As we add more formats to the 'thorough' group, however, it becomes
increasingly large and time consuming to run. What is needed is a make
target that can exercise all tests for an individual format, regardless
of speed classification.

This makes use of the previous enhancement to mtest2make.py to introduce
new meson suites 'block-$FORMAT-optional', which translate to new top
level make targets 'check-block-$FORMAT'. These new targets always run
all tests and as such do not need the SPEED variable to be set, but are
not triggered by 'make check' or 'make check-block'.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/Makefile.include         | 3 ++-
 tests/qemu-iotests/meson.build | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 7728098981..fc6fdcbbf9 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -14,7 +14,8 @@ check-help:
 	@echo " $(MAKE) check-unit               Run qobject tests"
 	@echo " $(MAKE) check-qapi-schema        Run QAPI schema tests"
 	@echo " $(MAKE) check-tracetool          Run tracetool generator tests"
-	@echo " $(MAKE) check-block              Run block tests"
+	@echo " $(MAKE) check-block              Run block tests (all formats)"
+	@echo " $(MAKE) check-block-FORMAT       Run block tests (only for FORMAT)"
 ifneq ($(filter $(all-check-targets), check-softfloat),)
 	@echo " $(MAKE) check-tcg                Run TCG tests"
 	@echo " $(MAKE) check-softfloat          Run FPU emulation tests"
diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 1a24d801a3..16a5e39476 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -62,7 +62,8 @@ foreach format, speed: qemu_iotests_formats
     endforeach
   endif
 
-  suites = []
+  # Every format gets put in the format specific suite
+  suites = ['block-' + format + '-optional']
   # Any format tagged quick or slow also gets added to slow
   # otherwise its tagged thorough
   if speed != 'thorough'
-- 
2.52.0

[PATCH v2 08/13] docs/devel/testing: expand documentation for 'make check-block'

[Daniel P. Berrangé]

Explain in greater detail what 'check-block' will run for each format,
and also document the new format specific targets.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 docs/devel/testing/main.rst    | 27 ++++++++++++++++++++++++---
 tests/qemu-iotests/meson.build |  2 ++
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index 0662766b5c..ec29cf7708 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -221,9 +221,30 @@ same commit that alters the generator code.
 check-block
 ~~~~~~~~~~~
 
-``make check-block`` runs a subset of the block layer iotests (the tests that
-are in the "auto" group).
-See the "QEMU iotests" section below for more information.
+There are a variety of ways to exercise the block layer I/O tests
+via make targets.
+
+A default ``make check`` or ``make check-block`` command will exercise
+the ``qcow2`` format, using the tests tagged into the ``auto`` group
+only.
+
+These targets accept the ``SPEED`` variable to augment the set of tests
+to run. A slightly more comprehensive test plan can be run by defining
+``SPEED=slow``, which enables all tests for the ``qcow2`` and ``raw``
+formats. The most comprehensive test plan can be run by defining
+``SPEED=thorough``, which enables all available tests for every format.
+
+This set of formats currently enabled for make integration are
+``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
+``vmdk``, and ``vpc``.
+
+Each of formats also has its own dedicated make target, named
+``make check-block-$FORMAT`` which will run all available tests for
+the designated format and does not require the ``SPEED`` variable
+to be set.
+
+See the "QEMU iotests" section below for more information on the
+block I/O test framework that is leveraged by these ``make`` targets.
 
 .. _qemu-iotests:
 
diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 16a5e39476..66b09d6b97 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -10,6 +10,8 @@ endif
 
 qemu_iotests_binaries = [qemu_img, qemu_io, qemu_nbd, qsd]
 qemu_iotests_env = {'PYTHON': python.full_path()}
+# If altering this definition, also update docs/devel/testing/main.rst
+# section on 'check-block' targets to reflect the changes
 qemu_iotests_formats = {
   'qcow2': 'quick',
   'raw': 'slow',
-- 
2.52.0

Re: [PATCH v2 08/13] docs/devel/testing: expand documentation for 'make check-block'

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> Explain in greater detail what 'check-block' will run for each format,
> and also document the new format specific targets.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   docs/devel/testing/main.rst    | 27 ++++++++++++++++++++++++---
>   tests/qemu-iotests/meson.build |  2 ++
>   2 files changed, 26 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> index 0662766b5c..ec29cf7708 100644
> --- a/docs/devel/testing/main.rst
> +++ b/docs/devel/testing/main.rst
> @@ -221,9 +221,30 @@ same commit that alters the generator code.
>   check-block
>   ~~~~~~~~~~~
>   
> -``make check-block`` runs a subset of the block layer iotests (the tests that
> -are in the "auto" group).
> -See the "QEMU iotests" section below for more information.
> +There are a variety of ways to exercise the block layer I/O tests
> +via make targets.
> +
> +A default ``make check`` or ``make check-block`` command will exercise
> +the ``qcow2`` format, using the tests tagged into the ``auto`` group
> +only.
> +
> +These targets accept the ``SPEED`` variable to augment the set of tests
> +to run. A slightly more comprehensive test plan can be run by defining
> +``SPEED=slow``, which enables all tests for the ``qcow2`` and ``raw``
> +formats. The most comprehensive test plan can be run by defining
> +``SPEED=thorough``, which enables all available tests for every format.

I'd maybe rather say "for most formats" instead of "for every format" since 
we still don't check stuff like "qcow1" etc.

Apart from that, patch looks fine to me, so with that change:
Reviewed-by: Thomas Huth <thuth@redhat.com>


> +This set of formats currently enabled for make integration are
> +``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
> +``vmdk``, and ``vpc``.
> +
> +Each of formats also has its own dedicated make target, named
> +``make check-block-$FORMAT`` which will run all available tests for
> +the designated format and does not require the ``SPEED`` variable
> +to be set.
> +
> +See the "QEMU iotests" section below for more information on the
> +block I/O test framework that is leveraged by these ``make`` targets.
>   
>   .. _qemu-iotests:
>   
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index 16a5e39476..66b09d6b97 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -10,6 +10,8 @@ endif
>   
>   qemu_iotests_binaries = [qemu_img, qemu_io, qemu_nbd, qsd]
>   qemu_iotests_env = {'PYTHON': python.full_path()}
> +# If altering this definition, also update docs/devel/testing/main.rst
> +# section on 'check-block' targets to reflect the changes
>   qemu_iotests_formats = {
>     'qcow2': 'quick',
>     'raw': 'slow',

[PATCH v2 09/13] tests: add nbd and luks to the I/O test suites

[Daniel P. Berrangé]

This introduces new suites for running I/O tests on NBD and LUKS
drivers, giving new make targets

 * make check-block-luks
 * make check-block-nbd

as well as adding their tests to 'make check-block SPEED=thorough'

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 docs/devel/testing/main.rst    | 4 ++--
 tests/qemu-iotests/meson.build | 4 +++-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index ec29cf7708..910ec11ab4 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -235,8 +235,8 @@ formats. The most comprehensive test plan can be run by defining
 ``SPEED=thorough``, which enables all available tests for every format.
 
 This set of formats currently enabled for make integration are
-``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
-``vmdk``, and ``vpc``.
+``luks``, ``nbd``, ``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``,
+``vhdx``, ``vmdk``, and ``vpc``.
 
 Each of formats also has its own dedicated make target, named
 ``make check-block-$FORMAT`` which will run all available tests for
diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index 66b09d6b97..744d0b6e88 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -15,12 +15,14 @@ qemu_iotests_env = {'PYTHON': python.full_path()}
 qemu_iotests_formats = {
   'qcow2': 'quick',
   'raw': 'slow',
+  'luks': 'thorough',
+  'nbd': 'thorough',
   'parallels': 'thorough',
   'qed': 'thorough',
   'vdi': 'thorough',
   'vhdx': 'thorough',
   'vmdk': 'thorough',
-  'vpc': 'thorough'
+  'vpc': 'thorough',
 }
 
 foreach k, v : emulators
-- 
2.52.0

Re: [PATCH v2 09/13] tests: add nbd and luks to the I/O test suites

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> This introduces new suites for running I/O tests on NBD and LUKS
> drivers, giving new make targets
> 
>   * make check-block-luks
>   * make check-block-nbd
> 
> as well as adding their tests to 'make check-block SPEED=thorough'
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   docs/devel/testing/main.rst    | 4 ++--
>   tests/qemu-iotests/meson.build | 4 +++-
>   2 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> index ec29cf7708..910ec11ab4 100644
> --- a/docs/devel/testing/main.rst
> +++ b/docs/devel/testing/main.rst
> @@ -235,8 +235,8 @@ formats. The most comprehensive test plan can be run by defining
>   ``SPEED=thorough``, which enables all available tests for every format.
>   
>   This set of formats currently enabled for make integration are

Should we change the wording now to "formats/protocols" since "luks" is 
rather a protocol than a format?

> -``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
> -``vmdk``, and ``vpc``.
> +``luks``, ``nbd``, ``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``,
> +``vhdx``, ``vmdk``, and ``vpc``.
>   
>   Each of formats also has its own dedicated make target, named
>   ``make check-block-$FORMAT`` which will run all available tests for
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index 66b09d6b97..744d0b6e88 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -15,12 +15,14 @@ qemu_iotests_env = {'PYTHON': python.full_path()}
>   qemu_iotests_formats = {

And change it to "qemu_iotests_blockdrivers" (or just "blockdrivers") now?

>     'qcow2': 'quick',
>     'raw': 'slow',
> +  'luks': 'thorough',
> +  'nbd': 'thorough',
>     'parallels': 'thorough',
>     'qed': 'thorough',
>     'vdi': 'thorough',
>     'vhdx': 'thorough',
>     'vmdk': 'thorough',
> -  'vpc': 'thorough'
> +  'vpc': 'thorough',
>   }
>   
>   foreach k, v : emulators

  Thomas

Re: [PATCH v2 09/13] tests: add nbd and luks to the I/O test suites

[Daniel P. Berrangé]

On Tue, Jan 13, 2026 at 01:52:45PM +0100, Thomas Huth wrote:
> On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> > This introduces new suites for running I/O tests on NBD and LUKS
> > drivers, giving new make targets
> > 
> >   * make check-block-luks
> >   * make check-block-nbd
> > 
> > as well as adding their tests to 'make check-block SPEED=thorough'
> > 
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >   docs/devel/testing/main.rst    | 4 ++--
> >   tests/qemu-iotests/meson.build | 4 +++-
> >   2 files changed, 5 insertions(+), 3 deletions(-)
> > 
> > diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> > index ec29cf7708..910ec11ab4 100644
> > --- a/docs/devel/testing/main.rst
> > +++ b/docs/devel/testing/main.rst
> > @@ -235,8 +235,8 @@ formats. The most comprehensive test plan can be run by defining
> >   ``SPEED=thorough``, which enables all available tests for every format.
> >   This set of formats currently enabled for make integration are
> 
> Should we change the wording now to "formats/protocols" since "luks" is
> rather a protocol than a format?

Actually 'luks' is a format, but 'nbd' is a protocol.

> 
> > -``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``, ``vhdx``,
> > -``vmdk``, and ``vpc``.
> > +``luks``, ``nbd``, ``parallels``, ``qcow2``, ``qed``, ``raw``, ``vdi``,
> > +``vhdx``, ``vmdk``, and ``vpc``.
> >   Each of formats also has its own dedicated make target, named
> >   ``make check-block-$FORMAT`` which will run all available tests for
> > diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> > index 66b09d6b97..744d0b6e88 100644
> > --- a/tests/qemu-iotests/meson.build
> > +++ b/tests/qemu-iotests/meson.build
> > @@ -15,12 +15,14 @@ qemu_iotests_env = {'PYTHON': python.full_path()}
> >   qemu_iotests_formats = {
> 
> And change it to "qemu_iotests_blockdrivers" (or just "blockdrivers") now?

Yes, I'll do this renaming in docs/makefiles/meson in a separate commit,
using 'driver' as a generic term to cover either protocols or formats.

> 
> >     'qcow2': 'quick',
> >     'raw': 'slow',
> > +  'luks': 'thorough',
> > +  'nbd': 'thorough',
> >     'parallels': 'thorough',
> >     'qed': 'thorough',
> >     'vdi': 'thorough',
> >     'vhdx': 'thorough',
> >     'vmdk': 'thorough',
> > -  'vpc': 'thorough'
> > +  'vpc': 'thorough',
> >   }
> >   foreach k, v : emulators
> 
>  Thomas
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v2 10/13] tests: skip I/O test 128 on GitLab due to broken dmsetup

[Daniel P. Berrangé]

The I/O test 128 uses 'dmsetup create' to create a device, optionally
using sudo to elevate privileges.

This works in GitLab CI, however, the resulting device never appears
in /dev leading the test to fail:

  1..1
  # running raw 128
  not ok raw 128
  ----------------------------------- stderr -----------------------------------
  --- /builds/berrange/qemu/tests/qemu-iotests/128.out
  +++ /builds/berrange/qemu/build/scratch/raw-file-128/128.out.bad
  @@ -1,5 +1,5 @@
   QA output created by 128

   == reading from error device ==
  -read failed: Input/output error
  +qemu-io: can't open device /dev/mapper/eiodev16546: Could not open '/dev/mapper/eiodev16546': No such file or directory
   *** done

  (test program exited with status code 1)

Most likely the container environment the tests are being run under
has a hand-populated /dev such that newly created devices from device
mapper won't automatically appear.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/128 | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/qemu-iotests/128 b/tests/qemu-iotests/128
index d0e00d24b1..b32e03beee 100755
--- a/tests/qemu-iotests/128
+++ b/tests/qemu-iotests/128
@@ -73,6 +73,8 @@ _supported_fmt raw
 _supported_proto file
 _supported_os Linux
 
+test -n "$GITLAB_CI" && _notrun "dmsetup not functional on GitLab CI runners"
+
 _setup_eiodev
 
 TEST_IMG="/dev/mapper/$devname"
-- 
2.52.0

Re: [PATCH v2 10/13] tests: skip I/O test 128 on GitLab due to broken dmsetup

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> The I/O test 128 uses 'dmsetup create' to create a device, optionally
> using sudo to elevate privileges.
> 
> This works in GitLab CI, however, the resulting device never appears
> in /dev leading the test to fail:
> 
>    1..1
>    # running raw 128
>    not ok raw 128
>    ----------------------------------- stderr -----------------------------------
>    --- /builds/berrange/qemu/tests/qemu-iotests/128.out
>    +++ /builds/berrange/qemu/build/scratch/raw-file-128/128.out.bad
>    @@ -1,5 +1,5 @@
>     QA output created by 128
> 
>     == reading from error device ==
>    -read failed: Input/output error
>    +qemu-io: can't open device /dev/mapper/eiodev16546: Could not open '/dev/mapper/eiodev16546': No such file or directory
>     *** done
> 
>    (test program exited with status code 1)
> 
> Most likely the container environment the tests are being run under
> has a hand-populated /dev such that newly created devices from device
> mapper won't automatically appear.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/qemu-iotests/128 | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/tests/qemu-iotests/128 b/tests/qemu-iotests/128
> index d0e00d24b1..b32e03beee 100755
> --- a/tests/qemu-iotests/128
> +++ b/tests/qemu-iotests/128
> @@ -73,6 +73,8 @@ _supported_fmt raw
>   _supported_proto file
>   _supported_os Linux
>   
> +test -n "$GITLAB_CI" && _notrun "dmsetup not functional on GitLab CI runners"
> +
>   _setup_eiodev
>   
>   TEST_IMG="/dev/mapper/$devname"

Could we maybe better check for the existance of $TEST_IMG for deciding 
whether to skip the test? ... so that this is also properly skipped in other 
container environments where /dev might not be populated right?

  Thomas

[PATCH v2 11/13] tests: fix check for sudo access in LUKS I/O test

[Daniel P. Berrangé]

The test did not cope with the possibility that 'sudo' was not installed
at all, merely that it was not configured. This broke tests in any CI
env which lacks 'sudo'.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/149 | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149
index c13343d7ef..6dff39a28a 100755
--- a/tests/qemu-iotests/149
+++ b/tests/qemu-iotests/149
@@ -95,11 +95,14 @@ def verify_passwordless_sudo():
 
     args = ["sudo", "-n", "/bin/true"]
 
-    proc = subprocess.Popen(args,
-                            stdin=subprocess.PIPE,
-                            stdout=subprocess.PIPE,
-                            stderr=subprocess.STDOUT,
-                            universal_newlines=True)
+    try:
+        proc = subprocess.Popen(args,
+                                stdin=subprocess.PIPE,
+                                stdout=subprocess.PIPE,
+                                stderr=subprocess.STDOUT,
+                                universal_newlines=True)
+    except FileNotFoundError as e:
+        iotests.notrun('requires sudo binary: %s' % e)
 
     msg = proc.communicate()[0]
 
-- 
2.52.0

Re: [PATCH v2 11/13] tests: fix check for sudo access in LUKS I/O test

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> The test did not cope with the possibility that 'sudo' was not installed
> at all, merely that it was not configured. This broke tests in any CI
> env which lacks 'sudo'.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/qemu-iotests/149 | 13 ++++++++-----
>   1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149
> index c13343d7ef..6dff39a28a 100755
> --- a/tests/qemu-iotests/149
> +++ b/tests/qemu-iotests/149
> @@ -95,11 +95,14 @@ def verify_passwordless_sudo():
>   
>       args = ["sudo", "-n", "/bin/true"]
>   
> -    proc = subprocess.Popen(args,
> -                            stdin=subprocess.PIPE,
> -                            stdout=subprocess.PIPE,
> -                            stderr=subprocess.STDOUT,
> -                            universal_newlines=True)
> +    try:
> +        proc = subprocess.Popen(args,
> +                                stdin=subprocess.PIPE,
> +                                stdout=subprocess.PIPE,
> +                                stderr=subprocess.STDOUT,
> +                                universal_newlines=True)
> +    except FileNotFoundError as e:
> +        iotests.notrun('requires sudo binary: %s' % e)
>   
>       msg = proc.communicate()[0]
>   

Reviewed-by: Thomas Huth <thuth@redhat.com>

[PATCH v2 12/13] tests: add QEMU_TEST_IO_SKIP for skipping I/O tests

[Daniel P. Berrangé]

The nature of block I/O tests is such that there can be unexpected false
positive failures in certain scenarios that have not been encountered
before, and sometimes non-deterministic failures that are hard to
reproduce.

Before enabling the I/O tests as gating jobs in CI, there needs to be a
mechanism to dynamically mark tests as skipped, without having to commit
code changes.

This introduces the QEMU_TEST_IO_SKIP environment variable that is set
to a list of FORMAT-OR-PROTOCOL:NAME pairs. The intent is that this
variable can be set as a GitLab CI pipeline variable to temporarily
disable a test while problems are being debugged.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 docs/devel/testing/main.rst      |  7 +++++++
 tests/qemu-iotests/testrunner.py | 16 ++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index 910ec11ab4..bb90ea5f15 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -285,6 +285,13 @@ that are specific to certain cache mode.
 More options are supported by the ``./check`` script, run ``./check -h`` for
 help.
 
+If a test program is known to be broken, it can be disabled by setting
+the ``QEMU_TEST_IO_SKIP`` environment variable with a list of tests to
+be skipped. The values are of the form FORMAT-OR-PROTOCOL:NAME, the
+leading component can be omitted to skip the test for all formats and
+protocols. For example ``export QEMU_TEST_IO_SKIP="luks:149 185 iov-padding``
+will skip for LUKS only, and ``185`` and ``iov-padding`` for all.
+
 Writing a new test case
 ~~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index dbe2dddc32..ecb5d4529f 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -145,6 +145,18 @@ def __init__(self, env: TestEnv, tap: bool = False,
 
         self._stack: contextlib.ExitStack
 
+        self.skip = {}
+        for rule in os.environ.get("QEMU_TEST_IO_SKIP", "").split(" "):
+            rule = rule.strip()
+            if rule == "":
+                continue
+            if ":" in rule:
+                fmt, name = rule.split(":")
+                if fmt in ("", env.imgfmt, env.imgproto):
+                    self.skip[name] = True
+            else:
+                self.skip[rule] = True
+
     def __enter__(self) -> 'TestRunner':
         self._stack = contextlib.ExitStack()
         self._stack.enter_context(self.env)
@@ -251,6 +263,10 @@ def do_run_test(self, test: str) -> TestResult:
                               description='No qualified output '
                                           f'(expected {f_reference})')
 
+        if f_test.name in self.skip:
+            return TestResult(status='not run',
+                              description='Listed in QEMU_TEST_IO_SKIP')
+
         args = [str(f_test.resolve())]
         env = self.env.prepare_subprocess(args)
 
-- 
2.52.0

Re: [PATCH v2 12/13] tests: add QEMU_TEST_IO_SKIP for skipping I/O tests

[Thomas Huth]

On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> The nature of block I/O tests is such that there can be unexpected false
> positive failures in certain scenarios that have not been encountered
> before, and sometimes non-deterministic failures that are hard to
> reproduce.
> 
> Before enabling the I/O tests as gating jobs in CI, there needs to be a
> mechanism to dynamically mark tests as skipped, without having to commit
> code changes.
> 
> This introduces the QEMU_TEST_IO_SKIP environment variable that is set
> to a list of FORMAT-OR-PROTOCOL:NAME pairs. The intent is that this
> variable can be set as a GitLab CI pipeline variable to temporarily
> disable a test while problems are being debugged.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   docs/devel/testing/main.rst      |  7 +++++++
>   tests/qemu-iotests/testrunner.py | 16 ++++++++++++++++
>   2 files changed, 23 insertions(+)
> 
> diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> index 910ec11ab4..bb90ea5f15 100644
> --- a/docs/devel/testing/main.rst
> +++ b/docs/devel/testing/main.rst
> @@ -285,6 +285,13 @@ that are specific to certain cache mode.
>   More options are supported by the ``./check`` script, run ``./check -h`` for
>   help.
>   
> +If a test program is known to be broken, it can be disabled by setting
> +the ``QEMU_TEST_IO_SKIP`` environment variable with a list of tests to
> +be skipped. The values are of the form FORMAT-OR-PROTOCOL:NAME, the
> +leading component can be omitted to skip the test for all formats and
> +protocols. For example ``export QEMU_TEST_IO_SKIP="luks:149 185 iov-padding``
> +will skip for LUKS only, and ``185`` and ``iov-padding`` for all.

"will skip 149 for LUKS" ?

Apart from that:
Reviewed-by: Thomas Huth <thuth@redhat.com>

[PATCH v2 13/13] gitlab: add jobs for thorough block tests

[Daniel P. Berrangé]

CI is only exercising the qcow2 'auto' tests currently. As a result we
get no exposure of changes which cause regressions in other block format
drivers.

This adds new CI jobs for each block format, that will run the target
'make check-block-$FORMAT'. The jobs are separate so that we have the
ability to make each format gating or not, depending on their level
of reliability.

There is currently an undiagnosed failure of job 185 with the qcow2 when
run in CI that is marked to be skipped, reported at:

  https://gitlab.com/qemu-project/qemu/-/issues/3270

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/buildtest.yml | 92 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index dfe954fe3c..91b398bd63 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -176,6 +176,98 @@ build-system-centos:
       x86_64-softmmu rx-softmmu sh4-softmmu
     MAKE_CHECK_ARGS: check-build
 
+block-luks-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-luks
+
+block-nbd-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-nbd
+
+block-parallels-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-parallels
+
+block-qcow2-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-qcow2
+    # https://gitlab.com/qemu-project/qemu/-/issues/3270
+    QEMU_TEST_IO_SKIP: qcow2:185
+
+block-qed-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-qed
+
+block-raw-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-raw
+
+block-vdi-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-vdi
+
+block-vhdx-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-vhdx
+
+block-vmdk-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-vmdk
+
+block-vpc-centos:
+  extends: .native_test_job_template
+  needs:
+    - job: build-system-centos
+      artifacts: true
+  variables:
+    IMAGE: centos9
+    MAKE_CHECK_ARGS: check-block-vpc
+
 # Previous QEMU release. Used for cross-version migration tests.
 build-previous-qemu:
   extends: .native_build_job_template
-- 
2.52.0