[PATCH] docs/devel/testing: iotests check script should be run via 'run'

[PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Peter Maydell]

The iotests section of the testing documentation suggests directly
running the "./check" script.  However if you do this then the
script complains:

WARNING: iotests is being run from outside of the configure-time python virtual environment
[...]
Please use the meson run script:
        /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check

Update the documentation to match this.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/devel/testing/main.rst | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
index b0ca37fe2ba..e26b8af0c79 100644
--- a/docs/devel/testing/main.rst
+++ b/docs/devel/testing/main.rst
@@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
 test files are named with numbers.
 
 To run iotests, make sure QEMU is built successfully, then switch to the
-``tests/qemu-iotests`` directory under the build directory, and run ``./check``
-with desired arguments from there.
+``tests/qemu-iotests`` directory under the build directory, and run
+the ``check`` script via the ``run`` wrapper which sets up the environment:
+
+.. code::
+
+   ../../run ./check [args]
 
 By default, "raw" format and "file" protocol is used; all tests will be
 executed, except the unsupported ones. You can override the format and protocol
@@ -282,16 +286,16 @@ with arguments:
 .. code::
 
   # test with qcow2 format
-  ./check -qcow2
+  ../../run ./check -qcow2
   # or test a different protocol
-  ./check -nbd
+  ../../run ./check -nbd
 
 It's also possible to list test numbers explicitly:
 
 .. code::
 
   # run selected cases with qcow2 format
-  ./check -qcow2 001 030 153
+  ../../run ./check -qcow2 001 030 153
 
 Cache mode can be selected with the "-c" option, which may help reveal bugs
 that are specific to certain cache mode.
-- 
2.43.0

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> The iotests section of the testing documentation suggests directly
> running the "./check" script.  However if you do this then the
> script complains:
>
> WARNING: iotests is being run from outside of the configure-time python virtual environment
> [...]
> Please use the meson run script:
>         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
>
> Update the documentation to match this.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
>  docs/devel/testing/main.rst | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> index b0ca37fe2ba..e26b8af0c79 100644
> --- a/docs/devel/testing/main.rst
> +++ b/docs/devel/testing/main.rst
> @@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
>  test files are named with numbers.
>  
>  To run iotests, make sure QEMU is built successfully, then switch to the
> -``tests/qemu-iotests`` directory under the build directory, and run ``./check``
> -with desired arguments from there.
> +``tests/qemu-iotests`` directory under the build directory, and run
> +the ``check`` script via the ``run`` wrapper which sets up the environment:
> +
> +.. code::
> +
> +   ../../run ./check [args]
>

This is possibly a personal preference thing but is there actually a
reason for cwd to be tests/qemu-iotests or should we frame it as:

  $BLD/run tests/qemu-iotests/check

which is how the functional tests are listed?


>  By default, "raw" format and "file" protocol is used; all tests will be
>  executed, except the unsupported ones. You can override the format and protocol
> @@ -282,16 +286,16 @@ with arguments:
>  .. code::
>  
>    # test with qcow2 format
> -  ./check -qcow2
> +  ../../run ./check -qcow2
>    # or test a different protocol
> -  ./check -nbd
> +  ../../run ./check -nbd
>  
>  It's also possible to list test numbers explicitly:
>  
>  .. code::
>  
>    # run selected cases with qcow2 format
> -  ./check -qcow2 001 030 153
> +  ../../run ./check -qcow2 001 030 153
>  
>  Cache mode can be selected with the "-c" option, which may help reveal bugs
>  that are specific to certain cache mode.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Peter Maydell]

On Tue, 3 Nov 2026 at 12:22, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > The iotests section of the testing documentation suggests directly
> > running the "./check" script.  However if you do this then the
> > script complains:
> >
> > WARNING: iotests is being run from outside of the configure-time python virtual environment
> > [...]
> > Please use the meson run script:
> >         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
> >
> > Update the documentation to match this.
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> > ---
> >  docs/devel/testing/main.rst | 14 +++++++++-----
> >  1 file changed, 9 insertions(+), 5 deletions(-)
> >
> > diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> > index b0ca37fe2ba..e26b8af0c79 100644
> > --- a/docs/devel/testing/main.rst
> > +++ b/docs/devel/testing/main.rst
> > @@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
> >  test files are named with numbers.
> >
> >  To run iotests, make sure QEMU is built successfully, then switch to the
> > -``tests/qemu-iotests`` directory under the build directory, and run ``./check``
> > -with desired arguments from there.
> > +``tests/qemu-iotests`` directory under the build directory, and run
> > +the ``check`` script via the ``run`` wrapper which sets up the environment:
> > +
> > +.. code::
> > +
> > +   ../../run ./check [args]
> >
>
> This is possibly a personal preference thing but is there actually a
> reason for cwd to be tests/qemu-iotests or should we frame it as:
>
>   $BLD/run tests/qemu-iotests/check
>
> which is how the functional tests are listed?

No idea; I'm just assuming the docs have a reason for telling
you to do that, and not changing it.

-- PMM

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Pierrick Bouvier]

On 11/3/2026 3:08 AM, Peter Maydell wrote:
> The iotests section of the testing documentation suggests directly
> running the "./check" script.  However if you do this then the
> script complains:
> 
> WARNING: iotests is being run from outside of the configure-time python virtual environment
> [...]
> Please use the meson run script:
>         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
>

A bit a side question,

Does it bring anything useful compared to:
./build/run meson test <this_test or suite>?
If not, maybe we can remove all this, including the extra doc.

We can also add test suites if that's the missing piece to run a
specific group of tests.

> Update the documentation to match this.
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
>  docs/devel/testing/main.rst | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
> 
> diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> index b0ca37fe2ba..e26b8af0c79 100644
> --- a/docs/devel/testing/main.rst
> +++ b/docs/devel/testing/main.rst
> @@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
>  test files are named with numbers.
>  
>  To run iotests, make sure QEMU is built successfully, then switch to the
> -``tests/qemu-iotests`` directory under the build directory, and run ``./check``
> -with desired arguments from there.
> +``tests/qemu-iotests`` directory under the build directory, and run
> +the ``check`` script via the ``run`` wrapper which sets up the environment:
> +
> +.. code::
> +
> +   ../../run ./check [args]
>  
>  By default, "raw" format and "file" protocol is used; all tests will be
>  executed, except the unsupported ones. You can override the format and protocol
> @@ -282,16 +286,16 @@ with arguments:
>  .. code::
>  
>    # test with qcow2 format
> -  ./check -qcow2
> +  ../../run ./check -qcow2
>    # or test a different protocol
> -  ./check -nbd
> +  ../../run ./check -nbd
>  
>  It's also possible to list test numbers explicitly:
>  
>  .. code::
>  
>    # run selected cases with qcow2 format
> -  ./check -qcow2 001 030 153
> +  ../../run ./check -qcow2 001 030 153
>  
>  Cache mode can be selected with the "-c" option, which may help reveal bugs
>  that are specific to certain cache mode.

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Peter Maydell]

On Thu, 5 Nov 2026 at 08:16, Pierrick Bouvier
<pierrick.bouvier@oss.qualcomm.com> wrote:
>
> On 11/3/2026 3:08 AM, Peter Maydell wrote:
> > The iotests section of the testing documentation suggests directly
> > running the "./check" script.  However if you do this then the
> > script complains:
> >
> > WARNING: iotests is being run from outside of the configure-time python virtual environment
> > [...]
> > Please use the meson run script:
> >         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
> >
>
> A bit a side question,
>
> Does it bring anything useful compared to:
> ./build/run meson test <this_test or suite>?
> If not, maybe we can remove all this, including the extra doc.
>
> We can also add test suites if that's the missing piece to run a
> specific group of tests.

You'd need to ask the iotests maintainers for that kind of question.
All I'm doing is following the instructions for "how do I run
the tests on block backend X".

-- PMM

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Daniel P. Berrangé]

On Thu, Nov 05, 2026 at 12:16:14AM -0800, Pierrick Bouvier wrote:
> On 11/3/2026 3:08 AM, Peter Maydell wrote:
> > The iotests section of the testing documentation suggests directly
> > running the "./check" script.  However if you do this then the
> > script complains:
> > 
> > WARNING: iotests is being run from outside of the configure-time python virtual environment
> > [...]
> > Please use the meson run script:
> >         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
> >
> 
> A bit a side question,
> 
> Does it bring anything useful compared to:
> ./build/run meson test <this_test or suite>?
> If not, maybe we can remove all this, including the extra doc.

Being able to run without meson in the middle makes debugging
problematic tests simpler IME.

We have direct execution capabilities for all our tests sets
(unit, qtest, functional, block) that devs can use when debugging
an individual failing test, so retaining docs for this is worthwhile.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Daniel P. Berrangé]

On Tue, Nov 03, 2026 at 12:22:50PM +0000, Alex Bennée wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
> > The iotests section of the testing documentation suggests directly
> > running the "./check" script.  However if you do this then the
> > script complains:
> >
> > WARNING: iotests is being run from outside of the configure-time python virtual environment
> > [...]
> > Please use the meson run script:
> >         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
> >
> > Update the documentation to match this.
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> > ---
> >  docs/devel/testing/main.rst | 14 +++++++++-----
> >  1 file changed, 9 insertions(+), 5 deletions(-)
> >
> > diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> > index b0ca37fe2ba..e26b8af0c79 100644
> > --- a/docs/devel/testing/main.rst
> > +++ b/docs/devel/testing/main.rst
> > @@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
> >  test files are named with numbers.
> >  
> >  To run iotests, make sure QEMU is built successfully, then switch to the
> > -``tests/qemu-iotests`` directory under the build directory, and run ``./check``
> > -with desired arguments from there.
> > +``tests/qemu-iotests`` directory under the build directory, and run
> > +the ``check`` script via the ``run`` wrapper which sets up the environment:
> > +
> > +.. code::
> > +
> > +   ../../run ./check [args]
> >
> 
> This is possibly a personal preference thing but is there actually a
> reason for cwd to be tests/qemu-iotests or should we frame it as:
> 
>   $BLD/run tests/qemu-iotests/check
> 
> which is how the functional tests are listed?

For the sake of docs it is probably a good idea to be run consistent
and illustrate use of 'run' from the root of the build  dir ie

  ./run tests/qemu-iotests/check

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Peter Maydell]

On Thu, 5 Nov 2026 at 09:34, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Nov 03, 2026 at 12:22:50PM +0000, Alex Bennée wrote:
> > Peter Maydell <peter.maydell@linaro.org> writes:
> >
> > > The iotests section of the testing documentation suggests directly
> > > running the "./check" script.  However if you do this then the
> > > script complains:
> > >
> > > WARNING: iotests is being run from outside of the configure-time python virtual environment
> > > [...]
> > > Please use the meson run script:
> > >         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
> > >
> > > Update the documentation to match this.
> > >
> > > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> > > ---
> > >  docs/devel/testing/main.rst | 14 +++++++++-----
> > >  1 file changed, 9 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> > > index b0ca37fe2ba..e26b8af0c79 100644
> > > --- a/docs/devel/testing/main.rst
> > > +++ b/docs/devel/testing/main.rst
> > > @@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
> > >  test files are named with numbers.
> > >
> > >  To run iotests, make sure QEMU is built successfully, then switch to the
> > > -``tests/qemu-iotests`` directory under the build directory, and run ``./check``
> > > -with desired arguments from there.
> > > +``tests/qemu-iotests`` directory under the build directory, and run
> > > +the ``check`` script via the ``run`` wrapper which sets up the environment:
> > > +
> > > +.. code::
> > > +
> > > +   ../../run ./check [args]
> > >
> >
> > This is possibly a personal preference thing but is there actually a
> > reason for cwd to be tests/qemu-iotests or should we frame it as:
> >
> >   $BLD/run tests/qemu-iotests/check
> >
> > which is how the functional tests are listed?
>
> For the sake of docs it is probably a good idea to be run consistent
> and illustrate use of 'run' from the root of the build  dir ie
>
>   ./run tests/qemu-iotests/check

Agreed if it works for all cases. Does it? I assumed the existing docs
were saying that it should be run from the qemu-iotests subdirectory
for a reason.

-- PMM

Re: [PATCH] docs/devel/testing: iotests check script should be run via 'run'

[Daniel P. Berrangé]

On Thu, Nov 05, 2026 at 10:20:07AM +0000, Peter Maydell wrote:
> On Thu, 5 Nov 2026 at 09:34, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Tue, Nov 03, 2026 at 12:22:50PM +0000, Alex Bennée wrote:
> > > Peter Maydell <peter.maydell@linaro.org> writes:
> > >
> > > > The iotests section of the testing documentation suggests directly
> > > > running the "./check" script.  However if you do this then the
> > > > script complains:
> > > >
> > > > WARNING: iotests is being run from outside of the configure-time python virtual environment
> > > > [...]
> > > > Please use the meson run script:
> > > >         /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check
> > > >
> > > > Update the documentation to match this.
> > > >
> > > > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> > > > ---
> > > >  docs/devel/testing/main.rst | 14 +++++++++-----
> > > >  1 file changed, 9 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst
> > > > index b0ca37fe2ba..e26b8af0c79 100644
> > > > --- a/docs/devel/testing/main.rst
> > > > +++ b/docs/devel/testing/main.rst
> > > > @@ -272,8 +272,12 @@ scripts.  The testing success criteria is golden output comparison, and the
> > > >  test files are named with numbers.
> > > >
> > > >  To run iotests, make sure QEMU is built successfully, then switch to the
> > > > -``tests/qemu-iotests`` directory under the build directory, and run ``./check``
> > > > -with desired arguments from there.
> > > > +``tests/qemu-iotests`` directory under the build directory, and run
> > > > +the ``check`` script via the ``run`` wrapper which sets up the environment:
> > > > +
> > > > +.. code::
> > > > +
> > > > +   ../../run ./check [args]
> > > >
> > >
> > > This is possibly a personal preference thing but is there actually a
> > > reason for cwd to be tests/qemu-iotests or should we frame it as:
> > >
> > >   $BLD/run tests/qemu-iotests/check
> > >
> > > which is how the functional tests are listed?
> >
> > For the sake of docs it is probably a good idea to be run consistent
> > and illustrate use of 'run' from the root of the build  dir ie
> >
> >   ./run tests/qemu-iotests/check
> 
> Agreed if it works for all cases. Does it? I assumed the existing docs
> were saying that it should be run from the qemu-iotests subdirectory
> for a reason.

I don't think there is a hard constraint on CWD for the iotests, as
I've often run from the root of the build dir and not seen failures.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|