Re: [PATCH v2 10/11] pytest: add pytest to the meson build system

[John Snow <jsnow@redhat.com>]

On Sun, Jul 10, 2022 at 1:01 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> Integrate the pytest framework with the meson build system. This will make meson
> run all the pytests under the pytest directory.
>
> Signed-off-by: Ani Sinha <ani@anisinha.ca>
> ---
>  tests/Makefile.include   |  4 +++-
>  tests/meson.build        |  1 +
>  tests/pytest/meson.build | 49 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 53 insertions(+), 1 deletion(-)
>  create mode 100644 tests/pytest/meson.build
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 3accb83b13..40755a6bd1 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -3,12 +3,14 @@
>  .PHONY: check-help
>  check-help:
>         @echo "Regression testing targets:"
> -       @echo " $(MAKE) check                  Run block, qapi-schema, unit, softfloat, qtest and decodetree tests"
> +       @echo " $(MAKE) check                  Run block, qapi-schema, unit, softfloat, qtest, pytest and decodetree tests"

Does this mean that "make check" *requires* an internet connection? If
so, I'm afraid that introduces some complications for downstreams
which require that "make check" can run without an internet
connection. It's something I've been trying to wrestle with as I split
the qemu.qmp library out of the QEMU tree, and I have been working
(slowly) on remedying it with some additional Makefile logic.

I have been looking into making a testing "dummy package" for python
with optional dependency groups that use a venv building script to
either pull from PyPI in online mode, or use system packages in
offline mode. In the case of offline mode, I am working on augmenting
the configure script to test that dependencies are met, and disabling
test groups when the correct dependencies cannot be located.

I hope to have another version of that series soon; it should be
trivial to add a new optional dependency group to it. I'll CC you on
the next version of the series.

--js

>         @echo " $(MAKE) bench                  Run speed tests"
>         @echo
>         @echo "Individual test suites:"
>         @echo " $(MAKE) check-qtest-TARGET     Run qtest tests for given target"
>         @echo " $(MAKE) check-qtest            Run qtest tests"
> +       @echo " $(MAKE) check-pytest           Run pytest tests"
> +       @echo " $(MAKE) check-pytest-TARGET    Run pytest for a given target"
>         @echo " $(MAKE) check-unit             Run qobject tests"
>         @echo " $(MAKE) check-qapi-schema      Run QAPI schema tests"
>         @echo " $(MAKE) check-block            Run block tests"
> diff --git a/tests/meson.build b/tests/meson.build
> index 8e318ec513..f344cbdc6c 100644
> --- a/tests/meson.build
> +++ b/tests/meson.build
> @@ -93,3 +93,4 @@ subdir('unit')
>  subdir('qapi-schema')
>  subdir('qtest')
>  subdir('migration')
> +subdir('pytest')
> diff --git a/tests/pytest/meson.build b/tests/pytest/meson.build
> new file mode 100644
> index 0000000000..e60d481ae4
> --- /dev/null
> +++ b/tests/pytest/meson.build
> @@ -0,0 +1,49 @@
> +slow_pytests = {
> +  'acpi-bits' : 120,
> +}
> +
> +pytests_generic = []
> +
> +# biosbits tests are currenly only supported on x86_64 platforms.
> +pytests_x86_64 = ['acpi-bits-test']
> +
> +pytest_executables = {}
> +other_deps = []
> +
> +subdir('acpi-bits')
> +
> +foreach dir : target_dirs
> +  if not dir.endswith('-softmmu')
> +    continue
> +  endif
> +
> +  target_base = dir.split('-')[0]
> +  pytest_emulator = emulators['qemu-system-' + target_base]
> +  target_pytests = get_variable('pytests_' + target_base, []) + pytests_generic
> +
> +  test_deps = roms
> +  pytest_env = environment()
> +  if have_tools
> +    pytest_env.set('QTEST_QEMU_IMG', './qemu-img')
> +    test_deps += [qemu_img]
> +  endif
> +  pytest_env.set('G_TEST_DBUS_DAEMON', meson.project_source_root() / 'tests/dbus-vmstate-daemon.sh')
> +  pytest_env.set('PYTEST_QEMU_BINARY', './qemu-system-' + target_base)
> +  pytest_env.set('PYTEST_SOURCE_ROOT', meson.project_source_root())
> +  if have_tools and have_vhost_user_blk_server
> +    pytest_env.set('PYTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon')
> +    test_deps += [qsd]
> +  endif
> +
> +  foreach test : target_pytests
> +    test('pytest-@0@/@1@'.format(target_base, test),
> +         pytest_executables[test],
> +         depends: [test_deps, pytest_emulator, emulator_modules, other_deps],
> +         env: pytest_env,
> +         args: ['--tap', '-k'],
> +         protocol: 'tap',
> +         timeout: slow_pytests.get(test, 30),
> +         priority: slow_pytests.get(test, 30),
> +         suite: ['pytest', 'pytest-' + target_base])
> +  endforeach
> +endforeach
> --
> 2.25.1
>