All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Huth <thuth@redhat.com>
To: "Alex Bennée" <alex.bennee@linaro.org>,
	qemu-devel@nongnu.org,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>
Cc: Ani Sinha <anisinha@redhat.com>,
	Richard Henderson <richard.henderson@linaro.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	"Daniel P . Berrange" <berrange@redhat.com>,
	John Snow <jsnow@redhat.com>,
	qemu-ppc@nongnu.org, Fabiano Rosas <farosas@suse.de>
Subject: [PATCH v2 04/23] tests/functional: Prepare the meson build system for the functional tests
Date: Wed, 24 Jul 2024 19:52:22 +0200	[thread overview]
Message-ID: <20240724175248.1389201-5-thuth@redhat.com> (raw)
In-Reply-To: <20240724175248.1389201-1-thuth@redhat.com>

Provide a meson.build file for the upcoming python-based functional
tests, and add some wrapper glue targets to the tests/Makefile.include
file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"),
while the "thorough" tests should only be run when running a
"make check-functional" test run (since these tests might download
additional assets from the internet).

The changes to the meson.build files are partly based on an earlier
patch by Ani Sinha.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include       | 11 ++++++
 tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
 tests/meson.build            |  1 +
 3 files changed, 78 insertions(+)
 create mode 100644 tests/functional/meson.build

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 167d8f29a4..d0c2ae30b7 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -9,6 +9,8 @@ check-help:
 	@echo "Individual test suites:"
 	@echo " $(MAKE) check-qtest-TARGET       Run qtest tests for given target"
 	@echo " $(MAKE) check-qtest              Run qtest tests"
+	@echo " $(MAKE) check-functional         Run python-based functional tests"
+	@echo " $(MAKE) check-functional-TARGET  Run functional tests for a given target"
 	@echo " $(MAKE) check-unit               Run qobject tests"
 	@echo " $(MAKE) check-qapi-schema        Run QAPI schema tests"
 	@echo " $(MAKE) check-block              Run block tests"
@@ -152,6 +154,15 @@ check-acceptance-deprecated-warning:
 
 check-acceptance: check-acceptance-deprecated-warning | check-avocado
 
+FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
+.PHONY: $(FUNCTIONAL_TARGETS)
+$(FUNCTIONAL_TARGETS):
+	@$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
+
+.PHONY: check-functional
+check-functional:
+	@$(MAKE) SPEED=thorough check-func check-func-quick
+
 # Consolidated targets
 
 .PHONY: check check-clean get-vm-images
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
new file mode 100644
index 0000000000..7fc9c4e513
--- /dev/null
+++ b/tests/functional/meson.build
@@ -0,0 +1,66 @@
+# QEMU functional tests:
+# Tests that are put in the 'quick' category are run by default during
+# 'make check'. Everything that should not be run during 'make check'
+# (e.g. tests that fetch assets from the internet) should be put into
+# the 'thorough' category instead.
+
+# Most tests run too slow with TCI enabled, so skip the functional tests there
+if get_option('tcg_interpreter')
+  subdir_done()
+endif
+
+# Timeouts for individual tests that can be slow e.g. with debugging enabled
+test_timeouts = {
+}
+
+tests_generic = [
+]
+
+tests_x86_64_quick = [
+]
+
+tests_x86_64_thorough = [
+]
+
+foreach speed : ['quick', 'thorough']
+  foreach dir : target_dirs
+    if not dir.endswith('-softmmu')
+      continue
+    endif
+
+    target_base = dir.split('-')[0]
+    test_emulator = emulators['qemu-system-' + target_base]
+
+    if speed == 'quick'
+      suites = ['func-quick', 'func-' + target_base]
+      target_tests = get_variable('tests_' + target_base + '_quick', []) + tests_generic
+    else
+      suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
+      target_tests = get_variable('tests_' + target_base + '_' + speed, [])
+    endif
+
+    test_deps = roms
+    test_env = environment()
+    if have_tools
+      test_env.set('QEMU_TEST_QEMU_IMG', meson.global_build_root() / 'qemu-img')
+      test_deps += [qemu_img]
+    endif
+    test_env.set('QEMU_TEST_QEMU_BINARY',
+                 meson.global_build_root() / 'qemu-system-' + target_base)
+    test_env.set('QEMU_BUILD_ROOT', meson.project_build_root())
+    test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
+                               meson.current_source_dir())
+
+    foreach test : target_tests
+      test('func-@0@/@1@'.format(target_base, test),
+           python,
+           depends: [test_deps, test_emulator, emulator_modules],
+           env: test_env,
+           args: [meson.current_source_dir() / 'test_' + test + '.py'],
+           protocol: 'tap',
+           timeout: test_timeouts.get(test, 60),
+           priority: test_timeouts.get(test, 60),
+           suite: suites)
+    endforeach
+  endforeach
+endforeach
diff --git a/tests/meson.build b/tests/meson.build
index acb6807094..3345ad2098 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -85,3 +85,4 @@ subdir('unit')
 subdir('qapi-schema')
 subdir('qtest')
 subdir('migration')
+subdir('functional')
-- 
2.45.2



  parent reply	other threads:[~2024-07-24 17:56 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-24 17:52 [PATCH v2 00/23] Convert avocado tests to normal Python unittests Thomas Huth
2024-07-24 17:52 ` [PATCH v2 01/23] python: Install pycotap in our venv if necessary Thomas Huth
2024-07-29 12:34   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 02/23] tests/functional: Add base classes for the upcoming pytest-based tests Thomas Huth
2024-07-29 12:35   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 03/23] tests/Makefile.include: Increase the level of indentation in the help text Thomas Huth
2024-07-25 11:58   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` Thomas Huth [this message]
2024-07-29 12:36   ` [PATCH v2 04/23] tests/functional: Prepare the meson build system for the functional tests Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 05/23] tests/functional: Convert simple avocado tests into standalone python tests Thomas Huth
2024-07-25  6:55   ` Philippe Mathieu-Daudé
2024-07-25 11:56     ` Philippe Mathieu-Daudé
2024-07-25 11:58   ` Philippe Mathieu-Daudé
2024-07-29 12:37     ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 06/23] tests/functional: Convert avocado tests that just need a small adjustment Thomas Huth
2024-07-25 12:04   ` Philippe Mathieu-Daudé
2024-07-29 12:37     ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 07/23] tests/functional: Implement fetch_asset() method for downloading assets Thomas Huth
2024-07-24 17:52 ` [PATCH v2 08/23] tests/functional: Convert some tests that download files via fetch_asset() Thomas Huth
2024-07-25 12:05   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 09/23] tests/functional: Add a function for extracting files from an archive Thomas Huth
2024-07-25 11:51   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 10/23] tests/functional: Convert some avocado tests that needed avocado.utils.archive Thomas Huth
2024-07-24 17:52 ` [PATCH v2 11/23] tests/functional: Set up logging Thomas Huth
2024-07-24 17:52 ` [PATCH v2 12/23] tests/functional: Convert the s390x avocado tests into standalone tests Thomas Huth
2024-07-24 17:52 ` [PATCH v2 13/23] tests/functional: Convert the x86_cpu_model_versions test Thomas Huth
2024-07-29 12:48   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 14/23] tests/functional: Convert the microblaze avocado tests into standalone tests Thomas Huth
2024-07-24 17:52 ` [PATCH v2 15/23] tests/functional: Convert the riscv_opensbi avocado test into a standalone test Thomas Huth
2024-07-25  5:44   ` Alistair Francis
2024-07-29 13:04   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 16/23] tests/functional: Convert the virtio_gpu " Thomas Huth
2024-07-24 17:52 ` [PATCH v2 17/23] tests/functional: Convert most ppc avocado tests into standalone tests Thomas Huth
2024-07-25 11:55   ` Philippe Mathieu-Daudé
2024-07-24 17:52 ` [PATCH v2 18/23] tests/functional: Convert the ppc_amiga avocado test into a standalone test Thomas Huth
2024-07-24 17:52 ` [PATCH v2 19/23] tests/functional: Convert the ppc_hv " Thomas Huth
2024-07-24 17:52 ` [PATCH v2 20/23] tests/functional: Convert the m68k nextcube test with tesseract Thomas Huth
2024-07-24 17:52 ` [PATCH v2 21/23] tests/functional: Convert the acpi-bits test into a standalone test Thomas Huth
2024-07-24 17:52 ` [PATCH v2 22/23] tests/functional: Convert the rx_gdbsim avocado " Thomas Huth
2024-07-24 17:52 ` [PATCH v2 23/23] gitlab-ci: Add "check-functional" to the build tests Thomas Huth
2024-07-24 23:35 ` [PATCH v2 00/23] Convert avocado tests to normal Python unittests Richard Henderson
2024-07-25  9:55   ` Daniel P. Berrangé
2024-07-25 10:42     ` Richard Henderson
2024-07-25 11:07       ` Daniel P. Berrangé
2024-07-26 13:03         ` Thomas Huth
2024-07-26 13:50           ` Cleber Rosa
2024-07-25 10:13   ` Thomas Huth
2024-07-25 10:50     ` Richard Henderson
2024-07-29 14:44   ` Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240724175248.1389201-5-thuth@redhat.com \
    --to=thuth@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=anisinha@redhat.com \
    --cc=berrange@redhat.com \
    --cc=farosas@suse.de \
    --cc=jsnow@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=richard.henderson@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.