qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Fabiano Rosas <farosas@suse.de>
To: qemu-devel@nongnu.org
Cc: "Juan Quintela" <quintela@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Daniel P . Berrangé" <berrange@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Yonggang Luo" <luoyonggang@gmail.com>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Laurent Vivier" <lvivier@redhat.com>
Subject: [RFC PATCH 2/2] tests/qtest: Pass migration tests individually to meson
Date: Mon,  3 Jul 2023 11:47:00 -0300	[thread overview]
Message-ID: <20230703144700.13536-3-farosas@suse.de> (raw)
In-Reply-To: <20230703144700.13536-1-farosas@suse.de>

Having glib tests (qtest) all defined in a single test file makes it
hard to know which test has failed when running CI. Create a new
'migration' test suite and move the migration tests individually to
meson.

For now, use the global migration-test timeout value, but we could set
a per-subtest timeout in the future.

Sample output:

$ ../configure --target-list=x86_64-softmmu,aarch64-softmmu ...
$ make check-migration
...
36/469 qemu:migration / /x86_64/migration/precopy/unix/plain                       OK    34.25s   1 subtests passed
37/469 qemu:migration / /x86_64/migration/multifd/tcp/tls/x509/default-host        OK     7.33s   1 subtests passed
39/469 qemu:migration / /x86_64/migration/multifd/tcp/tls/x509/allow-anon-client   OK     7.32s   1 subtests passed
40/469 qemu:migration / /aarch64/migration/postcopy/compress/plain                 SKIP   0.04s
41/469 qemu:migration / /aarch64/migration/postcopy/recovery/compress/plain        SKIP   0.04s
42/469 qemu:migration / /aarch64/migration/bad_dest                                OK     0.65s   1 subtests passed

Signed-off-by: Fabiano Rosas <farosas@suse.de>
---
 .gitlab-ci.d/windows.yml |  2 +-
 tests/qtest/meson.build  | 40 +++++++++++++++++++++++++++++++---------
 2 files changed, 32 insertions(+), 10 deletions(-)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index f889a468b5..04bc74a20e 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -84,7 +84,7 @@ msys2-64bit:
   - ..\msys64\usr\bin\bash -lc 'make'
   # qTests don't run successfully with "--without-default-devices",
   # so let's exclude the qtests from CI for now.
-  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS=\"--no-suite qtest\" || { cat meson-logs/testlog.txt; exit 1; } ;'
+  - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS=\"--no-suite qtest migration\" || { cat meson-logs/testlog.txt; exit 1; } ;'
 
 msys2-32bit:
   extends: .shared_msys2_builder
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 5fa6833ad7..43c140921c 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -369,14 +369,36 @@ foreach dir : target_dirs
         test: executable(test, src, dependencies: deps)
       }
     endif
-    test('qtest-@0@/@1@'.format(target_base, test),
-         qtest_executables[test],
-         depends: [test_deps, qtest_emulator, emulator_modules],
-         env: qtest_env,
-         args: ['--tap', '-k'],
-         protocol: 'tap',
-         timeout: slow_qtests.get(test, 30),
-         priority: slow_qtests.get(test, 30),
-         suite: ['qtest', 'qtest-' + target_base])
+
+    migtest = 'migration-test'
+    if test == migtest
+      migtests = run_command(python, files('gen_migration_tests_list.py'),
+                             meson.current_source_dir() / 'migration-test.c',
+                             check: true)
+
+      foreach item : migtests.stdout().strip().split('\n')
+        testname = '/@0@@1@'.format(target_base, item)
+
+        test(testname,
+             qtest_executables['migration-test'],
+             depends: [test_deps, qtest_emulator, emulator_modules],
+             env: qtest_env,
+             args: ['-k', '-p', testname],
+             protocol: 'tap',
+             timeout: slow_qtests.get(migtest),
+             priority: slow_qtests.get(migtest),
+             suite: ['migration'])
+      endforeach
+    else
+      test('qtest-@0@/@1@'.format(target_base, test),
+           qtest_executables[test],
+           depends: [test_deps, qtest_emulator, emulator_modules],
+           env: qtest_env,
+           args: ['--tap', '-k'],
+           protocol: 'tap',
+           timeout: slow_qtests.get(test, 30),
+           priority: slow_qtests.get(test, 30),
+           suite: ['qtest', 'qtest-' + target_base])
+      endif
   endforeach
 endforeach
-- 
2.35.3



  parent reply	other threads:[~2023-07-03 14:47 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-07-03 14:46 [RFC PATCH 0/2] tests/qtest: Run migration subtests via meson Fabiano Rosas
2023-07-03 14:46 ` [RFC PATCH 1/2] tests/qtest: Add a script to gather migration tests list Fabiano Rosas
2023-07-03 14:47 ` Fabiano Rosas [this message]
2023-07-03 15:17 ` [RFC PATCH 0/2] tests/qtest: Run migration subtests via meson Daniel P. Berrangé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230703144700.13536-3-farosas@suse.de \
    --to=farosas@suse.de \
    --cc=alex.bennee@linaro.org \
    --cc=berrange@redhat.com \
    --cc=bleal@redhat.com \
    --cc=luoyonggang@gmail.com \
    --cc=lvivier@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).