Re: [PATCH 1/4] tests/qtest: Add check-migration

["Daniel P. Berrangé" <berrange@redhat.com>]

On Thu, Oct 17, 2024 at 11:32:08AM -0300, Fabiano Rosas wrote:
> Add two new targets, check-migration and check-migration-quick to
> allow dividing migration tests into a quick set and a slow set. With
> this it'll be possible to reduce the amount of migration tests that
> run by default as part of 'make check'.
> 
> Keep under the 'migration-quick' suite only a few tests to serve as
> sanity check for every build and move the rest under the 'migration'
> suite.

I don't think we should need to have separate make targets
for each speed. I would expect users to be able to run

  $ make check SPEED=thorough
  $ make check-qtest SPEED=thorough

which is how we document doing this for functional tests.

If we want a way to let users more easily run individual
(or a subset of) qtest suites, how about we allow for
some filtering along the lines of:

  $ make check-qtest QTESTS=migration-test

as adding top level make-check-<blah> targets for each
different subset maintainers might want isn't scalable.

> 
> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> ---
>  meson.build             |  6 +++---
>  tests/Makefile.include  |  2 ++
>  tests/qtest/meson.build | 47 +++++++++++++++++++++++++++++++++--------
>  3 files changed, 43 insertions(+), 12 deletions(-)
> 
> diff --git a/meson.build b/meson.build
> index 4ea1984fc5..92d38691f9 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -3,9 +3,9 @@ project('qemu', ['c'], meson_version: '>=1.1.0',
>                            'b_staticpic=false', 'stdsplit=false', 'optimization=2', 'b_pie=true'],
>          version: files('VERSION'))
>  
> -add_test_setup('quick', exclude_suites: ['slow', 'thorough'], is_default: true)
> -add_test_setup('slow', exclude_suites: ['thorough'], env: ['G_TEST_SLOW=1', 'SPEED=slow'])
> -add_test_setup('thorough', env: ['G_TEST_SLOW=1', 'SPEED=thorough'])
> +add_test_setup('quick', exclude_suites: ['slow', 'thorough', 'migration'], is_default: true)
> +add_test_setup('slow', exclude_suites: ['thorough', 'migration-quick'], env: ['G_TEST_SLOW=1', 'SPEED=slow'])
> +add_test_setup('thorough', exclude_suites: ['migration-quick'], env: ['G_TEST_SLOW=1', 'SPEED=thorough'])
>  
>  meson.add_postconf_script(find_program('scripts/symlink-install-tree.py'))
>  
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 010369bd3a..79c1350bfb 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -11,6 +11,8 @@ check-help:
>  	@echo " $(MAKE) check-qtest              Run qtest tests"
>  	@echo " $(MAKE) check-functional         Run python-based functional tests"
>  	@echo " $(MAKE) check-functional-TARGET  Run functional tests for a given target"
> +	@echo " $(MAKE) check-migration-quick    Run a small set of migration tests"
> +	@echo " $(MAKE) check-migration          Run all migration tests"
>  	@echo " $(MAKE) check-unit               Run qobject tests"
>  	@echo " $(MAKE) check-qapi-schema        Run QAPI schema tests"
>  	@echo " $(MAKE) check-block              Run block tests"
> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
> index b207e38696..27a802474a 100644
> --- a/tests/qtest/meson.build
> +++ b/tests/qtest/meson.build
> @@ -406,14 +406,43 @@ foreach dir : target_dirs
>          test: executable(test, src, dependencies: deps)
>        }
>      endif
> -    test('qtest-@0@/@1@'.format(target_base, test),
> -         qtest_executables[test],
> -         depends: [test_deps, qtest_emulator, emulator_modules],
> -         env: qtest_env,
> -         args: ['--tap', '-k'],
> -         protocol: 'tap',
> -         timeout: slow_qtests.get(test, 60),
> -         priority: slow_qtests.get(test, 60),
> -         suite: ['qtest', 'qtest-' + target_base])
> +
> +    # The migration-test test runs several slow sub-tests. Add it to
> +    # two separate targets, one for executing a few tests
> +    # (migration-quick) and another for executing the full set
> +    # (migration). This is done to reduce the amount of tests that run
> +    # via make check.
> +    if test == 'migration-test'
> +      foreach opts : [
> +        {
> +          'test-args': ['--tap', '-k', '-m', 'slow'],
> +          'test-suite': ['migration']
> +        },
> +        {
> +          'test-args': ['--tap', '-k', '-m', 'quick'],
> +          'test-suite': ['migration-quick']
> +        }]
> +
> +        test(target_base,
> +             qtest_executables[test],
> +             depends: [test_deps, qtest_emulator, emulator_modules],
> +             env: qtest_env,
> +             args: opts['test-args'],
> +             protocol: 'tap',
> +             timeout: slow_qtests.get(test, 60),
> +             priority: slow_qtests.get(test, 60),
> +             suite: opts['test-suite'])
> +      endforeach
> +    else
> +      test('qtest-@0@/@1@'.format(target_base, test),
> +           qtest_executables[test],
> +           depends: [test_deps, qtest_emulator, emulator_modules],
> +           env: qtest_env,
> +           args: ['--tap', '-k'],
> +           protocol: 'tap',
> +           timeout: slow_qtests.get(test, 60),
> +           priority: slow_qtests.get(test, 60),
> +           suite: ['qtest', 'qtest-' + target_base])
> +    endif
>    endforeach
>  endforeach
> -- 
> 2.35.3
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|