Re: [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests

["Daniel P. Berrangé" <berrange@redhat.com>]

On Mon, Aug 26, 2024 at 10:18:44AM +0200, Thomas Huth wrote:
> On 23/08/2024 14.54, Philippe Mathieu-Daudé wrote:
> > On 21/8/24 10:27, Thomas Huth wrote:
> > > Provide a meson.build file for the upcoming python-based functional
> > > tests, and add some wrapper glue targets to the tests/Makefile.include
> > > file. We are going to use two "speed" modes for the functional tests:
> > > The "quick" tests can be run at any time (i.e. also during "make check"),
> > > while the "thorough" tests should only be run when running a
> > > "make check-functional" test run (since these tests might download
> > > additional assets from the internet).
> > > 
> > > The changes to the meson.build files are partly based on an earlier
> > > patch by Ani Sinha.
> > > 
> > > Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > ---
> > >   tests/Makefile.include       | 11 ++++++
> > >   tests/functional/meson.build | 66 ++++++++++++++++++++++++++++++++++++
> > >   tests/meson.build            |  1 +
> > >   3 files changed, 78 insertions(+)
> > >   create mode 100644 tests/functional/meson.build
> > 
> > 
> > > +# Timeouts for individual tests that can be slow e.g. with debugging enabled
> > > +test_timeouts = {
> > > +}
> > 
> > 
> > > +    foreach test : target_tests
> > > +      test('func-@0@/@1@'.format(target_base, test),
> > > +           python,
> > > +           depends: [test_deps, test_emulator, emulator_modules],
> > > +           env: test_env,
> > > +           args: [meson.current_source_dir() / 'test_' + test + '.py'],
> > > +           protocol: 'tap',
> > > +           timeout: test_timeouts.get(test, 60),
> > > +           priority: test_timeouts.get(test, 60),
> > 
> > IIUC with Avocado the timeout was for each test_func in a TestClass.
> > Now this is only per TestClass. Hopefully I'm wrong...
> 
> I think you're right ... we might need to adjust the meson timeouts here and
> there in case they are causing problems, but that's just business as usual
> (we had to do the same when enabling the meson timeouts for the qtests for
> example).

Meson timeouts are around the lifetime of whatever child process
we tell it to execute.  So, yes, the timeout is for the .py file
as a whole, not the individual TestClass within it.

This is different from Avocado, but not a big problem. Worst case
we'll adjust some meson level timeouts, or split overly long running
.py test files into multiple test .py files.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|