Re: [PATCH v2 00/23] Convert avocado tests to normal Python unittests

[Richard Henderson <richard.henderson@linaro.org>]

On 7/25/24 03:52, Thomas Huth wrote:
> The Avocado v88 that we use in QEMU is already on a life support
> system: It is not supported by upstream anymore, and with the latest
> versions of Python, it won't work anymore since it depends on the
> "imp" module that has been removed in Python 3.12.
> 
> There have been several attempts to update the test suite in QEMU
> to a newer version of Avocado, but so far no attempt has successfully
> been merged yet.
> 
> Additionally, the whole "make check" test suite in QEMU is using the
> meson test runner nowadays, so running the python-based tests via the
> Avocodo test runner looks and feels quite like an oddball, requiring
> the users to deal with the knowledge of multiple test runners in
> parallel (e.g. the timeout settings work completely differently).
> 
> So instead of trying to update the python-based test suite in QEMU
> to a newer version of Avocado, we should try to better integrate
> it with the meson test runner instead. Indeed most tests work quite
> nicely without the Avocado framework already, as you can see with
> this patch series - it does not convert all tests, just a subset so
> far, but this already proves that many tests only need small modifi-
> cations to work without Avocado.
> 
> Only tests that use the LinuxTest / LinuxDistro and LinuxSSHMixIn
> classes (e.g. based on cloud-init images or using SSH) really depend
> on the Avocado framework, so we'd need a solution for those if we
> want to continue using them. One solution might be to simply use the
> required functions from avocado.utils for these tests, and still run
> them via the meson test runner instead, but that needs some further
> investigation that will be done later.
> 
> 
> Now if you want to try out these patches: Apply the patches, then
> recompile and then run:
> 
>   make check-functional
> 
> You can also run single targets e.g. with:
> 
>   make check-functional-ppc
> 
> You can also run the tests without any test runner now by
> setting the PYTHONPATH environment variable to the "python" folder
> of your source tree, and by specifying the build directory via
> QEMU_BUILD_ROOT (if autodetection fails) and by specifying the
> QEMU binary via QEMU_TEST_QEMU_BINARY. For example:
> 
>   export PYTHONPATH=$HOME/qemu/python
>   export QEMU_TEST_QEMU_BINARY=qemu-system-x86_64
>   export QEMU_BUILD_ROOT=$HOME/qemu/build
>   ~/qemu/tests/functional/test_virtio_version.py
> 
> The logs of the tests can be found in the build directory under
> tests/functional/<arch>/<testname> - console log and general logs will
> be put in separate files there.
> 
> Still to be done: Update the documentation for this new test framework.

I'll say again that the download *must* be handled separately from the test with timeout. 
This is an absolute show-stopper.

I've tried this twice now, from a decently fast connection in central Brisbane, and have 
had multiple downloads be canceled by the timeout.  Since the download isn't clever enough 
to pick up where it left off, it will never succeed.


r~