Re: [PATCH v1 00/11] Convert avocado tests to normal Python unittests

[Thomas Huth <thuth@redhat.com>]

On 16/07/2024 19.03, Thomas Huth wrote:
> On 16/07/2024 18.51, Daniel P. Berrangé wrote:
>> On Tue, Jul 16, 2024 at 01:26:03PM +0200, Thomas Huth wrote:
> ...
>>> So instead of trying to update the python-based test suite in QEMU
>>> to a newer version of Avocado, we should maybe try to better integrate
>>> it with the meson test runner instead. Indeed most tests work quite
>>> nicely without the Avocado framework already, as you can see with
>>> this patch series - it does not convert all tests, just a subset so
>>> far, but this already proves that many tests only need small modifi-
>>> cations to work without Avocado.
> ...
>>> Now if you want to try out these patches: Apply the patches, then
>>> recompile and then run:
>>>
>>>   make check-functional
>>>
>>> You can also run single targets e.g. with:
>>>
>>>   make check-functional-ppc
>>>
>>> You can also run the tests without any test runner now by
>>> setting the PYTHONPATH environment variable to the "python" folder
>>> of your source tree, and by specifying the build directory via
>>> QEMU_BUILD_ROOT (if autodetection fails) and by specifying the
>>> QEMU binary via QEMU_TEST_QEMU_BINARY. For example:
>>>
>>>   export PYTHONPATH=$HOME/qemu/python
>>>   export QEMU_TEST_QEMU_BINARY=qemu-system-x86_64
>>>   export PYTHONPATH=$HOME/qemu/build
>>>   ~/qemu/tests/functional/test_virtio_version.py
>>
>> For the whole series as is
>>
>>   Tested-by: Daniel P. Berrangé <berrange@redhat.com>
>>
>> as it does what you claim it does here when I tried it.
> 
> Thanks!
> 
>>> The logs of the tests can be found in the build directory under
>>> tests/functional/<testname> - console log and general logs will
>>> be put in separate files there.
>>
>> As an example, one dir name appears to be:
>>
>>    __main__.MemAddrCheck.test_phybits_ok_pentium_pae
>>
>> I'd rather prefer it if the dir name matched the test script
>> file name - in this case test_mem_addr_space.py, as I don't
>> want to have to lookup which class names were defined inside
>> each test script. We could drop the "test_" prefix from the
>> method name too
>>
>> IOW, could we make this dir name be:
>>
>>    test_mem_addr_space.phybits_ok_pentium_pae
> 
> I can try to change that, indeed ... but the boilerplate code will increase 
> a little bit, I guess, since I cannot simply rely on the unittest.id() 
> function in that case anymore...

After looking at this for a while, I think it's maybe best to ditch the idea 
of making the .py files directly runnable and run the tests via a simple 
pycotap runner instead. Then you get proper module names:

$ pyvenv/bin/python3 -m pycotap test_virtio_version
TAP version 13
ok 1 test_virtio_version.VirtioVersionCheck.test_conventional_devs
ok 2 test_virtio_version.VirtioVersionCheck.test_modern_only_devs
1..2

  Thomas