Re: [PATCH v4 1/6] tests/requirements.txt: bump up avocado-framework version to 101.0

["Philippe Mathieu-Daudé" <philmd@linaro.org>]

On 21/4/23 06:09, Cleber Rosa wrote:
> 
> On 4/20/23 12:54, John Snow wrote:
>>
>>
>> On Thu, Apr 20, 2023, 5:19 AM Philippe Mathieu-Daudé 
>> <philmd@linaro.org> wrote:
>>
>>     On 17/4/23 19:44, Alex Bennée wrote:
>>     >
>>     > John Snow <jsnow@redhat.com> writes:
>>     >
>>     >> On Mon, Apr 17, 2023 at 9:43 AM Alex Bennée
>>     <alex.bennee@linaro.org> wrote:

>>     >>> diff --git a/tests/requirements.txt b/tests/requirements.txt
>>     >>> index 0ba561b6bd..a6f73da681 100644
>>     >>> --- a/tests/requirements.txt
>>     >>> +++ b/tests/requirements.txt
>>     >>> @@ -2,5 +2,5 @@
>>     >>>   # in the tests/venv Python virtual environment. For more info,
>>     >>>   # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
>>     >>>   # Note that qemu.git/python/ is always implicitly installed.
>>     >>> -avocado-framework==88.1
>>     >>> +avocado-framework==101.0
>>     >>>   pycdlib==1.11.0
>>     >>> --
>>     >>> 2.39.2
>>     >>>
>>     >>
>>     >> I thought there were test failures that prohibited us from
>>     bumping the
>>     >> Avocado-Framework version. Did those get rectified recently?
>>
>>     No, still not working on Darwin. At this point I'm thinking at 
>> sending
>>     a patch clarifying testing QEMU with Avocado is not supported on
>>     Darwin.
>>
>>
>> What's broken on Darwin?
>>
> Hi John,
> 
> IIRC, the main issue is that the legacy runner (the one active in 88.1) 
> pickles some stuff that do not play nice under Darwin.  It has never (to 
> the best of my knowledge) worked under Darwin.

Correct.

>> Is it broken with avocado-framework==88.1 too? (Is this a regression?)
>>
> It was already broken with avocado-framework==88.1,

Correct.

> but it should mostly 
> work with 101.0 (see previous reply).
> 
>> We might need to consider entirely separate issues:
>>
>> (1) We need to upgrade avocado-framework to a supported LTS version >= 
>> v90 for dependency harmony with qemu.git/python testing
>>
>> (2) We need to upgrade avocado-framework to >= v101.0 to fix the 
>> stated checksum issue in this patch
>>
>> (3) We would like avocado tests to work on Darwin. (Have they ever 
>> worked? When did they break? etc.)
>>
>> My understanding is that this patch helps with 1 and 2 but not 3, but 
>> if Darwin support isn't a regression, this is still a step forward.
>>
> To the best of my knowledge, this patch would help with 1, 2 and even 3.

As of today using v8.0.0:

$ make check-avocado
   GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 
tests/fp/berkeley-softfloat-3 dtc
   MKDIR   /Users/philmd/source/qemu/build/tests/results
   AVOCADO Downloading avocado tests VM image for aarch64
The image was downloaded:
Provider Version Architecture File
fedora   31      aarch64 
/Users/philmd/avocado/data/cache/by_location/4f156e531446a679cbfe13caef8b7c9f9f79aafa/Fedora-Cloud-Base-31-1.9.aarch64.qcow2
   AVOCADO tests/avocado
Fetching asset from 
tests/avocado/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_atf
Fetching asset from 
tests/avocado/boot_xen.py:BootXen.test_arm64_xen_411_and_dom0
Fetching asset from 
tests/avocado/boot_xen.py:BootXen.test_arm64_xen_414_and_dom0
Fetching asset from 
tests/avocado/boot_xen.py:BootXen.test_arm64_xen_415_and_dom0
Fetching asset from 
tests/avocado/machine_aarch64_virt.py:Aarch64VirtMachine.test_alpine_virt_tcg_gic_max
Fetching asset from 
tests/avocado/replay_kernel.py:ReplayKernelNormal.test_aarch64_virt
Fetching asset from 
tests/avocado/reverse_debugging.py:ReverseDebugging_AArch64.test_aarch64_virt
JOB ID     : 142a0b2d3960c7daccd657781d52f814e1617392
JOB LOG    : 
/Users/philmd/source/qemu/build/tests/results/job-2023-04-21T11.35-142a0b2/job.log

Avocado crashed: TypeError: cannot pickle '_thread.RLock' object
Traceback (most recent call last):

   File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/core/job.py", 
line 632, in run_tests
     summary |= suite.run(self)

   File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/core/suite.py", 
line 311, in run
     return self.runner.run_suite(job, self)

   File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/plugins/runner.py", 
line 390, in run_suite
     if not self.run_test(job, test_factory, queue, summary,

   File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/plugins/runner.py", 
line 170, in run_test
     proc.start()

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", 
line 121, in start
     self._popen = self._Popen(self)

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", 
line 224, in _Popen
     return _default_context.get_context().Process._Popen(process_obj)

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", 
line 288, in _Popen
     return Popen(process_obj)

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", 
line 32, in __init__
     super().__init__(process_obj)

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", 
line 19, in __init__
     self._launch(process_obj)

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", 
line 47, in _launch
     reduction.dump(process_obj, fp)

   File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/reduction.py", 
line 60, in dump
     ForkingPickler(file, protocol).dump(obj)

TypeError: cannot pickle '_thread.RLock' object

Please include the traceback info and command line used on your bug report
Report bugs visiting https://github.com/avocado-framework/avocado/issues/new
make: *** [check-avocado] Error 4

This is https://github.com/avocado-framework/avocado/issues/4888
which is still opened.

Manually using "avocado run --test-runner=nrunner" helps running the
tests but the console output is never displayed (see [*]) and once
finished Avocado lets various QEMU processes running in detached
mode, wasting resources.

[*] 
https://lore.kernel.org/qemu-devel/1d0baf8b-c757-265c-b206-07ca3f218b2a@linaro.org/