Re: acceptance-system-fedora failures

[Cleber Rosa <crosa@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2810 bytes --]

On Wed, Oct 07, 2020 at 07:20:49AM +0200, Philippe Mathieu-Daudé wrote:
> On 10/7/20 1:07 AM, John Snow wrote:
> > I'm seeing this gitlab test fail quite often in my Python work; I don't
> > *think* this has anything to do with my patches, but maybe I need to try
> > and bisect this more aggressively.
> > 
> > The very first hint of an error I see is on line 154:
> > 
> > https://gitlab.com/jsnow/qemu/-/jobs/776334918#L154
> > 
> > 22:05:36 ERROR|
> > 22:05:36 ERROR| Reproduced traceback from:
> > /builds/jsnow/qemu/build/tests/venv/lib64/python3.8/site-packages/avocado/core/test.py:753
> > 
> > 22:05:36 ERROR| Traceback (most recent call last):
> > 22:05:36 ERROR|   File
> > "/builds/jsnow/qemu/build/tests/acceptance/avocado_qemu/__init__.py",
> > line 171, in setUp
> > 22:05:36 ERROR|     self.cancel("No QEMU binary defined or found in the
> > build tree")
>

One very bad aspect of this output is that the test outcome (a test
cancelation) is determined by an exception handler by the runner, and
the "ERROR" prefix is indeed very misleading.

But yes, in short, it's not an *error*, but the test getting canceled.

> Last year the Avocado developers said we could have a clearer
> log error report, but meanwhile this verbose output is better
> that not having anything ¯\_(ツ)_/¯
>

With the new test runner ("avocado run --test-runner=nrunner") that
just made its way into Avocado 81.0, there's a much better separation
of what happens within the test, and within the runner.

The next step is to post the QEMU and "Avocado >= 82.0" integration, so
hopefully this will improve soon.

> > 
> > Is this a known problem?
> 
> "No QEMU binary defined or found in the build tree" is not a
> problem, it means a test is skipped because the qemu-system-$ARCH
> binary is not found. In your case this is because your job
> (acceptance-system-fedora) is based on build-system-fedora
> which only build the following targets:
> 
>     TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu
>       xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu
>       sparc64-softmmu
>

Right.

> Now I don't understand what binary the EmptyCPUModel/Migration tests
> are expecting. Maybe these tests only work when a single target is
> built? IOW not expecting that the python code searching for a binary
> return multiple ones? -> Eduardo/Cleber.
>

`avocado_qemu.pick_default_qemu()`, if not given an arch, will try to
find a target binary that matches the host.  The problem with picking
any (first available?) built target is the non-deterministic aspect.

BTW, with regards to how `avocado_qemu.pick_default_qemu()` will get
an arch, it can come either from a test parameter, or from an "arch"
tag.

Cheers,
- Cleber.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]