Re: [PATCH 2/3] avocado_qemu: add AVOCADO_DEFAULT_ARCH for cross-arch tests

[Fabiano Rosas <farosas@suse.de>]

Daniel Henrique Barboza <dbarboza@ventanamicro.com> writes:

> All avocado tests that are arch agnostic (i.e. does not set an 'arch'
> tag) are run with arch=None in pick_default_qemu_bin(), and then 'arch'
> is set to os.uname()[4], meaning that it will take the arch of the
> running host.
>
> This means that if one compiles QEMU binaries for non-x86 targets on an
> x86 machine, and then run 'make check-avocado', all arch agnostic tests
> will be cancelled because there's no qemu-system-x86_64 to be found.
>
> There is no particular reason to not allow these tests to be run with
> other arch binaries in a x86_64 host. Allow the developer to do it by
> adding a a new env variable called AVOCADO_DEFAULT_ARCH. Any 'arch' that
> is set by this variable will take precedence of setting it via
> os.uname()[4]. We can then run non-x86 binaries tests in a x86_64 host
> as follows:
>
> $ AVOCADO_DEFAULT_ARCH=riscv64 make check-avocado
> (...)
> RESULTS: PASS 11 | ERROR 0 | FAIL 0 | SKIP 1 | WARN 0 | INTERRUPT 0 | CANCEL 0

I don't understand why tags don't solve the problem. We
are already passing a tag for each target:

ifndef AVOCADO_TAGS
	AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
						 $(filter %-softmmu,$(TARGETS)))
else
	AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
endif

I then tried to tag migration.py with:

    :avocado: tags=arch:x86_64
    :avocado: tags=arch:aarch64

On an x86_64 machine with target-list=x86_64-softmmu,aarch64-softmmu,
only the x86_64 test runs. Even if I remove the x86_64 tag from the
avocado line. Possibly due to the --filter-by-tags-include-empty
options. But I would expect a second run with aarch64, even if it
failed.

If I use only:

    :avocado: tags=arch:riscv

and run:

python3 -m avocado --show=app run -t arch:riscv -t arch:x86_64 --failfast ../tests/avocado/migration.py

Then it complains about the binary, but the x86_64 binary is present! So
it looked at the tag after all:

CANCEL: No QEMU binary defined or found in the build tree for arch riscv
                                                                   ^^^^^

I don't know how to make this work, but I feel there should be a way to
have the framework select the correct test AND pass the correct arch
parameter along.