Re: [PATCH v2 03/16] tests/Makefile.include: add binary dependency to run-tcg-tests-% rules

["Alex Bennée" <alex.bennee@linaro.org>]

Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:

> On 5/26/2026 11:17 PM, Alex Bennée wrote:
>> Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:
>> 
<snip>
>> 
>> What do you mean? I run check-tcg all the time.
>> 
>> Which containers fail to build for you? The main cross-compiler one
>> (debian-all-test-cross) is built all the time.
>>
>
> Build is working out of the box now.
> However, it seems that only tests for aarch64, arm and x86_64 are run
> when using containers without having installed cross compilers.
> Maybe it's another carefully handcrafted feature of tcg Makefiles?

Works for me over here. Do you have podman or docker installed?

It's true the Makefile has to jump through some hoops to compute
TCG_TESTS_TARGETS from TCG_TESTS_WITH_COMPILERS (config-host.mak) and
CONFIGURED_TCG_TARGETS (via $(wildcard *-config-target.h)). Is that
where it fails?

As I've said before I'm not wedded to Makefiles, in fact I would look
forward to a world when I can run an individual test directly from
meson. However I'm not prepared to throw away functionality, especially
the ability of new developers/testers to build the tests without having
to install cross-compilers. That would take us back to the *before
times* when TCG tests where hardly ever run and tended to bitrot.

>>> But well, if the cross-container thing is *really* a blocker, we can
>>> just wrap build and run in a script per cross compiler, that does
>>> docker/podman build && docker/podman run and call that.
>> 
>> We still have the remnants of that in the docker.py script although for
>> most containers we moved to invoking docker (or podman) directly.
>>
>
> Maybe that's why I encountered issues in the past, and now it works out
> of the box.

The original script attempted to deal with caching layers and the
consequences of custom USER/UID on local machines. Nowadays it is mostly
used as a basic wrapper around "probe" for the runtime and is still used
by the binfmt_misc and toolchain wrappers.

For porting to meson it might be worth just having a minimal wrapper
just for building rather than mess with it.

>> Also bear in mind the containers are also used to cross build QEMU
>> itself. 
>> 
>>>
>>> Now we solved the problem with system tests, and cross compilers, is
>>> there another thing that is absolutely needed from Makefiles?
>>>
>>>>> If it's not the case, then we should definitely move into the direction
>>>>> of migrating all this to meson, and rely on meson tests infrastructure.
>>>>>
>
> To add on top of this, I would be happy to help with the transition, but
> I want to make sure you'll accept it first, with a list of
> requirements.

For check-tcg the requirements are simple:

  - support building tests with containers
  - handle all existing tests
    - per-arch *-user and *-softmmu
      - including post test validation checks (conditional-diff-out, validate-memory-counts.py, etc...)
      - per-target overrides (some targets are broken for some tests)
      - custom QEMU_OPTS for some tests
      - including sub-arch tests (e.g. x86_64 includes i386, mips64el includes mips64 etc...)
    - multiarch tests for all *-user targets
    - multiarch tests for all *-softmmu targets with boot.S
    - gdbstub tests
      - don't forget gdb feature probing for arch and arch features (e.g. GDB_HAS_SME_TILES)
    - record/replay tests
    - tcg plugin tests
      - modulate plugins over tests to avoid combinatorial explosion
      - filter some plugins from general testing (e.g. patch)
      - filter some plugins based on mode (e.g. syscall)
      - specific plugins with specific tests (e.g. patch, memory)
    - support build variations
      - compiler feature probing to gate tests
      - custom cflags for some tests
      - alternate cflag builds for some tests (e.g. sha512-[vector|sse|mvx|...])
      - build scripts (e.g. test-mmx.py)

with meson TCG_TEST_FILTER should be obsolete.

> I'm a bit tired of seeing patches in your subsystems get stuck for
> weeks, without any proper review, to finally be (re)posted again in a
> new series, blocked for more weeks, and finally reach upstream one month
> later or worse. It's truly exhausting for everyone contributing in such
> places.

What can I say - reviews are welcome. I always list what patches are
still waiting for review in the cover letter. I should also add for
testing/next I try to be extra careful that regressions don't slip in
because keeping the testing sub-system stable is important to support
things like bisection.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro