[PATCH v4 00/19] testing: various docker clean-ups post meson tcg conversion

[PATCH v4 00/19] testing: various docker clean-ups post meson tcg conversion

[Alex Bennée]

This cleans up a few bits and pieces although doesn't totally solve
the issue of an apparent stall while meson builds the docker image.
However with these changes there should be no difference between the
meson build and the equivalent "make docker-image-foo" invocation.

We also finally get rid of the ill-advised magic caching support in
docker.py.

v4
 - added Illya's fixes for out of tree and s390x tests
 - tweaked the revert so --disable-containers still passed to meson
 - don't define tests if no compiler available to avoid excess SKIPs

v3

 - Update make2test to trigger docker build from Makefile.include.
 - add Daniels's suggested console: true patch
 - fixed up some unref vars from the reverts

v2

I've restored the configure based compiler checks for so we can build
roms and the vdso blobs again. It looks like we could never build roms
with container based images so I've left it for now. Unlike the vdso
build we can't trigger a container build through Makefile.prereqs.

TODO
----

I also need to look at the timestamp stuff that Pierrick has suggested
to see if that improves the caching from the registry.

The following still need review:

  tests/tcg: add console property to docker image custom target
  scripts/make2mtest: add docker dependencies for check-tcg-% targets
  tests/docker: fix brown bag typo for docker-ccache
  tests/docker: swap --quiet for --verbose
  tests/docker: add container registry to configure and use
  tests/tcg: don't define tests when missing compilers
  tests/tcg: use --add-current-user when building containers
  configure: re-purpose the tcg test compiler configs for vdso building
  Revert "tests/tcg/meson.build: implement ./configure -cross-cc-* options"

Please test and review.

Alex.

Alex Bennée (16):
  Revert "configure: remove cross_cc and gdb detection logic"
  Revert "tests/tcg/meson.build: implement ./configure -cross-cc-*
    options"
  Revert "tests/tcg/meson.build: implement ./configure containers
    options"
  configure: re-purpose the tcg test compiler configs for vdso building
  tests/tcg: prefix meson built docker tags with qemu/
  tests/tcg: use --add-current-user when building containers
  tests/tcg: don't define tests when missing compilers
  tests/docker: add container registry to configure and use
  tests/docker: remove used _read_qemu_dockerfile
  tests/docker: strip remaining caching logic from docker.py
  tests/docker: add specific command to parser
  tests/docker: swap --quiet for --verbose
  tests/docker: fix brown bag typo for docker-ccache
  gitlab: update the @user id mapping file
  scripts/make2mtest: add docker dependencies for check-tcg-% targets
  tests/tcg: add console property to docker image custom target

Daniel P. Berrangé (1):
  configure: correctly honour --disable-containers

Ilya Leoshkevich (2):
  tests/tcg/s390x: Provide the libc bits needed by the system tests
  tests/tcg: Fix building outside of the source tree

 configure                               | 494 +++++++++++++++++++++++-
 tests/tcg/s390x/libc/stdio.h            |  11 +
 tests/tcg/s390x/libc/string.h           |  12 +
 .gitlab-map-auto                        |   8 +-
 linux-user/aarch64/Makefile.vdso        |   2 +-
 linux-user/arm/Makefile.vdso            |   2 +-
 linux-user/hppa/Makefile.vdso           |   2 +-
 linux-user/i386/Makefile.vdso           |   2 +-
 linux-user/loongarch64/Makefile.vdso    |   2 +-
 linux-user/ppc/Makefile.vdso            |   2 +-
 linux-user/riscv/Makefile.vdso          |   2 +-
 linux-user/s390x/Makefile.vdso          |   2 +-
 linux-user/sh4/Makefile.vdso            |   2 +-
 linux-user/x86_64/Makefile.vdso         |   2 +-
 meson_options.txt                       |  92 +----
 scripts/coverity-scan/run-coverity-scan |   2 +-
 scripts/gitlab-map-update               |   2 +-
 scripts/meson-buildoptions.sh           |   3 +
 scripts/mtest2make.py                   |  35 +-
 tests/docker/Makefile.include           |  16 +-
 tests/docker/docker.py                  | 188 ++++-----
 tests/tcg/meson.build                   |  66 ++--
 tests/tcg/s390x/system/meson.build      |   2 +
 23 files changed, 659 insertions(+), 292 deletions(-)
 create mode 100644 tests/tcg/s390x/libc/stdio.h
 create mode 100644 tests/tcg/s390x/libc/string.h

-- 
2.47.3

[PATCH v4 01/19] Revert "configure: remove cross_cc and gdb detection logic"

[Alex Bennée]

We want to restore the logic to configure as it's also used by the
firmware builds.

This reverts commit 031f935567ab8abfa4e84682ecea08915ca9656d.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure | 461 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 461 insertions(+)

diff --git a/configure b/configure
index aaa41fe47aa..1973f023b1e 100755
--- a/configure
+++ b/configure
@@ -1266,6 +1266,366 @@ EOF
   fi
 fi
 
+##########################################
+# functions to probe cross compilers
+
+if test "$container_command" = ""; then
+    container_command=$($python "$source_path"/tests/docker/docker.py probe)
+    test "$container_command" = "no" && container_command=""
+fi
+if test $use_containers = "yes" && test "$container_command" != ""; then
+    docker_py="$python $source_path/tests/docker/docker.py --command $container_command"
+fi
+
+# cross compilers defaults, can be overridden with --cross-cc-ARCH
+: ${cross_prefix_aarch64="aarch64-linux-gnu-"}
+: ${cross_prefix_aarch64_be="$cross_prefix_aarch64"}
+: ${cross_prefix_alpha="alpha-linux-gnu-"}
+: ${cross_prefix_arm="arm-linux-gnueabihf-"}
+: ${cross_prefix_armeb="$cross_prefix_arm"}
+: ${cross_prefix_hexagon="hexagon-unknown-linux-musl-"}
+: ${cross_prefix_loongarch64="loongarch64-unknown-linux-gnu-"}
+: ${cross_prefix_hppa="hppa-linux-gnu-"}
+: ${cross_prefix_i386="i686-linux-gnu-"}
+: ${cross_prefix_m68k="m68k-linux-gnu-"}
+: ${cross_prefix_microblaze="microblaze-linux-musl-"}
+: ${cross_prefix_mips64el="mips64el-linux-gnuabi64-"}
+: ${cross_prefix_mips64="mips64-linux-gnuabi64-"}
+: ${cross_prefix_mipsel="mipsel-linux-gnu-"}
+: ${cross_prefix_mips="mips-linux-gnu-"}
+: ${cross_prefix_ppc="powerpc-linux-gnu-"}
+: ${cross_prefix_ppc64="powerpc64-linux-gnu-"}
+: ${cross_prefix_ppc64le="$cross_prefix_ppc64"}
+: ${cross_prefix_riscv64="riscv64-linux-gnu-"}
+: ${cross_prefix_s390x="s390x-linux-gnu-"}
+: ${cross_prefix_sh4="sh4-linux-gnu-"}
+: ${cross_prefix_sparc64="sparc64-linux-gnu-"}
+: ${cross_prefix_sparc="$cross_prefix_sparc64"}
+: ${cross_prefix_tricore="tricore-"}
+: ${cross_prefix_x86_64="x86_64-linux-gnu-"}
+
+: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
+: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
+: ${cross_cc_armeb="$cross_cc_arm"}
+: ${cross_cc_cflags_armeb="-mbig-endian"}
+: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
+: ${cross_cc_cflags_hexagon="-mv73 -O2 -static"}
+: ${cross_cc_cflags_i386="-m32"}
+: ${cross_cc_cflags_ppc="-m32 -mbig-endian"}
+: ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
+: ${cross_cc_ppc64le="$cross_cc_ppc64"}
+: ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
+: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
+: ${cross_cc_sparc="$cross_cc_sparc64"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
+: ${cross_cc_cflags_x86_64="-m64 -mcx16"}
+
+compute_target_variable() {
+  eval "$2="
+  if eval test -n "\"\${cross_prefix_$1}\""; then
+    if eval has "\"\${cross_prefix_$1}\$3\""; then
+      eval "$2=\"\${cross_prefix_$1}\$3\""
+    fi
+  fi
+}
+
+have_target() {
+  for i; do
+    case " $target_list " in
+      *" $i "*) return 0;;
+      *) ;;
+    esac
+  done
+  return 1
+}
+
+# probe_target_compiler TARGET
+#
+# Look for a compiler for the given target, either native or cross.
+# Set variables target_* if a compiler is found, and container_cross_*
+# if a Docker-based cross-compiler image is known for the target.
+# Set got_cross_cc to yes/no depending on whether a non-container-based
+# compiler was found.
+#
+# If TARGET is a user-mode emulation target, also set build_static to
+# "y" if static linking is possible.
+#
+probe_target_compiler() {
+  # reset all output variables
+  got_cross_cc=no
+  container_image=
+  container_hosts=
+  container_cross_prefix=
+  container_cross_cc=
+  container_cross_ar=
+  container_cross_as=
+  container_cross_ld=
+  container_cross_nm=
+  container_cross_objcopy=
+  container_cross_ranlib=
+  container_cross_strip=
+
+  target_arch=${1%%-*}
+  case $target_arch in
+    aarch64) container_hosts="x86_64 aarch64" ;;
+    aarch64_be) container_hosts="x86_64 aarch64" ;;
+    alpha) container_hosts=x86_64 ;;
+    arm) container_hosts="x86_64 aarch64" ;;
+    hexagon) container_hosts=x86_64 ;;
+    hppa) container_hosts=x86_64 ;;
+    i386) container_hosts=x86_64 ;;
+    loongarch64) container_hosts=x86_64 ;;
+    m68k) container_hosts=x86_64 ;;
+    microblaze) container_hosts=x86_64 ;;
+    mips64el) container_hosts=x86_64 ;;
+    mips64) container_hosts=x86_64 ;;
+    mipsel) container_hosts=x86_64 ;;
+    mips) container_hosts=x86_64 ;;
+    ppc) container_hosts=x86_64 ;;
+    ppc64|ppc64le) container_hosts=x86_64 ;;
+    riscv64) container_hosts=x86_64 ;;
+    s390x) container_hosts=x86_64 ;;
+    sh4) container_hosts=x86_64 ;;
+    sparc64) container_hosts=x86_64 ;;
+    tricore) container_hosts=x86_64 ;;
+    x86_64) container_hosts="aarch64 ppc64le x86_64" ;;
+    xtensa*) container_hosts=x86_64 ;;
+  esac
+
+  for host in $container_hosts; do
+    test "$container_command" != "" || continue
+    test "$host" = "$cpu" || continue
+    case $target_arch in
+      # debian-all-test-cross architectures
+
+      aarch64_be)
+        container_image=debian-all-test-cross
+        container_cross_prefix=aarch64-linux-gnu-
+        ;;
+      alpha|hppa|m68k|mips|mipsel|riscv64|sh4|sparc64)
+        container_image=debian-all-test-cross
+        ;;
+      mips64)
+        container_image=debian-all-test-cross
+        container_cross_prefix=mips64-linux-gnuabi64-
+        ;;
+      ppc|ppc64|ppc64le)
+        container_image=debian-all-test-cross
+        container_cross_prefix=powerpc${target_arch#ppc}-linux-gnu-
+        ;;
+
+      # architectures with individual containers
+
+      aarch64)
+        # We don't have any bigendian build tools so we only use this for AArch64
+        container_image=debian-arm64-cross
+        ;;
+      arm)
+        # We don't have any bigendian build tools so we only use this for ARM
+        container_image=debian-armhf-cross
+        container_cross_prefix=arm-linux-gnueabihf-
+        ;;
+      hexagon)
+        container_cross_prefix=hexagon-unknown-linux-musl-
+        container_cross_cc=${container_cross_prefix}clang
+        ;;
+      i386)
+        container_image=debian-i686-cross
+        container_cross_prefix=i686-linux-gnu-
+        ;;
+      loongarch64)
+        container_image=debian-loongarch-cross
+        container_cross_prefix=loongarch64-unknown-linux-gnu-
+        ;;
+      microblaze)
+        container_cross_prefix=microblaze-linux-musl-
+        ;;
+      mips64el)
+        container_image=debian-all-test-cross
+        container_cross_prefix=mips64el-linux-gnuabi64-
+        ;;
+      tricore)
+        container_image=debian-tricore-cross
+        container_cross_prefix=tricore-
+        ;;
+      x86_64)
+        container_image=debian-amd64-cross
+        ;;
+      xtensa*)
+        container_image=debian-xtensa-cross
+
+        # default to the dc232b cpu
+        container_cross_prefix=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-
+        ;;
+    esac
+    # Debian and GNU architecture names usually match
+    : ${container_image:=debian-$target_arch-cross}
+    : ${container_cross_prefix:=$target_arch-linux-gnu-}
+    : ${container_cross_cc:=${container_cross_prefix}gcc}
+    : ${container_cross_ar:=${container_cross_prefix}ar}
+    : ${container_cross_as:=${container_cross_prefix}as}
+    : ${container_cross_ld:=${container_cross_prefix}ld}
+    : ${container_cross_nm:=${container_cross_prefix}nm}
+    : ${container_cross_objcopy:=${container_cross_prefix}objcopy}
+    : ${container_cross_ranlib:=${container_cross_prefix}ranlib}
+    : ${container_cross_strip:=${container_cross_prefix}strip}
+  done
+
+  try=cross
+  # For softmmu/roms also look for a bi-endian or multilib-enabled host compiler
+  if [ "${1%softmmu}" != "$1" ] || test "$target_arch" = "$cpu"; then
+      case "$target_arch:$cpu" in
+        aarch64_be:aarch64 | \
+        armeb:arm | \
+        i386:x86_64 | \
+        ppc*:ppc64 | \
+        sparc:sparc64 | \
+        "$cpu:$cpu")
+        try='native cross' ;;
+      esac
+  fi
+  eval "target_cflags=\${cross_cc_cflags_$target_arch}"
+  for thistry in $try; do
+    case $thistry in
+    native)
+      target_cc=$cc
+      target_ccas=$ccas
+      target_ar=$ar
+      target_as=$as
+      target_ld=$ld
+      target_nm=$nm
+      target_objcopy=$objcopy
+      target_ranlib=$ranlib
+      target_strip=$strip
+      ;;
+    cross)
+      target_cc=
+      if eval test -n "\"\${cross_cc_$target_arch}\""; then
+        if eval has "\"\${cross_cc_$target_arch}\""; then
+          eval "target_cc=\"\${cross_cc_$target_arch}\""
+        fi
+      else
+        compute_target_variable $target_arch target_cc gcc
+      fi
+      target_ccas=$target_cc
+      compute_target_variable $target_arch target_ar ar
+      compute_target_variable $target_arch target_as as
+      compute_target_variable $target_arch target_ld ld
+      compute_target_variable $target_arch target_nm nm
+      compute_target_variable $target_arch target_objcopy objcopy
+      compute_target_variable $target_arch target_ranlib ranlib
+      compute_target_variable $target_arch target_strip strip
+      ;;
+    esac
+
+    if test -n "$target_cc"; then
+      case $target_arch in
+        i386|x86_64)
+          if $target_cc --version | grep -qi "clang"; then
+            continue
+          fi
+          ;;
+      esac
+    elif test -n "$target_as" && test -n "$target_ld"; then
+      # Special handling for assembler only targets
+      case $target in
+        tricore-softmmu)
+          build_static=
+          got_cross_cc=yes
+          break
+          ;;
+        *)
+          continue
+          ;;
+      esac
+    else
+      continue
+    fi
+
+    write_c_skeleton
+    case $1 in
+      *-softmmu)
+        if do_compiler "$target_cc" $target_cflags -o $TMPO -c $TMPC &&
+          do_compiler "$target_cc" $target_cflags -r -nostdlib -o "${TMPDIR1}/${TMPB}2.o" "$TMPO" -lgcc; then
+          got_cross_cc=yes
+          break
+        fi
+        ;;
+      *)
+        if do_compiler "$target_cc" $target_cflags -o $TMPE $TMPC -static ; then
+          build_static=y
+          got_cross_cc=yes
+          break
+        fi
+        if do_compiler "$target_cc" $target_cflags -o $TMPE $TMPC ; then
+          build_static=
+          got_cross_cc=yes
+          break
+        fi
+        ;;
+    esac
+  done
+  if test $got_cross_cc != yes; then
+    build_static=
+    target_cc=
+    target_ccas=
+    target_ar=
+    target_as=
+    target_ld=
+    target_nm=
+    target_objcopy=
+    target_ranlib=
+    target_strip=
+  fi
+  test -n "$target_cc"
+}
+
+write_target_makefile() {
+  echo "PYTHON=$python"
+  echo "EXTRA_CFLAGS=$target_cflags"
+  if test -z "$target_cc" && test -z "$target_as"; then
+    test -z "$container_image" && error_exit "Internal error: could not find cross compiler for $1?"
+    echo "$1: docker-image-$container_image" >> Makefile.prereqs
+    if test -n "$container_cross_cc"; then
+      echo "CC=$docker_py cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
+      echo "CCAS=$docker_py cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
+    fi
+    echo "AR=$docker_py cc --cc $container_cross_ar -i qemu/$container_image -s $source_path --"
+    echo "AS=$docker_py cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
+    echo "LD=$docker_py cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
+    echo "NM=$docker_py cc --cc $container_cross_nm -i qemu/$container_image -s $source_path --"
+    echo "OBJCOPY=$docker_py cc --cc $container_cross_objcopy -i qemu/$container_image -s $source_path --"
+    echo "RANLIB=$docker_py cc --cc $container_cross_ranlib -i qemu/$container_image -s $source_path --"
+    echo "STRIP=$docker_py cc --cc $container_cross_strip -i qemu/$container_image -s $source_path --"
+  else
+    if test -n "$target_cc"; then
+      echo "CC=$target_cc"
+      echo "CCAS=$target_ccas"
+    fi
+    if test -n "$target_ar"; then
+      echo "AR=$target_ar"
+    fi
+    if test -n "$target_as"; then
+      echo "AS=$target_as"
+    fi
+    if test -n "$target_ld"; then
+      echo "LD=$target_ld"
+    fi
+    if test -n "$target_nm"; then
+      echo "NM=$target_nm"
+    fi
+    if test -n "$target_objcopy"; then
+      echo "OBJCOPY=$target_objcopy"
+    fi
+    if test -n "$target_ranlib"; then
+      echo "RANLIB=$target_ranlib"
+    fi
+    if test -n "$target_strip"; then
+      echo "STRIP=$target_strip"
+    fi
+  fi
+}
+
 #######################################
 # cross-compiled firmware targets
 
@@ -1299,6 +1659,48 @@ fi
 
 echo "# Automatically generated by configure - do not modify" > Makefile.prereqs
 
+# Mac OS X ships with a broken assembler
+if have_target i386-softmmu x86_64-softmmu && \
+        test "$host_os" != "darwin" && test "$host_os" != "sunos" && \
+        test "$host_os" != "haiku" && \
+        probe_target_compiler i386-softmmu; then
+    subdirs="$subdirs pc-bios/optionrom"
+    config_mak=pc-bios/optionrom/config.mak
+    echo "# Automatically generated by configure - do not modify" > $config_mak
+    echo "TOPSRC_DIR=$source_path" >> $config_mak
+    write_target_makefile >> $config_mak
+fi
+
+if have_target ppc-softmmu ppc64-softmmu && \
+        probe_target_compiler ppc-softmmu; then
+    subdirs="$subdirs pc-bios/vof"
+    config_mak=pc-bios/vof/config.mak
+    echo "# Automatically generated by configure - do not modify" > $config_mak
+    echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
+    write_target_makefile >> $config_mak
+fi
+
+# Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
+# (which is the lowest architecture level that Clang supports)
+if have_target s390x-softmmu && probe_target_compiler s390x-softmmu && \
+    GIT=git "$source_path/scripts/git-submodule.sh" "$git_submodules_action" roms/SLOF >> config.log 2>&1; then
+  write_c_skeleton
+  do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
+  has_z900=$?
+  if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
+    if [ $has_z900 != 0 ]; then
+      echo "WARNING: Your compiler does not support the z900!"
+      echo "         The s390-ccw bios will only work with guest CPUs >= z10."
+    fi
+    subdirs="$subdirs pc-bios/s390-ccw"
+    config_mak=pc-bios/s390-ccw/config-host.mak
+    echo "# Automatically generated by configure - do not modify" > $config_mak
+    echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
+    echo "GIT_SUBMODULES_ACTION=$git_submodules_action" >> $config_mak
+    write_target_makefile >> $config_mak
+  fi
+fi
+
 #######################################
 # generate config-host.mak
 
@@ -1322,6 +1724,65 @@ if test "$default_targets" = "yes"; then
   echo "CONFIG_DEFAULT_TARGETS=y" >> $config_host_mak
 fi
 
+tcg_tests_with_compilers=
+for target in $target_list; do
+  arch=${target%%-*}
+
+  case $target in
+    xtensa*-linux-user)
+      # the toolchain is not complete with headers, only build system tests
+      continue
+      ;;
+    *-softmmu)
+      test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
+      qemu="qemu-system-$arch"
+      ;;
+    *-linux-user|*-bsd-user)
+      qemu="qemu-$arch"
+      ;;
+  esac
+
+  if probe_target_compiler $target || test -n "$container_image"; then
+      test -n "$container_image" && build_static=y
+      mkdir -p "tests/tcg/$target"
+      config_target_mak=tests/tcg/$target/config-target.mak
+      ln -sf "$source_path/tests/tcg/Makefile.target" "tests/tcg/$target/Makefile"
+      echo "# Automatically generated by configure - do not modify" > "$config_target_mak"
+      echo "TARGET_NAME=$arch" >> "$config_target_mak"
+      echo "TARGET=$target" >> "$config_target_mak"
+      write_target_makefile "build-tcg-tests-$target" >> "$config_target_mak"
+      echo "BUILD_STATIC=$build_static" >> "$config_target_mak"
+      echo "QEMU=$PWD/$qemu" >> "$config_target_mak"
+
+      # will GDB work with these binaries?
+      if test "${gdb_arches#*$arch}" != "$gdb_arches"; then
+          echo "GDB=$gdb_bin" >> $config_target_mak
+      fi
+
+      if test "${gdb_arches#*$arch}" != "$gdb_arches" && version_ge $gdb_version 14.1; then
+          echo "GDB_HAS_SME_TILES=y" >> $config_target_mak
+      else
+          echo "GDB_HAS_SME_TILES=n" >> $config_target_mak
+      fi
+
+      if test "${gdb_arches#*aarch64}" != "$gdb_arches" && version_ge $gdb_version 15.1; then
+          echo "GDB_HAS_MTE=y" >> $config_target_mak
+      fi
+
+      if test "${gdb_arches#*aarch64}" != "$gdb_arches" && version_ge $gdb_version 16.0; then
+          # GDB has to support MTE in baremetal to allow debugging MTE in QEMU system mode
+          echo "GDB_SUPPORTS_MTE_IN_BAREMETAL=y" >> $config_target_mak
+      fi
+
+      echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> Makefile.prereqs
+      tcg_tests_with_compilers="$tcg_tests_with_compilers $target"
+  fi
+done
+
+if test "$tcg" = "enabled"; then
+    echo "TCG_TESTS_WITH_COMPILERS=$tcg_tests_with_compilers" >> $config_host_mak
+fi
+
 if test "$skip_meson" = no; then
   cross="config-meson.cross.new"
   meson_quote() {
-- 
2.47.3

[PATCH v4 02/19] Revert "tests/tcg/meson.build: implement ./configure -cross-cc-* options"

[Alex Bennée]

We want to keep configuring of the cross compiler within configure so
it can be used by the firmware.

This reverts commit 7e1d54da6faf10e1fc70f6b3d8247068b80b1597.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v3
  - declare empty container_command/use_containers to avoid unref var
---
 configure             | 31 ++++++++-------
 meson_options.txt     | 89 -------------------------------------------
 tests/tcg/meson.build | 31 +++------------
 3 files changed, 23 insertions(+), 128 deletions(-)

diff --git a/configure b/configure
index 1973f023b1e..bedde58b8cf 100755
--- a/configure
+++ b/configure
@@ -172,6 +172,7 @@ fi
 # some defaults, based on the host environment
 
 # default parameters
+container_command=""
 cpu=""
 cross_compile="no"
 cross_prefix=""
@@ -227,6 +228,19 @@ for opt do
   ;;
   --extra-ldflags=*) EXTRA_LDFLAGS="$EXTRA_LDFLAGS $optarg"
   ;;
+  --cross-cc-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --cross-cc-FOO option"
+  ;;
+  --cross-cc-cflags-*) cc_arch=${opt#--cross-cc-cflags-}; cc_arch=${cc_arch%%=*}
+                      eval "cross_cc_cflags_${cc_arch}=\$optarg"
+  ;;
+  --cross-cc-*) cc_arch=${opt#--cross-cc-}; cc_arch=${cc_arch%%=*}
+                eval "cross_cc_${cc_arch}=\$optarg"
+  ;;
+  --cross-prefix-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --cross-prefix-FOO option"
+  ;;
+  --cross-prefix-*) cc_arch=${opt#--cross-prefix-}; cc_arch=${cc_arch%%=*}
+                    eval "cross_prefix_${cc_arch}=\$optarg"
+  ;;
   --without-default-features) default_feature="no"
   ;;
   --wasm64-32bit-address-limit) wasm64_memory64="2"
@@ -246,6 +260,7 @@ ninja=""
 python=
 download="enabled"
 skip_meson=no
+use_containers=""
 rust="disabled"
 rust_target_triple=""
 
@@ -618,19 +633,9 @@ for opt do
   ;;
   --extra-ldflags=*)
   ;;
-  --cross-cc-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --cross-cc-FOO option"
-  ;;
-  --cross-cc-cflags-*) arch=${opt#--cross-cc-cflags-}; arch=${arch%%=*}
-                       meson_option_add "-Dtcg_tests_cross_cflags_${arch}=${optarg}"
-  ;;
-  --cross-cc-*) arch=${opt#--cross-cc-}; arch=${arch%%=*}
-                meson_option_add "-Dtcg_tests_cross_cc_${arch}=${optarg}"
-  ;;
-  --cross-prefix-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --cross-prefix-FOO option"
+  --cross-cc-*)
   ;;
-  # cross-prefix is just used as a syntactic sugar for -cross-cc.
-  --cross-prefix-*) arch=${opt#--cross-prefix-}; arch=${arch%%=*}
-                    meson_option_add "-Dtcg_tests_cross_cc_${arch}=${optarg}gcc"
+  --cross-prefix-*)
   ;;
   --enable-docs) docs=enabled
   ;;
@@ -1273,7 +1278,7 @@ if test "$container_command" = ""; then
     container_command=$($python "$source_path"/tests/docker/docker.py probe)
     test "$container_command" = "no" && container_command=""
 fi
-if test $use_containers = "yes" && test "$container_command" != ""; then
+if test "$use_containers" = "yes" && test "$container_command" != ""; then
     docker_py="$python $source_path/tests/docker/docker.py --command $container_command"
 fi
 
diff --git a/meson_options.txt b/meson_options.txt
index e3570a418aa..66d3689d14c 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -393,92 +393,3 @@ option('containers', type: 'boolean', value: true,
        description: 'use containers to cross compile tcg tests')
 option('container_command', type: 'string',
        description: 'command to build/run containers')
-
-option('tcg_tests_cross_cc_aarch64', type: 'string',
-       description: 'cc for aarch64 tcg tests')
-option('tcg_tests_cross_cflags_aarch64', type: 'string',
-       description: 'cflags for aarch64 tcg tests')
-option('tcg_tests_cross_cc_aarch64_be', type: 'string',
-       description: 'cc for aarch64_be tcg tests')
-option('tcg_tests_cross_cflags_aarch64_be', type: 'string',
-       description: 'cflags for aarch64_be tcg tests')
-option('tcg_tests_cross_cc_alpha', type: 'string',
-       description: 'cc for alpha tcg tests')
-option('tcg_tests_cross_cflags_alpha', type: 'string',
-       description: 'cflags for alpha tcg tests')
-option('tcg_tests_cross_cc_arm', type: 'string',
-       description: 'cc for arm tcg tests')
-option('tcg_tests_cross_cflags_arm', type: 'string',
-       description: 'cflags for arm tcg tests')
-option('tcg_tests_cross_cc_hexagon', type: 'string',
-       description: 'cc for hexagon tcg tests')
-option('tcg_tests_cross_cflags_hexagon', type: 'string',
-       description: 'cflags for hexagon tcg tests')
-option('tcg_tests_cross_cc_hppa', type: 'string',
-       description: 'cc for hppa tcg tests')
-option('tcg_tests_cross_cflags_hppa', type: 'string',
-       description: 'cflags for hppa tcg tests')
-option('tcg_tests_cross_cc_i386', type: 'string',
-       description: 'cc for i386 tcg tests')
-option('tcg_tests_cross_cflags_i386', type: 'string',
-       description: 'cflags for i386 tcg tests')
-option('tcg_tests_cross_cc_loongarch64', type: 'string',
-       description: 'cc for loongarch64 tcg tests')
-option('tcg_tests_cross_cflags_loongarch64', type: 'string',
-       description: 'cflags for loongarch64 tcg tests')
-option('tcg_tests_cross_cc_m68k', type: 'string',
-       description: 'cc for m68k tcg tests')
-option('tcg_tests_cross_cflags_m68k', type: 'string',
-       description: 'cflags for m68k tcg tests')
-option('tcg_tests_cross_cc_mips', type: 'string',
-       description: 'cc for mips tcg tests')
-option('tcg_tests_cross_cflags_mips', type: 'string',
-       description: 'cflags for mips tcg tests')
-option('tcg_tests_cross_cc_mips64', type: 'string',
-       description: 'cc for mips64 tcg tests')
-option('tcg_tests_cross_cflags_mips64', type: 'string',
-       description: 'cflags for mips64 tcg tests')
-option('tcg_tests_cross_cc_mips64el', type: 'string',
-       description: 'cc for mips64el tcg tests')
-option('tcg_tests_cross_cflags_mips64el', type: 'string',
-       description: 'cflags for mips64el tcg tests')
-option('tcg_tests_cross_cc_or1k', type: 'string',
-       description: 'cc for or1k tcg tests')
-option('tcg_tests_cross_cflags_or1k', type: 'string',
-       description: 'cflags for or1k tcg tests')
-option('tcg_tests_cross_cc_ppc64', type: 'string',
-       description: 'cc for ppc64 tcg tests')
-option('tcg_tests_cross_cflags_ppc64', type: 'string',
-       description: 'cflags for ppc64 tcg tests')
-option('tcg_tests_cross_cc_ppc64le', type: 'string',
-       description: 'cc for ppc64le tcg tests')
-option('tcg_tests_cross_cflags_ppc64le', type: 'string',
-       description: 'cflags for ppc64le tcg tests')
-option('tcg_tests_cross_cc_riscv64', type: 'string',
-       description: 'cc for riscv64 tcg tests')
-option('tcg_tests_cross_cflags_riscv64', type: 'string',
-       description: 'cflags for riscv64 tcg tests')
-option('tcg_tests_cross_cc_s390x', type: 'string',
-       description: 'cc for s390x tcg tests')
-option('tcg_tests_cross_cflags_s390x', type: 'string',
-       description: 'cflags for s390x tcg tests')
-option('tcg_tests_cross_cc_sh4', type: 'string',
-       description: 'cc for sh4 tcg tests')
-option('tcg_tests_cross_cflags_sh4', type: 'string',
-       description: 'cflags for sh4 tcg tests')
-option('tcg_tests_cross_cc_tricore', type: 'string',
-       description: 'cc for tricore tcg tests')
-option('tcg_tests_cross_cflags_tricore', type: 'string',
-       description: 'cflags for tricore tcg tests')
-option('tcg_tests_cross_cc_x86_64', type: 'string',
-       description: 'cc for x86_64 tcg tests')
-option('tcg_tests_cross_cflags_x86_64', type: 'string',
-       description: 'cflags for x86_64 tcg tests')
-option('tcg_tests_cross_cc_xtensa', type: 'string',
-       description: 'cc for xtensa tcg tests')
-option('tcg_tests_cross_cflags_xtensa', type: 'string',
-       description: 'cflags for xtensa tcg tests')
-option('tcg_tests_cross_cc_xtensaeb', type: 'string',
-       description: 'cc for xtensaeb tcg tests')
-option('tcg_tests_cross_cflags_xtensaeb', type: 'string',
-       description: 'cflags for xtensaeb tcg tests')
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index d41a228fb3d..ce208e686ff 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -185,8 +185,6 @@ foreach target, plan: tcg_tests
     continue
   endif
 
-  cc_arch = target.replace('-linux-user', '').replace('-softmmu', '')
-
   # Detect duplicated executables/tests, and report an error to force user to
   # choose how to deal with it.
   built_tests = {}
@@ -204,35 +202,16 @@ foreach target, plan: tcg_tests
   endforeach
 
   cc = find_program(plan['cc'], required : false)
-  cc_cflags = []
-
-  check_flags = []
-  check_required = false
-  cross_cc = get_option('tcg_tests_cross_cc_' + cc_arch)
-  if cross_cc != ''
-    # we make sure cross cc exists
-    cc = find_program(cross_cc, required: true)
-    check_required = true
-  endif
-  cross_cflags = get_option('tcg_tests_cross_cflags_' + cc_arch)
-  if cross_cflags != ''
-    cc_cflags = cross_cflags.split()
-    # and that cross cc can compile programs
-    check_flags = cc_cflags
-    check_required = true
-  endif
-
   cc_from_system = cc.found()
   has_cc = cc.found()
   if has_cc
+    check_flags = ['-static']
     if target.endswith('softmmu')
-      check_flags += ['-nostdlib', '-ffreestanding', '-r']
-    else
-      check_flags += ['-static']
+      check_flags = ['-nostdlib', '-ffreestanding', '-r']
     endif
 
     cmd = run_command([cc, files('test_cc.c'), check_flags, '-o', '/dev/null'],
-                      check: check_required)
+                      check: false)
     has_cc = cmd.returncode() == 0
     cc_from_system = has_cc
   endif
@@ -368,9 +347,9 @@ foreach target, plan: tcg_tests
         endif
       endif
 
-      cflags = cc_cflags
+      cflags = []
       if 'cflags' in setup
-        cflags += setup['cflags']
+        cflags = setup['cflags']
       endif
 
       if 'cc_feat' in setup
-- 
2.47.3

Re: [PATCH v4 02/19] Revert "tests/tcg/meson.build: implement ./configure -cross-cc-* options"

[Pierrick Bouvier]

On 9/16/2026 8:29 AM, Alex Bennée wrote:
> We want to keep configuring of the cross compiler within configure so
> it can be used by the firmware.
>

Would that be possible to add in command message that it disable effect
of those options for tcg-tests?

> This reverts commit 7e1d54da6faf10e1fc70f6b3d8247068b80b1597.
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v3
>   - declare empty container_command/use_containers to avoid unref var
> ---
>  configure             | 31 ++++++++-------
>  meson_options.txt     | 89 -------------------------------------------
>  tests/tcg/meson.build | 31 +++------------
>  3 files changed, 23 insertions(+), 128 deletions(-)
> 

For the rest,
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

Regards,
Pierrick

Re: [PATCH v4 02/19] Revert "tests/tcg/meson.build: implement ./configure -cross-cc-* options"

[Alex Bennée]

Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:

> On 9/16/2026 8:29 AM, Alex Bennée wrote:
>> We want to keep configuring of the cross compiler within configure so
>> it can be used by the firmware.
>>
>
> Would that be possible to add in command message that it disable effect
> of those options for tcg-tests?

I'm wondering if we should revert this at all, or just merge with the
meson commands (because currently this breaks the MacOS tcg checks)
>
>> This reverts commit 7e1d54da6faf10e1fc70f6b3d8247068b80b1597.
>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> 
>> ---
>> v3
>>   - declare empty container_command/use_containers to avoid unref var
>> ---
>>  configure             | 31 ++++++++-------
>>  meson_options.txt     | 89 -------------------------------------------
>>  tests/tcg/meson.build | 31 +++------------
>>  3 files changed, 23 insertions(+), 128 deletions(-)
>> 
>
> For the rest,
> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>
> Regards,
> Pierrick

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v4 02/19] Revert "tests/tcg/meson.build: implement ./configure -cross-cc-* options"

[Pierrick Bouvier]

On 9/17/2026 6:24 AM, Alex Bennée wrote:
> Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:
> 
>> On 9/16/2026 8:29 AM, Alex Bennée wrote:
>>> We want to keep configuring of the cross compiler within configure so
>>> it can be used by the firmware.
>>>
>>
>> Would that be possible to add in command message that it disable effect
>> of those options for tcg-tests?
> 
> I'm wondering if we should revert this at all, or just merge with the
> meson commands (because currently this breaks the MacOS tcg checks)

That's what I mentioned a few times indeed.

I think it would be better to keep those options, so everything works as
expected, and just merge back with config.mak generation.

Yes, it means we pass information from configure step to meson, but I
don't see the problem. Also, we'll need to pass this information anyway
if we want to integrate build for vdso in meson directly.

>>
>>> This reverts commit 7e1d54da6faf10e1fc70f6b3d8247068b80b1597.
>>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>>
>>> ---
>>> v3
>>>   - declare empty container_command/use_containers to avoid unref var
>>> ---
>>>  configure             | 31 ++++++++-------
>>>  meson_options.txt     | 89 -------------------------------------------
>>>  tests/tcg/meson.build | 31 +++------------
>>>  3 files changed, 23 insertions(+), 128 deletions(-)
>>>
>>
>> For the rest,
>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>
>> Regards,
>> Pierrick
> 

[PATCH v4 03/19] Revert "tests/tcg/meson.build: implement ./configure containers options"

[Alex Bennée]

We want to keep the container configuration inside configure for the
firmware builds.

This partially reverts commit 522b299ca4316d8cad57f9496ffcba87dcc20719
while keeping the plumbing to disable containers within meson.

Tested-by: Eric Farman <farman@linux.ibm.com>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
[AJB: untangle changes since]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Jared Rossi <jrossi@linux.ibm.com>

---
v4
  - tweak revert to keep disable reported
  - update commit msg
---
 configure             | 14 ++++++++++----
 meson_options.txt     |  3 +++
 tests/tcg/meson.build |  3 +++
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/configure b/configure
index bedde58b8cf..6084a7a065e 100755
--- a/configure
+++ b/configure
@@ -260,7 +260,7 @@ ninja=""
 python=
 download="enabled"
 skip_meson=no
-use_containers=""
+use_containers="yes"
 rust="disabled"
 rust_target_triple=""
 
@@ -727,11 +727,14 @@ for opt do
   ;;
   --disable-plugins) plugins="no"
   ;;
-  --enable-containers) # enabled by default
+  --enable-containers) use_containers="yes"
+                       meson_option_add -Dcontainers=true
   ;;
-  --disable-containers) meson_option_add -Dcontainers=false
+  --disable-containers) use_containers="no"
+                        meson_option_add -Dcontainers=false
   ;;
-  --container-command=*) meson_option_add -Dcontainer_command="$optarg"
+  --container-command=*) container_command="$optarg"
+                         meson_option_add -Dcontainer_command="$optarg"
   ;;
   --rust-target-triple=*) rust_target_triple="$optarg"
   ;;
@@ -1718,6 +1721,9 @@ echo all: >> $config_host_mak
 
 echo "SRC_PATH=$source_path" >> $config_host_mak
 echo "TARGET_DIRS=$target_list" >> $config_host_mak
+if test "$container_command" != ""; then
+    echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak
+fi
 echo "SUBDIRS=$subdirs" >> $config_host_mak
 echo "PYTHON=$python" >> $config_host_mak
 echo "MKVENV_ENSUREGROUP=$mkvenv ensuregroup $mkvenv_online_flag" >> $config_host_mak
diff --git a/meson_options.txt b/meson_options.txt
index 66d3689d14c..daf301ed48a 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -389,6 +389,9 @@ option('rust', type: 'feature', value: 'disabled',
        description: 'Rust support')
 option('strict_rust_lints', type: 'boolean', value: false,
        description: 'Enable stricter set of Rust warnings')
+option('container_registry', type: 'string',
+       value: 'registry.gitlab.com/qemu-project/qemu',
+       description: 'container registry to cache from')
 option('containers', type: 'boolean', value: true,
        description: 'use containers to cross compile tcg tests')
 option('container_command', type: 'string',
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index ce208e686ff..890da7b81e9 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -101,6 +101,9 @@ if get_option('containers')
   endif
 endif
 tcg_tests_summary += {'container command': container_command}
+if get_option('container_registry') != ''
+  tcg_tests_summary += {'container registry': get_option('container_registry')}
+endif
 
 # plugins come first, as we need to build the list
 test_plugins = {}
-- 
2.47.3

Re: [PATCH v4 03/19] Revert "tests/tcg/meson.build: implement ./configure containers options"

[Pierrick Bouvier]

On 9/16/2026 8:29 AM, Alex Bennée wrote:
> We want to keep the container configuration inside configure for the
> firmware builds.
> 
> This partially reverts commit 522b299ca4316d8cad57f9496ffcba87dcc20719
> while keeping the plumbing to disable containers within meson.
> 
> Tested-by: Eric Farman <farman@linux.ibm.com>
> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> [AJB: untangle changes since]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Tested-by: Jared Rossi <jrossi@linux.ibm.com>
> 
> ---
> v4
>   - tweak revert to keep disable reported
>   - update commit msg
> ---
>  configure             | 14 ++++++++++----
>  meson_options.txt     |  3 +++
>  tests/tcg/meson.build |  3 +++
>  3 files changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/configure b/configure
> index bedde58b8cf..6084a7a065e 100755
> --- a/configure
> +++ b/configure
> @@ -260,7 +260,7 @@ ninja=""
>  python=
>  download="enabled"
>  skip_meson=no
> -use_containers=""
> +use_containers="yes"
>  rust="disabled"
>  rust_target_triple=""
>  
> @@ -727,11 +727,14 @@ for opt do
>    ;;
>    --disable-plugins) plugins="no"
>    ;;
> -  --enable-containers) # enabled by default
> +  --enable-containers) use_containers="yes"
> +                       meson_option_add -Dcontainers=true
>    ;;
> -  --disable-containers) meson_option_add -Dcontainers=false
> +  --disable-containers) use_containers="no"
> +                        meson_option_add -Dcontainers=false
>    ;;
> -  --container-command=*) meson_option_add -Dcontainer_command="$optarg"
> +  --container-command=*) container_command="$optarg"
> +                         meson_option_add -Dcontainer_command="$optarg"

Not sure if we want to keep this option (meson container_command).
Related if in tests/tcg/meson.build can be removed also.

>    ;;
>    --rust-target-triple=*) rust_target_triple="$optarg"
>    ;;
> @@ -1718,6 +1721,9 @@ echo all: >> $config_host_mak
>  
>  echo "SRC_PATH=$source_path" >> $config_host_mak
>  echo "TARGET_DIRS=$target_list" >> $config_host_mak
> +if test "$container_command" != ""; then
> +    echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak
> +fi
>  echo "SUBDIRS=$subdirs" >> $config_host_mak
>  echo "PYTHON=$python" >> $config_host_mak
>  echo "MKVENV_ENSUREGROUP=$mkvenv ensuregroup $mkvenv_online_flag" >> $config_host_mak
> diff --git a/meson_options.txt b/meson_options.txt
> index 66d3689d14c..daf301ed48a 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -389,6 +389,9 @@ option('rust', type: 'feature', value: 'disabled',
>         description: 'Rust support')
>  option('strict_rust_lints', type: 'boolean', value: false,
>         description: 'Enable stricter set of Rust warnings')
> +option('container_registry', type: 'string',
> +       value: 'registry.gitlab.com/qemu-project/qemu',
> +       description: 'container registry to cache from')
>  option('containers', type: 'boolean', value: true,
>         description: 'use containers to cross compile tcg tests')
>  option('container_command', type: 'string',
> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
> index ce208e686ff..890da7b81e9 100644
> --- a/tests/tcg/meson.build
> +++ b/tests/tcg/meson.build
> @@ -101,6 +101,9 @@ if get_option('containers')
>    endif
>  endif
>  tcg_tests_summary += {'container command': container_command}
> +if get_option('container_registry') != ''
> +  tcg_tests_summary += {'container registry': get_option('container_registry')}
> +endif
> 

Rebase issue?

>  # plugins come first, as we need to build the list
>  test_plugins = {}

Re: [PATCH v4 03/19] Revert "tests/tcg/meson.build: implement ./configure containers options"

[Alex Bennée]

Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:

> On 9/16/2026 8:29 AM, Alex Bennée wrote:
>> We want to keep the container configuration inside configure for the
>> firmware builds.
>> 
>> This partially reverts commit 522b299ca4316d8cad57f9496ffcba87dcc20719
>> while keeping the plumbing to disable containers within meson.
>> 
>> Tested-by: Eric Farman <farman@linux.ibm.com>
>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>> [AJB: untangle changes since]
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Tested-by: Jared Rossi <jrossi@linux.ibm.com>
>> 
>> ---
>> v4
>>   - tweak revert to keep disable reported
>>   - update commit msg
>> ---
>>  configure             | 14 ++++++++++----
>>  meson_options.txt     |  3 +++
>>  tests/tcg/meson.build |  3 +++
>>  3 files changed, 16 insertions(+), 4 deletions(-)
>> 
>> diff --git a/configure b/configure
>> index bedde58b8cf..6084a7a065e 100755
>> --- a/configure
>> +++ b/configure
>> @@ -260,7 +260,7 @@ ninja=""
>>  python=
>>  download="enabled"
>>  skip_meson=no
>> -use_containers=""
>> +use_containers="yes"
>>  rust="disabled"
>>  rust_target_triple=""
>>  
>> @@ -727,11 +727,14 @@ for opt do
>>    ;;
>>    --disable-plugins) plugins="no"
>>    ;;
>> -  --enable-containers) # enabled by default
>> +  --enable-containers) use_containers="yes"
>> +                       meson_option_add -Dcontainers=true
>>    ;;
>> -  --disable-containers) meson_option_add -Dcontainers=false
>> +  --disable-containers) use_containers="no"
>> +                        meson_option_add -Dcontainers=false
>>    ;;
>> -  --container-command=*) meson_option_add -Dcontainer_command="$optarg"
>> +  --container-command=*) container_command="$optarg"
>> +                         meson_option_add -Dcontainer_command="$optarg"
>
> Not sure if we want to keep this option (meson container_command).
> Related if in tests/tcg/meson.build can be removed also.

We want Makefile.include and meson to use the same container runtime
surely?

>
>>    ;;
>>    --rust-target-triple=*) rust_target_triple="$optarg"
>>    ;;
>> @@ -1718,6 +1721,9 @@ echo all: >> $config_host_mak
>>  
>>  echo "SRC_PATH=$source_path" >> $config_host_mak
>>  echo "TARGET_DIRS=$target_list" >> $config_host_mak
>> +if test "$container_command" != ""; then
>> +    echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak
>> +fi
>>  echo "SUBDIRS=$subdirs" >> $config_host_mak
>>  echo "PYTHON=$python" >> $config_host_mak
>>  echo "MKVENV_ENSUREGROUP=$mkvenv ensuregroup $mkvenv_online_flag" >> $config_host_mak
>> diff --git a/meson_options.txt b/meson_options.txt
>> index 66d3689d14c..daf301ed48a 100644
>> --- a/meson_options.txt
>> +++ b/meson_options.txt
>> @@ -389,6 +389,9 @@ option('rust', type: 'feature', value: 'disabled',
>>         description: 'Rust support')
>>  option('strict_rust_lints', type: 'boolean', value: false,
>>         description: 'Enable stricter set of Rust warnings')
>> +option('container_registry', type: 'string',
>> +       value: 'registry.gitlab.com/qemu-project/qemu',
>> +       description: 'container registry to cache from')
>>  option('containers', type: 'boolean', value: true,
>>         description: 'use containers to cross compile tcg tests')
>>  option('container_command', type: 'string',
>> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>> index ce208e686ff..890da7b81e9 100644
>> --- a/tests/tcg/meson.build
>> +++ b/tests/tcg/meson.build
>> @@ -101,6 +101,9 @@ if get_option('containers')
>>    endif
>>  endif
>>  tcg_tests_summary += {'container command': container_command}
>> +if get_option('container_registry') != ''
>> +  tcg_tests_summary += {'container registry': get_option('container_registry')}
>> +endif
>> 
>
> Rebase issue?

Possibly - it got messy with the reverts.

>
>>  # plugins come first, as we need to build the list
>>  test_plugins = {}

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v4 03/19] Revert "tests/tcg/meson.build: implement ./configure containers options"

[Pierrick Bouvier]

On 9/17/2026 2:27 AM, Alex Bennée wrote:
> Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:
> 
>> On 9/16/2026 8:29 AM, Alex Bennée wrote:
>>> We want to keep the container configuration inside configure for the
>>> firmware builds.
>>>
>>> This partially reverts commit 522b299ca4316d8cad57f9496ffcba87dcc20719
>>> while keeping the plumbing to disable containers within meson.
>>>
>>> Tested-by: Eric Farman <farman@linux.ibm.com>
>>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>>> [AJB: untangle changes since]
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> Tested-by: Jared Rossi <jrossi@linux.ibm.com>
>>>
>>> ---
>>> v4
>>>   - tweak revert to keep disable reported
>>>   - update commit msg
>>> ---
>>>  configure             | 14 ++++++++++----
>>>  meson_options.txt     |  3 +++
>>>  tests/tcg/meson.build |  3 +++
>>>  3 files changed, 16 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/configure b/configure
>>> index bedde58b8cf..6084a7a065e 100755
>>> --- a/configure
>>> +++ b/configure
>>> @@ -260,7 +260,7 @@ ninja=""
>>>  python=
>>>  download="enabled"
>>>  skip_meson=no
>>> -use_containers=""
>>> +use_containers="yes"
>>>  rust="disabled"
>>>  rust_target_triple=""
>>>  
>>> @@ -727,11 +727,14 @@ for opt do
>>>    ;;
>>>    --disable-plugins) plugins="no"
>>>    ;;
>>> -  --enable-containers) # enabled by default
>>> +  --enable-containers) use_containers="yes"
>>> +                       meson_option_add -Dcontainers=true
>>>    ;;
>>> -  --disable-containers) meson_option_add -Dcontainers=false
>>> +  --disable-containers) use_containers="no"
>>> +                        meson_option_add -Dcontainers=false
>>>    ;;
>>> -  --container-command=*) meson_option_add -Dcontainer_command="$optarg"
>>> +  --container-command=*) container_command="$optarg"
>>> +                         meson_option_add -Dcontainer_command="$optarg"
>>
>> Not sure if we want to keep this option (meson container_command).
>> Related if in tests/tcg/meson.build can be removed also.
> 
> We want Makefile.include and meson to use the same container runtime
> surely?
>

Agree. It seems like the general direction was "revert" everything, but
yes, I think it would be better to keep all meson options as they are
currently.

>>
>>>    ;;
>>>    --rust-target-triple=*) rust_target_triple="$optarg"
>>>    ;;
>>> @@ -1718,6 +1721,9 @@ echo all: >> $config_host_mak
>>>  
>>>  echo "SRC_PATH=$source_path" >> $config_host_mak
>>>  echo "TARGET_DIRS=$target_list" >> $config_host_mak
>>> +if test "$container_command" != ""; then
>>> +    echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak
>>> +fi
>>>  echo "SUBDIRS=$subdirs" >> $config_host_mak
>>>  echo "PYTHON=$python" >> $config_host_mak
>>>  echo "MKVENV_ENSUREGROUP=$mkvenv ensuregroup $mkvenv_online_flag" >> $config_host_mak
>>> diff --git a/meson_options.txt b/meson_options.txt
>>> index 66d3689d14c..daf301ed48a 100644
>>> --- a/meson_options.txt
>>> +++ b/meson_options.txt
>>> @@ -389,6 +389,9 @@ option('rust', type: 'feature', value: 'disabled',
>>>         description: 'Rust support')
>>>  option('strict_rust_lints', type: 'boolean', value: false,
>>>         description: 'Enable stricter set of Rust warnings')
>>> +option('container_registry', type: 'string',
>>> +       value: 'registry.gitlab.com/qemu-project/qemu',
>>> +       description: 'container registry to cache from')
>>>  option('containers', type: 'boolean', value: true,
>>>         description: 'use containers to cross compile tcg tests')
>>>  option('container_command', type: 'string',
>>> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
>>> index ce208e686ff..890da7b81e9 100644
>>> --- a/tests/tcg/meson.build
>>> +++ b/tests/tcg/meson.build
>>> @@ -101,6 +101,9 @@ if get_option('containers')
>>>    endif
>>>  endif
>>>  tcg_tests_summary += {'container command': container_command}
>>> +if get_option('container_registry') != ''
>>> +  tcg_tests_summary += {'container registry': get_option('container_registry')}
>>> +endif
>>>
>>
>> Rebase issue?
> 
> Possibly - it got messy with the reverts.
> 
>>
>>>  # plugins come first, as we need to build the list
>>>  test_plugins = {}
> 

[PATCH v4 04/19] configure: correctly honour --disable-containers

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The configure script originally only probed for 'runc' command
in order to handle cross-compilation containers. The Makefile
under tests/docker would then probe again when used.

In order to eliminate the additional probeing, we changed
configure in commit c4ce04cfb7460d46c0262d437a7f35bb5c5fc449
to always probe for 'runc', so the result could be passed
along to tests/docker make rules.

That commit overlooked that the 'probe_target_compiler' func
was relying on '$runc' == 'no' as a proxy for the state of
the --disable-containers arg.

When we started unconditionally probing, that short cut no
longer works and we must explicitly check "$use_containers"
too.

Fixes: c4ce04cfb7460d46c0262d437a7f35bb5c5fc449
Reported-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 configure | 1 +
 1 file changed, 1 insertion(+)

diff --git a/configure b/configure
index 6084a7a065e..4c7bf406b88 100755
--- a/configure
+++ b/configure
@@ -1401,6 +1401,7 @@ probe_target_compiler() {
   esac
 
   for host in $container_hosts; do
+    test "$use_containers" = "yes" || continue
     test "$container_command" != "" || continue
     test "$host" = "$cpu" || continue
     case $target_arch in
-- 
2.47.3

[PATCH v4 05/19] configure: re-purpose the tcg test compiler configs for vdso building

[Alex Bennée]

All the tcg tests are in meson now so there is no reason to put the
config-target.mak files in those directories. We also don't need the
GDB feature tests or to emit prerequsistes.

The vdso re-build blob does use the compiler definitions though so
we can emit the details for that and tweak the vdso makefiles to point
at the right place. Note some of the vdso makefiles build both 32 and
64 bit versions from the same compiler.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure                            | 69 +++++++++++-----------------
 linux-user/aarch64/Makefile.vdso     |  2 +-
 linux-user/arm/Makefile.vdso         |  2 +-
 linux-user/hppa/Makefile.vdso        |  2 +-
 linux-user/i386/Makefile.vdso        |  2 +-
 linux-user/loongarch64/Makefile.vdso |  2 +-
 linux-user/ppc/Makefile.vdso         |  2 +-
 linux-user/riscv/Makefile.vdso       |  2 +-
 linux-user/s390x/Makefile.vdso       |  2 +-
 linux-user/sh4/Makefile.vdso         |  2 +-
 linux-user/x86_64/Makefile.vdso      |  2 +-
 11 files changed, 36 insertions(+), 53 deletions(-)

diff --git a/configure b/configure
index 4c7bf406b88..67967cd294c 100755
--- a/configure
+++ b/configure
@@ -1736,65 +1736,48 @@ if test "$default_targets" = "yes"; then
   echo "CONFIG_DEFAULT_TARGETS=y" >> $config_host_mak
 fi
 
-tcg_tests_with_compilers=
+# Export the compiler details needed for rebuilding the vdso blobs
 for target in $target_list; do
-  arch=${target%%-*}
 
+  # Skip anything that is not a linux-user target
   case $target in
+    *-linux-user)
+      ;;
+    *)
+      continue
+      ;;
+  esac
+
+  arch=${target%%-*}
+
+  case $arch in
     xtensa*-linux-user)
-      # the toolchain is not complete with headers, only build system tests
+      # the toolchain can only build system tests due to missing headers
+      continue
+      ;;
+    aarch64_be-linux-user)
+      # there is no BE user space
       continue
       ;;
-    *-softmmu)
-      test -f "$source_path/tests/tcg/$arch/Makefile.softmmu-target" || continue
-      qemu="qemu-system-$arch"
+    aarch64|arm|hppa|i386|loongarch64|ppc64|riscv64|s390x|sh4|x86_64)
+      # arches which need a vdso, some 64 bit arches build both
       ;;
-    *-linux-user|*-bsd-user)
-      qemu="qemu-$arch"
+    *)
+      continue
       ;;
   esac
 
-  if probe_target_compiler $target || test -n "$container_image"; then
-      test -n "$container_image" && build_static=y
-      mkdir -p "tests/tcg/$target"
-      config_target_mak=tests/tcg/$target/config-target.mak
-      ln -sf "$source_path/tests/tcg/Makefile.target" "tests/tcg/$target/Makefile"
+  # note container_image is set by a successful call to probe_target_compiler
+  if probe_target_compiler "$target" || test -n "$container_image"; then
+      mkdir -p "linux-user/$arch"
+      config_target_mak=linux-user/$arch/config-target.mak
       echo "# Automatically generated by configure - do not modify" > "$config_target_mak"
       echo "TARGET_NAME=$arch" >> "$config_target_mak"
       echo "TARGET=$target" >> "$config_target_mak"
-      write_target_makefile "build-tcg-tests-$target" >> "$config_target_mak"
-      echo "BUILD_STATIC=$build_static" >> "$config_target_mak"
-      echo "QEMU=$PWD/$qemu" >> "$config_target_mak"
-
-      # will GDB work with these binaries?
-      if test "${gdb_arches#*$arch}" != "$gdb_arches"; then
-          echo "GDB=$gdb_bin" >> $config_target_mak
-      fi
-
-      if test "${gdb_arches#*$arch}" != "$gdb_arches" && version_ge $gdb_version 14.1; then
-          echo "GDB_HAS_SME_TILES=y" >> $config_target_mak
-      else
-          echo "GDB_HAS_SME_TILES=n" >> $config_target_mak
-      fi
-
-      if test "${gdb_arches#*aarch64}" != "$gdb_arches" && version_ge $gdb_version 15.1; then
-          echo "GDB_HAS_MTE=y" >> $config_target_mak
-      fi
-
-      if test "${gdb_arches#*aarch64}" != "$gdb_arches" && version_ge $gdb_version 16.0; then
-          # GDB has to support MTE in baremetal to allow debugging MTE in QEMU system mode
-          echo "GDB_SUPPORTS_MTE_IN_BAREMETAL=y" >> $config_target_mak
-      fi
-
-      echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> Makefile.prereqs
-      tcg_tests_with_compilers="$tcg_tests_with_compilers $target"
+      write_target_makefile "update-linux-vdso" >> "$config_target_mak"
   fi
 done
 
-if test "$tcg" = "enabled"; then
-    echo "TCG_TESTS_WITH_COMPILERS=$tcg_tests_with_compilers" >> $config_host_mak
-fi
-
 if test "$skip_meson" = no; then
   cross="config-meson.cross.new"
   meson_quote() {
diff --git a/linux-user/aarch64/Makefile.vdso b/linux-user/aarch64/Makefile.vdso
index c33a679c0f2..82ed2011087 100644
--- a/linux-user/aarch64/Makefile.vdso
+++ b/linux-user/aarch64/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/aarch64-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/aarch64/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/aarch64
 VPATH += $(SUBDIR)
diff --git a/linux-user/arm/Makefile.vdso b/linux-user/arm/Makefile.vdso
index ede489e2363..16ae8d7aaae 100644
--- a/linux-user/arm/Makefile.vdso
+++ b/linux-user/arm/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/arm-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/arm/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/arm
 VPATH += $(SUBDIR)
diff --git a/linux-user/hppa/Makefile.vdso b/linux-user/hppa/Makefile.vdso
index f4537ae7169..78642c42661 100644
--- a/linux-user/hppa/Makefile.vdso
+++ b/linux-user/hppa/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/hppa-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/hppa/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/hppa
 VPATH += $(SUBDIR)
diff --git a/linux-user/i386/Makefile.vdso b/linux-user/i386/Makefile.vdso
index 95bc616f6d6..ca2e37fa6b6 100644
--- a/linux-user/i386/Makefile.vdso
+++ b/linux-user/i386/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/i386-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/i386/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/i386
 VPATH += $(SUBDIR)
diff --git a/linux-user/loongarch64/Makefile.vdso b/linux-user/loongarch64/Makefile.vdso
index 1d760b1e47e..ceb61101209 100644
--- a/linux-user/loongarch64/Makefile.vdso
+++ b/linux-user/loongarch64/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/loongarch64-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/loongarch64/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/loongarch64
 VPATH += $(SUBDIR)
diff --git a/linux-user/ppc/Makefile.vdso b/linux-user/ppc/Makefile.vdso
index e2b8facbb50..6b9be843817 100644
--- a/linux-user/ppc/Makefile.vdso
+++ b/linux-user/ppc/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/ppc64-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/ppc64/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/ppc
 VPATH += $(SUBDIR)
diff --git a/linux-user/riscv/Makefile.vdso b/linux-user/riscv/Makefile.vdso
index 2c257dbfda5..2eefdb70cf9 100644
--- a/linux-user/riscv/Makefile.vdso
+++ b/linux-user/riscv/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/riscv64-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/riscv64/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/riscv
 VPATH += $(SUBDIR)
diff --git a/linux-user/s390x/Makefile.vdso b/linux-user/s390x/Makefile.vdso
index e82bf9e29ff..51f7c50a2de 100644
--- a/linux-user/s390x/Makefile.vdso
+++ b/linux-user/s390x/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/s390x-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/s390x/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/s390x
 VPATH += $(SUBDIR)
diff --git a/linux-user/sh4/Makefile.vdso b/linux-user/sh4/Makefile.vdso
index 272c5d262de..ed2cc11b825 100644
--- a/linux-user/sh4/Makefile.vdso
+++ b/linux-user/sh4/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/sh4-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/sh4/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/sh4
 VPATH += $(SUBDIR)
diff --git a/linux-user/x86_64/Makefile.vdso b/linux-user/x86_64/Makefile.vdso
index 26552b66dba..56fe87af72b 100644
--- a/linux-user/x86_64/Makefile.vdso
+++ b/linux-user/x86_64/Makefile.vdso
@@ -1,4 +1,4 @@
-include $(BUILD_DIR)/tests/tcg/x86_64-linux-user/config-target.mak
+include $(BUILD_DIR)/linux-user/x86_64/config-target.mak
 
 SUBDIR = $(SRC_PATH)/linux-user/x86_64
 VPATH += $(SUBDIR)
-- 
2.47.3

Re: [PATCH v4 05/19] configure: re-purpose the tcg test compiler configs for vdso building

[Pierrick Bouvier]

On 9/16/2026 8:29 AM, Alex Bennée wrote:
> All the tcg tests are in meson now so there is no reason to put the
> config-target.mak files in those directories. We also don't need the
> GDB feature tests or to emit prerequsistes.
> 
> The vdso re-build blob does use the compiler definitions though so
> we can emit the details for that and tweak the vdso makefiles to point
> at the right place. Note some of the vdso makefiles build both 32 and
> 64 bit versions from the same compiler.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  configure                            | 69 +++++++++++-----------------
>  linux-user/aarch64/Makefile.vdso     |  2 +-
>  linux-user/arm/Makefile.vdso         |  2 +-
>  linux-user/hppa/Makefile.vdso        |  2 +-
>  linux-user/i386/Makefile.vdso        |  2 +-
>  linux-user/loongarch64/Makefile.vdso |  2 +-
>  linux-user/ppc/Makefile.vdso         |  2 +-
>  linux-user/riscv/Makefile.vdso       |  2 +-
>  linux-user/s390x/Makefile.vdso       |  2 +-
>  linux-user/sh4/Makefile.vdso         |  2 +-
>  linux-user/x86_64/Makefile.vdso      |  2 +-
>  11 files changed, 36 insertions(+), 53 deletions(-)
> 

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

[PATCH v4 06/19] tests/tcg/s390x: Provide the libc bits needed by the system tests

[Alex Bennée]

From: Ilya Leoshkevich <iii@linux.ibm.com>

The meson conversion dropped the SLOF libc include path that commit
3fa010d5317 ("tests/tcg/s390x: Use the SLOF libc headers for the
multiarch tests") had added, breaking the build on systems without a
cross-libc, such as Fedora.

Rather than reintroducing a dependency on the SLOF submodule, which no
other tcg test has, declare the three functions that the reused pc-bios
code actually needs: memcpy(), memset(), and printf().

Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Reported-by: Alex Bennée <alex.bennee@linaro.org>
Closes: https://patchew.org/QEMU/20260910100120.2076092-1-berrange@redhat.com/
Suggested-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Fixes: cd81cf86834 ("tests/tcg/s390x: system tests")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Message-ID: <20260915110959.492195-1-iii@linux.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/s390x/libc/stdio.h       | 11 +++++++++++
 tests/tcg/s390x/libc/string.h      | 12 ++++++++++++
 tests/tcg/s390x/system/meson.build |  2 ++
 3 files changed, 25 insertions(+)
 create mode 100644 tests/tcg/s390x/libc/stdio.h
 create mode 100644 tests/tcg/s390x/libc/string.h

diff --git a/tests/tcg/s390x/libc/stdio.h b/tests/tcg/s390x/libc/stdio.h
new file mode 100644
index 00000000000..c3e45995f65
--- /dev/null
+++ b/tests/tcg/s390x/libc/stdio.h
@@ -0,0 +1,11 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#ifndef QEMU_TESTS_S390X_STDIO_H
+#define QEMU_TESTS_S390X_STDIO_H
+
+#include <minilib.h>
+
+#define printf ml_printf
+
+#endif
diff --git a/tests/tcg/s390x/libc/string.h b/tests/tcg/s390x/libc/string.h
new file mode 100644
index 00000000000..1cabeb90adf
--- /dev/null
+++ b/tests/tcg/s390x/libc/string.h
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#ifndef QEMU_TESTS_S390X_STRING_H
+#define QEMU_TESTS_S390X_STRING_H
+
+#include <stddef.h>
+
+void *memcpy(void *dest, const void *src, size_t n);
+void *memset(void *dest, int c, size_t size);
+
+#endif
diff --git a/tests/tcg/s390x/system/meson.build b/tests/tcg/s390x/system/meson.build
index ca07f503058..1e360e70821 100644
--- a/tests/tcg/s390x/system/meson.build
+++ b/tests/tcg/s390x/system/meson.build
@@ -6,11 +6,13 @@ link_script = files('../softmmu.ld')[0]
 head64 = files('../head64.S')[0]
 console = files('../console.c')[0]
 ipl_inc = meson.project_source_root()/'include'/'hw'/'s390x'/'ipl'
+libc_inc = meson.current_source_dir()/'..'/'libc'
 cflags = ['-nostdlib',
           '-ffreestanding',
           '-Wa,--noexecstack',
           '-I', minilib_dir, minilib_printf,
           '-I', ipl_inc,
+          '-I', libc_inc,
           '-march=z13',
           head64, console]
 asmflags = ['-nostdlib',
-- 
2.47.3

[PATCH v4 07/19] tests/tcg: prefix meson built docker tags with qemu/

[Alex Bennée]

With the meson changes we now build the images directly by calling
docker.py. However the Makefile.include builds prefix all the images
with qemu/:

  # General rule for building docker images.
  docker-image-%: $(DOCKER_FILES_DIR)/%.docker
            $(call quiet-command,			\
                  DOCKER_BUILDKIT=1 $(CONTAINER_COMMAND) build \
                  $(if $(DOCKER_V),,--quiet)		\
                  $(if $(NOCACHE),--no-cache,		\
                          $(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \
                  --build-arg BUILDKIT_INLINE_CACHE=1 	\
                  $(if $(NOUSER),,			\
                          --build-arg USER=$(USER)	\
                          --build-arg UID=$(UID))		\
                  -t qemu/$* - < $< $(if $(DOCKER_V),,> /dev/null),\
                  "BUILD", $*)

So to ensure we don't end up building every container twice make sure
the naming conventions line up.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/meson.build | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 890da7b81e9..d2a837f103d 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -237,7 +237,7 @@ foreach target, plan: tcg_tests
     image_name = 'image-' + cc_dockerfile
     has_docker = docker_host_arch_supported and docker_supported
     if cc_dockerfile not in image_targets and has_docker
-      cmd = [docker_wrapper, 'build', '-f', dockerfile, '-t', cc_dockerfile]
+      cmd = [docker_wrapper, 'build', '-f', dockerfile, '-t', 'qemu/' + cc_dockerfile]
       t = custom_target(image_name, command: cmd,
                         build_by_default: false,
                         output: 'no_output_' + image_name)
@@ -252,7 +252,7 @@ foreach target, plan: tcg_tests
       here = meson.project_build_root()
       cc = [docker_wrapper, 'run', '--run-as-current-user',
             '-w', here, '-v', mount,
-            cc_dockerfile, plan['cc']]
+            'qemu/' + cc_dockerfile, plan['cc']]
       has_cc = true
     endif
   endif
@@ -261,7 +261,7 @@ foreach target, plan: tcg_tests
   if has_cc
     summary_cc = plan['cc']
     if not cc_from_system
-      summary_cc += ' (from \'' + plan['cc_dockerfile'] + '\' container)'
+      summary_cc += ' (from \'qemu/' + plan['cc_dockerfile'] + '\' container)'
     endif
   endif
   tcg_tests_summary += {'cc for ' + target: summary_cc}
-- 
2.47.3

Re: [PATCH v4 07/19] tests/tcg: prefix meson built docker tags with qemu/

[Richard Henderson]

On 9/16/26 05:30, Alex Bennée wrote:
> With the meson changes we now build the images directly by calling
> docker.py. However the Makefile.include builds prefix all the images
> with qemu/:
> 
>    # General rule for building docker images.
>    docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>              $(call quiet-command,			\
>                    DOCKER_BUILDKIT=1 $(CONTAINER_COMMAND) build \
>                    $(if $(DOCKER_V),,--quiet)		\
>                    $(if $(NOCACHE),--no-cache,		\
>                            $(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \
>                    --build-arg BUILDKIT_INLINE_CACHE=1 	\
>                    $(if $(NOUSER),,			\
>                            --build-arg USER=$(USER)	\
>                            --build-arg UID=$(UID))		\
>                    -t qemu/$* - < $< $(if $(DOCKER_V),,> /dev/null),\
>                    "BUILD", $*)
> 
> So to ensure we don't end up building every container twice make sure
> the naming conventions line up.
> 
> Reviewed-by: Pierrick Bouvier<pierrick.bouvier@oss.qualcomm.com>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> ---
>   tests/tcg/meson.build | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

[PATCH v4 08/19] tests/tcg: use --add-current-user when building containers

[Alex Bennée]

As the meson tcg build routes the container builds through docker.py
we can also clean up its hack which involved appending a RUN line to
the dockerfile. All the dockerfiles that care will work with the
--build-arg.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 13 ++++++-------
 tests/tcg/meson.build  |  3 ++-
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 315fa2ff848..6090a9dc957 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -297,13 +297,6 @@ def build_image(self, tag, docker_dir, dockerfile,
                                              dir=docker_dir, suffix=".docker")
         tmp_df.write(dockerfile)
 
-        if user:
-            uid = os.getuid()
-            uname = getpass.getuser()
-            tmp_df.write("\n")
-            tmp_df.write("RUN id %s 2>/dev/null || useradd -u %d -U %s" %
-                         (uname, uid, uname))
-
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s\n" % (checksum))
         for f, c in extra_files_cksum:
@@ -315,6 +308,12 @@ def build_image(self, tag, docker_dir, dockerfile,
         if self._buildkit:
             build_args += ["--build-arg", "BUILDKIT_INLINE_CACHE=1"]
 
+        if user:
+            uid = os.getuid()
+            uname = getpass.getuser()
+            build_args += ["--build-arg", "USER=%s" % uname,
+                           "--build-arg", "UID=%s" % uid]
+
         if registry is not None:
             pull_args = ["pull", "%s/%s" % (registry, tag)]
             self._do(pull_args, quiet=quiet)
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index d2a837f103d..d317f0ee310 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -237,7 +237,8 @@ foreach target, plan: tcg_tests
     image_name = 'image-' + cc_dockerfile
     has_docker = docker_host_arch_supported and docker_supported
     if cc_dockerfile not in image_targets and has_docker
-      cmd = [docker_wrapper, 'build', '-f', dockerfile, '-t', 'qemu/' + cc_dockerfile]
+      cmd = [docker_wrapper, 'build', '-f', dockerfile, '-t', 'qemu/' + cc_dockerfile,
+             '--add-current-user']
       t = custom_target(image_name, command: cmd,
                         build_by_default: false,
                         output: 'no_output_' + image_name)
-- 
2.47.3

Re: [PATCH v4 08/19] tests/tcg: use --add-current-user when building containers

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> As the meson tcg build routes the container builds through docker.py
> we can also clean up its hack which involved appending a RUN line to
> the dockerfile. All the dockerfiles that care will work with the
> --build-arg.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/docker.py | 13 ++++++-------
>  tests/tcg/meson.build  |  3 ++-
>  2 files changed, 8 insertions(+), 8 deletions(-)
> 

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

[PATCH v4 09/19] tests/tcg: don't define tests when missing compilers

[Alex Bennée]

Skip defining the tests for an architecture if we can't build them.
Otherwise we add a bunch of noise to the reported results. We still do
report when we skip because the compiler doesn't have the features we
need to build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/meson.build | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index d317f0ee310..904a5a02242 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -264,8 +264,11 @@ foreach target, plan: tcg_tests
     if not cc_from_system
       summary_cc += ' (from \'qemu/' + plan['cc_dockerfile'] + '\' container)'
     endif
+    tcg_tests_summary += {'cc for ' + target: summary_cc}
+  else
+    # no compiler, skip this target
+    continue
   endif
-  tcg_tests_summary += {'cc for ' + target: summary_cc}
 
   folder = plan['folder']
   gdb_arch = plan['gdb_arch']
@@ -373,6 +376,9 @@ foreach target, plan: tcg_tests
           cc_has_feat += {cc_feat: avail}
         endif
 
+        # While we skip defining tests for whole architectures if we
+        # don't have the compiler we do skip individual tests where
+        # the compiler is missing a feature.
         if not cc_has_feat[cc_feat]
           skip_test = 'compiler feature ' + cc_feat + ' not supported'
           built_tests += {exe_name: exe_name}
@@ -380,10 +386,8 @@ foreach target, plan: tcg_tests
         cflags += cc_feat_cflags[cc_feat]
       endif
 
-      if not has_cc
-        skip_test = 'compiler ' + plan['cc'] + ' not available'
-        built_tests += {exe_name: exe_name}
-      endif
+      # We should either have a compiler or have skipped the whole target
+      assert(has_cc)
 
       exe_depends = build_test_depends
       if 'depends' in setup
-- 
2.47.3

Re: [PATCH v4 09/19] tests/tcg: don't define tests when missing compilers

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> Skip defining the tests for an architecture if we can't build them.
> Otherwise we add a bunch of noise to the reported results. We still do
> report when we skip because the compiler doesn't have the features we
> need to build.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/tcg/meson.build | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
> 
> diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
> index d317f0ee310..904a5a02242 100644
> --- a/tests/tcg/meson.build
> +++ b/tests/tcg/meson.build
> @@ -264,8 +264,11 @@ foreach target, plan: tcg_tests
>      if not cc_from_system
>        summary_cc += ' (from \'qemu/' + plan['cc_dockerfile'] + '\' container)'
>      endif
> +    tcg_tests_summary += {'cc for ' + target: summary_cc}
> +  else
> +    # no compiler, skip this target
> +    continue

This was purposefully implemented to not use a continue.
This ensures we always check all tests definitions are correct, whether
or not we build them. In other words, it validates meson.build
systematically. With this patch, only tests definitions for which a
compiler is present, which is incorrect.

The alternative is to not declare the final test (test() call at the
end), in case cc is not present. Reuse skip_test variable, and check if
cc is present at this step instead.

This way, we keep benefit of checking meson.build, and the benefit of
this patch.

>    endif
> -  tcg_tests_summary += {'cc for ' + target: summary_cc}
>  
>    folder = plan['folder']
>    gdb_arch = plan['gdb_arch']
> @@ -373,6 +376,9 @@ foreach target, plan: tcg_tests
>            cc_has_feat += {cc_feat: avail}
>          endif
>  
> +        # While we skip defining tests for whole architectures if we
> +        # don't have the compiler we do skip individual tests where
> +        # the compiler is missing a feature.
>          if not cc_has_feat[cc_feat]
>            skip_test = 'compiler feature ' + cc_feat + ' not supported'
>            built_tests += {exe_name: exe_name}
> @@ -380,10 +386,8 @@ foreach target, plan: tcg_tests
>          cflags += cc_feat_cflags[cc_feat]
>        endif
>  
> -      if not has_cc
> -        skip_test = 'compiler ' + plan['cc'] + ' not available'
> -        built_tests += {exe_name: exe_name}
> -      endif
> +      # We should either have a compiler or have skipped the whole target
> +      assert(has_cc)
>  
>        exe_depends = build_test_depends
>        if 'depends' in setup

Re: [PATCH v4 09/19] tests/tcg: don't define tests when missing compilers

[Richard Henderson]

On 9/16/26 06:41, Pierrick Bouvier wrote:
> The alternative is to not declare the final test (test() call at the
> end), in case cc is not present. Reuse skip_test variable, and check if
> cc is present at this step instead.
> 
> This way, we keep benefit of checking meson.build, and the benefit of
> this patch.

Good idea.


r~

Re: [PATCH v4 09/19] tests/tcg: don't define tests when missing compilers

[Pierrick Bouvier]

On 9/16/2026 5:00 PM, Richard Henderson wrote:
> On 9/16/26 06:41, Pierrick Bouvier wrote:
>> The alternative is to not declare the final test (test() call at the
>> end), in case cc is not present. Reuse skip_test variable, and check if
>> cc is present at this step instead.
>>
>> This way, we keep benefit of checking meson.build, and the benefit of
>> this patch.
> 
> Good idea.
>

Implemented with:

--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -508,10 +508,14 @@ foreach target, plan: tcg_tests
         cmd = ['sh', '-c', 'echo "' + skip_test + '" && exit 77']
       endif

-      test(test_name, env, args: cmd,
-           depends: test_depends,
-           timeout: 120,
-           suite: ['tcg', 'tcg-' + target])
+      # Do not add a skipped test when compiler is missing,
+      # though we still check all tests are correctly defined.
+      if has_cc
+        test(test_name, env, args: cmd,
+             depends: test_depends,
+             timeout: 120,
+             suite: ['tcg', 'tcg-' + target])
+      endif
       added_tests += {test_name: true}
     endforeach
   endforeach

It can replace current patch.

> 
> r~

[PATCH v4 10/19] tests/docker: add container registry to configure and use

[Alex Bennée]

Previously we had the default baked into Makefile.include but as the
meson container logic will also benefit from it we update the configure
machinery.

For now we allow direct make invocations to override but for the
normal case it is in one place now.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - fixed conflicts from the revert.
v3
  - regen meson-buildoptions.sh
v4
  - merge conflicts
---
 configure                     | 8 ++++++++
 scripts/meson-buildoptions.sh | 3 +++
 tests/docker/Makefile.include | 4 +---
 tests/tcg/meson.build         | 6 ++++++
 4 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 67967cd294c..b0e1e694a6c 100755
--- a/configure
+++ b/configure
@@ -173,6 +173,7 @@ fi
 
 # default parameters
 container_command=""
+container_registry="registry.gitlab.com/qemu-project/qemu"
 cpu=""
 cross_compile="no"
 cross_prefix=""
@@ -736,6 +737,9 @@ for opt do
   --container-command=*) container_command="$optarg"
                          meson_option_add -Dcontainer_command="$optarg"
   ;;
+  --container-registry=*) container_registry="$optarg"
+                          meson_option_add -Dcontainer_registry="$optarg"
+  ;;
   --rust-target-triple=*) rust_target_triple="$optarg"
   ;;
   --gdb=*) meson_option_add -Dgdb="$optarg"
@@ -870,6 +874,7 @@ Advanced options (experts only):
   --cpu=CPU                Build for host CPU [$cpu]
   --disable-containers     don't use containers for cross-building
   --container-command=CMD  which container command to use [autodetect]
+  --container-registry=URL which container registry to cache from [$container_registry]
   --gdb=GDB-path           gdb to use for gdbstub tests [autodetect]
   --wasm64-32bit-address-limit Restrict wasm64 address space to 32-bit (default
                                is to use the whole 64-bit range).
@@ -1732,6 +1737,9 @@ echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
 echo "MESON=$meson" >> $config_host_mak
 echo "NINJA=$ninja" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
+if test -n "$container_registry"; then
+  echo "DOCKER_DEFAULT_REGISTRY=$container_registry" >> $config_host_mak
+fi
 if test "$default_targets" = "yes"; then
   echo "CONFIG_DEFAULT_TARGETS=y" >> $config_host_mak
 fi
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index ba3c9de216b..0cb1d48c982 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -10,6 +10,9 @@ meson_options_help() {
   printf "%s\n" '  --block-drv-rw-whitelist=VALUE'
   printf "%s\n" '                           set block driver read-write whitelist (by default'
   printf "%s\n" '                           affects only QEMU, not tools like qemu-img)'
+  printf "%s\n" '  --container-registry=VALUE'
+  printf "%s\n" '                           container registry to cache from'
+  printf "%s\n" '                           [registry.gitlab.com/qemu-project/qemu]'
   printf "%s\n" '  --datadir=VALUE          Data file directory [share]'
   printf "%s\n" '  --disable-coroutine-pool coroutine freelist (better performance)'
   printf "%s\n" '  --disable-debug-info     Enable debug symbols and other information'
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 0adddb6a5c6..74bf471f171 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -11,9 +11,7 @@ USER = $(if $(NOUSER),,$(shell id -un))
 UID = $(if $(NOUSER),,$(shell id -u))
 
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
-ifeq ($(HOST_ARCH),x86_64)
-DOCKER_DEFAULT_REGISTRY := registry.gitlab.com/qemu-project/qemu
-endif
+DOCKER_DEFAULT_REGISTRY ?= registry.gitlab.com/qemu-project/qemu
 DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTRY))
 
 CONTAINER_COMMAND ?= $(shell $(SRC_PATH)/tests/docker/docker.py probe)
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 904a5a02242..87e5614fc83 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -104,6 +104,9 @@ tcg_tests_summary += {'container command': container_command}
 if get_option('container_registry') != ''
   tcg_tests_summary += {'container registry': get_option('container_registry')}
 endif
+if get_option('container_registry') != ''
+  tcg_tests_summary += {'container registry': get_option('container_registry')}
+endif
 
 # plugins come first, as we need to build the list
 test_plugins = {}
@@ -239,6 +242,9 @@ foreach target, plan: tcg_tests
     if cc_dockerfile not in image_targets and has_docker
       cmd = [docker_wrapper, 'build', '-f', dockerfile, '-t', 'qemu/' + cc_dockerfile,
              '--add-current-user']
+      if get_option('container_registry') != ''
+        cmd += ['--registry', get_option('container_registry')]
+      endif
       t = custom_target(image_name, command: cmd,
                         build_by_default: false,
                         output: 'no_output_' + image_name)
-- 
2.47.3

Re: [PATCH v4 10/19] tests/docker: add container registry to configure and use

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> Previously we had the default baked into Makefile.include but as the
> meson container logic will also benefit from it we update the configure
> machinery.
> 
> For now we allow direct make invocations to override but for the
> normal case it is in one place now.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v2
>   - fixed conflicts from the revert.
> v3
>   - regen meson-buildoptions.sh
> v4
>   - merge conflicts
> ---
>  configure                     | 8 ++++++++
>  scripts/meson-buildoptions.sh | 3 +++
>  tests/docker/Makefile.include | 4 +---
>  tests/tcg/meson.build         | 6 ++++++
>  4 files changed, 18 insertions(+), 3 deletions(-)
> 

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

[PATCH v4 11/19] tests/docker: remove used _read_qemu_dockerfile

[Alex Bennée]

This is no longer needed since the functionality using it was removed.

Fixes: 07056db1b50 (tests/docker: remove FROM qemu/ support from docker.py)
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 10 ----------
 1 file changed, 10 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 6090a9dc957..fd0cbdfe856 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -175,16 +175,6 @@ def _check_binfmt_misc(executable):
     return interp, True
 
 
-def _read_qemu_dockerfile(img_name):
-    # special case for Debian linux-user images
-    if img_name.startswith("debian") and img_name.endswith("user"):
-        img_name = "debian-bootstrap"
-
-    df = os.path.join(os.path.dirname(__file__), "dockerfiles",
-                      img_name + ".docker")
-    return _read_dockerfile(df)
-
-
 def _dockerfile_verify_flat(df):
     "Verify we do not include other qemu/ layers"
     for l in df.splitlines():
-- 
2.47.3

[PATCH v4 12/19] tests/docker: strip remaining caching logic from docker.py

[Alex Bennée]

The caching logic was a remnant of an older time when we had multiple
shared layers building that went into the final image. It was arguable
that it worked back then but it is definitely not needed now so into
the bin it goes.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 148 ++++++++++++++---------------------------
 1 file changed, 49 insertions(+), 99 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index fd0cbdfe856..a2e97061240 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -15,7 +15,6 @@
 import sys
 import subprocess
 import json
-import hashlib
 import atexit
 import uuid
 import argparse
@@ -27,7 +26,6 @@
 from tarfile import TarFile, TarInfo
 from io import StringIO, BytesIO
 from shutil import copy, rmtree
-from datetime import datetime, timedelta
 
 
 FILTERED_ENV_NAMES = ['ftp_proxy', 'http_proxy', 'https_proxy']
@@ -35,20 +33,9 @@
 
 DEVNULL = open(os.devnull, 'wb')
 
-def _bytes_checksum(bytes):
-    """Calculate a digest string unique to the text content"""
-    return hashlib.sha1(bytes).hexdigest()
-
-def _text_checksum(text):
-    """Calculate a digest string unique to the text content"""
-    return _bytes_checksum(text.encode('utf-8'))
-
 def _read_dockerfile(path):
     return open(path, 'rt', encoding='utf-8').read()
 
-def _file_checksum(filename):
-    return _bytes_checksum(open(filename, 'rb').read())
-
 
 def _guess_engine_command():
     """ Guess a working engine command or raise exception if not found"""
@@ -263,38 +250,15 @@ def inspect_tag(self, tag):
         except subprocess.CalledProcessError:
             return None
 
-    def get_image_creation_time(self, info):
-        return json.loads(info)[0]["Created"]
-
-    def get_image_dockerfile_checksum(self, tag):
-        resp = self.inspect_tag(tag)
-        labels = json.loads(resp)[0]["Config"].get("Labels", {})
-        return labels.get("com.qemu.dockerfile-checksum", "")
-
-    def build_image(self, tag, docker_dir, dockerfile,
-                    quiet=True, user=False, argv=None, registry=None,
-                    extra_files_cksum=[]):
+    def build_image(self, tag, docker_dir, dockerfile_path,
+                    quiet=True, user=False, argv=None, registry=None):
         if argv is None:
             argv = []
 
-        if not _dockerfile_verify_flat(dockerfile):
+        if not _dockerfile_verify_flat(_read_dockerfile(dockerfile_path)):
             return -1
 
-        checksum = _text_checksum(dockerfile)
-
-        tmp_df = tempfile.NamedTemporaryFile(mode="w+t",
-                                             encoding='utf-8',
-                                             dir=docker_dir, suffix=".docker")
-        tmp_df.write(dockerfile)
-
-        tmp_df.write("\n")
-        tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s\n" % (checksum))
-        for f, c in extra_files_cksum:
-            tmp_df.write("LABEL com.qemu.%s-checksum=%s\n" % (f, c))
-
-        tmp_df.flush()
-
-        build_args = ["build", "-t", tag, "-f", tmp_df.name]
+        build_args = ["build", "-t", tag, "-f", dockerfile_path]
         if self._buildkit:
             build_args += ["--build-arg", "BUILDKIT_INLINE_CACHE=1"]
 
@@ -320,13 +284,6 @@ def update_image(self, tag, tarball, quiet=True):
 
         self._do_check(["build", "-t", tag, "-"], quiet=quiet, stdin=tarball)
 
-    def image_matches_dockerfile(self, tag, dockerfile):
-        try:
-            checksum = self.get_image_dockerfile_checksum(tag)
-        except Exception:
-            return False
-        return checksum == _text_checksum(dockerfile)
-
     def run(self, cmd, keep, quiet, as_user=False):
         label = uuid.uuid4().hex
         if not keep:
@@ -412,61 +369,54 @@ def args(self, parser):
                             help="Dockerfile name")
 
     def run(self, args, argv):
-        dockerfile = _read_dockerfile(args.dockerfile)
         tag = args.tag
 
         dkr = Docker(args.command)
-        if "--no-cache" not in argv and \
-           dkr.image_matches_dockerfile(tag, dockerfile):
-            pass
-        else:
-            # Create a docker context directory for the build
-            docker_dir = tempfile.mkdtemp(prefix="docker_build")
-
-            # Validate binfmt_misc will work
-            if args.skip_binfmt:
-                qpath = args.include_executable
-            elif args.include_executable:
-                qpath, enabled = _check_binfmt_misc(args.include_executable)
-                if not enabled:
-                    return 1
-
-            # Is there a .pre file to run in the build context?
-            docker_pre = os.path.splitext(args.dockerfile)[0]+".pre"
-            if os.path.exists(docker_pre):
-                stdout = DEVNULL if args.quiet else None
-                rc = subprocess.call(os.path.realpath(docker_pre),
-                                     cwd=docker_dir, stdout=stdout)
-                if rc == 3:
-                    print("Skip")
-                    return 0
-                elif rc != 0:
-                    print("%s exited with code %d" % (docker_pre, rc))
-                    return 1
-
-            # Copy any extra files into the Docker context. These can be
-            # included by the use of the ADD directive in the Dockerfile.
-            cksum = []
-            if args.include_executable:
-                # FIXME: there is no checksum of this executable and the linked
-                # libraries, once the image built any change of this executable
-                # or any library won't trigger another build.
-                _copy_binary_with_libs(args.include_executable,
-                                       qpath, docker_dir)
-
-            for filename in args.extra_files or []:
-                _copy_with_mkdir(filename, docker_dir)
-                cksum += [(filename, _file_checksum(filename))]
-
-            argv += ["--build-arg=" + k.lower() + "=" + v
-                     for k, v in os.environ.items()
-                     if k.lower() in FILTERED_ENV_NAMES]
-            dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, user=args.user,
-                            argv=argv, registry=args.registry,
-                            extra_files_cksum=cksum)
-
-            rmtree(docker_dir)
+
+        # Create a docker context directory for the build
+        docker_dir = tempfile.mkdtemp(prefix="docker_build")
+
+        # Validate binfmt_misc will work
+        if args.skip_binfmt:
+            qpath = args.include_executable
+        elif args.include_executable:
+            qpath, enabled = _check_binfmt_misc(args.include_executable)
+            if not enabled:
+                return 1
+
+        # Is there a .pre file to run in the build context?
+        docker_pre = os.path.splitext(args.dockerfile)[0]+".pre"
+        if os.path.exists(docker_pre):
+            stdout = DEVNULL if args.quiet else None
+            rc = subprocess.call(os.path.realpath(docker_pre),
+                                 cwd=docker_dir, stdout=stdout)
+            if rc == 3:
+                print("Skip")
+                return 0
+            elif rc != 0:
+                print("%s exited with code %d" % (docker_pre, rc))
+                return 1
+
+        # Copy any extra files into the Docker context. These can be
+        # included by the use of the ADD directive in the Dockerfile.
+        if args.include_executable:
+            # FIXME: there is no checksum of this executable and the linked
+            # libraries, once the image built any change of this executable
+            # or any library won't trigger another build.
+            _copy_binary_with_libs(args.include_executable,
+                                   qpath, docker_dir)
+
+        for filename in args.extra_files or []:
+            _copy_with_mkdir(filename, docker_dir)
+
+        argv += ["--build-arg=" + k.lower() + "=" + v
+                 for k, v in os.environ.items()
+                 if k.lower() in FILTERED_ENV_NAMES]
+        dkr.build_image(tag, docker_dir, args.dockerfile,
+                        quiet=args.quiet, user=args.user,
+                        argv=argv, registry=args.registry)
+
+        rmtree(docker_dir)
 
         return 0
 
-- 
2.47.3

[PATCH v4 13/19] tests/docker: add specific command to parser

[Alex Bennée]

A future change will want to use additional flags which clash with
docker/podman command lines. Currently they would get snarfed up by
argparser. Make the pass-through commands explicit to avoid this.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 scripts/coverity-scan/run-coverity-scan | 2 +-
 tests/docker/docker.py                  | 5 ++++-
 tests/tcg/meson.build                   | 2 +-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/scripts/coverity-scan/run-coverity-scan b/scripts/coverity-scan/run-coverity-scan
index 2f9ccd5e44b..0e1361606e0 100755
--- a/scripts/coverity-scan/run-coverity-scan
+++ b/scripts/coverity-scan/run-coverity-scan
@@ -381,7 +381,7 @@ if [ "$DOCKER" = yes ]; then
     # Arrange for this docker run to get access to the sources with -v.
     # We pass through all the configuration from the outer script to the inner.
     export COVERITY_EMAIL COVERITY_BUILD_CMD
-    tests/docker/docker.py run -it --env COVERITY_EMAIL --env COVERITY_BUILD_CMD \
+    tests/docker/docker.py run -- -it --env COVERITY_EMAIL --env COVERITY_BUILD_CMD \
            -v "$SECRETDIR:/work" coverity-scanner \
            ./run-coverity-scan --version "$VERSION" \
            --description "$DESCRIPTION" $ARGS --tokenfile /work/token \
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index a2e97061240..d7eb4cf6811 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -336,9 +336,12 @@ def args(self, parser):
                             help="Don't remove image when command completes")
         parser.add_argument("--run-as-current-user", action="store_true",
                             help="Run container using the current user's uid")
+        parser.add_argument('cmd', nargs='*',
+                            help="""The command to run. You should precede with
+                            -- to avoid confusion about its flags""")
 
     def run(self, args, argv):
-        return Docker(args.command).run(argv, args.keep, quiet=args.quiet,
+        return Docker(args.command).run(args.cmd, args.keep, quiet=args.quiet,
                                         as_user=args.run_as_current_user)
 
 
diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 87e5614fc83..80205f6f98a 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -257,7 +257,7 @@ foreach target, plan: tcg_tests
       mount = meson.project_source_root()
       mount = mount + ':' + mount
       here = meson.project_build_root()
-      cc = [docker_wrapper, 'run', '--run-as-current-user',
+      cc = [docker_wrapper, 'run', '--run-as-current-user', '--',
             '-w', here, '-v', mount,
             'qemu/' + cc_dockerfile, plan['cc']]
       has_cc = true
-- 
2.47.3

[PATCH v4 14/19] tests/docker: swap --quiet for --verbose

[Alex Bennée]

Running the docker build in meson spams the output so default to quiet
and make --verbose the options. If we want to expose verbosity in
meson we can deal with that later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v3
  - dropped accidental libvirt-ci inclusion
---
 tests/docker/Makefile.include | 10 +++++-----
 tests/docker/docker.py        | 22 ++++++++++++----------
 2 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 74bf471f171..85825d74dfb 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -72,13 +72,13 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 			DEB_TYPE=$(DEB_TYPE) 					\
 			$(if $(DEB_URL),DEB_URL=$(DEB_URL),)			\
 			$(DOCKER_SCRIPT) build -t qemu/debian-$* -f $< 		\
-			$(if $(DOCKER_V),,--quiet) $(if $(NOCACHE),--no-cache) 	\
+			$(if $(DOCKER_V),--verbose) $(if $(NOCACHE),--no-cache) \
 			$(if $(NOUSER),,--add-current-user) 			\
 			$(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))	\
 			$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)), \
 			"BUILD","binfmt debian-$* (debootstrapped)"),		\
 		$(call quiet-command,						\
-			$(DOCKER_SCRIPT) check --quiet qemu/debian-$* $< || 	\
+			$(DOCKER_SCRIPT) check qemu/debian-$* $< || 		\
 			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
 			"CHECK", "debian-$* exists"))
 
@@ -103,17 +103,17 @@ debian-toolchain-run = \
 	$(if $(NOCACHE)$(NOFETCH),					\
 		$(call quiet-command,					\
 			$(DOCKER_SCRIPT) build -t qemu/$1 -f $< 	\
-			$(if $(DOCKER_V),,--quiet) 			\
+			$(if $(DOCKER_V),--verbose) 			\
 			$(if $(NOCACHE),--no-cache)			\
 			--registry $(DOCKER_REGISTRY) --extra-files	\
 			$(DOCKER_FILES_DIR)/$1.d/build-toolchain.sh,	\
 			"BUILD", $1),				        \
 		$(call quiet-command,					\
-			$(DOCKER_SCRIPT) fetch $(if $(DOCKER_V),,--quiet) \
+			$(DOCKER_SCRIPT) fetch $(if $(DOCKER_V),--verbose) \
 				qemu/$1 $(DOCKER_REGISTRY),		\
 			"FETCH", $1)					\
 		$(call quiet-command,					\
-			$(DOCKER_SCRIPT) update $(if $(DOCKER_V),,--quiet) \
+			$(DOCKER_SCRIPT) update $(if $(DOCKER_V),--verbose) \
 				qemu/$1 				\
 				$(if $(NOUSER),,--add-current-user) 	\
 			"PREPARE", $1))
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index d7eb4cf6811..10cb4cbde11 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -312,8 +312,8 @@ class SubCommand(object):
     name = None  # Subcommand name
 
     def shared_args(self, parser):
-        parser.add_argument("--quiet", action="store_true",
-                            help="Run quietly unless an error occurred")
+        parser.add_argument("--verbose", "-v", action="store_true",
+                            help="Run verbosely (default is quiet)")
 
     def args(self, parser):
         """Setup argument parser"""
@@ -341,7 +341,8 @@ def args(self, parser):
                             -- to avoid confusion about its flags""")
 
     def run(self, args, argv):
-        return Docker(args.command).run(args.cmd, args.keep, quiet=args.quiet,
+        return Docker(args.command).run(args.cmd, args.keep,
+                                        quiet=not args.verbose,
                                         as_user=args.run_as_current_user)
 
 
@@ -390,7 +391,7 @@ def run(self, args, argv):
         # Is there a .pre file to run in the build context?
         docker_pre = os.path.splitext(args.dockerfile)[0]+".pre"
         if os.path.exists(docker_pre):
-            stdout = DEVNULL if args.quiet else None
+            stdout = None if args.verbose else DEVNULL
             rc = subprocess.call(os.path.realpath(docker_pre),
                                  cwd=docker_dir, stdout=stdout)
             if rc == 3:
@@ -416,7 +417,7 @@ def run(self, args, argv):
                  for k, v in os.environ.items()
                  if k.lower() in FILTERED_ENV_NAMES]
         dkr.build_image(tag, docker_dir, args.dockerfile,
-                        quiet=args.quiet, user=args.user,
+                        quiet=not args.verbose, user=args.user,
                         argv=argv, registry=args.registry)
 
         rmtree(docker_dir)
@@ -435,9 +436,9 @@ def args(self, parser):
 
     def run(self, args, argv):
         dkr = Docker(args.command)
-        dkr.command(cmd="pull", quiet=args.quiet,
+        dkr.command(cmd="pull", quiet=not args.verbose,
                     argv=["%s/%s" % (args.registry, args.tag)])
-        dkr.command(cmd="tag", quiet=args.quiet,
+        dkr.command(cmd="tag", quiet=not args.verbose,
                     argv=["%s/%s" % (args.registry, args.tag), args.tag])
 
 
@@ -513,7 +514,7 @@ def run(self, args, argv):
 
         # Run the build with our tarball context
         dkr = Docker(args.command)
-        dkr.update_image(args.tag, tmp, quiet=args.quiet)
+        dkr.update_image(args.tag, tmp, quiet=not args.verbose)
 
         return 0
 
@@ -532,7 +533,8 @@ class ImagesCommand(SubCommand):
     name = "images"
 
     def run(self, args, argv):
-        return Docker(args.command).command("images", argv, args.quiet)
+        return Docker(args.command).command("images", argv,
+                                            quiet=not args.verbose)
 
 
 class ProbeCommand(SubCommand):
@@ -574,7 +576,7 @@ def run(self, args, argv):
                 cmd += ["-v", "%s:%s:ro,z" % (p, p)]
         cmd += [args.image, args.cc]
         cmd += argv
-        return Docker(args.command).run(cmd, False, quiet=args.quiet,
+        return Docker(args.command).run(cmd, False, quiet=not args.verbose,
                                         as_user=True)
 
 
-- 
2.47.3

Re: [PATCH v4 14/19] tests/docker: swap --quiet for --verbose

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> Running the docker build in meson spams the output so default to quiet
> and make --verbose the options. If we want to expose verbosity in
> meson we can deal with that later.
>

It "spams" output only when build is really done. If image is already
built, then it's silent. I don't think it's worth silencing this
completely by default. Maybe we can postpone this patch to when it will
be correctly integrated with meson, there is no emergency or fix here.

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v3
>   - dropped accidental libvirt-ci inclusion
> ---
>  tests/docker/Makefile.include | 10 +++++-----
>  tests/docker/docker.py        | 22 ++++++++++++----------
>  2 files changed, 17 insertions(+), 15 deletions(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 74bf471f171..85825d74dfb 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -72,13 +72,13 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
>  			DEB_TYPE=$(DEB_TYPE) 					\
>  			$(if $(DEB_URL),DEB_URL=$(DEB_URL),)			\
>  			$(DOCKER_SCRIPT) build -t qemu/debian-$* -f $< 		\
> -			$(if $(DOCKER_V),,--quiet) $(if $(NOCACHE),--no-cache) 	\
> +			$(if $(DOCKER_V),--verbose) $(if $(NOCACHE),--no-cache) \
>  			$(if $(NOUSER),,--add-current-user) 			\
>  			$(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES))	\
>  			$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)), \
>  			"BUILD","binfmt debian-$* (debootstrapped)"),		\
>  		$(call quiet-command,						\
> -			$(DOCKER_SCRIPT) check --quiet qemu/debian-$* $< || 	\
> +			$(DOCKER_SCRIPT) check qemu/debian-$* $< || 		\
>  			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
>  			"CHECK", "debian-$* exists"))
>  
> @@ -103,17 +103,17 @@ debian-toolchain-run = \
>  	$(if $(NOCACHE)$(NOFETCH),					\
>  		$(call quiet-command,					\
>  			$(DOCKER_SCRIPT) build -t qemu/$1 -f $< 	\
> -			$(if $(DOCKER_V),,--quiet) 			\
> +			$(if $(DOCKER_V),--verbose) 			\
>  			$(if $(NOCACHE),--no-cache)			\
>  			--registry $(DOCKER_REGISTRY) --extra-files	\
>  			$(DOCKER_FILES_DIR)/$1.d/build-toolchain.sh,	\
>  			"BUILD", $1),				        \
>  		$(call quiet-command,					\
> -			$(DOCKER_SCRIPT) fetch $(if $(DOCKER_V),,--quiet) \
> +			$(DOCKER_SCRIPT) fetch $(if $(DOCKER_V),--verbose) \
>  				qemu/$1 $(DOCKER_REGISTRY),		\
>  			"FETCH", $1)					\
>  		$(call quiet-command,					\
> -			$(DOCKER_SCRIPT) update $(if $(DOCKER_V),,--quiet) \
> +			$(DOCKER_SCRIPT) update $(if $(DOCKER_V),--verbose) \
>  				qemu/$1 				\
>  				$(if $(NOUSER),,--add-current-user) 	\
>  			"PREPARE", $1))
> diff --git a/tests/docker/docker.py b/tests/docker/docker.py
> index d7eb4cf6811..10cb4cbde11 100755
> --- a/tests/docker/docker.py
> +++ b/tests/docker/docker.py
> @@ -312,8 +312,8 @@ class SubCommand(object):
>      name = None  # Subcommand name
>  
>      def shared_args(self, parser):
> -        parser.add_argument("--quiet", action="store_true",
> -                            help="Run quietly unless an error occurred")
> +        parser.add_argument("--verbose", "-v", action="store_true",
> +                            help="Run verbosely (default is quiet)")
>  
>      def args(self, parser):
>          """Setup argument parser"""
> @@ -341,7 +341,8 @@ def args(self, parser):
>                              -- to avoid confusion about its flags""")
>  
>      def run(self, args, argv):
> -        return Docker(args.command).run(args.cmd, args.keep, quiet=args.quiet,
> +        return Docker(args.command).run(args.cmd, args.keep,
> +                                        quiet=not args.verbose,
>                                          as_user=args.run_as_current_user)
>  
>  
> @@ -390,7 +391,7 @@ def run(self, args, argv):
>          # Is there a .pre file to run in the build context?
>          docker_pre = os.path.splitext(args.dockerfile)[0]+".pre"
>          if os.path.exists(docker_pre):
> -            stdout = DEVNULL if args.quiet else None
> +            stdout = None if args.verbose else DEVNULL
>              rc = subprocess.call(os.path.realpath(docker_pre),
>                                   cwd=docker_dir, stdout=stdout)
>              if rc == 3:
> @@ -416,7 +417,7 @@ def run(self, args, argv):
>                   for k, v in os.environ.items()
>                   if k.lower() in FILTERED_ENV_NAMES]
>          dkr.build_image(tag, docker_dir, args.dockerfile,
> -                        quiet=args.quiet, user=args.user,
> +                        quiet=not args.verbose, user=args.user,
>                          argv=argv, registry=args.registry)
>  
>          rmtree(docker_dir)
> @@ -435,9 +436,9 @@ def args(self, parser):
>  
>      def run(self, args, argv):
>          dkr = Docker(args.command)
> -        dkr.command(cmd="pull", quiet=args.quiet,
> +        dkr.command(cmd="pull", quiet=not args.verbose,
>                      argv=["%s/%s" % (args.registry, args.tag)])
> -        dkr.command(cmd="tag", quiet=args.quiet,
> +        dkr.command(cmd="tag", quiet=not args.verbose,
>                      argv=["%s/%s" % (args.registry, args.tag), args.tag])
>  
>  
> @@ -513,7 +514,7 @@ def run(self, args, argv):
>  
>          # Run the build with our tarball context
>          dkr = Docker(args.command)
> -        dkr.update_image(args.tag, tmp, quiet=args.quiet)
> +        dkr.update_image(args.tag, tmp, quiet=not args.verbose)
>  
>          return 0
>  
> @@ -532,7 +533,8 @@ class ImagesCommand(SubCommand):
>      name = "images"
>  
>      def run(self, args, argv):
> -        return Docker(args.command).command("images", argv, args.quiet)
> +        return Docker(args.command).command("images", argv,
> +                                            quiet=not args.verbose)
>  
>  
>  class ProbeCommand(SubCommand):
> @@ -574,7 +576,7 @@ def run(self, args, argv):
>                  cmd += ["-v", "%s:%s:ro,z" % (p, p)]
>          cmd += [args.image, args.cc]
>          cmd += argv
> -        return Docker(args.command).run(cmd, False, quiet=args.quiet,
> +        return Docker(args.command).run(cmd, False, quiet=not args.verbose,
>                                          as_user=True)
>  
>  

[PATCH v4 15/19] tests/docker: fix brown bag typo for docker-ccache

[Alex Bennée]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 85825d74dfb..fa873e09b4f 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -197,7 +197,7 @@ docker-help: docker
 # Where QEMU caches build artefacts
 DOCKER_QEMU_CACHE_DIR := $$HOME/.cache/qemu
 # Use a global constant ccache directory to speed up repetitive builds
-DOCKER_QEMU_CCACHE_DIR := DOCKER_QEMU_CACHE_DIR/docker-ccache
+DOCKER_QEMU_CCACHE_DIR := $$DOCKER_QEMU_CACHE_DIR/docker-ccache
 
 # This rule if for directly running against an arbitrary docker target.
 # It is called by the expanded docker targets (e.g. make
-- 
2.47.3

Re: [PATCH v4 15/19] tests/docker: fix brown bag typo for docker-ccache

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

[PATCH v4 16/19] gitlab: update the @user id mapping file

[Alex Bennée]

I needed to fix a POSIX-ism in the script as well.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20260914184534.706637-1-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-map-auto          | 8 +++++---
 scripts/gitlab-map-update | 2 +-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/.gitlab-map-auto b/.gitlab-map-auto
index 28ce448f1bf..3f717a15d4e 100644
--- a/.gitlab-map-auto
+++ b/.gitlab-map-auto
@@ -12,11 +12,12 @@ a1xndr	Alexander Bulekov
 adi-g15-ibm	Aditya Gupta
 agraf	Alexander Graf
 alex.williamson	Alex Williamson
-aliang1	Aihua Liang
 alistair23	Alistair Francis
+amachhiw	Amit Machhiwal
 anisinha	Ani Sinha
 anthony-linaro	Anthony Roberts
 anthonyper	Anthony PERARD
+aroxell	Anders Roxell
 berrange	Daniel P. Berrangé
 birkelund	Klaus Jensen
 bonzini	Paolo Bonzini
@@ -36,10 +37,10 @@ dwmw2	David Woodhouse
 eauger1	Eric Auger
 ebblake	Eric Blake
 edgar.iglesias	Edgar E. Iglesias
+efarman	Eric Farman
 ehabkost	Eduardo Habkost
 eldondev	Eldon
 epilys	Manos Pitsidianakis
-famzheng	Fam Zheng
 farosas	Fabiano Rosas
 gautammenghani	Gautam Menghani
 gkurz	Greg Kurz
@@ -51,7 +52,6 @@ imammedo	Igor Mammedov
 jasowang	Jason Wang
 jmacarthur	Jim MacArthur
 jsnow	John Snow
-juan.quintela	Juan Quintela
 kbastian-qemu	Bastian Koppelmann
 kmwolf	Kevin Wolf
 kostyanf14	Kostiantyn Kostiuk
@@ -67,6 +67,7 @@ mauromatteo.cascella	Mauro Matteo Cascella
 mcayland	Mark Cave-Ayland
 mdroth	Michael Roth
 mediouni-m	M. Mediouni
+mjrosato	Matthew Rosato
 mjt0k	Michael Tokarev
 mstredhat	MST
 npiggin	npiggin
@@ -79,6 +80,7 @@ pipo.sk	Peter Krempa
 pkrempa	Peter Krempa (work)
 pm215	Peter Maydell
 qemu-janitor	Qemu Janitor
+raphael.s.norwitz	Raphael Norwitz
 rathc	Chinmay Rath
 rth7680	Richard Henderson
 schoenebeck	Christian Schoenebeck
diff --git a/scripts/gitlab-map-update b/scripts/gitlab-map-update
index c134c226aa2..743ad11900d 100755
--- a/scripts/gitlab-map-update
+++ b/scripts/gitlab-map-update
@@ -33,7 +33,7 @@ glab api --paginate /projects/qemu-project%2fqemu/members | \
     .gitlab-map-auto.tmp
 test $? != 0 && fail=1
 
-if test $fail == 0
+if test $fail -eq 0
 then
   grep '^#' .gitlab-map-auto > .gitlab-map-auto.new
   LC_ALL=C sort .gitlab-map-auto.tmp | uniq >> .gitlab-map-auto.new
-- 
2.47.3

Re: [PATCH v4 16/19] gitlab: update the @user id mapping file

[Amit Machhiwal]

On 2026/09/16 04:30 PM, Alex Bennée wrote:
> I needed to fix a POSIX-ism in the script as well.
> 
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-ID: <20260914184534.706637-1-alex.bennee@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .gitlab-map-auto          | 8 +++++---
>  scripts/gitlab-map-update | 2 +-
>  2 files changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/.gitlab-map-auto b/.gitlab-map-auto
> index 28ce448f1bf..3f717a15d4e 100644
> --- a/.gitlab-map-auto
> +++ b/.gitlab-map-auto
> @@ -12,11 +12,12 @@ a1xndr	Alexander Bulekov
>  adi-g15-ibm	Aditya Gupta
>  agraf	Alexander Graf
>  alex.williamson	Alex Williamson
> -aliang1	Aihua Liang
>  alistair23	Alistair Francis
> +amachhiw	Amit Machhiwal

Thanks for adding my ID, Alex!

Acked-by: Amit Machhiwal <amachhiw@linux.ibm.com>

Thanks,
Amit

[PATCH v4 17/19] scripts/make2mtest: add docker dependencies for check-tcg-% targets

[Alex Bennée]

This should trigger the building of the docker image when the user
calls "make check-tcg" or one of the arch specific targets such as
"make check-tcg-m68k-linux-user".

We still keep the internal meson dependency so individual tests can
launched and the docker build verified.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 scripts/mtest2make.py | 35 +++++++++++++++++++++++++++++------
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 383ea68b16d..ab894d90d47 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -14,6 +14,7 @@ class Suite(object):
     def __init__(self):
         self.deps = set()
         self.speeds = set()
+        self.docker_deps = set()
 
     def names(self, base):
         return [f'{base}-{speed}' for speed in self.speeds]
@@ -40,7 +41,22 @@ def names(self, base):
 
 introspect = json.load(sys.stdin)
 
-def process_tests(test, targets, suites):
+targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
+           for t in introspect['targets']}
+
+# collect the docker images uses by the tests. This is totally a hacky
+# heuristic that fishes the images out of the test custom commands by
+# looking for the qemu/image pattern.
+targets_docker = {}
+for t in introspect['targets']:
+    for src in t.get('target_sources', []):
+        for arg in src.get('compiler', []):
+            if arg.startswith('qemu/'):
+                img = arg.split('/', 1)[1]
+                targets_docker[t['id']] = f'docker-image-{img}'
+                break
+
+def process_tests(test, targets, targets_docker, suites):
     executable = test['cmd'][0]
     try:
         executable = os.path.relpath(executable)
@@ -51,6 +67,8 @@ def process_tests(test, targets, suites):
     deps = itertools.chain.from_iterable(deps)
     deps = list(deps)
 
+    docker_deps = [targets_docker[x] for x in test['depends'] if x in targets_docker]
+
     test_suites = test['suite'] or ['default']
     for s in test_suites:
         # The suite name in the introspection info is "PROJECT" or "PROJECT:SUITE"
@@ -59,6 +77,7 @@ def process_tests(test, targets, suites):
             if s == 'slow' or s == 'thorough':
                 continue
         suites[s].deps.update(deps)
+        suites[s].docker_deps.update(docker_deps)
         if s.endswith('-slow'):
             s = s[:-5]
             suites[s].speeds.add('slow')
@@ -81,6 +100,9 @@ def emit_prolog(suites, prefix):
     print(f'all-{prefix}-xml = {all_xml}')
     print(f'.PHONY: {prefix} do-meson-{prefix} {prefix}-report.junit.xml $(all-{prefix}-targets) $(all-{prefix}-xml)')
     print(f'ninja-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.deps))')
+    print(f'docker-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.docker-deps))')
+    print(f'docker-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.docker-goals.$g))')
+    print(f'run-ninja: $(sort $(docker-cmd-goals))')
     print(f'{prefix}-build: run-ninja')
     print(f'{prefix} $(all-{prefix}-targets): do-meson-{prefix}')
     print(f'do-meson-{prefix}: run-ninja; $(if $(MAKE.n),,+)$(MESON) test $(.{prefix}.mtestargs)')
@@ -93,6 +115,10 @@ def emit_suite(name, suite, prefix):
     print()
     print(f'.{prefix}-{tgtname}.deps = {deps}')
     print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
+    if suite.docker_deps:
+        docker_deps = ' '.join(sorted(suite.docker_deps))
+        print(f'.{prefix}-{tgtname}.docker-deps = {docker_deps}')
+        print(f'.docker-goals.check-build += $(.{prefix}-{tgtname}.docker-deps)')
 
     names = ' '.join(sorted(suite.names(name)))
     targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
@@ -105,19 +131,16 @@ def emit_suite(name, suite, prefix):
     print(f".{prefix}.mtest-suites += {name} $(call .speed.$(SPEED), {names})")
     print(f'endif')
 
-targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
-           for t in introspect['targets']}
-
 testsuites = defaultdict(Suite)
 for test in introspect['tests']:
-    process_tests(test, targets, testsuites)
+    process_tests(test, targets, targets_docker, testsuites)
 emit_prolog(testsuites, 'check')
 for name, suite in testsuites.items():
     emit_suite(name, suite, 'check')
 
 benchsuites = defaultdict(Suite)
 for test in introspect['benchmarks']:
-    process_tests(test, targets, benchsuites)
+    process_tests(test, targets, targets_docker, benchsuites)
 emit_prolog(benchsuites, 'bench')
 for name, suite in benchsuites.items():
     emit_suite(name, suite, 'bench')
-- 
2.47.3

Re: [PATCH v4 17/19] scripts/make2mtest: add docker dependencies for check-tcg-% targets

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> This should trigger the building of the docker image when the user
> calls "make check-tcg" or one of the arch specific targets such as
> "make check-tcg-m68k-linux-user".
> 
> We still keep the internal meson dependency so individual tests can
> launched and the docker build verified.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  scripts/mtest2make.py | 35 +++++++++++++++++++++++++++++------
>  1 file changed, 29 insertions(+), 6 deletions(-)
> 
> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
> index 383ea68b16d..ab894d90d47 100644
> --- a/scripts/mtest2make.py
> +++ b/scripts/mtest2make.py
> @@ -14,6 +14,7 @@ class Suite(object):
>      def __init__(self):
>          self.deps = set()
>          self.speeds = set()
> +        self.docker_deps = set()
>  
>      def names(self, base):
>          return [f'{base}-{speed}' for speed in self.speeds]
> @@ -40,7 +41,22 @@ def names(self, base):
>  
>  introspect = json.load(sys.stdin)
>  
> -def process_tests(test, targets, suites):
> +targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
> +           for t in introspect['targets']}
> +
> +# collect the docker images uses by the tests. This is totally a hacky
> +# heuristic that fishes the images out of the test custom commands by
> +# looking for the qemu/image pattern.
> +targets_docker = {}
> +for t in introspect['targets']:
> +    for src in t.get('target_sources', []):
> +        for arg in src.get('compiler', []):
> +            if arg.startswith('qemu/'):
> +                img = arg.split('/', 1)[1]
> +                targets_docker[t['id']] = f'docker-image-{img}'
> +                break
> +
> +def process_tests(test, targets, targets_docker, suites):
>      executable = test['cmd'][0]
>      try:
>          executable = os.path.relpath(executable)
> @@ -51,6 +67,8 @@ def process_tests(test, targets, suites):
>      deps = itertools.chain.from_iterable(deps)
>      deps = list(deps)
>  
> +    docker_deps = [targets_docker[x] for x in test['depends'] if x in targets_docker]
> +
>      test_suites = test['suite'] or ['default']
>      for s in test_suites:
>          # The suite name in the introspection info is "PROJECT" or "PROJECT:SUITE"
> @@ -59,6 +77,7 @@ def process_tests(test, targets, suites):
>              if s == 'slow' or s == 'thorough':
>                  continue
>          suites[s].deps.update(deps)
> +        suites[s].docker_deps.update(docker_deps)
>          if s.endswith('-slow'):
>              s = s[:-5]
>              suites[s].speeds.add('slow')
> @@ -81,6 +100,9 @@ def emit_prolog(suites, prefix):
>      print(f'all-{prefix}-xml = {all_xml}')
>      print(f'.PHONY: {prefix} do-meson-{prefix} {prefix}-report.junit.xml $(all-{prefix}-targets) $(all-{prefix}-xml)')
>      print(f'ninja-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.deps))')
> +    print(f'docker-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.docker-deps))')
> +    print(f'docker-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.docker-goals.$g))')
> +    print(f'run-ninja: $(sort $(docker-cmd-goals))')
>      print(f'{prefix}-build: run-ninja')
>      print(f'{prefix} $(all-{prefix}-targets): do-meson-{prefix}')
>      print(f'do-meson-{prefix}: run-ninja; $(if $(MAKE.n),,+)$(MESON) test $(.{prefix}.mtestargs)')
> @@ -93,6 +115,10 @@ def emit_suite(name, suite, prefix):
>      print()
>      print(f'.{prefix}-{tgtname}.deps = {deps}')
>      print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
> +    if suite.docker_deps:
> +        docker_deps = ' '.join(sorted(suite.docker_deps))
> +        print(f'.{prefix}-{tgtname}.docker-deps = {docker_deps}')
> +        print(f'.docker-goals.check-build += $(.{prefix}-{tgtname}.docker-deps)')
>  
>      names = ' '.join(sorted(suite.names(name)))
>      targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
> @@ -105,19 +131,16 @@ def emit_suite(name, suite, prefix):
>      print(f".{prefix}.mtest-suites += {name} $(call .speed.$(SPEED), {names})")
>      print(f'endif')
>  
> -targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
> -           for t in introspect['targets']}
> -
>  testsuites = defaultdict(Suite)
>  for test in introspect['tests']:
> -    process_tests(test, targets, testsuites)
> +    process_tests(test, targets, targets_docker, testsuites)
>  emit_prolog(testsuites, 'check')
>  for name, suite in testsuites.items():
>      emit_suite(name, suite, 'check')
>  
>  benchsuites = defaultdict(Suite)
>  for test in introspect['benchmarks']:
> -    process_tests(test, targets, benchsuites)
> +    process_tests(test, targets, targets_docker, benchsuites)
>  emit_prolog(benchsuites, 'bench')
>  for name, suite in benchsuites.items():
>      emit_suite(name, suite, 'bench')

From previous answer on v3, this patch does duplicate existing
dependencies. If we don't have a specific reason to repeat it on
Makefile side, it should not be included in this series.

Regards,
Pierrick

Re: [PATCH v4 17/19] scripts/make2mtest: add docker dependencies for check-tcg-% targets

[Alex Bennée]

Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:

> On 9/16/2026 8:30 AM, Alex Bennée wrote:
>> This should trigger the building of the docker image when the user
>> calls "make check-tcg" or one of the arch specific targets such as
>> "make check-tcg-m68k-linux-user".
>> 
>> We still keep the internal meson dependency so individual tests can
>> launched and the docker build verified.
>> 
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  scripts/mtest2make.py | 35 +++++++++++++++++++++++++++++------
>>  1 file changed, 29 insertions(+), 6 deletions(-)
>> 
>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
>> index 383ea68b16d..ab894d90d47 100644
>> --- a/scripts/mtest2make.py
>> +++ b/scripts/mtest2make.py
>> @@ -14,6 +14,7 @@ class Suite(object):
>>      def __init__(self):
>>          self.deps = set()
>>          self.speeds = set()
>> +        self.docker_deps = set()
>>  
>>      def names(self, base):
>>          return [f'{base}-{speed}' for speed in self.speeds]
>> @@ -40,7 +41,22 @@ def names(self, base):
>>  
>>  introspect = json.load(sys.stdin)
>>  
>> -def process_tests(test, targets, suites):
>> +targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
>> +           for t in introspect['targets']}
>> +
>> +# collect the docker images uses by the tests. This is totally a hacky
>> +# heuristic that fishes the images out of the test custom commands by
>> +# looking for the qemu/image pattern.
>> +targets_docker = {}
>> +for t in introspect['targets']:
>> +    for src in t.get('target_sources', []):
>> +        for arg in src.get('compiler', []):
>> +            if arg.startswith('qemu/'):
>> +                img = arg.split('/', 1)[1]
>> +                targets_docker[t['id']] = f'docker-image-{img}'
>> +                break
>> +
>> +def process_tests(test, targets, targets_docker, suites):
>>      executable = test['cmd'][0]
>>      try:
>>          executable = os.path.relpath(executable)
>> @@ -51,6 +67,8 @@ def process_tests(test, targets, suites):
>>      deps = itertools.chain.from_iterable(deps)
>>      deps = list(deps)
>>  
>> +    docker_deps = [targets_docker[x] for x in test['depends'] if x in targets_docker]
>> +
>>      test_suites = test['suite'] or ['default']
>>      for s in test_suites:
>>          # The suite name in the introspection info is "PROJECT" or "PROJECT:SUITE"
>> @@ -59,6 +77,7 @@ def process_tests(test, targets, suites):
>>              if s == 'slow' or s == 'thorough':
>>                  continue
>>          suites[s].deps.update(deps)
>> +        suites[s].docker_deps.update(docker_deps)
>>          if s.endswith('-slow'):
>>              s = s[:-5]
>>              suites[s].speeds.add('slow')
>> @@ -81,6 +100,9 @@ def emit_prolog(suites, prefix):
>>      print(f'all-{prefix}-xml = {all_xml}')
>>      print(f'.PHONY: {prefix} do-meson-{prefix} {prefix}-report.junit.xml $(all-{prefix}-targets) $(all-{prefix}-xml)')
>>      print(f'ninja-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.deps))')
>> +    print(f'docker-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.docker-deps))')
>> +    print(f'docker-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.docker-goals.$g))')
>> +    print(f'run-ninja: $(sort $(docker-cmd-goals))')
>>      print(f'{prefix}-build: run-ninja')
>>      print(f'{prefix} $(all-{prefix}-targets): do-meson-{prefix}')
>>      print(f'do-meson-{prefix}: run-ninja; $(if $(MAKE.n),,+)$(MESON) test $(.{prefix}.mtestargs)')
>> @@ -93,6 +115,10 @@ def emit_suite(name, suite, prefix):
>>      print()
>>      print(f'.{prefix}-{tgtname}.deps = {deps}')
>>      print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
>> +    if suite.docker_deps:
>> +        docker_deps = ' '.join(sorted(suite.docker_deps))
>> +        print(f'.{prefix}-{tgtname}.docker-deps = {docker_deps}')
>> +        print(f'.docker-goals.check-build += $(.{prefix}-{tgtname}.docker-deps)')
>>  
>>      names = ' '.join(sorted(suite.names(name)))
>>      targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
>> @@ -105,19 +131,16 @@ def emit_suite(name, suite, prefix):
>>      print(f".{prefix}.mtest-suites += {name} $(call .speed.$(SPEED), {names})")
>>      print(f'endif')
>>  
>> -targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
>> -           for t in introspect['targets']}
>> -
>>  testsuites = defaultdict(Suite)
>>  for test in introspect['tests']:
>> -    process_tests(test, targets, testsuites)
>> +    process_tests(test, targets, targets_docker, testsuites)
>>  emit_prolog(testsuites, 'check')
>>  for name, suite in testsuites.items():
>>      emit_suite(name, suite, 'check')
>>  
>>  benchsuites = defaultdict(Suite)
>>  for test in introspect['benchmarks']:
>> -    process_tests(test, targets, benchsuites)
>> +    process_tests(test, targets, targets_docker, benchsuites)
>>  emit_prolog(benchsuites, 'bench')
>>  for name, suite in benchsuites.items():
>>      emit_suite(name, suite, 'bench')
>
> From previous answer on v3, this patch does duplicate existing
> dependencies. If we don't have a specific reason to repeat it on
> Makefile side, it should not be included in this series.

The reason was to trigger the build outside meson where it doesn't stall
the output. Maybe we can drop not we have the console fix.

>
> Regards,
> Pierrick

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v4 17/19] scripts/make2mtest: add docker dependencies for check-tcg-% targets

[Pierrick Bouvier]

On 9/17/2026 2:29 AM, Alex Bennée wrote:
> Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com> writes:
> 
>> On 9/16/2026 8:30 AM, Alex Bennée wrote:
>>> This should trigger the building of the docker image when the user
>>> calls "make check-tcg" or one of the arch specific targets such as
>>> "make check-tcg-m68k-linux-user".
>>>
>>> We still keep the internal meson dependency so individual tests can
>>> launched and the docker build verified.
>>>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> ---
>>>  scripts/mtest2make.py | 35 +++++++++++++++++++++++++++++------
>>>  1 file changed, 29 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
>>> index 383ea68b16d..ab894d90d47 100644
>>> --- a/scripts/mtest2make.py
>>> +++ b/scripts/mtest2make.py
>>> @@ -14,6 +14,7 @@ class Suite(object):
>>>      def __init__(self):
>>>          self.deps = set()
>>>          self.speeds = set()
>>> +        self.docker_deps = set()
>>>  
>>>      def names(self, base):
>>>          return [f'{base}-{speed}' for speed in self.speeds]
>>> @@ -40,7 +41,22 @@ def names(self, base):
>>>  
>>>  introspect = json.load(sys.stdin)
>>>  
>>> -def process_tests(test, targets, suites):
>>> +targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
>>> +           for t in introspect['targets']}
>>> +
>>> +# collect the docker images uses by the tests. This is totally a hacky
>>> +# heuristic that fishes the images out of the test custom commands by
>>> +# looking for the qemu/image pattern.
>>> +targets_docker = {}
>>> +for t in introspect['targets']:
>>> +    for src in t.get('target_sources', []):
>>> +        for arg in src.get('compiler', []):
>>> +            if arg.startswith('qemu/'):
>>> +                img = arg.split('/', 1)[1]
>>> +                targets_docker[t['id']] = f'docker-image-{img}'
>>> +                break
>>> +
>>> +def process_tests(test, targets, targets_docker, suites):
>>>      executable = test['cmd'][0]
>>>      try:
>>>          executable = os.path.relpath(executable)
>>> @@ -51,6 +67,8 @@ def process_tests(test, targets, suites):
>>>      deps = itertools.chain.from_iterable(deps)
>>>      deps = list(deps)
>>>  
>>> +    docker_deps = [targets_docker[x] for x in test['depends'] if x in targets_docker]
>>> +
>>>      test_suites = test['suite'] or ['default']
>>>      for s in test_suites:
>>>          # The suite name in the introspection info is "PROJECT" or "PROJECT:SUITE"
>>> @@ -59,6 +77,7 @@ def process_tests(test, targets, suites):
>>>              if s == 'slow' or s == 'thorough':
>>>                  continue
>>>          suites[s].deps.update(deps)
>>> +        suites[s].docker_deps.update(docker_deps)
>>>          if s.endswith('-slow'):
>>>              s = s[:-5]
>>>              suites[s].speeds.add('slow')
>>> @@ -81,6 +100,9 @@ def emit_prolog(suites, prefix):
>>>      print(f'all-{prefix}-xml = {all_xml}')
>>>      print(f'.PHONY: {prefix} do-meson-{prefix} {prefix}-report.junit.xml $(all-{prefix}-targets) $(all-{prefix}-xml)')
>>>      print(f'ninja-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.deps))')
>>> +    print(f'docker-cmd-goals += $(foreach s, $(.{prefix}.mtest-suites), $(.{prefix}-$s.docker-deps))')
>>> +    print(f'docker-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.docker-goals.$g))')
>>> +    print(f'run-ninja: $(sort $(docker-cmd-goals))')
>>>      print(f'{prefix}-build: run-ninja')
>>>      print(f'{prefix} $(all-{prefix}-targets): do-meson-{prefix}')
>>>      print(f'do-meson-{prefix}: run-ninja; $(if $(MAKE.n),,+)$(MESON) test $(.{prefix}.mtestargs)')
>>> @@ -93,6 +115,10 @@ def emit_suite(name, suite, prefix):
>>>      print()
>>>      print(f'.{prefix}-{tgtname}.deps = {deps}')
>>>      print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
>>> +    if suite.docker_deps:
>>> +        docker_deps = ' '.join(sorted(suite.docker_deps))
>>> +        print(f'.{prefix}-{tgtname}.docker-deps = {docker_deps}')
>>> +        print(f'.docker-goals.check-build += $(.{prefix}-{tgtname}.docker-deps)')
>>>  
>>>      names = ' '.join(sorted(suite.names(name)))
>>>      targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
>>> @@ -105,19 +131,16 @@ def emit_suite(name, suite, prefix):
>>>      print(f".{prefix}.mtest-suites += {name} $(call .speed.$(SPEED), {names})")
>>>      print(f'endif')
>>>  
>>> -targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
>>> -           for t in introspect['targets']}
>>> -
>>>  testsuites = defaultdict(Suite)
>>>  for test in introspect['tests']:
>>> -    process_tests(test, targets, testsuites)
>>> +    process_tests(test, targets, targets_docker, testsuites)
>>>  emit_prolog(testsuites, 'check')
>>>  for name, suite in testsuites.items():
>>>      emit_suite(name, suite, 'check')
>>>  
>>>  benchsuites = defaultdict(Suite)
>>>  for test in introspect['benchmarks']:
>>> -    process_tests(test, targets, benchsuites)
>>> +    process_tests(test, targets, targets_docker, benchsuites)
>>>  emit_prolog(benchsuites, 'bench')
>>>  for name, suite in benchsuites.items():
>>>      emit_suite(name, suite, 'bench')
>>
>> From previous answer on v3, this patch does duplicate existing
>> dependencies. If we don't have a specific reason to repeat it on
>> Makefile side, it should not be included in this series.
> 
> The reason was to trigger the build outside meson where it doesn't stall
> the output. Maybe we can drop not we have the console fix.
>

Yes, it doesn't stall build anymore, and it's adding duplication. That's
safe to remove current patch.

>>
>> Regards,
>> Pierrick
> 

[PATCH v4 18/19] tests/tcg: add console property to docker image custom target

[Alex Bennée]

According to the meson docs this serialises the target in a special
"console" pool which also disables the buffering of stdout. This helps
avoid the appearance of a massive stall when a docker image is built.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/meson.build | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 80205f6f98a..6c675a554cf 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -247,6 +247,7 @@ foreach target, plan: tcg_tests
       endif
       t = custom_target(image_name, command: cmd,
                         build_by_default: false,
+                        console: true,
                         output: 'no_output_' + image_name)
       image_targets += {cc_dockerfile: t}
     endif
-- 
2.47.3

Re: [PATCH v4 18/19] tests/tcg: add console property to docker image custom target

[Pierrick Bouvier]

On 9/16/2026 8:30 AM, Alex Bennée wrote:
> According to the meson docs this serialises the target in a special
> "console" pool which also disables the buffering of stdout. This helps
> avoid the appearance of a massive stall when a docker image is built.
> 
> Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/tcg/meson.build | 1 +
>  1 file changed, 1 insertion(+)
> 

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

[PATCH v4 19/19] tests/tcg: Fix building outside of the source tree

[Alex Bennée]

From: Ilya Leoshkevich <iii@linux.ibm.com>

Running TCG tests from a build directory that is outside of the source
tree fails with:

    [...]/qemu/tests/tcg/multiarch/libs/float_helpers.c:228:1: fatal error: opening dependency file tests/tcg/s390x-linux-user-float_convs.d: No such file or directory

This is because the build directory is not mounted into the container.
Add this mount. Do not bother with preventing overlapping mounts, they
are harmless.

Fixes: 27939ad414ca ("tests/tcg/meson.build: introduce cc_dockerfile and cc_docker_host_arch")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20260915110855.491985-1-iii@linux.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/tcg/meson.build b/tests/tcg/meson.build
index 6c675a554cf..c566820c097 100644
--- a/tests/tcg/meson.build
+++ b/tests/tcg/meson.build
@@ -259,7 +259,7 @@ foreach target, plan: tcg_tests
       mount = mount + ':' + mount
       here = meson.project_build_root()
       cc = [docker_wrapper, 'run', '--run-as-current-user', '--',
-            '-w', here, '-v', mount,
+            '-w', here, '-v', here + ':' + here, '-v', mount,
             'qemu/' + cc_dockerfile, plan['cc']]
       has_cc = true
     endif
-- 
2.47.3

Re: [PATCH v4 00/19] testing: various docker clean-ups post meson tcg conversion

[Richard Henderson]

On 9/16/26 05:29, Alex Bennée wrote:
> This cleans up a few bits and pieces although doesn't totally solve
> the issue of an apparent stall while meson builds the docker image.
> However with these changes there should be no difference between the
> meson build and the equivalent "make docker-image-foo" invocation.
> 
> We also finally get rid of the ill-advised magic caching support in
> docker.py.
> 
> v4
>   - added Illya's fixes for out of tree and s390x tests
>   - tweaked the revert so --disable-containers still passed to meson
>   - don't define tests if no compiler available to avoid excess SKIPs
> 
> v3
> 
>   - Update make2test to trigger docker build from Makefile.include.
>   - add Daniels's suggested console: true patch
>   - fixed up some unref vars from the reverts
> 
> v2
> 
> I've restored the configure based compiler checks for so we can build
> roms and the vdso blobs again. It looks like we could never build roms
> with container based images so I've left it for now. Unlike the vdso
> build we can't trigger a container build through Makefile.prereqs.
> 
> TODO
> ----
> 
> I also need to look at the timestamp stuff that Pierrick has suggested
> to see if that improves the caching from the registry.
> 
> The following still need review:
> 
>    tests/tcg: add console property to docker image custom target
>    scripts/make2mtest: add docker dependencies for check-tcg-% targets
>    tests/docker: fix brown bag typo for docker-ccache
>    tests/docker: swap --quiet for --verbose
>    tests/docker: add container registry to configure and use
>    tests/tcg: don't define tests when missing compilers
>    tests/tcg: use --add-current-user when building containers
>    configure: re-purpose the tcg test compiler configs for vdso building
>    Revert "tests/tcg/meson.build: implement ./configure -cross-cc-* options"
> 
> Please test and review.

While I don't really understand some of the container intricacies, this 
does fix all of my build and test issues, so

Tested-by: Richard Henderson <richard.henderson@linaro.org>

r~