[PULL 0/6] testing updates

[PULL 0/6] testing updates

[Alex Bennée]

The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:

  Merge tag 'hppa-fixes-pull-request' of https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27 +0000)

are available in the Git repository at:

  https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1

for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:

  gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)

----------------------------------------------------------------
testing updates:

  - fix minor shell-ism that can break check-tcg
  - turn off verbose logging on custom runners
  - make configure echo call in CI
  - fix unused variable in linux-test
  - add binary compiler docker image for hexagon
  - disable doc and gui builds for tci and disable-tcg builds

----------------------------------------------------------------
Alex Bennée (3):
      gitlab: turn off verbose logging for make check on custom runners
      configure: repeat ourselves for the benefit of CI
      tests/tcg: fix unused variable in linux-test

Mukilan Thiyagarajan (2):
      configure: Fix check-tcg not executing any tests
      tests/docker: use prebuilt toolchain for debian-hexagon-cross

Thomas Huth (1):
      gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs

 configure                                          |  11 +-
 tests/tcg/multiarch/linux/linux-test.c             |   6 +-
 .gitlab-ci.d/buildtest.yml                         |  10 +-
 .gitlab-ci.d/container-cross.yml                   |  19 +--
 .gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml |  12 +-
 .../custom-runners/ubuntu-22.04-aarch32.yml        |   2 +-
 .../custom-runners/ubuntu-22.04-aarch64.yml        |  12 +-
 MAINTAINERS                                        |   1 -
 tests/docker/Makefile.include                      |   4 -
 .../debian-hexagon-cross.d/build-toolchain.sh      | 141 ---------------------
 .../docker/dockerfiles/debian-hexagon-cross.docker |  53 +++-----
 11 files changed, 47 insertions(+), 224 deletions(-)
 delete mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh


-- 
2.34.1

[PULL 1/6] configure: Fix check-tcg not executing any tests

[Alex Bennée]

From: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>

After configuring with --target-list=hexagon-linux-user
running `make check-tcg` just prints the following:

```
make: Nothing to be done for 'check-tcg'
```

In the probe_target_compiler function, the 'break'
command is used incorrectly. There are no lexically
enclosing loops associated with that break command which
is an unspecfied behaviour in the POSIX standard.

The dash shell implementation aborts the currently executing
loop, in this case, causing the rest of the logic for the loop
in line 2490 to be skipped, which means no Makefiles are
generated for the tcg target tests.

Fixes: c3b570b5a9a24d25 (configure: don't enable
cross compilers unless in target_list)

Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Link: https://patchew.org/QEMU/20221207082309.9966-1-quic._5Fmthiyaga@quicinc.com/
Message-Id: <20221207082309.9966-1-quic_mthiyaga@quicinc.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221221090411.1995037-2-alex.bennee@linaro.org>

diff --git a/configure b/configure
index 26c7bc5154..7a804fb657 100755
--- a/configure
+++ b/configure
@@ -1881,9 +1881,7 @@ probe_target_compiler() {
   # We shall skip configuring the target compiler if the user didn't
   # bother enabling an appropriate guest. This avoids building
   # extraneous firmware images and tests.
-  if test "${target_list#*$1}" != "$1"; then
-      break;
-  else
+  if test "${target_list#*$1}" = "$1"; then
       return 1
   fi
 
-- 
2.34.1

[PULL 2/6] gitlab: turn off verbose logging for make check on custom runners

[Alex Bennée]

The verbosity adds a lot of unnecessary output to the CI logs which
end up getting truncated anyway. We can always extract information
from the meson test logs on a failure and for the custom runners its
generally easier to re-create failures anyway.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221221090411.1995037-3-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
index 0c835939db..fcaef9e5ef 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
@@ -19,9 +19,9 @@ ubuntu-20.04-s390x-all-linux-static:
  - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
- - make --output-sync -j`nproc` check-tcg V=1
+ - make --output-sync -j`nproc` check-tcg
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-20.04-s390x-all:
@@ -40,7 +40,7 @@ ubuntu-20.04-s390x-all:
  - ../configure --disable-libssh
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-20.04-s390x-alldbg:
@@ -63,7 +63,7 @@ ubuntu-20.04-s390x-alldbg:
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make clean
  - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-20.04-s390x-clang:
@@ -85,7 +85,7 @@ ubuntu-20.04-s390x-clang:
  - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-20.04-s390x-tci:
@@ -127,5 +127,5 @@ ubuntu-20.04-s390x-notcg:
  - ../configure --disable-libssh --disable-tcg
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
index 1a2f9b8dbe..2c386fa3e9 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
@@ -21,5 +21,5 @@ ubuntu-22.04-aarch32-all:
  - ../configure --cross-prefix=arm-linux-gnueabihf-
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
index ce0b18af6f..abeb33eaff 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
@@ -19,9 +19,9 @@ ubuntu-22.04-aarch64-all-linux-static:
  - ../configure --enable-debug --static --disable-system --disable-pie
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
- - make --output-sync -j`nproc --ignore=40` check-tcg V=1
+ - make --output-sync -j`nproc --ignore=40` check-tcg
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-22.04-aarch64-all:
@@ -43,7 +43,7 @@ ubuntu-22.04-aarch64-all:
  - ../configure
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-22.04-aarch64-alldbg:
@@ -62,7 +62,7 @@ ubuntu-22.04-aarch64-alldbg:
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make clean
  - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-22.04-aarch64-clang:
@@ -84,7 +84,7 @@ ubuntu-22.04-aarch64-clang:
  - ../configure --disable-libssh --cc=clang-10 --cxx=clang++-10 --enable-sanitizers
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
 
 ubuntu-22.04-aarch64-tci:
@@ -126,5 +126,5 @@ ubuntu-22.04-aarch64-notcg:
  - ../configure --disable-tcg
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
    || { cat meson-logs/testlog.txt; exit 1; } ;
-- 
2.34.1

[PULL 3/6] configure: repeat ourselves for the benefit of CI

[Alex Bennée]

Our CI system echos the lines it executes but not the expansions. For
the sake of a line of extra verbosity during the configure phase lets
echo the invocation of script to stdout as well as the log when on CI.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-4-alex.bennee@linaro.org>

diff --git a/configure b/configure
index 7a804fb657..d89e883844 100755
--- a/configure
+++ b/configure
@@ -83,9 +83,10 @@ rm -f config.log
 # Print a helpful header at the top of config.log
 echo "# QEMU configure log $(date)" >> config.log
 printf "# Configured with:" >> config.log
-printf " '%s'" "$0" "$@" >> config.log
-echo >> config.log
-echo "#" >> config.log
+# repeat the invocation to log and stdout for CI
+invoke=$(printf " '%s'" "$0" "$@")
+test -n "$GITLAB_CI" && echo "configuring with: $invoke"
+{ echo "$invoke"; echo; echo "#"; } >> config.log
 
 quote_sh() {
     printf "%s" "$1" | sed "s,','\\\\'',g; s,.*,'&',"
-- 
2.34.1

[PULL 4/6] tests/tcg: fix unused variable in linux-test

[Alex Bennée]

The latest hexagon compiler picks up that we never consume wcount.
Given the name of the #define that rcount checks against is WCOUNT_MAX
I figured the check just got missed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-5-alex.bennee@linaro.org>

diff --git a/tests/tcg/multiarch/linux/linux-test.c b/tests/tcg/multiarch/linux/linux-test.c
index 5a2a4f2258..64f57cb287 100644
--- a/tests/tcg/multiarch/linux/linux-test.c
+++ b/tests/tcg/multiarch/linux/linux-test.c
@@ -354,13 +354,17 @@ static void test_pipe(void)
             if (FD_ISSET(fds[0], &rfds)) {
                 chk_error(read(fds[0], &ch, 1));
                 rcount++;
-                if (rcount >= WCOUNT_MAX)
+                if (rcount >= WCOUNT_MAX) {
                     break;
+                }
             }
             if (FD_ISSET(fds[1], &wfds)) {
                 ch = 'a';
                 chk_error(write(fds[1], &ch, 1));
                 wcount++;
+                if (wcount >= WCOUNT_MAX) {
+                    break;
+                }
             }
         }
     }
-- 
2.34.1

[PULL 5/6] tests/docker: use prebuilt toolchain for debian-hexagon-cross

[Alex Bennée]

From: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>

The current docker image for cross compiling hexagon guests
is manually built since it takes >2 hours to build from source.

This patch:
 1. Solves the above issue by using the prebuilt clang
    toolchain hosted on CodeLinaro [1] and maintained by QUIC [2].
 2. The dockerfile is also switched from multi-stage to single stage
    build to allow the CI docker engine to reuse the layer cache.
 3. Re-enables the hexagon-cross-container job to be always run in
    CI and makes it a non-optional dependency for the
    build-user-hexagon job.

The changes for 1 & 2 together bring down the build time to
~3 minutes in GitLab CI when cache is reused and ~9 minutes
when cache cannot be reused.

[1]: https://github.com/CodeLinaro/hexagon-builder
[2]: https://github.com/quic/toolchain_for_hexagon/releases/

Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
[AJB: also tweak MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221219144354.11659-1-quic_mthiyaga@quicinc.com>
Message-Id: <20221221090411.1995037-6-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index d21b4a1fd4..93302a96e2 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -275,14 +275,10 @@ build-user-static:
     CONFIGURE_ARGS: --disable-tools --disable-system --static
     MAKE_CHECK_ARGS: check-tcg
 
-# Because the hexagon cross-compiler takes so long to build we don't rely
-# on the CI system to build it and hence this job has an optional dependency
-# declared. The image is manually uploaded.
 build-user-hexagon:
   extends: .native_build_job_template
   needs:
     job: hexagon-cross-container
-    optional: true
   variables:
     IMAGE: debian-hexagon-cross
     TARGETS: hexagon-linux-user
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 2d560e9764..5486dc43c6 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -37,28 +37,11 @@ armhf-debian-cross-container:
 # We never want to build hexagon in the CI system and by default we
 # always want to refer to the master registry where it lives.
 hexagon-cross-container:
-  extends: .base_job_template
-  image: docker:stable
+  extends: .container_job_template
   stage: containers
   variables:
     NAME: debian-hexagon-cross
-    GIT_DEPTH: 1
     QEMU_JOB_ONLY_FORKS: 1
-  services:
-    - docker:dind
-  before_script:
-    - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
-    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
-    - docker info
-    - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
-  script:
-    - echo "TAG:$TAG"
-    - echo "COMMON_TAG:$COMMON_TAG"
-    - docker pull $COMMON_TAG
-    - docker tag $COMMON_TAG $TAG
-    - docker push "$TAG"
-  after_script:
-    - docker logout
 
 hppa-debian-cross-container:
   extends: .container_job_template
diff --git a/MAINTAINERS b/MAINTAINERS
index 716d5a24ad..005a2d3ed2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -204,7 +204,6 @@ F: tests/tcg/hexagon/
 F: disas/hexagon.c
 F: configs/targets/hexagon-linux-user/default.mak
 F: docker/dockerfiles/debian-hexagon-cross.docker
-F: docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
 
 Hexagon idef-parser
 M: Alessandro Di Federico <ale@rev.ng>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index fc7a3b7e71..665ddde518 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -109,10 +109,6 @@ debian-toolchain-run = \
 			"PREPARE", $1))
 debian-toolchain = $(call debian-toolchain-run,$(patsubst docker-image-%,%,$1))
 
-docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.docker \
-	$(DOCKER_FILES_DIR)/debian-hexagon-cross.d/build-toolchain.sh
-	$(call debian-toolchain, $@)
-
 docker-image-debian-microblaze-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
     $(DOCKER_FILES_DIR)/debian-microblaze-cross.d/build-toolchain.sh
 	$(call debian-toolchain, $@)
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh
deleted file mode 100755
index 19b1c9f83e..0000000000
--- a/tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh
+++ /dev/null
@@ -1,141 +0,0 @@
-#!/bin/bash
-
-set -e
-
-BASE=$(readlink -f ${PWD})
-
-TOOLCHAIN_INSTALL=$(readlink -f "$TOOLCHAIN_INSTALL")
-ROOTFS=$(readlink -f "$ROOTFS")
-
-TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
-HEX_SYSROOT=${TOOLCHAIN_INSTALL}/hexagon-unknown-linux-musl
-HEX_TOOLS_TARGET_BASE=${HEX_SYSROOT}/usr
-
-function cdp() {
-  DIR="$1"
-  mkdir -p "$DIR"
-  cd "$DIR"
-}
-
-function fetch() {
-  DIR="$1"
-  URL="$2"
-  TEMP="$(readlink -f "$PWD/tmp.tar.gz")"
-  wget --quiet "$URL" -O "$TEMP"
-  cdp "$DIR"
-  tar xaf "$TEMP" --strip-components=1
-  rm "$TEMP"
-  cd -
-}
-
-build_llvm_clang() {
-  fetch "$BASE/llvm-project" "$LLVM_URL"
-  cdp "$BASE/build-llvm"
-
-  cmake -G Ninja \
-    -DCMAKE_BUILD_TYPE=Release \
-    -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \
-    -DLLVM_ENABLE_LLD=ON \
-    -DLLVM_TARGETS_TO_BUILD="Hexagon" \
-    -DLLVM_ENABLE_PROJECTS="clang;lld" \
-    "$BASE/llvm-project/llvm"
-  ninja all install
-  cd ${TOOLCHAIN_BIN}
-  ln -sf clang hexagon-unknown-linux-musl-clang
-  ln -sf clang++ hexagon-unknown-linux-musl-clang++
-  ln -sf llvm-ar hexagon-unknown-linux-musl-ar
-  ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump
-  ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy
-  ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf
-  ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib
-
-  # workaround for now:
-  cat <<EOF > hexagon-unknown-linux-musl.cfg
--G0 --sysroot=${HEX_SYSROOT}
-EOF
-}
-
-build_clang_rt() {
-  cdp "$BASE/build-clang_rt"
-  cmake -G Ninja \
-    -DCMAKE_BUILD_TYPE=Release \
-    -DLLVM_CONFIG_PATH="$BASE/build-llvm/bin/llvm-config" \
-    -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \
-    -DCMAKE_SYSTEM_NAME=Linux \
-    -DCMAKE_C_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
-    -DCMAKE_ASM_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
-    -DCMAKE_INSTALL_PREFIX=${HEX_TOOLS_TARGET_BASE} \
-    -DCMAKE_CROSSCOMPILING=ON \
-    -DCMAKE_C_COMPILER_FORCED=ON \
-    -DCMAKE_CXX_COMPILER_FORCED=ON \
-    -DCOMPILER_RT_BUILD_BUILTINS=ON \
-    -DCOMPILER_RT_BUILTINS_ENABLE_PIC=OFF \
-    -DCMAKE_SIZEOF_VOID_P=4 \
-    -DCOMPILER_RT_OS_DIR= \
-    -DCAN_TARGET_hexagon=1 \
-    -DCAN_TARGET_x86_64=0 \
-    -DCOMPILER_RT_SUPPORTED_ARCH=hexagon \
-    -DLLVM_ENABLE_PROJECTS="compiler-rt" \
-    "$BASE/llvm-project/compiler-rt"
-  ninja install-compiler-rt
-}
-
-build_musl_headers() {
-  fetch "$BASE/musl" "$MUSL_URL"
-  cd "$BASE/musl"
-  make clean
-  CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
-    CROSS_COMPILE=hexagon-unknown-linux-musl \
-    LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
-    CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
-    ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
-  PATH=${TOOLCHAIN_BIN}:$PATH make CROSS_COMPILE= install-headers
-
-  cd ${HEX_SYSROOT}/..
-  ln -sf hexagon-unknown-linux-musl hexagon
-}
-
-build_kernel_headers() {
-  fetch "$BASE/linux" "$LINUX_URL"
-  mkdir -p "$BASE/build-linux"
-  cd "$BASE/linux"
-  make O=../build-linux ARCH=hexagon \
-   KBUILD_CFLAGS_KERNEL="-mlong-calls" \
-   CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
-   LD=${TOOLCHAIN_BIN}/ld.lld \
-   KBUILD_VERBOSE=1 comet_defconfig
-  make mrproper
-
-  cd "$BASE/build-linux"
-  make \
-    ARCH=hexagon \
-    CC=${TOOLCHAIN_BIN}/clang \
-    INSTALL_HDR_PATH=${HEX_TOOLS_TARGET_BASE} \
-    V=1 \
-    headers_install
-}
-
-build_musl() {
-  cd "$BASE/musl"
-  make clean
-  CROSS_COMPILE=hexagon-unknown-linux-musl- \
-    AR=llvm-ar \
-    RANLIB=llvm-ranlib \
-    STRIP=llvm-strip \
-    CC=clang \
-    LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
-    CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
-    ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
-  PATH=${TOOLCHAIN_BIN}/:$PATH make CROSS_COMPILE= install
-  cd ${HEX_TOOLS_TARGET_BASE}/lib
-  ln -sf libc.so ld-musl-hexagon.so
-  ln -sf ld-musl-hexagon.so ld-musl-hexagon.so.1
-  cdp ${HEX_TOOLS_TARGET_BASE}/../lib
-  ln -sf ../usr/lib/ld-musl-hexagon.so.1
-}
-
-build_llvm_clang
-build_kernel_headers
-build_musl_headers
-build_clang_rt
-build_musl
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker
index c4238e893f..8a0d748343 100644
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -7,44 +7,29 @@
 #
 FROM docker.io/library/debian:11-slim
 
-# Install common build utilities
-RUN apt update && \
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+RUN apt-get update && \
     DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
-    apt install -y --no-install-recommends \
-        bison \
+# Install common build utilities
+    apt-get install -y --no-install-recommends \
+        curl \
+        xz-utils \
         ca-certificates \
-        clang \
-        cmake \
+        bison \
         flex \
-        gcc \
-        lld \
-        make \
-        ninja-build \
-        python3 \
-        rsync \
-        wget \
-        xz-utils
-
-ENV TOOLCHAIN_INSTALL /usr/local
-ENV ROOTFS /usr/local
-
-ENV LLVM_URL https://github.com/llvm/llvm-project/archive/bfcd21876adc3498065e4da92799f613e730d475.tar.gz
-ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz
-ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz
-
-ADD build-toolchain.sh /root/hexagon-toolchain/build-toolchain.sh
-
-RUN cd /root/hexagon-toolchain && ./build-toolchain.sh
-
-FROM docker.io/library/debian:11-slim
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+        git \
+        ninja-build && \
 # Install QEMU build deps for use in CI
-RUN apt update && \
-    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-    DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy bison flex git ninja-build && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
     apt build-dep -yy --arch-only qemu
-COPY --from=0 /usr/local /usr/local
-ENV PATH $PATH:/usr/local/bin/
+
+
+ENV TOOLCHAIN_INSTALL /opt
+ENV TOOLCHAIN_RELEASE 15.0.3
+ENV TOOLCHAIN_BASENAME "clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-linux-musl"
+ENV TOOLCHAIN_URL https://codelinaro.jfrog.io/artifactory/codelinaro-toolchain-for-hexagon/v${TOOLCHAIN_RELEASE}/${TOOLCHAIN_BASENAME}.tar.xz
+
+RUN curl -#SL "$TOOLCHAIN_URL" | tar -xJC "$TOOLCHAIN_INSTALL"
+ENV PATH $PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin
-- 
2.34.1

[PULL 6/6] gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

These jobs use their own "script:" section and thus do not profit from
the global "--disable-docs" from the template. While we're at it, disable
also some GUI front ends here since we do not gain any additional test
coverage by compiling those here again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221208135945.99975-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-7-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 93302a96e2..f09a898c3e 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -244,6 +244,7 @@ build-tcg-disabled:
     - mkdir build
     - cd build
     - ../configure --disable-tcg --audio-drv-list="" --with-coroutine=ucontext
+                   --disable-docs --disable-sdl --disable-gtk --disable-vnc
       || { cat config.log meson-logs/meson-log.txt && exit 1; }
     - make -j"$JOBS"
     - make check-unit
@@ -530,8 +531,9 @@ build-tci:
     - TARGETS="aarch64 alpha arm hppa m68k microblaze ppc64 s390x x86_64"
     - mkdir build
     - cd build
-    - ../configure --enable-tcg-interpreter
-        --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" || { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - ../configure --enable-tcg-interpreter --disable-docs --disable-gtk --disable-vnc
+        --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)"
+        || { cat config.log meson-logs/meson-log.txt && exit 1; }
     - make -j"$JOBS"
     - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test
     - for tg in $TARGETS ; do
-- 
2.34.1

Re: [PULL 0/6] testing updates

[Peter Maydell]

On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>
>   Merge tag 'hppa-fixes-pull-request' of https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>
> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>
>   gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>
> ----------------------------------------------------------------
> testing updates:
>
>   - fix minor shell-ism that can break check-tcg
>   - turn off verbose logging on custom runners
>   - make configure echo call in CI
>   - fix unused variable in linux-test
>   - add binary compiler docker image for hexagon
>   - disable doc and gui builds for tci and disable-tcg builds
>

Gitlab's CI marks this with a "yaml invalid" tag:
https://gitlab.com/qemu-project/qemu/-/pipelines/729324088

and the error:
'build-user-hexagon' job needs 'hexagon-cross-container' job, but
'hexagon-cross-container' is not in any previous stage

thanks
-- PMM

RE: [PULL 0/6] testing updates

[Mukilan Thiyagarajan (QUIC)]

I believe the error is caused by the QEMU_JOB_ONLY_FORKS: 1 line
which needs to be removed from the definition of hexagon-cross-container.

Alex, Peter, 

Shall I raise a patch to remove this line? Should the patch be
created against the testing/next branch?

Thanks,
Mukilan

-----Original Message-----
From: qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org <qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org> On Behalf Of Peter Maydell
Sent: Thursday, December 22, 2022 4:45 PM
To: Alex Bennée <alex.bennee@linaro.org>
Cc: qemu-devel@nongnu.org
Subject: Re: [PULL 0/6] testing updates

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>
>   Merge tag 'hppa-fixes-pull-request' of 
> https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27 
> +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>
> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>
>   gitlab-ci: Disable docs and GUIs for the build-tci and 
> build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>
> ----------------------------------------------------------------
> testing updates:
>
>   - fix minor shell-ism that can break check-tcg
>   - turn off verbose logging on custom runners
>   - make configure echo call in CI
>   - fix unused variable in linux-test
>   - add binary compiler docker image for hexagon
>   - disable doc and gui builds for tci and disable-tcg builds
>

Gitlab's CI marks this with a "yaml invalid" tag:
https://gitlab.com/qemu-project/qemu/-/pipelines/729324088

and the error:
'build-user-hexagon' job needs 'hexagon-cross-container' job, but 'hexagon-cross-container' is not in any previous stage

thanks
-- PMM

Re: [PULL 0/6] testing updates

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>>
>>   Merge tag 'hppa-fixes-pull-request' of
>> https://github.com/hdeller/qemu-hppa into staging (2022-12-20
>> 15:32:27 +0000)
>>
>> are available in the Git repository at:
>>
>>   https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>>
>> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>>
>>   gitlab-ci: Disable docs and GUIs for the build-tci and
>> build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>>
>> ----------------------------------------------------------------
>> testing updates:
>>
>>   - fix minor shell-ism that can break check-tcg
>>   - turn off verbose logging on custom runners
>>   - make configure echo call in CI
>>   - fix unused variable in linux-test
>>   - add binary compiler docker image for hexagon
>>   - disable doc and gui builds for tci and disable-tcg builds
>>
>
> Gitlab's CI marks this with a "yaml invalid" tag:
> https://gitlab.com/qemu-project/qemu/-/pipelines/729324088
>
> and the error:
> 'build-user-hexagon' job needs 'hexagon-cross-container' job, but
> 'hexagon-cross-container' is not in any previous stage

Ahh and doesn't show up on my run:

  https://gitlab.com/stsquad/qemu/-/pipelines/728840940

I guess because it is qemu-project only.

>
> thanks
> -- PMM


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PULL 0/6] testing updates

[Alex Bennée]

"Mukilan Thiyagarajan (QUIC)" <quic_mthiyaga@quicinc.com> writes:

> I believe the error is caused by the QEMU_JOB_ONLY_FORKS: 1 line
> which needs to be removed from the definition of hexagon-cross-container.
>
> Alex, Peter, 
>
> Shall I raise a patch to remove this line? Should the patch be
> created against the testing/next branch?

Just show the patch here and I'll manually re-spin.

>
> Thanks,
> Mukilan
>
> -----Original Message-----
> From: qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org
> <qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org> On Behalf Of
> Peter Maydell
> Sent: Thursday, December 22, 2022 4:45 PM
> To: Alex Bennée <alex.bennee@linaro.org>
> Cc: qemu-devel@nongnu.org
> Subject: Re: [PULL 0/6] testing updates
>
> WARNING: This email originated from outside of Qualcomm. Please be
> wary of any links or attachments, and do not enable macros.
>
> On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>>
>>   Merge tag 'hppa-fixes-pull-request' of 
>> https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27 
>> +0000)
>>
>> are available in the Git repository at:
>>
>>   https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>>
>> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>>
>>   gitlab-ci: Disable docs and GUIs for the build-tci and 
>> build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>>
>> ----------------------------------------------------------------
>> testing updates:
>>
>>   - fix minor shell-ism that can break check-tcg
>>   - turn off verbose logging on custom runners
>>   - make configure echo call in CI
>>   - fix unused variable in linux-test
>>   - add binary compiler docker image for hexagon
>>   - disable doc and gui builds for tci and disable-tcg builds
>>
>
> Gitlab's CI marks this with a "yaml invalid" tag:
> https://gitlab.com/qemu-project/qemu/-/pipelines/729324088
>
> and the error:
> 'build-user-hexagon' job needs 'hexagon-cross-container' job, but 'hexagon-cross-container' is not in any previous stage
>
> thanks
> -- PMM


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PULL 0/6] testing updates

[Mukilan Thiyagarajan]

Thank you, Alex!

Here is the patch to fix the issue. I've also removed
the comment which is no longer applicable.

Regards,
Mukilan

---
 .gitlab-ci.d/container-cross.yml | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 5486dc43c6..e0d75d5824 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -34,14 +34,11 @@ armhf-debian-cross-container:
   variables:
     NAME: debian-armhf-cross
 
-# We never want to build hexagon in the CI system and by default we
-# always want to refer to the master registry where it lives.
 hexagon-cross-container:
   extends: .container_job_template
   stage: containers
   variables:
     NAME: debian-hexagon-cross
-    QEMU_JOB_ONLY_FORKS: 1
 
 hppa-debian-cross-container:
   extends: .container_job_template
-- 
2.17.1