* [PULL 1/6] configure: Fix check-tcg not executing any tests
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
@ 2022-12-21 14:40 ` Alex Bennée
2022-12-21 14:40 ` [PULL 2/6] gitlab: turn off verbose logging for make check on custom runners Alex Bennée
` (5 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-21 14:40 UTC (permalink / raw)
To: qemu-devel
Cc: peter.maydell, Mukilan Thiyagarajan, Richard Henderson,
Alex Bennée, Paolo Bonzini, Thomas Huth
From: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
After configuring with --target-list=hexagon-linux-user
running `make check-tcg` just prints the following:
```
make: Nothing to be done for 'check-tcg'
```
In the probe_target_compiler function, the 'break'
command is used incorrectly. There are no lexically
enclosing loops associated with that break command which
is an unspecfied behaviour in the POSIX standard.
The dash shell implementation aborts the currently executing
loop, in this case, causing the rest of the logic for the loop
in line 2490 to be skipped, which means no Makefiles are
generated for the tcg target tests.
Fixes: c3b570b5a9a24d25 (configure: don't enable
cross compilers unless in target_list)
Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Link: https://patchew.org/QEMU/20221207082309.9966-1-quic._5Fmthiyaga@quicinc.com/
Message-Id: <20221207082309.9966-1-quic_mthiyaga@quicinc.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221221090411.1995037-2-alex.bennee@linaro.org>
diff --git a/configure b/configure
index 26c7bc5154..7a804fb657 100755
--- a/configure
+++ b/configure
@@ -1881,9 +1881,7 @@ probe_target_compiler() {
# We shall skip configuring the target compiler if the user didn't
# bother enabling an appropriate guest. This avoids building
# extraneous firmware images and tests.
- if test "${target_list#*$1}" != "$1"; then
- break;
- else
+ if test "${target_list#*$1}" = "$1"; then
return 1
fi
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PULL 2/6] gitlab: turn off verbose logging for make check on custom runners
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
2022-12-21 14:40 ` [PULL 1/6] configure: Fix check-tcg not executing any tests Alex Bennée
@ 2022-12-21 14:40 ` Alex Bennée
2022-12-21 14:40 ` [PULL 3/6] configure: repeat ourselves for the benefit of CI Alex Bennée
` (4 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-21 14:40 UTC (permalink / raw)
To: qemu-devel
Cc: peter.maydell, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé, Wainer dos Santos Moschetta,
Beraldo Leal
The verbosity adds a lot of unnecessary output to the CI logs which
end up getting truncated anyway. We can always extract information
from the meson test logs on a failure and for the custom runners its
generally easier to re-create failures anyway.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221221090411.1995037-3-alex.bennee@linaro.org>
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
index 0c835939db..fcaef9e5ef 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml
@@ -19,9 +19,9 @@ ubuntu-20.04-s390x-all-linux-static:
- ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
- - make --output-sync -j`nproc` check-tcg V=1
+ - make --output-sync -j`nproc` check-tcg
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-20.04-s390x-all:
@@ -40,7 +40,7 @@ ubuntu-20.04-s390x-all:
- ../configure --disable-libssh
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-20.04-s390x-alldbg:
@@ -63,7 +63,7 @@ ubuntu-20.04-s390x-alldbg:
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make clean
- make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-20.04-s390x-clang:
@@ -85,7 +85,7 @@ ubuntu-20.04-s390x-clang:
- ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-20.04-s390x-tci:
@@ -127,5 +127,5 @@ ubuntu-20.04-s390x-notcg:
- ../configure --disable-libssh --disable-tcg
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
index 1a2f9b8dbe..2c386fa3e9 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
@@ -21,5 +21,5 @@ ubuntu-22.04-aarch32-all:
- ../configure --cross-prefix=arm-linux-gnueabihf-
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
index ce0b18af6f..abeb33eaff 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
@@ -19,9 +19,9 @@ ubuntu-22.04-aarch64-all-linux-static:
- ../configure --enable-debug --static --disable-system --disable-pie
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
- - make --output-sync -j`nproc --ignore=40` check-tcg V=1
+ - make --output-sync -j`nproc --ignore=40` check-tcg
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-22.04-aarch64-all:
@@ -43,7 +43,7 @@ ubuntu-22.04-aarch64-all:
- ../configure
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-22.04-aarch64-alldbg:
@@ -62,7 +62,7 @@ ubuntu-22.04-aarch64-alldbg:
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make clean
- make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-22.04-aarch64-clang:
@@ -84,7 +84,7 @@ ubuntu-22.04-aarch64-clang:
- ../configure --disable-libssh --cc=clang-10 --cxx=clang++-10 --enable-sanitizers
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
ubuntu-22.04-aarch64-tci:
@@ -126,5 +126,5 @@ ubuntu-22.04-aarch64-notcg:
- ../configure --disable-tcg
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
- make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check V=1
+ - make --output-sync -j`nproc --ignore=40` check
|| { cat meson-logs/testlog.txt; exit 1; } ;
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PULL 3/6] configure: repeat ourselves for the benefit of CI
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
2022-12-21 14:40 ` [PULL 1/6] configure: Fix check-tcg not executing any tests Alex Bennée
2022-12-21 14:40 ` [PULL 2/6] gitlab: turn off verbose logging for make check on custom runners Alex Bennée
@ 2022-12-21 14:40 ` Alex Bennée
2022-12-21 14:40 ` [PULL 4/6] tests/tcg: fix unused variable in linux-test Alex Bennée
` (3 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-21 14:40 UTC (permalink / raw)
To: qemu-devel
Cc: peter.maydell, Alex Bennée, Philippe Mathieu-Daudé,
Paolo Bonzini, Thomas Huth
Our CI system echos the lines it executes but not the expansions. For
the sake of a line of extra verbosity during the configure phase lets
echo the invocation of script to stdout as well as the log when on CI.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-4-alex.bennee@linaro.org>
diff --git a/configure b/configure
index 7a804fb657..d89e883844 100755
--- a/configure
+++ b/configure
@@ -83,9 +83,10 @@ rm -f config.log
# Print a helpful header at the top of config.log
echo "# QEMU configure log $(date)" >> config.log
printf "# Configured with:" >> config.log
-printf " '%s'" "$0" "$@" >> config.log
-echo >> config.log
-echo "#" >> config.log
+# repeat the invocation to log and stdout for CI
+invoke=$(printf " '%s'" "$0" "$@")
+test -n "$GITLAB_CI" && echo "configuring with: $invoke"
+{ echo "$invoke"; echo; echo "#"; } >> config.log
quote_sh() {
printf "%s" "$1" | sed "s,','\\\\'',g; s,.*,'&',"
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PULL 4/6] tests/tcg: fix unused variable in linux-test
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
` (2 preceding siblings ...)
2022-12-21 14:40 ` [PULL 3/6] configure: repeat ourselves for the benefit of CI Alex Bennée
@ 2022-12-21 14:40 ` Alex Bennée
2022-12-21 14:40 ` [PULL 5/6] tests/docker: use prebuilt toolchain for debian-hexagon-cross Alex Bennée
` (2 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-21 14:40 UTC (permalink / raw)
To: qemu-devel; +Cc: peter.maydell, Alex Bennée, Philippe Mathieu-Daudé
The latest hexagon compiler picks up that we never consume wcount.
Given the name of the #define that rcount checks against is WCOUNT_MAX
I figured the check just got missed.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-5-alex.bennee@linaro.org>
diff --git a/tests/tcg/multiarch/linux/linux-test.c b/tests/tcg/multiarch/linux/linux-test.c
index 5a2a4f2258..64f57cb287 100644
--- a/tests/tcg/multiarch/linux/linux-test.c
+++ b/tests/tcg/multiarch/linux/linux-test.c
@@ -354,13 +354,17 @@ static void test_pipe(void)
if (FD_ISSET(fds[0], &rfds)) {
chk_error(read(fds[0], &ch, 1));
rcount++;
- if (rcount >= WCOUNT_MAX)
+ if (rcount >= WCOUNT_MAX) {
break;
+ }
}
if (FD_ISSET(fds[1], &wfds)) {
ch = 'a';
chk_error(write(fds[1], &ch, 1));
wcount++;
+ if (wcount >= WCOUNT_MAX) {
+ break;
+ }
}
}
}
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PULL 5/6] tests/docker: use prebuilt toolchain for debian-hexagon-cross
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
` (3 preceding siblings ...)
2022-12-21 14:40 ` [PULL 4/6] tests/tcg: fix unused variable in linux-test Alex Bennée
@ 2022-12-21 14:40 ` Alex Bennée
2022-12-21 14:40 ` [PULL 6/6] gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs Alex Bennée
2022-12-22 11:14 ` [PULL 0/6] testing updates Peter Maydell
6 siblings, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-21 14:40 UTC (permalink / raw)
To: qemu-devel
Cc: peter.maydell, Mukilan Thiyagarajan, Alex Bennée,
Philippe Mathieu-Daudé, Thomas Huth,
Wainer dos Santos Moschetta, Beraldo Leal
From: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
The current docker image for cross compiling hexagon guests
is manually built since it takes >2 hours to build from source.
This patch:
1. Solves the above issue by using the prebuilt clang
toolchain hosted on CodeLinaro [1] and maintained by QUIC [2].
2. The dockerfile is also switched from multi-stage to single stage
build to allow the CI docker engine to reuse the layer cache.
3. Re-enables the hexagon-cross-container job to be always run in
CI and makes it a non-optional dependency for the
build-user-hexagon job.
The changes for 1 & 2 together bring down the build time to
~3 minutes in GitLab CI when cache is reused and ~9 minutes
when cache cannot be reused.
[1]: https://github.com/CodeLinaro/hexagon-builder
[2]: https://github.com/quic/toolchain_for_hexagon/releases/
Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
[AJB: also tweak MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221219144354.11659-1-quic_mthiyaga@quicinc.com>
Message-Id: <20221221090411.1995037-6-alex.bennee@linaro.org>
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index d21b4a1fd4..93302a96e2 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -275,14 +275,10 @@ build-user-static:
CONFIGURE_ARGS: --disable-tools --disable-system --static
MAKE_CHECK_ARGS: check-tcg
-# Because the hexagon cross-compiler takes so long to build we don't rely
-# on the CI system to build it and hence this job has an optional dependency
-# declared. The image is manually uploaded.
build-user-hexagon:
extends: .native_build_job_template
needs:
job: hexagon-cross-container
- optional: true
variables:
IMAGE: debian-hexagon-cross
TARGETS: hexagon-linux-user
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 2d560e9764..5486dc43c6 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -37,28 +37,11 @@ armhf-debian-cross-container:
# We never want to build hexagon in the CI system and by default we
# always want to refer to the master registry where it lives.
hexagon-cross-container:
- extends: .base_job_template
- image: docker:stable
+ extends: .container_job_template
stage: containers
variables:
NAME: debian-hexagon-cross
- GIT_DEPTH: 1
QEMU_JOB_ONLY_FORKS: 1
- services:
- - docker:dind
- before_script:
- - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
- - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
- - docker info
- - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
- script:
- - echo "TAG:$TAG"
- - echo "COMMON_TAG:$COMMON_TAG"
- - docker pull $COMMON_TAG
- - docker tag $COMMON_TAG $TAG
- - docker push "$TAG"
- after_script:
- - docker logout
hppa-debian-cross-container:
extends: .container_job_template
diff --git a/MAINTAINERS b/MAINTAINERS
index 716d5a24ad..005a2d3ed2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -204,7 +204,6 @@ F: tests/tcg/hexagon/
F: disas/hexagon.c
F: configs/targets/hexagon-linux-user/default.mak
F: docker/dockerfiles/debian-hexagon-cross.docker
-F: docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
Hexagon idef-parser
M: Alessandro Di Federico <ale@rev.ng>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index fc7a3b7e71..665ddde518 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -109,10 +109,6 @@ debian-toolchain-run = \
"PREPARE", $1))
debian-toolchain = $(call debian-toolchain-run,$(patsubst docker-image-%,%,$1))
-docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.docker \
- $(DOCKER_FILES_DIR)/debian-hexagon-cross.d/build-toolchain.sh
- $(call debian-toolchain, $@)
-
docker-image-debian-microblaze-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
$(DOCKER_FILES_DIR)/debian-microblaze-cross.d/build-toolchain.sh
$(call debian-toolchain, $@)
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh
deleted file mode 100755
index 19b1c9f83e..0000000000
--- a/tests/docker/dockerfiles/debian-hexagon-cross.d/build-toolchain.sh
+++ /dev/null
@@ -1,141 +0,0 @@
-#!/bin/bash
-
-set -e
-
-BASE=$(readlink -f ${PWD})
-
-TOOLCHAIN_INSTALL=$(readlink -f "$TOOLCHAIN_INSTALL")
-ROOTFS=$(readlink -f "$ROOTFS")
-
-TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
-HEX_SYSROOT=${TOOLCHAIN_INSTALL}/hexagon-unknown-linux-musl
-HEX_TOOLS_TARGET_BASE=${HEX_SYSROOT}/usr
-
-function cdp() {
- DIR="$1"
- mkdir -p "$DIR"
- cd "$DIR"
-}
-
-function fetch() {
- DIR="$1"
- URL="$2"
- TEMP="$(readlink -f "$PWD/tmp.tar.gz")"
- wget --quiet "$URL" -O "$TEMP"
- cdp "$DIR"
- tar xaf "$TEMP" --strip-components=1
- rm "$TEMP"
- cd -
-}
-
-build_llvm_clang() {
- fetch "$BASE/llvm-project" "$LLVM_URL"
- cdp "$BASE/build-llvm"
-
- cmake -G Ninja \
- -DCMAKE_BUILD_TYPE=Release \
- -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \
- -DLLVM_ENABLE_LLD=ON \
- -DLLVM_TARGETS_TO_BUILD="Hexagon" \
- -DLLVM_ENABLE_PROJECTS="clang;lld" \
- "$BASE/llvm-project/llvm"
- ninja all install
- cd ${TOOLCHAIN_BIN}
- ln -sf clang hexagon-unknown-linux-musl-clang
- ln -sf clang++ hexagon-unknown-linux-musl-clang++
- ln -sf llvm-ar hexagon-unknown-linux-musl-ar
- ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump
- ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy
- ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf
- ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib
-
- # workaround for now:
- cat <<EOF > hexagon-unknown-linux-musl.cfg
--G0 --sysroot=${HEX_SYSROOT}
-EOF
-}
-
-build_clang_rt() {
- cdp "$BASE/build-clang_rt"
- cmake -G Ninja \
- -DCMAKE_BUILD_TYPE=Release \
- -DLLVM_CONFIG_PATH="$BASE/build-llvm/bin/llvm-config" \
- -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \
- -DCMAKE_SYSTEM_NAME=Linux \
- -DCMAKE_C_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
- -DCMAKE_ASM_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
- -DCMAKE_INSTALL_PREFIX=${HEX_TOOLS_TARGET_BASE} \
- -DCMAKE_CROSSCOMPILING=ON \
- -DCMAKE_C_COMPILER_FORCED=ON \
- -DCMAKE_CXX_COMPILER_FORCED=ON \
- -DCOMPILER_RT_BUILD_BUILTINS=ON \
- -DCOMPILER_RT_BUILTINS_ENABLE_PIC=OFF \
- -DCMAKE_SIZEOF_VOID_P=4 \
- -DCOMPILER_RT_OS_DIR= \
- -DCAN_TARGET_hexagon=1 \
- -DCAN_TARGET_x86_64=0 \
- -DCOMPILER_RT_SUPPORTED_ARCH=hexagon \
- -DLLVM_ENABLE_PROJECTS="compiler-rt" \
- "$BASE/llvm-project/compiler-rt"
- ninja install-compiler-rt
-}
-
-build_musl_headers() {
- fetch "$BASE/musl" "$MUSL_URL"
- cd "$BASE/musl"
- make clean
- CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
- CROSS_COMPILE=hexagon-unknown-linux-musl \
- LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
- CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
- ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
- PATH=${TOOLCHAIN_BIN}:$PATH make CROSS_COMPILE= install-headers
-
- cd ${HEX_SYSROOT}/..
- ln -sf hexagon-unknown-linux-musl hexagon
-}
-
-build_kernel_headers() {
- fetch "$BASE/linux" "$LINUX_URL"
- mkdir -p "$BASE/build-linux"
- cd "$BASE/linux"
- make O=../build-linux ARCH=hexagon \
- KBUILD_CFLAGS_KERNEL="-mlong-calls" \
- CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
- LD=${TOOLCHAIN_BIN}/ld.lld \
- KBUILD_VERBOSE=1 comet_defconfig
- make mrproper
-
- cd "$BASE/build-linux"
- make \
- ARCH=hexagon \
- CC=${TOOLCHAIN_BIN}/clang \
- INSTALL_HDR_PATH=${HEX_TOOLS_TARGET_BASE} \
- V=1 \
- headers_install
-}
-
-build_musl() {
- cd "$BASE/musl"
- make clean
- CROSS_COMPILE=hexagon-unknown-linux-musl- \
- AR=llvm-ar \
- RANLIB=llvm-ranlib \
- STRIP=llvm-strip \
- CC=clang \
- LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
- CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
- ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
- PATH=${TOOLCHAIN_BIN}/:$PATH make CROSS_COMPILE= install
- cd ${HEX_TOOLS_TARGET_BASE}/lib
- ln -sf libc.so ld-musl-hexagon.so
- ln -sf ld-musl-hexagon.so ld-musl-hexagon.so.1
- cdp ${HEX_TOOLS_TARGET_BASE}/../lib
- ln -sf ../usr/lib/ld-musl-hexagon.so.1
-}
-
-build_llvm_clang
-build_kernel_headers
-build_musl_headers
-build_clang_rt
-build_musl
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker
index c4238e893f..8a0d748343 100644
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -7,44 +7,29 @@
#
FROM docker.io/library/debian:11-slim
-# Install common build utilities
-RUN apt update && \
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- bison \
+# Install common build utilities
+ apt-get install -y --no-install-recommends \
+ curl \
+ xz-utils \
ca-certificates \
- clang \
- cmake \
+ bison \
flex \
- gcc \
- lld \
- make \
- ninja-build \
- python3 \
- rsync \
- wget \
- xz-utils
-
-ENV TOOLCHAIN_INSTALL /usr/local
-ENV ROOTFS /usr/local
-
-ENV LLVM_URL https://github.com/llvm/llvm-project/archive/bfcd21876adc3498065e4da92799f613e730d475.tar.gz
-ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz
-ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz
-
-ADD build-toolchain.sh /root/hexagon-toolchain/build-toolchain.sh
-
-RUN cd /root/hexagon-toolchain && ./build-toolchain.sh
-
-FROM docker.io/library/debian:11-slim
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+ git \
+ ninja-build && \
# Install QEMU build deps for use in CI
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
- DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy bison flex git ninja-build && \
DEBIAN_FRONTEND=noninteractive eatmydata \
apt build-dep -yy --arch-only qemu
-COPY --from=0 /usr/local /usr/local
-ENV PATH $PATH:/usr/local/bin/
+
+
+ENV TOOLCHAIN_INSTALL /opt
+ENV TOOLCHAIN_RELEASE 15.0.3
+ENV TOOLCHAIN_BASENAME "clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-linux-musl"
+ENV TOOLCHAIN_URL https://codelinaro.jfrog.io/artifactory/codelinaro-toolchain-for-hexagon/v${TOOLCHAIN_RELEASE}/${TOOLCHAIN_BASENAME}.tar.xz
+
+RUN curl -#SL "$TOOLCHAIN_URL" | tar -xJC "$TOOLCHAIN_INSTALL"
+ENV PATH $PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PULL 6/6] gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
` (4 preceding siblings ...)
2022-12-21 14:40 ` [PULL 5/6] tests/docker: use prebuilt toolchain for debian-hexagon-cross Alex Bennée
@ 2022-12-21 14:40 ` Alex Bennée
2022-12-22 11:14 ` [PULL 0/6] testing updates Peter Maydell
6 siblings, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-21 14:40 UTC (permalink / raw)
To: qemu-devel
Cc: peter.maydell, Thomas Huth, Alex Bennée,
Philippe Mathieu-Daudé, Wainer dos Santos Moschetta,
Beraldo Leal
From: Thomas Huth <thuth@redhat.com>
These jobs use their own "script:" section and thus do not profit from
the global "--disable-docs" from the template. While we're at it, disable
also some GUI front ends here since we do not gain any additional test
coverage by compiling those here again.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221208135945.99975-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221221090411.1995037-7-alex.bennee@linaro.org>
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 93302a96e2..f09a898c3e 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -244,6 +244,7 @@ build-tcg-disabled:
- mkdir build
- cd build
- ../configure --disable-tcg --audio-drv-list="" --with-coroutine=ucontext
+ --disable-docs --disable-sdl --disable-gtk --disable-vnc
|| { cat config.log meson-logs/meson-log.txt && exit 1; }
- make -j"$JOBS"
- make check-unit
@@ -530,8 +531,9 @@ build-tci:
- TARGETS="aarch64 alpha arm hppa m68k microblaze ppc64 s390x x86_64"
- mkdir build
- cd build
- - ../configure --enable-tcg-interpreter
- --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" || { cat config.log meson-logs/meson-log.txt && exit 1; }
+ - ../configure --enable-tcg-interpreter --disable-docs --disable-gtk --disable-vnc
+ --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)"
+ || { cat config.log meson-logs/meson-log.txt && exit 1; }
- make -j"$JOBS"
- make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test
- for tg in $TARGETS ; do
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PULL 0/6] testing updates
2022-12-21 14:40 [PULL 0/6] testing updates Alex Bennée
` (5 preceding siblings ...)
2022-12-21 14:40 ` [PULL 6/6] gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs Alex Bennée
@ 2022-12-22 11:14 ` Peter Maydell
2022-12-22 11:35 ` Mukilan Thiyagarajan (QUIC)
2022-12-22 20:31 ` Alex Bennée
6 siblings, 2 replies; 12+ messages in thread
From: Peter Maydell @ 2022-12-22 11:14 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel
On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>
> Merge tag 'hppa-fixes-pull-request' of https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27 +0000)
>
> are available in the Git repository at:
>
> https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>
> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>
> gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>
> ----------------------------------------------------------------
> testing updates:
>
> - fix minor shell-ism that can break check-tcg
> - turn off verbose logging on custom runners
> - make configure echo call in CI
> - fix unused variable in linux-test
> - add binary compiler docker image for hexagon
> - disable doc and gui builds for tci and disable-tcg builds
>
Gitlab's CI marks this with a "yaml invalid" tag:
https://gitlab.com/qemu-project/qemu/-/pipelines/729324088
and the error:
'build-user-hexagon' job needs 'hexagon-cross-container' job, but
'hexagon-cross-container' is not in any previous stage
thanks
-- PMM
^ permalink raw reply [flat|nested] 12+ messages in thread* RE: [PULL 0/6] testing updates
2022-12-22 11:14 ` [PULL 0/6] testing updates Peter Maydell
@ 2022-12-22 11:35 ` Mukilan Thiyagarajan (QUIC)
2022-12-22 20:32 ` Alex Bennée
2022-12-22 20:31 ` Alex Bennée
1 sibling, 1 reply; 12+ messages in thread
From: Mukilan Thiyagarajan (QUIC) @ 2022-12-22 11:35 UTC (permalink / raw)
To: Peter Maydell, Alex Bennée; +Cc: qemu-devel@nongnu.org
I believe the error is caused by the QEMU_JOB_ONLY_FORKS: 1 line
which needs to be removed from the definition of hexagon-cross-container.
Alex, Peter,
Shall I raise a patch to remove this line? Should the patch be
created against the testing/next branch?
Thanks,
Mukilan
-----Original Message-----
From: qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org <qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org> On Behalf Of Peter Maydell
Sent: Thursday, December 22, 2022 4:45 PM
To: Alex Bennée <alex.bennee@linaro.org>
Cc: qemu-devel@nongnu.org
Subject: Re: [PULL 0/6] testing updates
WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>
> Merge tag 'hppa-fixes-pull-request' of
> https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27
> +0000)
>
> are available in the Git repository at:
>
> https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>
> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>
> gitlab-ci: Disable docs and GUIs for the build-tci and
> build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>
> ----------------------------------------------------------------
> testing updates:
>
> - fix minor shell-ism that can break check-tcg
> - turn off verbose logging on custom runners
> - make configure echo call in CI
> - fix unused variable in linux-test
> - add binary compiler docker image for hexagon
> - disable doc and gui builds for tci and disable-tcg builds
>
Gitlab's CI marks this with a "yaml invalid" tag:
https://gitlab.com/qemu-project/qemu/-/pipelines/729324088
and the error:
'build-user-hexagon' job needs 'hexagon-cross-container' job, but 'hexagon-cross-container' is not in any previous stage
thanks
-- PMM
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PULL 0/6] testing updates
2022-12-22 11:35 ` Mukilan Thiyagarajan (QUIC)
@ 2022-12-22 20:32 ` Alex Bennée
2022-12-23 5:04 ` Mukilan Thiyagarajan
0 siblings, 1 reply; 12+ messages in thread
From: Alex Bennée @ 2022-12-22 20:32 UTC (permalink / raw)
To: Mukilan Thiyagarajan (QUIC); +Cc: Peter Maydell, qemu-devel@nongnu.org
"Mukilan Thiyagarajan (QUIC)" <quic_mthiyaga@quicinc.com> writes:
> I believe the error is caused by the QEMU_JOB_ONLY_FORKS: 1 line
> which needs to be removed from the definition of hexagon-cross-container.
>
> Alex, Peter,
>
> Shall I raise a patch to remove this line? Should the patch be
> created against the testing/next branch?
Just show the patch here and I'll manually re-spin.
>
> Thanks,
> Mukilan
>
> -----Original Message-----
> From: qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org
> <qemu-devel-bounces+quic_mthiyaga=quicinc.com@nongnu.org> On Behalf Of
> Peter Maydell
> Sent: Thursday, December 22, 2022 4:45 PM
> To: Alex Bennée <alex.bennee@linaro.org>
> Cc: qemu-devel@nongnu.org
> Subject: Re: [PULL 0/6] testing updates
>
> WARNING: This email originated from outside of Qualcomm. Please be
> wary of any links or attachments, and do not enable macros.
>
> On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>>
>> Merge tag 'hppa-fixes-pull-request' of
>> https://github.com/hdeller/qemu-hppa into staging (2022-12-20 15:32:27
>> +0000)
>>
>> are available in the Git repository at:
>>
>> https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>>
>> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>>
>> gitlab-ci: Disable docs and GUIs for the build-tci and
>> build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>>
>> ----------------------------------------------------------------
>> testing updates:
>>
>> - fix minor shell-ism that can break check-tcg
>> - turn off verbose logging on custom runners
>> - make configure echo call in CI
>> - fix unused variable in linux-test
>> - add binary compiler docker image for hexagon
>> - disable doc and gui builds for tci and disable-tcg builds
>>
>
> Gitlab's CI marks this with a "yaml invalid" tag:
> https://gitlab.com/qemu-project/qemu/-/pipelines/729324088
>
> and the error:
> 'build-user-hexagon' job needs 'hexagon-cross-container' job, but 'hexagon-cross-container' is not in any previous stage
>
> thanks
> -- PMM
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PULL 0/6] testing updates
2022-12-22 20:32 ` Alex Bennée
@ 2022-12-23 5:04 ` Mukilan Thiyagarajan
0 siblings, 0 replies; 12+ messages in thread
From: Mukilan Thiyagarajan @ 2022-12-23 5:04 UTC (permalink / raw)
To: alex.bennee; +Cc: qemu-devel, peter.maydell, Mukilan Thiyagarajan
Thank you, Alex!
Here is the patch to fix the issue. I've also removed
the comment which is no longer applicable.
Regards,
Mukilan
---
.gitlab-ci.d/container-cross.yml | 3 ---
1 file changed, 3 deletions(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 5486dc43c6..e0d75d5824 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -34,14 +34,11 @@ armhf-debian-cross-container:
variables:
NAME: debian-armhf-cross
-# We never want to build hexagon in the CI system and by default we
-# always want to refer to the master registry where it lives.
hexagon-cross-container:
extends: .container_job_template
stage: containers
variables:
NAME: debian-hexagon-cross
- QEMU_JOB_ONLY_FORKS: 1
hppa-debian-cross-container:
extends: .container_job_template
--
2.17.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PULL 0/6] testing updates
2022-12-22 11:14 ` [PULL 0/6] testing updates Peter Maydell
2022-12-22 11:35 ` Mukilan Thiyagarajan (QUIC)
@ 2022-12-22 20:31 ` Alex Bennée
1 sibling, 0 replies; 12+ messages in thread
From: Alex Bennée @ 2022-12-22 20:31 UTC (permalink / raw)
To: Peter Maydell; +Cc: qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> On Wed, 21 Dec 2022 at 14:40, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 8540a1f69578afb3b37866b1ce5bec46a9f6efbc:
>>
>> Merge tag 'hppa-fixes-pull-request' of
>> https://github.com/hdeller/qemu-hppa into staging (2022-12-20
>> 15:32:27 +0000)
>>
>> are available in the Git repository at:
>>
>> https://gitlab.com/stsquad/qemu.git tags/pull-testing-next-211222-1
>>
>> for you to fetch changes up to 7a8ec48480c116db93e0d91688be1dcf34147795:
>>
>> gitlab-ci: Disable docs and GUIs for the build-tci and
>> build-tcg-disabled jobs (2022-12-21 11:19:05 +0000)
>>
>> ----------------------------------------------------------------
>> testing updates:
>>
>> - fix minor shell-ism that can break check-tcg
>> - turn off verbose logging on custom runners
>> - make configure echo call in CI
>> - fix unused variable in linux-test
>> - add binary compiler docker image for hexagon
>> - disable doc and gui builds for tci and disable-tcg builds
>>
>
> Gitlab's CI marks this with a "yaml invalid" tag:
> https://gitlab.com/qemu-project/qemu/-/pipelines/729324088
>
> and the error:
> 'build-user-hexagon' job needs 'hexagon-cross-container' job, but
> 'hexagon-cross-container' is not in any previous stage
Ahh and doesn't show up on my run:
https://gitlab.com/stsquad/qemu/-/pipelines/728840940
I guess because it is qemu-project only.
>
> thanks
> -- PMM
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 12+ messages in thread