[PATCH 0/5] testing/next - arm custom runner tweaks

qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed

* [PATCH 0/5] testing/next - arm custom runner tweaks
@ 2025-10-14 10:43 Alex Bennée
  2025-10-14 10:43 ` [PATCH 1/5] tests/lcitool: bump to latest version and regenerate (!wait on upstream) Alex Bennée
                   ` (5 more replies)
  0 siblings, 6 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 10:43 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Thomas Huth, qemu-arm, Gustavo Romero,
	Alex Bennée, Philippe Mathieu-Daudé, BALATON Zoltan,
	qemu-s390x

This isn't quite ready yet as there is an in-flight MR for libvirt-ci
and the final patch has to manually drop a couple of packages which
don't cleanly install. I don't know if we should deal with that
upstream or in the lcitool metadata. For reference the two packages
are:

  libcurl4-gnutls-dev:armhf
  libglusterfs-dev:armhf

which I don't think are crucial to coverage. The main motivation was
to include the crypto libs so the block io tests don't fail.

However the the bulk of it works to:

  - improve utilisation of the new CI machines
  - remove hacks that get in the way of private test runners
  - fully drive ansible packages with lcitool
  - widen the testing of 32 bit Arm on the custom runner

Please review.

Alex.

Alex Bennée (5):
  tests/lcitool: bump to latest version and regenerate (!wait on
    upstream)
  gitlab: use template for ubuntu-24.04-aarch64 jobs
  gitlab: modernise the ubuntu24.04-aarch32 jobs
  scripts/setup: update the armhf overlay
  scripts/setup: update build-environment to use armhf yaml (!WIP)

 .../custom-runners/ubuntu-24.04-aarch32.yml   |  15 +-
 .../custom-runners/ubuntu-24.04-aarch64.yml   | 230 ++++++++----------
 scripts/ci/setup/ubuntu/build-environment.yml |  22 +-
 .../ci/setup/ubuntu/ubuntu-2404-aarch64.yaml  |   4 +-
 ...armhf-cross.yml => ubuntu-2404-armhf.yaml} |  67 +----
 .../ci/setup/ubuntu/ubuntu-2404-s390x.yaml    |   4 +-
 tests/docker/dockerfiles/alpine.docker        |  10 +-
 tests/docker/dockerfiles/centos9.docker       |  10 +-
 .../dockerfiles/debian-amd64-cross.docker     |  16 +-
 .../dockerfiles/debian-arm64-cross.docker     |  16 +-
 .../dockerfiles/debian-armhf-cross.docker     |  16 +-
 .../dockerfiles/debian-i686-cross.docker      |  16 +-
 .../dockerfiles/debian-mips64el-cross.docker  |  16 +-
 .../dockerfiles/debian-mipsel-cross.docker    |  16 +-
 .../dockerfiles/debian-ppc64el-cross.docker   |  16 +-
 .../dockerfiles/debian-riscv64-cross.docker   |  16 +-
 .../dockerfiles/debian-s390x-cross.docker     |  16 +-
 tests/docker/dockerfiles/debian.docker        |  10 +-
 .../dockerfiles/fedora-rust-nightly.docker    |  10 +-
 .../dockerfiles/fedora-win64-cross.docker     |  14 +-
 tests/docker/dockerfiles/fedora.docker        |  10 +-
 tests/docker/dockerfiles/opensuse-leap.docker |  10 +-
 tests/docker/dockerfiles/ubuntu2204.docker    |  10 +-
 tests/lcitool/libvirt-ci                      |   2 +-
 tests/lcitool/refresh                         |  11 +
 25 files changed, 259 insertions(+), 324 deletions(-)
 rename scripts/ci/setup/ubuntu/{ubuntu-2204-armhf-cross.yml => ubuntu-2404-armhf.yaml} (63%)

-- 
2.47.3



^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH 1/5] tests/lcitool: bump to latest version and regenerate (!wait on upstream)
  2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
@ 2025-10-14 10:43 ` Alex Bennée
  2025-10-14 10:43 ` [PATCH 2/5] gitlab: use template for ubuntu-24.04-aarch64 jobs Alex Bennée
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 10:43 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Thomas Huth, qemu-arm, Gustavo Romero,
	Alex Bennée, Philippe Mathieu-Daudé, BALATON Zoltan,
	qemu-s390x

Now that lcitool can build cross images for Ubuntu and expose just the
foreign layer to yaml output re-generate the armhf package list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
upstream MR: https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/552
---
 scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml |  4 ++--
 scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml   |  4 ++--
 tests/docker/dockerfiles/alpine.docker           | 10 +++++-----
 tests/docker/dockerfiles/centos9.docker          | 10 +++++-----
 .../docker/dockerfiles/debian-amd64-cross.docker | 16 ++++++++--------
 .../docker/dockerfiles/debian-arm64-cross.docker | 16 ++++++++--------
 .../docker/dockerfiles/debian-armhf-cross.docker | 16 ++++++++--------
 .../docker/dockerfiles/debian-i686-cross.docker  | 16 ++++++++--------
 .../dockerfiles/debian-mips64el-cross.docker     | 16 ++++++++--------
 .../dockerfiles/debian-mipsel-cross.docker       | 16 ++++++++--------
 .../dockerfiles/debian-ppc64el-cross.docker      | 16 ++++++++--------
 .../dockerfiles/debian-riscv64-cross.docker      | 16 ++++++++--------
 .../docker/dockerfiles/debian-s390x-cross.docker | 16 ++++++++--------
 tests/docker/dockerfiles/debian.docker           | 10 +++++-----
 .../dockerfiles/fedora-rust-nightly.docker       | 10 +++++-----
 .../docker/dockerfiles/fedora-win64-cross.docker | 14 +++++++-------
 tests/docker/dockerfiles/fedora.docker           | 10 +++++-----
 tests/docker/dockerfiles/opensuse-leap.docker    | 10 +++++-----
 tests/docker/dockerfiles/ubuntu2204.docker       | 10 +++++-----
 tests/lcitool/libvirt-ci                         |  2 +-
 20 files changed, 119 insertions(+), 119 deletions(-)

diff --git a/scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml b/scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml
index ce632d97108..70063db198e 100644
--- a/scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml
+++ b/scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml
@@ -26,7 +26,7 @@ packages:
   - git
   - hostname
   - libaio-dev
-  - libasan6
+  - libasan8
   - libasound2-dev
   - libattr1-dev
   - libbpf-dev
@@ -37,7 +37,7 @@ packages:
   - libcap-ng-dev
   - libcapstone-dev
   - libcbor-dev
-  - libclang-dev
+  - libclang-rt-dev
   - libcmocka-dev
   - libcurl4-gnutls-dev
   - libdaxctl-dev
diff --git a/scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml b/scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml
index f45f75c9602..4f1a49be34a 100644
--- a/scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml
+++ b/scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml
@@ -26,7 +26,7 @@ packages:
   - git
   - hostname
   - libaio-dev
-  - libasan6
+  - libasan8
   - libasound2-dev
   - libattr1-dev
   - libbpf-dev
@@ -37,7 +37,7 @@ packages:
   - libcap-ng-dev
   - libcapstone-dev
   - libcbor-dev
-  - libclang-dev
+  - libclang-rt-dev
   - libcmocka-dev
   - libcurl4-gnutls-dev
   - libdaxctl-dev
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index 52adf9ccbb7..1da375c9158 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -126,11 +126,11 @@ RUN apk update && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 # https://gitlab.alpinelinux.org/alpine/aports/-/issues/17463
 RUN apk add clang19-libclang
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/centos9.docker b/tests/docker/dockerfiles/centos9.docker
index 0674d778262..ff3e8069b45 100644
--- a/tests/docker/dockerfiles/centos9.docker
+++ b/tests/docker/dockerfiles/centos9.docker
@@ -132,11 +132,11 @@ RUN dnf distro-sync -y && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 # As a final step configure the user (if env is defined)
 ARG USER
 ARG UID
diff --git a/tests/docker/dockerfiles/debian-amd64-cross.docker b/tests/docker/dockerfiles/debian-amd64-cross.docker
index 7f4674400df..26a39940126 100644
--- a/tests/docker/dockerfiles/debian-amd64-cross.docker
+++ b/tests/docker/dockerfiles/debian-amd64-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture amd64 && \
@@ -178,9 +178,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/x86_64-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-linux-gnu-gcc
 
-ENV ABI "x86_64-linux-gnu"
-ENV MESON_OPTS "--cross-file=x86_64-linux-gnu"
-ENV RUST_TARGET "x86_64-unknown-linux-gnu"
+ENV ABI="x86_64-linux-gnu"
+ENV MESON_OPTS="--cross-file=x86_64-linux-gnu"
+ENV RUST_TARGET="x86_64-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=x86_64-linux-gnu-
 ENV DEF_TARGET_LIST x86_64-softmmu,x86_64-linux-user,i386-softmmu,i386-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-arm64-cross.docker b/tests/docker/dockerfiles/debian-arm64-cross.docker
index c7cd54ee5ca..4b0163fa1b9 100644
--- a/tests/docker/dockerfiles/debian-arm64-cross.docker
+++ b/tests/docker/dockerfiles/debian-arm64-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture arm64 && \
@@ -177,9 +177,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/aarch64-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/aarch64-linux-gnu-gcc
 
-ENV ABI "aarch64-linux-gnu"
-ENV MESON_OPTS "--cross-file=aarch64-linux-gnu"
-ENV RUST_TARGET "aarch64-unknown-linux-gnu"
+ENV ABI="aarch64-linux-gnu"
+ENV MESON_OPTS="--cross-file=aarch64-linux-gnu"
+ENV RUST_TARGET="aarch64-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=aarch64-linux-gnu-
 ENV DEF_TARGET_LIST aarch64-softmmu,aarch64-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
index 627d41c6dee..742031ce0a2 100644
--- a/tests/docker/dockerfiles/debian-armhf-cross.docker
+++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture armhf && \
@@ -174,9 +174,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/arm-linux-gnueabihf && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-gcc
 
-ENV ABI "arm-linux-gnueabihf"
-ENV MESON_OPTS "--cross-file=arm-linux-gnueabihf"
-ENV RUST_TARGET "armv7-unknown-linux-gnueabihf"
+ENV ABI="arm-linux-gnueabihf"
+ENV MESON_OPTS="--cross-file=arm-linux-gnueabihf"
+ENV RUST_TARGET="armv7-unknown-linux-gnueabihf"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
 ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-i686-cross.docker b/tests/docker/dockerfiles/debian-i686-cross.docker
index 2998764065f..1ccfde2e4bb 100644
--- a/tests/docker/dockerfiles/debian-i686-cross.docker
+++ b/tests/docker/dockerfiles/debian-i686-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture i386 && \
@@ -174,9 +174,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/i686-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-linux-gnu-gcc
 
-ENV ABI "i686-linux-gnu"
-ENV MESON_OPTS "--cross-file=i686-linux-gnu"
-ENV RUST_TARGET "i686-unknown-linux-gnu"
+ENV ABI="i686-linux-gnu"
+ENV MESON_OPTS="--cross-file=i686-linux-gnu"
+ENV RUST_TARGET="i686-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=i686-linux-gnu-
 ENV DEF_TARGET_LIST i386-softmmu,i386-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker
index 6e88777f76d..eeb77212d5d 100644
--- a/tests/docker/dockerfiles/debian-mips64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker
@@ -71,11 +71,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture mips64el && \
@@ -173,9 +173,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/mips64el-linux-gnuabi64 && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-gcc
 
-ENV ABI "mips64el-linux-gnuabi64"
-ENV MESON_OPTS "--cross-file=mips64el-linux-gnuabi64"
-ENV RUST_TARGET "mips64el-unknown-linux-gnuabi64"
+ENV ABI="mips64el-linux-gnuabi64"
+ENV MESON_OPTS="--cross-file=mips64el-linux-gnuabi64"
+ENV RUST_TARGET="mips64el-unknown-linux-gnuabi64"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mips64el-linux-gnuabi64-
 ENV DEF_TARGET_LIST mips64el-softmmu,mips64el-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/docker/dockerfiles/debian-mipsel-cross.docker
index 5f4e3fa9636..9b3dca44f15 100644
--- a/tests/docker/dockerfiles/debian-mipsel-cross.docker
+++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker
@@ -71,11 +71,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture mipsel && \
@@ -173,9 +173,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/mipsel-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc
 
-ENV ABI "mipsel-linux-gnu"
-ENV MESON_OPTS "--cross-file=mipsel-linux-gnu"
-ENV RUST_TARGET "mipsel-unknown-linux-gnu"
+ENV ABI="mipsel-linux-gnu"
+ENV MESON_OPTS="--cross-file=mipsel-linux-gnu"
+ENV RUST_TARGET="mipsel-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=mipsel-linux-gnu-
 ENV DEF_TARGET_LIST mipsel-softmmu,mipsel-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
index dfa690616d1..15c6b436d31 100644
--- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture ppc64el && \
@@ -176,9 +176,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/powerpc64le-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-gcc
 
-ENV ABI "powerpc64le-linux-gnu"
-ENV MESON_OPTS "--cross-file=powerpc64le-linux-gnu"
-ENV RUST_TARGET "powerpc64le-unknown-linux-gnu"
+ENV ABI="powerpc64le-linux-gnu"
+ENV MESON_OPTS="--cross-file=powerpc64le-linux-gnu"
+ENV RUST_TARGET="powerpc64le-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=powerpc64le-linux-gnu-
 ENV DEF_TARGET_LIST ppc64-softmmu,ppc64-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
index 09b2953f32c..7a395cc594c 100644
--- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture riscv64 && \
@@ -179,9 +179,9 @@ endian = 'little'\n" > /usr/local/share/meson/cross/riscv64-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/riscv64-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/riscv64-linux-gnu-gcc
 
-ENV ABI "riscv64-linux-gnu"
-ENV MESON_OPTS "--cross-file=riscv64-linux-gnu"
-ENV RUST_TARGET "riscv64gc-unknown-linux-gnu"
+ENV ABI="riscv64-linux-gnu"
+ENV MESON_OPTS="--cross-file=riscv64-linux-gnu"
+ENV RUST_TARGET="riscv64gc-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu-
 ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian-s390x-cross.docker b/tests/docker/dockerfiles/debian-s390x-cross.docker
index 09a78c15baf..57aa3209c5c 100644
--- a/tests/docker/dockerfiles/debian-s390x-cross.docker
+++ b/tests/docker/dockerfiles/debian-s390x-cross.docker
@@ -72,11 +72,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN export DEBIAN_FRONTEND=noninteractive && \
     dpkg --add-architecture s390x && \
@@ -175,9 +175,9 @@ endian = 'big'\n" > /usr/local/share/meson/cross/s390x-linux-gnu && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-cc && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/s390x-linux-gnu-gcc
 
-ENV ABI "s390x-linux-gnu"
-ENV MESON_OPTS "--cross-file=s390x-linux-gnu"
-ENV RUST_TARGET "s390x-unknown-linux-gnu"
+ENV ABI="s390x-linux-gnu"
+ENV MESON_OPTS="--cross-file=s390x-linux-gnu"
+ENV RUST_TARGET="s390x-unknown-linux-gnu"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=s390x-linux-gnu-
 ENV DEF_TARGET_LIST s390x-softmmu,s390x-linux-user
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/debian.docker b/tests/docker/dockerfiles/debian.docker
index 8dd893be4b0..bbb920c6815 100644
--- a/tests/docker/dockerfiles/debian.docker
+++ b/tests/docker/dockerfiles/debian.docker
@@ -152,11 +152,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 # netmap/cscope/global
 RUN DEBIAN_FRONTEND=noninteractive eatmydata \
   apt install -y --no-install-recommends \
diff --git a/tests/docker/dockerfiles/fedora-rust-nightly.docker b/tests/docker/dockerfiles/fedora-rust-nightly.docker
index 7d31c9f4060..e4233b53cbe 100644
--- a/tests/docker/dockerfiles/fedora-rust-nightly.docker
+++ b/tests/docker/dockerfiles/fedora-rust-nightly.docker
@@ -152,11 +152,11 @@ exec "$@"\n' > /usr/bin/nosync && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 RUN dnf install -y wget
 ENV RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo
 ENV RUSTC=/usr/local/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc
diff --git a/tests/docker/dockerfiles/fedora-win64-cross.docker b/tests/docker/dockerfiles/fedora-win64-cross.docker
index c76a70c3684..e6f84dd556f 100644
--- a/tests/docker/dockerfiles/fedora-win64-cross.docker
+++ b/tests/docker/dockerfiles/fedora-win64-cross.docker
@@ -72,11 +72,11 @@ exec "$@"\n' > /usr/bin/nosync && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 
 RUN nosync dnf install -y \
                mingw-w64-tools \
@@ -110,8 +110,8 @@ RUN nosync dnf install -y \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-g++ && \
     ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-gcc
 
-ENV ABI "x86_64-w64-mingw32"
-ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw64.meson"
+ENV ABI="x86_64-w64-mingw32"
+ENV MESON_OPTS="--cross-file=/usr/share/mingw/toolchain-mingw64.meson"
 ENV QEMU_CONFIGURE_OPTS --cross-prefix=x86_64-w64-mingw32-
 ENV DEF_TARGET_LIST x86_64-softmmu
 # As a final step configure the user (if env is defined)
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 891a740fcbc..632259378c9 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -152,11 +152,11 @@ exec "$@"\n' > /usr/bin/nosync && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 # As a final step configure the user (if env is defined)
 ARG USER
 ARG UID
diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
index 75e17477807..def0ca9db46 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -138,11 +138,11 @@ RUN /usr/bin/pip3.11 install \
                      sphinx \
                      sphinx-rtd-theme
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3.11"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3.11"
 # As a final step configure the user (if env is defined)
 ARG USER
 ARG UID
diff --git a/tests/docker/dockerfiles/ubuntu2204.docker b/tests/docker/dockerfiles/ubuntu2204.docker
index b393db55a8c..e9f059d0659 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
+++ b/tests/docker/dockerfiles/ubuntu2204.docker
@@ -150,11 +150,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 
 RUN /usr/bin/pip3 install meson==1.8.1
 
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
+ENV LANG="en_US.UTF-8"
+ENV MAKE="/usr/bin/make"
+ENV NINJA="/usr/bin/ninja"
+ENV PYTHON="/usr/bin/python3"
 ENV RUSTC=/usr/bin/rustc-1.77
 ENV RUSTDOC=/usr/bin/rustdoc-1.77
 ENV CARGO_HOME=/usr/local/cargo
diff --git a/tests/lcitool/libvirt-ci b/tests/lcitool/libvirt-ci
index 9da20ff7c3b..1b943062a29 160000
--- a/tests/lcitool/libvirt-ci
+++ b/tests/lcitool/libvirt-ci
@@ -1 +1 @@
-Subproject commit 9da20ff7c3bc9067804a7561c2ff87583b434853
+Subproject commit 1b943062a29cf56b5e6d7aedf865843d65ba69a7
-- 
2.47.3



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 2/5] gitlab: use template for ubuntu-24.04-aarch64 jobs
  2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
  2025-10-14 10:43 ` [PATCH 1/5] tests/lcitool: bump to latest version and regenerate (!wait on upstream) Alex Bennée
@ 2025-10-14 10:43 ` Alex Bennée
  2025-10-14 10:43 ` [PATCH 3/5] gitlab: modernise the ubuntu24.04-aarch32 jobs Alex Bennée
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 10:43 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Thomas Huth, qemu-arm, Gustavo Romero,
	Alex Bennée, Philippe Mathieu-Daudé, BALATON Zoltan,
	qemu-s390x

Most of the test is pure boilerplate so to save ourselves from
repetition move all the main bits into a minimal copy of
native_build_job_template but without the caching.

We keep all the current allow_fail and configure setups but do take
the opportunity to replace the -j`nproc --ignore=40` hack with
something that almost, but not quite, saturates the machine its being
built on.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .../custom-runners/ubuntu-24.04-aarch64.yml   | 230 ++++++++----------
 1 file changed, 96 insertions(+), 134 deletions(-)

diff --git a/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch64.yml b/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch64.yml
index d26c7827f45..46db9ae0138 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch64.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch64.yml
@@ -2,150 +2,112 @@
 # setup by the scripts/ci/setup/ubuntu/build-environment.yml task
 # "Install basic packages to build QEMU on Ubuntu 24.04"
 
+.ubuntu_aarch64_template:
+  extends: .custom_runner_template
+  needs: []
+  stage: build
+  tags:
+    - ubuntu_24.04
+    - aarch64
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+    - if: "$AARCH64_RUNNER_AVAILABLE"
+  before_script:
+    - source scripts/ci/gitlab-ci-section
+    - section_start setup "Pre-script setup"
+    - JOBS=$(expr $(nproc) - 4)
+    - section_end setup
+  script:
+    - mkdir build
+    - cd build
+    - section_start configure "Running configure"
+    - ../configure $CONFIGURE_ARGS ||
+          { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - section_end configure
+    - section_start build "Building QEMU"
+    - make --output-sync -j"$JOBS"
+    - section_end build
+    - section_start test "Running tests"
+    - if test -n "$MAKE_CHECK_ARGS";
+      then
+        make -j"$JOBS" $MAKE_CHECK_ARGS ;
+      fi
+    - section_end test
+
 ubuntu-24.04-aarch64-all-linux-static:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
- - if: "$AARCH64_RUNNER_AVAILABLE"
- script:
- - mkdir build
- - cd build
- # Disable -static-pie due to build error with system libc:
- # https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1987438
- - ../configure --enable-debug --static --disable-system --disable-pie
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
- - make check-tcg
- - make --output-sync -j`nproc --ignore=40` check
+  extends: .ubuntu_aarch64_template
+  variables:
+    # Disable -static-pie due to build error with system libc:
+    # https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1987438
+    CONFIGURE_ARGS: --enable-debug --static --disable-system --disable-pie
+    MAKE_CHECK_ARGS: check-tcg
 
 ubuntu-24.04-aarch64-all:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$AARCH64_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check
+  extends: .ubuntu_aarch64_template
+  variables:
+    MAKE_CHECK_ARGS: check
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+      when: manual
+      allow_failure: true
+    - if: "$AARCH64_RUNNER_AVAILABLE"
+      when: manual
+      allow_failure: true
 
 ubuntu-24.04-aarch64-without-defaults:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$AARCH64_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --disable-user --without-default-devices --without-default-features
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check
+  extends: .ubuntu_aarch64_template
+  variables:
+    CONFIGURE_ARGS: --disable-user --without-default-devices --without-default-features
+    MAKE_CHECK_ARGS: check
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+      when: manual
+      allow_failure: true
+    - if: "$AARCH64_RUNNER_AVAILABLE"
+      when: manual
+      allow_failure: true
 
 ubuntu-24.04-aarch64-alldbg:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
- - if: "$AARCH64_RUNNER_AVAILABLE"
- script:
- - mkdir build
- - cd build
- - ../configure --enable-debug
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make clean
- - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check
+  extends: .ubuntu_aarch64_template
+  variables:
+    CONFIGURE_ARGS: --enable-debug
+    MAKE_CHECK_ARGS: check-tcg
 
 ubuntu-24.04-aarch64-clang:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$AARCH64_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-ubsan
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check
+  extends: .ubuntu_aarch64_template
+  variables:
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-ubsan
+    MAKE_CHECK_ARGS: check
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+      when: manual
+      allow_failure: true
+    - if: "$AARCH64_RUNNER_AVAILABLE"
+      when: manual
+      allow_failure: true
 
 ubuntu-24.04-aarch64-tci:
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$AARCH64_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --enable-tcg-interpreter
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
+  extends: .ubuntu_aarch64_template
+  variables:
+    CONFIGURE_ARGS: --enable-tcg-interpreter
+    MAKE_CHECK_ARGS: check
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+      when: manual
+      allow_failure: true
+    - if: "$AARCH64_RUNNER_AVAILABLE"
+      when: manual
+      allow_failure: true
 
 ubuntu-24.04-aarch64-notcg:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - aarch64
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$AARCH64_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --disable-tcg --with-devices-aarch64=minimal
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check
+  extends: .ubuntu_aarch64_template
+  variables:
+    CONFIGURE_ARGS: --disable-tcg --with-devices-aarch64=minimal
+    MAKE_CHECK_ARGS: check
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+      when: manual
+      allow_failure: true
+    - if: "$AARCH64_RUNNER_AVAILABLE"
+      when: manual
+      allow_failure: true
-- 
2.47.3



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 3/5] gitlab: modernise the ubuntu24.04-aarch32 jobs
  2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
  2025-10-14 10:43 ` [PATCH 1/5] tests/lcitool: bump to latest version and regenerate (!wait on upstream) Alex Bennée
  2025-10-14 10:43 ` [PATCH 2/5] gitlab: use template for ubuntu-24.04-aarch64 jobs Alex Bennée
@ 2025-10-14 10:43 ` Alex Bennée
  2025-10-14 10:43 ` [PATCH 4/5] scripts/setup: update the armhf overlay Alex Bennée
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 10:43 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Thomas Huth, qemu-arm, Gustavo Romero,
	Alex Bennée, Philippe Mathieu-Daudé, BALATON Zoltan,
	qemu-s390x

We only have one job here so no point using a template but we might as
well update the JOBS calculation and use our fancy section markers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .../custom-runners/ubuntu-24.04-aarch32.yml       | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch32.yml b/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch32.yml
index 75029c9187e..3e7fdc8fa98 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch32.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-24.04-aarch32.yml
@@ -16,10 +16,21 @@ ubuntu-24.04-aarch32-all:
  - if: "$AARCH32_RUNNER_AVAILABLE"
    when: manual
    allow_failure: true
+ before_script:
+   - source scripts/ci/gitlab-ci-section
+   - section_start setup "Pre-script setup"
+   - JOBS=$(expr $(nproc) - 4)
+   - section_end setup
  script:
  - mkdir build
  - cd build
+ - section_start configure "Running configure"
  - ../configure --cross-prefix=arm-linux-gnueabihf-
    || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc --ignore=40`
- - make --output-sync -j`nproc --ignore=40` check
+ - section_end configure
+ - section_start build "Building QEMU"
+ - make --output-sync -j"$JOBS"
+ - section_end build
+ - section_start test "Running tests"
+ - make --output-sync -j"$JOBS" check
+ - section_end test
-- 
2.47.3



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 4/5] scripts/setup: update the armhf overlay
  2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
                   ` (2 preceding siblings ...)
  2025-10-14 10:43 ` [PATCH 3/5] gitlab: modernise the ubuntu24.04-aarch32 jobs Alex Bennée
@ 2025-10-14 10:43 ` Alex Bennée
  2025-10-14 10:43 ` [PATCH 5/5] scripts/setup: update build-environment to use armhf yaml (!WIP) Alex Bennée
  2025-10-14 10:47 ` [PATCH 0/5] testing/next - arm custom runner tweaks Thomas Huth
  5 siblings, 0 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 10:43 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Thomas Huth, qemu-arm, Gustavo Romero,
	Alex Bennée, Philippe Mathieu-Daudé, BALATON Zoltan,
	qemu-s390x

We certainly didn't generate the armhf list via lcitool as it couldn't
deal with ubuntu or foreign layers. Now we can properly generate the
list.

Fixes: 1417704564a (tests/lcitool: generate package lists for ansible)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 ...armhf-cross.yml => ubuntu-2404-armhf.yaml} | 65 +++----------------
 tests/lcitool/refresh                         | 11 ++++
 2 files changed, 20 insertions(+), 56 deletions(-)
 rename scripts/ci/setup/ubuntu/{ubuntu-2204-armhf-cross.yml => ubuntu-2404-armhf.yaml} (65%)

diff --git a/scripts/ci/setup/ubuntu/ubuntu-2204-armhf-cross.yml b/scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml
similarity index 65%
rename from scripts/ci/setup/ubuntu/ubuntu-2204-armhf-cross.yml
rename to scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml
index 0cc34cd10b9..f06f95514bd 100644
--- a/scripts/ci/setup/ubuntu/ubuntu-2204-armhf-cross.yml
+++ b/scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml
@@ -1,64 +1,13 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool variables --cross-arch armv7l ubuntu-2204 qemu
+#  $ lcitool variables --host-arch aarch64 --cross-arch armv7l ubuntu-2404 qemu
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
 packages:
-  - bash
-  - bc
-  - bison
-  - bsdextrautils
-  - bzip2
-  - ca-certificates
-  - ccache
-  - dbus
-  - debianutils
-  - diffutils
-  - exuberant-ctags
-  - findutils
-  - flex
-  - gcc
-  - gcovr
-  - gettext
-  - git
-  - hostname
-  - libglib2.0-dev
-  - libpcre2-dev
-  - libsndio-dev
-  - libspice-protocol-dev
-  - llvm
-  - locales
-  - make
-  - meson
-  - mtools
-  - ncat
-  - ninja-build
-  - openssh-client
-  - pkgconf
-  - python3
-  - python3-numpy
-  - python3-opencv
-  - python3-pillow
-  - python3-pip
-  - python3-sphinx
-  - python3-sphinx-rtd-theme
-  - python3-tomli
-  - python3-venv
-  - python3-yaml
-  - rpm2cpio
-  - sed
-  - socat
-  - sparse
-  - swtpm
-  - tar
-  - tesseract-ocr
-  - tesseract-ocr-eng
-  - xorriso
-  - zstd
   - gcc-arm-linux-gnueabihf
   - libaio-dev:armhf
-  - libasan6:armhf
+  - libasan8:armhf
   - libasound2-dev:armhf
   - libattr1-dev:armhf
   - libbpf-dev:armhf
@@ -68,6 +17,7 @@ packages:
   - libcacard-dev:armhf
   - libcap-ng-dev:armhf
   - libcapstone-dev:armhf
+  - libcbor-dev:armhf
   - libcmocka-dev:armhf
   - libcurl4-gnutls-dev:armhf
   - libdaxctl-dev:armhf
@@ -82,7 +32,7 @@ packages:
   - libglusterfs-dev:armhf
   - libgnutls28-dev:armhf
   - libgtk-3-dev:armhf
-  - libibumad-dev:armhf
+  - libgtk-vnc-2.0-dev:armhf
   - libibverbs-dev:armhf
   - libiscsi-dev:armhf
   - libjemalloc-dev:armhf
@@ -94,11 +44,11 @@ packages:
   - libnfs-dev:armhf
   - libnuma-dev:armhf
   - libpam0g-dev:armhf
+  - libpcre2-dev:armhf
   - libpipewire-0.3-dev:armhf
   - libpixman-1-dev:armhf
   - libpng-dev:armhf
   - libpulse-dev:armhf
-  - librbd-dev:armhf
   - librdmacm-dev:armhf
   - libsasl2-dev:armhf
   - libsdl2-dev:armhf
@@ -107,8 +57,11 @@ packages:
   - libselinux1-dev:armhf
   - libslirp-dev:armhf
   - libsnappy-dev:armhf
+  - libsndio-dev:armhf
+  - libspice-protocol-dev:armhf
   - libspice-server-dev:armhf
   - libssh-dev:armhf
+  - libstd-rust-dev:armhf
   - libsystemd-dev:armhf
   - libtasn1-6-dev:armhf
   - libubsan1:armhf
@@ -119,7 +72,7 @@ packages:
   - libvdeplug-dev:armhf
   - libvirglrenderer-dev:armhf
   - libvte-2.91-dev:armhf
-  - libxen-dev:armhf
+  - libxdp-dev:armhf
   - libzstd-dev:armhf
   - nettle-dev:armhf
   - systemtap-sdt-dev:armhf
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 056cfb6e9d7..8ad45da19c1 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -99,6 +99,16 @@ def generate_yaml(os, target, arch, trailer=None):
                          arch, target, "qemu"]
     generate(filename, cmd, trailer)
 
+# an overlay file only cares about the additional packages for cross
+# so we can skip all the host packages needed.
+def generate_cross_overlay_yaml(os, target, arch, name, cross, trailer=None):
+    filename = Path(src_dir, "scripts", "ci", "setup", os, f"{target}-{name}.yaml")
+    cmd = lcitool_cmd + ["variables", "--format", "yaml",
+                         "--layer", "foreign",
+                         "-a", arch, "--cross", cross,
+                         target, "qemu"]
+    generate(filename, cmd, trailer)
+
 
 alpine_extras = [
     "# https://gitlab.alpinelinux.org/alpine/aports/-/issues/17463\n",
@@ -271,6 +281,7 @@ try:
     # Ansible package lists
     #
     generate_yaml("ubuntu", "ubuntu-2404", "aarch64")
+    generate_cross_overlay_yaml("ubuntu", "ubuntu-2404", "aarch64", "armhf", cross="armv7l")
     generate_yaml("ubuntu", "ubuntu-2404", "s390x")
 
 
-- 
2.47.3



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH 5/5] scripts/setup: update build-environment to use armhf yaml (!WIP)
  2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
                   ` (3 preceding siblings ...)
  2025-10-14 10:43 ` [PATCH 4/5] scripts/setup: update the armhf overlay Alex Bennée
@ 2025-10-14 10:43 ` Alex Bennée
  2025-10-14 10:47 ` [PATCH 0/5] testing/next - arm custom runner tweaks Thomas Huth
  5 siblings, 0 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 10:43 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Thomas Huth, qemu-arm, Gustavo Romero,
	Alex Bennée, Philippe Mathieu-Daudé, BALATON Zoltan,
	qemu-s390x

Now we have properly updated our yaml to contain just the cross
packages lets injest that in the build-environment script.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
TODO
  - currently I hand hack some packages that won't install out.
---
 scripts/ci/setup/ubuntu/build-environment.yml | 22 +++++++++----------
 .../ci/setup/ubuntu/ubuntu-2404-armhf.yaml    |  2 --
 2 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/scripts/ci/setup/ubuntu/build-environment.yml b/scripts/ci/setup/ubuntu/build-environment.yml
index 6042750cb4d..f84157dc655 100644
--- a/scripts/ci/setup/ubuntu/build-environment.yml
+++ b/scripts/ci/setup/ubuntu/build-environment.yml
@@ -47,19 +47,19 @@
         - ansible_facts['distribution'] == 'Ubuntu'
         - ansible_facts['distribution_version'] == '24.04'
 
+    # the package lists are updated by "make lcitool-refresh"
+    - name: Include the armhf package lists based on OS and architecture
+      include_vars:
+        file: "ubuntu-2404-armhf.yaml"
+        name: armhf_packages
+      when:
+        - ansible_facts['distribution'] == 'Ubuntu'
+        - ansible_facts['distribution_version'] == '24.04'
+        - ansible_facts['architecture'] == 'aarch64'
+
     - name: Install armhf cross-compile packages to build QEMU on AArch64 Ubuntu 24.04
       package:
-        name:
-          - binutils-arm-linux-gnueabihf
-          - gcc-arm-linux-gnueabihf
-          - libblkid-dev:armhf
-          - libc6-dev:armhf
-          - libffi-dev:armhf
-          - libglib2.0-dev:armhf
-          - libmount-dev:armhf
-          - libpcre2-dev:armhf
-          - libpixman-1-dev:armhf
-          - zlib1g-dev:armhf
+        name: "{{ armhf_packages.packages }}"
       when:
         - ansible_facts['distribution'] == 'Ubuntu'
         - ansible_facts['distribution_version'] == '24.04'
diff --git a/scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml b/scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml
index f06f95514bd..e361357a0ea 100644
--- a/scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml
+++ b/scripts/ci/setup/ubuntu/ubuntu-2404-armhf.yaml
@@ -19,7 +19,6 @@ packages:
   - libcapstone-dev:armhf
   - libcbor-dev:armhf
   - libcmocka-dev:armhf
-  - libcurl4-gnutls-dev:armhf
   - libdaxctl-dev:armhf
   - libdrm-dev:armhf
   - libepoxy-dev:armhf
@@ -29,7 +28,6 @@ packages:
   - libgbm-dev:armhf
   - libgcrypt20-dev:armhf
   - libglib2.0-dev:armhf
-  - libglusterfs-dev:armhf
   - libgnutls28-dev:armhf
   - libgtk-3-dev:armhf
   - libgtk-vnc-2.0-dev:armhf
-- 
2.47.3



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/5] testing/next - arm custom runner tweaks
  2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
                   ` (4 preceding siblings ...)
  2025-10-14 10:43 ` [PATCH 5/5] scripts/setup: update build-environment to use armhf yaml (!WIP) Alex Bennée
@ 2025-10-14 10:47 ` Thomas Huth
  2025-10-14 17:29   ` Alex Bennée
  5 siblings, 1 reply; 10+ messages in thread
From: Thomas Huth @ 2025-10-14 10:47 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: Peter Maydell, qemu-arm, Gustavo Romero,
	Philippe Mathieu-Daudé, BALATON Zoltan, qemu-s390x

On 14/10/2025 12.43, Alex Bennée wrote:
> This isn't quite ready yet as there is an in-flight MR for libvirt-ci
> and the final patch has to manually drop a couple of packages which
> don't cleanly install. I don't know if we should deal with that
> upstream or in the lcitool metadata. For reference the two packages
> are:
> 
>    libcurl4-gnutls-dev:armhf
>    libglusterfs-dev:armhf

FYI, glusterfs is deprecated in QEMU since a while:

  https://gitlab.com/qemu-project/qemu/-/commit/b873463821343c6f702c4195f2168790b09cf44e

... so you could also add a patch that removes it from QEMU now, I guess.

  Thomas



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/5] testing/next - arm custom runner tweaks
  2025-10-14 10:47 ` [PATCH 0/5] testing/next - arm custom runner tweaks Thomas Huth
@ 2025-10-14 17:29   ` Alex Bennée
  2025-10-14 17:53     ` Richard Henderson
  2025-10-14 18:19     ` Thomas Huth
  0 siblings, 2 replies; 10+ messages in thread
From: Alex Bennée @ 2025-10-14 17:29 UTC (permalink / raw)
  To: Thomas Huth
  Cc: qemu-devel, Peter Maydell, qemu-arm, Gustavo Romero,
	Philippe Mathieu-Daudé, BALATON Zoltan, qemu-s390x

Thomas Huth <thuth@redhat.com> writes:

> On 14/10/2025 12.43, Alex Bennée wrote:
>> This isn't quite ready yet as there is an in-flight MR for libvirt-ci
>> and the final patch has to manually drop a couple of packages which
>> don't cleanly install. I don't know if we should deal with that
>> upstream or in the lcitool metadata. For reference the two packages
>> are:
>>    libcurl4-gnutls-dev:armhf
>>    libglusterfs-dev:armhf
>
> FYI, glusterfs is deprecated in QEMU since a while:
>
>  https://gitlab.com/qemu-project/qemu/-/commit/b873463821343c6f702c4195f2168790b09cf44e
>
> ... so you could also add a patch that removes it from QEMU now, I
> guess.

I can do that.


Richard and Phillipe reminded me that we are due to drop 32 bit hosts
real soon now so maybe I should just remove the runner and the hacks
needed to get it working?

>
>  Thomas

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/5] testing/next - arm custom runner tweaks
  2025-10-14 17:29   ` Alex Bennée
@ 2025-10-14 17:53     ` Richard Henderson
  2025-10-14 18:19     ` Thomas Huth
  1 sibling, 0 replies; 10+ messages in thread
From: Richard Henderson @ 2025-10-14 17:53 UTC (permalink / raw)
  To: qemu-devel

On 10/14/25 10:29, Alex Bennée wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> On 14/10/2025 12.43, Alex Bennée wrote:
>>> This isn't quite ready yet as there is an in-flight MR for libvirt-ci
>>> and the final patch has to manually drop a couple of packages which
>>> don't cleanly install. I don't know if we should deal with that
>>> upstream or in the lcitool metadata. For reference the two packages
>>> are:
>>>     libcurl4-gnutls-dev:armhf
>>>     libglusterfs-dev:armhf
>>
>> FYI, glusterfs is deprecated in QEMU since a while:
>>
>>   https://gitlab.com/qemu-project/qemu/-/commit/b873463821343c6f702c4195f2168790b09cf44e
>>
>> ... so you could also add a patch that removes it from QEMU now, I
>> guess.
> 
> I can do that.
> 
> 
> Richard and Phillipe reminded me that we are due to drop 32 bit hosts
> real soon now so maybe I should just remove the runner and the hacks
> needed to get it working?

I think that's for the best.


r~


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH 0/5] testing/next - arm custom runner tweaks
  2025-10-14 17:29   ` Alex Bennée
  2025-10-14 17:53     ` Richard Henderson
@ 2025-10-14 18:19     ` Thomas Huth
  1 sibling, 0 replies; 10+ messages in thread
From: Thomas Huth @ 2025-10-14 18:19 UTC (permalink / raw)
  To: Alex Bennée
  Cc: qemu-devel, Peter Maydell, qemu-arm, Gustavo Romero,
	Philippe Mathieu-Daudé, BALATON Zoltan, qemu-s390x,
	Liviu Ionescu

On 14/10/2025 19.29, Alex Bennée wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> On 14/10/2025 12.43, Alex Bennée wrote:
>>> This isn't quite ready yet as there is an in-flight MR for libvirt-ci
>>> and the final patch has to manually drop a couple of packages which
>>> don't cleanly install. I don't know if we should deal with that
>>> upstream or in the lcitool metadata. For reference the two packages
>>> are:
>>>     libcurl4-gnutls-dev:armhf
>>>     libglusterfs-dev:armhf
>>
>> FYI, glusterfs is deprecated in QEMU since a while:
>>
>>   https://gitlab.com/qemu-project/qemu/-/commit/b873463821343c6f702c4195f2168790b09cf44e
>>
>> ... so you could also add a patch that removes it from QEMU now, I
>> guess.
> 
> I can do that.
> 
> 
> Richard and Phillipe reminded me that we are due to drop 32 bit hosts
> real soon now so maybe I should just remove the runner and the hacks
> needed to get it working?

Let see ... the last time I tried to deprecate 32-bit arm, I've been told 
that the Raspberry Pis still use a 32-bit OS by default:

  https://lore.kernel.org/qemu-devel/F852C238-77B8-4E24-9494-8D060EB78F9F@livius.net/

... and the right time to drop 32-bit arm is when most Pi users install a 
64-bit OS by default:

  https://lore.kernel.org/qemu-devel/379B4C23-61C2-4AA3-A851-EEFC98A9AFF6@livius.net/

So looking at https://www.raspberrypi.com/software/operating-systems/ now, 
it seems like the 64-bit version is the default nowadays (at least it is the 
first option there, and no more recommendation for the 32-bit version), so 
yes, it's likely about time to get rid of the 32-bit arm hosts now!

  Thomas



^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2025-10-14 18:20 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-14 10:43 [PATCH 0/5] testing/next - arm custom runner tweaks Alex Bennée
2025-10-14 10:43 ` [PATCH 1/5] tests/lcitool: bump to latest version and regenerate (!wait on upstream) Alex Bennée
2025-10-14 10:43 ` [PATCH 2/5] gitlab: use template for ubuntu-24.04-aarch64 jobs Alex Bennée
2025-10-14 10:43 ` [PATCH 3/5] gitlab: modernise the ubuntu24.04-aarch32 jobs Alex Bennée
2025-10-14 10:43 ` [PATCH 4/5] scripts/setup: update the armhf overlay Alex Bennée
2025-10-14 10:43 ` [PATCH 5/5] scripts/setup: update build-environment to use armhf yaml (!WIP) Alex Bennée
2025-10-14 10:47 ` [PATCH 0/5] testing/next - arm custom runner tweaks Thomas Huth
2025-10-14 17:29   ` Alex Bennée
2025-10-14 17:53     ` Richard Henderson
2025-10-14 18:19     ` Thomas Huth

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).