[XEN PATCH v2 0/6] automation: Refresh the remaining Debian containers

[XEN PATCH v2 0/6] automation: Refresh the remaining Debian containers

[Javi Merino]

This is v2 of [0], to refresh the remaining Debian 12 containers. The
containers now use heredocs for readability, and use
"apt-get --no-install-recommends" to minimize the installation of
unnecessary packages.  All containers except Debian 12 arm64 and
x86_64 run as a normal user.  Debian 12 arm64 and x86_64 have been
kept running as root as Stefano asked for it as it would break the
xilinx workers that are currently in flux.

The updated containers have been tested in
https://gitlab.com/xen-project/people/javimerino/xen/-/pipelines/1526520801

Changes since v1[0]:
  - In containerfiles, repeat the "LABEL" intruction for each label and
    avoid the backslash at the end of lines.
  - Use ENV VAR=value consistently in dockerfiles.
  - Dropped the patch that moved the Debian 12 arm64 and x86_64
    containers to run as a normal user.
  - Added a patch to drop the "-y" in "apt-get update"
  - Move libnl-3-dev to the libxl section in the Debian 12 x86_64
    dockerfile.
  - Add missing requires for libxenguest dombuilder in the Debian 12
    x86_64 dockerfile
  - Move the expect dependency to the section for tests

[0] https://lore.kernel.org/xen-devel/cover.1729760501.git.javi.merino@cloud.com

Javi Merino (6):
  CI: Refresh the Debian 12 x86_64 container
  CI: Refresh the Debian 12 x86_32 container
  CI: Refresh the Debian 12 arm64 container
  CI: Refresh the Debian 12 arm32 cross compile container
  CI: Refresh the Debian 12 cppcheck container
  CI: Don't use -y with apt-get update

 automation/build/debian/11-ppc64le.dockerfile |  2 +-
 automation/build/debian/11-riscv64.dockerfile |  2 +-
 .../debian/12-arm64v8-arm32-gcc.dockerfile    | 28 +++++++
 .../debian/12-arm64v8-cppcheck.dockerfile     | 79 +++++++++++++++++++
 automation/build/debian/12-arm64v8.dockerfile | 68 ++++++++++++++++
 automation/build/debian/12-ppc64le.dockerfile |  2 +-
 automation/build/debian/12-riscv64.dockerfile |  2 +-
 automation/build/debian/12-x86_32.dockerfile  | 50 ++++++++++++
 .../build/debian/12-x86_64-gcc-ibt.dockerfile |  4 +-
 automation/build/debian/12-x86_64.dockerfile  | 71 +++++++++++++++++
 .../bookworm-arm64v8-arm32-gcc.dockerfile     | 24 ------
 .../build/debian/bookworm-arm64v8.dockerfile  | 55 -------------
 .../build/debian/bookworm-cppcheck.dockerfile | 54 -------------
 .../build/debian/bookworm-i386.dockerfile     | 50 ------------
 automation/build/debian/bookworm.dockerfile   | 57 -------------
 .../build/ubuntu/16.04-x86_64.dockerfile      |  2 +-
 .../build/ubuntu/18.04-x86_64.dockerfile      |  2 +-
 .../build/ubuntu/20.04-x86_64.dockerfile      |  2 +-
 .../build/ubuntu/22.04-x86_64.dockerfile      |  2 +-
 .../build/ubuntu/24.04-x86_64.dockerfile      |  2 +-
 automation/gitlab-ci/build.yaml               | 68 ++++++++--------
 automation/gitlab-ci/test.yaml                | 32 ++++----
 automation/scripts/containerize               | 10 +--
 23 files changed, 362 insertions(+), 306 deletions(-)
 create mode 100644 automation/build/debian/12-arm64v8-arm32-gcc.dockerfile
 create mode 100644 automation/build/debian/12-arm64v8-cppcheck.dockerfile
 create mode 100644 automation/build/debian/12-arm64v8.dockerfile
 create mode 100644 automation/build/debian/12-x86_32.dockerfile
 create mode 100644 automation/build/debian/12-x86_64.dockerfile
 delete mode 100644 automation/build/debian/bookworm-arm64v8-arm32-gcc.dockerfile
 delete mode 100644 automation/build/debian/bookworm-arm64v8.dockerfile
 delete mode 100644 automation/build/debian/bookworm-cppcheck.dockerfile
 delete mode 100644 automation/build/debian/bookworm-i386.dockerfile
 delete mode 100644 automation/build/debian/bookworm.dockerfile

-- 
2.45.2

[XEN PATCH v2 1/6] CI: Refresh the Debian 12 x86_64 container

[Javi Merino]

Rework the container to use heredocs for readability, and use
apt-get --no-install-recommends to keep the size down.

This reduces the size of the (uncompressed) container from 3.44GB to
1.97GB.

The container is left running the builds and tests as root to avoid
breaking the xilinx runners.

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---

Changes in dependencies:

$ diff -u <(git show origin/staging:automation/build/debian/bookworm.dockerfile | awk '/&&/{f=0};f;/apt-get \-\-quiet/{f=1};' | perl -ne 's/ \\$//;s/^ +//; /^#/ or print' | sort) <(awk '/^ +\)/{f=0};f;/DEPS=\(/{f=1}' automation/build/debian/12-x86_64.dockerfile | perl -ne 's/^\s+//; /^#/ or print' | sort)          
--- /proc/self/fd/16    2024-11-05 14:06:10.160095997 +0000
+++ /proc/self/fd/17    2024-11-05 14:06:10.160095997 +0000
@@ -1,34 +1,29 @@
 acpica-tools
-apt-transport-https
 bcc
 bin86
 bison
 build-essential
 busybox-static
+ca-certificates
 checkpolicy
 clang
 cpio
 expect
 flex
-git
-gnupg
-golang
-libaio-dev
-libfindlib-ocaml-dev
-libglib2.0-dev
+git-core
+golang-go
+libbz2-dev
 liblzma-dev
+liblzo2-dev
 libncurses5-dev
 libnl-3-dev
-libpixman-1-dev
 libyajl-dev
-markdown
-nasm
+libzstd-dev
+ocaml-findlib
 ocaml-nox
 ovmf
-pandoc
 pkg-config
 python3-dev
 python3-setuptools
 qemu-system-x86
-transfig
 uuid-dev

 automation/build/debian/12-x86_64.dockerfile | 71 ++++++++++++++++++++
 automation/build/debian/bookworm.dockerfile  | 57 ----------------
 automation/gitlab-ci/build.yaml              | 20 +++---
 automation/gitlab-ci/test.yaml               | 14 ++--
 automation/scripts/containerize              |  2 +-
 5 files changed, 89 insertions(+), 75 deletions(-)
 create mode 100644 automation/build/debian/12-x86_64.dockerfile
 delete mode 100644 automation/build/debian/bookworm.dockerfile

diff --git a/automation/build/debian/12-x86_64.dockerfile b/automation/build/debian/12-x86_64.dockerfile
new file mode 100644
index 000000000000..c440748f2336
--- /dev/null
+++ b/automation/build/debian/12-x86_64.dockerfile
@@ -0,0 +1,71 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/amd64 debian:bookworm
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# build depends
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    apt-get update
+    DEPS=(
+        # Xen
+        bison
+        build-essential
+        checkpolicy
+        clang
+        flex
+
+        # Tools (general)
+        ca-certificates
+        git-core
+        pkg-config
+        wget
+        # libxenguest dombuilder
+        libbz2-dev
+        liblzma-dev
+        liblzo2-dev
+        libzstd-dev
+        zlib1g-dev
+        # libacpi
+        acpica-tools
+        # libxl
+        uuid-dev
+        libnl-3-dev
+        libyajl-dev
+        # RomBIOS
+        bcc
+        bin86
+        # xentop
+        libncurses5-dev
+        # Python bindings
+        python3-dev
+        python3-setuptools
+        # Golang bindings
+        golang-go
+        # Ocaml bindings/oxenstored
+        ocaml-nox
+        ocaml-findlib
+
+        # for test phase, qemu-smoke-* jobs
+        expect
+        qemu-system-x86
+
+        # for qemu-alpine-x86_64-gcc
+        busybox-static
+        cpio
+
+        # For *-efi jobs
+        ovmf
+    )
+
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+
+    rm -rf /var/lib/apt/lists*
+EOF
+
+USER root
+WORKDIR /build
diff --git a/automation/build/debian/bookworm.dockerfile b/automation/build/debian/bookworm.dockerfile
deleted file mode 100644
index 72e01aa58b55..000000000000
--- a/automation/build/debian/bookworm.dockerfile
+++ /dev/null
@@ -1,57 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/amd64 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        zlib1g-dev \
-        libncurses5-dev \
-        python3-dev \
-        python3-setuptools \
-        uuid-dev \
-        libyajl-dev \
-        libaio-dev \
-        libglib2.0-dev \
-        clang \
-        libpixman-1-dev \
-        pkg-config \
-        flex \
-        bison \
-        acpica-tools \
-        bin86 \
-        bcc \
-        liblzma-dev \
-        libnl-3-dev \
-        ocaml-nox \
-        libfindlib-ocaml-dev \
-        markdown \
-        transfig \
-        pandoc \
-        checkpolicy \
-        wget \
-        git \
-        nasm \
-        gnupg \
-        apt-transport-https \
-        golang \
-        # for test phase, qemu-smoke-* jobs
-        qemu-system-x86 \
-        expect \
-        # For *-efi jobs
-        ovmf \
-        # for test phase, qemu-alpine-* jobs
-        cpio \
-        busybox-static \
-        && \
-        apt-get autoremove -y && \
-        apt-get clean && \
-        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index af2b1ceba3a4..d64a7e40f3f1 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -345,15 +345,15 @@ alpine-3.18-gcc-debug:
       CONFIG_UNSUPPORTED=y
       CONFIG_ARGO=y
 
-debian-bookworm-gcc-debug:
+debian-12-x86_64-gcc-debug:
   extends: .gcc-x86-64-build-debug
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
 
-debian-bookworm-clang-debug:
+debian-12-x86_64-clang-debug:
   extends: .clang-x86-64-build-debug
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
 
 debian-12-ppc64le-gcc-debug:
   extends: .gcc-ppc64le-cross-build-debug
@@ -557,20 +557,20 @@ debian-12-x86_64-gcc-ibt:
     EXTRA_FIXED_RANDCONFIG: |
       CONFIG_XEN_IBT=y
 
-debian-bookworm-clang:
+debian-12-x86_64-clang:
   extends: .clang-x86-64-build
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
 
-debian-bookworm-gcc:
+debian-12-x86_64-gcc:
   extends: .gcc-x86-64-build
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
 
-debian-bookworm-gcc-randconfig:
+debian-12-x86_64-gcc-randconfig:
   extends: .gcc-x86-64-build
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
     RANDCONFIG: y
 
 debian-bookworm-32-clang-debug:
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index e8f57e87bd19..5b89cfa33cf8 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -43,7 +43,7 @@
 .qemu-x86-64:
   extends: .test-jobs-common
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
     LOGFILE: qemu-smoke-x86-64.log
   artifacts:
     paths:
@@ -155,7 +155,7 @@
 build-each-commit-gcc:
   extends: .test-jobs-common
   variables:
-    CONTAINER: debian:bookworm
+    CONTAINER: debian:12-x86_64
     XEN_TARGET_ARCH: x86_64
     CC: gcc
   script:
@@ -477,35 +477,35 @@ qemu-smoke-x86-64-gcc:
   script:
     - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
   needs:
-    - debian-bookworm-gcc-debug
+    - debian-12-x86_64-gcc-debug
 
 qemu-smoke-x86-64-clang:
   extends: .qemu-smoke-x86-64
   script:
     - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
   needs:
-    - debian-bookworm-clang-debug
+    - debian-12-x86_64-clang-debug
 
 qemu-smoke-x86-64-gcc-pvh:
   extends: .qemu-smoke-x86-64
   script:
     - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
   needs:
-    - debian-bookworm-gcc-debug
+    - debian-12-x86_64-gcc-debug
 
 qemu-smoke-x86-64-clang-pvh:
   extends: .qemu-smoke-x86-64
   script:
     - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
   needs:
-    - debian-bookworm-clang-debug
+    - debian-12-x86_64-clang-debug
 
 qemu-smoke-x86-64-gcc-efi:
   extends: .qemu-smoke-x86-64
   script:
     - ./automation/scripts/qemu-smoke-x86-64-efi.sh pv 2>&1 | tee ${LOGFILE}
   needs:
-    - debian-bookworm-gcc-debug
+    - debian-12-x86_64-gcc-debug
 
 qemu-smoke-riscv64-gcc:
   extends: .qemu-riscv64
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index 6ac02c42d124..ea6e1a9b18f4 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -34,7 +34,7 @@ case "_${CONTAINER}" in
     _bullseye-riscv64) CONTAINER="${BASE}/debian:11-riscv64" ;;
     _bookworm-riscv64) CONTAINER="${BASE}/debian:12-riscv64" ;;
     _bookworm-x86_64-gcc-ibt) CONTAINER="${BASE}/debian:12-x86_64-gcc-ibt" ;;
-    _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;;
+    _bookworm|_bookworm-x86_64|_) CONTAINER="${BASE}/debian:12-x86_64" ;;
     _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;;
     _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;;
     _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;;
-- 
2.45.2

[XEN PATCH v2 2/6] CI: Refresh the Debian 12 x86_32 container

[Javi Merino]

Rework the container to be non-root, use heredocs for readability, and
use apt-get --no-install-recommends to keep the size down.  Rename the
job to x86_32, to be consistent with XEN_TARGET_ARCH and the
naming scheme of all the other CI jobs:
${VERSION}-${ARCH}-${BUILD_NAME}

Remove build dependencies for building QEMU, as we don't do it since
e305256e69b1 ("CI: Stop building QEMU in general").

Remove build dependencies for the documentation as we don't have to
build it for every single arch.

This reduces the size of the container from 2.22GB to 1.32Gb.

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---
 automation/build/debian/12-x86_32.dockerfile  | 50 +++++++++++++++++++
 .../build/debian/bookworm-i386.dockerfile     | 50 -------------------
 automation/gitlab-ci/build.yaml               |  8 +--
 automation/scripts/containerize               |  2 +-
 4 files changed, 55 insertions(+), 55 deletions(-)
 create mode 100644 automation/build/debian/12-x86_32.dockerfile
 delete mode 100644 automation/build/debian/bookworm-i386.dockerfile

diff --git a/automation/build/debian/12-x86_32.dockerfile b/automation/build/debian/12-x86_32.dockerfile
new file mode 100644
index 000000000000..b1cabf8d2f6d
--- /dev/null
+++ b/automation/build/debian/12-x86_32.dockerfile
@@ -0,0 +1,50 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/i386 debian:bookworm
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# build depends
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    useradd --create-home user
+
+    apt-get update
+    DEPS=(
+        # Xen
+        bison
+        build-essential
+        checkpolicy
+        clang
+        flex
+
+        # Tools (general)
+        ca-certificates
+        git-core
+        pkg-config
+        wget
+        # libacpi
+        acpica-tools
+        # libxl
+        uuid-dev
+        libyajl-dev
+        # xentop
+        libncurses5-dev
+        # Python bindings
+        python3-dev
+        python3-setuptools
+        # Ocaml bindings/oxenstored
+        ocaml-nox
+        ocaml-findlib
+    )
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+
+    rm -rf /var/lib/apt/lists*
+EOF
+
+USER user
+WORKDIR /build
+ENTRYPOINT ["linux32"]
diff --git a/automation/build/debian/bookworm-i386.dockerfile b/automation/build/debian/bookworm-i386.dockerfile
deleted file mode 100644
index 66fa3121c54e..000000000000
--- a/automation/build/debian/bookworm-i386.dockerfile
+++ /dev/null
@@ -1,50 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/i386 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-ENTRYPOINT ["linux32"]
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        zlib1g-dev \
-        libncurses5-dev \
-        python3-dev \
-        python3-setuptools \
-        uuid-dev \
-        libyajl-dev \
-        libaio-dev \
-        libglib2.0-dev \
-        clang \
-        libpixman-1-dev \
-        pkg-config \
-        flex \
-        bison \
-        acpica-tools \
-        bin86 \
-        bcc \
-        liblzma-dev \
-        libc6-dev \
-        libnl-3-dev \
-        ocaml-nox \
-        libfindlib-ocaml-dev \
-        markdown \
-        transfig \
-        pandoc \
-        checkpolicy \
-        wget \
-        git \
-        nasm \
-        apt-transport-https \
-        && \
-        apt-get autoremove -y && \
-        apt-get clean && \
-        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index d64a7e40f3f1..871beb70e157 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -573,15 +573,15 @@ debian-12-x86_64-gcc-randconfig:
     CONTAINER: debian:12-x86_64
     RANDCONFIG: y
 
-debian-bookworm-32-clang-debug:
+debian-12-x86_32-clang-debug:
   extends: .clang-x86-32-build-debug
   variables:
-    CONTAINER: debian:bookworm-i386
+    CONTAINER: debian:12-x86_32
 
-debian-bookworm-32-gcc-debug:
+debian-12-x86_32-gcc-debug:
   extends: .gcc-x86-32-build-debug
   variables:
-    CONTAINER: debian:bookworm-i386
+    CONTAINER: debian:12-x86_32
 
 fedora-40-x86_64-gcc:
   extends: .gcc-x86-64-build
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index ea6e1a9b18f4..d72c22c103ff 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -35,7 +35,7 @@ case "_${CONTAINER}" in
     _bookworm-riscv64) CONTAINER="${BASE}/debian:12-riscv64" ;;
     _bookworm-x86_64-gcc-ibt) CONTAINER="${BASE}/debian:12-x86_64-gcc-ibt" ;;
     _bookworm|_bookworm-x86_64|_) CONTAINER="${BASE}/debian:12-x86_64" ;;
-    _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;;
+    _bookworm-i386|_bookworm-x86_32) CONTAINER="${BASE}/debian:12-x86_32" ;;
     _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;;
     _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;;
     _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;;
-- 
2.45.2

[XEN PATCH v2 3/6] CI: Refresh the Debian 12 arm64 container

[Javi Merino]

Rework the container to use heredocs for readability and use
apt-get --no-install-recommends to keep the size down.  Rename the job
to debian-12-arm64-* to follow the naming scheme of all the other CI
jobs.

This reduces the size of the debian:12-arm64v8 from 2.25GB down to 1.62GB.

The container is left running the builds and tests as root to avoid
breaking the xilinx runners.

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---
 automation/build/debian/12-arm64v8.dockerfile | 68 +++++++++++++++++++
 .../build/debian/bookworm-arm64v8.dockerfile  | 55 ---------------
 automation/gitlab-ci/build.yaml               |  8 +--
 automation/gitlab-ci/test.yaml                |  4 +-
 automation/scripts/containerize               |  2 +-
 5 files changed, 75 insertions(+), 62 deletions(-)
 create mode 100644 automation/build/debian/12-arm64v8.dockerfile
 delete mode 100644 automation/build/debian/bookworm-arm64v8.dockerfile

diff --git a/automation/build/debian/12-arm64v8.dockerfile b/automation/build/debian/12-arm64v8.dockerfile
new file mode 100644
index 000000000000..fa53eb174862
--- /dev/null
+++ b/automation/build/debian/12-arm64v8.dockerfile
@@ -0,0 +1,68 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/arm64/v8 debian:bookworm
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# build depends
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    apt-get update
+    DEPS=(
+        # Xen
+        bison
+        build-essential
+        flex
+        # Flask
+        checkpolicy
+
+        # Tools (general)
+        git-core
+        libext2fs-dev
+        libfdt-dev
+        libglib2.0-dev
+        libpixman-1-dev
+        pkg-config
+        wget
+        # libxenguest dombuilder
+        libbz2-dev
+        liblzma-dev
+        liblzo2-dev
+        libzstd-dev
+        # libacpi
+        acpica-tools
+        # libxl
+        uuid-dev
+        libyajl-dev
+        # Python bindings
+        python3-dev
+        python3-setuptools
+        # Golang bindings
+        golang-go
+        # Ocaml bindings/oxenstored
+        ocaml-nox
+        ocaml-findlib
+
+        # for test phase, qemu-smoke-* jobs
+        busybox-static
+        ca-certificates
+        cpio
+        curl
+        device-tree-compiler
+        expect
+        u-boot-qemu
+        # for imagebuilder
+        file
+        u-boot-tools
+    )
+
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+
+    rm -rf /var/lib/apt/lists*
+EOF
+
+USER root
+WORKDIR /build
diff --git a/automation/build/debian/bookworm-arm64v8.dockerfile b/automation/build/debian/bookworm-arm64v8.dockerfile
deleted file mode 100644
index da8ae7512589..000000000000
--- a/automation/build/debian/bookworm-arm64v8.dockerfile
+++ /dev/null
@@ -1,55 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/arm64/v8 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        zlib1g-dev \
-        libncurses5-dev \
-        python3-dev \
-        python3-setuptools \
-        uuid-dev \
-        libyajl-dev \
-        libaio-dev \
-        libglib2.0-dev \
-        clang \
-        libpixman-1-dev \
-        pkg-config \
-        flex \
-        bison \
-        acpica-tools \
-        libfdt-dev \
-        bin86 \
-        bcc \
-        liblzma-dev \
-        libnl-3-dev \
-        ocaml-nox \
-        libfindlib-ocaml-dev \
-        markdown \
-        transfig \
-        pandoc \
-        checkpolicy \
-        wget \
-        git \
-        nasm \
-        # for test phase, qemu-smoke-* jobs
-        u-boot-qemu \
-        u-boot-tools \
-        device-tree-compiler \
-        curl \
-        cpio \
-        busybox-static \
-        expect \
-        && \
-        apt-get autoremove -y && \
-        apt-get clean && \
-        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 871beb70e157..8ee1049d857c 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -411,15 +411,15 @@ debian-bookworm-gcc-arm32-debug-earlyprintk:
 
 # Arm builds
 
-debian-bookworm-gcc-arm64:
+debian-12-arm64-gcc:
   extends: .gcc-arm64-build
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
 
-debian-bookworm-gcc-debug-arm64:
+debian-12-arm64-gcc-debug:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
 
 alpine-3.18-gcc-arm64:
   extends: .gcc-arm64-build
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 5b89cfa33cf8..3a4d0eb7e15d 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -17,7 +17,7 @@
 .qemu-arm64:
   extends: .test-jobs-common
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
     LOGFILE: qemu-smoke-arm64.log
   artifacts:
     paths:
@@ -30,7 +30,7 @@
 .qemu-arm32:
   extends: .test-jobs-common
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
     LOGFILE: qemu-smoke-arm32.log
   artifacts:
     paths:
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index d72c22c103ff..1b75c8d93724 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -37,7 +37,7 @@ case "_${CONTAINER}" in
     _bookworm|_bookworm-x86_64|_) CONTAINER="${BASE}/debian:12-x86_64" ;;
     _bookworm-i386|_bookworm-x86_32) CONTAINER="${BASE}/debian:12-x86_32" ;;
     _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;;
-    _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;;
+    _bookworm-arm64v8) CONTAINER="${BASE}/debian:12-arm64v8" ;;
     _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;;
     _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;;
     _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;;
-- 
2.45.2

[XEN PATCH v2 4/6] CI: Refresh the Debian 12 arm32 cross compile container

[Javi Merino]

Rework the container to user heredocs for readability and use
apt-get --no-install-recommends to keep the size down.  Rename the CI
jobs to debian-12-arm32-<variant> to follow the naming scheme of all
the other CI jobs.

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---
 .../debian/12-arm64v8-arm32-gcc.dockerfile    | 28 +++++++++++++++++++
 .../bookworm-arm64v8-arm32-gcc.dockerfile     | 24 ----------------
 automation/gitlab-ci/build.yaml               | 20 ++++++-------
 automation/gitlab-ci/test.yaml                | 14 +++++-----
 automation/scripts/containerize               |  2 +-
 5 files changed, 46 insertions(+), 42 deletions(-)
 create mode 100644 automation/build/debian/12-arm64v8-arm32-gcc.dockerfile
 delete mode 100644 automation/build/debian/bookworm-arm64v8-arm32-gcc.dockerfile

diff --git a/automation/build/debian/12-arm64v8-arm32-gcc.dockerfile b/automation/build/debian/12-arm64v8-arm32-gcc.dockerfile
new file mode 100644
index 000000000000..01756bfcbf72
--- /dev/null
+++ b/automation/build/debian/12-arm64v8-arm32-gcc.dockerfile
@@ -0,0 +1,28 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/arm64/v8 debian:bookworm
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV CROSS_COMPILE=/usr/bin/arm-linux-gnueabihf-
+
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    useradd --create-home user
+
+    apt-get update
+    DEPS=(
+        bison
+        build-essential
+        flex
+        gcc-arm-linux-gnueabihf
+    )
+    apt-get --yes --no-install-recommends install "${DEPS[@]}"
+
+    rm -rf /var/lib/apt/lists*
+EOF
+
+USER user
+WORKDIR /build
diff --git a/automation/build/debian/bookworm-arm64v8-arm32-gcc.dockerfile b/automation/build/debian/bookworm-arm64v8-arm32-gcc.dockerfile
deleted file mode 100644
index 95b3f0428372..000000000000
--- a/automation/build/debian/bookworm-arm64v8-arm32-gcc.dockerfile
+++ /dev/null
@@ -1,24 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/arm64/v8 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV USER root
-ENV CROSS_COMPILE /usr/bin/arm-linux-gnueabihf-
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        flex \
-        bison \
-        git \
-        gcc-arm-linux-gnueabihf \
-        && \
-        apt-get autoremove -y && \
-        apt-get clean && \
-        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 8ee1049d857c..3f87187443e2 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -371,39 +371,39 @@ debian-12-riscv64-gcc-debug:
 
 # Arm32 cross-build
 
-debian-bookworm-gcc-arm32:
+debian-12-arm32-gcc:
   extends: .gcc-arm32-cross-build
   variables:
-    CONTAINER: debian:bookworm-arm64v8-arm32-gcc
+    CONTAINER: debian:12-arm64v8-arm32-gcc
     HYPERVISOR_ONLY: y
 
-debian-bookworm-gcc-arm32-debug:
+debian-12-arm32-gcc-debug:
   extends: .gcc-arm32-cross-build-debug
   variables:
-    CONTAINER: debian:bookworm-arm64v8-arm32-gcc
+    CONTAINER: debian:12-arm64v8-arm32-gcc
     HYPERVISOR_ONLY: y
 
-debian-bookworm-gcc-arm32-randconfig:
+debian-12-arm32-gcc-randconfig:
   extends: .gcc-arm32-cross-build
   variables:
-    CONTAINER: debian:bookworm-arm64v8-arm32-gcc
+    CONTAINER: debian:12-arm64v8-arm32-gcc
     HYPERVISOR_ONLY: y
     RANDCONFIG: y
 
-debian-bookworm-gcc-arm32-debug-staticmem:
+debian-12-arm32-gcc-debug-staticmem:
   extends: .gcc-arm32-cross-build-debug
   variables:
-    CONTAINER: debian:bookworm-arm64v8-arm32-gcc
+    CONTAINER: debian:12-arm64v8-arm32-gcc
     HYPERVISOR_ONLY: y
     EXTRA_XEN_CONFIG: |
       CONFIG_EXPERT=y
       CONFIG_UNSUPPORTED=y
       CONFIG_STATIC_MEMORY=y
 
-debian-bookworm-gcc-arm32-debug-earlyprintk:
+debian-12-arm32-gcc-debug-earlyprintk:
   extends: .gcc-arm32-cross-build-debug
   variables:
-    CONTAINER: debian:bookworm-arm64v8-arm32-gcc
+    CONTAINER: debian:12-arm64v8-arm32-gcc
     HYPERVISOR_ONLY: y
     EXTRA_XEN_CONFIG: |
       CONFIG_EARLY_UART_CHOICE_PL011=y
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 3a4d0eb7e15d..fa452ccf7a9e 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -414,7 +414,7 @@ qemu-smoke-dom0less-arm32-gcc:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32
+    - debian-12-arm32-gcc
 
 qemu-smoke-dom0less-arm32-gcc-debug:
   extends: .qemu-arm32
@@ -422,7 +422,7 @@ qemu-smoke-dom0less-arm32-gcc-debug:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32-debug
+    - debian-12-arm32-gcc-debug
 
 qemu-smoke-dom0less-arm32-gcc-debug-staticmem:
   extends: .qemu-arm32
@@ -430,7 +430,7 @@ qemu-smoke-dom0less-arm32-gcc-debug-staticmem:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh static-mem 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32-debug-staticmem
+    - debian-12-arm32-gcc-debug-staticmem
 
 qemu-smoke-dom0less-arm32-gcc-debug-gzip:
   extends: .qemu-arm32
@@ -438,7 +438,7 @@ qemu-smoke-dom0less-arm32-gcc-debug-gzip:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh gzip 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32-debug
+    - debian-12-arm32-gcc-debug
 
 qemu-smoke-dom0less-arm32-gcc-without-dom0:
   extends: .qemu-arm32
@@ -446,7 +446,7 @@ qemu-smoke-dom0less-arm32-gcc-without-dom0:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh without-dom0 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32
+    - debian-12-arm32-gcc
 
 qemu-smoke-dom0less-arm32-gcc-debug-without-dom0:
   extends: .qemu-arm32
@@ -454,7 +454,7 @@ qemu-smoke-dom0less-arm32-gcc-debug-without-dom0:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh without-dom0 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32-debug
+    - debian-12-arm32-gcc-debug
 
 qemu-smoke-dom0less-arm32-gcc-debug-earlyprintk:
   extends: .qemu-arm32
@@ -462,7 +462,7 @@ qemu-smoke-dom0less-arm32-gcc-debug-earlyprintk:
     - ./automation/scripts/qemu-smoke-dom0less-arm32.sh earlyprintk 2>&1 | tee ${LOGFILE}
   needs:
     - *arm32-test-needs
-    - debian-bookworm-gcc-arm32-debug-earlyprintk
+    - debian-12-arm32-gcc-debug-earlyprintk
 
 qemu-alpine-x86_64-gcc:
   extends: .qemu-x86-64
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index 1b75c8d93724..c9988bfe927d 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -36,7 +36,7 @@ case "_${CONTAINER}" in
     _bookworm-x86_64-gcc-ibt) CONTAINER="${BASE}/debian:12-x86_64-gcc-ibt" ;;
     _bookworm|_bookworm-x86_64|_) CONTAINER="${BASE}/debian:12-x86_64" ;;
     _bookworm-i386|_bookworm-x86_32) CONTAINER="${BASE}/debian:12-x86_32" ;;
-    _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;;
+    _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:12-arm64v8-arm32-gcc" ;;
     _bookworm-arm64v8) CONTAINER="${BASE}/debian:12-arm64v8" ;;
     _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;;
     _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;;
-- 
2.45.2

[XEN PATCH v2 5/6] CI: Refresh the Debian 12 cppcheck container

[Javi Merino]

Rework the container to build and run cppcheck as a normal user.  User
heredocs for readability and use apt-get --no-install-recommends to
keep the size down.

Changed the libpcre3-dev dependency to libpcre3, as the -dev package
is only needed for building, not for running.

With the cleanups, the size of the container is reduced from 882MB to
782MB.

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---
 .../debian/12-arm64v8-cppcheck.dockerfile     | 79 +++++++++++++++++++
 .../build/debian/bookworm-cppcheck.dockerfile | 54 -------------
 automation/gitlab-ci/build.yaml               | 12 +--
 automation/scripts/containerize               |  2 +-
 4 files changed, 86 insertions(+), 61 deletions(-)
 create mode 100644 automation/build/debian/12-arm64v8-cppcheck.dockerfile
 delete mode 100644 automation/build/debian/bookworm-cppcheck.dockerfile

diff --git a/automation/build/debian/12-arm64v8-cppcheck.dockerfile b/automation/build/debian/12-arm64v8-cppcheck.dockerfile
new file mode 100644
index 000000000000..21c006e74f2e
--- /dev/null
+++ b/automation/build/debian/12-arm64v8-cppcheck.dockerfile
@@ -0,0 +1,79 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/arm64/v8 debian:bookworm AS builder
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV CPPCHECK_VERSION=2.7
+
+# dependencies for cppcheck build
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    apt-get update
+    DEPS=(
+        build-essential
+        ca-certificates
+        curl
+        libpcre3-dev
+        python-is-python3
+    )
+
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+
+    rm -rf /var/lib/apt/lists*
+EOF
+
+RUN useradd --home /build --create-home user
+
+WORKDIR /build
+USER user
+
+# cppcheck release build (see cppcheck readme.md)
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    curl -fsSL https://github.com/danmar/cppcheck/archive/"$CPPCHECK_VERSION".tar.gz | tar xvz
+    cd cppcheck-"$CPPCHECK_VERSION"
+
+    MAKE_OPTS=(
+        MATCHCOMPILER=yes
+        DESTDIR=/build/out
+        FILESDIR="/usr/share/cppcheck"
+        HAVE_RULES=yes CXXFLAGS="-O2 -DNDEBUG -Wall -Wno-sign-compare -Wno-unused-function"
+    )
+    make install -j$(nproc) "${MAKE_OPTS[@]}"
+EOF
+
+FROM --platform=linux/arm64/v8 debian:bookworm
+COPY --from=builder /build/out/usr/bin/cppcheck /usr/bin/cppcheck
+COPY --from=builder /build/out/usr/share/cppcheck /usr/share/cppcheck
+
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# dependencies for cppcheck analysis including Xen-only build/cross-build
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    useradd --create-home user
+
+    apt-get update
+    DEPS=(
+        bison
+        build-essential
+        python-is-python3
+        libpcre3
+        flex
+        gcc-arm-linux-gnueabihf
+        gcc-x86-64-linux-gnu
+    )
+    apt-get --yes --no-install-recommends install "${DEPS[@]}"
+    rm -rf /var/lib/apt/lists*
+EOF
+
+USER user
+WORKDIR /build
diff --git a/automation/build/debian/bookworm-cppcheck.dockerfile b/automation/build/debian/bookworm-cppcheck.dockerfile
deleted file mode 100644
index fe4cd4a1aaab..000000000000
--- a/automation/build/debian/bookworm-cppcheck.dockerfile
+++ /dev/null
@@ -1,54 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/arm64/v8 debian:bookworm AS builder
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV CPPCHECK_VERSION=2.7
-ENV USER root
-
-# dependencies for cppcheck build
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        curl \
-        build-essential \
-        python-is-python3 \
-        libpcre3-dev
-
-RUN mkdir /build
-WORKDIR /build
-
-# cppcheck release build (see cppcheck readme.md)
-RUN curl -fsSLO https://github.com/danmar/cppcheck/archive/"$CPPCHECK_VERSION".tar.gz && \
-    tar xvzf "$CPPCHECK_VERSION".tar.gz && \
-    cd cppcheck-"$CPPCHECK_VERSION" && \
-    make install -j$(nproc) \
-        MATCHCOMPILER=yes \
-        FILESDIR=/usr/share/cppcheck \
-        HAVE_RULES=yes CXXFLAGS="-O2 -DNDEBUG -Wall -Wno-sign-compare -Wno-unused-function"
-
-FROM --platform=linux/arm64/v8 debian:bookworm
-COPY --from=builder /usr/bin/cppcheck /usr/bin/cppcheck
-COPY --from=builder /usr/share/cppcheck /usr/share/cppcheck
-
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# dependencies for cppcheck analysis including Xen-only build/cross-build
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        python-is-python3 \
-        libpcre3-dev \
-        flex \
-        bison \
-        gcc-arm-linux-gnueabihf \
-        gcc-x86-64-linux-gnu \
-        && \
-        apt-get autoremove -y && \
-        apt-get clean && \
-        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 3f87187443e2..57b013334b6c 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -492,26 +492,26 @@ yocto-qemux86-64:
 
 # Cppcheck analysis jobs
 
-debian-bookworm-gcc-cppcheck:
+debian-12-x86_64-gcc-cppcheck:
   extends: .gcc-x86-64-cross-build
   variables:
-    CONTAINER: debian:bookworm-cppcheck
+    CONTAINER: debian:12-arm64v8-cppcheck
     CROSS_COMPILE: /usr/bin/x86_64-linux-gnu-
     CPPCHECK: y
     HYPERVISOR_ONLY: y
 
-debian-bookworm-gcc-arm32-cppcheck:
+debian-12-arm32-gcc-cppcheck:
   extends: .gcc-arm32-cross-build
   variables:
-    CONTAINER: debian:bookworm-cppcheck
+    CONTAINER: debian:12-arm64v8-cppcheck
     CROSS_COMPILE: /usr/bin/arm-linux-gnueabihf-
     CPPCHECK: y
     HYPERVISOR_ONLY: y
 
-debian-bookworm-gcc-arm64-cppcheck:
+debian-12-arm64-gcc-cppcheck:
   extends: .gcc-arm64-build
   variables:
-    CONTAINER: debian:bookworm-cppcheck
+    CONTAINER: debian:12-arm64v8-cppcheck
     CPPCHECK: y
     HYPERVISOR_ONLY: y
 
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index c9988bfe927d..e5502c81a759 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -38,7 +38,7 @@ case "_${CONTAINER}" in
     _bookworm-i386|_bookworm-x86_32) CONTAINER="${BASE}/debian:12-x86_32" ;;
     _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:12-arm64v8-arm32-gcc" ;;
     _bookworm-arm64v8) CONTAINER="${BASE}/debian:12-arm64v8" ;;
-    _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;;
+    _bookworm-cppcheck) CONTAINER="${BASE}/debian:12-arm64v8-cppcheck" ;;
     _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;;
     _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;;
     _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;;
-- 
2.45.2

[XEN PATCH v2 6/6] CI: Don't use -y with apt-get update

[Javi Merino]

apt-get update refreshes the package lists.  -y doesn't do anything
here.  It is needed for "apt-get install" or "apt-get upgrade" but not
for apt-get update.  Drop it.

Signed-off-by: Javi Merino <javi.merino@cloud.com>
---
 automation/build/debian/11-ppc64le.dockerfile        | 2 +-
 automation/build/debian/11-riscv64.dockerfile        | 2 +-
 automation/build/debian/12-ppc64le.dockerfile        | 2 +-
 automation/build/debian/12-riscv64.dockerfile        | 2 +-
 automation/build/debian/12-x86_64-gcc-ibt.dockerfile | 4 ++--
 automation/build/ubuntu/16.04-x86_64.dockerfile      | 2 +-
 automation/build/ubuntu/18.04-x86_64.dockerfile      | 2 +-
 automation/build/ubuntu/20.04-x86_64.dockerfile      | 2 +-
 automation/build/ubuntu/22.04-x86_64.dockerfile      | 2 +-
 automation/build/ubuntu/24.04-x86_64.dockerfile      | 2 +-
 10 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/automation/build/debian/11-ppc64le.dockerfile b/automation/build/debian/11-ppc64le.dockerfile
index d846b670d02c..385a0a071db2 100644
--- a/automation/build/debian/11-ppc64le.dockerfile
+++ b/automation/build/debian/11-ppc64le.dockerfile
@@ -13,7 +13,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/debian/11-riscv64.dockerfile b/automation/build/debian/11-riscv64.dockerfile
index 1c99bc89ea1a..a55047c5019a 100644
--- a/automation/build/debian/11-riscv64.dockerfile
+++ b/automation/build/debian/11-riscv64.dockerfile
@@ -13,7 +13,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/debian/12-ppc64le.dockerfile b/automation/build/debian/12-ppc64le.dockerfile
index 1d5b486bb47f..da518aab4e10 100644
--- a/automation/build/debian/12-ppc64le.dockerfile
+++ b/automation/build/debian/12-ppc64le.dockerfile
@@ -13,7 +13,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/debian/12-riscv64.dockerfile b/automation/build/debian/12-riscv64.dockerfile
index 8d7233d6757d..bce8f1015f7c 100644
--- a/automation/build/debian/12-riscv64.dockerfile
+++ b/automation/build/debian/12-riscv64.dockerfile
@@ -13,7 +13,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/debian/12-x86_64-gcc-ibt.dockerfile b/automation/build/debian/12-x86_64-gcc-ibt.dockerfile
index 3ce60e7d9466..9b5d11fca648 100644
--- a/automation/build/debian/12-x86_64-gcc-ibt.dockerfile
+++ b/automation/build/debian/12-x86_64-gcc-ibt.dockerfile
@@ -6,7 +6,7 @@ ENV DEBIAN_FRONTEND=noninteractive
 RUN <<EOF
 #!/bin/bash
     set -e
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         bison
@@ -62,7 +62,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/ubuntu/16.04-x86_64.dockerfile b/automation/build/ubuntu/16.04-x86_64.dockerfile
index d8617bfade5e..9cc8ca89e8e0 100644
--- a/automation/build/ubuntu/16.04-x86_64.dockerfile
+++ b/automation/build/ubuntu/16.04-x86_64.dockerfile
@@ -11,7 +11,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/ubuntu/18.04-x86_64.dockerfile b/automation/build/ubuntu/18.04-x86_64.dockerfile
index 336697ad954b..aefe52125a22 100644
--- a/automation/build/ubuntu/18.04-x86_64.dockerfile
+++ b/automation/build/ubuntu/18.04-x86_64.dockerfile
@@ -11,7 +11,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/ubuntu/20.04-x86_64.dockerfile b/automation/build/ubuntu/20.04-x86_64.dockerfile
index 39fb297519ee..1ee20a13ac6b 100644
--- a/automation/build/ubuntu/20.04-x86_64.dockerfile
+++ b/automation/build/ubuntu/20.04-x86_64.dockerfile
@@ -11,7 +11,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/ubuntu/22.04-x86_64.dockerfile b/automation/build/ubuntu/22.04-x86_64.dockerfile
index 6aa3c4d1881d..a9a9b84930fb 100644
--- a/automation/build/ubuntu/22.04-x86_64.dockerfile
+++ b/automation/build/ubuntu/22.04-x86_64.dockerfile
@@ -11,7 +11,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
diff --git a/automation/build/ubuntu/24.04-x86_64.dockerfile b/automation/build/ubuntu/24.04-x86_64.dockerfile
index c46d152abf10..2005723b31ad 100644
--- a/automation/build/ubuntu/24.04-x86_64.dockerfile
+++ b/automation/build/ubuntu/24.04-x86_64.dockerfile
@@ -11,7 +11,7 @@ RUN <<EOF
 
     useradd --create-home user
 
-    apt-get -y update
+    apt-get update
 
     DEPS=(
         # Xen
-- 
2.45.2

Re: [XEN PATCH v2 6/6] CI: Don't use -y with apt-get update

[Andrew Cooper]

On 06/11/2024 1:05 pm, Javi Merino wrote:
> apt-get update refreshes the package lists.  -y doesn't do anything
> here.  It is needed for "apt-get install" or "apt-get upgrade" but not
> for apt-get update.  Drop it.
>
> Signed-off-by: Javi Merino <javi.merino@cloud.com>

Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>

> ---
>  automation/build/debian/11-ppc64le.dockerfile        | 2 +-
>  automation/build/debian/11-riscv64.dockerfile        | 2 +-
>  automation/build/debian/12-ppc64le.dockerfile        | 2 +-
>  automation/build/debian/12-riscv64.dockerfile        | 2 +-
>  automation/build/debian/12-x86_64-gcc-ibt.dockerfile | 4 ++--
>  automation/build/ubuntu/16.04-x86_64.dockerfile      | 2 +-
>  automation/build/ubuntu/18.04-x86_64.dockerfile      | 2 +-
>  automation/build/ubuntu/20.04-x86_64.dockerfile      | 2 +-
>  automation/build/ubuntu/22.04-x86_64.dockerfile      | 2 +-
>  automation/build/ubuntu/24.04-x86_64.dockerfile      | 2 +-

Judging by this file list, the change is independent of patches 1-5, so
can go in straight away.

~Andrew

Re: [XEN PATCH v2 1/6] CI: Refresh the Debian 12 x86_64 container

[Andrew Cooper]

On 06/11/2024 1:05 pm, Javi Merino wrote:
> Rework the container to use heredocs for readability, and use
> apt-get --no-install-recommends to keep the size down.
>
> This reduces the size of the (uncompressed) container from 3.44GB to
> 1.97GB.
>
> The container is left running the builds and tests as root to avoid
> breaking the xilinx runners.
>
> Signed-off-by: Javi Merino <javi.merino@cloud.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Re: [XEN PATCH v2 3/6] CI: Refresh the Debian 12 arm64 container

[Stefano Stabellini]

On Wed, 6 Nov 2024, Javi Merino wrote:
> Rework the container to use heredocs for readability and use
> apt-get --no-install-recommends to keep the size down.  Rename the job
> to debian-12-arm64-* to follow the naming scheme of all the other CI
> jobs.
> 
> This reduces the size of the debian:12-arm64v8 from 2.25GB down to 1.62GB.
> 
> The container is left running the builds and tests as root to avoid
> breaking the xilinx runners.
> 
> Signed-off-by: Javi Merino <javi.merino@cloud.com>


Hi Javi,

I generated a new dependency diff, this time it should be correct, see
the diff appended below. Most of the differences are not concerning,
but I have a couple of questions.


> --- /tmp/1a	2024-11-06 16:12:09.235734221 +0000
> +++ /tmp/2a	2024-11-06 16:12:15.203659831 +0000
> @@ -1,36 +1,32 @@
>  acpica-tools
> -bcc
> -bin86
>  bison
>  build-essential
>  busybox-static
> +ca-certificates
>  checkpolicy
> -clang
>  cpio
>  curl
>  device-tree-compiler
>  expect
> +file
>  flex
> -git
> -libaio-dev

I think this one might be needed to build QEMU but given that we don't
have any build jobs building QEMU on ARM today anymore, it could be
removed. We are going to add a new build job to build QEMU on ARM at
some point soon but we can add any relevant dependency back at that
time.


> +git-core
> +golang-go
> +libbz2-dev
> +libext2fs-dev
>  libfdt-dev
> -libfindlib-ocaml-dev
>  libglib2.0-dev
>  liblzma-dev
> -libncurses5-dev
> -libnl-3-dev

I think it is the same for these two


> +liblzo2-dev
>  libpixman-1-dev
>  libyajl-dev
> -markdown
> -nasm
> +libzstd-dev
> +ocaml-findlib
>  ocaml-nox
> -pandoc

The removal of pandoc and markdown both concerns me. Wouldn't this cause
the documents under docs/ not to be built anymore?


>  pkg-config
>  python3-dev
>  python3-setuptools
> -transfig
>  u-boot-qemu
>  u-boot-tools
>  uuid-dev
>  wget
> -zlib1g-dev

Re: [XEN PATCH v2 2/6] CI: Refresh the Debian 12 x86_32 container

[Andrew Cooper]

On 06/11/2024 1:05 pm, Javi Merino wrote:
> Rework the container to be non-root, use heredocs for readability, and
> use apt-get --no-install-recommends to keep the size down.  Rename the
> job to x86_32, to be consistent with XEN_TARGET_ARCH and the
> naming scheme of all the other CI jobs:
> ${VERSION}-${ARCH}-${BUILD_NAME}
>
> Remove build dependencies for building QEMU, as we don't do it since
> e305256e69b1 ("CI: Stop building QEMU in general").

The container never had ninja/meson so Qemu hasn't built in it for many
years, irrespective of e305256e69b1.

>
> Remove build dependencies for the documentation as we don't have to
> build it for every single arch.
>
> This reduces the size of the container from 2.22GB to 1.32Gb.
>
> Signed-off-by: Javi Merino <javi.merino@cloud.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

I'll tweak the commit message on commit, to save needing a repost.

~Andrew

Re: [XEN PATCH v2 3/6] CI: Refresh the Debian 12 arm64 container

[Andrew Cooper]

On 06/11/2024 4:23 pm, Stefano Stabellini wrote:
> On Wed, 6 Nov 2024, Javi Merino wrote:
>> Rework the container to use heredocs for readability and use
>> apt-get --no-install-recommends to keep the size down.  Rename the job
>> to debian-12-arm64-* to follow the naming scheme of all the other CI
>> jobs.
>>
>> This reduces the size of the debian:12-arm64v8 from 2.25GB down to 1.62GB.
>>
>> The container is left running the builds and tests as root to avoid
>> breaking the xilinx runners.
>>
>> Signed-off-by: Javi Merino <javi.merino@cloud.com>
>
> Hi Javi,
>
> I generated a new dependency diff, this time it should be correct, see
> the diff appended below. Most of the differences are not concerning,
> but I have a couple of questions.

Javi has moved elsewhere, so I guess I'm picking this up.

>
>
>> --- /tmp/1a	2024-11-06 16:12:09.235734221 +0000
>> +++ /tmp/2a	2024-11-06 16:12:15.203659831 +0000
>> @@ -1,36 +1,32 @@
>>  acpica-tools
>> -bcc
>> -bin86
>>  bison
>>  build-essential
>>  busybox-static
>> +ca-certificates
>>  checkpolicy
>> -clang
>>  cpio
>>  curl
>>  device-tree-compiler
>>  expect
>> +file
>>  flex
>> -git
>> -libaio-dev
> I think this one might be needed to build QEMU but given that we don't
> have any build jobs building QEMU on ARM today anymore, it could be
> removed. We are going to add a new build job to build QEMU on ARM at
> some point soon but we can add any relevant dependency back at that
> time.

Deps can be added back when (if) they're needed.  It's far easier to do
it this way around, than the other.

There's more that can be stripped, I think.

Without QEMU, we can drop libglib2.0-dev, libpixman-1-dev.

acpica-tools seems like an x86-ism to me.


I am surprised at the addition of "file".  It's listed as needed for the
test phase, but if it wasn't present before, how did things work?

That said, I really would prefer to strip out all the test container
stuff and get that working through artefacts, which will reduce test time.

>> +git-core
>> +golang-go
>> +libbz2-dev
>> +libext2fs-dev
>>  libfdt-dev
>> -libfindlib-ocaml-dev
>>  libglib2.0-dev
>>  liblzma-dev
>> -libncurses5-dev
>> -libnl-3-dev
> I think it is the same for these two

curses is for xentop.  nl-3 is for `xl colo` and surely not needed.

>
>
>> +liblzo2-dev
>>  libpixman-1-dev
>>  libyajl-dev
>> -markdown
>> -nasm
>> +libzstd-dev
>> +ocaml-findlib
>>  ocaml-nox
>> -pandoc
> The removal of pandoc and markdown both concerns me. Wouldn't this cause
> the documents under docs/ not to be built anymore?

This is intentional, because they're huge.  I'm going to do docs builds
separately, including wiring up -Werror.

>
>
>>  pkg-config
>>  python3-dev
>>  python3-setuptools
>> -transfig
>>  u-boot-qemu
>>  u-boot-tools
>>  uuid-dev
>>  wget
>> -zlib1g-dev

zlib wants to say.  It's part of the xenguest dombuilder.

~Andrew