[PATCH 0/3] CI: Cleanup/Upgrade of Linux used for testing

[PATCH 0/3] CI: Cleanup/Upgrade of Linux used for testing

[Andrew Cooper]

Switch fully on to test-artefacts for Linux.

Patch 3 is a repost.  Patch 1 supercedes a prior posting.

https://gitlab.com/xen-project/hardware/xen-staging/-/pipelines/1762221755

Andrew Cooper (3):
  CI: Update x86 tests from Linux 6.1.19 to 6.6.56
  CI: Update ARM64 tests from Linux 6.6.74 to 6.6.86
  CI: Drop stale test-artifacts/alpine/3.18.dockerfile

 automation/gitlab-ci/build.yaml               | 24 -------
 automation/gitlab-ci/test.yaml                |  8 ++-
 .../tests-artifacts/alpine/3.18.dockerfile    | 67 -------------------
 .../tests-artifacts/kernel/6.1.19.dockerfile  | 41 ------------
 .../kernel/6.6.74-arm64v8.dockerfile          | 38 -----------
 5 files changed, 6 insertions(+), 172 deletions(-)
 delete mode 100644 automation/tests-artifacts/alpine/3.18.dockerfile
 delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile
 delete mode 100644 automation/tests-artifacts/kernel/6.6.74-arm64v8.dockerfile

-- 
2.39.5

[PATCH 1/3] CI: Update x86 tests from Linux 6.1.19 to 6.6.56

[Andrew Cooper]

Linux 6.6.56 was already added to test-artifacts for the argo testing, and
this removes one moving part while cleaning things up.

Drop the associated export job, and dockerfile.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Also, somewhere between 6.6.56 and 6.6.74, PVH stops working
---
 automation/gitlab-ci/build.yaml               | 13 ------
 automation/gitlab-ci/test.yaml                |  4 +-
 .../tests-artifacts/kernel/6.1.19.dockerfile  | 41 -------------------
 3 files changed, 3 insertions(+), 55 deletions(-)
 delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 169bebe3c775..ab758243c1ec 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -308,19 +308,6 @@ qemu-system-aarch64-6.0.0-arm32-export:
   tags:
     - arm64
 
-# x86_64 test artifacts
-
-kernel-6.1.19-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
-  script:
-    - mkdir binaries && cp /bzImage binaries/bzImage
-  artifacts:
-    paths:
-      - binaries/bzImage
-  tags:
-    - x86_64
-
 # Jobs below this line
 
 # Build jobs needed for tests
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index d05b9a98afa6..a23ff8a6e7fe 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -16,7 +16,9 @@
   - qemu-system-aarch64-6.0.0-arm32-export
 
 .x86-64-test-needs: &x86-64-test-needs
-  - kernel-6.1.19-export
+  - project: xen-project/hardware/test-artifacts
+    job: linux-6.6.56-x86_64
+    ref: master
   - project: xen-project/hardware/test-artifacts
     job: x86_64-rootfs-alpine-3.18
     ref: master
diff --git a/automation/tests-artifacts/kernel/6.1.19.dockerfile b/automation/tests-artifacts/kernel/6.1.19.dockerfile
deleted file mode 100644
index 073eaa0e11fa..000000000000
--- a/automation/tests-artifacts/kernel/6.1.19.dockerfile
+++ /dev/null
@@ -1,41 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/amd64 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV LINUX_VERSION=6.1.19
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        bc \
-        curl \
-        flex \
-        bison \
-        libelf-dev \
-        && \
-    apt-get autoremove -y && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-
-# Build the kernel
-RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
-    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
-    cd linux-"$LINUX_VERSION" && \
-    make defconfig && \
-    make xen.config && \
-    scripts/config --enable BRIDGE && \
-    scripts/config --enable IGC && \
-    scripts/config --enable TUN && \
-    cp .config .config.orig && \
-    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
-    make -j$(nproc) bzImage && \
-    cp arch/x86/boot/bzImage / && \
-    cd /build && \
-    rm -rf linux-"$LINUX_VERSION"*
-- 
2.39.5

Re: [PATCH 1/3] CI: Update x86 tests from Linux 6.1.19 to 6.6.56

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 470 bytes --]

On Thu, Apr 10, 2025 at 01:05:18PM +0100, Andrew Cooper wrote:
> Linux 6.6.56 was already added to test-artifacts for the argo testing, and
> this removes one moving part while cleaning things up.
> 
> Drop the associated export job, and dockerfile.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 2/3] CI: Update ARM64 tests from Linux 6.6.74 to 6.6.86

[Andrew Cooper]

Switch over to test-artifacts.

Drop the associated export job, and dockerfile.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/gitlab-ci/build.yaml               | 11 ------
 automation/gitlab-ci/test.yaml                |  4 +-
 .../kernel/6.6.74-arm64v8.dockerfile          | 38 -------------------
 3 files changed, 3 insertions(+), 50 deletions(-)
 delete mode 100644 automation/tests-artifacts/kernel/6.6.74-arm64v8.dockerfile

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index ab758243c1ec..40dcd31f1bce 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -275,17 +275,6 @@ alpine-3.18-arm64-rootfs-export:
   tags:
     - arm64
 
-kernel-6.6.74-arm64-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.6.74-arm64v8
-  script:
-    - mkdir binaries && cp /Image binaries/Image
-  artifacts:
-    paths:
-      - binaries/Image
-  tags:
-    - arm64
-
 qemu-system-aarch64-6.0.0-arm64-export:
   extends: .test-jobs-artifact-common
   image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-aarch64:6.0.0-arm64v8
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index a23ff8a6e7fe..59a2de28c864 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -8,8 +8,10 @@
   - when: on_success
 
 .arm64-test-needs: &arm64-test-needs
+  - project: xen-project/hardware/test-artifacts
+    job: linux-6.6.86-arm64
+    ref: master
   - alpine-3.18-arm64-rootfs-export
-  - kernel-6.6.74-arm64-export
   - qemu-system-aarch64-6.0.0-arm64-export
 
 .arm32-test-needs: &arm32-test-needs
diff --git a/automation/tests-artifacts/kernel/6.6.74-arm64v8.dockerfile b/automation/tests-artifacts/kernel/6.6.74-arm64v8.dockerfile
deleted file mode 100644
index 73e51454257d..000000000000
--- a/automation/tests-artifacts/kernel/6.6.74-arm64v8.dockerfile
+++ /dev/null
@@ -1,38 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/arm64/v8 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV LINUX_VERSION=6.6.74
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        bc \
-        curl \
-        flex \
-        bison \
-        libssl-dev \
-        && \
-    \
-    # Build the kernel
-    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
-    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
-    cd linux-"$LINUX_VERSION" && \
-    make defconfig && \
-    sed -i 's/CONFIG_IPV6=m/CONFIG_IPV6=y/g' .config && \
-    sed -i 's/CONFIG_BRIDGE=m/CONFIG_BRIDGE=y/g' .config && \
-    sed -i 's/# CONFIG_XEN_NETDEV_BACKEND is not set/CONFIG_XEN_NETDEV_BACKEND=y/g' .config && \
-    make -j$(nproc) Image.gz && \
-    cp arch/arm64/boot/Image / && \
-    cd /build && \
-    rm -rf linux-"$LINUX_VERSION"* && \
-    apt-get autoremove -y && \
-    apt-get clean && \
-    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
2.39.5

Re: [PATCH 2/3] CI: Update ARM64 tests from Linux 6.6.74 to 6.6.86

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

On Thu, Apr 10, 2025 at 01:05:19PM +0100, Andrew Cooper wrote:
> Switch over to test-artifacts.
> 
> Drop the associated export job, and dockerfile.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH 3/3] CI: Drop stale test-artifacts/alpine/3.18.dockerfile

[Andrew Cooper]

This should have been dropped too, as the authoritative copy lives in another
repo now.

Fixes: babe11b46c1a ("CI: Drop alpine-3.18-rootfs-export and use test-artefacts")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Repost
---
 .../tests-artifacts/alpine/3.18.dockerfile    | 67 -------------------
 1 file changed, 67 deletions(-)
 delete mode 100644 automation/tests-artifacts/alpine/3.18.dockerfile

diff --git a/automation/tests-artifacts/alpine/3.18.dockerfile b/automation/tests-artifacts/alpine/3.18.dockerfile
deleted file mode 100644
index 311a92889b87..000000000000
--- a/automation/tests-artifacts/alpine/3.18.dockerfile
+++ /dev/null
@@ -1,67 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/amd64 alpine:3.18
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-RUN \
-  # apk
-  apk update && \
-  \
-  # xen runtime deps
-  apk add musl && \
-  apk add libgcc && \
-  apk add openrc && \
-  apk add busybox && \
-  apk add sudo && \
-  apk add dbus && \
-  apk add bash && \
-  apk add python3 && \
-  apk add zlib && \
-  apk add ncurses && \
-  apk add yajl && \
-  apk add libaio && \
-  apk add xz && \
-  apk add util-linux && \
-  apk add argp-standalone && \
-  apk add libfdt && \
-  apk add glib && \
-  apk add pixman && \
-  apk add curl && \
-  apk add udev && \
-  apk add pciutils && \
-  apk add libelf && \
-  \
-  # Xen
-  cd / && \
-  # Minimal ramdisk environment in case of cpio output
-  rc-update add udev && \
-  rc-update add udev-trigger && \
-  rc-update add udev-settle && \
-  rc-update add loopback sysinit && \
-  rc-update add bootmisc boot && \
-  rc-update add devfs sysinit && \
-  rc-update add dmesg sysinit && \
-  rc-update add hostname boot && \
-  rc-update add hwclock boot && \
-  rc-update add hwdrivers sysinit && \
-  rc-update add modules boot && \
-  rc-update add killprocs shutdown && \
-  rc-update add mount-ro shutdown && \
-  rc-update add savecache shutdown && \
-  rc-update add local default && \
-  cp -a /sbin/init /init && \
-  echo "ttyS0" >> /etc/securetty && \
-  echo "hvc0" >> /etc/securetty && \
-  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
-  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
-  echo > /etc/modules && \
-  passwd -d "root" root && \
-  \
-  # Create rootfs
-  cd / && \
-  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
-- 
2.39.5

Re: [PATCH 3/3] CI: Drop stale test-artifacts/alpine/3.18.dockerfile

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

On Thu, Apr 10, 2025 at 01:05:20PM +0100, Andrew Cooper wrote:
> This should have been dropped too, as the authoritative copy lives in another
> repo now.
> 
> Fixes: babe11b46c1a ("CI: Drop alpine-3.18-rootfs-export and use test-artefacts")
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]