[XEN PATCH v1 0/3] automation: add x86_64 test (linux argo)

[XEN PATCH v1 0/3] automation: add x86_64 test (linux argo)

[victorm.lira]

From: Victor Lira <victorm.lira@amd.com>

Add x86_64 hardware test that creates a Xen Argo communication
connection between two PVH domains.

To accomplish this, add new build artifacts for Linux and Argo, and
update the xilinx x86_64 test script.

Victor Lira (3):
  automation: add linux 6.6.56 artifact
  automation: add linux argo test artifacts
  automation: add x86_64 test (linux argo)

 automation/gitlab-ci/build.yaml               | 34 +++++++
 automation/gitlab-ci/test.yaml                | 10 ++
 .../scripts/xilinx-smoke-dom0-x86_64.sh       | 76 ++++++++++-----
 .../tests-artifacts/argo/6.6.56.dockerfile    | 95 +++++++++++++++++++
 .../tests-artifacts/kernel/6.6.56.dockerfile  | 54 +++++++++++
 5 files changed, 244 insertions(+), 25 deletions(-)
 create mode 100644 automation/tests-artifacts/argo/6.6.56.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/6.6.56.dockerfile

--
2.25.1

[XEN PATCH v1 1/3] automation: add linux 6.6.56 artifact

[victorm.lira]

From: Victor Lira <victorm.lira@amd.com>

Add dockerfile for building container image that holds linux 6.6.56
bzImage for x86_64, using the same build process as the current 6.1.19
image

Signed-off-by: Victor Lira <victorm.lira@amd.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org
---
 .../tests-artifacts/kernel/6.6.56.dockerfile  | 54 +++++++++++++++++++
 1 file changed, 54 insertions(+)
 create mode 100644 automation/tests-artifacts/kernel/6.6.56.dockerfile

diff --git a/automation/tests-artifacts/kernel/6.6.56.dockerfile b/automation/tests-artifacts/kernel/6.6.56.dockerfile
new file mode 100644
index 0000000000..5b6f2fe527
--- /dev/null
+++ b/automation/tests-artifacts/kernel/6.6.56.dockerfile
@@ -0,0 +1,54 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/amd64 debian:bookworm
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LINUX_VERSION=6.6.56
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        libssl-dev \
+        bc \
+        curl \
+        flex \
+        bison \
+        libelf-dev \
+        && \
+    apt-get autoremove -y && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/* && \
+    \
+    # Build the kernel
+    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
+    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
+    cd linux-"$LINUX_VERSION" && \
+    make defconfig && \
+    make xen.config && \
+    scripts/config --enable BRIDGE && \
+    scripts/config --enable IGC && \
+    scripts/config --enable TUN && \
+    cp .config .config.orig && \
+    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
+    make -j$(nproc) bzImage && \
+    cp arch/x86/boot/bzImage / && \
+    cd /build && \
+    rm -rf linux-"$LINUX_VERSION"* && \
+    \
+    # Clean up
+    apt-get --yes purge \
+        build-essential \
+        libssl-dev \
+        bc \
+        curl \
+        flex \
+        bison \
+        libelf-dev \
+        && \
+    apt-get autoremove -y
--
2.25.1

[XEN PATCH v1 2/3] automation: add linux argo test artifacts

[victorm.lira]

From: Victor Lira <victorm.lira@amd.com>

Add dockerfile for building container image that holds test binaries for
Xen Argo test on Linux 6.6.56 x86_64

The build produces the following:
- xen-argo.ko       Linux kernel module
- lib/libargo*      Linux shared library
- argo-exec         Linux user test program

Signed-off-by: Victor Lira <victorm.lira@amd.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org
---
 .../tests-artifacts/argo/6.6.56.dockerfile    | 95 +++++++++++++++++++
 1 file changed, 95 insertions(+)
 create mode 100644 automation/tests-artifacts/argo/6.6.56.dockerfile

diff --git a/automation/tests-artifacts/argo/6.6.56.dockerfile b/automation/tests-artifacts/argo/6.6.56.dockerfile
new file mode 100644
index 0000000000..a8e199ad3b
--- /dev/null
+++ b/automation/tests-artifacts/argo/6.6.56.dockerfile
@@ -0,0 +1,95 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/amd64 alpine:3.18
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV LINUX_VERSION=6.6.56
+ENV USER root
+
+RUN mkdir /build /artifacts
+WORKDIR /build
+ENV BUILDDIR=/build/
+ENV COPYDIR=/artifacts/
+ENV ARGO_SHA="705a7a8a624b42e13e655d3042059b8a85cdf6a3"
+ENV ARGOEXEC_SHA="d900429f6640acc6f68a3d3a4c945d7da60625d8"
+
+RUN apk --no-cache add \
+  \
+  musl-dev  \
+  build-base \
+  libc6-compat \
+  linux-headers \
+  bash \
+  git \
+  curl \
+  flex \
+  bison \
+  elfutils-dev \
+  autoconf \
+  automake \
+  libtool && \
+  \
+  # Prepare Linux sources
+  curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSION".tar.xz && \
+  tar xJf linux-"$LINUX_VERSION".tar.xz && \
+  cd linux-"$LINUX_VERSION" && \
+  make ARCH=x86 defconfig && \
+  make ARCH=x86 xen.config && \
+  ./scripts/config --enable BRIDGE && \
+  ./scripts/config --enable IGC && \
+  ./scripts/config --enable TUN && \
+  cp .config .config.orig && \
+  cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
+  make ARCH=x86 olddefconfig && \
+  make ARCH=x86 modules_prepare && \
+  \
+  # Build Linux kernel module for Xen Argo
+  cd "${BUILDDIR}" && \
+  git clone \
+      --depth=1 --branch=master \
+      https://github.com/OpenXT/linux-xen-argo.git && \
+  git -C "${BUILDDIR}/linux-xen-argo" switch --detach "${ARGO_SHA}" && \
+  make -C "linux-${LINUX_VERSION}" M="${BUILDDIR}/linux-xen-argo/argo-linux" \
+      CFLAGS_MODULE="-Wno-error" KBUILD_MODPOST_WARN=1 && \
+  cp "linux-xen-argo/argo-linux/xen-argo.ko" "${COPYDIR}/xen-argo.ko" && \
+  \
+  # Build Linux libargo shared library, applying fixes to build in Alpine
+  cd "${BUILDDIR}/linux-xen-argo/libargo" && \
+  sed -i "s|AM_INIT_AUTOMAKE|AC_CONFIG_AUX_DIR(.)\nAM_INIT_AUTOMAKE|" configure.ac && \
+  sed -i "s/__SOCKADDR_COMMON (sxenargo_)/sa_family_t sxenargo_family/" src/libargo.h && \
+  sed -i "s/__SOCKADDR_COMMON_SIZE/(sizeof (unsigned short int))/" src/libargo.h && \
+  autoreconf --install && \
+  ./configure  --prefix="${COPYDIR}" CPPFLAGS="-I${PWD}/../argo-linux/include" && \
+  make && \
+  make install && \
+  \
+  # Build Linux user program, modifying for xilinx argo test
+  cd "${BUILDDIR}" && \
+  wget "https://raw.githubusercontent.com/OpenXT/xenclient-oe/${ARGOEXEC_SHA}/recipes-openxt/argo-exec/argo-exec/argo-exec.c" && \
+  sed -i "s|#include <xen/xen.h>||" argo-exec.c && \
+  sed -i "s|ret = shuffle(s, fds\[0\], fds\[1\]);|ret = shuffle(s, 0, 1);|" argo-exec.c && \
+  gcc -I"${BUILDDIR}/linux-xen-argo/libargo/src" \
+      -I"${BUILDDIR}/linux-xen-argo/argo-linux/include" \
+      -L"${COPYDIR}/lib/" \
+      -o argo-exec argo-exec.c -largo && \
+  cp argo-exec "${COPYDIR}" && \
+  \
+  # Clean up
+  cd /build && \
+  rm -fr linux-"$LINUX_VERSION"* linux-xen-argo argo-exec.c && \
+  \
+  apk del \
+  \
+  musl-dev  \
+  build-base \
+  libc6-compat \
+  linux-headers \
+  bash \
+  git \
+  curl \
+  flex \
+  bison \
+  elfutils-dev \
+  autoconf \
+  automake \
+  libtool
--
2.25.1

[XEN PATCH v1 3/3] automation: add x86_64 test (linux argo)

[victorm.lira]

From: Victor Lira <victorm.lira@amd.com>

Add x86_64 hardware test that creates a Xen Argo communication
connection between two PVH domains. In the test, dom0 creates a domU and
listens for messages sent by the domU through Argo.

To accomplish this, add Xen build jobs to export Linux and argo artifacts,
and build Xen with CONFIG_ARGO=y.

Update the xilinx x86_64 test script to support the new test, and add
"sync_console" to command line to avoid an issue with console messages
being lost.

Signed-off-by: Victor Lira <victorm.lira@amd.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: xen-devel@lists.xenproject.org
---
 automation/gitlab-ci/build.yaml               | 34 +++++++++
 automation/gitlab-ci/test.yaml                | 10 +++
 .../scripts/xilinx-smoke-dom0-x86_64.sh       | 76 +++++++++++++------
 3 files changed, 95 insertions(+), 25 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index c2db69ecf8..827759fb7b 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -326,6 +326,30 @@ kernel-6.1.19-export:
   tags:
     - x86_64

+kernel-6.6.56-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.6.56
+  script:
+    - mkdir binaries && cp /bzImage binaries/bzImage
+  artifacts:
+    paths:
+      - binaries/bzImage
+  tags:
+    - x86_64
+
+argo-6.6.56-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/argo:6.6.56
+  script:
+    - mkdir binaries && cp -ar /artifacts/* binaries
+  artifacts:
+    paths:
+      - binaries/xen-argo.ko
+      - binaries/lib/
+      - binaries/argo-exec
+  tags:
+    - x86_64
+
 # Jobs below this line

 # Build jobs needed for tests
@@ -341,6 +365,16 @@ alpine-3.18-gcc-debug:
     CONTAINER: alpine:3.18
     BUILD_QEMU_XEN: y

+alpine-3.18-gcc-debug-argo:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: alpine:3.18
+    EXTRA_XEN_CONFIG: |
+      CONFIG_EXPERT=y
+      CONFIG_UNSUPPORTED=y
+      CONFIG_ARGO=y
+    BUILD_QEMU_XEN: n
+
 debian-bookworm-gcc-debug:
   extends: .gcc-x86-64-build-debug
   variables:
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index b27c2be174..849860627d 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -192,6 +192,16 @@ xilinx-smoke-dom0-x86_64-gcc-debug:
     - *x86-64-test-needs
     - alpine-3.18-gcc-debug

+xilinx-smoke-dom0-x86_64-gcc-debug-argo:
+  extends: .xilinx-x86_64
+  script:
+    - ./automation/scripts/xilinx-smoke-dom0-x86_64.sh argo 2>&1 | tee ${LOGFILE}
+  needs:
+    - alpine-3.18-gcc-debug-argo
+    - kernel-6.6.56-export
+    - alpine-3.18-rootfs-export
+    - argo-6.6.56-export
+
 adl-smoke-x86-64-gcc-debug:
   extends: .adl-x86-64
   script:
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 7027f083ba..f70cfdc155 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -9,6 +9,8 @@ fatal() {
     exit 1
 }

+WORKDIR="${PWD}"
+
 # Test parameter defaults.
 TEST="$1"
 PASS_MSG="Test passed: ${TEST}"
@@ -24,9 +26,10 @@ kernel = "/boot/vmlinuz"
 ramdisk = "/boot/initrd-domU"
 extra = "root=/dev/ram0 console=hvc0"
 memory = 512
-vif = [ "bridge=xenbr0", ]
-disk = [ ]
 '
+DOMU_CFG_EXTRA=""
+copy_domU_files () { :; }
+copy_dom0_files () { :; }

 # Select test variant.
 if [ "${TEST}" = "ping" ]; then
@@ -39,6 +42,14 @@ done
 echo \"${DOMU_MSG}\"
 "
     DOM0_CMD="
+brctl addbr xenbr0
+brctl addif xenbr0 eth0
+ifconfig eth0 up
+ifconfig xenbr0 up
+ifconfig xenbr0 192.168.0.1
+# get domU console content into test log
+tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
+xl create /etc/xen/domU.cfg
 set +x
 until grep -q \"${DOMU_MSG}\" /var/log/xen/console/guest-domU.log; do
     sleep 1
@@ -46,6 +57,34 @@ done
 set -x
 echo \"${PASS_MSG}\"
 "
+    DOMU_CFG_EXTRA='
+vif = [ "bridge=xenbr0", ]
+disk = [ ]
+'
+elif [ "${TEST}" = "argo" ]
+then
+    PASS_MSG="TEST: Message from DOMU"
+    XEN_CMD_EXTRA="argo=1,mac-permissive=1"
+    DOMU_CMD="
+insmod /root/xen-argo.ko
+until false
+do
+  echo \"${PASS_MSG}\"
+  sleep 1
+done | argo-exec -p 28333 -d 0 -- /bin/echo
+"
+    DOM0_CMD="
+insmod /root/xen-argo.ko
+xl create /etc/xen/domU.cfg
+argo-exec -l -p 28333 -- /bin/echo
+"
+copy_dom0_files ()
+{
+    cp "${WORKDIR}/binaries/xen-argo.ko" "root/"
+    cp -ar "${WORKDIR}/binaries/lib/"* "usr/local/lib/"
+    cp "${WORKDIR}/binaries/argo-exec" "usr/local/bin/"
+}
+copy_domU_files () { copy_dom0_files; }
 else
     fatal "Unknown test: ${TEST}"
 fi
@@ -54,18 +93,18 @@ fi
 mkdir -p rootfs
 cd rootfs
 tar xzf ../binaries/initrd.tar.gz
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
+mkdir proc run srv sys
 rm var/run
 echo "#!/bin/sh
-
+set -x
+export LD_LIBRARY_PATH=/usr/local/lib
+PATH=/usr/local/bin:/usr/local/sbin:\$PATH
 ${DOMU_CMD}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 echo "rc_verbose=yes" >> etc/rc.conf
 sed -i -e 's/^Welcome/domU \0/' etc/issue
+copy_domU_files
 find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
 cd ..
 rm -rf rootfs
@@ -74,37 +113,24 @@ rm -rf rootfs
 mkdir -p rootfs
 cd rootfs
 tar xzf ../binaries/initrd.tar.gz
-mkdir boot
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
+mkdir boot proc run srv sys
 rm var/run
 cp -ar ../binaries/dist/install/* .
 echo "#!/bin/bash
-
+set -x
 export LD_LIBRARY_PATH=/usr/local/lib
 bash /etc/init.d/xencommons start
-
-brctl addbr xenbr0
-brctl addif xenbr0 eth0
-ifconfig eth0 up
-ifconfig xenbr0 up
-ifconfig xenbr0 192.168.0.1
-
-# get domU console content into test log
-tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
-xl create /etc/xen/domU.cfg
 ${DOM0_CMD}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "${DOMU_CFG}" > etc/xen/domU.cfg
+echo "${DOMU_CFG}${DOMU_CFG_EXTRA}" > etc/xen/domU.cfg
 echo "rc_verbose=yes" >> etc/rc.conf
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
 cp ../binaries/bzImage boot/vmlinuz
 cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
+copy_dom0_files
 find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
 cd ..

@@ -116,7 +142,7 @@ cp -f binaries/bzImage ${TFTP}/${TEST_BOARD}/vmlinuz
 cp -f binaries/dom0-rootfs.cpio.gz ${TFTP}/${TEST_BOARD}/initrd-dom0
 echo "
 net_default_server=10.0.6.1
-multiboot2 (tftp)/${TEST_BOARD}/xen ${XEN_CMDLINE}
+multiboot2 (tftp)/${TEST_BOARD}/xen ${XEN_CMDLINE} sync_console
 module2 (tftp)/${TEST_BOARD}/vmlinuz console=hvc0 root=/dev/ram0 earlyprintk=xen
 module2 (tftp)/${TEST_BOARD}/initrd-dom0
 boot
--
2.25.1

Re: [XEN PATCH v1 0/3] automation: add x86_64 test (linux argo)

[Andrew Cooper]

On 17/10/2024 6:18 pm, victorm.lira@amd.com wrote:
> From: Victor Lira <victorm.lira@amd.com>
>
> Add x86_64 hardware test that creates a Xen Argo communication
> connection between two PVH domains.
>
> To accomplish this, add new build artifacts for Linux and Argo, and
> update the xilinx x86_64 test script.
>
> Victor Lira (3):
>   automation: add linux 6.6.56 artifact
>   automation: add linux argo test artifacts
>   automation: add x86_64 test (linux argo)
>
>  automation/gitlab-ci/build.yaml               | 34 +++++++
>  automation/gitlab-ci/test.yaml                | 10 ++
>  .../scripts/xilinx-smoke-dom0-x86_64.sh       | 76 ++++++++++-----
>  .../tests-artifacts/argo/6.6.56.dockerfile    | 95 +++++++++++++++++++
>  .../tests-artifacts/kernel/6.6.56.dockerfile  | 54 +++++++++++
>  5 files changed, 244 insertions(+), 25 deletions(-)
>  create mode 100644 automation/tests-artifacts/argo/6.6.56.dockerfile
>  create mode 100644 automation/tests-artifacts/kernel/6.6.56.dockerfile

I'm afraid that we need to stop doing this test-artefacts like this.

The *-export pattern is nonsense (wasting lots of runner network&disk
and time just to copy data back into Github's artefact store), and the
main reason why we're at ~2T of data total for Xen-project.

We need a project wide expires_in: setting, which will prune all of our
not-most-recent content, and probably make most of that 2T disappear.

But, the test jobs stating what artefacts they want are perfectly
capable of pulling artefacts from somewhere other than an earlier nop
build job, meaning that we could get away with having one artefact
shared by everything, not one artefact copied per user per pipeline into
the artefact store.


I was thinking of experimenting with a separate top-level repo that does
nothing but has a few manual runs to populate artefacts, and having the
Xen tests pull artefacts from here rather than from earlier build jobs.

But, I've not had a chance to play yet, so I don't know for sure if this
is a workable direction.

Alternatively, if anyone has a better idea, I'm all ears.  But, it is
very clear that the *-export pattern is a giant bodge and there are
better ways.


Other notes.  Your dockerfiles use syntax:1 but not heredocs, and
they're also root containers.

Please follow the pattern in
https://lore.kernel.org/xen-devel/178560672106e37648304f5e597cc0b16c8db6db.1729170005.git.javi.merino@cloud.com/T/#u
or one of plenty other containers I've already converted in the past few
months.

Please do not add a new build job just to get a new minor variation of
Xen.  Just enable ARGO in general Xen build.

Although, given that you are now putting in testing from an unsupported
feature, we really need to bump the suggestion to move it to at least
Tech Preview.  (CC Dan/Chris).

~Andrew

Re: [XEN PATCH v1 0/3] automation: add x86_64 test (linux argo)

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 3392 bytes --]

On Thu, 17 Oct 2024, Andrew Cooper wrote:
> On 17/10/2024 6:18 pm, victorm.lira@amd.com wrote:
> > From: Victor Lira <victorm.lira@amd.com>
> >
> > Add x86_64 hardware test that creates a Xen Argo communication
> > connection between two PVH domains.
> >
> > To accomplish this, add new build artifacts for Linux and Argo, and
> > update the xilinx x86_64 test script.
> >
> > Victor Lira (3):
> >   automation: add linux 6.6.56 artifact
> >   automation: add linux argo test artifacts
> >   automation: add x86_64 test (linux argo)
> >
> >  automation/gitlab-ci/build.yaml               | 34 +++++++
> >  automation/gitlab-ci/test.yaml                | 10 ++
> >  .../scripts/xilinx-smoke-dom0-x86_64.sh       | 76 ++++++++++-----
> >  .../tests-artifacts/argo/6.6.56.dockerfile    | 95 +++++++++++++++++++
> >  .../tests-artifacts/kernel/6.6.56.dockerfile  | 54 +++++++++++
> >  5 files changed, 244 insertions(+), 25 deletions(-)
> >  create mode 100644 automation/tests-artifacts/argo/6.6.56.dockerfile
> >  create mode 100644 automation/tests-artifacts/kernel/6.6.56.dockerfile
> 
> I'm afraid that we need to stop doing this test-artefacts like this.
> 
> The *-export pattern is nonsense (wasting lots of runner network&disk
> and time just to copy data back into Github's artefact store), and the
> main reason why we're at ~2T of data total for Xen-project.
> 
> We need a project wide expires_in: setting, which will prune all of our
> not-most-recent content, and probably make most of that 2T disappear.

I looked at this for an hour and couldn't find a way to do it
unfortunately. expires_in can only be changed if you are an Admin of the
Gitlab instance, meaning an Admin of gitlab.com.

However, there is a setting in each repository (not one global at the
gitlab.com/xen-project level) called "Keep artifacts from most recent
successful jobs" that can be disabled. I did it manually for my repo.

The big offender is patchew. I went and disabled "Keep artifacts from
most recent successful jobs" but I didn't see any immediate changes in
storage utilization. But it is expected to take time to propagate.


> But, the test jobs stating what artefacts they want are perfectly
> capable of pulling artefacts from somewhere other than an earlier nop
> build job, meaning that we could get away with having one artefact
> shared by everything, not one artefact copied per user per pipeline into
> the artefact store.
> 
> 
> I was thinking of experimenting with a separate top-level repo that does
> nothing but has a few manual runs to populate artefacts, and having the
> Xen tests pull artefacts from here rather than from earlier build jobs.
> 
> But, I've not had a chance to play yet, so I don't know for sure if this
> is a workable direction.
>
> Alternatively, if anyone has a better idea, I'm all ears.  But, it is
> very clear that the *-export pattern is a giant bodge and there are
> better ways.
> 
> Other notes.  Your dockerfiles use syntax:1 but not heredocs, and
> they're also root containers.
> 
> Please follow the pattern in
> https://lore.kernel.org/xen-devel/178560672106e37648304f5e597cc0b16c8db6db.1729170005.git.javi.merino@cloud.com/T/#u
> or one of plenty other containers I've already converted in the past few
> months.
> 
> Please do not add a new build job just to get a new minor variation of
> Xen.  Just enable ARGO in general Xen build.

+1

Re: [XEN PATCH v1 0/3] automation: add x86_64 test (linux argo)

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 454 bytes --]

On Thu, Oct 17, 2024 at 08:25:57PM +0100, Andrew Cooper wrote:
> I was thinking of experimenting with a separate top-level repo that does
> nothing but has a few manual runs to populate artefacts, and having the
> Xen tests pull artefacts from here rather than from earlier build jobs.

This sounds like a good idea. It may also help with rebuilding
containers when needed?

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]