[PATCH v3 0/6] CI: switch to CPIO archives

[PATCH v3 0/6] CI: switch to CPIO archives

[Andrew Cooper]

This is an amalgamation of Marek's work and mine, tested at each stage on all
hardware runners.  See patches for details.

Andrew Cooper (3):
  CI: Rename intermediate artefacts in qemu-* scripts
  CI: Switch to new argo artefact
  CI: Include microcode for x86 hardware jobs

Marek Marczykowski-Górecki (3):
  CI: avoid repacking initrd as part of the test job
  CI: remove now unused alpine-3.18-arm64-rootfs job and its container
  CI: save toolstack artifact as cpio.gz

 automation/gitlab-ci/build.yaml               | 12 +---
 automation/gitlab-ci/test.yaml                | 16 +++--
 automation/scripts/build                      |  2 +-
 automation/scripts/qemu-alpine-x86_64.sh      | 32 ++++-----
 automation/scripts/qemu-smoke-dom0-arm64.sh   | 30 ++++-----
 .../scripts/qemu-smoke-dom0less-arm64.sh      | 17 ++---
 automation/scripts/qubes-x86-64.sh            | 37 +++++------
 .../scripts/xilinx-smoke-dom0-x86_64.sh       | 50 +++++++-------
 .../scripts/xilinx-smoke-dom0less-arm64.sh    | 33 ++++------
 .../alpine/3.18-arm64v8.dockerfile            | 66 -------------------
 10 files changed, 101 insertions(+), 194 deletions(-)
 delete mode 100644 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile


base-commit: f401eae6ec446671ad79deac84421d56808d43f2
-- 
2.39.5

[PATCH v3 1/6] CI: Rename intermediate artefacts in qemu-* scripts

[Andrew Cooper]

Right now, we have initrd.cpio.gz as domU, and initrd.tar.gz as the base for
dom0.

Rename initrd.cpio.gz to domU-rootfs.cpio.gz, and xen-rootfs.cpio.gz to
dom0-rootfs.cpio.gz to make it clearer which is which.  Rename the VM from
test to domU.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

v3:
 * New

https://gitlab.com/xen-project/hardware/xen-staging/-/pipelines/1765822634
---
 automation/scripts/qemu-alpine-x86_64.sh    | 16 ++++++++--------
 automation/scripts/qemu-smoke-dom0-arm64.sh | 16 ++++++++--------
 2 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 89bdb4df7dac..569bd766d31e 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -25,7 +25,7 @@ mount -t devtmpfs devtmpfs /dev
 chmod +x initrd/init
 # DomU rootfs
 cd initrd
-find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+find . | cpio -H newc -o | gzip > ../domU-rootfs.cpio.gz
 cd ..
 
 # initrd.tar.gz is Dom0 rootfs
@@ -38,15 +38,15 @@ mkdir srv
 mkdir sys
 rm var/run
 cp -ar ../dist/install/* .
-mv ../initrd.cpio.gz ./root
+mv ../domU-rootfs.cpio.gz ./root
 cp ../bzImage ./root
-echo "name=\"test\"
+echo "name=\"domU\"
 memory=512
 vcpus=1
 kernel=\"/root/bzImage\"
-ramdisk=\"/root/initrd.cpio.gz\"
+ramdisk=\"/root/domU-rootfs.cpio.gz\"
 extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\"
-" > root/test.cfg
+" > root/domU.cfg
 echo "#!/bin/bash
 
 set -x
@@ -56,13 +56,13 @@ bash /etc/init.d/xencommons start
 
 xl list
 
-xl -vvv create -c /root/test.cfg
+xl -vvv create -c /root/domU.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 echo "rc_verbose=yes" >> etc/rc.conf
 # rebuild Dom0 rootfs
-find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip > ../dom0-rootfs.cpio.gz
 cd ../..
 
 cat >> binaries/pxelinux.0 << EOF
@@ -70,7 +70,7 @@ cat >> binaries/pxelinux.0 << EOF
 
 kernel xen console=com1 console_timestamps=boot
 module bzImage console=hvc0
-module xen-rootfs.cpio.gz
+module dom0-rootfs.cpio.gz
 boot
 EOF
 
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 51d037b0003e..e8e49ded245a 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -24,7 +24,7 @@ mount -t devtmpfs devtmpfs /dev
 /bin/sh" > initrd/init
 chmod +x initrd/init
 cd initrd
-find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+find . | cpio -H newc -o | gzip > ../domU-rootfs.cpio.gz
 cd ..
 
 mkdir -p rootfs
@@ -36,15 +36,15 @@ mkdir srv
 mkdir sys
 rm var/run
 cp -ar ../dist/install/* .
-mv ../initrd.cpio.gz ./root
+mv ../domU-rootfs.cpio.gz ./root
 cp ../Image ./root
-echo "name=\"test\"
+echo "name=\"domU\"
 memory=512
 vcpus=1
 kernel=\"/root/Image\"
-ramdisk=\"/root/initrd.cpio.gz\"
+ramdisk=\"/root/domU-rootfs.cpio.gz\"
 extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\"
-" > root/test.cfg
+" > root/domU.cfg
 echo "#!/bin/bash
 
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -52,12 +52,12 @@ bash /etc/init.d/xencommons start
 
 xl list
 
-xl -vvv create -c /root/test.cfg
+xl -vvv create -c /root/domU.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 echo "rc_verbose=yes" >> etc/rc.conf
-find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip > ../dom0-rootfs.cpio.gz
 cd ../..
 
 # XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
@@ -78,7 +78,7 @@ MEMORY_END="0xC0000000"
 DEVICE_TREE="virt-gicv2.dtb"
 XEN="xen"
 DOM0_KERNEL="Image"
-DOM0_RAMDISK="xen-rootfs.cpio.gz"
+DOM0_RAMDISK="dom0-rootfs.cpio.gz"
 XEN_CMD="console=dtuart dom0_mem=1024M console_timestamps=boot"
 
 NUM_DOMUS=0
-- 
2.39.5

[PATCH v3 2/6] CI: avoid repacking initrd as part of the test job

[Andrew Cooper]

From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Use the new test-artifacts which provide rootfs.cpio.gz rather than
initrd.tar.gz.  rootfs.cpio.gz also has all the necessary top-level
directories, and includes the rc_verbose setting, so these modifications can
be dropped.

Having that, do not repack the whole initrd, but only pack modified
files and rely on Linux handling of concatenated archives.
This allows packing just test-related files (which includes the whole
toolstack), instead of the whole initrd.

For xilinx-smoke-dom0-x86_64.sh, this involves instructing grub not to unzip
the archive, as doing so corrupts it.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

v3:
 * Tested (bugfixed) on xilinx-* runners
 * Rearrange logic so the order of concantination is clearer (relevant for
   subsequent patches)

https://gitlab.com/xen-project/hardware/xen-staging/-/pipelines/1765676583
---
 automation/gitlab-ci/test.yaml                |  8 +++--
 automation/scripts/qemu-alpine-x86_64.sh      | 16 ++++------
 automation/scripts/qemu-smoke-dom0-arm64.sh   | 14 ++++----
 .../scripts/qemu-smoke-dom0less-arm64.sh      | 15 ++++-----
 automation/scripts/qubes-x86-64.sh            | 32 +++++++------------
 .../scripts/xilinx-smoke-dom0-x86_64.sh       | 27 ++++++++--------
 .../scripts/xilinx-smoke-dom0less-arm64.sh    | 30 +++++++----------
 7 files changed, 61 insertions(+), 81 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 59a2de28c864..51229cbe561d 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -11,7 +11,9 @@
   - project: xen-project/hardware/test-artifacts
     job: linux-6.6.86-arm64
     ref: master
-  - alpine-3.18-arm64-rootfs-export
+  - project: xen-project/hardware/test-artifacts
+    job: alpine-3.18-arm64-rootfs
+    ref: master
   - qemu-system-aarch64-6.0.0-arm64-export
 
 .arm32-test-needs: &arm32-test-needs
@@ -22,7 +24,7 @@
     job: linux-6.6.56-x86_64
     ref: master
   - project: xen-project/hardware/test-artifacts
-    job: x86_64-rootfs-alpine-3.18
+    job: alpine-3.18-x86_64-rootfs
     ref: master
 
 .qemu-arm64:
@@ -243,7 +245,7 @@ xilinx-smoke-dom0-x86_64-gcc-debug-argo:
       job: x86_64-kernel-linux-6.6.56
       ref: master
     - project: xen-project/hardware/test-artifacts
-      job: x86_64-rootfs-alpine-3.18
+      job: alpine-3.18-x86_64-rootfs
       ref: master
     - project: xen-project/hardware/test-artifacts
       job: x86_64-argo-linux-6.6.56
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 569bd766d31e..c7dd12197862 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -28,16 +28,14 @@ cd initrd
 find . | cpio -H newc -o | gzip > ../domU-rootfs.cpio.gz
 cd ..
 
-# initrd.tar.gz is Dom0 rootfs
+# Dom0 rootfs
+cp rootfs.cpio.gz dom0-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xvzf ../initrd.tar.gz
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
-rm var/run
 cp -ar ../dist/install/* .
+mkdir -p root etc/local.d
 mv ../domU-rootfs.cpio.gz ./root
 cp ../bzImage ./root
 echo "name=\"domU\"
@@ -60,9 +58,7 @@ xl -vvv create -c /root/domU.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-# rebuild Dom0 rootfs
-find . | cpio -H newc -o | gzip > ../dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../dom0-rootfs.cpio.gz
 cd ../..
 
 cat >> binaries/pxelinux.0 << EOF
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index e8e49ded245a..c0cf61ff8f7b 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -27,15 +27,14 @@ cd initrd
 find . | cpio -H newc -o | gzip > ../domU-rootfs.cpio.gz
 cd ..
 
+# Dom0 rootfs
+cp rootfs.cpio.gz dom0-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xvzf ../initrd.tar.gz
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
-rm var/run
 cp -ar ../dist/install/* .
+mkdir -p etc/local.d root
 mv ../domU-rootfs.cpio.gz ./root
 cp ../Image ./root
 echo "name=\"domU\"
@@ -56,8 +55,7 @@ xl -vvv create -c /root/domU.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../dom0-rootfs.cpio.gz
 cd ../..
 
 # XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index f72d20936181..8e939f0b7214 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -114,16 +114,14 @@ cd initrd
 find . | cpio --create --format='newc' | gzip > ../binaries/initrd
 cd ..
 
-# DOM0 rootfs
+# Dom0 rootfs
+cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
-rm var/run
 cp -ar ../binaries/dist/install/* .
+mkdir -p etc/local.d
 
 echo "#!/bin/bash
 
@@ -142,8 +140,7 @@ xl network-attach 1 type=vif
 ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 # ImageBuilder
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index c4be71464246..e2376cb51cb4 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -164,16 +164,13 @@ ${domU_extra_config}
 "
 
 if [ -n "$domU_check" ]; then
-    # DomU
+    # DomU rootfs
+    cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
+
+    # test-local configuration
     mkdir -p rootfs
     cd rootfs
-    # fakeroot is needed to preserve device nodes in rootless podman container
-    fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
-    mkdir proc
-    mkdir run
-    mkdir srv
-    mkdir sys
-    rm var/run
+    mkdir -p etc/local.d
     echo "#!/bin/sh
 
 echo 8 > /proc/sys/kernel/printk
@@ -181,26 +178,22 @@ echo 8 > /proc/sys/kernel/printk
 ${domU_check}
 " > etc/local.d/xen.start
     chmod +x etc/local.d/xen.start
-    echo "rc_verbose=yes" >> etc/rc.conf
     echo "domU Welcome to Alpine Linux
 Kernel \r on an \m (\l)
 
 " > etc/issue
-    find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
+    find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
     cd ..
     rm -rf rootfs
 fi
 
-# DOM0 rootfs
+# Dom0 rootfs
+cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
-mkdir boot
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
-rm var/run
+mkdir -p boot etc/local.d
 cp -ar ../binaries/dist/install/* .
 cp -ar ../binaries/tests .
 cp -a ../automation/scripts/run-tools-tests tests/
@@ -237,7 +230,6 @@ fi
 chmod +x etc/local.d/xen.start
 echo "$domU_config" > etc/xen/domU.cfg
 
-echo "rc_verbose=yes" >> etc/rc.conf
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
@@ -245,7 +237,7 @@ cp ../binaries/bzImage boot/vmlinuz
 if [ -n "$domU_check" ]; then
     cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
 fi
-find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 7e5382931b85..1fc52d78db16 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -80,6 +80,7 @@ argo-exec -l -p 28333 -- /bin/echo
 "
 copy_dom0_files ()
 {
+    mkdir -p root usr/local/lib usr/local/bin
     cp "${WORKDIR}/binaries/xen-argo.ko" "root/"
     cp -ar "${WORKDIR}/binaries/lib/"* "usr/local/lib/"
     cp "${WORKDIR}/binaries/argo-exec" "usr/local/bin/"
@@ -89,12 +90,13 @@ else
     fatal "Unknown test: ${TEST}"
 fi
 
-# Set up domU rootfs.
+# DomU rootfs
+cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
-mkdir proc run srv sys
-rm var/run
+mkdir -p etc/local.d
 echo "#!/bin/sh
 set -x
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -102,22 +104,22 @@ PATH=/usr/local/bin:/usr/local/sbin:\$PATH
 ${DOMU_CMD}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
 echo "domU Welcome to Alpine Linux
 Kernel \r on an \m (\l)
 
 " > etc/issue
 copy_domU_files
-find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
 cd ..
 rm -rf rootfs
 
-# Set up dom0 rootfs.
+# Dom0 rootfs
+cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
-mkdir boot proc run srv sys
-rm var/run
+mkdir -p boot etc/local.d
 cp -ar ../binaries/dist/install/* .
 echo "#!/bin/bash
 set -x
@@ -127,14 +129,13 @@ ${DOM0_CMD}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 echo "${DOMU_CFG}${DOMU_CFG_EXTRA}" > etc/xen/domU.cfg
-echo "rc_verbose=yes" >> etc/rc.conf
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
 cp ../binaries/bzImage boot/vmlinuz
 cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
 copy_dom0_files
-find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 # Load software into TFTP server directory.
@@ -147,7 +148,7 @@ echo "
 net_default_server=10.0.6.1
 multiboot2 (tftp)/${TEST_BOARD}/xen ${XEN_CMDLINE} sync_console
 module2 (tftp)/${TEST_BOARD}/vmlinuz console=hvc0 root=/dev/ram0 earlyprintk=xen
-module2 (tftp)/${TEST_BOARD}/initrd-dom0
+module2 --nounzip (tftp)/${TEST_BOARD}/initrd-dom0
 boot
 " > ${TFTP}/${TEST_BOARD}/grub.cfg
 
diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
index 3e1fcf6bf93c..a3031ecab0dc 100755
--- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
+++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
@@ -38,36 +38,31 @@ echo \"${passed}\"
 "
 fi
 
-# DomU
+# DomU rootfs
+cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
-rm var/run
+mkdir -p etc/local.d
 echo "#!/bin/sh
 
 ${domU_check}
 /bin/sh" > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
 cd ..
 rm -rf rootfs
 
-# DOM0 rootfs
+# Dom0 rootfs
+cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+
+# test-local configuration
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
-mkdir proc
-mkdir run
-mkdir srv
-mkdir sys
-rm var/run
 cp -ar ../binaries/dist/install/* .
 
+mkdir -p etc/local.d
 echo "#!/bin/bash
 
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -78,8 +73,7 @@ bash /etc/init.d/xencommons start
 ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 
-- 
2.39.5

[PATCH v3 3/6] CI: remove now unused alpine-3.18-arm64-rootfs job and its container

[Andrew Cooper]

From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

This got moved to test-artifacts.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 automation/gitlab-ci/build.yaml               | 11 ----
 .../alpine/3.18-arm64v8.dockerfile            | 66 -------------------
 2 files changed, 77 deletions(-)
 delete mode 100644 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 40dcd31f1bce..1b82b359d01f 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -264,17 +264,6 @@
 
 # Arm test artifacts
 
-alpine-3.18-arm64-rootfs-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18-arm64v8
-  script:
-    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
-  artifacts:
-    paths:
-      - binaries/initrd.tar.gz
-  tags:
-    - arm64
-
 qemu-system-aarch64-6.0.0-arm64-export:
   extends: .test-jobs-artifact-common
   image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-aarch64:6.0.0-arm64v8
diff --git a/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile b/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
deleted file mode 100644
index a4542f703997..000000000000
--- a/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
+++ /dev/null
@@ -1,66 +0,0 @@
-# syntax=docker/dockerfile:1
-FROM --platform=linux/arm64/v8 alpine:3.18
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="xen-devel@lists.xenproject.org"
-
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-RUN \
-  # apk
-  apk update && \
-  \
-  # xen runtime deps
-  apk add musl && \
-  apk add openrc && \
-  apk add busybox && \
-  apk add sudo && \
-  apk add dbus && \
-  apk add bash && \
-  apk add python3 && \
-  apk add zlib && \
-  apk add ncurses && \
-  apk add yajl && \
-  apk add libaio && \
-  apk add xz && \
-  apk add util-linux && \
-  apk add argp-standalone && \
-  apk add libfdt && \
-  apk add glib && \
-  apk add pixman && \
-  apk add curl && \
-  apk add udev && \
-  \
-  # Xen
-  cd / && \
-  # Minimal ramdisk environment in case of cpio output
-  rc-update add udev && \
-  rc-update add udev-trigger && \
-  rc-update add udev-settle && \
-  rc-update add networking sysinit && \
-  rc-update add loopback sysinit && \
-  rc-update add bootmisc boot && \
-  rc-update add devfs sysinit && \
-  rc-update add dmesg sysinit && \
-  rc-update add hostname boot && \
-  rc-update add hwclock boot && \
-  rc-update add hwdrivers sysinit && \
-  rc-update add killprocs shutdown && \
-  rc-update add modloop sysinit && \
-  rc-update add modules boot && \
-  rc-update add mount-ro shutdown && \
-  rc-update add savecache shutdown && \
-  rc-update add sysctl boot && \
-  rc-update add local default && \
-  cp -a /sbin/init /init && \
-  echo "ttyS0" >> /etc/securetty && \
-  echo "hvc0" >> /etc/securetty && \
-  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
-  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
-  passwd -d "root" root && \
-  \
-  # Create rootfs
-  cd / && \
-  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
-- 
2.39.5

[PATCH v3 4/6] CI: Switch to new argo artefact

[Andrew Cooper]

The argo artefact is now a cpio archive and can be appended to dom?-rootfs
directly, rather than being part of of the overlay.  Drop the
copy_dom?_files() functions.

xen-argo.ko has moved into the standard location, so update the insmod path.

No practical change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Jason Andryuk <jason.andryuk@amd.com>
CC: Daniel P. Smith <dpsmith@apertussolutions.com>

v3:
 * Rebase over change to rootfs handling

https://gitlab.com/xen-project/hardware/xen-staging/-/pipelines/1766180593
---
 automation/gitlab-ci/test.yaml                |  2 +-
 .../scripts/xilinx-smoke-dom0-x86_64.sh       | 22 +++++++------------
 2 files changed, 9 insertions(+), 15 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 51229cbe561d..d46da1c43d05 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -242,7 +242,7 @@ xilinx-smoke-dom0-x86_64-gcc-debug-argo:
   needs:
     - alpine-3.18-gcc-debug
     - project: xen-project/hardware/test-artifacts
-      job: x86_64-kernel-linux-6.6.56
+      job: linux-6.6.56-x86_64
       ref: master
     - project: xen-project/hardware/test-artifacts
       job: alpine-3.18-x86_64-rootfs
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 1fc52d78db16..85007dcc290a 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -28,8 +28,6 @@ extra = "root=/dev/ram0 console=hvc0"
 memory = 512
 '
 DOMU_CFG_EXTRA=""
-copy_domU_files () { :; }
-copy_dom0_files () { :; }
 
 # Select test variant.
 if [ "${TEST}" = "ping" ]; then
@@ -66,7 +64,7 @@ then
     PASS_MSG="TEST: Message from DOMU"
     XEN_CMD_EXTRA="argo=1,mac-permissive=1"
     DOMU_CMD="
-insmod /root/xen-argo.ko
+insmod /lib/modules/\$(uname -r)/updates/xen-argo.ko
 until false
 do
   echo \"${PASS_MSG}\"
@@ -74,24 +72,19 @@ do
 done | argo-exec -p 28333 -d 0 -- /bin/echo
 "
     DOM0_CMD="
-insmod /root/xen-argo.ko
+insmod /lib/modules/\$(uname -r)/updates/xen-argo.ko
 xl -vvv create /etc/xen/domU.cfg
 argo-exec -l -p 28333 -- /bin/echo
 "
-copy_dom0_files ()
-{
-    mkdir -p root usr/local/lib usr/local/bin
-    cp "${WORKDIR}/binaries/xen-argo.ko" "root/"
-    cp -ar "${WORKDIR}/binaries/lib/"* "usr/local/lib/"
-    cp "${WORKDIR}/binaries/argo-exec" "usr/local/bin/"
-}
-copy_domU_files () { copy_dom0_files; }
 else
     fatal "Unknown test: ${TEST}"
 fi
 
 # DomU rootfs
 cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
+if [[ "${TEST}" == argo ]]; then
+    cat binaries/argo.cpio.gz >> binaries/domU-rootfs.cpio.gz
+fi
 
 # test-local configuration
 mkdir -p rootfs
@@ -108,13 +101,15 @@ echo "domU Welcome to Alpine Linux
 Kernel \r on an \m (\l)
 
 " > etc/issue
-copy_domU_files
 find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
 cd ..
 rm -rf rootfs
 
 # Dom0 rootfs
 cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+if [[ "${TEST}" == argo ]]; then
+    cat binaries/argo.cpio.gz >> binaries/dom0-rootfs.cpio.gz
+fi
 
 # test-local configuration
 mkdir -p rootfs
@@ -134,7 +129,6 @@ echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
 cp ../binaries/bzImage boot/vmlinuz
 cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
-copy_dom0_files
 find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
-- 
2.39.5

[PATCH v3 5/6] CI: save toolstack artifact as cpio.gz

[Andrew Cooper]

From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

This avoids the need to re-compress it in every test job.  This saves minutes
of wallclock time.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

v3:
 * Rebase
 * Rename to xen-tools.cpio.gz to be more specific

https://gitlab.com/xen-project/hardware/xen-staging/-/pipelines/1766186423
---
 automation/scripts/build                          | 2 +-
 automation/scripts/qemu-alpine-x86_64.sh          | 2 +-
 automation/scripts/qemu-smoke-dom0-arm64.sh       | 2 +-
 automation/scripts/qemu-smoke-dom0less-arm64.sh   | 2 +-
 automation/scripts/qubes-x86-64.sh                | 4 +++-
 automation/scripts/xilinx-smoke-dom0-x86_64.sh    | 4 ++--
 automation/scripts/xilinx-smoke-dom0less-arm64.sh | 3 +--
 7 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/automation/scripts/build b/automation/scripts/build
index 522efe774ef3..51effec81ddd 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -108,7 +108,7 @@ else
     # Note: Some smoke tests depending on finding binaries/xen on a full build
     # even though dist/ contains everything, while some containers don't even
     # build Xen
-    cp -r dist binaries/
+    (cd dist/install; find | cpio -o -H newc | gzip) > binaries/xen-tools.cpio.gz
     cp -r tools/tests binaries/
     collect_xen_artefacts
 fi
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index c7dd12197862..fe86caa39304 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -30,11 +30,11 @@ cd ..
 
 # Dom0 rootfs
 cp rootfs.cpio.gz dom0-rootfs.cpio.gz
+cat xen-tools.cpio.gz >> dom0-rootfs.cpio.gz
 
 # test-local configuration
 mkdir -p rootfs
 cd rootfs
-cp -ar ../dist/install/* .
 mkdir -p root etc/local.d
 mv ../domU-rootfs.cpio.gz ./root
 cp ../bzImage ./root
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index c0cf61ff8f7b..61dd24b9abef 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -29,11 +29,11 @@ cd ..
 
 # Dom0 rootfs
 cp rootfs.cpio.gz dom0-rootfs.cpio.gz
+cat xen-tools.cpio.gz >> dom0-rootfs.cpio.gz
 
 # test-local configuration
 mkdir -p rootfs
 cd rootfs
-cp -ar ../dist/install/* .
 mkdir -p etc/local.d root
 mv ../domU-rootfs.cpio.gz ./root
 cp ../Image ./root
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index 8e939f0b7214..58f11dd31038 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -116,11 +116,11 @@ cd ..
 
 # Dom0 rootfs
 cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 
 # test-local configuration
 mkdir -p rootfs
 cd rootfs
-cp -ar ../binaries/dist/install/* .
 mkdir -p etc/local.d
 
 echo "#!/bin/bash
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index e2376cb51cb4..3947027d6e4d 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -189,12 +189,12 @@ fi
 
 # Dom0 rootfs
 cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 
 # test-local configuration
 mkdir -p rootfs
 cd rootfs
 mkdir -p boot etc/local.d
-cp -ar ../binaries/dist/install/* .
 cp -ar ../binaries/tests .
 cp -a ../automation/scripts/run-tools-tests tests/
 
@@ -228,8 +228,10 @@ else
 fi
 
 chmod +x etc/local.d/xen.start
+mkdir -p etc/xen
 echo "$domU_config" > etc/xen/domU.cfg
 
+mkdir -p etc/default
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 85007dcc290a..1f8cc172c630 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -107,6 +107,7 @@ rm -rf rootfs
 
 # Dom0 rootfs
 cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 if [[ "${TEST}" == argo ]]; then
     cat binaries/argo.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 fi
@@ -114,8 +115,7 @@ fi
 # test-local configuration
 mkdir -p rootfs
 cd rootfs
-mkdir -p boot etc/local.d
-cp -ar ../binaries/dist/install/* .
+mkdir -p boot etc/local.d etc/xen etc/default
 echo "#!/bin/bash
 set -x
 export LD_LIBRARY_PATH=/usr/local/lib
diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
index a3031ecab0dc..59f3fe7edb3e 100755
--- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
+++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
@@ -56,12 +56,11 @@ rm -rf rootfs
 
 # Dom0 rootfs
 cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 
 # test-local configuration
 mkdir -p rootfs
 cd rootfs
-cp -ar ../binaries/dist/install/* .
-
 mkdir -p etc/local.d
 echo "#!/bin/bash
 
-- 
2.39.5

[PATCH v3 6/6] CI: Include microcode for x86 hardware jobs

[Andrew Cooper]

All the x86 hardware runners are out of date, to varying degrees, in terms of
microcode.

Microcode must be uncompressed and prepended to the initrd in order for Xen to
be able to use it during early boot.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

v3:
 * New

https://gitlab.com/xen-project/hardware/xen-staging/-/pipelines/1766187024
---
 automation/gitlab-ci/build.yaml                | 1 +
 automation/gitlab-ci/test.yaml                 | 6 ++++++
 automation/scripts/qubes-x86-64.sh             | 3 ++-
 automation/scripts/xilinx-smoke-dom0-x86_64.sh | 3 ++-
 4 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 1b82b359d01f..ac5367874526 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -306,6 +306,7 @@ alpine-3.18-gcc-debug:
       CONFIG_ARGO=y
       CONFIG_UBSAN=y
       CONFIG_UBSAN_FATAL=y
+      CONFIG_UCODE_SCAN_DEFAULT=y
       CONFIG_XHCI=y
 
 debian-12-x86_64-gcc-debug:
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index d46da1c43d05..ef17def0120a 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -26,6 +26,9 @@
   - project: xen-project/hardware/test-artifacts
     job: alpine-3.18-x86_64-rootfs
     ref: master
+  - project: xen-project/hardware/test-artifacts
+    job: microcode-x86_64
+    ref: master
 
 .qemu-arm64:
   extends: .test-jobs-common
@@ -250,6 +253,9 @@ xilinx-smoke-dom0-x86_64-gcc-debug-argo:
     - project: xen-project/hardware/test-artifacts
       job: x86_64-argo-linux-6.6.56
       ref: master
+    - project: xen-project/hardware/test-artifacts
+      job: microcode-x86_64
+      ref: master
 
 adl-smoke-x86-64-gcc-debug:
   extends: .adl-x86-64
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 3947027d6e4d..1f90e7002c73 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -188,7 +188,8 @@ Kernel \r on an \m (\l)
 fi
 
 # Dom0 rootfs
-cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+cp binaries/ucode.cpio binaries/dom0-rootfs.cpio.gz
+cat binaries/rootfs.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 
 # test-local configuration
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 1f8cc172c630..5bb44188dc3d 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -106,7 +106,8 @@ cd ..
 rm -rf rootfs
 
 # Dom0 rootfs
-cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
+cp binaries/ucode.cpio binaries/dom0-rootfs.cpio.gz
+cat binaries/rootfs.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 if [[ "${TEST}" == argo ]]; then
     cat binaries/argo.cpio.gz >> binaries/dom0-rootfs.cpio.gz
-- 
2.39.5

[PATCH RFC 7/6] CI: Adjust how domU is packaged in dom0

[Andrew Cooper]

Package domU in /boot for dom0 and insert into the uncompressed part of dom0's
rootfs, rather than recompressing it as part of the overlay.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Doug Goldstein <cardoe@cardoe.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

A little RFC.  It wants extending to the other tests too.
---
 automation/scripts/qubes-x86-64.sh | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 1f90e7002c73..7ce077dfeaee 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -185,10 +185,22 @@ Kernel \r on an \m (\l)
     find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
     cd ..
     rm -rf rootfs
+
+    # Package domU kernel+rootfs in /boot for dom0 (uncompressed)
+    mkdir -p rootfs/boot
+    cd rootfs
+    cp ../binaries/bzImage boot/vmlinuz
+    cp ../binaries/domU-rootfs.cpio.gz boot/
+    find . | cpio -H newc -o > ../binaries/domU-in-dom0.cpio
+    cd ..
+    rm -rf rootfs
 fi
 
 # Dom0 rootfs
 cp binaries/ucode.cpio binaries/dom0-rootfs.cpio.gz
+if [ -e binaries/domU-in-dom0.cpio ]; then
+    cat binaries/domU-in-dom0.cpio >> binaries/dom0-rootfs.cpio.gz
+fi
 cat binaries/rootfs.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
 
@@ -236,10 +248,6 @@ mkdir -p etc/default
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
-cp ../binaries/bzImage boot/vmlinuz
-if [ -n "$domU_check" ]; then
-    cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
-fi
 find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
-- 
2.39.5

Re: [PATCH v3 1/6] CI: Rename intermediate artefacts in qemu-* scripts

[Anthony PERARD]

On Mon, Apr 14, 2025 at 12:08:58PM +0100, Andrew Cooper wrote:
> Right now, we have initrd.cpio.gz as domU, and initrd.tar.gz as the base for
> dom0.
> 
> Rename initrd.cpio.gz to domU-rootfs.cpio.gz, and xen-rootfs.cpio.gz to
> dom0-rootfs.cpio.gz to make it clearer which is which.  Rename the VM from
> test to domU.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

This also rename some long option into short options without an
explanation, but the change looks fine. (I usually prefer long options
in scripts because it means you don't need to check `man` to figure out
what a command line does.)

Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v3 2/6] CI: avoid repacking initrd as part of the test job

[Anthony PERARD]

On Mon, Apr 14, 2025 at 12:08:59PM +0100, Andrew Cooper wrote:
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> Use the new test-artifacts which provide rootfs.cpio.gz rather than
> initrd.tar.gz.  rootfs.cpio.gz also has all the necessary top-level
> directories, and includes the rc_verbose setting, so these modifications can
> be dropped.
> 
> Having that, do not repack the whole initrd, but only pack modified
> files and rely on Linux handling of concatenated archives.
> This allows packing just test-related files (which includes the whole
> toolstack), instead of the whole initrd.
> 
> For xilinx-smoke-dom0-x86_64.sh, this involves instructing grub not to unzip
> the archive, as doing so corrupts it.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

This patch seems to depends on the other patch series for
"test-artifacts" repo.

I've tried to compare the rootfs fs generated by this repo, and the one
generated in the "test-artifacts" repo, and I think there's a few
changes, at least for the arm64 rootfs as the changes have probably
already sailed for x86_64, namely:

    Those don't exist in the new rootfs:
        rc-update add networking sysinit
        rc-update add modloop sysinit
        rc-update add modules boot
        rc-update add sysctl boot
    There's `echo > /etc/modules` been added.

But I guess none of those matter, as it already works on x86_64 tests.

Overall, looks good, more common code between tests!

Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v3 3/6] CI: remove now unused alpine-3.18-arm64-rootfs job and its container

[Anthony PERARD]

On Mon, Apr 14, 2025 at 12:09:00PM +0100, Andrew Cooper wrote:
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> This got moved to test-artifacts.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v3 4/6] CI: Switch to new argo artefact

[Anthony PERARD]

On Mon, Apr 14, 2025 at 12:09:01PM +0100, Andrew Cooper wrote:
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 51229cbe561d..d46da1c43d05 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -242,7 +242,7 @@ xilinx-smoke-dom0-x86_64-gcc-debug-argo:
>    needs:
>      - alpine-3.18-gcc-debug
>      - project: xen-project/hardware/test-artifacts
> -      job: x86_64-kernel-linux-6.6.56
> +      job: linux-6.6.56-x86_64
>        ref: master
>      - project: xen-project/hardware/test-artifacts
>        job: alpine-3.18-x86_64-rootfs


Don't you need to remove the dependency on "x86_64-argo-linux-6.6.56"
which is just out of context, as I think this is now part of
"linux-6.6.56-x86_64" job.

Besides that:
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v3 4/6] CI: Switch to new argo artefact

[Andrew Cooper]

On 14/04/2025 6:24 pm, Anthony PERARD wrote:
> On Mon, Apr 14, 2025 at 12:09:01PM +0100, Andrew Cooper wrote:
>> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
>> index 51229cbe561d..d46da1c43d05 100644
>> --- a/automation/gitlab-ci/test.yaml
>> +++ b/automation/gitlab-ci/test.yaml
>> @@ -242,7 +242,7 @@ xilinx-smoke-dom0-x86_64-gcc-debug-argo:
>>    needs:
>>      - alpine-3.18-gcc-debug
>>      - project: xen-project/hardware/test-artifacts
>> -      job: x86_64-kernel-linux-6.6.56
>> +      job: linux-6.6.56-x86_64
>>        ref: master
>>      - project: xen-project/hardware/test-artifacts
>>        job: alpine-3.18-x86_64-rootfs
>
> Don't you need to remove the dependency on "x86_64-argo-linux-6.6.56"
> which is just out of context, as I think this is now part of
> "linux-6.6.56-x86_64" job.

Yes.  Sorry, this was a bad rebase taking out my "ref:
andrewcoop-test"'s through the series.

>
> Besides that:
> Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks.

~Andrew

Re: [PATCH v3 2/6] CI: avoid repacking initrd as part of the test job

[Andrew Cooper]

On 14/04/2025 5:57 pm, Anthony PERARD wrote:
> On Mon, Apr 14, 2025 at 12:08:59PM +0100, Andrew Cooper wrote:
>> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>>
>> Use the new test-artifacts which provide rootfs.cpio.gz rather than
>> initrd.tar.gz.  rootfs.cpio.gz also has all the necessary top-level
>> directories, and includes the rc_verbose setting, so these modifications can
>> be dropped.
>>
>> Having that, do not repack the whole initrd, but only pack modified
>> files and rely on Linux handling of concatenated archives.
>> This allows packing just test-related files (which includes the whole
>> toolstack), instead of the whole initrd.
>>
>> For xilinx-smoke-dom0-x86_64.sh, this involves instructing grub not to unzip
>> the archive, as doing so corrupts it.
>>
>> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> This patch seems to depends on the other patch series for
> "test-artifacts" repo.

Yes, there's a large interlink, although the other series is fully
committed now.

>
> I've tried to compare the rootfs fs generated by this repo, and the one
> generated in the "test-artifacts" repo, and I think there's a few
> changes, at least for the arm64 rootfs as the changes have probably
> already sailed for x86_64, namely:
>
>     Those don't exist in the new rootfs:
>         rc-update add networking sysinit
>         rc-update add modloop sysinit
>         rc-update add modules boot
>         rc-update add sysctl boot
>     There's `echo > /etc/modules` been added.
>
> But I guess none of those matter, as it already works on x86_64 tests.

Those changes are discussed in the other series, in the relevant patches.

modules is deliberately removed; we don't build them (properly), and the
scan on boot puts errors onto the console.  Emptying /etc/modules is to
remove af_packet and ipv6, again because we don't have those as modules
(we build ipv6 in).

Modloop seems to be scanning other devices for filesystems.  networking
depends on sysctl, but I can't see any of them being used, and the arm64
tests work fine without them

>
> Overall, looks good, more common code between tests!

Indeed.  We can always put them back in if they turn out to be needed.

> Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks.

~Andrew

Re: [PATCH v3 5/6] CI: save toolstack artifact as cpio.gz

[Anthony PERARD]

On Mon, Apr 14, 2025 at 12:09:02PM +0100, Andrew Cooper wrote:
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> This avoids the need to re-compress it in every test job.  This saves minutes
> of wallclock time.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,

-- 
Anthony PERARD

Re: [PATCH RFC 7/6] CI: Adjust how domU is packaged in dom0

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 2378 bytes --]

On Mon, Apr 14, 2025 at 01:47:15PM +0100, Andrew Cooper wrote:
> Package domU in /boot for dom0 and insert into the uncompressed part of dom0's
> rootfs, rather than recompressing it as part of the overlay.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Anthony PERARD <anthony.perard@vates.tech>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Doug Goldstein <cardoe@cardoe.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> A little RFC.  It wants extending to the other tests too.
> ---
>  automation/scripts/qubes-x86-64.sh | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index 1f90e7002c73..7ce077dfeaee 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -185,10 +185,22 @@ Kernel \r on an \m (\l)
>      find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
>      cd ..
>      rm -rf rootfs
> +
> +    # Package domU kernel+rootfs in /boot for dom0 (uncompressed)
> +    mkdir -p rootfs/boot
> +    cd rootfs
> +    cp ../binaries/bzImage boot/vmlinuz
> +    cp ../binaries/domU-rootfs.cpio.gz boot/

This changes the name of domU rootfs - was initrd-domU, yet the domU.cfg
is not updated.

> +    find . | cpio -H newc -o > ../binaries/domU-in-dom0.cpio
> +    cd ..
> +    rm -rf rootfs
>  fi
>  
>  # Dom0 rootfs
>  cp binaries/ucode.cpio binaries/dom0-rootfs.cpio.gz
> +if [ -e binaries/domU-in-dom0.cpio ]; then
> +    cat binaries/domU-in-dom0.cpio >> binaries/dom0-rootfs.cpio.gz
> +fi
>  cat binaries/rootfs.cpio.gz >> binaries/dom0-rootfs.cpio.gz
>  cat binaries/xen-tools.cpio.gz >> binaries/dom0-rootfs.cpio.gz
>  
> @@ -236,10 +248,6 @@ mkdir -p etc/default
>  echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
>  echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
>  mkdir -p var/log/xen/console
> -cp ../binaries/bzImage boot/vmlinuz
> -if [ -n "$domU_check" ]; then
> -    cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
> -fi
>  find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
>  cd ..
>  
> -- 
> 2.39.5
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v3 6/6] CI: Include microcode for x86 hardware jobs

[Anthony PERARD]

On Mon, Apr 14, 2025 at 12:09:03PM +0100, Andrew Cooper wrote:
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 1b82b359d01f..ac5367874526 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -306,6 +306,7 @@ alpine-3.18-gcc-debug:
>        CONFIG_ARGO=y
>        CONFIG_UBSAN=y
>        CONFIG_UBSAN_FATAL=y
> +      CONFIG_UCODE_SCAN_DEFAULT=y

Is there a change that this patch series gets backported? Because that
new Kconfig option won't exist.

Othewise, patch looks fine:
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v3 6/6] CI: Include microcode for x86 hardware jobs

[Andrew Cooper]

On 14/04/2025 6:45 pm, Anthony PERARD wrote:
> On Mon, Apr 14, 2025 at 12:09:03PM +0100, Andrew Cooper wrote:
>> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
>> index 1b82b359d01f..ac5367874526 100644
>> --- a/automation/gitlab-ci/build.yaml
>> +++ b/automation/gitlab-ci/build.yaml
>> @@ -306,6 +306,7 @@ alpine-3.18-gcc-debug:
>>        CONFIG_ARGO=y
>>        CONFIG_UBSAN=y
>>        CONFIG_UBSAN_FATAL=y
>> +      CONFIG_UCODE_SCAN_DEFAULT=y
> Is there a change

DYM "chance" ?

>  that this patch series gets backported? Because that
> new Kconfig option won't exist.

Yes, I do intend to backport this whole series in due course, and yes,
I'm aware.

> Othewise, patch looks fine:
> Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks.

~Andrew

Re: [PATCH v3 6/6] CI: Include microcode for x86 hardware jobs

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]

On Mon, Apr 14, 2025 at 06:47:07PM +0100, Andrew Cooper wrote:
> On 14/04/2025 6:45 pm, Anthony PERARD wrote:
> > On Mon, Apr 14, 2025 at 12:09:03PM +0100, Andrew Cooper wrote:
> >> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> >> index 1b82b359d01f..ac5367874526 100644
> >> --- a/automation/gitlab-ci/build.yaml
> >> +++ b/automation/gitlab-ci/build.yaml
> >> @@ -306,6 +306,7 @@ alpine-3.18-gcc-debug:
> >>        CONFIG_ARGO=y
> >>        CONFIG_UBSAN=y
> >>        CONFIG_UBSAN_FATAL=y
> >> +      CONFIG_UCODE_SCAN_DEFAULT=y
> > Is there a change
> 
> DYM "chance" ?
> 
> >  that this patch series gets backported? Because that
> > new Kconfig option won't exist.
> 
> Yes, I do intend to backport this whole series in due course, and yes,
> I'm aware.

A more backport-friendly way would be add ucode=scan to xen cmdline.

> > Othewise, patch looks fine:
> > Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
> 
> Thanks.
> 
> ~Andrew

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v3 6/6] CI: Include microcode for x86 hardware jobs

[Andrew Cooper]

On 14/04/2025 6:55 pm, Marek Marczykowski-Górecki wrote:
> On Mon, Apr 14, 2025 at 06:47:07PM +0100, Andrew Cooper wrote:
>> On 14/04/2025 6:45 pm, Anthony PERARD wrote:
>>> On Mon, Apr 14, 2025 at 12:09:03PM +0100, Andrew Cooper wrote:
>>>> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
>>>> index 1b82b359d01f..ac5367874526 100644
>>>> --- a/automation/gitlab-ci/build.yaml
>>>> +++ b/automation/gitlab-ci/build.yaml
>>>> @@ -306,6 +306,7 @@ alpine-3.18-gcc-debug:
>>>>        CONFIG_ARGO=y
>>>>        CONFIG_UBSAN=y
>>>>        CONFIG_UBSAN_FATAL=y
>>>> +      CONFIG_UCODE_SCAN_DEFAULT=y
>>> Is there a change
>> DYM "chance" ?
>>
>>>  that this patch series gets backported? Because that
>>> new Kconfig option won't exist.
>> Yes, I do intend to backport this whole series in due course, and yes,
>> I'm aware.
> A more backport-friendly way would be add ucode=scan to xen cmdline.

Yeah, but they're too long already IMO.

Needing to override defaults to make our CI system useful is a good hint
that the defaults are wrong.

Same for console_timestamps, which isn't even deployed consistently
across the testing.

~Andrew

Re: [PATCH v3 2/6] CI: avoid repacking initrd as part of the test job

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 3446 bytes --]

On Mon, 14 Apr 2025, Andrew Cooper wrote:
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> 
> Use the new test-artifacts which provide rootfs.cpio.gz rather than
> initrd.tar.gz.  rootfs.cpio.gz also has all the necessary top-level
> directories, and includes the rc_verbose setting, so these modifications can
> be dropped.
> 
> Having that, do not repack the whole initrd, but only pack modified
> files and rely on Linux handling of concatenated archives.
> This allows packing just test-related files (which includes the whole
> toolstack), instead of the whole initrd.
> 
> For xilinx-smoke-dom0-x86_64.sh, this involves instructing grub not to unzip
> the archive, as doing so corrupts it.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

[...]


> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> index c4be71464246..e2376cb51cb4 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -164,16 +164,13 @@ ${domU_extra_config}
>  "
>  
>  if [ -n "$domU_check" ]; then
> -    # DomU
> +    # DomU rootfs
> +    cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
> +
> +    # test-local configuration
>      mkdir -p rootfs
>      cd rootfs
> -    # fakeroot is needed to preserve device nodes in rootless podman container
> -    fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
> -    mkdir proc
> -    mkdir run
> -    mkdir srv
> -    mkdir sys
> -    rm var/run
> +    mkdir -p etc/local.d
>      echo "#!/bin/sh

I am worried about the drop of fakeroot because the newly created dirs
and files we'll have user as owner instead of root.

I suggest we fix this for consistency, and cpio -R 0:0 might suffice.


>  echo 8 > /proc/sys/kernel/printk
> @@ -181,26 +178,22 @@ echo 8 > /proc/sys/kernel/printk
>  ${domU_check}
>  " > etc/local.d/xen.start
>      chmod +x etc/local.d/xen.start
> -    echo "rc_verbose=yes" >> etc/rc.conf
>      echo "domU Welcome to Alpine Linux
>  Kernel \r on an \m (\l)
>  
>  " > etc/issue
> -    find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
> +    find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
>      cd ..
>      rm -rf rootfs
>  fi
>  
> -# DOM0 rootfs
> +# Dom0 rootfs
> +cp binaries/rootfs.cpio.gz binaries/dom0-rootfs.cpio.gz
> +
> +# test-local configuration
>  mkdir -p rootfs
>  cd rootfs
> -fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
> -mkdir boot
> -mkdir proc
> -mkdir run
> -mkdir srv
> -mkdir sys
> -rm var/run
> +mkdir -p boot etc/local.d
>  cp -ar ../binaries/dist/install/* .
>  cp -ar ../binaries/tests .
>  cp -a ../automation/scripts/run-tools-tests tests/
> @@ -237,7 +230,6 @@ fi
>  chmod +x etc/local.d/xen.start
>  echo "$domU_config" > etc/xen/domU.cfg
>  
> -echo "rc_verbose=yes" >> etc/rc.conf
>  echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
>  echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
>  mkdir -p var/log/xen/console
> @@ -245,7 +237,7 @@ cp ../binaries/bzImage boot/vmlinuz
>  if [ -n "$domU_check" ]; then
>      cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
>  fi
> -find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
> +find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
>  cd ..

Re: [PATCH v3 2/6] CI: avoid repacking initrd as part of the test job

[Andrew Cooper]

On 16/04/2025 12:48 am, Stefano Stabellini wrote:
>> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
>> index c4be71464246..e2376cb51cb4 100755
>> --- a/automation/scripts/qubes-x86-64.sh
>> +++ b/automation/scripts/qubes-x86-64.sh
>> @@ -164,16 +164,13 @@ ${domU_extra_config}
>>  "
>>  
>>  if [ -n "$domU_check" ]; then
>> -    # DomU
>> +    # DomU rootfs
>> +    cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
>> +
>> +    # test-local configuration
>>      mkdir -p rootfs
>>      cd rootfs
>> -    # fakeroot is needed to preserve device nodes in rootless podman container
>> -    fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
>> -    mkdir proc
>> -    mkdir run
>> -    mkdir srv
>> -    mkdir sys
>> -    rm var/run
>> +    mkdir -p etc/local.d
>>      echo "#!/bin/sh
> I am worried about the drop of fakeroot because the newly created dirs
> and files we'll have user as owner instead of root.
>
> I suggest we fix this for consistency, and cpio -R 0:0 might suffice.

https://gitlab.com/xen-project/hardware/xen-staging/-/jobs/9737393986#L1087

It seems to be all root anyway.  I'm not entirely sure why, but it does
explain why everything is working, given the attempt to have non-root
containers.

~Andrew

Re: [PATCH v3 2/6] CI: avoid repacking initrd as part of the test job

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

On Wed, 16 Apr 2025, Andrew Cooper wrote:
> On 16/04/2025 12:48 am, Stefano Stabellini wrote:
> >> diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
> >> index c4be71464246..e2376cb51cb4 100755
> >> --- a/automation/scripts/qubes-x86-64.sh
> >> +++ b/automation/scripts/qubes-x86-64.sh
> >> @@ -164,16 +164,13 @@ ${domU_extra_config}
> >>  "
> >>  
> >>  if [ -n "$domU_check" ]; then
> >> -    # DomU
> >> +    # DomU rootfs
> >> +    cp binaries/rootfs.cpio.gz binaries/domU-rootfs.cpio.gz
> >> +
> >> +    # test-local configuration
> >>      mkdir -p rootfs
> >>      cd rootfs
> >> -    # fakeroot is needed to preserve device nodes in rootless podman container
> >> -    fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
> >> -    mkdir proc
> >> -    mkdir run
> >> -    mkdir srv
> >> -    mkdir sys
> >> -    rm var/run
> >> +    mkdir -p etc/local.d
> >>      echo "#!/bin/sh
> > I am worried about the drop of fakeroot because the newly created dirs
> > and files we'll have user as owner instead of root.
> >
> > I suggest we fix this for consistency, and cpio -R 0:0 might suffice.
> 
> https://gitlab.com/xen-project/hardware/xen-staging/-/jobs/9737393986#L1087
> 
> It seems to be all root anyway.  I'm not entirely sure why, but it does
> explain why everything is working, given the attempt to have non-root
> containers.

Hi Andrew, thank you for looking into this. I am truly surprised to see
xen.start marked as root.root. Given that it is what we want I will not
complain though :-)