[PATCH v2 00/12] Several CI cleanups and improvements, plus yet another new runner

[PATCH v2 00/12] Several CI cleanups and improvements, plus yet another new runner

[Marek Marczykowski-Górecki]

Some of those patches are collected from my stubdomain test series, others are
made while getting the new runner working. All of the cleanups can be applied
independently, but most if not all are needed for the "CI: add AMD Zen 4 HW
runner" patch adding new runner.

Last two/four patches (including test-artifacts patches) are
optimizations that cut time of some jobs almost by half.

As usual, besides the patches, somebody need to click on "hal9012" runner in
the relevant projects.

Marek Marczykowski-Górecki (12):
  CI: prevent grub unpacking initramfs
  CI: switch qubes runners to use console.exp
  CI: enable XHCI console in Xen debug build on Alpine
  CI: include domU kernel messages in the console output log
  CI: increase verbosity of starting a domain
  CI: consistently use DOCKER_CMD in makefiles
  CI: wait for the network interface in PCI passthrough tests
  CI: switch test kernel from 6.1.19 to 6.12.21
  CI: adjust resolving network interface into PCI device
  CI: add AMD Zen 4 HW runner
  CI: avoid repacking initrd as part of the test job
  CI: save toolstack artifact as cpio.gz

 automation/build/Makefile                                 |  4 +-
 automation/build/alpine/3.18-arm64v8.dockerfile           |  1 +-
 automation/gitlab-ci/build.yaml                           | 10 +-
 automation/gitlab-ci/test.yaml                            | 58 ++++-
 automation/scripts/build                                  |  2 +-
 automation/scripts/console.exp                            | 23 +-
 automation/scripts/qemu-alpine-x86_64.sh                  | 15 +-
 automation/scripts/qemu-smoke-dom0-arm32.sh               |  2 +-
 automation/scripts/qemu-smoke-dom0-arm64.sh               | 15 +-
 automation/scripts/qemu-smoke-dom0less-arm64.sh           |  9 +-
 automation/scripts/qubes-x86-64.sh                        | 97 +++-----
 automation/scripts/xilinx-smoke-dom0-x86_64.sh            | 29 +-
 automation/scripts/xilinx-smoke-dom0less-arm64.sh         | 16 +-
 automation/tests-artifacts/Makefile                       |  4 +-
 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile |  5 +-
 automation/tests-artifacts/alpine/3.18.dockerfile         |  1 +-
 automation/tests-artifacts/kernel/6.1.19.dockerfile       | 41 +---
 automation/tests-artifacts/kernel/6.12.21.dockerfile      | 43 ++++-
 18 files changed, 216 insertions(+), 159 deletions(-)
 delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/6.12.21.dockerfile

base-commit: a235f856e4bbd270b085590e1f5fc9599234dcdf
-- 
git-series 0.9.1

[PATCH v2 01/12] CI: prevent grub unpacking initramfs

[Marek Marczykowski-Górecki]

It fails on larger initramfs (~250MB one) and sometimes even smaller
depending on memory size/memory map, let Linux do it.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 automation/scripts/qubes-x86-64.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index f27296445901..8e78b7984e98 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -248,7 +248,7 @@ CONTROLLER=control@thor.testnet
 echo "
 multiboot2 (http)/gitlab-ci/xen $CONSOLE_OPTS loglvl=all guest_loglvl=all dom0_mem=4G console_timestamps=boot $extra_xen_opts
 module2 (http)/gitlab-ci/vmlinuz console=hvc0 root=/dev/ram0 earlyprintk=xen
-module2 (http)/gitlab-ci/initrd-dom0
+module2 --nounzip (http)/gitlab-ci/initrd-dom0
 " > $TFTP/grub.cfg
 
 echo "#!ipxe
-- 
git-series 0.9.1

[PATCH v2 02/12] CI: switch qubes runners to use console.exp

[Marek Marczykowski-Górecki]

It appears as sometimes it takes more time for Xen even start booting,
mostly due to firmware and fetching large boot files by grub. In some
jobs the current timeout is pretty close to the actual time needed, and
sometimes (rarely for now) test fails due to timeout expiring in the
middle of dom0 booting. This will be happening more often if the
initramfs will grow (and with more complex tests).
This has been observed on some dom0pvh-hvm jobs, at least on runners hw3
and hw11.

Switch to using expect (console.exp) for more robust test output
handling. This allows waiting separately for Xen starting to boot and
then for the test to complete. For now, set both of those to 120s, which
pessimistically bumps timeout for the whole test to 240s (from 120s).

Some messages use regex, use 'expect -re' for all of them for
consistency, even though not all strictly need that (yet).

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v2:
- replace previous "ci: increase timeout for hw tests" with changing how
  console is interacted with

This needs a containers rebuild.
---
 automation/build/alpine/3.18-arm64v8.dockerfile |  1 +-
 automation/scripts/console.exp                  | 23 ++++++--
 automation/scripts/qubes-x86-64.sh              | 52 ++++--------------
 3 files changed, 32 insertions(+), 44 deletions(-)

diff --git a/automation/build/alpine/3.18-arm64v8.dockerfile b/automation/build/alpine/3.18-arm64v8.dockerfile
index 19fe46f8418f..b8482d5bf43f 100644
--- a/automation/build/alpine/3.18-arm64v8.dockerfile
+++ b/automation/build/alpine/3.18-arm64v8.dockerfile
@@ -48,3 +48,4 @@ RUN apk --no-cache add \
   # qubes test deps
   openssh-client \
   fakeroot \
+  expect \
diff --git a/automation/scripts/console.exp b/automation/scripts/console.exp
index 31ce97b91b63..d1689fa5bf7f 100755
--- a/automation/scripts/console.exp
+++ b/automation/scripts/console.exp
@@ -28,21 +28,34 @@ if {[info exists env(UBOOT_CMD)]} {
     send "$env(UBOOT_CMD)\r"
 }
 
+if {[info exists env(BOOT_MSG)]} {
+    expect -re "$env(BOOT_MSG)"
+}
+
+if {[info exists env(WAKEUP_CMD)]} {
+    expect -re "$env(SUSPEND_MSG)"
+
+    # keep it suspended a bit, then wakeup
+    sleep 30
+
+    system "$env(WAKEUP_CMD)"
+}
+
 if {[info exists env(LOG_MSG)]} {
     expect {
-        "$env(PASSED)" {
-            expect "$env(LOG_MSG)"
+        -re "$env(PASSED)" {
+            expect -re "$env(LOG_MSG)"
             exit 0
         }
-        "$env(LOG_MSG)" {
-            expect "$env(PASSED)"
+        -re "$env(LOG_MSG)" {
+            expect -re "$env(PASSED)"
             exit 0
         }
     }
 }
 
 expect {
-    "$env(PASSED)" {
+    -re "$env(PASSED)" {
         exit 0
     }
 }
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 8e78b7984e98..0eac410f4168 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-set -ex
+set -ex -o pipefail
 
 # One of:
 #  - ""             PV dom0,  PVH domU
@@ -263,52 +263,26 @@ cp -f binaries/xen $TFTP/xen
 cp -f binaries/bzImage $TFTP/vmlinuz
 cp -f binaries/dom0-rootfs.cpio.gz $TFTP/initrd-dom0
 
-# start logging the serial; this gives interactive console, don't close its
-# stdin to not close it; the 'cat' is important, plain redirection would hang
-# until somebody opens the pipe; opening and closing the pipe is used to close
-# the console
-mkfifo /tmp/console-stdin
-cat /tmp/console-stdin |\
-ssh $CONTROLLER console | tee smoke.serial | sed 's/\r//' &
-
 # start the system pointing at gitlab-ci predefined config
 ssh $CONTROLLER gitlabci poweron
-trap "ssh $CONTROLLER poweroff; : > /tmp/console-stdin" EXIT
+trap "ssh $CONTROLLER poweroff" EXIT
 
 if [ -n "$wait_and_wakeup" ]; then
-    # wait for suspend or a timeout
-    until grep "$wait_and_wakeup" smoke.serial || [ $timeout -le 0 ]; do
-        sleep 1;
-        : $((--timeout))
-    done
-    if [ $timeout -le 0 ]; then
-        echo "ERROR: suspend timeout, aborting"
-        exit 1
-    fi
-    # keep it suspended a bit, then wakeup
-    sleep 30
-    ssh $CONTROLLER wake
+    export SUSPEND_MSG="$wait_and_wakeup"
+    export WAKEUP_CMD="ssh $CONTROLLER wake"
 fi
 
-set +x
-until grep "^Welcome to Alpine Linux" smoke.serial || [ $timeout -le 0 ]; do
-    sleep 1;
-    : $((--timeout))
-done
-set -x
-
-tail -n 100 smoke.serial
-
-if [ $timeout -le 0 ]; then
-    echo "ERROR: test timeout, aborting"
-    exit 1
-fi
+export PASSED="${passed}"
+export BOOT_MSG="Latest ChangeSet: "
+export LOG_MSG="\nWelcome to Alpine Linux"
+export TEST_CMD="ssh $CONTROLLER console"
+export TEST_LOG="smoke.serial"
+export TEST_TIMEOUT="$timeout"
+./automation/scripts/console.exp | sed 's/\r\+$//'
+TEST_RESULT=$?
 
 if [ -n "$retrieve_xml" ]; then
     nc -w 10 "$SUT_ADDR" 8080 > tests-junit.xml </dev/null
 fi
 
-sleep 1
-
-(grep -q "^Welcome to Alpine Linux" smoke.serial && grep -q "${passed}" smoke.serial) || exit 1
-exit 0
+exit "$TEST_RESULT"
-- 
git-series 0.9.1

[PATCH v2 03/12] CI: enable XHCI console in Xen debug build on Alpine

[Marek Marczykowski-Górecki]

This build is used for hardware tests, and some runners use XHCI console

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
 automation/gitlab-ci/build.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 2513908b059b..169bebe3c775 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -341,6 +341,7 @@ alpine-3.18-gcc-debug:
       CONFIG_ARGO=y
       CONFIG_UBSAN=y
       CONFIG_UBSAN_FATAL=y
+      CONFIG_XHCI=y
 
 debian-12-x86_64-gcc-debug:
   extends: .gcc-x86-64-build-debug
-- 
git-series 0.9.1

[PATCH v2 04/12] CI: include domU kernel messages in the console output log

[Marek Marczykowski-Górecki]

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
 automation/scripts/qubes-x86-64.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 0eac410f4168..7bf42d2e3d31 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -175,6 +175,8 @@ if [ -n "$domU_check" ]; then
     rm var/run
     echo "#!/bin/sh
 
+echo 8 > /proc/sys/kernel/printk
+
 ${domU_check}
 " > etc/local.d/xen.start
     chmod +x etc/local.d/xen.start
-- 
git-series 0.9.1

[PATCH v2 05/12] CI: increase verbosity of starting a domain

[Marek Marczykowski-Górecki]

And start collecting qemu log earlier, so it isn't lost in case of a
timeout during domain startup.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v2:
- adjust xilinx-smoke-dom0-x86_64.sh too
---
 automation/scripts/qemu-alpine-x86_64.sh       | 2 +-
 automation/scripts/qemu-smoke-dom0-arm32.sh    | 2 +-
 automation/scripts/qemu-smoke-dom0-arm64.sh    | 2 +-
 automation/scripts/qubes-x86-64.sh             | 4 ++--
 automation/scripts/xilinx-smoke-dom0-x86_64.sh | 4 ++--
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 1ff689b577e3..17e2141d625e 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -56,7 +56,7 @@ bash /etc/init.d/xencommons start
 
 xl list
 
-xl create -c /root/test.cfg
+xl -vvv create -c /root/test.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
index e1cd83880928..0c60a66e25e3 100755
--- a/automation/scripts/qemu-smoke-dom0-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
@@ -21,7 +21,7 @@ echo "#!/bin/bash
 
 xl list
 
-xl create -c /root/test.cfg
+xl -vvv create -c /root/test.cfg
 
 " > ./root/xen.start
 echo "bash /root/xen.start" >> ./etc/init.d/xen-watchdog
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 4d22a124df11..8774a8701232 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -52,7 +52,7 @@ bash /etc/init.d/xencommons start
 
 xl list
 
-xl create -c /root/test.cfg
+xl -vvv create -c /root/test.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 7bf42d2e3d31..17fcbd8fa7db 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -122,7 +122,6 @@ echo \"${passed}\"
 "
 
         dom0_check="
-tail -F /var/log/xen/qemu-dm-domU.log &
 until grep -q \"^domU Welcome to Alpine Linux\" /var/log/xen/console/guest-domU.log; do
     sleep 1
 done
@@ -222,7 +221,8 @@ if [ -n "$domU_check" ]; then
     echo "
 # get domU console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
-xl create /etc/xen/domU.cfg
+tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu-dm) /\" &
+xl -vvv create /etc/xen/domU.cfg
 ${dom0_check}
 " >> etc/local.d/xen.start
 else
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index f70cfdc1552a..7834ffbe0593 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -49,7 +49,7 @@ ifconfig xenbr0 up
 ifconfig xenbr0 192.168.0.1
 # get domU console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
-xl create /etc/xen/domU.cfg
+xl -vvv create /etc/xen/domU.cfg
 set +x
 until grep -q \"${DOMU_MSG}\" /var/log/xen/console/guest-domU.log; do
     sleep 1
@@ -75,7 +75,7 @@ done | argo-exec -p 28333 -d 0 -- /bin/echo
 "
     DOM0_CMD="
 insmod /root/xen-argo.ko
-xl create /etc/xen/domU.cfg
+xl -vvv create /etc/xen/domU.cfg
 argo-exec -l -p 28333 -- /bin/echo
 "
 copy_dom0_files ()
-- 
git-series 0.9.1

[PATCH v2 06/12] CI: consistently use DOCKER_CMD in makefiles

[Marek Marczykowski-Górecki]

This allows rebuilding containers using podman too.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 automation/build/Makefile           | 4 ++--
 automation/tests-artifacts/Makefile | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/automation/build/Makefile b/automation/build/Makefile
index 4df43b040777..fedf7524dacd 100644
--- a/automation/build/Makefile
+++ b/automation/build/Makefile
@@ -31,8 +31,8 @@ clean:
 define CLEAN_RULE
 .PHONY: clean-$(1)
 clean-$(1):
-	if [ -n "$$$$(docker image ls -q $(REGISTRY)/$(subst /,:,$(1)))" ]; then \
-		docker image rm $(REGISTRY)/$(subst /,:,$(1)); \
+	if [ -n "$$$$($(DOCKER_CMD) image ls -q $(REGISTRY)/$(subst /,:,$(1)))" ]; then \
+		$(DOCKER_CMD) image rm $(REGISTRY)/$(subst /,:,$(1)); \
 	fi
 
 endef
diff --git a/automation/tests-artifacts/Makefile b/automation/tests-artifacts/Makefile
index d055cd696bed..80a60a94f3f7 100644
--- a/automation/tests-artifacts/Makefile
+++ b/automation/tests-artifacts/Makefile
@@ -10,9 +10,9 @@ help:
 	@echo "To push container builds, set the env var PUSH"
 
 %: %.dockerfile ## Builds containers
-	docker build --pull -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
+	$(DOCKER_CMD) build --pull -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
 	@if [ ! -z $${PUSH+x} ]; then \
-		docker push $(REGISTRY)/$(@D):$(@F); \
+		$(DOCKER_CMD) push $(REGISTRY)/$(@D):$(@F); \
 	fi
 
 .PHONY: all
-- 
git-series 0.9.1

[PATCH v2 07/12] CI: wait for the network interface in PCI passthrough tests

[Marek Marczykowski-Górecki]

The network driver initializes asynchronously, and it may not be ready
yet by the time the startup script is called. This is especially the
case for USB network adapter (where the PCI device is the USB
controller) in the upcoming runner.

Don't bother about separate timeout - test timeout will cover this part
too.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 automation/scripts/qubes-x86-64.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 17fcbd8fa7db..9359e8914fb2 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -104,6 +104,7 @@ on_reboot = "destroy"
         domU_check="
 set -x -e
 interface=eth0
+while ! [ -e \"/sys/class/net/\$interface\" ]; do sleep 1; done
 ip link set \"\$interface\" up
 timeout 30s udhcpc -i \"\$interface\"
 pingip=\$(ip -o -4 r show default|cut -f 3 -d ' ')
-- 
git-series 0.9.1

[PATCH v2 09/12] CI: adjust resolving network interface into PCI device

[Marek Marczykowski-Górecki]

Change how PCI device lookup is done to handle also USB devices, in
which case get the USB controller. Instead of taking basename of the
'device' symlink, resolve the full path (example:
/sys/devices/pci0000:00/0000:00:09.0/usb4/4-7/4-7:1.0) and take the
first part after pci0000:00. Theoretically it could be a bridge, but VM
has flat PCI topology.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
 automation/scripts/qubes-x86-64.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 9359e8914fb2..861e302d845b 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -110,7 +110,8 @@ timeout 30s udhcpc -i \"\$interface\"
 pingip=\$(ip -o -4 r show default|cut -f 3 -d ' ')
 ping -c 10 \"\$pingip\"
 echo domU started
-pcidevice=\$(basename \$(readlink /sys/class/net/\$interface/device))
+pcidevice=\$(realpath /sys/class/net/\$interface/device |
+             sed 's#.*pci0000:00/\\([^/]*\\).*#\\1#')
 lspci -vs \$pcidevice
 "
         if [ -n "$PCIDEV_INTR" ]; then
-- 
git-series 0.9.1

[PATCH v2 10/12] CI: add AMD Zen 4 HW runner

[Marek Marczykowski-Górecki]

This is AMD Ryzen 5 7640U in Framework Laptop AMD.
It has several things different than the other runners.
First of all, the console is using XHCI debug capability.
And then, this system doesn't have normal wired ethernet. But is has one
on USB, and this one is used for booting. For this, enable
CONFIG_USB_RTL8152 in the Linux build.

Include some basic tests, plus PCI passthrough.

This machine doesn't support S3. S0ix pretends to be working with
Qubes's dom0 (kernel + userspace), but it hangs on suspend with Alpine
used in test. But even when it doesn't hang, it doesn't really reach
deep sleep, so skip this test for now here.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Requires containers rebuild, especially tests-artifacts/kernel/6.12.21.

Do not apply yet, until issues found by those tests are fixed. That
would be at least third issue found using this system, so lets have it
in CI.
---
 automation/gitlab-ci/test.yaml                       | 56 +++++++++++++-
 automation/tests-artifacts/kernel/6.12.21.dockerfile |  2 +-
 2 files changed, 58 insertions(+)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index a13dd040bd26..feb2dd10ab30 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -188,6 +188,16 @@
   tags:
     - qubes-hw11
 
+.zen4-x86-64:
+  extends: .adl-x86-64
+  variables:
+    PCIDEV: "c3:00.4"
+    PCIDEV_INTR: "MSI-X"
+    CONSOLE_OPTS: "console=xhci dbgp=xhci@pcic1:00.3,share=yes"
+    SUT_ADDR: test-12.testnet
+  tags:
+    - qubes-hw12
+
 # Test jobs
 build-each-commit-gcc:
   extends: .test-jobs-common
@@ -479,6 +489,52 @@ zen3p-tools-tests-pvh-x86-64-gcc-debug:
     - *x86-64-test-needs
     - alpine-3.18-gcc-debug
 
+zen4-smoke-x86-64-gcc-debug:
+  extends: .zen4-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh 2>&1 | tee ${LOGFILE}
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.18-gcc-debug
+
+zen4-smoke-x86-64-dom0pvh gcc-debug:
+  extends: .zen4-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh dom0pvh 2>&1 | tee ${LOGFILE}
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.18-gcc-debug
+
+zen4-pci-hvm-x86-64-gcc-debug:
+  extends: .zen4-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh pci-hvm 2>&1 | tee ${LOGFILE}
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.18-gcc-debug
+
+zen4-tools-tests-pv-x86-64-gcc-debug:
+  extends: .zen4-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh tools-tests-pv 2>&1 | tee ${LOGFILE}
+  artifacts:
+    reports:
+      junit: tests-junit.xml
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.18-gcc-debug
+
+zen4-tools-tests-pvh-x86-64-gcc-debug:
+  extends: .zen4-x86-64
+  script:
+    - ./automation/scripts/qubes-x86-64.sh tools-tests-pvh 2>&1 | tee ${LOGFILE}
+  artifacts:
+    reports:
+      junit: tests-junit.xml
+  needs:
+    - *x86-64-test-needs
+    - alpine-3.18-gcc-debug
+
 qemu-smoke-dom0-arm64-gcc:
   extends: .qemu-arm64
   script:
diff --git a/automation/tests-artifacts/kernel/6.12.21.dockerfile b/automation/tests-artifacts/kernel/6.12.21.dockerfile
index d7d34031cab5..4e0a68ec61eb 100644
--- a/automation/tests-artifacts/kernel/6.12.21.dockerfile
+++ b/automation/tests-artifacts/kernel/6.12.21.dockerfile
@@ -19,6 +19,7 @@ RUN apt-get update && \
         flex \
         bison \
         libelf-dev \
+        libssl-dev \
         && \
     apt-get autoremove -y && \
     apt-get clean && \
@@ -33,6 +34,7 @@ RUN curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"$LINUX_VERSI
     scripts/config --enable BRIDGE && \
     scripts/config --enable IGC && \
     scripts/config --enable TUN && \
+    scripts/config --enable USB_RTL8152 && \
     cp .config .config.orig && \
     cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
     make -j$(nproc) bzImage && \
-- 
git-series 0.9.1

[PATCH v2 11/12] CI: avoid repacking initrd as part of the test job

[Marek Marczykowski-Górecki]

The prerequisite for this is building initrd.cpio.gz, not only
initrd.tar.gz. That's a change in the test-artifacts repository.

Having that, do not repack the whole initrd, but only pack modified
files and rely on Linux handling of concatenated archives.
This allows packing just test-related files (which includes the whole
toolstack), instead of the whole initrd.

DomU initrd handling is a bit more complicated thing. It's sent to the
target host as part of the dom0 initrd. But to modify/extend it, the
"overlay" need to be applied already on the target system (otherwise
controller host would need to repack it anyway). So, make the
initrd.cpio.gz in test-artifacts include boot/initrd-domU already, and
add test-specific files as boot/initrd-domU-overlay that is concatenated
by dom0 boot script.

Since full initrd is not unpacked now when preparing domU (and dom0)
rootfs, a couple of minor changes are needed to how they are prepared.
This includes creating whole etc/issue file, instead of modifying
existing one, and a need to create a couple directories.

And since there is now initrd.cpio.gz artifact, rename temporary domU
initrd to initrd-domU.cpio.gz

Finally, move adding "rc_verbose=yes" to /etc/rc.conf to initrd
building.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
New in v2

In fact, alpine/3.18.dockerfile in this repository is not what
is used to build initrd.cpio.gz for test here, but add it anyway in case
there are some other tests using it that I missed. If it's really
unused, it probably should be removed, but that's a separate patch.

Jobs xilinx-* are untested, as I don't have necessary access.
---
 automation/gitlab-ci/build.yaml                           |  5 +-
 automation/scripts/qemu-alpine-x86_64.sh                  | 11 +--
 automation/scripts/qemu-smoke-dom0-arm64.sh               | 11 +--
 automation/scripts/qemu-smoke-dom0less-arm64.sh           |  7 +--
 automation/scripts/qubes-x86-64.sh                        | 29 +++-----
 automation/scripts/xilinx-smoke-dom0-x86_64.sh            | 23 +++---
 automation/scripts/xilinx-smoke-dom0less-arm64.sh         | 14 +---
 automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile |  5 +-
 automation/tests-artifacts/alpine/3.18.dockerfile         |  1 +-
 9 files changed, 55 insertions(+), 51 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index d41f27fc94bf..8e1ff0178469 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -268,10 +268,13 @@ alpine-3.18-arm64-rootfs-export:
   extends: .test-jobs-artifact-common
   image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.18-arm64v8
   script:
-    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
+    - mkdir binaries
+    - cp /initrd.tar.gz binaries/initrd.tar.gz
+    - cp /initrd.cpio.gz binaries/initrd.cpio.gz
   artifacts:
     paths:
       - binaries/initrd.tar.gz
+      - binaries/initrd.cpio.gz
   tags:
     - arm64
 
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 17e2141d625e..9b47b662c3f2 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -25,20 +25,19 @@ mount -t devtmpfs devtmpfs /dev
 chmod +x initrd/init
 # DomU rootfs
 cd initrd
-find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+find . | cpio --create --format='newc' | gzip > ../initrd-domU.cpio.gz
 cd ..
 
 # initrd.tar.gz is Dom0 rootfs
 mkdir -p rootfs
 cd rootfs
-tar xvzf ../initrd.tar.gz
 mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-rm var/run
 cp -ar ../dist/install/* .
-mv ../initrd.cpio.gz ./root
+mkdir -p root etc/local.d
+mv ../initrd-domU.cpio.gz ./root/initrd.cpio.gz
 cp ../bzImage ./root
 echo "name=\"test\"
 memory=512
@@ -60,9 +59,9 @@ xl -vvv create -c /root/test.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
 # rebuild Dom0 rootfs
-find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
+cp ../initrd.cpio.gz ../xen-rootfs.cpio.gz
+find . |cpio -H newc -o|gzip >> ../xen-rootfs.cpio.gz
 cd ../..
 
 cat >> binaries/pxelinux.0 << EOF
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 8774a8701232..533b286528f1 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -24,19 +24,18 @@ mount -t devtmpfs devtmpfs /dev
 /bin/sh" > initrd/init
 chmod +x initrd/init
 cd initrd
-find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+find . | cpio --create --format='newc' | gzip > ../initrd-domU.cpio.gz
 cd ..
 
 mkdir -p rootfs
 cd rootfs
-tar xvzf ../initrd.tar.gz
 mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-rm var/run
 cp -ar ../dist/install/* .
-mv ../initrd.cpio.gz ./root
+mkdir -p etc/local.d root
+mv ../initrd-domU.cpio.gz ./root/initrd.cpio.gz
 cp ../Image ./root
 echo "name=\"test\"
 memory=512
@@ -56,8 +55,8 @@ xl -vvv create -c /root/test.cfg
 
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
+cp ../initrd.cpio.gz ../xen-rootfs.cpio.gz
+find . |cpio -H newc -o|gzip >> ../xen-rootfs.cpio.gz
 cd ../..
 
 # XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index f72d20936181..70085e26c139 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -117,13 +117,12 @@ cd ..
 # DOM0 rootfs
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
 mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-rm var/run
 cp -ar ../binaries/dist/install/* .
+mkdir -p etc/local.d
 
 echo "#!/bin/bash
 
@@ -142,8 +141,8 @@ xl network-attach 1 type=vif
 ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 # ImageBuilder
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 861e302d845b..399d72e0600d 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -167,13 +167,10 @@ if [ -n "$domU_check" ]; then
     # DomU
     mkdir -p rootfs
     cd rootfs
-    # fakeroot is needed to preserve device nodes in rootless podman container
-    fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
     mkdir proc
     mkdir run
-    mkdir srv
     mkdir sys
-    rm var/run
+    mkdir -p etc/local.d
     echo "#!/bin/sh
 
 echo 8 > /proc/sys/kernel/printk
@@ -181,23 +178,23 @@ echo 8 > /proc/sys/kernel/printk
 ${domU_check}
 " > etc/local.d/xen.start
     chmod +x etc/local.d/xen.start
-    echo "rc_verbose=yes" >> etc/rc.conf
-    sed -i -e 's/^Welcome/domU \0/' etc/issue
-    find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
+    echo "domU Welcome to Alpine Linux 3.18
+Kernel \r on an \m (\l)
+
+" > etc/issue
+    find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs-overlay.cpio.gz
     cd ..
     rm -rf rootfs
 fi
 
-# DOM0 rootfs
+# DOM0 rootfs - this will be an overlay over alpine's initrd
 mkdir -p rootfs
 cd rootfs
-fakeroot -s ../fakeroot-save tar xzf ../binaries/initrd.tar.gz
 mkdir boot
 mkdir proc
 mkdir run
-mkdir srv
 mkdir sys
-rm var/run
+mkdir -p etc/local.d
 cp -ar ../binaries/dist/install/* .
 cp -ar ../binaries/tests .
 cp -a ../automation/scripts/run-tools-tests tests/
@@ -221,6 +218,8 @@ fi
 
 if [ -n "$domU_check" ]; then
     echo "
+# append test-specific files to domU initrd
+cat /boot/initrd-domU-overlay >> boot/initrd-domU
 # get domU console content into test log
 tail -F /var/log/xen/console/guest-domU.log 2>/dev/null | sed -e \"s/^/(domU) /\" &
 tail -F /var/log/xen/qemu-dm-domU.log 2>/dev/null | sed -e \"s/^/(qemu-dm) /\" &
@@ -234,18 +233,18 @@ fi
 chmod +x etc/local.d/xen.start
 echo "$domU_config" > etc/xen/domU.cfg
 
-echo "rc_verbose=yes" >> etc/rc.conf
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
 cp ../binaries/bzImage boot/vmlinuz
 if [ -n "$domU_check" ]; then
-    cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
+    cp ../binaries/domU-rootfs-overlay.cpio.gz boot/initrd-domU-overlay
 fi
-find . | fakeroot -i ../fakeroot-save cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+# take base initrd and append test-specific files
+cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
-
 TFTP=/scratch/gitlab-runner/tftp
 CONTROLLER=control@thor.testnet
 
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 7834ffbe0593..6b9f0e4a8dc7 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -80,6 +80,7 @@ argo-exec -l -p 28333 -- /bin/echo
 "
 copy_dom0_files ()
 {
+    mkdir -p root usr/local/lib
     cp "${WORKDIR}/binaries/xen-argo.ko" "root/"
     cp -ar "${WORKDIR}/binaries/lib/"* "usr/local/lib/"
     cp "${WORKDIR}/binaries/argo-exec" "usr/local/bin/"
@@ -92,9 +93,8 @@ fi
 # Set up domU rootfs.
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
 mkdir proc run srv sys
-rm var/run
+mkdir -p etc/local.d
 echo "#!/bin/sh
 set -x
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -102,36 +102,39 @@ PATH=/usr/local/bin:/usr/local/sbin:\$PATH
 ${DOMU_CMD}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-sed -i -e 's/^Welcome/domU \0/' etc/issue
+echo "domU Welcome to Alpine Linux 3.18
+Kernel \r on an \m (\l)
+
+" > etc/issue
 copy_domU_files
-find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs-overlay.cpio.gz
 cd ..
 rm -rf rootfs
 
 # Set up dom0 rootfs.
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
 mkdir boot proc run srv sys
-rm var/run
 cp -ar ../binaries/dist/install/* .
+mkdir -p etc/local.d
 echo "#!/bin/bash
 set -x
 export LD_LIBRARY_PATH=/usr/local/lib
 bash /etc/init.d/xencommons start
+# append test-specific files to domU initrd
+cat /boot/initrd-domU-overlay >> boot/initrd-domU
 ${DOM0_CMD}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 echo "${DOMU_CFG}${DOMU_CFG_EXTRA}" > etc/xen/domU.cfg
-echo "rc_verbose=yes" >> etc/rc.conf
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
 cp ../binaries/bzImage boot/vmlinuz
-cp ../binaries/domU-rootfs.cpio.gz boot/initrd-domU
+cp ../binaries/domU-rootfs-overlay.cpio.gz boot/initrd-domU-overlay
 copy_dom0_files
-find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 # Load software into TFTP server directory.
diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
index b24ad11b8cac..81be60e2026e 100755
--- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
+++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
@@ -41,33 +41,31 @@ fi
 # DomU
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
 mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-rm var/run
+mkdir -p etc/local.d
 echo "#!/bin/sh
 
 ${domU_check}
 /bin/sh" > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
+cp ../binaries/initrd.cpio.gz ../binaries/domU-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
 cd ..
 rm -rf rootfs
 
 # DOM0 rootfs
 mkdir -p rootfs
 cd rootfs
-tar xzf ../binaries/initrd.tar.gz
 mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-rm var/run
 cp -ar ../binaries/dist/install/* .
 
+mkdir -p etc/local.d
 echo "#!/bin/bash
 
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -78,8 +76,8 @@ bash /etc/init.d/xencommons start
 ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
-echo "rc_verbose=yes" >> etc/rc.conf
-find . | cpio -H newc -o | gzip > ../binaries/dom0-rootfs.cpio.gz
+cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
 
diff --git a/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile b/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
index a4542f703997..78d465dc6a39 100644
--- a/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
+++ b/automation/tests-artifacts/alpine/3.18-arm64v8.dockerfile
@@ -60,7 +60,10 @@ RUN \
   echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
   echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
   passwd -d "root" root && \
+  echo "rc_verbose=yes" >> /etc/rc.conf && \
   \
   # Create rootfs
   cd / && \
-  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
+  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var && \
+  find bin dev etc home init lib mnt opt root sbin usr var |\
+    cpio -o -H newc | gzip > /initrd.cpio.gz
diff --git a/automation/tests-artifacts/alpine/3.18.dockerfile b/automation/tests-artifacts/alpine/3.18.dockerfile
index 311a92889b87..ff124fef49a8 100644
--- a/automation/tests-artifacts/alpine/3.18.dockerfile
+++ b/automation/tests-artifacts/alpine/3.18.dockerfile
@@ -61,6 +61,7 @@ RUN \
   echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
   echo > /etc/modules && \
   passwd -d "root" root && \
+  echo "rc_verbose=yes" >> /etc/rc.conf && \
   \
   # Create rootfs
   cd / && \
-- 
git-series 0.9.1

[PATCH v2 12/12] CI: save toolstack artifact as cpio.gz

[Marek Marczykowski-Górecki]

This avoids the need to re-compress it in every test job.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
New in v2
---
 automation/scripts/build                          | 2 +-
 automation/scripts/qemu-alpine-x86_64.sh          | 2 +-
 automation/scripts/qemu-smoke-dom0-arm64.sh       | 2 +-
 automation/scripts/qemu-smoke-dom0less-arm64.sh   | 2 +-
 automation/scripts/qubes-x86-64.sh                | 4 +++-
 automation/scripts/xilinx-smoke-dom0-x86_64.sh    | 4 ++--
 automation/scripts/xilinx-smoke-dom0less-arm64.sh | 2 +-
 7 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/automation/scripts/build b/automation/scripts/build
index 522efe774ef3..365534895047 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -108,7 +108,7 @@ else
     # Note: Some smoke tests depending on finding binaries/xen on a full build
     # even though dist/ contains everything, while some containers don't even
     # build Xen
-    cp -r dist binaries/
+    (cd dist/install; find | cpio -o -H newc | gzip) > binaries/toolstack.cpio.gz
     cp -r tools/tests binaries/
     collect_xen_artefacts
 fi
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 9b47b662c3f2..81e8ff24327a 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -35,7 +35,6 @@ mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-cp -ar ../dist/install/* .
 mkdir -p root etc/local.d
 mv ../initrd-domU.cpio.gz ./root/initrd.cpio.gz
 cp ../bzImage ./root
@@ -61,6 +60,7 @@ xl -vvv create -c /root/test.cfg
 chmod +x etc/local.d/xen.start
 # rebuild Dom0 rootfs
 cp ../initrd.cpio.gz ../xen-rootfs.cpio.gz
+cat ../toolstack.cpio.gz >> ../xen-rootfs.cpio.gz
 find . |cpio -H newc -o|gzip >> ../xen-rootfs.cpio.gz
 cd ../..
 
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 533b286528f1..2e5246f2c9b6 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -33,7 +33,6 @@ mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-cp -ar ../dist/install/* .
 mkdir -p etc/local.d root
 mv ../initrd-domU.cpio.gz ./root/initrd.cpio.gz
 cp ../Image ./root
@@ -56,6 +55,7 @@ xl -vvv create -c /root/test.cfg
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 cp ../initrd.cpio.gz ../xen-rootfs.cpio.gz
+cat ../toolstack.cpio.gz >> ../xen-rootfs.cpio.gz
 find . |cpio -H newc -o|gzip >> ../xen-rootfs.cpio.gz
 cd ../..
 
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index 70085e26c139..93f4fb410690 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -121,7 +121,6 @@ mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-cp -ar ../binaries/dist/install/* .
 mkdir -p etc/local.d
 
 echo "#!/bin/bash
@@ -142,6 +141,7 @@ ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+cat ../binaries/toolstack.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
 find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
diff --git a/automation/scripts/qubes-x86-64.sh b/automation/scripts/qubes-x86-64.sh
index 399d72e0600d..4fe59fce8999 100755
--- a/automation/scripts/qubes-x86-64.sh
+++ b/automation/scripts/qubes-x86-64.sh
@@ -195,7 +195,6 @@ mkdir proc
 mkdir run
 mkdir sys
 mkdir -p etc/local.d
-cp -ar ../binaries/dist/install/* .
 cp -ar ../binaries/tests .
 cp -a ../automation/scripts/run-tools-tests tests/
 
@@ -231,8 +230,10 @@ else
 fi
 
 chmod +x etc/local.d/xen.start
+mkdir -p etc/xen
 echo "$domU_config" > etc/xen/domU.cfg
 
+mkdir -p etc/default
 echo "XENCONSOLED_TRACE=all" >> etc/default/xencommons
 echo "QEMU_XEN=/bin/false" >> etc/default/xencommons
 mkdir -p var/log/xen/console
@@ -242,6 +243,7 @@ if [ -n "$domU_check" ]; then
 fi
 # take base initrd and append test-specific files
 cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+cat ../binaries/toolstack.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
 find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 6b9f0e4a8dc7..ea0f952975c7 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -115,8 +115,7 @@ rm -rf rootfs
 mkdir -p rootfs
 cd rootfs
 mkdir boot proc run srv sys
-cp -ar ../binaries/dist/install/* .
-mkdir -p etc/local.d
+mkdir -p etc/local.d etc/xen etc/default
 echo "#!/bin/bash
 set -x
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -134,6 +133,7 @@ cp ../binaries/bzImage boot/vmlinuz
 cp ../binaries/domU-rootfs-overlay.cpio.gz boot/initrd-domU-overlay
 copy_dom0_files
 cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+cat ../binaries/toolstack.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
 find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
index 81be60e2026e..dbb955df0478 100755
--- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
+++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
@@ -63,7 +63,6 @@ mkdir proc
 mkdir run
 mkdir srv
 mkdir sys
-cp -ar ../binaries/dist/install/* .
 
 mkdir -p etc/local.d
 echo "#!/bin/bash
@@ -77,6 +76,7 @@ ${dom0_check}
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 cp ../binaries/initrd.cpio.gz ../binaries/dom0-rootfs.cpio.gz
+cat ../binaries/toolstack.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
 find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
 cd ..
 
-- 
git-series 0.9.1

[PATCH test-artifacts v2 13/12] scripts: add rc_verbose=yes to /etc/rc.conf

[Marek Marczykowski-Górecki]

This is done in every single test job, so do it at the initrd build
time.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 scripts/x86_64-rootfs-alpine.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
index f8b04c5..b70b3a5 100755
--- a/scripts/x86_64-rootfs-alpine.sh
+++ b/scripts/x86_64-rootfs-alpine.sh
@@ -50,6 +50,7 @@ echo "ttyS0" >> /etc/securetty
 echo "hvc0" >> /etc/securetty
 echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab
 echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab
+echo "rc_verbose=yes" >> /etc/rc.conf
 echo > /etc/modules
 passwd -d "root" root
 
-- 
2.48.1

[PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Marek Marczykowski-Górecki]

Build initrd format directly digestable by the kernel. Additionally,
include it itself inside as boot/initrd-domU file, for domU booting in
tests.
This should avoid the need to repack tar -> cpio as part of the test
job.

Keep generating initrd.tar.gz as that's still used by older branches.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 scripts/x86_64-rootfs-alpine.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
index b70b3a5..13e85fa 100755
--- a/scripts/x86_64-rootfs-alpine.sh
+++ b/scripts/x86_64-rootfs-alpine.sh
@@ -58,3 +58,8 @@ passwd -d "root" root
 cd /
 tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
     bin dev etc home init lib mnt opt root sbin usr var
+mkdir boot
+find bin dev etc home init lib mnt opt root sbin usr var |\
+    cpio -o -H newc | gzip > boot/initrd-domU
+find bin boot dev etc home init lib mnt opt root sbin usr var |\
+    cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
-- 
2.48.1

Re: [PATCH test-artifacts v2 13/12] scripts: add rc_verbose=yes to /etc/rc.conf

[Andrew Cooper]

On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> This is done in every single test job, so do it at the initrd build
> time.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Andrew Cooper]

On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> Build initrd format directly digestable by the kernel. Additionally,
> include it itself inside as boot/initrd-domU file, for domU booting in
> tests.
> This should avoid the need to repack tar -> cpio as part of the test
> job.
>
> Keep generating initrd.tar.gz as that's still used by older branches.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

> ---
>  scripts/x86_64-rootfs-alpine.sh | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> index b70b3a5..13e85fa 100755
> --- a/scripts/x86_64-rootfs-alpine.sh
> +++ b/scripts/x86_64-rootfs-alpine.sh
> @@ -58,3 +58,8 @@ passwd -d "root" root
>  cd /
>  tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>      bin dev etc home init lib mnt opt root sbin usr var

What's required to drop the tar version?  Presumably merging the rest of
your series?

> +mkdir boot
> +find bin dev etc home init lib mnt opt root sbin usr var |\
> +    cpio -o -H newc | gzip > boot/initrd-domU
> +find bin boot dev etc home init lib mnt opt root sbin usr var |\
> +    cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Andrew Cooper]

On 07/04/2025 5:17 pm, Andrew Cooper wrote:
> On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
>> Build initrd format directly digestable by the kernel. Additionally,
>> include it itself inside as boot/initrd-domU file, for domU booting in
>> tests.
>> This should avoid the need to repack tar -> cpio as part of the test
>> job.
>>
>> Keep generating initrd.tar.gz as that's still used by older branches.
>>
>> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
>> ---
>>  scripts/x86_64-rootfs-alpine.sh | 5 +++++
>>  1 file changed, 5 insertions(+)
>>
>> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
>> index b70b3a5..13e85fa 100755
>> --- a/scripts/x86_64-rootfs-alpine.sh
>> +++ b/scripts/x86_64-rootfs-alpine.sh
>> @@ -58,3 +58,8 @@ passwd -d "root" root
>>  cd /
>>  tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>>      bin dev etc home init lib mnt opt root sbin usr var
> What's required to drop the tar version?  Presumably merging the rest of
> your series?

NVM, I can't read.  I'm tempted to fold in:

#TODO, drop .tar.gz when Xen 4.20 drops out of testing

That said, I expect we'll be backporting these changes.

~Andrew

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 2088 bytes --]

On Mon, Apr 07, 2025 at 05:22:05PM +0100, Andrew Cooper wrote:
> On 07/04/2025 5:17 pm, Andrew Cooper wrote:
> > On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> >> Build initrd format directly digestable by the kernel. Additionally,
> >> include it itself inside as boot/initrd-domU file, for domU booting in
> >> tests.
> >> This should avoid the need to repack tar -> cpio as part of the test
> >> job.
> >>
> >> Keep generating initrd.tar.gz as that's still used by older branches.
> >>
> >> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >
> >> ---
> >>  scripts/x86_64-rootfs-alpine.sh | 5 +++++
> >>  1 file changed, 5 insertions(+)
> >>
> >> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> >> index b70b3a5..13e85fa 100755
> >> --- a/scripts/x86_64-rootfs-alpine.sh
> >> +++ b/scripts/x86_64-rootfs-alpine.sh
> >> @@ -58,3 +58,8 @@ passwd -d "root" root
> >>  cd /
> >>  tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> >>      bin dev etc home init lib mnt opt root sbin usr var
> > What's required to drop the tar version?  Presumably merging the rest of
> > your series?
> 
> NVM, I can't read.  I'm tempted to fold in:
> 
> #TODO, drop .tar.gz when Xen 4.20 drops out of testing
> 
> That said, I expect we'll be backporting these changes.


There is one more difference: the cpio.gz contains the whole thing
twice. Once as rootfs for dom0 and then another as boot/initrd-domU.
Dropping .tar.gz is probably a good idea at some point, so I'm okay with
such comment added. But I imagine some future tests may benefit from
just one layer, which may want introducing another cpio.gz without
boot/initrd-domU included. In fact, even right now that might be useful
- for example dom0less arm64 test uses just busybox for domU rootfs, not
the whole archive (so with my changes dom0 rootfs has unused
boot/initrd-domU included).

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Andrew Cooper]

On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> Build initrd format directly digestable by the kernel. Additionally,
> include it itself inside as boot/initrd-domU file, for domU booting in
> tests.
> This should avoid the need to repack tar -> cpio as part of the test
> job.
>
> Keep generating initrd.tar.gz as that's still used by older branches.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
>  scripts/x86_64-rootfs-alpine.sh | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> index b70b3a5..13e85fa 100755
> --- a/scripts/x86_64-rootfs-alpine.sh
> +++ b/scripts/x86_64-rootfs-alpine.sh
> @@ -58,3 +58,8 @@ passwd -d "root" root
>  cd /
>  tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>      bin dev etc home init lib mnt opt root sbin usr var
> +mkdir boot
> +find bin dev etc home init lib mnt opt root sbin usr var |\
> +    cpio -o -H newc | gzip > boot/initrd-domU
> +find bin boot dev etc home init lib mnt opt root sbin usr var |\
> +    cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \

So, after looking at this a bit more, a few notes.

Trailing \ needs dropping.

initrd-domU probably ought to be named initrd-domU-base.cpio.gz so it's
clear what it is.  These are easy to fix up.

Looking through the current initrd.tar.gz, we've got:

$ ls -lah ./var/cache/apk/
total 1.9M
drwxr-xr-x 2 andrew andrew 4.0K Nov  8 15:12 .
drwxr-xr-x 4 andrew andrew 4.0K Sep  6  2024 ..
-rw-r--r-- 1 andrew andrew 425K Nov  8 15:12 APKINDEX.73fc500b.tar.gz
-rw-r--r-- 1 andrew andrew 1.4M Nov  8 15:12 APKINDEX.eb46142e.tar.gz

which (alone) is 5% of the size of the initrd.  I'll submit a patch
separately to drop this.

But doesn't this result in initrd.tar.gz being included in initrd-domU,
and then (doubly) in initrd.cpio.gz ?

~Andrew

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Andrew Cooper]

On 07/04/2025 6:11 pm, Andrew Cooper wrote:
> On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
>> Build initrd format directly digestable by the kernel. Additionally,
>> include it itself inside as boot/initrd-domU file, for domU booting in
>> tests.
>> This should avoid the need to repack tar -> cpio as part of the test
>> job.
>>
>> Keep generating initrd.tar.gz as that's still used by older branches.
>>
>> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>> ---
>>  scripts/x86_64-rootfs-alpine.sh | 5 +++++
>>  1 file changed, 5 insertions(+)
>>
>> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
>> index b70b3a5..13e85fa 100755
>> --- a/scripts/x86_64-rootfs-alpine.sh
>> +++ b/scripts/x86_64-rootfs-alpine.sh
>> @@ -58,3 +58,8 @@ passwd -d "root" root
>>  cd /
>>  tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>>      bin dev etc home init lib mnt opt root sbin usr var
>> +mkdir boot
>> +find bin dev etc home init lib mnt opt root sbin usr var |\
>> +    cpio -o -H newc | gzip > boot/initrd-domU
>> +find bin boot dev etc home init lib mnt opt root sbin usr var |\
>> +    cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
> So, after looking at this a bit more, a few notes.
>
> Trailing \ needs dropping.
>
> initrd-domU probably ought to be named initrd-domU-base.cpio.gz so it's
> clear what it is.  These are easy to fix up.
>
> Looking through the current initrd.tar.gz, we've got:
>
> $ ls -lah ./var/cache/apk/
> total 1.9M
> drwxr-xr-x 2 andrew andrew 4.0K Nov  8 15:12 .
> drwxr-xr-x 4 andrew andrew 4.0K Sep  6  2024 ..
> -rw-r--r-- 1 andrew andrew 425K Nov  8 15:12 APKINDEX.73fc500b.tar.gz
> -rw-r--r-- 1 andrew andrew 1.4M Nov  8 15:12 APKINDEX.eb46142e.tar.gz
>
> which (alone) is 5% of the size of the initrd.  I'll submit a patch
> separately to drop this.
>
> But doesn't this result in initrd.tar.gz being included in initrd-domU,
> and then (doubly) in initrd.cpio.gz ?

No, because $WORKDIR is nothing, and we put it in /binaries at the top
level which is excluded.

~Andrew

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 2387 bytes --]

On Mon, Apr 07, 2025 at 06:11:29PM +0100, Andrew Cooper wrote:
> On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> > Build initrd format directly digestable by the kernel. Additionally,
> > include it itself inside as boot/initrd-domU file, for domU booting in
> > tests.
> > This should avoid the need to repack tar -> cpio as part of the test
> > job.
> >
> > Keep generating initrd.tar.gz as that's still used by older branches.
> >
> > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > ---
> >  scripts/x86_64-rootfs-alpine.sh | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> > index b70b3a5..13e85fa 100755
> > --- a/scripts/x86_64-rootfs-alpine.sh
> > +++ b/scripts/x86_64-rootfs-alpine.sh
> > @@ -58,3 +58,8 @@ passwd -d "root" root
> >  cd /
> >  tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> >      bin dev etc home init lib mnt opt root sbin usr var
> > +mkdir boot
> > +find bin dev etc home init lib mnt opt root sbin usr var |\
> > +    cpio -o -H newc | gzip > boot/initrd-domU
> > +find bin boot dev etc home init lib mnt opt root sbin usr var |\
> > +    cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
> 
> So, after looking at this a bit more, a few notes.
> 
> Trailing \ needs dropping.
> 
> initrd-domU probably ought to be named initrd-domU-base.cpio.gz so it's
> clear what it is.  These are easy to fix up.

Technically, it's useful as is already if you just want to check if domU
starts.

> Looking through the current initrd.tar.gz, we've got:
> 
> $ ls -lah ./var/cache/apk/
> total 1.9M
> drwxr-xr-x 2 andrew andrew 4.0K Nov  8 15:12 .
> drwxr-xr-x 4 andrew andrew 4.0K Sep  6  2024 ..
> -rw-r--r-- 1 andrew andrew 425K Nov  8 15:12 APKINDEX.73fc500b.tar.gz
> -rw-r--r-- 1 andrew andrew 1.4M Nov  8 15:12 APKINDEX.eb46142e.tar.gz
> 
> which (alone) is 5% of the size of the initrd.  I'll submit a patch
> separately to drop this.

Makes sense.

> But doesn't this result in initrd.tar.gz being included in initrd-domU,
> and then (doubly) in initrd.cpio.gz ?

No, find (same as tar earlier) has explicit list of dirs to look at,
which don't include initrd.* nor "binaries".

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 06/12] CI: consistently use DOCKER_CMD in makefiles

[Andrew Cooper]

On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> This allows rebuilding containers using podman too.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
>  automation/build/Makefile           | 4 ++--
>  automation/tests-artifacts/Makefile | 4 ++--
>  2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/automation/build/Makefile b/automation/build/Makefile
> index 4df43b040777..fedf7524dacd 100644
> --- a/automation/build/Makefile
> +++ b/automation/build/Makefile
> @@ -31,8 +31,8 @@ clean:
>  define CLEAN_RULE
>  .PHONY: clean-$(1)
>  clean-$(1):
> -	if [ -n "$$$$(docker image ls -q $(REGISTRY)/$(subst /,:,$(1)))" ]; then \
> -		docker image rm $(REGISTRY)/$(subst /,:,$(1)); \
> +	if [ -n "$$$$($(DOCKER_CMD) image ls -q $(REGISTRY)/$(subst /,:,$(1)))" ]; then \
> +		$(DOCKER_CMD) image rm $(REGISTRY)/$(subst /,:,$(1)); \
>  	fi
>  
>  endef
> diff --git a/automation/tests-artifacts/Makefile b/automation/tests-artifacts/Makefile
> index d055cd696bed..80a60a94f3f7 100644
> --- a/automation/tests-artifacts/Makefile
> +++ b/automation/tests-artifacts/Makefile
> @@ -10,9 +10,9 @@ help:
>  	@echo "To push container builds, set the env var PUSH"
>  
>  %: %.dockerfile ## Builds containers
> -	docker build --pull -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
> +	$(DOCKER_CMD) build --pull -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
>  	@if [ ! -z $${PUSH+x} ]; then \
> -		docker push $(REGISTRY)/$(@D):$(@F); \
> +		$(DOCKER_CMD) push $(REGISTRY)/$(@D):$(@F); \
>  	fi
>  
>  .PHONY: all

I'm going to apply this commit to the test artefacts repo too, as it's
got a copy of this Makefile in need of the same change.

Re: [PATCH v2 08/12] CI: switch test kernel from 6.1.19 to 6.12.21

[Andrew Cooper]

On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> The jump is pretty big, but should be enough for a longer time. The
> change right now is motivated adding new runner with AMD Zen4.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
>  automation/gitlab-ci/build.yaml                      |  4 +-
>  automation/gitlab-ci/test.yaml                       |  2 +-
>  automation/tests-artifacts/kernel/6.1.19.dockerfile  | 41 +-------------
>  automation/tests-artifacts/kernel/6.12.21.dockerfile | 41 +++++++++++++-
>  4 files changed, 44 insertions(+), 44 deletions(-)
>  delete mode 100644 automation/tests-artifacts/kernel/6.1.19.dockerfile
>  create mode 100644 automation/tests-artifacts/kernel/6.12.21.dockerfile

I'd really prefer not to continue extending the *-export mess, but that
in turn needs fixes in the test-artefacts repo first.  It should
simplify later parts of this series.

~Andrew

Re: [PATCH v2 02/12] CI: switch qubes runners to use console.exp

[Andrew Cooper]

On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> diff --git a/automation/scripts/console.exp b/automation/scripts/console.exp
> index 31ce97b91b63..d1689fa5bf7f 100755
> --- a/automation/scripts/console.exp
> +++ b/automation/scripts/console.exp
> @@ -28,21 +28,34 @@ if {[info exists env(UBOOT_CMD)]} {
>      send "$env(UBOOT_CMD)\r"
>  }
>  
> +if {[info exists env(BOOT_MSG)]} {
> +    expect -re "$env(BOOT_MSG)"
> +}

We're gaining an increasingly complex set of environment variables to
control this script.  In particular, this is more a signs-of-life
message than a completion.

I think this patch wants splitting into two.  First doing the -re
changes, and adding BOOT_MSG, and some comments explaining how each part
is supposed to be used.  I expect the xilinx- jobs want to use BOOT_MSG
too, whereas I expect we don't want the qemu/cirrus smoke tests to use
it, seeing as they're all direct-boot in qemu.

The, the second patch is converting qubes-x86-64.sh to use it, adding
WAKEUP_CMD/SYSPEND_MSG as they're pretty specific.

~Andrew

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Anthony PERARD]

On Mon, Apr 07, 2025 at 06:31:06PM +0200, Marek Marczykowski-Górecki wrote:
> There is one more difference: the cpio.gz contains the whole thing
> twice. Once as rootfs for dom0 and then another as boot/initrd-domU.
> Dropping .tar.gz is probably a good idea at some point, so I'm okay with
> such comment added. But I imagine some future tests may benefit from
> just one layer, which may want introducing another cpio.gz without
> boot/initrd-domU included. In fact, even right now that might be useful
> - for example dom0less arm64 test uses just busybox for domU rootfs, not
> the whole archive (so with my changes dom0 rootfs has unused
> boot/initrd-domU included).

In such case, would it make sense to prepare several initrd.cpio.gz? A
common one, then having overlays of files we want to add or replace to
the initrd of spefic test. I think osstest used to do something like
that, with command that would be:
    cat common.cpio.gz overlay.cpio.gz > initrd.cpio.gz

If it matter, is seem that the "file.cpio" used to be made with
`cpio -Hnewc -o`

Cheers,

--

Anthony Perard | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: [PATCH test-artifacts v2 14/12] scripts: build initrd cpio

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 1859 bytes --]

On Tue, Apr 08, 2025 at 12:06:51PM +0000, Anthony PERARD wrote:
> On Mon, Apr 07, 2025 at 06:31:06PM +0200, Marek Marczykowski-Górecki wrote:
> > There is one more difference: the cpio.gz contains the whole thing
> > twice. Once as rootfs for dom0 and then another as boot/initrd-domU.
> > Dropping .tar.gz is probably a good idea at some point, so I'm okay with
> > such comment added. But I imagine some future tests may benefit from
> > just one layer, which may want introducing another cpio.gz without
> > boot/initrd-domU included. In fact, even right now that might be useful
> > - for example dom0less arm64 test uses just busybox for domU rootfs, not
> > the whole archive (so with my changes dom0 rootfs has unused
> > boot/initrd-domU included).
> 
> In such case, would it make sense to prepare several initrd.cpio.gz? A
> common one, then having overlays of files we want to add or replace to
> the initrd of spefic test. I think osstest used to do something like
> that, with command that would be:
>     cat common.cpio.gz overlay.cpio.gz > initrd.cpio.gz
> 
> If it matter, is seem that the "file.cpio" used to be made with
> `cpio -Hnewc -o`

This is already how final initrd is constructed with my changes.
Having said that, maybe test-artifacts should not include
boot/initrd-domU in initrd.cpio.gz initially, but have a separate cpio
that has just boot/initrd-domU there. Most tests do use this domU
initrd, but since they need to concatenate something anyway, maybe
starting with two files instead of one doesn't make much difference?

So, it would be:
- initrd.cpio.gz - plain rootfs, for dom0 (or domU in dom0less tests)
- initrd-in-boot.cpio.gz - the above initrd.cpio.gz packed again as
  boot/initrd-domU

What do you think?

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]