* [PATCH v1 0/4] CI: updates to XTF CI runners
@ 2025-04-16 5:32 dmkhn
2025-04-16 5:32 ` [PATCH v1 1/4] CI: unify x86 XTF test runner dmkhn
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: dmkhn @ 2025-04-16 5:32 UTC (permalink / raw)
To: xen-devel; +Cc: andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
The series started from adding new argo XTF CI job and ended up with updating
all XTF runners and related CI jobs.
It unifies the XTF runner scripts so that it is possible to use one script for
executing any XTF test under QEMU. That simplifies running XTFs locally and in
CI.
Patch 1 reworks x86 XTF runner.
Patch 2 reworks x86 EFI XTF.
Patch 3 reworks Arm64 XTF runner.
Patch 4 adds new argo x86 XTF CI job.
CI run: https://gitlab.com/xen-project/people/dmukhin/xen/-/pipelines/1770437024
Denis Mukhin (4):
CI: unify x86 XTF test runner
CI: switch x86 EFI smoke test runner to qemu-xtf.sh
CI: switch arm64 XTF test runner to qemu-xtf.sh
CI: add argo x86 XTF test
automation/gitlab-ci/test.yaml | 23 ++-
automation/scripts/.gitignore | 9 ++
.../scripts/include/configs/xtf-arm64-config | 2 +
.../scripts/include/configs/xtf-x86-64-config | 0
.../include/configs/xtf-x86-64-efi-config | 0
automation/scripts/include/xtf-arm64 | 81 ++++++++++
automation/scripts/include/xtf-runner | 141 ++++++++++++++++++
automation/scripts/include/xtf-x86-64 | 31 ++++
automation/scripts/include/xtf-x86-64-efi | 52 +++++++
automation/scripts/qemu-smoke-x86-64-efi.sh | 43 ------
automation/scripts/qemu-smoke-x86-64.sh | 26 ----
automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 ---------
automation/scripts/qemu-xtf.sh | 26 ++++
13 files changed, 358 insertions(+), 144 deletions(-)
create mode 100644 automation/scripts/.gitignore
create mode 100644 automation/scripts/include/configs/xtf-arm64-config
create mode 100644 automation/scripts/include/configs/xtf-x86-64-config
create mode 100644 automation/scripts/include/configs/xtf-x86-64-efi-config
create mode 100644 automation/scripts/include/xtf-arm64
create mode 100644 automation/scripts/include/xtf-runner
create mode 100644 automation/scripts/include/xtf-x86-64
create mode 100644 automation/scripts/include/xtf-x86-64-efi
delete mode 100755 automation/scripts/qemu-smoke-x86-64-efi.sh
delete mode 100755 automation/scripts/qemu-smoke-x86-64.sh
delete mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh
create mode 100755 automation/scripts/qemu-xtf.sh
--
2.34.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v1 1/4] CI: unify x86 XTF test runner
2025-04-16 5:32 [PATCH v1 0/4] CI: updates to XTF CI runners dmkhn
@ 2025-04-16 5:32 ` dmkhn
2025-04-17 0:16 ` Stefano Stabellini
2025-04-16 5:32 ` [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh dmkhn
` (2 subsequent siblings)
3 siblings, 1 reply; 13+ messages in thread
From: dmkhn @ 2025-04-16 5:32 UTC (permalink / raw)
To: xen-devel; +Cc: andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
From: Denis Mukhin <dmukhin@ford.com>
Add test runner script qemu-xtf.sh which is allows any XTF x86 test to be
easily executed. Test runner is invoked from the qemu-smoke* jobs with the
hardcoded parameters.
Each x86 XTF job lead time is reduced a bit since only the test-related code
is built, not the entire XTF project.
Add .gitignore to avoid committing test artifacts by mistake.
Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
automation/gitlab-ci/test.yaml | 8 +-
automation/scripts/.gitignore | 6 +
.../scripts/include/configs/xtf-x86-64-config | 0
automation/scripts/include/xtf-runner | 134 ++++++++++++++++++
automation/scripts/include/xtf-x86-64 | 31 ++++
automation/scripts/qemu-smoke-x86-64.sh | 26 ----
automation/scripts/qemu-xtf.sh | 26 ++++
7 files changed, 201 insertions(+), 30 deletions(-)
create mode 100644 automation/scripts/.gitignore
create mode 100644 automation/scripts/include/configs/xtf-x86-64-config
create mode 100644 automation/scripts/include/xtf-runner
create mode 100644 automation/scripts/include/xtf-x86-64
delete mode 100755 automation/scripts/qemu-smoke-x86-64.sh
create mode 100755 automation/scripts/qemu-xtf.sh
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 5ce445b78f..3adc841335 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -659,28 +659,28 @@ qemu-alpine-x86_64-gcc:
qemu-smoke-x86-64-gcc:
extends: .qemu-smoke-x86-64
script:
- - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh x86-64 pv64 example 2>&1 | tee ${LOGFILE}
needs:
- debian-12-x86_64-gcc-debug
qemu-smoke-x86-64-clang:
extends: .qemu-smoke-x86-64
script:
- - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh x86-64 pv64 example 2>&1 | tee ${LOGFILE}
needs:
- debian-12-x86_64-clang-debug
qemu-smoke-x86-64-gcc-pvh:
extends: .qemu-smoke-x86-64
script:
- - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh x86-64 hvm64 example 2>&1 | tee ${LOGFILE}
needs:
- debian-12-x86_64-gcc-debug
qemu-smoke-x86-64-clang-pvh:
extends: .qemu-smoke-x86-64
script:
- - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh x86-64 hvm64 example 2>&1 | tee ${LOGFILE}
needs:
- debian-12-x86_64-clang-debug
diff --git a/automation/scripts/.gitignore b/automation/scripts/.gitignore
new file mode 100644
index 0000000000..2f2d6e1ebd
--- /dev/null
+++ b/automation/scripts/.gitignore
@@ -0,0 +1,6 @@
+!include
+
+binaries
+smoke.serial
+xen
+xtf*/
diff --git a/automation/scripts/include/configs/xtf-x86-64-config b/automation/scripts/include/configs/xtf-x86-64-config
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/automation/scripts/include/xtf-runner b/automation/scripts/include/xtf-runner
new file mode 100644
index 0000000000..55b7b34b89
--- /dev/null
+++ b/automation/scripts/include/xtf-runner
@@ -0,0 +1,134 @@
+#!/bin/bash
+#
+# XTF test utilities.
+#
+# Environment variables:
+# BOOT_MSG: Expected boot message
+# FW_PREFIX: Firmware images path including '/' at the end
+# PASSED: XTF test printout in case of a pass
+# QEMU_PREFIX: QEMU path including '/' at the end
+# TEST_LOG: Output log file
+# UBOOT_CMD: U-Boot command line
+# WORKDIR: Test working directory
+# XEN_BINARY: Xen binary location
+# XEN_CONSOLE: Xen console device name
+# XTF_SRC_CONFIG: XTF config file
+# XTF_SRC_BRANCH: XTF branch
+# XTF_SRC_URI: XTF source code URI
+
+function die()
+{
+ set +x
+ echo "FATAL: $*" >&2
+ exit 1
+}
+
+[ -z "$TOP" ] && die "\$TOP is not set"
+
+# Output log file
+TEST_LOG="${TEST_LOG:-smoke.serial}"
+# XTF test printout in case of a pass
+PASSED="${PASSED:-Test result: SUCCESS}"
+# Expected boot message
+BOOT_MSG="${BOOT_MSG:-Latest ChangeSet: }"
+# Test working directory
+WORKDIR="${WORKDIR:-binaries}"
+# XTF source code
+XTF_SRC_CONFIG="${XTF_CONFIG:-${TOP}/include/configs/xtf-${ARCH}-config}"
+
+# Build an XTF test binary.
+# $1 Test variant.
+# $2 Test name.
+function xtf_build_binary()
+{
+ local xtf_variant=$1
+ local xtf_name=$2
+ local xtf_dir="xtf-${ARCH}"
+
+ # Crude check for local testing
+ if [ ! -d ${xtf_dir}/.git ]; then
+ git clone ${XTF_SRC_URI} ${xtf_dir} -b ${XTF_SRC_BRANCH}
+ fi
+
+ make \
+ -C ${xtf_dir} \
+ -j$(nproc) \
+ $(tr '\n' ' ' < ${XTF_SRC_CONFIG}) \
+ TESTS=tests/${xtf_name}
+
+ export XTF_NAME="${xtf_name}"
+ export XTF_VARIANT="${xtf_variant}"
+ export XTF_WORKDIR="$(readlink -f ${xtf_dir})"
+ export XTF_BINARY="${XTF_WORKDIR}/tests/${xtf_name}/test-${xtf_variant}-${xtf_name}"
+}
+
+# Build Xen command line for running an XTF test.
+# $1 Test variant.
+# $2 Test name.
+function xtf_build_cmdline()
+{
+ local xtf_variant=$1
+ local xtf_name=$2
+ declare -a cmdline=()
+
+ cmdline+=("loglvl=all noreboot console_timestamps=boot")
+ cmdline+=("console=${XEN_CONSOLE}")
+
+ # NB: OK to have hvm64, which is x86-only variant
+ if [[ $xtf_variant == "hvm64" ]]; then
+ cmdline+=("dom0-iommu=none dom0=pvh")
+ fi
+
+ export XEN_CMDLINE="${cmdline[@]}"
+}
+
+# Build an XTF test environment.
+# $1 Test variant.
+# $2 Test name.
+function xtf_build_test()
+{
+ local v=$1
+ local xtf_name=$2
+ local xtf_variant=""
+
+ for x in ${XTF_SRC_VARIANTS}; do
+ if [[ "${x}" == "${v}" ]]; then
+ xtf_variant=${v}
+ break
+ fi
+ done
+ if [[ -z $xtf_variant ]]; then
+ die "unsupported test variant '$1', supported variants: ${XTF_SRC_VARIANTS}"
+ fi
+
+ xtf_build_binary ${xtf_variant} ${xtf_name}
+ xtf_build_cmdline ${xtf_variant} ${xtf_name}
+}
+
+# Execute an XTF test.
+function xtf_run_test()
+{
+ rm -f ${TEST_LOG}
+ export BOOT_MSG PASSED TEST_CMD TEST_LOG UBOOT_CMD
+ ${TOP}/console.exp | sed 's/\r\+$//'
+}
+
+# Setup environment and run an XTF test.
+# $1 Test variant.
+# $2 Test name.
+function xtf_test()
+{
+ # Out: FW_*, QEMU_*, XEN_{BINARY,CONSOLE}, XTF_SRC_*
+ xtf_arch_prepare
+
+ # In: XTF_SRC_*
+ # OUt: XTF_{BINARY,NAME,VARIANT,WORKDIR} and XEN_CMDLINE
+ xtf_build_test $@
+
+ # In: FW_*, QEMU_*, XTF_*, XEN_*
+ # Out: BOOT_MSG, PASSED, TEST_{CMD,LOG}, UBOOT_CMD
+ xtf_arch_setup
+
+ # In: BOOT_MSG, PASSED, TEST_{CMD,LOG}, UBOOT_CMD
+ xtf_run_test
+}
diff --git a/automation/scripts/include/xtf-x86-64 b/automation/scripts/include/xtf-x86-64
new file mode 100644
index 0000000000..edddf18b38
--- /dev/null
+++ b/automation/scripts/include/xtf-x86-64
@@ -0,0 +1,31 @@
+#!/bin/bash
+#
+# XTF test utilities (x86_64).
+#
+
+# Arch-specific environment overrides.
+function xtf_arch_prepare()
+{
+ export FW_PREFIX="${FW_PREFIX:-}"
+ export QEMU_PREFIX="${QEMU_PREFIX:-}"
+ export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen}"
+ export XEN_CONSOLE="${XEN_CONSOLE:-com1}"
+ export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-master}"
+ export XTF_SRC_URI="${XTF_SRC_URI:-https://xenbits.xen.org/git-http/xtf.git}"
+ export XTF_SRC_VARIANTS="hvm64 pv64"
+}
+
+# Perform arch-specific XTF environment setup.
+function xtf_arch_setup()
+{
+ export TEST_CMD="${QEMU_PREFIX}qemu-system-x86_64 \
+ -no-reboot \
+ -nographic \
+ -monitor none \
+ -serial stdio \
+ -m 512 \
+ -kernel ${XEN_BINARY} \
+ -initrd ${XTF_BINARY} \
+ -append \"${XEN_CMDLINE}\" \
+ "
+}
diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
deleted file mode 100755
index da0c26cc2f..0000000000
--- a/automation/scripts/qemu-smoke-x86-64.sh
+++ /dev/null
@@ -1,26 +0,0 @@
-#!/bin/bash
-
-set -ex -o pipefail
-
-# variant should be either pv or pvh
-variant=$1
-
-# Clone and build XTF
-git clone https://xenbits.xen.org/git-http/xtf.git
-cd xtf && make -j$(nproc) && cd -
-
-case $variant in
- pvh) k=test-hvm64-example extra="dom0-iommu=none dom0=pvh" ;;
- *) k=test-pv64-example extra= ;;
-esac
-
-rm -f smoke.serial
-export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
- -initrd xtf/tests/example/$k \
- -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
- -m 512 -monitor none -serial stdio"
-
-export TEST_LOG="smoke.serial"
-export PASSED="Test result: SUCCESS"
-
-./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-xtf.sh b/automation/scripts/qemu-xtf.sh
new file mode 100755
index 0000000000..2e16d4aece
--- /dev/null
+++ b/automation/scripts/qemu-xtf.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+#
+# XTF test runner (QEMU).
+#
+
+set -e -o pipefail
+
+if [ $# -lt 3 ]; then
+ echo "Usage: $(basename $0) ARCH XTF-VARIANT XTF-NAME"
+ exit 1
+fi
+
+export ARCH="$1"
+shift
+
+export TOP="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+if [ ! -f "${TOP}/include/xtf-${ARCH}" ]; then
+ echo "unsupported architecture '${ARCH}'" >&2
+ exit 1
+fi
+
+set -x
+source ${TOP}/include/xtf-runner
+source ${TOP}/include/xtf-${ARCH}
+
+xtf_test $@
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh
2025-04-16 5:32 [PATCH v1 0/4] CI: updates to XTF CI runners dmkhn
2025-04-16 5:32 ` [PATCH v1 1/4] CI: unify x86 XTF test runner dmkhn
@ 2025-04-16 5:32 ` dmkhn
2025-04-17 0:14 ` Stefano Stabellini
2025-04-16 5:32 ` [PATCH v1 3/4] CI: switch arm64 XTF " dmkhn
2025-04-16 5:32 ` [PATCH v1 4/4] CI: add argo x86 XTF test dmkhn
3 siblings, 1 reply; 13+ messages in thread
From: dmkhn @ 2025-04-16 5:32 UTC (permalink / raw)
To: xen-devel; +Cc: andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
From: Denis Mukhin <dmukhin@ford.com>
Use qemu-xtf.sh for qemu-smoke-x86-64-gcc-efi job.
Lead time is reduced a bit since not all XTF code base is built, just the
required test.
Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
automation/gitlab-ci/test.yaml | 2 +-
.../include/configs/xtf-x86-64-efi-config | 0
automation/scripts/include/xtf-x86-64-efi | 52 +++++++++++++++++++
automation/scripts/qemu-smoke-x86-64-efi.sh | 43 ---------------
4 files changed, 53 insertions(+), 44 deletions(-)
create mode 100644 automation/scripts/include/configs/xtf-x86-64-efi-config
create mode 100644 automation/scripts/include/xtf-x86-64-efi
delete mode 100755 automation/scripts/qemu-smoke-x86-64-efi.sh
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 3adc841335..ca1e4eb528 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -687,7 +687,7 @@ qemu-smoke-x86-64-clang-pvh:
qemu-smoke-x86-64-gcc-efi:
extends: .qemu-smoke-x86-64
script:
- - ./automation/scripts/qemu-smoke-x86-64-efi.sh pv 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh x86-64-efi hvm64 example 2>&1 | tee ${LOGFILE}
needs:
- debian-12-x86_64-gcc-debug
diff --git a/automation/scripts/include/configs/xtf-x86-64-efi-config b/automation/scripts/include/configs/xtf-x86-64-efi-config
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/automation/scripts/include/xtf-x86-64-efi b/automation/scripts/include/xtf-x86-64-efi
new file mode 100644
index 0000000000..79622d5a6c
--- /dev/null
+++ b/automation/scripts/include/xtf-x86-64-efi
@@ -0,0 +1,52 @@
+#!/bin/bash
+#
+# XTF test utilities (x86_64, EFI).
+#
+
+# Arch-specific environment overrides.
+function xtf_arch_prepare()
+{
+ export FW_PREFIX="${FW_PREFIX:-/usr/share/OVMF/}"
+ export QEMU_PREFIX="${QEMU_PREFIX:-}"
+ export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen.efi}"
+ export XEN_CONSOLE="${XEN_CONSOLE:-com1}"
+ export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-master}"
+ export XTF_SRC_URI="${XTF_SRC_URI:-https://xenbits.xen.org/git-http/xtf.git}"
+ export XTF_SRC_VARIANTS="hvm64 pv64"
+}
+
+# Perform arch-specific XTF environment setup.
+function xtf_arch_setup()
+{
+ local esp_dir="${WORKDIR}/boot-esp"
+ local efi_dir="${esp_dir}/EFI/BOOT"
+
+ # Generate EFI boot environment
+ mkdir -p ${efi_dir}
+ cp ${XEN_BINARY} ${efi_dir}/BOOTX64.EFI
+ cp ${XTF_BINARY} ${efi_dir}/kernel
+
+ cat > ${efi_dir}/BOOTX64.cfg <<EOF
+[global]
+default=test
+
+[test]
+options=${XEN_CMDLINE}
+kernel=kernel
+EOF
+
+ # NB: OVMF_CODE.fd is read-only, no need to copy
+ cp ${FW_PREFIX}OVMF_VARS.fd ${WORKDIR}
+
+ export TEST_CMD="${QEMU_PREFIX}qemu-system-x86_64 \
+ -no-reboot \
+ -nographic \
+ -monitor none \
+ -serial stdio \
+ -m 512 \
+ -M q35,kernel-irqchip=split \
+ -drive if=pflash,format=raw,readonly=on,file=${FW_PREFIX}OVMF_CODE.fd \
+ -drive if=pflash,format=raw,file=${WORKDIR}/OVMF_VARS.fd \
+ -drive file=fat:rw:${esp_dir},media=disk,index=0,format=raw \
+ "
+}
diff --git a/automation/scripts/qemu-smoke-x86-64-efi.sh b/automation/scripts/qemu-smoke-x86-64-efi.sh
deleted file mode 100755
index 7572722be6..0000000000
--- a/automation/scripts/qemu-smoke-x86-64-efi.sh
+++ /dev/null
@@ -1,43 +0,0 @@
-#!/bin/bash
-
-set -ex -o pipefail
-
-# variant should be either pv or pvh
-variant=$1
-
-# Clone and build XTF
-git clone https://xenbits.xen.org/git-http/xtf.git
-cd xtf && make -j$(nproc) && cd -
-
-case $variant in
- pvh) k=test-hvm64-example extra="dom0-iommu=none dom0=pvh" ;;
- *) k=test-pv64-example extra= ;;
-esac
-
-mkdir -p boot-esp/EFI/BOOT
-cp binaries/xen.efi boot-esp/EFI/BOOT/BOOTX64.EFI
-cp xtf/tests/example/$k boot-esp/EFI/BOOT/kernel
-
-cat > boot-esp/EFI/BOOT/BOOTX64.cfg <<EOF
-[global]
-default=test
-
-[test]
-options=loglvl=all console=com1 noreboot console_timestamps=boot $extra
-kernel=kernel
-EOF
-
-cp /usr/share/OVMF/OVMF_CODE.fd OVMF_CODE.fd
-cp /usr/share/OVMF/OVMF_VARS.fd OVMF_VARS.fd
-
-rm -f smoke.serial
-export TEST_CMD="qemu-system-x86_64 -nographic -M q35,kernel-irqchip=split \
- -drive if=pflash,format=raw,readonly=on,file=OVMF_CODE.fd \
- -drive if=pflash,format=raw,file=OVMF_VARS.fd \
- -drive file=fat:rw:boot-esp,media=disk,index=0,format=raw \
- -m 512 -monitor none -serial stdio"
-
-export TEST_LOG="smoke.serial"
-export PASSED="Test result: SUCCESS"
-
-./automation/scripts/console.exp | sed 's/\r\+$//'
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v1 3/4] CI: switch arm64 XTF test runner to qemu-xtf.sh
2025-04-16 5:32 [PATCH v1 0/4] CI: updates to XTF CI runners dmkhn
2025-04-16 5:32 ` [PATCH v1 1/4] CI: unify x86 XTF test runner dmkhn
2025-04-16 5:32 ` [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh dmkhn
@ 2025-04-16 5:32 ` dmkhn
2025-04-17 0:12 ` Stefano Stabellini
2025-04-16 5:32 ` [PATCH v1 4/4] CI: add argo x86 XTF test dmkhn
3 siblings, 1 reply; 13+ messages in thread
From: dmkhn @ 2025-04-16 5:32 UTC (permalink / raw)
To: xen-devel; +Cc: andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
From: Denis Mukhin <dmukhin@ford.com>
Hook arm64 QEMU configuration to qemu-xtf.sh and use new script in arm64 CI
jobs.
Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
automation/gitlab-ci/test.yaml | 4 +-
automation/scripts/.gitignore | 3 +
.../scripts/include/configs/xtf-arm64-config | 2 +
automation/scripts/include/xtf-arm64 | 81 +++++++++++++++++++
automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 ----------------
5 files changed, 88 insertions(+), 70 deletions(-)
create mode 100644 automation/scripts/include/configs/xtf-arm64-config
create mode 100644 automation/scripts/include/xtf-arm64
delete mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index ca1e4eb528..d6e4a0a622 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -571,7 +571,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-earlyprintk:
qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
extends: .qemu-arm64
script:
- - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh arm64 mmu64le hyp-xen-version 2>&1 | tee ${LOGFILE}
needs:
- alpine-3.18-gcc-arm64
- qemu-system-aarch64-6.0.0-arm64-export
@@ -579,7 +579,7 @@ qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
extends: .qemu-arm64
script:
- - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
+ - ./automation/scripts/qemu-xtf.sh arm64 mmu64le hyp-xen-version 2>&1 | tee ${LOGFILE}
needs:
- alpine-3.18-gcc-debug-arm64
- qemu-system-aarch64-6.0.0-arm64-export
diff --git a/automation/scripts/.gitignore b/automation/scripts/.gitignore
index 2f2d6e1ebd..f853da4d89 100644
--- a/automation/scripts/.gitignore
+++ b/automation/scripts/.gitignore
@@ -1,6 +1,9 @@
!include
binaries
+imagebuilder
smoke.serial
xen
xtf*/
+
+*.rom
diff --git a/automation/scripts/include/configs/xtf-arm64-config b/automation/scripts/include/configs/xtf-arm64-config
new file mode 100644
index 0000000000..9942740927
--- /dev/null
+++ b/automation/scripts/include/configs/xtf-arm64-config
@@ -0,0 +1,2 @@
+CONFIG_GICV2=y
+CONFIG_SBSA_UART=y
diff --git a/automation/scripts/include/xtf-arm64 b/automation/scripts/include/xtf-arm64
new file mode 100644
index 0000000000..68487fe617
--- /dev/null
+++ b/automation/scripts/include/xtf-arm64
@@ -0,0 +1,81 @@
+#!/bin/bash
+#
+# XTF test utilities (arm64).
+#
+
+# Arch-specific environment overrides.
+function xtf_arch_prepare()
+{
+ export FW_PREFIX="${FW_PREFIX:-/usr/lib/u-boot/qemu_arm64/}"
+ export QEMU_PREFIX="${QEMU_PREFIX:-${WORKDIR}/}"
+ export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen}"
+ export XEN_CONSOLE="${XEN_CONSOLE:-dtuart}"
+ export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-xtf-arm}"
+ export XTF_SRC_URI="${XTF_SRC_URI:-https://gitlab.com/xen-project/fusa/xtf.git}"
+ export XTF_SRC_VARIANTS="mmu64le"
+}
+
+# Perform arch-specific XTF environment setup.
+function xtf_arch_setup()
+{
+ # QEMU looks for "efi-virtio.rom" even if it is unneeded
+ curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+ # Crude check for local testing
+ if [ ! -d imagebuilder/.git ]; then
+ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
+ fi
+
+ cat > ${WORKDIR}/config <<EOF
+MEMORY_START="0x40000000"
+MEMORY_END="0xC0000000"
+
+XEN="xen"
+DEVICE_TREE="virt-gicv2.dtb"
+
+XEN_CMD="${XEN_CMDLINE}"
+
+DOMU_KERNEL[0]="xtf-test"
+DOMU_MEM[0]="128"
+
+NUM_DOMUS=1
+
+LOAD_CMD="tftpb"
+UBOOT_SOURCE="boot.source"
+UBOOT_SCRIPT="boot.scr"
+EOF
+ cp ${XTF_BINARY} ${WORKDIR}/xtf-test
+
+ # Generate virt-gicv2.dtb
+ ${WORKDIR}/qemu-system-aarch64 \
+ -machine virtualization=true \
+ -cpu cortex-a57 \
+ -machine type=virt \
+ -m 2048 \
+ -smp 2 \
+ -display none \
+ -machine dumpdtb=${WORKDIR}/virt-gicv2.dtb
+
+ # Generate U-Boot environment
+ bash -x imagebuilder/scripts/uboot-script-gen \
+ -t tftp \
+ -d ${WORKDIR}/ \
+ -c ${WORKDIR}/config
+
+ export TEST_CMD="${QEMU_PREFIX}qemu-system-aarch64 \
+ -machine virtualization=true \
+ -cpu cortex-a57 \
+ -machine type=virt \
+ -no-reboot \
+ -nographic \
+ -monitor none \
+ -serial stdio \
+ -m 2048 \
+ -smp 2 \
+ -device virtio-net-pci,netdev=n0 \
+ -netdev user,id=n0,tftp=${WORKDIR} \
+ -bios ${FW_PREFIX}u-boot.bin \
+ "
+
+ export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
+}
diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
deleted file mode 100755
index 436f460c3c..0000000000
--- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
+++ /dev/null
@@ -1,68 +0,0 @@
-#!/bin/bash
-
-set -ex -o pipefail
-
-# Name of the XTF test
-xtf_test=$1
-
-# Message returned by XTF in case of success
-passed="Test result: SUCCESS"
-
-# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
-curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
-./binaries/qemu-system-aarch64 \
- -machine virtualization=true \
- -cpu cortex-a57 -machine type=virt \
- -m 2048 -smp 2 -display none \
- -machine dumpdtb=binaries/virt-gicv2.dtb
-
-# XTF
-# Build a single XTF test passed as a first parameter to the script.
-# Build XTF with GICv2 support to match Qemu configuration and with SBSA UART
-# support, so that the test will use an emulated UART for printing messages.
-# This will allow us to run the test on both debug and non-debug Xen builds.
-rm -rf xtf
-git clone https://gitlab.com/xen-project/fusa/xtf.git -b xtf-arm
-make -C xtf TESTS=tests/${xtf_test} CONFIG_SBSA_UART=y CONFIG_GICV2=y -j$(nproc)
-cp xtf/tests/${xtf_test}/test-mmu64le-${xtf_test} binaries/xtf-test
-
-# ImageBuilder
-echo 'MEMORY_START="0x40000000"
-MEMORY_END="0xC0000000"
-
-XEN="xen"
-DEVICE_TREE="virt-gicv2.dtb"
-
-XEN_CMD="console=dtuart console_timestamps=boot"
-
-DOMU_KERNEL[0]="xtf-test"
-DOMU_MEM[0]="128"
-
-NUM_DOMUS=1
-
-LOAD_CMD="tftpb"
-UBOOT_SOURCE="boot.source"
-UBOOT_SCRIPT="boot.scr"' > binaries/config
-
-rm -rf imagebuilder
-git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
-bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
-
-# Run the test
-rm -f smoke.serial
-export TEST_CMD="./binaries/qemu-system-aarch64 \
- -machine virtualization=true \
- -cpu cortex-a57 -machine type=virt \
- -m 2048 -monitor none -serial stdio \
- -smp 2 \
- -no-reboot \
- -device virtio-net-pci,netdev=n0 \
- -netdev user,id=n0,tftp=binaries \
- -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
-
-export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
-export BOOT_MSG="Latest ChangeSet: "
-export TEST_LOG="smoke.serial"
-export PASSED="${passed}"
-
-./automation/scripts/console.exp | sed 's/\r\+$//'
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v1 4/4] CI: add argo x86 XTF test
2025-04-16 5:32 [PATCH v1 0/4] CI: updates to XTF CI runners dmkhn
` (2 preceding siblings ...)
2025-04-16 5:32 ` [PATCH v1 3/4] CI: switch arm64 XTF " dmkhn
@ 2025-04-16 5:32 ` dmkhn
2025-04-17 0:11 ` Stefano Stabellini
3 siblings, 1 reply; 13+ messages in thread
From: dmkhn @ 2025-04-16 5:32 UTC (permalink / raw)
To: xen-devel; +Cc: andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
From: Denis Mukhin <dmukhin@ford.com>
Introduce new CI job to run x86 XTF argo test under QEMU to smoke test argo
feature functionality in upstream CI.
The new job lead time is ~30s, limit max job duration to 60s.
Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
automation/gitlab-ci/test.yaml | 9 +++++++++
automation/scripts/include/xtf-runner | 7 +++++++
2 files changed, 16 insertions(+)
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index d6e4a0a622..9001efb45c 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -691,6 +691,15 @@ qemu-smoke-x86-64-gcc-efi:
needs:
- debian-12-x86_64-gcc-debug
+qemu-xtf-argo-x86_64-gcc-debug:
+ extends: .qemu-smoke-x86-64
+ variables:
+ TEST_TIMEOUT_OVERRIDE: 60
+ script:
+ - ./automation/scripts/qemu-xtf.sh x86-64 pv64 argo 2>&1 | tee ${LOGFILE}
+ needs:
+ - alpine-3.18-gcc-debug
+
qemu-smoke-riscv64-gcc:
extends: .qemu-riscv64
script:
diff --git a/automation/scripts/include/xtf-runner b/automation/scripts/include/xtf-runner
index 55b7b34b89..12470b3979 100644
--- a/automation/scripts/include/xtf-runner
+++ b/automation/scripts/include/xtf-runner
@@ -70,6 +70,9 @@ function xtf_build_cmdline()
local xtf_variant=$1
local xtf_name=$2
declare -a cmdline=()
+ declare -A per_test_args=(
+ [argo]="argo=1 mac-permissive=1"
+ )
cmdline+=("loglvl=all noreboot console_timestamps=boot")
cmdline+=("console=${XEN_CONSOLE}")
@@ -79,6 +82,10 @@ function xtf_build_cmdline()
cmdline+=("dom0-iommu=none dom0=pvh")
fi
+ if [[ -v per_test_args[${xtf_name}] ]]; then
+ cmdline+=("${per_test_args[${xtf_name}]}")
+ fi
+
export XEN_CMDLINE="${cmdline[@]}"
}
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v1 4/4] CI: add argo x86 XTF test
2025-04-16 5:32 ` [PATCH v1 4/4] CI: add argo x86 XTF test dmkhn
@ 2025-04-17 0:11 ` Stefano Stabellini
2025-04-17 18:37 ` dmkhn
0 siblings, 1 reply; 13+ messages in thread
From: Stefano Stabellini @ 2025-04-17 0:11 UTC (permalink / raw)
To: dmkhn; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> From: Denis Mukhin <dmukhin@ford.com>
>
> Introduce new CI job to run x86 XTF argo test under QEMU to smoke test argo
> feature functionality in upstream CI.
>
> The new job lead time is ~30s, limit max job duration to 60s.
>
> Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> ---
> automation/gitlab-ci/test.yaml | 9 +++++++++
> automation/scripts/include/xtf-runner | 7 +++++++
> 2 files changed, 16 insertions(+)
>
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index d6e4a0a622..9001efb45c 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -691,6 +691,15 @@ qemu-smoke-x86-64-gcc-efi:
> needs:
> - debian-12-x86_64-gcc-debug
>
> +qemu-xtf-argo-x86_64-gcc-debug:
> + extends: .qemu-smoke-x86-64
> + variables:
> + TEST_TIMEOUT_OVERRIDE: 60
> + script:
> + - ./automation/scripts/qemu-xtf.sh x86-64 pv64 argo 2>&1 | tee ${LOGFILE}
> + needs:
> + - alpine-3.18-gcc-debug
> +
> qemu-smoke-riscv64-gcc:
> extends: .qemu-riscv64
> script:
> diff --git a/automation/scripts/include/xtf-runner b/automation/scripts/include/xtf-runner
> index 55b7b34b89..12470b3979 100644
> --- a/automation/scripts/include/xtf-runner
> +++ b/automation/scripts/include/xtf-runner
> @@ -70,6 +70,9 @@ function xtf_build_cmdline()
> local xtf_variant=$1
> local xtf_name=$2
> declare -a cmdline=()
> + declare -A per_test_args=(
> + [argo]="argo=1 mac-permissive=1"
> + )
> +
> cmdline+=("loglvl=all noreboot console_timestamps=boot")
> cmdline+=("console=${XEN_CONSOLE}")
> @@ -79,6 +82,10 @@ function xtf_build_cmdline()
> cmdline+=("dom0-iommu=none dom0=pvh")
> fi
>
> + if [[ -v per_test_args[${xtf_name}] ]]; then
> + cmdline+=("${per_test_args[${xtf_name}]}")
> + fi
> +
This is all within the same bash function so I think we could avoid
using the per_test_args array and just do:
if test "$xtf_name"
then
cmdline+=("argo=1 mac-permissive=1")
fi
The indirection is not required as far as I can tell. I do realize it is
a matter of taste and it works either way so I wouldn't insist.
> export XEN_CMDLINE="${cmdline[@]}"
> }
>
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 3/4] CI: switch arm64 XTF test runner to qemu-xtf.sh
2025-04-16 5:32 ` [PATCH v1 3/4] CI: switch arm64 XTF " dmkhn
@ 2025-04-17 0:12 ` Stefano Stabellini
2025-04-17 18:51 ` dmkhn
0 siblings, 1 reply; 13+ messages in thread
From: Stefano Stabellini @ 2025-04-17 0:12 UTC (permalink / raw)
To: dmkhn; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> From: Denis Mukhin <dmukhin@ford.com>
>
> Hook arm64 QEMU configuration to qemu-xtf.sh and use new script in arm64 CI
> jobs.
>
> Signed-off-by: Denis Mukhin <dmukhin@ford.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> automation/gitlab-ci/test.yaml | 4 +-
> automation/scripts/.gitignore | 3 +
> .../scripts/include/configs/xtf-arm64-config | 2 +
> automation/scripts/include/xtf-arm64 | 81 +++++++++++++++++++
> automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 ----------------
> 5 files changed, 88 insertions(+), 70 deletions(-)
> create mode 100644 automation/scripts/include/configs/xtf-arm64-config
> create mode 100644 automation/scripts/include/xtf-arm64
> delete mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh
>
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index ca1e4eb528..d6e4a0a622 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -571,7 +571,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-earlyprintk:
> qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
> extends: .qemu-arm64
> script:
> - - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh arm64 mmu64le hyp-xen-version 2>&1 | tee ${LOGFILE}
> needs:
> - alpine-3.18-gcc-arm64
> - qemu-system-aarch64-6.0.0-arm64-export
> @@ -579,7 +579,7 @@ qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
> qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
> extends: .qemu-arm64
> script:
> - - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh arm64 mmu64le hyp-xen-version 2>&1 | tee ${LOGFILE}
> needs:
> - alpine-3.18-gcc-debug-arm64
> - qemu-system-aarch64-6.0.0-arm64-export
> diff --git a/automation/scripts/.gitignore b/automation/scripts/.gitignore
> index 2f2d6e1ebd..f853da4d89 100644
> --- a/automation/scripts/.gitignore
> +++ b/automation/scripts/.gitignore
> @@ -1,6 +1,9 @@
> !include
>
> binaries
> +imagebuilder
> smoke.serial
> xen
> xtf*/
> +
> +*.rom
> diff --git a/automation/scripts/include/configs/xtf-arm64-config b/automation/scripts/include/configs/xtf-arm64-config
> new file mode 100644
> index 0000000000..9942740927
> --- /dev/null
> +++ b/automation/scripts/include/configs/xtf-arm64-config
> @@ -0,0 +1,2 @@
> +CONFIG_GICV2=y
> +CONFIG_SBSA_UART=y
> diff --git a/automation/scripts/include/xtf-arm64 b/automation/scripts/include/xtf-arm64
> new file mode 100644
> index 0000000000..68487fe617
> --- /dev/null
> +++ b/automation/scripts/include/xtf-arm64
> @@ -0,0 +1,81 @@
> +#!/bin/bash
> +#
> +# XTF test utilities (arm64).
> +#
> +
> +# Arch-specific environment overrides.
> +function xtf_arch_prepare()
> +{
> + export FW_PREFIX="${FW_PREFIX:-/usr/lib/u-boot/qemu_arm64/}"
> + export QEMU_PREFIX="${QEMU_PREFIX:-${WORKDIR}/}"
> + export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen}"
> + export XEN_CONSOLE="${XEN_CONSOLE:-dtuart}"
> + export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-xtf-arm}"
> + export XTF_SRC_URI="${XTF_SRC_URI:-https://gitlab.com/xen-project/fusa/xtf.git}"
> + export XTF_SRC_VARIANTS="mmu64le"
> +}
> +
> +# Perform arch-specific XTF environment setup.
> +function xtf_arch_setup()
> +{
> + # QEMU looks for "efi-virtio.rom" even if it is unneeded
> + curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> +
> + # Crude check for local testing
> + if [ ! -d imagebuilder/.git ]; then
> + git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
> + fi
> +
> + cat > ${WORKDIR}/config <<EOF
> +MEMORY_START="0x40000000"
> +MEMORY_END="0xC0000000"
> +
> +XEN="xen"
> +DEVICE_TREE="virt-gicv2.dtb"
> +
> +XEN_CMD="${XEN_CMDLINE}"
> +
> +DOMU_KERNEL[0]="xtf-test"
> +DOMU_MEM[0]="128"
> +
> +NUM_DOMUS=1
> +
> +LOAD_CMD="tftpb"
> +UBOOT_SOURCE="boot.source"
> +UBOOT_SCRIPT="boot.scr"
> +EOF
> + cp ${XTF_BINARY} ${WORKDIR}/xtf-test
> +
> + # Generate virt-gicv2.dtb
> + ${WORKDIR}/qemu-system-aarch64 \
> + -machine virtualization=true \
> + -cpu cortex-a57 \
> + -machine type=virt \
> + -m 2048 \
> + -smp 2 \
> + -display none \
> + -machine dumpdtb=${WORKDIR}/virt-gicv2.dtb
> +
> + # Generate U-Boot environment
> + bash -x imagebuilder/scripts/uboot-script-gen \
> + -t tftp \
> + -d ${WORKDIR}/ \
> + -c ${WORKDIR}/config
> +
> + export TEST_CMD="${QEMU_PREFIX}qemu-system-aarch64 \
> + -machine virtualization=true \
> + -cpu cortex-a57 \
> + -machine type=virt \
> + -no-reboot \
> + -nographic \
> + -monitor none \
> + -serial stdio \
> + -m 2048 \
> + -smp 2 \
> + -device virtio-net-pci,netdev=n0 \
> + -netdev user,id=n0,tftp=${WORKDIR} \
> + -bios ${FW_PREFIX}u-boot.bin \
> + "
> +
> + export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> +}
> diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> deleted file mode 100755
> index 436f460c3c..0000000000
> --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
> +++ /dev/null
> @@ -1,68 +0,0 @@
> -#!/bin/bash
> -
> -set -ex -o pipefail
> -
> -# Name of the XTF test
> -xtf_test=$1
> -
> -# Message returned by XTF in case of success
> -passed="Test result: SUCCESS"
> -
> -# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
> -curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> -./binaries/qemu-system-aarch64 \
> - -machine virtualization=true \
> - -cpu cortex-a57 -machine type=virt \
> - -m 2048 -smp 2 -display none \
> - -machine dumpdtb=binaries/virt-gicv2.dtb
> -
> -# XTF
> -# Build a single XTF test passed as a first parameter to the script.
> -# Build XTF with GICv2 support to match Qemu configuration and with SBSA UART
> -# support, so that the test will use an emulated UART for printing messages.
> -# This will allow us to run the test on both debug and non-debug Xen builds.
> -rm -rf xtf
> -git clone https://gitlab.com/xen-project/fusa/xtf.git -b xtf-arm
> -make -C xtf TESTS=tests/${xtf_test} CONFIG_SBSA_UART=y CONFIG_GICV2=y -j$(nproc)
> -cp xtf/tests/${xtf_test}/test-mmu64le-${xtf_test} binaries/xtf-test
> -
> -# ImageBuilder
> -echo 'MEMORY_START="0x40000000"
> -MEMORY_END="0xC0000000"
> -
> -XEN="xen"
> -DEVICE_TREE="virt-gicv2.dtb"
> -
> -XEN_CMD="console=dtuart console_timestamps=boot"
> -
> -DOMU_KERNEL[0]="xtf-test"
> -DOMU_MEM[0]="128"
> -
> -NUM_DOMUS=1
> -
> -LOAD_CMD="tftpb"
> -UBOOT_SOURCE="boot.source"
> -UBOOT_SCRIPT="boot.scr"' > binaries/config
> -
> -rm -rf imagebuilder
> -git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
> -bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
> -
> -# Run the test
> -rm -f smoke.serial
> -export TEST_CMD="./binaries/qemu-system-aarch64 \
> - -machine virtualization=true \
> - -cpu cortex-a57 -machine type=virt \
> - -m 2048 -monitor none -serial stdio \
> - -smp 2 \
> - -no-reboot \
> - -device virtio-net-pci,netdev=n0 \
> - -netdev user,id=n0,tftp=binaries \
> - -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
> -
> -export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> -export BOOT_MSG="Latest ChangeSet: "
> -export TEST_LOG="smoke.serial"
> -export PASSED="${passed}"
> -
> -./automation/scripts/console.exp | sed 's/\r\+$//'
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh
2025-04-16 5:32 ` [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh dmkhn
@ 2025-04-17 0:14 ` Stefano Stabellini
2025-04-17 18:34 ` dmkhn
0 siblings, 1 reply; 13+ messages in thread
From: Stefano Stabellini @ 2025-04-17 0:14 UTC (permalink / raw)
To: dmkhn; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> From: Denis Mukhin <dmukhin@ford.com>
>
> Use qemu-xtf.sh for qemu-smoke-x86-64-gcc-efi job.
>
> Lead time is reduced a bit since not all XTF code base is built, just the
> required test.
>
> Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> ---
> automation/gitlab-ci/test.yaml | 2 +-
> .../include/configs/xtf-x86-64-efi-config | 0
> automation/scripts/include/xtf-x86-64-efi | 52 +++++++++++++++++++
> automation/scripts/qemu-smoke-x86-64-efi.sh | 43 ---------------
> 4 files changed, 53 insertions(+), 44 deletions(-)
> create mode 100644 automation/scripts/include/configs/xtf-x86-64-efi-config
> create mode 100644 automation/scripts/include/xtf-x86-64-efi
> delete mode 100755 automation/scripts/qemu-smoke-x86-64-efi.sh
>
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 3adc841335..ca1e4eb528 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -687,7 +687,7 @@ qemu-smoke-x86-64-clang-pvh:
> qemu-smoke-x86-64-gcc-efi:
> extends: .qemu-smoke-x86-64
> script:
> - - ./automation/scripts/qemu-smoke-x86-64-efi.sh pv 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh x86-64-efi hvm64 example 2>&1 | tee ${LOGFILE}
> needs:
> - debian-12-x86_64-gcc-debug
>
> diff --git a/automation/scripts/include/configs/xtf-x86-64-efi-config b/automation/scripts/include/configs/xtf-x86-64-efi-config
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/automation/scripts/include/xtf-x86-64-efi b/automation/scripts/include/xtf-x86-64-efi
> new file mode 100644
> index 0000000000..79622d5a6c
> --- /dev/null
> +++ b/automation/scripts/include/xtf-x86-64-efi
> @@ -0,0 +1,52 @@
> +#!/bin/bash
> +#
> +# XTF test utilities (x86_64, EFI).
> +#
> +
> +# Arch-specific environment overrides.
> +function xtf_arch_prepare()
> +{
> + export FW_PREFIX="${FW_PREFIX:-/usr/share/OVMF/}"
> + export QEMU_PREFIX="${QEMU_PREFIX:-}"
> + export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen.efi}"
Any chance we can avoid using WORKDIR? Or alternatively, we can remove
TOP? Or remove both? :-)
Maybe it is best to keep WORKDIR and remove TOP, I am not sure. I am
just trying to reduce the amount of variables used and that we depend
upon.
I assume you tested the pipeline and it worked, right?
> + export XEN_CONSOLE="${XEN_CONSOLE:-com1}"
> + export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-master}"
> + export XTF_SRC_URI="${XTF_SRC_URI:-https://xenbits.xen.org/git-http/xtf.git}"
> + export XTF_SRC_VARIANTS="hvm64 pv64"
> +}
> +
> +# Perform arch-specific XTF environment setup.
> +function xtf_arch_setup()
> +{
> + local esp_dir="${WORKDIR}/boot-esp"
> + local efi_dir="${esp_dir}/EFI/BOOT"
> +
> + # Generate EFI boot environment
> + mkdir -p ${efi_dir}
> + cp ${XEN_BINARY} ${efi_dir}/BOOTX64.EFI
> + cp ${XTF_BINARY} ${efi_dir}/kernel
> +
> + cat > ${efi_dir}/BOOTX64.cfg <<EOF
> +[global]
> +default=test
> +
> +[test]
> +options=${XEN_CMDLINE}
> +kernel=kernel
> +EOF
> +
> + # NB: OVMF_CODE.fd is read-only, no need to copy
> + cp ${FW_PREFIX}OVMF_VARS.fd ${WORKDIR}
> +
> + export TEST_CMD="${QEMU_PREFIX}qemu-system-x86_64 \
> + -no-reboot \
> + -nographic \
> + -monitor none \
> + -serial stdio \
> + -m 512 \
> + -M q35,kernel-irqchip=split \
> + -drive if=pflash,format=raw,readonly=on,file=${FW_PREFIX}OVMF_CODE.fd \
> + -drive if=pflash,format=raw,file=${WORKDIR}/OVMF_VARS.fd \
> + -drive file=fat:rw:${esp_dir},media=disk,index=0,format=raw \
> + "
> +}
> diff --git a/automation/scripts/qemu-smoke-x86-64-efi.sh b/automation/scripts/qemu-smoke-x86-64-efi.sh
> deleted file mode 100755
> index 7572722be6..0000000000
> --- a/automation/scripts/qemu-smoke-x86-64-efi.sh
> +++ /dev/null
> @@ -1,43 +0,0 @@
> -#!/bin/bash
> -
> -set -ex -o pipefail
> -
> -# variant should be either pv or pvh
> -variant=$1
> -
> -# Clone and build XTF
> -git clone https://xenbits.xen.org/git-http/xtf.git
> -cd xtf && make -j$(nproc) && cd -
> -
> -case $variant in
> - pvh) k=test-hvm64-example extra="dom0-iommu=none dom0=pvh" ;;
> - *) k=test-pv64-example extra= ;;
> -esac
> -
> -mkdir -p boot-esp/EFI/BOOT
> -cp binaries/xen.efi boot-esp/EFI/BOOT/BOOTX64.EFI
> -cp xtf/tests/example/$k boot-esp/EFI/BOOT/kernel
> -
> -cat > boot-esp/EFI/BOOT/BOOTX64.cfg <<EOF
> -[global]
> -default=test
> -
> -[test]
> -options=loglvl=all console=com1 noreboot console_timestamps=boot $extra
> -kernel=kernel
> -EOF
> -
> -cp /usr/share/OVMF/OVMF_CODE.fd OVMF_CODE.fd
> -cp /usr/share/OVMF/OVMF_VARS.fd OVMF_VARS.fd
> -
> -rm -f smoke.serial
> -export TEST_CMD="qemu-system-x86_64 -nographic -M q35,kernel-irqchip=split \
> - -drive if=pflash,format=raw,readonly=on,file=OVMF_CODE.fd \
> - -drive if=pflash,format=raw,file=OVMF_VARS.fd \
> - -drive file=fat:rw:boot-esp,media=disk,index=0,format=raw \
> - -m 512 -monitor none -serial stdio"
> -
> -export TEST_LOG="smoke.serial"
> -export PASSED="Test result: SUCCESS"
> -
> -./automation/scripts/console.exp | sed 's/\r\+$//'
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 1/4] CI: unify x86 XTF test runner
2025-04-16 5:32 ` [PATCH v1 1/4] CI: unify x86 XTF test runner dmkhn
@ 2025-04-17 0:16 ` Stefano Stabellini
2025-04-17 18:50 ` dmkhn
0 siblings, 1 reply; 13+ messages in thread
From: Stefano Stabellini @ 2025-04-17 0:16 UTC (permalink / raw)
To: dmkhn; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, sstabellini, dmukhin
On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> From: Denis Mukhin <dmukhin@ford.com>
>
> Add test runner script qemu-xtf.sh which is allows any XTF x86 test to be
> easily executed. Test runner is invoked from the qemu-smoke* jobs with the
> hardcoded parameters.
>
> Each x86 XTF job lead time is reduced a bit since only the test-related code
> is built, not the entire XTF project.
>
> Add .gitignore to avoid committing test artifacts by mistake.
>
> Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> ---
> automation/gitlab-ci/test.yaml | 8 +-
> automation/scripts/.gitignore | 6 +
> .../scripts/include/configs/xtf-x86-64-config | 0
> automation/scripts/include/xtf-runner | 134 ++++++++++++++++++
> automation/scripts/include/xtf-x86-64 | 31 ++++
> automation/scripts/qemu-smoke-x86-64.sh | 26 ----
> automation/scripts/qemu-xtf.sh | 26 ++++
> 7 files changed, 201 insertions(+), 30 deletions(-)
> create mode 100644 automation/scripts/.gitignore
> create mode 100644 automation/scripts/include/configs/xtf-x86-64-config
> create mode 100644 automation/scripts/include/xtf-runner
> create mode 100644 automation/scripts/include/xtf-x86-64
> delete mode 100755 automation/scripts/qemu-smoke-x86-64.sh
> create mode 100755 automation/scripts/qemu-xtf.sh
>
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 5ce445b78f..3adc841335 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -659,28 +659,28 @@ qemu-alpine-x86_64-gcc:
> qemu-smoke-x86-64-gcc:
> extends: .qemu-smoke-x86-64
> script:
> - - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh x86-64 pv64 example 2>&1 | tee ${LOGFILE}
> needs:
> - debian-12-x86_64-gcc-debug
>
> qemu-smoke-x86-64-clang:
> extends: .qemu-smoke-x86-64
> script:
> - - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh x86-64 pv64 example 2>&1 | tee ${LOGFILE}
> needs:
> - debian-12-x86_64-clang-debug
>
> qemu-smoke-x86-64-gcc-pvh:
> extends: .qemu-smoke-x86-64
> script:
> - - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh x86-64 hvm64 example 2>&1 | tee ${LOGFILE}
> needs:
> - debian-12-x86_64-gcc-debug
>
> qemu-smoke-x86-64-clang-pvh:
> extends: .qemu-smoke-x86-64
> script:
> - - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
> + - ./automation/scripts/qemu-xtf.sh x86-64 hvm64 example 2>&1 | tee ${LOGFILE}
> needs:
> - debian-12-x86_64-clang-debug
>
> diff --git a/automation/scripts/.gitignore b/automation/scripts/.gitignore
> new file mode 100644
> index 0000000000..2f2d6e1ebd
> --- /dev/null
> +++ b/automation/scripts/.gitignore
> @@ -0,0 +1,6 @@
> +!include
> +
> +binaries
> +smoke.serial
> +xen
> +xtf*/
I am not sure this works as intended: I thought the paths are relative
to the location of the .gitignore file, but I could be wrong.
> diff --git a/automation/scripts/include/configs/xtf-x86-64-config b/automation/scripts/include/configs/xtf-x86-64-config
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/automation/scripts/include/xtf-runner b/automation/scripts/include/xtf-runner
> new file mode 100644
> index 0000000000..55b7b34b89
> --- /dev/null
> +++ b/automation/scripts/include/xtf-runner
> @@ -0,0 +1,134 @@
> +#!/bin/bash
> +#
> +# XTF test utilities.
> +#
> +# Environment variables:
> +# BOOT_MSG: Expected boot message
> +# FW_PREFIX: Firmware images path including '/' at the end
> +# PASSED: XTF test printout in case of a pass
> +# QEMU_PREFIX: QEMU path including '/' at the end
> +# TEST_LOG: Output log file
> +# UBOOT_CMD: U-Boot command line
> +# WORKDIR: Test working directory
> +# XEN_BINARY: Xen binary location
> +# XEN_CONSOLE: Xen console device name
> +# XTF_SRC_CONFIG: XTF config file
> +# XTF_SRC_BRANCH: XTF branch
> +# XTF_SRC_URI: XTF source code URI
> +
> +function die()
> +{
> + set +x
> + echo "FATAL: $*" >&2
> + exit 1
> +}
> +
> +[ -z "$TOP" ] && die "\$TOP is not set"
> +
> +# Output log file
> +TEST_LOG="${TEST_LOG:-smoke.serial}"
> +# XTF test printout in case of a pass
> +PASSED="${PASSED:-Test result: SUCCESS}"
> +# Expected boot message
> +BOOT_MSG="${BOOT_MSG:-Latest ChangeSet: }"
> +# Test working directory
> +WORKDIR="${WORKDIR:-binaries}"
> +# XTF source code
> +XTF_SRC_CONFIG="${XTF_CONFIG:-${TOP}/include/configs/xtf-${ARCH}-config}"
Should this be XTF_SRC_CONFIG="${XTF_SRC_CONFIG:- ?
> +# Build an XTF test binary.
> +# $1 Test variant.
> +# $2 Test name.
> +function xtf_build_binary()
> +{
> + local xtf_variant=$1
> + local xtf_name=$2
> + local xtf_dir="xtf-${ARCH}"
> +
> + # Crude check for local testing
> + if [ ! -d ${xtf_dir}/.git ]; then
> + git clone ${XTF_SRC_URI} ${xtf_dir} -b ${XTF_SRC_BRANCH}
> + fi
> +
> + make \
> + -C ${xtf_dir} \
> + -j$(nproc) \
> + $(tr '\n' ' ' < ${XTF_SRC_CONFIG}) \
> + TESTS=tests/${xtf_name}
> +
> + export XTF_NAME="${xtf_name}"
> + export XTF_VARIANT="${xtf_variant}"
> + export XTF_WORKDIR="$(readlink -f ${xtf_dir})"
> + export XTF_BINARY="${XTF_WORKDIR}/tests/${xtf_name}/test-${xtf_variant}-${xtf_name}"
> +}
> +
> +# Build Xen command line for running an XTF test.
> +# $1 Test variant.
> +# $2 Test name.
> +function xtf_build_cmdline()
> +{
> + local xtf_variant=$1
> + local xtf_name=$2
> + declare -a cmdline=()
> +
> + cmdline+=("loglvl=all noreboot console_timestamps=boot")
> + cmdline+=("console=${XEN_CONSOLE}")
> +
> + # NB: OK to have hvm64, which is x86-only variant
> + if [[ $xtf_variant == "hvm64" ]]; then
> + cmdline+=("dom0-iommu=none dom0=pvh")
> + fi
> +
> + export XEN_CMDLINE="${cmdline[@]}"
> +}
> +
> +# Build an XTF test environment.
> +# $1 Test variant.
> +# $2 Test name.
> +function xtf_build_test()
> +{
> + local v=$1
> + local xtf_name=$2
> + local xtf_variant=""
> +
> + for x in ${XTF_SRC_VARIANTS}; do
> + if [[ "${x}" == "${v}" ]]; then
> + xtf_variant=${v}
> + break
> + fi
> + done
> + if [[ -z $xtf_variant ]]; then
> + die "unsupported test variant '$1', supported variants: ${XTF_SRC_VARIANTS}"
> + fi
> +
> + xtf_build_binary ${xtf_variant} ${xtf_name}
> + xtf_build_cmdline ${xtf_variant} ${xtf_name}
> +}
> +
> +# Execute an XTF test.
> +function xtf_run_test()
> +{
> + rm -f ${TEST_LOG}
> + export BOOT_MSG PASSED TEST_CMD TEST_LOG UBOOT_CMD
> + ${TOP}/console.exp | sed 's/\r\+$//'
> +}
> +
> +# Setup environment and run an XTF test.
> +# $1 Test variant.
> +# $2 Test name.
> +function xtf_test()
> +{
> + # Out: FW_*, QEMU_*, XEN_{BINARY,CONSOLE}, XTF_SRC_*
> + xtf_arch_prepare
> +
> + # In: XTF_SRC_*
> + # OUt: XTF_{BINARY,NAME,VARIANT,WORKDIR} and XEN_CMDLINE
> + xtf_build_test $@
> +
> + # In: FW_*, QEMU_*, XTF_*, XEN_*
> + # Out: BOOT_MSG, PASSED, TEST_{CMD,LOG}, UBOOT_CMD
> + xtf_arch_setup
> +
> + # In: BOOT_MSG, PASSED, TEST_{CMD,LOG}, UBOOT_CMD
> + xtf_run_test
> +}
> diff --git a/automation/scripts/include/xtf-x86-64 b/automation/scripts/include/xtf-x86-64
> new file mode 100644
> index 0000000000..edddf18b38
> --- /dev/null
> +++ b/automation/scripts/include/xtf-x86-64
> @@ -0,0 +1,31 @@
> +#!/bin/bash
> +#
> +# XTF test utilities (x86_64).
> +#
> +
> +# Arch-specific environment overrides.
> +function xtf_arch_prepare()
> +{
> + export FW_PREFIX="${FW_PREFIX:-}"
> + export QEMU_PREFIX="${QEMU_PREFIX:-}"
It looks like QEMU_PREFIX is only used within this file (and other
similar files in later patches). If so, maybe for the sake of reducing
global variables and variables in general we could get rid of
QEMU_PREFIX.
> + export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen}"
> + export XEN_CONSOLE="${XEN_CONSOLE:-com1}"
Instead of XEN_CONSOLE, I'd suggest to have an arch-specific variable
for Xen command line options where we can put the console as well as
other things. However, it is also totally fine as is.
In fact, I think you went above and beyond in terms of generic
abstractions, and I think it would have been OK to go with fewer
variables, and more ad-hoc custom code.
> + export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-master}"
> + export XTF_SRC_URI="${XTF_SRC_URI:-https://xenbits.xen.org/git-http/xtf.git}"
> + export XTF_SRC_VARIANTS="hvm64 pv64"
> +}
> +
> +# Perform arch-specific XTF environment setup.
> +function xtf_arch_setup()
> +{
> + export TEST_CMD="${QEMU_PREFIX}qemu-system-x86_64 \
> + -no-reboot \
> + -nographic \
> + -monitor none \
> + -serial stdio \
> + -m 512 \
> + -kernel ${XEN_BINARY} \
> + -initrd ${XTF_BINARY} \
> + -append \"${XEN_CMDLINE}\" \
> + "
> +}
> diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
> deleted file mode 100755
> index da0c26cc2f..0000000000
> --- a/automation/scripts/qemu-smoke-x86-64.sh
> +++ /dev/null
> @@ -1,26 +0,0 @@
> -#!/bin/bash
> -
> -set -ex -o pipefail
> -
> -# variant should be either pv or pvh
> -variant=$1
> -
> -# Clone and build XTF
> -git clone https://xenbits.xen.org/git-http/xtf.git
> -cd xtf && make -j$(nproc) && cd -
> -
> -case $variant in
> - pvh) k=test-hvm64-example extra="dom0-iommu=none dom0=pvh" ;;
> - *) k=test-pv64-example extra= ;;
> -esac
> -
> -rm -f smoke.serial
> -export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
> - -initrd xtf/tests/example/$k \
> - -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
> - -m 512 -monitor none -serial stdio"
> -
> -export TEST_LOG="smoke.serial"
> -export PASSED="Test result: SUCCESS"
> -
> -./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-xtf.sh b/automation/scripts/qemu-xtf.sh
> new file mode 100755
> index 0000000000..2e16d4aece
> --- /dev/null
> +++ b/automation/scripts/qemu-xtf.sh
> @@ -0,0 +1,26 @@
> +#!/bin/bash
> +#
> +# XTF test runner (QEMU).
> +#
> +
> +set -e -o pipefail
> +
> +if [ $# -lt 3 ]; then
> + echo "Usage: $(basename $0) ARCH XTF-VARIANT XTF-NAME"
> + exit 1
> +fi
> +
> +export ARCH="$1"
> +shift
> +
> +export TOP="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
> +if [ ! -f "${TOP}/include/xtf-${ARCH}" ]; then
> + echo "unsupported architecture '${ARCH}'" >&2
> + exit 1
> +fi
This seems a bit complex.. maybe we can assume that we are in xen.git/ ?
> +set -x
> +source ${TOP}/include/xtf-runner
> +source ${TOP}/include/xtf-${ARCH}
So here we could do:
source automation/scripts/include/xtf-runner
? Or is it important to be able to detect and calculate the directory?
If so, why not use TOP=$(pwd) or TOP=${PWD} or TOP=$(dirname "$0")?
By the way, if we need a variable for the top directory I would call it
XEN_ROOT instead of TOP.
I see the patches are using both TOP and WORKDIR, maybe we can get rid
of one of them? This is only a matter of taste but I think it would be
simpler with fewer variables and most of the others look unavoidable.
Everything else looks OK to me.
> +xtf_test $@
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh
2025-04-17 0:14 ` Stefano Stabellini
@ 2025-04-17 18:34 ` dmkhn
0 siblings, 0 replies; 13+ messages in thread
From: dmkhn @ 2025-04-17 18:34 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, dmukhin
On Wed, Apr 16, 2025 at 05:14:41PM -0700, Stefano Stabellini wrote:
> On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> > From: Denis Mukhin <dmukhin@ford.com>
> >
> > Use qemu-xtf.sh for qemu-smoke-x86-64-gcc-efi job.
> >
> > Lead time is reduced a bit since not all XTF code base is built, just the
> > required test.
> >
> > Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> > ---
> > automation/gitlab-ci/test.yaml | 2 +-
> > .../include/configs/xtf-x86-64-efi-config | 0
> > automation/scripts/include/xtf-x86-64-efi | 52 +++++++++++++++++++
> > automation/scripts/qemu-smoke-x86-64-efi.sh | 43 ---------------
> > 4 files changed, 53 insertions(+), 44 deletions(-)
> > create mode 100644 automation/scripts/include/configs/xtf-x86-64-efi-config
> > create mode 100644 automation/scripts/include/xtf-x86-64-efi
> > delete mode 100755 automation/scripts/qemu-smoke-x86-64-efi.sh
> >
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index 3adc841335..ca1e4eb528 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -687,7 +687,7 @@ qemu-smoke-x86-64-clang-pvh:
> > qemu-smoke-x86-64-gcc-efi:
> > extends: .qemu-smoke-x86-64
> > script:
> > - - ./automation/scripts/qemu-smoke-x86-64-efi.sh pv 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh x86-64-efi hvm64 example 2>&1 | tee ${LOGFILE}
> > needs:
> > - debian-12-x86_64-gcc-debug
> >
> > diff --git a/automation/scripts/include/configs/xtf-x86-64-efi-config b/automation/scripts/include/configs/xtf-x86-64-efi-config
> > new file mode 100644
> > index 0000000000..e69de29bb2
> > diff --git a/automation/scripts/include/xtf-x86-64-efi b/automation/scripts/include/xtf-x86-64-efi
> > new file mode 100644
> > index 0000000000..79622d5a6c
> > --- /dev/null
> > +++ b/automation/scripts/include/xtf-x86-64-efi
> > @@ -0,0 +1,52 @@
> > +#!/bin/bash
> > +#
> > +# XTF test utilities (x86_64, EFI).
> > +#
> > +
> > +# Arch-specific environment overrides.
> > +function xtf_arch_prepare()
> > +{
> > + export FW_PREFIX="${FW_PREFIX:-/usr/share/OVMF/}"
> > + export QEMU_PREFIX="${QEMU_PREFIX:-}"
> > + export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen.efi}"
>
> Any chance we can avoid using WORKDIR? Or alternatively, we can remove
> TOP? Or remove both? :-)
Yeah, it grew a bit convoluted, but I decided to give it a try to collect
opinions.
Thanks for the feedback!
These two variables have different meanings:
- WORKDIR is a scratch location for keeping artifacts like DTB,
EFI config, imagebuilder config.
- TOP (looks like at the very least needs a better name) is the helper
variable pointint to the location of automation/scripts.
>
> Maybe it is best to keep WORKDIR and remove TOP, I am not sure. I am
> just trying to reduce the amount of variables used and that we depend
> upon.
I think it should be easy to drop TOP.
>
> I assume you tested the pipeline and it worked, right?
Yes, the pipeline is green:
https://gitlab.com/xen-project/people/dmukhin/xen/-/pipelines/1770437024
>
>
> > + export XEN_CONSOLE="${XEN_CONSOLE:-com1}"
> > + export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-master}"
> > + export XTF_SRC_URI="${XTF_SRC_URI:-https://xenbits.xen.org/git-http/xtf.git}"
> > + export XTF_SRC_VARIANTS="hvm64 pv64"
> > +}
> > +
> > +# Perform arch-specific XTF environment setup.
> > +function xtf_arch_setup()
> > +{
> > + local esp_dir="${WORKDIR}/boot-esp"
> > + local efi_dir="${esp_dir}/EFI/BOOT"
> > +
> > + # Generate EFI boot environment
> > + mkdir -p ${efi_dir}
> > + cp ${XEN_BINARY} ${efi_dir}/BOOTX64.EFI
> > + cp ${XTF_BINARY} ${efi_dir}/kernel
> > +
> > + cat > ${efi_dir}/BOOTX64.cfg <<EOF
> > +[global]
> > +default=test
> > +
> > +[test]
> > +options=${XEN_CMDLINE}
> > +kernel=kernel
> > +EOF
> > +
> > + # NB: OVMF_CODE.fd is read-only, no need to copy
> > + cp ${FW_PREFIX}OVMF_VARS.fd ${WORKDIR}
> > +
> > + export TEST_CMD="${QEMU_PREFIX}qemu-system-x86_64 \
> > + -no-reboot \
> > + -nographic \
> > + -monitor none \
> > + -serial stdio \
> > + -m 512 \
> > + -M q35,kernel-irqchip=split \
> > + -drive if=pflash,format=raw,readonly=on,file=${FW_PREFIX}OVMF_CODE.fd \
> > + -drive if=pflash,format=raw,file=${WORKDIR}/OVMF_VARS.fd \
> > + -drive file=fat:rw:${esp_dir},media=disk,index=0,format=raw \
> > + "
> > +}
> > diff --git a/automation/scripts/qemu-smoke-x86-64-efi.sh b/automation/scripts/qemu-smoke-x86-64-efi.sh
> > deleted file mode 100755
> > index 7572722be6..0000000000
> > --- a/automation/scripts/qemu-smoke-x86-64-efi.sh
> > +++ /dev/null
> > @@ -1,43 +0,0 @@
> > -#!/bin/bash
> > -
> > -set -ex -o pipefail
> > -
> > -# variant should be either pv or pvh
> > -variant=$1
> > -
> > -# Clone and build XTF
> > -git clone https://xenbits.xen.org/git-http/xtf.git
> > -cd xtf && make -j$(nproc) && cd -
> > -
> > -case $variant in
> > - pvh) k=test-hvm64-example extra="dom0-iommu=none dom0=pvh" ;;
> > - *) k=test-pv64-example extra= ;;
> > -esac
> > -
> > -mkdir -p boot-esp/EFI/BOOT
> > -cp binaries/xen.efi boot-esp/EFI/BOOT/BOOTX64.EFI
> > -cp xtf/tests/example/$k boot-esp/EFI/BOOT/kernel
> > -
> > -cat > boot-esp/EFI/BOOT/BOOTX64.cfg <<EOF
> > -[global]
> > -default=test
> > -
> > -[test]
> > -options=loglvl=all console=com1 noreboot console_timestamps=boot $extra
> > -kernel=kernel
> > -EOF
> > -
> > -cp /usr/share/OVMF/OVMF_CODE.fd OVMF_CODE.fd
> > -cp /usr/share/OVMF/OVMF_VARS.fd OVMF_VARS.fd
> > -
> > -rm -f smoke.serial
> > -export TEST_CMD="qemu-system-x86_64 -nographic -M q35,kernel-irqchip=split \
> > - -drive if=pflash,format=raw,readonly=on,file=OVMF_CODE.fd \
> > - -drive if=pflash,format=raw,file=OVMF_VARS.fd \
> > - -drive file=fat:rw:boot-esp,media=disk,index=0,format=raw \
> > - -m 512 -monitor none -serial stdio"
> > -
> > -export TEST_LOG="smoke.serial"
> > -export PASSED="Test result: SUCCESS"
> > -
> > -./automation/scripts/console.exp | sed 's/\r\+$//'
> > --
> > 2.34.1
> >
> >
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 4/4] CI: add argo x86 XTF test
2025-04-17 0:11 ` Stefano Stabellini
@ 2025-04-17 18:37 ` dmkhn
0 siblings, 0 replies; 13+ messages in thread
From: dmkhn @ 2025-04-17 18:37 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, dmukhin
On Wed, Apr 16, 2025 at 05:11:51PM -0700, Stefano Stabellini wrote:
> On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> > From: Denis Mukhin <dmukhin@ford.com>
> >
> > Introduce new CI job to run x86 XTF argo test under QEMU to smoke test argo
> > feature functionality in upstream CI.
> >
> > The new job lead time is ~30s, limit max job duration to 60s.
> >
> > Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> > ---
> > automation/gitlab-ci/test.yaml | 9 +++++++++
> > automation/scripts/include/xtf-runner | 7 +++++++
> > 2 files changed, 16 insertions(+)
> >
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index d6e4a0a622..9001efb45c 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -691,6 +691,15 @@ qemu-smoke-x86-64-gcc-efi:
> > needs:
> > - debian-12-x86_64-gcc-debug
> >
> > +qemu-xtf-argo-x86_64-gcc-debug:
> > + extends: .qemu-smoke-x86-64
> > + variables:
> > + TEST_TIMEOUT_OVERRIDE: 60
> > + script:
> > + - ./automation/scripts/qemu-xtf.sh x86-64 pv64 argo 2>&1 | tee ${LOGFILE}
> > + needs:
> > + - alpine-3.18-gcc-debug
> > +
> > qemu-smoke-riscv64-gcc:
> > extends: .qemu-riscv64
> > script:
> > diff --git a/automation/scripts/include/xtf-runner b/automation/scripts/include/xtf-runner
> > index 55b7b34b89..12470b3979 100644
> > --- a/automation/scripts/include/xtf-runner
> > +++ b/automation/scripts/include/xtf-runner
> > @@ -70,6 +70,9 @@ function xtf_build_cmdline()
> > local xtf_variant=$1
> > local xtf_name=$2
> > declare -a cmdline=()
> > + declare -A per_test_args=(
> > + [argo]="argo=1 mac-permissive=1"
> > + )
> > +
> > cmdline+=("loglvl=all noreboot console_timestamps=boot")
> > cmdline+=("console=${XEN_CONSOLE}")
> > @@ -79,6 +82,10 @@ function xtf_build_cmdline()
> > cmdline+=("dom0-iommu=none dom0=pvh")
> > fi
> >
> > + if [[ -v per_test_args[${xtf_name}] ]]; then
> > + cmdline+=("${per_test_args[${xtf_name}]}")
> > + fi
> > +
>
> This is all within the same bash function so I think we could avoid
> using the per_test_args array and just do:
>
> if test "$xtf_name"
> then
> cmdline+=("argo=1 mac-permissive=1")
> fi
>
> The indirection is not required as far as I can tell. I do realize it is
> a matter of taste and it works either way so I wouldn't insist.
Some XTFs may need special configuration.
I added this to be a placeholder for test-specific Xen run-time configuration.
>
>
>
> > export XEN_CMDLINE="${cmdline[@]}"
> > }
> >
> > --
> > 2.34.1
> >
> >
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 1/4] CI: unify x86 XTF test runner
2025-04-17 0:16 ` Stefano Stabellini
@ 2025-04-17 18:50 ` dmkhn
0 siblings, 0 replies; 13+ messages in thread
From: dmkhn @ 2025-04-17 18:50 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, dmukhin
On Wed, Apr 16, 2025 at 05:16:49PM -0700, Stefano Stabellini wrote:
> On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> > From: Denis Mukhin <dmukhin@ford.com>
> >
> > Add test runner script qemu-xtf.sh which is allows any XTF x86 test to be
> > easily executed. Test runner is invoked from the qemu-smoke* jobs with the
> > hardcoded parameters.
> >
> > Each x86 XTF job lead time is reduced a bit since only the test-related code
> > is built, not the entire XTF project.
> >
> > Add .gitignore to avoid committing test artifacts by mistake.
> >
> > Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> > ---
> > automation/gitlab-ci/test.yaml | 8 +-
> > automation/scripts/.gitignore | 6 +
> > .../scripts/include/configs/xtf-x86-64-config | 0
> > automation/scripts/include/xtf-runner | 134 ++++++++++++++++++
> > automation/scripts/include/xtf-x86-64 | 31 ++++
> > automation/scripts/qemu-smoke-x86-64.sh | 26 ----
> > automation/scripts/qemu-xtf.sh | 26 ++++
> > 7 files changed, 201 insertions(+), 30 deletions(-)
> > create mode 100644 automation/scripts/.gitignore
> > create mode 100644 automation/scripts/include/configs/xtf-x86-64-config
> > create mode 100644 automation/scripts/include/xtf-runner
> > create mode 100644 automation/scripts/include/xtf-x86-64
> > delete mode 100755 automation/scripts/qemu-smoke-x86-64.sh
> > create mode 100755 automation/scripts/qemu-xtf.sh
> >
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index 5ce445b78f..3adc841335 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -659,28 +659,28 @@ qemu-alpine-x86_64-gcc:
> > qemu-smoke-x86-64-gcc:
> > extends: .qemu-smoke-x86-64
> > script:
> > - - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh x86-64 pv64 example 2>&1 | tee ${LOGFILE}
> > needs:
> > - debian-12-x86_64-gcc-debug
> >
> > qemu-smoke-x86-64-clang:
> > extends: .qemu-smoke-x86-64
> > script:
> > - - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh x86-64 pv64 example 2>&1 | tee ${LOGFILE}
> > needs:
> > - debian-12-x86_64-clang-debug
> >
> > qemu-smoke-x86-64-gcc-pvh:
> > extends: .qemu-smoke-x86-64
> > script:
> > - - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh x86-64 hvm64 example 2>&1 | tee ${LOGFILE}
> > needs:
> > - debian-12-x86_64-gcc-debug
> >
> > qemu-smoke-x86-64-clang-pvh:
> > extends: .qemu-smoke-x86-64
> > script:
> > - - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh x86-64 hvm64 example 2>&1 | tee ${LOGFILE}
> > needs:
> > - debian-12-x86_64-clang-debug
> >
> > diff --git a/automation/scripts/.gitignore b/automation/scripts/.gitignore
> > new file mode 100644
> > index 0000000000..2f2d6e1ebd
> > --- /dev/null
> > +++ b/automation/scripts/.gitignore
> > @@ -0,0 +1,6 @@
> > +!include
> > +
> > +binaries
> > +smoke.serial
> > +xen
> > +xtf*/
>
> I am not sure this works as intended: I thought the paths are relative
> to the location of the .gitignore file, but I could be wrong.
git allows .gitignore at any level in the source tree so that lenghty
paths could be avoided.
>
>
> > diff --git a/automation/scripts/include/configs/xtf-x86-64-config b/automation/scripts/include/configs/xtf-x86-64-config
> > new file mode 100644
> > index 0000000000..e69de29bb2
> > diff --git a/automation/scripts/include/xtf-runner b/automation/scripts/include/xtf-runner
> > new file mode 100644
> > index 0000000000..55b7b34b89
> > --- /dev/null
> > +++ b/automation/scripts/include/xtf-runner
> > @@ -0,0 +1,134 @@
> > +#!/bin/bash
> > +#
> > +# XTF test utilities.
> > +#
> > +# Environment variables:
> > +# BOOT_MSG: Expected boot message
> > +# FW_PREFIX: Firmware images path including '/' at the end
> > +# PASSED: XTF test printout in case of a pass
> > +# QEMU_PREFIX: QEMU path including '/' at the end
> > +# TEST_LOG: Output log file
> > +# UBOOT_CMD: U-Boot command line
> > +# WORKDIR: Test working directory
> > +# XEN_BINARY: Xen binary location
> > +# XEN_CONSOLE: Xen console device name
> > +# XTF_SRC_CONFIG: XTF config file
> > +# XTF_SRC_BRANCH: XTF branch
> > +# XTF_SRC_URI: XTF source code URI
> > +
> > +function die()
> > +{
> > + set +x
> > + echo "FATAL: $*" >&2
> > + exit 1
> > +}
> > +
> > +[ -z "$TOP" ] && die "\$TOP is not set"
> > +
> > +# Output log file
> > +TEST_LOG="${TEST_LOG:-smoke.serial}"
> > +# XTF test printout in case of a pass
> > +PASSED="${PASSED:-Test result: SUCCESS}"
> > +# Expected boot message
> > +BOOT_MSG="${BOOT_MSG:-Latest ChangeSet: }"
> > +# Test working directory
> > +WORKDIR="${WORKDIR:-binaries}"
> > +# XTF source code
> > +XTF_SRC_CONFIG="${XTF_CONFIG:-${TOP}/include/configs/xtf-${ARCH}-config}"
>
> Should this be XTF_SRC_CONFIG="${XTF_SRC_CONFIG:- ?
Thanks for the catch!
>
>
> > +# Build an XTF test binary.
> > +# $1 Test variant.
> > +# $2 Test name.
> > +function xtf_build_binary()
> > +{
> > + local xtf_variant=$1
> > + local xtf_name=$2
> > + local xtf_dir="xtf-${ARCH}"
> > +
> > + # Crude check for local testing
> > + if [ ! -d ${xtf_dir}/.git ]; then
> > + git clone ${XTF_SRC_URI} ${xtf_dir} -b ${XTF_SRC_BRANCH}
> > + fi
> > +
> > + make \
> > + -C ${xtf_dir} \
> > + -j$(nproc) \
> > + $(tr '\n' ' ' < ${XTF_SRC_CONFIG}) \
> > + TESTS=tests/${xtf_name}
> > +
> > + export XTF_NAME="${xtf_name}"
> > + export XTF_VARIANT="${xtf_variant}"
> > + export XTF_WORKDIR="$(readlink -f ${xtf_dir})"
> > + export XTF_BINARY="${XTF_WORKDIR}/tests/${xtf_name}/test-${xtf_variant}-${xtf_name}"
> > +}
> > +
> > +# Build Xen command line for running an XTF test.
> > +# $1 Test variant.
> > +# $2 Test name.
> > +function xtf_build_cmdline()
> > +{
> > + local xtf_variant=$1
> > + local xtf_name=$2
> > + declare -a cmdline=()
> > +
> > + cmdline+=("loglvl=all noreboot console_timestamps=boot")
> > + cmdline+=("console=${XEN_CONSOLE}")
> > +
> > + # NB: OK to have hvm64, which is x86-only variant
> > + if [[ $xtf_variant == "hvm64" ]]; then
> > + cmdline+=("dom0-iommu=none dom0=pvh")
> > + fi
> > +
> > + export XEN_CMDLINE="${cmdline[@]}"
> > +}
> > +
> > +# Build an XTF test environment.
> > +# $1 Test variant.
> > +# $2 Test name.
> > +function xtf_build_test()
> > +{
> > + local v=$1
> > + local xtf_name=$2
> > + local xtf_variant=""
> > +
> > + for x in ${XTF_SRC_VARIANTS}; do
> > + if [[ "${x}" == "${v}" ]]; then
> > + xtf_variant=${v}
> > + break
> > + fi
> > + done
> > + if [[ -z $xtf_variant ]]; then
> > + die "unsupported test variant '$1', supported variants: ${XTF_SRC_VARIANTS}"
> > + fi
> > +
> > + xtf_build_binary ${xtf_variant} ${xtf_name}
> > + xtf_build_cmdline ${xtf_variant} ${xtf_name}
> > +}
> > +
> > +# Execute an XTF test.
> > +function xtf_run_test()
> > +{
> > + rm -f ${TEST_LOG}
> > + export BOOT_MSG PASSED TEST_CMD TEST_LOG UBOOT_CMD
> > + ${TOP}/console.exp | sed 's/\r\+$//'
> > +}
> > +
> > +# Setup environment and run an XTF test.
> > +# $1 Test variant.
> > +# $2 Test name.
> > +function xtf_test()
> > +{
> > + # Out: FW_*, QEMU_*, XEN_{BINARY,CONSOLE}, XTF_SRC_*
> > + xtf_arch_prepare
> > +
> > + # In: XTF_SRC_*
> > + # OUt: XTF_{BINARY,NAME,VARIANT,WORKDIR} and XEN_CMDLINE
> > + xtf_build_test $@
> > +
> > + # In: FW_*, QEMU_*, XTF_*, XEN_*
> > + # Out: BOOT_MSG, PASSED, TEST_{CMD,LOG}, UBOOT_CMD
> > + xtf_arch_setup
> > +
> > + # In: BOOT_MSG, PASSED, TEST_{CMD,LOG}, UBOOT_CMD
> > + xtf_run_test
> > +}
> > diff --git a/automation/scripts/include/xtf-x86-64 b/automation/scripts/include/xtf-x86-64
> > new file mode 100644
> > index 0000000000..edddf18b38
> > --- /dev/null
> > +++ b/automation/scripts/include/xtf-x86-64
> > @@ -0,0 +1,31 @@
> > +#!/bin/bash
> > +#
> > +# XTF test utilities (x86_64).
> > +#
> > +
> > +# Arch-specific environment overrides.
> > +function xtf_arch_prepare()
> > +{
> > + export FW_PREFIX="${FW_PREFIX:-}"
> > + export QEMU_PREFIX="${QEMU_PREFIX:-}"
>
> It looks like QEMU_PREFIX is only used within this file (and other
> similar files in later patches). If so, maybe for the sake of reducing
> global variables and variables in general we could get rid of
> QEMU_PREFIX.
Turned out that Arm and x86 CI jobs use different paths to QEMU binaries.
dev host may have some other location, that's why I parameterized QEMU
location.
>
>
> > + export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen}"
> > + export XEN_CONSOLE="${XEN_CONSOLE:-com1}"
>
> Instead of XEN_CONSOLE, I'd suggest to have an arch-specific variable
> for Xen command line options where we can put the console as well as
> other things. However, it is also totally fine as is.
I initially did that, but then there's a big portion of command line parameters
which is shared across all supported architectures, with console= pointing to
arch-specific console device.
I will add arch-specific variable and drop XEN_CONSOLE.
>
> In fact, I think you went above and beyond in terms of generic
> abstractions, and I think it would have been OK to go with fewer
> variables, and more ad-hoc custom code.
>
>
> > + export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-master}"
> > + export XTF_SRC_URI="${XTF_SRC_URI:-https://xenbits.xen.org/git-http/xtf.git}"
> > + export XTF_SRC_VARIANTS="hvm64 pv64"
> > +}
> > +
> > +# Perform arch-specific XTF environment setup.
> > +function xtf_arch_setup()
> > +{
> > + export TEST_CMD="${QEMU_PREFIX}qemu-system-x86_64 \
> > + -no-reboot \
> > + -nographic \
> > + -monitor none \
> > + -serial stdio \
> > + -m 512 \
> > + -kernel ${XEN_BINARY} \
> > + -initrd ${XTF_BINARY} \
> > + -append \"${XEN_CMDLINE}\" \
> > + "
> > +}
> > diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
> > deleted file mode 100755
> > index da0c26cc2f..0000000000
> > --- a/automation/scripts/qemu-smoke-x86-64.sh
> > +++ /dev/null
> > @@ -1,26 +0,0 @@
> > -#!/bin/bash
> > -
> > -set -ex -o pipefail
> > -
> > -# variant should be either pv or pvh
> > -variant=$1
> > -
> > -# Clone and build XTF
> > -git clone https://xenbits.xen.org/git-http/xtf.git
> > -cd xtf && make -j$(nproc) && cd -
> > -
> > -case $variant in
> > - pvh) k=test-hvm64-example extra="dom0-iommu=none dom0=pvh" ;;
> > - *) k=test-pv64-example extra= ;;
> > -esac
> > -
> > -rm -f smoke.serial
> > -export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
> > - -initrd xtf/tests/example/$k \
> > - -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
> > - -m 512 -monitor none -serial stdio"
> > -
> > -export TEST_LOG="smoke.serial"
> > -export PASSED="Test result: SUCCESS"
> > -
> > -./automation/scripts/console.exp | sed 's/\r\+$//'
> > diff --git a/automation/scripts/qemu-xtf.sh b/automation/scripts/qemu-xtf.sh
> > new file mode 100755
> > index 0000000000..2e16d4aece
> > --- /dev/null
> > +++ b/automation/scripts/qemu-xtf.sh
> > @@ -0,0 +1,26 @@
> > +#!/bin/bash
> > +#
> > +# XTF test runner (QEMU).
> > +#
> > +
> > +set -e -o pipefail
> > +
> > +if [ $# -lt 3 ]; then
> > + echo "Usage: $(basename $0) ARCH XTF-VARIANT XTF-NAME"
> > + exit 1
> > +fi
> > +
> > +export ARCH="$1"
> > +shift
> > +
> > +export TOP="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
> > +if [ ! -f "${TOP}/include/xtf-${ARCH}" ]; then
> > + echo "unsupported architecture '${ARCH}'" >&2
> > + exit 1
> > +fi
>
> This seems a bit complex.. maybe we can assume that we are in xen.git/ ?
Will do that.
>
>
> > +set -x
> > +source ${TOP}/include/xtf-runner
> > +source ${TOP}/include/xtf-${ARCH}
>
> So here we could do:
>
> source automation/scripts/include/xtf-runner
>
> ? Or is it important to be able to detect and calculate the directory?
> If so, why not use TOP=$(pwd) or TOP=${PWD} or TOP=$(dirname "$0")?
> By the way, if we need a variable for the top directory I would call it
> XEN_ROOT instead of TOP.
>
> I see the patches are using both TOP and WORKDIR, maybe we can get rid
> of one of them? This is only a matter of taste but I think it would be
> simpler with fewer variables and most of the others look unavoidable.
I will try to eliminate TOP uses.
>
> Everything else looks OK to me.
>
>
> > +xtf_test $@
> > --
> > 2.34.1
> >
> >
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v1 3/4] CI: switch arm64 XTF test runner to qemu-xtf.sh
2025-04-17 0:12 ` Stefano Stabellini
@ 2025-04-17 18:51 ` dmkhn
0 siblings, 0 replies; 13+ messages in thread
From: dmkhn @ 2025-04-17 18:51 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, andrew.cooper3, cardoe, marmarek, dmukhin
On Wed, Apr 16, 2025 at 05:12:55PM -0700, Stefano Stabellini wrote:
> On Wed, 16 Apr 2025, dmkhn@proton.me wrote:
> > From: Denis Mukhin <dmukhin@ford.com>
> >
> > Hook arm64 QEMU configuration to qemu-xtf.sh and use new script in arm64 CI
> > jobs.
> >
> > Signed-off-by: Denis Mukhin <dmukhin@ford.com>
>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Thanks!
>
>
> > ---
> > automation/gitlab-ci/test.yaml | 4 +-
> > automation/scripts/.gitignore | 3 +
> > .../scripts/include/configs/xtf-arm64-config | 2 +
> > automation/scripts/include/xtf-arm64 | 81 +++++++++++++++++++
> > automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 ----------------
> > 5 files changed, 88 insertions(+), 70 deletions(-)
> > create mode 100644 automation/scripts/include/configs/xtf-arm64-config
> > create mode 100644 automation/scripts/include/xtf-arm64
> > delete mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh
> >
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index ca1e4eb528..d6e4a0a622 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -571,7 +571,7 @@ qemu-smoke-dom0less-arm64-gcc-debug-earlyprintk:
> > qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
> > extends: .qemu-arm64
> > script:
> > - - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh arm64 mmu64le hyp-xen-version 2>&1 | tee ${LOGFILE}
> > needs:
> > - alpine-3.18-gcc-arm64
> > - qemu-system-aarch64-6.0.0-arm64-export
> > @@ -579,7 +579,7 @@ qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
> > qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
> > extends: .qemu-arm64
> > script:
> > - - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
> > + - ./automation/scripts/qemu-xtf.sh arm64 mmu64le hyp-xen-version 2>&1 | tee ${LOGFILE}
> > needs:
> > - alpine-3.18-gcc-debug-arm64
> > - qemu-system-aarch64-6.0.0-arm64-export
> > diff --git a/automation/scripts/.gitignore b/automation/scripts/.gitignore
> > index 2f2d6e1ebd..f853da4d89 100644
> > --- a/automation/scripts/.gitignore
> > +++ b/automation/scripts/.gitignore
> > @@ -1,6 +1,9 @@
> > !include
> >
> > binaries
> > +imagebuilder
> > smoke.serial
> > xen
> > xtf*/
> > +
> > +*.rom
> > diff --git a/automation/scripts/include/configs/xtf-arm64-config b/automation/scripts/include/configs/xtf-arm64-config
> > new file mode 100644
> > index 0000000000..9942740927
> > --- /dev/null
> > +++ b/automation/scripts/include/configs/xtf-arm64-config
> > @@ -0,0 +1,2 @@
> > +CONFIG_GICV2=y
> > +CONFIG_SBSA_UART=y
> > diff --git a/automation/scripts/include/xtf-arm64 b/automation/scripts/include/xtf-arm64
> > new file mode 100644
> > index 0000000000..68487fe617
> > --- /dev/null
> > +++ b/automation/scripts/include/xtf-arm64
> > @@ -0,0 +1,81 @@
> > +#!/bin/bash
> > +#
> > +# XTF test utilities (arm64).
> > +#
> > +
> > +# Arch-specific environment overrides.
> > +function xtf_arch_prepare()
> > +{
> > + export FW_PREFIX="${FW_PREFIX:-/usr/lib/u-boot/qemu_arm64/}"
> > + export QEMU_PREFIX="${QEMU_PREFIX:-${WORKDIR}/}"
> > + export XEN_BINARY="${XEN_BINARY:-${WORKDIR}/xen}"
> > + export XEN_CONSOLE="${XEN_CONSOLE:-dtuart}"
> > + export XTF_SRC_BRANCH="${XTF_SRC_BRANCH:-xtf-arm}"
> > + export XTF_SRC_URI="${XTF_SRC_URI:-https://gitlab.com/xen-project/fusa/xtf.git}"
> > + export XTF_SRC_VARIANTS="mmu64le"
> > +}
> > +
> > +# Perform arch-specific XTF environment setup.
> > +function xtf_arch_setup()
> > +{
> > + # QEMU looks for "efi-virtio.rom" even if it is unneeded
> > + curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> > +
> > + # Crude check for local testing
> > + if [ ! -d imagebuilder/.git ]; then
> > + git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
> > + fi
> > +
> > + cat > ${WORKDIR}/config <<EOF
> > +MEMORY_START="0x40000000"
> > +MEMORY_END="0xC0000000"
> > +
> > +XEN="xen"
> > +DEVICE_TREE="virt-gicv2.dtb"
> > +
> > +XEN_CMD="${XEN_CMDLINE}"
> > +
> > +DOMU_KERNEL[0]="xtf-test"
> > +DOMU_MEM[0]="128"
> > +
> > +NUM_DOMUS=1
> > +
> > +LOAD_CMD="tftpb"
> > +UBOOT_SOURCE="boot.source"
> > +UBOOT_SCRIPT="boot.scr"
> > +EOF
> > + cp ${XTF_BINARY} ${WORKDIR}/xtf-test
> > +
> > + # Generate virt-gicv2.dtb
> > + ${WORKDIR}/qemu-system-aarch64 \
> > + -machine virtualization=true \
> > + -cpu cortex-a57 \
> > + -machine type=virt \
> > + -m 2048 \
> > + -smp 2 \
> > + -display none \
> > + -machine dumpdtb=${WORKDIR}/virt-gicv2.dtb
> > +
> > + # Generate U-Boot environment
> > + bash -x imagebuilder/scripts/uboot-script-gen \
> > + -t tftp \
> > + -d ${WORKDIR}/ \
> > + -c ${WORKDIR}/config
> > +
> > + export TEST_CMD="${QEMU_PREFIX}qemu-system-aarch64 \
> > + -machine virtualization=true \
> > + -cpu cortex-a57 \
> > + -machine type=virt \
> > + -no-reboot \
> > + -nographic \
> > + -monitor none \
> > + -serial stdio \
> > + -m 2048 \
> > + -smp 2 \
> > + -device virtio-net-pci,netdev=n0 \
> > + -netdev user,id=n0,tftp=${WORKDIR} \
> > + -bios ${FW_PREFIX}u-boot.bin \
> > + "
> > +
> > + export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> > +}
> > diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> > deleted file mode 100755
> > index 436f460c3c..0000000000
> > --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
> > +++ /dev/null
> > @@ -1,68 +0,0 @@
> > -#!/bin/bash
> > -
> > -set -ex -o pipefail
> > -
> > -# Name of the XTF test
> > -xtf_test=$1
> > -
> > -# Message returned by XTF in case of success
> > -passed="Test result: SUCCESS"
> > -
> > -# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
> > -curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> > -./binaries/qemu-system-aarch64 \
> > - -machine virtualization=true \
> > - -cpu cortex-a57 -machine type=virt \
> > - -m 2048 -smp 2 -display none \
> > - -machine dumpdtb=binaries/virt-gicv2.dtb
> > -
> > -# XTF
> > -# Build a single XTF test passed as a first parameter to the script.
> > -# Build XTF with GICv2 support to match Qemu configuration and with SBSA UART
> > -# support, so that the test will use an emulated UART for printing messages.
> > -# This will allow us to run the test on both debug and non-debug Xen builds.
> > -rm -rf xtf
> > -git clone https://gitlab.com/xen-project/fusa/xtf.git -b xtf-arm
> > -make -C xtf TESTS=tests/${xtf_test} CONFIG_SBSA_UART=y CONFIG_GICV2=y -j$(nproc)
> > -cp xtf/tests/${xtf_test}/test-mmu64le-${xtf_test} binaries/xtf-test
> > -
> > -# ImageBuilder
> > -echo 'MEMORY_START="0x40000000"
> > -MEMORY_END="0xC0000000"
> > -
> > -XEN="xen"
> > -DEVICE_TREE="virt-gicv2.dtb"
> > -
> > -XEN_CMD="console=dtuart console_timestamps=boot"
> > -
> > -DOMU_KERNEL[0]="xtf-test"
> > -DOMU_MEM[0]="128"
> > -
> > -NUM_DOMUS=1
> > -
> > -LOAD_CMD="tftpb"
> > -UBOOT_SOURCE="boot.source"
> > -UBOOT_SCRIPT="boot.scr"' > binaries/config
> > -
> > -rm -rf imagebuilder
> > -git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
> > -bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
> > -
> > -# Run the test
> > -rm -f smoke.serial
> > -export TEST_CMD="./binaries/qemu-system-aarch64 \
> > - -machine virtualization=true \
> > - -cpu cortex-a57 -machine type=virt \
> > - -m 2048 -monitor none -serial stdio \
> > - -smp 2 \
> > - -no-reboot \
> > - -device virtio-net-pci,netdev=n0 \
> > - -netdev user,id=n0,tftp=binaries \
> > - -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
> > -
> > -export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> > -export BOOT_MSG="Latest ChangeSet: "
> > -export TEST_LOG="smoke.serial"
> > -export PASSED="${passed}"
> > -
> > -./automation/scripts/console.exp | sed 's/\r\+$//'
> > --
> > 2.34.1
> >
> >
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2025-04-17 18:51 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-04-16 5:32 [PATCH v1 0/4] CI: updates to XTF CI runners dmkhn
2025-04-16 5:32 ` [PATCH v1 1/4] CI: unify x86 XTF test runner dmkhn
2025-04-17 0:16 ` Stefano Stabellini
2025-04-17 18:50 ` dmkhn
2025-04-16 5:32 ` [PATCH v1 2/4] CI: switch x86 EFI smoke test runner to qemu-xtf.sh dmkhn
2025-04-17 0:14 ` Stefano Stabellini
2025-04-17 18:34 ` dmkhn
2025-04-16 5:32 ` [PATCH v1 3/4] CI: switch arm64 XTF " dmkhn
2025-04-17 0:12 ` Stefano Stabellini
2025-04-17 18:51 ` dmkhn
2025-04-16 5:32 ` [PATCH v1 4/4] CI: add argo x86 XTF test dmkhn
2025-04-17 0:11 ` Stefano Stabellini
2025-04-17 18:37 ` dmkhn
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.