* [XEN PATCH v2 0/2] automation: test script fixes (pipefail)
@ 2024-08-29 22:34 victorm.lira
2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira
2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira
0 siblings, 2 replies; 6+ messages in thread
From: victorm.lira @ 2024-08-29 22:34 UTC (permalink / raw)
To: xen-devel; +Cc: Victor Lira
From: Victor Lira <victorm.lira@amd.com>
Separated into two patches to improve clarity and updated to use the
"pipefail" function instead of "mkfifo".
Victor Lira (2):
automation: fix qemu test false success
automation: use expect utility in xilinx tests
.../build/ubuntu/xenial-xilinx.dockerfile | 1 +
automation/gitlab-ci/test.yaml | 2 ++
.../scripts/{qemu-key.exp => console.exp} | 10 ++++-----
automation/scripts/qemu-alpine-x86_64.sh | 9 ++++----
automation/scripts/qemu-smoke-dom0-arm32.sh | 9 ++++----
automation/scripts/qemu-smoke-dom0-arm64.sh | 9 ++++----
.../scripts/qemu-smoke-dom0less-arm32.sh | 9 ++++----
.../scripts/qemu-smoke-dom0less-arm64.sh | 9 ++++----
automation/scripts/qemu-smoke-ppc64le.sh | 9 ++++----
automation/scripts/qemu-smoke-riscv64.sh | 9 ++++----
automation/scripts/qemu-smoke-x86-64.sh | 9 ++++----
automation/scripts/qemu-xtf-dom0less-arm64.sh | 9 ++++----
.../scripts/xilinx-smoke-dom0-x86_64.sh | 22 +++++++++----------
.../scripts/xilinx-smoke-dom0less-arm64.sh | 19 ++++++++--------
14 files changed, 64 insertions(+), 71 deletions(-)
rename automation/scripts/{qemu-key.exp => console.exp} (78%)
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/2] automation: fix false success in qemu tests
2024-08-29 22:34 [XEN PATCH v2 0/2] automation: test script fixes (pipefail) victorm.lira
@ 2024-08-29 22:34 ` victorm.lira
2024-08-30 0:48 ` Stefano Stabellini
2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira
1 sibling, 1 reply; 6+ messages in thread
From: victorm.lira @ 2024-08-29 22:34 UTC (permalink / raw)
To: xen-devel
Cc: Victor Lira, Andrew Cooper, Jan Beulich, Julien Grall,
Stefano Stabellini, Michal Orzel
From: Victor Lira <victorm.lira@amd.com>
Fix flaw in qemu-*.sh tests that producess a false success. The following
lines produces success despite the "expect" script producing nonzero exit
status:
set +e
...
./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
(end of file)
The default exit status for a pipeline using "|" operator is that of the
rightmost command. Fix this by setting the "pipefail" option in the shell,
and removing "set +e" allowing the expect script to determine the result.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
---
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Michal Orzel <michal.orzel@amd.com>
Cc: xen-devel@lists.xenproject.org
---
automation/scripts/qemu-alpine-x86_64.sh | 3 +--
automation/scripts/qemu-key.exp | 2 +-
automation/scripts/qemu-smoke-dom0-arm32.sh | 3 +--
automation/scripts/qemu-smoke-dom0-arm64.sh | 3 +--
automation/scripts/qemu-smoke-dom0less-arm32.sh | 3 +--
automation/scripts/qemu-smoke-dom0less-arm64.sh | 3 +--
automation/scripts/qemu-smoke-ppc64le.sh | 3 +--
automation/scripts/qemu-smoke-riscv64.sh | 3 +--
automation/scripts/qemu-smoke-x86-64.sh | 3 +--
automation/scripts/qemu-xtf-dom0less-arm64.sh | 3 +--
10 files changed, 10 insertions(+), 19 deletions(-)
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 42a89e86b021..93914c41bc24 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
# DomU Busybox
cd binaries
@@ -76,7 +76,6 @@ EOF
# Run the test
rm -f smoke.serial
-set +e
export QEMU_CMD="qemu-system-x86_64 \
-cpu qemu64,+svm \
-m 2G -smp 2 \
diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/qemu-key.exp
index 787f1f08cb96..66c416483146 100755
--- a/automation/scripts/qemu-key.exp
+++ b/automation/scripts/qemu-key.exp
@@ -14,7 +14,7 @@ eval spawn $env(QEMU_CMD)
expect_after {
-re "(.*)\r" {
- exp_continue
+ exp_continue -continue_timer
}
timeout {send_error "ERROR-Timeout!\n"; exit 1}
eof {send_error "ERROR-EOF!\n"; exit 1}
diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
index fd64b19358ae..7c282eff3a58 100755
--- a/automation/scripts/qemu-smoke-dom0-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
serial_log="$(pwd)/smoke.serial"
@@ -77,7 +77,6 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
rm -f ${serial_log}
-set +e
export QEMU_CMD="./qemu-system-arm \
-machine virt \
-machine virtualization=true \
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index e0cea742b0a0..81f210f7f50e 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
# DomU Busybox
cd binaries
@@ -93,7 +93,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
# Run the test
rm -f smoke.serial
-set +e
export QEMU_CMD="./binaries/qemu-system-aarch64 \
-machine virtualization=true \
-cpu cortex-a57 -machine type=virt \
diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh
index e824cb7c2a3d..38e8a0b0bd7d 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
test_variant=$1
@@ -130,7 +130,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
# Run the test
rm -f ${serial_log}
-set +e
export QEMU_CMD="./qemu-system-arm \
-machine virt \
-machine virtualization=true \
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index f42ba5d196bc..ea67650e17da 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
test_variant=$1
@@ -204,7 +204,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
# Run the test
rm -f smoke.serial
-set +e
export QEMU_CMD="./binaries/qemu-system-aarch64 \
-machine virtualization=true \
-cpu cortex-a57 -machine type=virt,gic-version=$gic_version \
diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh
index 594f92c19cc2..49e189c37058 100755
--- a/automation/scripts/qemu-smoke-ppc64le.sh
+++ b/automation/scripts/qemu-smoke-ppc64le.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
serial_log="$(pwd)/smoke.serial"
@@ -9,7 +9,6 @@ machine=$1
# Run the test
rm -f ${serial_log}
-set +e
export QEMU_CMD="qemu-system-ppc64 \
-bios skiboot.lid \
diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh
index c2595f657ff3..422ee03e0d26 100755
--- a/automation/scripts/qemu-smoke-riscv64.sh
+++ b/automation/scripts/qemu-smoke-riscv64.sh
@@ -1,10 +1,9 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
# Run the test
rm -f smoke.serial
-set +e
export QEMU_CMD="qemu-system-riscv64 \
-M virt \
diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
index 3440b1761db4..7495185d9fc6 100755
--- a/automation/scripts/qemu-smoke-x86-64.sh
+++ b/automation/scripts/qemu-smoke-x86-64.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
# variant should be either pv or pvh
variant=$1
@@ -15,7 +15,6 @@ case $variant in
esac
rm -f smoke.serial
-set +e
export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
-initrd xtf/tests/example/$k \
-append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
index 4042fe50602b..acef1637e25b 100755
--- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
+++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
# Name of the XTF test
xtf_test=$1
@@ -50,7 +50,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
# Run the test
rm -f smoke.serial
-set +e
export QEMU_CMD="./binaries/qemu-system-aarch64 \
-machine virtualization=true \
-cpu cortex-a57 -machine type=virt \
--
2.25.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v2 2/2] automation: use expect utility in xilinx tests
2024-08-29 22:34 [XEN PATCH v2 0/2] automation: test script fixes (pipefail) victorm.lira
2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira
@ 2024-08-29 22:34 ` victorm.lira
2024-08-30 0:48 ` Stefano Stabellini
2024-08-30 12:35 ` Andrew Cooper
1 sibling, 2 replies; 6+ messages in thread
From: victorm.lira @ 2024-08-29 22:34 UTC (permalink / raw)
To: xen-devel
Cc: Victor Lira, Andrew Cooper, Jan Beulich, Julien Grall,
Stefano Stabellini, Michal Orzel
From: Victor Lira <victorm.lira@amd.com>
Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty))
This patch introduced a CI failure due to a timeout in xilinx-x86_64 test.
Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility
to determine test result and allow early exit from tests.
Add "expect" to xilinx container environment (dockerfile).
Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be
used by both QEMU and hardware tests.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
---
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Michal Orzel <michal.orzel@amd.com>
Cc: xen-devel@lists.xenproject.org
---
.../build/ubuntu/xenial-xilinx.dockerfile | 1 +
automation/gitlab-ci/test.yaml | 2 ++
.../scripts/{qemu-key.exp => console.exp} | 8 +++----
automation/scripts/qemu-alpine-x86_64.sh | 6 ++---
automation/scripts/qemu-smoke-dom0-arm32.sh | 6 ++---
automation/scripts/qemu-smoke-dom0-arm64.sh | 6 ++---
.../scripts/qemu-smoke-dom0less-arm32.sh | 6 ++---
.../scripts/qemu-smoke-dom0less-arm64.sh | 6 ++---
automation/scripts/qemu-smoke-ppc64le.sh | 6 ++---
automation/scripts/qemu-smoke-riscv64.sh | 6 ++---
automation/scripts/qemu-smoke-x86-64.sh | 6 ++---
automation/scripts/qemu-xtf-dom0less-arm64.sh | 6 ++---
.../scripts/xilinx-smoke-dom0-x86_64.sh | 22 +++++++++----------
.../scripts/xilinx-smoke-dom0less-arm64.sh | 19 ++++++++--------
14 files changed, 54 insertions(+), 52 deletions(-)
rename automation/scripts/{qemu-key.exp => console.exp} (83%)
diff --git a/automation/build/ubuntu/xenial-xilinx.dockerfile b/automation/build/ubuntu/xenial-xilinx.dockerfile
index f03d62e8bd3f..6107d8b7711f 100644
--- a/automation/build/ubuntu/xenial-xilinx.dockerfile
+++ b/automation/build/ubuntu/xenial-xilinx.dockerfile
@@ -20,6 +20,7 @@ RUN apt-get update && \
git \
gzip \
file \
+ expect \
&& \
apt-get autoremove -y && \
apt-get clean && \
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 3b339f387fbe..e3ebe3745994 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -84,6 +84,7 @@
variables:
CONTAINER: ubuntu:xenial-xilinx
LOGFILE: qemu-smoke-xilinx.log
+ TEST_TIMEOUT: 120
artifacts:
paths:
- smoke.serial
@@ -103,6 +104,7 @@
LOGFILE: xilinx-smoke-x86_64.log
XEN_CMD_CONSOLE: "console=com2 com2=115200,8n1,0x2F8,4"
TEST_BOARD: "crater"
+ TEST_TIMEOUT: 500
artifacts:
paths:
- smoke.serial
diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/console.exp
similarity index 83%
rename from automation/scripts/qemu-key.exp
rename to automation/scripts/console.exp
index 66c416483146..f538aa6bd06c 100755
--- a/automation/scripts/qemu-key.exp
+++ b/automation/scripts/console.exp
@@ -1,16 +1,16 @@
#!/usr/bin/expect -f
-if {[info exists env(QEMU_TIMEOUT)]} {
- set timeout $env(QEMU_TIMEOUT)
+if {[info exists env(TEST_TIMEOUT)]} {
+ set timeout $env(TEST_TIMEOUT)
} else {
set timeout 1500
}
-log_file -a $env(QEMU_LOG)
+log_file -a $env(TEST_LOG)
match_max 10000
-eval spawn $env(QEMU_CMD)
+eval spawn $env(TEST_CMD)
expect_after {
-re "(.*)\r" {
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
index 93914c41bc24..1ff689b577e3 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -76,7 +76,7 @@ EOF
# Run the test
rm -f smoke.serial
-export QEMU_CMD="qemu-system-x86_64 \
+export TEST_CMD="qemu-system-x86_64 \
-cpu qemu64,+svm \
-m 2G -smp 2 \
-monitor none -serial stdio \
@@ -84,8 +84,8 @@ export QEMU_CMD="qemu-system-x86_64 \
-device virtio-net-pci,netdev=n0 \
-netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
export LOG_MSG="Domain-0"
export PASSED="BusyBox"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
index 7c282eff3a58..e1cd83880928 100755
--- a/automation/scripts/qemu-smoke-dom0-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
@@ -77,7 +77,7 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
rm -f ${serial_log}
-export QEMU_CMD="./qemu-system-arm \
+export TEST_CMD="./qemu-system-arm \
-machine virt \
-machine virtualization=true \
-smp 4 \
@@ -91,8 +91,8 @@ export QEMU_CMD="./qemu-system-arm \
-bios /usr/lib/u-boot/qemu_arm/u-boot.bin"
export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
-export QEMU_LOG="${serial_log}"
+export TEST_LOG="${serial_log}"
export LOG_MSG="Domain-0"
export PASSED="/ #"
-../automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+../automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 81f210f7f50e..4d22a124df11 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -93,7 +93,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
# Run the test
rm -f smoke.serial
-export QEMU_CMD="./binaries/qemu-system-aarch64 \
+export TEST_CMD="./binaries/qemu-system-aarch64 \
-machine virtualization=true \
-cpu cortex-a57 -machine type=virt \
-m 2048 -monitor none -serial stdio \
@@ -104,8 +104,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
-bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
export LOG_MSG="Domain-0"
export PASSED="BusyBox"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh
index 38e8a0b0bd7d..41f6e5d8e615 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh
@@ -130,7 +130,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
# Run the test
rm -f ${serial_log}
-export QEMU_CMD="./qemu-system-arm \
+export TEST_CMD="./qemu-system-arm \
-machine virt \
-machine virtualization=true \
-smp 4 \
@@ -144,8 +144,8 @@ export QEMU_CMD="./qemu-system-arm \
-bios /usr/lib/u-boot/qemu_arm/u-boot.bin"
export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
-export QEMU_LOG="${serial_log}"
+export TEST_LOG="${serial_log}"
export LOG_MSG="${dom0_prompt}"
export PASSED="${passed}"
-../automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+../automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index ea67650e17da..83e1866ca6c5 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -204,7 +204,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
# Run the test
rm -f smoke.serial
-export QEMU_CMD="./binaries/qemu-system-aarch64 \
+export TEST_CMD="./binaries/qemu-system-aarch64 \
-machine virtualization=true \
-cpu cortex-a57 -machine type=virt,gic-version=$gic_version \
-m 2048 -monitor none -serial stdio \
@@ -215,8 +215,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
-bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
export LOG_MSG="Welcome to Alpine Linux"
export PASSED="${passed}"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh
index 49e189c37058..617096ad1fa8 100755
--- a/automation/scripts/qemu-smoke-ppc64le.sh
+++ b/automation/scripts/qemu-smoke-ppc64le.sh
@@ -10,7 +10,7 @@ machine=$1
# Run the test
rm -f ${serial_log}
-export QEMU_CMD="qemu-system-ppc64 \
+export TEST_CMD="qemu-system-ppc64 \
-bios skiboot.lid \
-M $machine \
-m 2g \
@@ -21,7 +21,7 @@ export QEMU_CMD="qemu-system-ppc64 \
-serial stdio \
-kernel binaries/xen"
-export QEMU_LOG="${serial_log}"
+export TEST_LOG="${serial_log}"
export PASSED="Hello, ppc64le!"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh
index 422ee03e0d26..8f755d0a6a99 100755
--- a/automation/scripts/qemu-smoke-riscv64.sh
+++ b/automation/scripts/qemu-smoke-riscv64.sh
@@ -5,14 +5,14 @@ set -ex -o pipefail
# Run the test
rm -f smoke.serial
-export QEMU_CMD="qemu-system-riscv64 \
+export TEST_CMD="qemu-system-riscv64 \
-M virt \
-smp 1 \
-nographic \
-m 2g \
-kernel binaries/xen"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
export PASSED="All set up"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
index 7495185d9fc6..da0c26cc2f82 100755
--- a/automation/scripts/qemu-smoke-x86-64.sh
+++ b/automation/scripts/qemu-smoke-x86-64.sh
@@ -15,12 +15,12 @@ case $variant in
esac
rm -f smoke.serial
-export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
+export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
-initrd xtf/tests/example/$k \
-append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
-m 512 -monitor none -serial stdio"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
export PASSED="Test result: SUCCESS"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
index acef1637e25b..9608de6ec033 100755
--- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
+++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
@@ -50,7 +50,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
# Run the test
rm -f smoke.serial
-export QEMU_CMD="./binaries/qemu-system-aarch64 \
+export TEST_CMD="./binaries/qemu-system-aarch64 \
-machine virtualization=true \
-cpu cortex-a57 -machine type=virt \
-m 2048 -monitor none -serial stdio \
@@ -61,7 +61,7 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
-bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
export PASSED="${passed}"
-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 4559e2b9ee1f..ef6e1361a95c 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -1,8 +1,8 @@
-#!/bin/sh
+#!/usr/bin/env bash
# Run x86_64 dom0 tests on hardware.
-set -ex
+set -ex -o pipefail
fatal() {
echo "$(basename "$0") $*" >&2
@@ -27,7 +27,6 @@ memory = 512
vif = [ "bridge=xenbr0", ]
disk = [ ]
'
-TIMEOUT_SECONDS=200
# Select test variant.
if [ "${TEST}" = "ping" ]; then
@@ -125,20 +124,19 @@ boot
# Power cycle board and collect serial port output.
SERIAL_DEV="/dev/serial/${TEST_BOARD}"
-SERIAL_CMD="cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'"
sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2
sleep 5
sh /scratch/gitlab-runner/${TEST_BOARD}.sh 1
sleep 5
set +e
stty -F ${SERIAL_DEV} 115200
-timeout -k 1 ${TIMEOUT_SECONDS} nohup sh -c "${SERIAL_CMD}"
-sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2
-
-set -e
-if grep -q "${PASS_MSG}" smoke.serial; then
- exit 0
-fi
+# Capture test result and power off board before exiting.
+export PASSED="${PASS_MSG}"
+export TEST_CMD="cat ${SERIAL_DEV}"
+export TEST_LOG="smoke.serial"
-fatal "Test failed"
+./automation/scripts/console.exp | sed 's/\r\+$//'
+TEST_RESULT=$?
+sh "/scratch/gitlab-runner/${TEST_BOARD}.sh" 2
+exit ${TEST_RESULT}
diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
index 18aa07f0a273..b24ad11b8cac 100755
--- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
+++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-set -ex
+set -ex -o pipefail
test_variant=$1
@@ -137,13 +137,14 @@ cd $START
SERIAL_DEV="/dev/serial/zynq"
set +e
stty -F ${SERIAL_DEV} 115200
-timeout -k 1 120 nohup sh -c "cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'"
-# stop the board
-cd /scratch/gitlab-runner
-bash zcu102.sh 2
-cd $START
+# Capture test result and power off board before exiting.
+export PASSED="${passed}"
+export LOG_MSG="Welcome to Alpine Linux"
+export TEST_CMD="cat ${SERIAL_DEV}"
+export TEST_LOG="smoke.serial"
-set -e
-(grep -q "^Welcome to Alpine Linux" smoke.serial && grep -q "${passed}" smoke.serial) || exit 1
-exit 0
+./automation/scripts/console.exp | sed 's/\r\+$//'
+TEST_RESULT=$?
+sh "/scratch/gitlab-runner/zcu102.sh" 2
+exit ${TEST_RESULT}
--
2.25.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 1/2] automation: fix false success in qemu tests
2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira
@ 2024-08-30 0:48 ` Stefano Stabellini
0 siblings, 0 replies; 6+ messages in thread
From: Stefano Stabellini @ 2024-08-30 0:48 UTC (permalink / raw)
To: Victor Lira
Cc: xen-devel, Andrew Cooper, Jan Beulich, Julien Grall,
Stefano Stabellini, Michal Orzel
On Thu, 29 Aug 2024, victorm.lira@amd.com wrote:
> From: Victor Lira <victorm.lira@amd.com>
>
> Fix flaw in qemu-*.sh tests that producess a false success. The following
> lines produces success despite the "expect" script producing nonzero exit
> status:
>
> set +e
> ...
> ./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> (end of file)
>
> The default exit status for a pipeline using "|" operator is that of the
> rightmost command. Fix this by setting the "pipefail" option in the shell,
> and removing "set +e" allowing the expect script to determine the result.
>
> Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Julien Grall <julien@xen.org>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Michal Orzel <michal.orzel@amd.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> automation/scripts/qemu-alpine-x86_64.sh | 3 +--
> automation/scripts/qemu-key.exp | 2 +-
> automation/scripts/qemu-smoke-dom0-arm32.sh | 3 +--
> automation/scripts/qemu-smoke-dom0-arm64.sh | 3 +--
> automation/scripts/qemu-smoke-dom0less-arm32.sh | 3 +--
> automation/scripts/qemu-smoke-dom0less-arm64.sh | 3 +--
> automation/scripts/qemu-smoke-ppc64le.sh | 3 +--
> automation/scripts/qemu-smoke-riscv64.sh | 3 +--
> automation/scripts/qemu-smoke-x86-64.sh | 3 +--
> automation/scripts/qemu-xtf-dom0less-arm64.sh | 3 +--
> 10 files changed, 10 insertions(+), 19 deletions(-)
>
> diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> index 42a89e86b021..93914c41bc24 100755
> --- a/automation/scripts/qemu-alpine-x86_64.sh
> +++ b/automation/scripts/qemu-alpine-x86_64.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> # DomU Busybox
> cd binaries
> @@ -76,7 +76,6 @@ EOF
>
> # Run the test
> rm -f smoke.serial
> -set +e
> export QEMU_CMD="qemu-system-x86_64 \
> -cpu qemu64,+svm \
> -m 2G -smp 2 \
> diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/qemu-key.exp
> index 787f1f08cb96..66c416483146 100755
> --- a/automation/scripts/qemu-key.exp
> +++ b/automation/scripts/qemu-key.exp
> @@ -14,7 +14,7 @@ eval spawn $env(QEMU_CMD)
>
> expect_after {
> -re "(.*)\r" {
> - exp_continue
> + exp_continue -continue_timer
> }
> timeout {send_error "ERROR-Timeout!\n"; exit 1}
> eof {send_error "ERROR-EOF!\n"; exit 1}
> diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
> index fd64b19358ae..7c282eff3a58 100755
> --- a/automation/scripts/qemu-smoke-dom0-arm32.sh
> +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> serial_log="$(pwd)/smoke.serial"
>
> @@ -77,7 +77,6 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
> bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
>
> rm -f ${serial_log}
> -set +e
> export QEMU_CMD="./qemu-system-arm \
> -machine virt \
> -machine virtualization=true \
> diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
> index e0cea742b0a0..81f210f7f50e 100755
> --- a/automation/scripts/qemu-smoke-dom0-arm64.sh
> +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> # DomU Busybox
> cd binaries
> @@ -93,7 +93,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
>
> # Run the test
> rm -f smoke.serial
> -set +e
> export QEMU_CMD="./binaries/qemu-system-aarch64 \
> -machine virtualization=true \
> -cpu cortex-a57 -machine type=virt \
> diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh
> index e824cb7c2a3d..38e8a0b0bd7d 100755
> --- a/automation/scripts/qemu-smoke-dom0less-arm32.sh
> +++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> test_variant=$1
>
> @@ -130,7 +130,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
>
> # Run the test
> rm -f ${serial_log}
> -set +e
> export QEMU_CMD="./qemu-system-arm \
> -machine virt \
> -machine virtualization=true \
> diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
> index f42ba5d196bc..ea67650e17da 100755
> --- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
> +++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> test_variant=$1
>
> @@ -204,7 +204,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
>
> # Run the test
> rm -f smoke.serial
> -set +e
> export QEMU_CMD="./binaries/qemu-system-aarch64 \
> -machine virtualization=true \
> -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \
> diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh
> index 594f92c19cc2..49e189c37058 100755
> --- a/automation/scripts/qemu-smoke-ppc64le.sh
> +++ b/automation/scripts/qemu-smoke-ppc64le.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> serial_log="$(pwd)/smoke.serial"
>
> @@ -9,7 +9,6 @@ machine=$1
>
> # Run the test
> rm -f ${serial_log}
> -set +e
>
> export QEMU_CMD="qemu-system-ppc64 \
> -bios skiboot.lid \
> diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh
> index c2595f657ff3..422ee03e0d26 100755
> --- a/automation/scripts/qemu-smoke-riscv64.sh
> +++ b/automation/scripts/qemu-smoke-riscv64.sh
> @@ -1,10 +1,9 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> # Run the test
> rm -f smoke.serial
> -set +e
>
> export QEMU_CMD="qemu-system-riscv64 \
> -M virt \
> diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
> index 3440b1761db4..7495185d9fc6 100755
> --- a/automation/scripts/qemu-smoke-x86-64.sh
> +++ b/automation/scripts/qemu-smoke-x86-64.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> # variant should be either pv or pvh
> variant=$1
> @@ -15,7 +15,6 @@ case $variant in
> esac
>
> rm -f smoke.serial
> -set +e
> export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
> -initrd xtf/tests/example/$k \
> -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
> diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> index 4042fe50602b..acef1637e25b 100755
> --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
> +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> # Name of the XTF test
> xtf_test=$1
> @@ -50,7 +50,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
>
> # Run the test
> rm -f smoke.serial
> -set +e
> export QEMU_CMD="./binaries/qemu-system-aarch64 \
> -machine virtualization=true \
> -cpu cortex-a57 -machine type=virt \
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] automation: use expect utility in xilinx tests
2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira
@ 2024-08-30 0:48 ` Stefano Stabellini
2024-08-30 12:35 ` Andrew Cooper
1 sibling, 0 replies; 6+ messages in thread
From: Stefano Stabellini @ 2024-08-30 0:48 UTC (permalink / raw)
To: Victor Lira
Cc: xen-devel, Andrew Cooper, Jan Beulich, Julien Grall,
Stefano Stabellini, Michal Orzel
On Thu, 29 Aug 2024, victorm.lira@amd.com wrote:
> From: Victor Lira <victorm.lira@amd.com>
>
> Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty))
> This patch introduced a CI failure due to a timeout in xilinx-x86_64 test.
>
> Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility
> to determine test result and allow early exit from tests.
> Add "expect" to xilinx container environment (dockerfile).
> Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be
> used by both QEMU and hardware tests.
>
> Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Julien Grall <julien@xen.org>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Michal Orzel <michal.orzel@amd.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> .../build/ubuntu/xenial-xilinx.dockerfile | 1 +
> automation/gitlab-ci/test.yaml | 2 ++
> .../scripts/{qemu-key.exp => console.exp} | 8 +++----
> automation/scripts/qemu-alpine-x86_64.sh | 6 ++---
> automation/scripts/qemu-smoke-dom0-arm32.sh | 6 ++---
> automation/scripts/qemu-smoke-dom0-arm64.sh | 6 ++---
> .../scripts/qemu-smoke-dom0less-arm32.sh | 6 ++---
> .../scripts/qemu-smoke-dom0less-arm64.sh | 6 ++---
> automation/scripts/qemu-smoke-ppc64le.sh | 6 ++---
> automation/scripts/qemu-smoke-riscv64.sh | 6 ++---
> automation/scripts/qemu-smoke-x86-64.sh | 6 ++---
> automation/scripts/qemu-xtf-dom0less-arm64.sh | 6 ++---
> .../scripts/xilinx-smoke-dom0-x86_64.sh | 22 +++++++++----------
> .../scripts/xilinx-smoke-dom0less-arm64.sh | 19 ++++++++--------
> 14 files changed, 54 insertions(+), 52 deletions(-)
> rename automation/scripts/{qemu-key.exp => console.exp} (83%)
>
> diff --git a/automation/build/ubuntu/xenial-xilinx.dockerfile b/automation/build/ubuntu/xenial-xilinx.dockerfile
> index f03d62e8bd3f..6107d8b7711f 100644
> --- a/automation/build/ubuntu/xenial-xilinx.dockerfile
> +++ b/automation/build/ubuntu/xenial-xilinx.dockerfile
> @@ -20,6 +20,7 @@ RUN apt-get update && \
> git \
> gzip \
> file \
> + expect \
> && \
> apt-get autoremove -y && \
> apt-get clean && \
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 3b339f387fbe..e3ebe3745994 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -84,6 +84,7 @@
> variables:
> CONTAINER: ubuntu:xenial-xilinx
> LOGFILE: qemu-smoke-xilinx.log
> + TEST_TIMEOUT: 120
> artifacts:
> paths:
> - smoke.serial
> @@ -103,6 +104,7 @@
> LOGFILE: xilinx-smoke-x86_64.log
> XEN_CMD_CONSOLE: "console=com2 com2=115200,8n1,0x2F8,4"
> TEST_BOARD: "crater"
> + TEST_TIMEOUT: 500
> artifacts:
> paths:
> - smoke.serial
> diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/console.exp
> similarity index 83%
> rename from automation/scripts/qemu-key.exp
> rename to automation/scripts/console.exp
> index 66c416483146..f538aa6bd06c 100755
> --- a/automation/scripts/qemu-key.exp
> +++ b/automation/scripts/console.exp
> @@ -1,16 +1,16 @@
> #!/usr/bin/expect -f
>
> -if {[info exists env(QEMU_TIMEOUT)]} {
> - set timeout $env(QEMU_TIMEOUT)
> +if {[info exists env(TEST_TIMEOUT)]} {
> + set timeout $env(TEST_TIMEOUT)
> } else {
> set timeout 1500
> }
>
> -log_file -a $env(QEMU_LOG)
> +log_file -a $env(TEST_LOG)
>
> match_max 10000
>
> -eval spawn $env(QEMU_CMD)
> +eval spawn $env(TEST_CMD)
>
> expect_after {
> -re "(.*)\r" {
> diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> index 93914c41bc24..1ff689b577e3 100755
> --- a/automation/scripts/qemu-alpine-x86_64.sh
> +++ b/automation/scripts/qemu-alpine-x86_64.sh
> @@ -76,7 +76,7 @@ EOF
>
> # Run the test
> rm -f smoke.serial
> -export QEMU_CMD="qemu-system-x86_64 \
> +export TEST_CMD="qemu-system-x86_64 \
> -cpu qemu64,+svm \
> -m 2G -smp 2 \
> -monitor none -serial stdio \
> @@ -84,8 +84,8 @@ export QEMU_CMD="qemu-system-x86_64 \
> -device virtio-net-pci,netdev=n0 \
> -netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0"
>
> -export QEMU_LOG="smoke.serial"
> +export TEST_LOG="smoke.serial"
> export LOG_MSG="Domain-0"
> export PASSED="BusyBox"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh
> index 7c282eff3a58..e1cd83880928 100755
> --- a/automation/scripts/qemu-smoke-dom0-arm32.sh
> +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
> @@ -77,7 +77,7 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git
> bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
>
> rm -f ${serial_log}
> -export QEMU_CMD="./qemu-system-arm \
> +export TEST_CMD="./qemu-system-arm \
> -machine virt \
> -machine virtualization=true \
> -smp 4 \
> @@ -91,8 +91,8 @@ export QEMU_CMD="./qemu-system-arm \
> -bios /usr/lib/u-boot/qemu_arm/u-boot.bin"
>
> export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> -export QEMU_LOG="${serial_log}"
> +export TEST_LOG="${serial_log}"
> export LOG_MSG="Domain-0"
> export PASSED="/ #"
>
> -../automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +../automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh
> index 81f210f7f50e..4d22a124df11 100755
> --- a/automation/scripts/qemu-smoke-dom0-arm64.sh
> +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
> @@ -93,7 +93,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
>
> # Run the test
> rm -f smoke.serial
> -export QEMU_CMD="./binaries/qemu-system-aarch64 \
> +export TEST_CMD="./binaries/qemu-system-aarch64 \
> -machine virtualization=true \
> -cpu cortex-a57 -machine type=virt \
> -m 2048 -monitor none -serial stdio \
> @@ -104,8 +104,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
> -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
>
> export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> -export QEMU_LOG="smoke.serial"
> +export TEST_LOG="smoke.serial"
> export LOG_MSG="Domain-0"
> export PASSED="BusyBox"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh
> index 38e8a0b0bd7d..41f6e5d8e615 100755
> --- a/automation/scripts/qemu-smoke-dom0less-arm32.sh
> +++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh
> @@ -130,7 +130,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config
>
> # Run the test
> rm -f ${serial_log}
> -export QEMU_CMD="./qemu-system-arm \
> +export TEST_CMD="./qemu-system-arm \
> -machine virt \
> -machine virtualization=true \
> -smp 4 \
> @@ -144,8 +144,8 @@ export QEMU_CMD="./qemu-system-arm \
> -bios /usr/lib/u-boot/qemu_arm/u-boot.bin"
>
> export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> -export QEMU_LOG="${serial_log}"
> +export TEST_LOG="${serial_log}"
> export LOG_MSG="${dom0_prompt}"
> export PASSED="${passed}"
>
> -../automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +../automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh
> index ea67650e17da..83e1866ca6c5 100755
> --- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
> +++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
> @@ -204,7 +204,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
>
> # Run the test
> rm -f smoke.serial
> -export QEMU_CMD="./binaries/qemu-system-aarch64 \
> +export TEST_CMD="./binaries/qemu-system-aarch64 \
> -machine virtualization=true \
> -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \
> -m 2048 -monitor none -serial stdio \
> @@ -215,8 +215,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
> -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
>
> export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> -export QEMU_LOG="smoke.serial"
> +export TEST_LOG="smoke.serial"
> export LOG_MSG="Welcome to Alpine Linux"
> export PASSED="${passed}"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh
> index 49e189c37058..617096ad1fa8 100755
> --- a/automation/scripts/qemu-smoke-ppc64le.sh
> +++ b/automation/scripts/qemu-smoke-ppc64le.sh
> @@ -10,7 +10,7 @@ machine=$1
> # Run the test
> rm -f ${serial_log}
>
> -export QEMU_CMD="qemu-system-ppc64 \
> +export TEST_CMD="qemu-system-ppc64 \
> -bios skiboot.lid \
> -M $machine \
> -m 2g \
> @@ -21,7 +21,7 @@ export QEMU_CMD="qemu-system-ppc64 \
> -serial stdio \
> -kernel binaries/xen"
>
> -export QEMU_LOG="${serial_log}"
> +export TEST_LOG="${serial_log}"
> export PASSED="Hello, ppc64le!"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh
> index 422ee03e0d26..8f755d0a6a99 100755
> --- a/automation/scripts/qemu-smoke-riscv64.sh
> +++ b/automation/scripts/qemu-smoke-riscv64.sh
> @@ -5,14 +5,14 @@ set -ex -o pipefail
> # Run the test
> rm -f smoke.serial
>
> -export QEMU_CMD="qemu-system-riscv64 \
> +export TEST_CMD="qemu-system-riscv64 \
> -M virt \
> -smp 1 \
> -nographic \
> -m 2g \
> -kernel binaries/xen"
>
> -export QEMU_LOG="smoke.serial"
> +export TEST_LOG="smoke.serial"
> export PASSED="All set up"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
> index 7495185d9fc6..da0c26cc2f82 100755
> --- a/automation/scripts/qemu-smoke-x86-64.sh
> +++ b/automation/scripts/qemu-smoke-x86-64.sh
> @@ -15,12 +15,12 @@ case $variant in
> esac
>
> rm -f smoke.serial
> -export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
> +export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
> -initrd xtf/tests/example/$k \
> -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \
> -m 512 -monitor none -serial stdio"
>
> -export QEMU_LOG="smoke.serial"
> +export TEST_LOG="smoke.serial"
> export PASSED="Test result: SUCCESS"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> index acef1637e25b..9608de6ec033 100755
> --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
> +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> @@ -50,7 +50,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
>
> # Run the test
> rm -f smoke.serial
> -export QEMU_CMD="./binaries/qemu-system-aarch64 \
> +export TEST_CMD="./binaries/qemu-system-aarch64 \
> -machine virtualization=true \
> -cpu cortex-a57 -machine type=virt \
> -m 2048 -monitor none -serial stdio \
> @@ -61,7 +61,7 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
> -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"
>
> export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"
> -export QEMU_LOG="smoke.serial"
> +export TEST_LOG="smoke.serial"
> export PASSED="${passed}"
>
> -./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> index 4559e2b9ee1f..ef6e1361a95c 100755
> --- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> +++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> @@ -1,8 +1,8 @@
> -#!/bin/sh
> +#!/usr/bin/env bash
>
> # Run x86_64 dom0 tests on hardware.
>
> -set -ex
> +set -ex -o pipefail
>
> fatal() {
> echo "$(basename "$0") $*" >&2
> @@ -27,7 +27,6 @@ memory = 512
> vif = [ "bridge=xenbr0", ]
> disk = [ ]
> '
> -TIMEOUT_SECONDS=200
>
> # Select test variant.
> if [ "${TEST}" = "ping" ]; then
> @@ -125,20 +124,19 @@ boot
>
> # Power cycle board and collect serial port output.
> SERIAL_DEV="/dev/serial/${TEST_BOARD}"
> -SERIAL_CMD="cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'"
> sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2
> sleep 5
> sh /scratch/gitlab-runner/${TEST_BOARD}.sh 1
> sleep 5
> set +e
> stty -F ${SERIAL_DEV} 115200
> -timeout -k 1 ${TIMEOUT_SECONDS} nohup sh -c "${SERIAL_CMD}"
> -sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2
> -
> -set -e
>
> -if grep -q "${PASS_MSG}" smoke.serial; then
> - exit 0
> -fi
> +# Capture test result and power off board before exiting.
> +export PASSED="${PASS_MSG}"
> +export TEST_CMD="cat ${SERIAL_DEV}"
> +export TEST_LOG="smoke.serial"
>
> -fatal "Test failed"
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> +TEST_RESULT=$?
> +sh "/scratch/gitlab-runner/${TEST_BOARD}.sh" 2
> +exit ${TEST_RESULT}
> diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
> index 18aa07f0a273..b24ad11b8cac 100755
> --- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
> +++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -set -ex
> +set -ex -o pipefail
>
> test_variant=$1
>
> @@ -137,13 +137,14 @@ cd $START
> SERIAL_DEV="/dev/serial/zynq"
> set +e
> stty -F ${SERIAL_DEV} 115200
> -timeout -k 1 120 nohup sh -c "cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'"
>
> -# stop the board
> -cd /scratch/gitlab-runner
> -bash zcu102.sh 2
> -cd $START
> +# Capture test result and power off board before exiting.
> +export PASSED="${passed}"
> +export LOG_MSG="Welcome to Alpine Linux"
> +export TEST_CMD="cat ${SERIAL_DEV}"
> +export TEST_LOG="smoke.serial"
>
> -set -e
> -(grep -q "^Welcome to Alpine Linux" smoke.serial && grep -q "${passed}" smoke.serial) || exit 1
> -exit 0
> +./automation/scripts/console.exp | sed 's/\r\+$//'
> +TEST_RESULT=$?
> +sh "/scratch/gitlab-runner/zcu102.sh" 2
> +exit ${TEST_RESULT}
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] automation: use expect utility in xilinx tests
2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira
2024-08-30 0:48 ` Stefano Stabellini
@ 2024-08-30 12:35 ` Andrew Cooper
1 sibling, 0 replies; 6+ messages in thread
From: Andrew Cooper @ 2024-08-30 12:35 UTC (permalink / raw)
To: victorm.lira, xen-devel
Cc: Jan Beulich, Julien Grall, Stefano Stabellini, Michal Orzel
On 29/08/2024 11:34 pm, victorm.lira@amd.com wrote:
> From: Victor Lira <victorm.lira@amd.com>
>
> Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty))
> This patch introduced a CI failure due to a timeout in xilinx-x86_64 test.
>
> Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility
> to determine test result and allow early exit from tests.
> Add "expect" to xilinx container environment (dockerfile).
> Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be
> used by both QEMU and hardware tests.
This is committed now, but just a quick note for the future. "Fixes:"
is a tag, and it normally lives here just above the SoB.
If you need to refer to it in the commit message, as you do here, then
the normal way is "Commit $sha ("$subject") introduced ..." in a regular
sentence.
> Signed-off-by: Victor Lira <victorm.lira@amd.com>
~Andrew
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-08-30 12:35 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-08-29 22:34 [XEN PATCH v2 0/2] automation: test script fixes (pipefail) victorm.lira
2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira
2024-08-30 0:48 ` Stefano Stabellini
2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira
2024-08-30 0:48 ` Stefano Stabellini
2024-08-30 12:35 ` Andrew Cooper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.