* [XEN PATCH v2 0/2] automation: test script fixes (pipefail)
@ 2024-08-29 22:34 victorm.lira
2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira
2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira
0 siblings, 2 replies; 6+ messages in thread
From: victorm.lira @ 2024-08-29 22:34 UTC (permalink / raw)
To: xen-devel; +Cc: Victor Lira
From: Victor Lira <victorm.lira@amd.com>
Separated into two patches to improve clarity and updated to use the
"pipefail" function instead of "mkfifo".
Victor Lira (2):
automation: fix qemu test false success
automation: use expect utility in xilinx tests
.../build/ubuntu/xenial-xilinx.dockerfile | 1 +
automation/gitlab-ci/test.yaml | 2 ++
.../scripts/{qemu-key.exp => console.exp} | 10 ++++-----
automation/scripts/qemu-alpine-x86_64.sh | 9 ++++----
automation/scripts/qemu-smoke-dom0-arm32.sh | 9 ++++----
automation/scripts/qemu-smoke-dom0-arm64.sh | 9 ++++----
.../scripts/qemu-smoke-dom0less-arm32.sh | 9 ++++----
.../scripts/qemu-smoke-dom0less-arm64.sh | 9 ++++----
automation/scripts/qemu-smoke-ppc64le.sh | 9 ++++----
automation/scripts/qemu-smoke-riscv64.sh | 9 ++++----
automation/scripts/qemu-smoke-x86-64.sh | 9 ++++----
automation/scripts/qemu-xtf-dom0less-arm64.sh | 9 ++++----
.../scripts/xilinx-smoke-dom0-x86_64.sh | 22 +++++++++----------
.../scripts/xilinx-smoke-dom0less-arm64.sh | 19 ++++++++--------
14 files changed, 64 insertions(+), 71 deletions(-)
rename automation/scripts/{qemu-key.exp => console.exp} (78%)
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v2 1/2] automation: fix false success in qemu tests 2024-08-29 22:34 [XEN PATCH v2 0/2] automation: test script fixes (pipefail) victorm.lira @ 2024-08-29 22:34 ` victorm.lira 2024-08-30 0:48 ` Stefano Stabellini 2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira 1 sibling, 1 reply; 6+ messages in thread From: victorm.lira @ 2024-08-29 22:34 UTC (permalink / raw) To: xen-devel Cc: Victor Lira, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Michal Orzel From: Victor Lira <victorm.lira@amd.com> Fix flaw in qemu-*.sh tests that producess a false success. The following lines produces success despite the "expect" script producing nonzero exit status: set +e ... ./automation/scripts/qemu-key.exp | sed 's/\r\+$//' (end of file) The default exit status for a pipeline using "|" operator is that of the rightmost command. Fix this by setting the "pipefail" option in the shell, and removing "set +e" allowing the expect script to determine the result. Signed-off-by: Victor Lira <victorm.lira@amd.com> --- Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Jan Beulich <jbeulich@suse.com> Cc: Julien Grall <julien@xen.org> Cc: Stefano Stabellini <sstabellini@kernel.org> Cc: Michal Orzel <michal.orzel@amd.com> Cc: xen-devel@lists.xenproject.org --- automation/scripts/qemu-alpine-x86_64.sh | 3 +-- automation/scripts/qemu-key.exp | 2 +- automation/scripts/qemu-smoke-dom0-arm32.sh | 3 +-- automation/scripts/qemu-smoke-dom0-arm64.sh | 3 +-- automation/scripts/qemu-smoke-dom0less-arm32.sh | 3 +-- automation/scripts/qemu-smoke-dom0less-arm64.sh | 3 +-- automation/scripts/qemu-smoke-ppc64le.sh | 3 +-- automation/scripts/qemu-smoke-riscv64.sh | 3 +-- automation/scripts/qemu-smoke-x86-64.sh | 3 +-- automation/scripts/qemu-xtf-dom0less-arm64.sh | 3 +-- 10 files changed, 10 insertions(+), 19 deletions(-) diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh index 42a89e86b021..93914c41bc24 100755 --- a/automation/scripts/qemu-alpine-x86_64.sh +++ b/automation/scripts/qemu-alpine-x86_64.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail # DomU Busybox cd binaries @@ -76,7 +76,6 @@ EOF # Run the test rm -f smoke.serial -set +e export QEMU_CMD="qemu-system-x86_64 \ -cpu qemu64,+svm \ -m 2G -smp 2 \ diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/qemu-key.exp index 787f1f08cb96..66c416483146 100755 --- a/automation/scripts/qemu-key.exp +++ b/automation/scripts/qemu-key.exp @@ -14,7 +14,7 @@ eval spawn $env(QEMU_CMD) expect_after { -re "(.*)\r" { - exp_continue + exp_continue -continue_timer } timeout {send_error "ERROR-Timeout!\n"; exit 1} eof {send_error "ERROR-EOF!\n"; exit 1} diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh index fd64b19358ae..7c282eff3a58 100755 --- a/automation/scripts/qemu-smoke-dom0-arm32.sh +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail serial_log="$(pwd)/smoke.serial" @@ -77,7 +77,6 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config rm -f ${serial_log} -set +e export QEMU_CMD="./qemu-system-arm \ -machine virt \ -machine virtualization=true \ diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh index e0cea742b0a0..81f210f7f50e 100755 --- a/automation/scripts/qemu-smoke-dom0-arm64.sh +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail # DomU Busybox cd binaries @@ -93,7 +93,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf # Run the test rm -f smoke.serial -set +e export QEMU_CMD="./binaries/qemu-system-aarch64 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt \ diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh index e824cb7c2a3d..38e8a0b0bd7d 100755 --- a/automation/scripts/qemu-smoke-dom0less-arm32.sh +++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail test_variant=$1 @@ -130,7 +130,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config # Run the test rm -f ${serial_log} -set +e export QEMU_CMD="./qemu-system-arm \ -machine virt \ -machine virtualization=true \ diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh index f42ba5d196bc..ea67650e17da 100755 --- a/automation/scripts/qemu-smoke-dom0less-arm64.sh +++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail test_variant=$1 @@ -204,7 +204,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf # Run the test rm -f smoke.serial -set +e export QEMU_CMD="./binaries/qemu-system-aarch64 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \ diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh index 594f92c19cc2..49e189c37058 100755 --- a/automation/scripts/qemu-smoke-ppc64le.sh +++ b/automation/scripts/qemu-smoke-ppc64le.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail serial_log="$(pwd)/smoke.serial" @@ -9,7 +9,6 @@ machine=$1 # Run the test rm -f ${serial_log} -set +e export QEMU_CMD="qemu-system-ppc64 \ -bios skiboot.lid \ diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh index c2595f657ff3..422ee03e0d26 100755 --- a/automation/scripts/qemu-smoke-riscv64.sh +++ b/automation/scripts/qemu-smoke-riscv64.sh @@ -1,10 +1,9 @@ #!/bin/bash -set -ex +set -ex -o pipefail # Run the test rm -f smoke.serial -set +e export QEMU_CMD="qemu-system-riscv64 \ -M virt \ diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh index 3440b1761db4..7495185d9fc6 100755 --- a/automation/scripts/qemu-smoke-x86-64.sh +++ b/automation/scripts/qemu-smoke-x86-64.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail # variant should be either pv or pvh variant=$1 @@ -15,7 +15,6 @@ case $variant in esac rm -f smoke.serial -set +e export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \ -initrd xtf/tests/example/$k \ -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \ diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh index 4042fe50602b..acef1637e25b 100755 --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail # Name of the XTF test xtf_test=$1 @@ -50,7 +50,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf # Run the test rm -f smoke.serial -set +e export QEMU_CMD="./binaries/qemu-system-aarch64 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt \ -- 2.25.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 1/2] automation: fix false success in qemu tests 2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira @ 2024-08-30 0:48 ` Stefano Stabellini 0 siblings, 0 replies; 6+ messages in thread From: Stefano Stabellini @ 2024-08-30 0:48 UTC (permalink / raw) To: Victor Lira Cc: xen-devel, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Michal Orzel On Thu, 29 Aug 2024, victorm.lira@amd.com wrote: > From: Victor Lira <victorm.lira@amd.com> > > Fix flaw in qemu-*.sh tests that producess a false success. The following > lines produces success despite the "expect" script producing nonzero exit > status: > > set +e > ... > ./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > (end of file) > > The default exit status for a pipeline using "|" operator is that of the > rightmost command. Fix this by setting the "pipefail" option in the shell, > and removing "set +e" allowing the expect script to determine the result. > > Signed-off-by: Victor Lira <victorm.lira@amd.com> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org> > --- > Cc: Andrew Cooper <andrew.cooper3@citrix.com> > Cc: Jan Beulich <jbeulich@suse.com> > Cc: Julien Grall <julien@xen.org> > Cc: Stefano Stabellini <sstabellini@kernel.org> > Cc: Michal Orzel <michal.orzel@amd.com> > Cc: xen-devel@lists.xenproject.org > --- > automation/scripts/qemu-alpine-x86_64.sh | 3 +-- > automation/scripts/qemu-key.exp | 2 +- > automation/scripts/qemu-smoke-dom0-arm32.sh | 3 +-- > automation/scripts/qemu-smoke-dom0-arm64.sh | 3 +-- > automation/scripts/qemu-smoke-dom0less-arm32.sh | 3 +-- > automation/scripts/qemu-smoke-dom0less-arm64.sh | 3 +-- > automation/scripts/qemu-smoke-ppc64le.sh | 3 +-- > automation/scripts/qemu-smoke-riscv64.sh | 3 +-- > automation/scripts/qemu-smoke-x86-64.sh | 3 +-- > automation/scripts/qemu-xtf-dom0less-arm64.sh | 3 +-- > 10 files changed, 10 insertions(+), 19 deletions(-) > > diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh > index 42a89e86b021..93914c41bc24 100755 > --- a/automation/scripts/qemu-alpine-x86_64.sh > +++ b/automation/scripts/qemu-alpine-x86_64.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > # DomU Busybox > cd binaries > @@ -76,7 +76,6 @@ EOF > > # Run the test > rm -f smoke.serial > -set +e > export QEMU_CMD="qemu-system-x86_64 \ > -cpu qemu64,+svm \ > -m 2G -smp 2 \ > diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/qemu-key.exp > index 787f1f08cb96..66c416483146 100755 > --- a/automation/scripts/qemu-key.exp > +++ b/automation/scripts/qemu-key.exp > @@ -14,7 +14,7 @@ eval spawn $env(QEMU_CMD) > > expect_after { > -re "(.*)\r" { > - exp_continue > + exp_continue -continue_timer > } > timeout {send_error "ERROR-Timeout!\n"; exit 1} > eof {send_error "ERROR-EOF!\n"; exit 1} > diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh > index fd64b19358ae..7c282eff3a58 100755 > --- a/automation/scripts/qemu-smoke-dom0-arm32.sh > +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > serial_log="$(pwd)/smoke.serial" > > @@ -77,7 +77,6 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git > bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config > > rm -f ${serial_log} > -set +e > export QEMU_CMD="./qemu-system-arm \ > -machine virt \ > -machine virtualization=true \ > diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh > index e0cea742b0a0..81f210f7f50e 100755 > --- a/automation/scripts/qemu-smoke-dom0-arm64.sh > +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > # DomU Busybox > cd binaries > @@ -93,7 +93,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf > > # Run the test > rm -f smoke.serial > -set +e > export QEMU_CMD="./binaries/qemu-system-aarch64 \ > -machine virtualization=true \ > -cpu cortex-a57 -machine type=virt \ > diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh > index e824cb7c2a3d..38e8a0b0bd7d 100755 > --- a/automation/scripts/qemu-smoke-dom0less-arm32.sh > +++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > test_variant=$1 > > @@ -130,7 +130,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config > > # Run the test > rm -f ${serial_log} > -set +e > export QEMU_CMD="./qemu-system-arm \ > -machine virt \ > -machine virtualization=true \ > diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh > index f42ba5d196bc..ea67650e17da 100755 > --- a/automation/scripts/qemu-smoke-dom0less-arm64.sh > +++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > test_variant=$1 > > @@ -204,7 +204,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf > > # Run the test > rm -f smoke.serial > -set +e > export QEMU_CMD="./binaries/qemu-system-aarch64 \ > -machine virtualization=true \ > -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \ > diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh > index 594f92c19cc2..49e189c37058 100755 > --- a/automation/scripts/qemu-smoke-ppc64le.sh > +++ b/automation/scripts/qemu-smoke-ppc64le.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > serial_log="$(pwd)/smoke.serial" > > @@ -9,7 +9,6 @@ machine=$1 > > # Run the test > rm -f ${serial_log} > -set +e > > export QEMU_CMD="qemu-system-ppc64 \ > -bios skiboot.lid \ > diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh > index c2595f657ff3..422ee03e0d26 100755 > --- a/automation/scripts/qemu-smoke-riscv64.sh > +++ b/automation/scripts/qemu-smoke-riscv64.sh > @@ -1,10 +1,9 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > # Run the test > rm -f smoke.serial > -set +e > > export QEMU_CMD="qemu-system-riscv64 \ > -M virt \ > diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh > index 3440b1761db4..7495185d9fc6 100755 > --- a/automation/scripts/qemu-smoke-x86-64.sh > +++ b/automation/scripts/qemu-smoke-x86-64.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > # variant should be either pv or pvh > variant=$1 > @@ -15,7 +15,6 @@ case $variant in > esac > > rm -f smoke.serial > -set +e > export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \ > -initrd xtf/tests/example/$k \ > -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \ > diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh > index 4042fe50602b..acef1637e25b 100755 > --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh > +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > # Name of the XTF test > xtf_test=$1 > @@ -50,7 +50,6 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf > > # Run the test > rm -f smoke.serial > -set +e > export QEMU_CMD="./binaries/qemu-system-aarch64 \ > -machine virtualization=true \ > -cpu cortex-a57 -machine type=virt \ > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 2/2] automation: use expect utility in xilinx tests 2024-08-29 22:34 [XEN PATCH v2 0/2] automation: test script fixes (pipefail) victorm.lira 2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira @ 2024-08-29 22:34 ` victorm.lira 2024-08-30 0:48 ` Stefano Stabellini 2024-08-30 12:35 ` Andrew Cooper 1 sibling, 2 replies; 6+ messages in thread From: victorm.lira @ 2024-08-29 22:34 UTC (permalink / raw) To: xen-devel Cc: Victor Lira, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Michal Orzel From: Victor Lira <victorm.lira@amd.com> Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty)) This patch introduced a CI failure due to a timeout in xilinx-x86_64 test. Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility to determine test result and allow early exit from tests. Add "expect" to xilinx container environment (dockerfile). Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be used by both QEMU and hardware tests. Signed-off-by: Victor Lira <victorm.lira@amd.com> --- Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Jan Beulich <jbeulich@suse.com> Cc: Julien Grall <julien@xen.org> Cc: Stefano Stabellini <sstabellini@kernel.org> Cc: Michal Orzel <michal.orzel@amd.com> Cc: xen-devel@lists.xenproject.org --- .../build/ubuntu/xenial-xilinx.dockerfile | 1 + automation/gitlab-ci/test.yaml | 2 ++ .../scripts/{qemu-key.exp => console.exp} | 8 +++---- automation/scripts/qemu-alpine-x86_64.sh | 6 ++--- automation/scripts/qemu-smoke-dom0-arm32.sh | 6 ++--- automation/scripts/qemu-smoke-dom0-arm64.sh | 6 ++--- .../scripts/qemu-smoke-dom0less-arm32.sh | 6 ++--- .../scripts/qemu-smoke-dom0less-arm64.sh | 6 ++--- automation/scripts/qemu-smoke-ppc64le.sh | 6 ++--- automation/scripts/qemu-smoke-riscv64.sh | 6 ++--- automation/scripts/qemu-smoke-x86-64.sh | 6 ++--- automation/scripts/qemu-xtf-dom0less-arm64.sh | 6 ++--- .../scripts/xilinx-smoke-dom0-x86_64.sh | 22 +++++++++---------- .../scripts/xilinx-smoke-dom0less-arm64.sh | 19 ++++++++-------- 14 files changed, 54 insertions(+), 52 deletions(-) rename automation/scripts/{qemu-key.exp => console.exp} (83%) diff --git a/automation/build/ubuntu/xenial-xilinx.dockerfile b/automation/build/ubuntu/xenial-xilinx.dockerfile index f03d62e8bd3f..6107d8b7711f 100644 --- a/automation/build/ubuntu/xenial-xilinx.dockerfile +++ b/automation/build/ubuntu/xenial-xilinx.dockerfile @@ -20,6 +20,7 @@ RUN apt-get update && \ git \ gzip \ file \ + expect \ && \ apt-get autoremove -y && \ apt-get clean && \ diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index 3b339f387fbe..e3ebe3745994 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -84,6 +84,7 @@ variables: CONTAINER: ubuntu:xenial-xilinx LOGFILE: qemu-smoke-xilinx.log + TEST_TIMEOUT: 120 artifacts: paths: - smoke.serial @@ -103,6 +104,7 @@ LOGFILE: xilinx-smoke-x86_64.log XEN_CMD_CONSOLE: "console=com2 com2=115200,8n1,0x2F8,4" TEST_BOARD: "crater" + TEST_TIMEOUT: 500 artifacts: paths: - smoke.serial diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/console.exp similarity index 83% rename from automation/scripts/qemu-key.exp rename to automation/scripts/console.exp index 66c416483146..f538aa6bd06c 100755 --- a/automation/scripts/qemu-key.exp +++ b/automation/scripts/console.exp @@ -1,16 +1,16 @@ #!/usr/bin/expect -f -if {[info exists env(QEMU_TIMEOUT)]} { - set timeout $env(QEMU_TIMEOUT) +if {[info exists env(TEST_TIMEOUT)]} { + set timeout $env(TEST_TIMEOUT) } else { set timeout 1500 } -log_file -a $env(QEMU_LOG) +log_file -a $env(TEST_LOG) match_max 10000 -eval spawn $env(QEMU_CMD) +eval spawn $env(TEST_CMD) expect_after { -re "(.*)\r" { diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh index 93914c41bc24..1ff689b577e3 100755 --- a/automation/scripts/qemu-alpine-x86_64.sh +++ b/automation/scripts/qemu-alpine-x86_64.sh @@ -76,7 +76,7 @@ EOF # Run the test rm -f smoke.serial -export QEMU_CMD="qemu-system-x86_64 \ +export TEST_CMD="qemu-system-x86_64 \ -cpu qemu64,+svm \ -m 2G -smp 2 \ -monitor none -serial stdio \ @@ -84,8 +84,8 @@ export QEMU_CMD="qemu-system-x86_64 \ -device virtio-net-pci,netdev=n0 \ -netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0" -export QEMU_LOG="smoke.serial" +export TEST_LOG="smoke.serial" export LOG_MSG="Domain-0" export PASSED="BusyBox" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh index 7c282eff3a58..e1cd83880928 100755 --- a/automation/scripts/qemu-smoke-dom0-arm32.sh +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh @@ -77,7 +77,7 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config rm -f ${serial_log} -export QEMU_CMD="./qemu-system-arm \ +export TEST_CMD="./qemu-system-arm \ -machine virt \ -machine virtualization=true \ -smp 4 \ @@ -91,8 +91,8 @@ export QEMU_CMD="./qemu-system-arm \ -bios /usr/lib/u-boot/qemu_arm/u-boot.bin" export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" -export QEMU_LOG="${serial_log}" +export TEST_LOG="${serial_log}" export LOG_MSG="Domain-0" export PASSED="/ #" -../automation/scripts/qemu-key.exp | sed 's/\r\+$//' +../automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh index 81f210f7f50e..4d22a124df11 100755 --- a/automation/scripts/qemu-smoke-dom0-arm64.sh +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh @@ -93,7 +93,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf # Run the test rm -f smoke.serial -export QEMU_CMD="./binaries/qemu-system-aarch64 \ +export TEST_CMD="./binaries/qemu-system-aarch64 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt \ -m 2048 -monitor none -serial stdio \ @@ -104,8 +104,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \ -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin" export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" -export QEMU_LOG="smoke.serial" +export TEST_LOG="smoke.serial" export LOG_MSG="Domain-0" export PASSED="BusyBox" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh index 38e8a0b0bd7d..41f6e5d8e615 100755 --- a/automation/scripts/qemu-smoke-dom0less-arm32.sh +++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh @@ -130,7 +130,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config # Run the test rm -f ${serial_log} -export QEMU_CMD="./qemu-system-arm \ +export TEST_CMD="./qemu-system-arm \ -machine virt \ -machine virtualization=true \ -smp 4 \ @@ -144,8 +144,8 @@ export QEMU_CMD="./qemu-system-arm \ -bios /usr/lib/u-boot/qemu_arm/u-boot.bin" export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" -export QEMU_LOG="${serial_log}" +export TEST_LOG="${serial_log}" export LOG_MSG="${dom0_prompt}" export PASSED="${passed}" -../automation/scripts/qemu-key.exp | sed 's/\r\+$//' +../automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh index ea67650e17da..83e1866ca6c5 100755 --- a/automation/scripts/qemu-smoke-dom0less-arm64.sh +++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh @@ -204,7 +204,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf # Run the test rm -f smoke.serial -export QEMU_CMD="./binaries/qemu-system-aarch64 \ +export TEST_CMD="./binaries/qemu-system-aarch64 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \ -m 2048 -monitor none -serial stdio \ @@ -215,8 +215,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \ -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin" export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" -export QEMU_LOG="smoke.serial" +export TEST_LOG="smoke.serial" export LOG_MSG="Welcome to Alpine Linux" export PASSED="${passed}" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh index 49e189c37058..617096ad1fa8 100755 --- a/automation/scripts/qemu-smoke-ppc64le.sh +++ b/automation/scripts/qemu-smoke-ppc64le.sh @@ -10,7 +10,7 @@ machine=$1 # Run the test rm -f ${serial_log} -export QEMU_CMD="qemu-system-ppc64 \ +export TEST_CMD="qemu-system-ppc64 \ -bios skiboot.lid \ -M $machine \ -m 2g \ @@ -21,7 +21,7 @@ export QEMU_CMD="qemu-system-ppc64 \ -serial stdio \ -kernel binaries/xen" -export QEMU_LOG="${serial_log}" +export TEST_LOG="${serial_log}" export PASSED="Hello, ppc64le!" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh index 422ee03e0d26..8f755d0a6a99 100755 --- a/automation/scripts/qemu-smoke-riscv64.sh +++ b/automation/scripts/qemu-smoke-riscv64.sh @@ -5,14 +5,14 @@ set -ex -o pipefail # Run the test rm -f smoke.serial -export QEMU_CMD="qemu-system-riscv64 \ +export TEST_CMD="qemu-system-riscv64 \ -M virt \ -smp 1 \ -nographic \ -m 2g \ -kernel binaries/xen" -export QEMU_LOG="smoke.serial" +export TEST_LOG="smoke.serial" export PASSED="All set up" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh index 7495185d9fc6..da0c26cc2f82 100755 --- a/automation/scripts/qemu-smoke-x86-64.sh +++ b/automation/scripts/qemu-smoke-x86-64.sh @@ -15,12 +15,12 @@ case $variant in esac rm -f smoke.serial -export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \ +export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \ -initrd xtf/tests/example/$k \ -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \ -m 512 -monitor none -serial stdio" -export QEMU_LOG="smoke.serial" +export TEST_LOG="smoke.serial" export PASSED="Test result: SUCCESS" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh index acef1637e25b..9608de6ec033 100755 --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh @@ -50,7 +50,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf # Run the test rm -f smoke.serial -export QEMU_CMD="./binaries/qemu-system-aarch64 \ +export TEST_CMD="./binaries/qemu-system-aarch64 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt \ -m 2048 -monitor none -serial stdio \ @@ -61,7 +61,7 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \ -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin" export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" -export QEMU_LOG="smoke.serial" +export TEST_LOG="smoke.serial" export PASSED="${passed}" -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' +./automation/scripts/console.exp | sed 's/\r\+$//' diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh index 4559e2b9ee1f..ef6e1361a95c 100755 --- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh +++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh @@ -1,8 +1,8 @@ -#!/bin/sh +#!/usr/bin/env bash # Run x86_64 dom0 tests on hardware. -set -ex +set -ex -o pipefail fatal() { echo "$(basename "$0") $*" >&2 @@ -27,7 +27,6 @@ memory = 512 vif = [ "bridge=xenbr0", ] disk = [ ] ' -TIMEOUT_SECONDS=200 # Select test variant. if [ "${TEST}" = "ping" ]; then @@ -125,20 +124,19 @@ boot # Power cycle board and collect serial port output. SERIAL_DEV="/dev/serial/${TEST_BOARD}" -SERIAL_CMD="cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'" sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2 sleep 5 sh /scratch/gitlab-runner/${TEST_BOARD}.sh 1 sleep 5 set +e stty -F ${SERIAL_DEV} 115200 -timeout -k 1 ${TIMEOUT_SECONDS} nohup sh -c "${SERIAL_CMD}" -sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2 - -set -e -if grep -q "${PASS_MSG}" smoke.serial; then - exit 0 -fi +# Capture test result and power off board before exiting. +export PASSED="${PASS_MSG}" +export TEST_CMD="cat ${SERIAL_DEV}" +export TEST_LOG="smoke.serial" -fatal "Test failed" +./automation/scripts/console.exp | sed 's/\r\+$//' +TEST_RESULT=$? +sh "/scratch/gitlab-runner/${TEST_BOARD}.sh" 2 +exit ${TEST_RESULT} diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh index 18aa07f0a273..b24ad11b8cac 100755 --- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh +++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh @@ -1,6 +1,6 @@ #!/bin/bash -set -ex +set -ex -o pipefail test_variant=$1 @@ -137,13 +137,14 @@ cd $START SERIAL_DEV="/dev/serial/zynq" set +e stty -F ${SERIAL_DEV} 115200 -timeout -k 1 120 nohup sh -c "cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'" -# stop the board -cd /scratch/gitlab-runner -bash zcu102.sh 2 -cd $START +# Capture test result and power off board before exiting. +export PASSED="${passed}" +export LOG_MSG="Welcome to Alpine Linux" +export TEST_CMD="cat ${SERIAL_DEV}" +export TEST_LOG="smoke.serial" -set -e -(grep -q "^Welcome to Alpine Linux" smoke.serial && grep -q "${passed}" smoke.serial) || exit 1 -exit 0 +./automation/scripts/console.exp | sed 's/\r\+$//' +TEST_RESULT=$? +sh "/scratch/gitlab-runner/zcu102.sh" 2 +exit ${TEST_RESULT} -- 2.25.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] automation: use expect utility in xilinx tests 2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira @ 2024-08-30 0:48 ` Stefano Stabellini 2024-08-30 12:35 ` Andrew Cooper 1 sibling, 0 replies; 6+ messages in thread From: Stefano Stabellini @ 2024-08-30 0:48 UTC (permalink / raw) To: Victor Lira Cc: xen-devel, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Michal Orzel On Thu, 29 Aug 2024, victorm.lira@amd.com wrote: > From: Victor Lira <victorm.lira@amd.com> > > Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty)) > This patch introduced a CI failure due to a timeout in xilinx-x86_64 test. > > Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility > to determine test result and allow early exit from tests. > Add "expect" to xilinx container environment (dockerfile). > Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be > used by both QEMU and hardware tests. > > Signed-off-by: Victor Lira <victorm.lira@amd.com> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org> > --- > Cc: Andrew Cooper <andrew.cooper3@citrix.com> > Cc: Jan Beulich <jbeulich@suse.com> > Cc: Julien Grall <julien@xen.org> > Cc: Stefano Stabellini <sstabellini@kernel.org> > Cc: Michal Orzel <michal.orzel@amd.com> > Cc: xen-devel@lists.xenproject.org > --- > .../build/ubuntu/xenial-xilinx.dockerfile | 1 + > automation/gitlab-ci/test.yaml | 2 ++ > .../scripts/{qemu-key.exp => console.exp} | 8 +++---- > automation/scripts/qemu-alpine-x86_64.sh | 6 ++--- > automation/scripts/qemu-smoke-dom0-arm32.sh | 6 ++--- > automation/scripts/qemu-smoke-dom0-arm64.sh | 6 ++--- > .../scripts/qemu-smoke-dom0less-arm32.sh | 6 ++--- > .../scripts/qemu-smoke-dom0less-arm64.sh | 6 ++--- > automation/scripts/qemu-smoke-ppc64le.sh | 6 ++--- > automation/scripts/qemu-smoke-riscv64.sh | 6 ++--- > automation/scripts/qemu-smoke-x86-64.sh | 6 ++--- > automation/scripts/qemu-xtf-dom0less-arm64.sh | 6 ++--- > .../scripts/xilinx-smoke-dom0-x86_64.sh | 22 +++++++++---------- > .../scripts/xilinx-smoke-dom0less-arm64.sh | 19 ++++++++-------- > 14 files changed, 54 insertions(+), 52 deletions(-) > rename automation/scripts/{qemu-key.exp => console.exp} (83%) > > diff --git a/automation/build/ubuntu/xenial-xilinx.dockerfile b/automation/build/ubuntu/xenial-xilinx.dockerfile > index f03d62e8bd3f..6107d8b7711f 100644 > --- a/automation/build/ubuntu/xenial-xilinx.dockerfile > +++ b/automation/build/ubuntu/xenial-xilinx.dockerfile > @@ -20,6 +20,7 @@ RUN apt-get update && \ > git \ > gzip \ > file \ > + expect \ > && \ > apt-get autoremove -y && \ > apt-get clean && \ > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml > index 3b339f387fbe..e3ebe3745994 100644 > --- a/automation/gitlab-ci/test.yaml > +++ b/automation/gitlab-ci/test.yaml > @@ -84,6 +84,7 @@ > variables: > CONTAINER: ubuntu:xenial-xilinx > LOGFILE: qemu-smoke-xilinx.log > + TEST_TIMEOUT: 120 > artifacts: > paths: > - smoke.serial > @@ -103,6 +104,7 @@ > LOGFILE: xilinx-smoke-x86_64.log > XEN_CMD_CONSOLE: "console=com2 com2=115200,8n1,0x2F8,4" > TEST_BOARD: "crater" > + TEST_TIMEOUT: 500 > artifacts: > paths: > - smoke.serial > diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/console.exp > similarity index 83% > rename from automation/scripts/qemu-key.exp > rename to automation/scripts/console.exp > index 66c416483146..f538aa6bd06c 100755 > --- a/automation/scripts/qemu-key.exp > +++ b/automation/scripts/console.exp > @@ -1,16 +1,16 @@ > #!/usr/bin/expect -f > > -if {[info exists env(QEMU_TIMEOUT)]} { > - set timeout $env(QEMU_TIMEOUT) > +if {[info exists env(TEST_TIMEOUT)]} { > + set timeout $env(TEST_TIMEOUT) > } else { > set timeout 1500 > } > > -log_file -a $env(QEMU_LOG) > +log_file -a $env(TEST_LOG) > > match_max 10000 > > -eval spawn $env(QEMU_CMD) > +eval spawn $env(TEST_CMD) > > expect_after { > -re "(.*)\r" { > diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh > index 93914c41bc24..1ff689b577e3 100755 > --- a/automation/scripts/qemu-alpine-x86_64.sh > +++ b/automation/scripts/qemu-alpine-x86_64.sh > @@ -76,7 +76,7 @@ EOF > > # Run the test > rm -f smoke.serial > -export QEMU_CMD="qemu-system-x86_64 \ > +export TEST_CMD="qemu-system-x86_64 \ > -cpu qemu64,+svm \ > -m 2G -smp 2 \ > -monitor none -serial stdio \ > @@ -84,8 +84,8 @@ export QEMU_CMD="qemu-system-x86_64 \ > -device virtio-net-pci,netdev=n0 \ > -netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0" > > -export QEMU_LOG="smoke.serial" > +export TEST_LOG="smoke.serial" > export LOG_MSG="Domain-0" > export PASSED="BusyBox" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh b/automation/scripts/qemu-smoke-dom0-arm32.sh > index 7c282eff3a58..e1cd83880928 100755 > --- a/automation/scripts/qemu-smoke-dom0-arm32.sh > +++ b/automation/scripts/qemu-smoke-dom0-arm32.sh > @@ -77,7 +77,7 @@ git clone --depth 1 https://gitlab.com/xen-project/imagebuilder.git > bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config > > rm -f ${serial_log} > -export QEMU_CMD="./qemu-system-arm \ > +export TEST_CMD="./qemu-system-arm \ > -machine virt \ > -machine virtualization=true \ > -smp 4 \ > @@ -91,8 +91,8 @@ export QEMU_CMD="./qemu-system-arm \ > -bios /usr/lib/u-boot/qemu_arm/u-boot.bin" > > export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" > -export QEMU_LOG="${serial_log}" > +export TEST_LOG="${serial_log}" > export LOG_MSG="Domain-0" > export PASSED="/ #" > > -../automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +../automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh b/automation/scripts/qemu-smoke-dom0-arm64.sh > index 81f210f7f50e..4d22a124df11 100755 > --- a/automation/scripts/qemu-smoke-dom0-arm64.sh > +++ b/automation/scripts/qemu-smoke-dom0-arm64.sh > @@ -93,7 +93,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf > > # Run the test > rm -f smoke.serial > -export QEMU_CMD="./binaries/qemu-system-aarch64 \ > +export TEST_CMD="./binaries/qemu-system-aarch64 \ > -machine virtualization=true \ > -cpu cortex-a57 -machine type=virt \ > -m 2048 -monitor none -serial stdio \ > @@ -104,8 +104,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \ > -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin" > > export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" > -export QEMU_LOG="smoke.serial" > +export TEST_LOG="smoke.serial" > export LOG_MSG="Domain-0" > export PASSED="BusyBox" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh b/automation/scripts/qemu-smoke-dom0less-arm32.sh > index 38e8a0b0bd7d..41f6e5d8e615 100755 > --- a/automation/scripts/qemu-smoke-dom0less-arm32.sh > +++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh > @@ -130,7 +130,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config > > # Run the test > rm -f ${serial_log} > -export QEMU_CMD="./qemu-system-arm \ > +export TEST_CMD="./qemu-system-arm \ > -machine virt \ > -machine virtualization=true \ > -smp 4 \ > @@ -144,8 +144,8 @@ export QEMU_CMD="./qemu-system-arm \ > -bios /usr/lib/u-boot/qemu_arm/u-boot.bin" > > export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" > -export QEMU_LOG="${serial_log}" > +export TEST_LOG="${serial_log}" > export LOG_MSG="${dom0_prompt}" > export PASSED="${passed}" > > -../automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +../automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh b/automation/scripts/qemu-smoke-dom0less-arm64.sh > index ea67650e17da..83e1866ca6c5 100755 > --- a/automation/scripts/qemu-smoke-dom0less-arm64.sh > +++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh > @@ -204,7 +204,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf > > # Run the test > rm -f smoke.serial > -export QEMU_CMD="./binaries/qemu-system-aarch64 \ > +export TEST_CMD="./binaries/qemu-system-aarch64 \ > -machine virtualization=true \ > -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \ > -m 2048 -monitor none -serial stdio \ > @@ -215,8 +215,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \ > -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin" > > export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" > -export QEMU_LOG="smoke.serial" > +export TEST_LOG="smoke.serial" > export LOG_MSG="Welcome to Alpine Linux" > export PASSED="${passed}" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-ppc64le.sh b/automation/scripts/qemu-smoke-ppc64le.sh > index 49e189c37058..617096ad1fa8 100755 > --- a/automation/scripts/qemu-smoke-ppc64le.sh > +++ b/automation/scripts/qemu-smoke-ppc64le.sh > @@ -10,7 +10,7 @@ machine=$1 > # Run the test > rm -f ${serial_log} > > -export QEMU_CMD="qemu-system-ppc64 \ > +export TEST_CMD="qemu-system-ppc64 \ > -bios skiboot.lid \ > -M $machine \ > -m 2g \ > @@ -21,7 +21,7 @@ export QEMU_CMD="qemu-system-ppc64 \ > -serial stdio \ > -kernel binaries/xen" > > -export QEMU_LOG="${serial_log}" > +export TEST_LOG="${serial_log}" > export PASSED="Hello, ppc64le!" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-riscv64.sh b/automation/scripts/qemu-smoke-riscv64.sh > index 422ee03e0d26..8f755d0a6a99 100755 > --- a/automation/scripts/qemu-smoke-riscv64.sh > +++ b/automation/scripts/qemu-smoke-riscv64.sh > @@ -5,14 +5,14 @@ set -ex -o pipefail > # Run the test > rm -f smoke.serial > > -export QEMU_CMD="qemu-system-riscv64 \ > +export TEST_CMD="qemu-system-riscv64 \ > -M virt \ > -smp 1 \ > -nographic \ > -m 2g \ > -kernel binaries/xen" > > -export QEMU_LOG="smoke.serial" > +export TEST_LOG="smoke.serial" > export PASSED="All set up" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh > index 7495185d9fc6..da0c26cc2f82 100755 > --- a/automation/scripts/qemu-smoke-x86-64.sh > +++ b/automation/scripts/qemu-smoke-x86-64.sh > @@ -15,12 +15,12 @@ case $variant in > esac > > rm -f smoke.serial > -export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \ > +export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \ > -initrd xtf/tests/example/$k \ > -append \"loglvl=all console=com1 noreboot console_timestamps=boot $extra\" \ > -m 512 -monitor none -serial stdio" > > -export QEMU_LOG="smoke.serial" > +export TEST_LOG="smoke.serial" > export PASSED="Test result: SUCCESS" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh > index acef1637e25b..9608de6ec033 100755 > --- a/automation/scripts/qemu-xtf-dom0less-arm64.sh > +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh > @@ -50,7 +50,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf > > # Run the test > rm -f smoke.serial > -export QEMU_CMD="./binaries/qemu-system-aarch64 \ > +export TEST_CMD="./binaries/qemu-system-aarch64 \ > -machine virtualization=true \ > -cpu cortex-a57 -machine type=virt \ > -m 2048 -monitor none -serial stdio \ > @@ -61,7 +61,7 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \ > -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin" > > export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000" > -export QEMU_LOG="smoke.serial" > +export TEST_LOG="smoke.serial" > export PASSED="${passed}" > > -./automation/scripts/qemu-key.exp | sed 's/\r\+$//' > +./automation/scripts/console.exp | sed 's/\r\+$//' > diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh b/automation/scripts/xilinx-smoke-dom0-x86_64.sh > index 4559e2b9ee1f..ef6e1361a95c 100755 > --- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh > +++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh > @@ -1,8 +1,8 @@ > -#!/bin/sh > +#!/usr/bin/env bash > > # Run x86_64 dom0 tests on hardware. > > -set -ex > +set -ex -o pipefail > > fatal() { > echo "$(basename "$0") $*" >&2 > @@ -27,7 +27,6 @@ memory = 512 > vif = [ "bridge=xenbr0", ] > disk = [ ] > ' > -TIMEOUT_SECONDS=200 > > # Select test variant. > if [ "${TEST}" = "ping" ]; then > @@ -125,20 +124,19 @@ boot > > # Power cycle board and collect serial port output. > SERIAL_DEV="/dev/serial/${TEST_BOARD}" > -SERIAL_CMD="cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'" > sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2 > sleep 5 > sh /scratch/gitlab-runner/${TEST_BOARD}.sh 1 > sleep 5 > set +e > stty -F ${SERIAL_DEV} 115200 > -timeout -k 1 ${TIMEOUT_SECONDS} nohup sh -c "${SERIAL_CMD}" > -sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2 > - > -set -e > > -if grep -q "${PASS_MSG}" smoke.serial; then > - exit 0 > -fi > +# Capture test result and power off board before exiting. > +export PASSED="${PASS_MSG}" > +export TEST_CMD="cat ${SERIAL_DEV}" > +export TEST_LOG="smoke.serial" > > -fatal "Test failed" > +./automation/scripts/console.exp | sed 's/\r\+$//' > +TEST_RESULT=$? > +sh "/scratch/gitlab-runner/${TEST_BOARD}.sh" 2 > +exit ${TEST_RESULT} > diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh b/automation/scripts/xilinx-smoke-dom0less-arm64.sh > index 18aa07f0a273..b24ad11b8cac 100755 > --- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh > +++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh > @@ -1,6 +1,6 @@ > #!/bin/bash > > -set -ex > +set -ex -o pipefail > > test_variant=$1 > > @@ -137,13 +137,14 @@ cd $START > SERIAL_DEV="/dev/serial/zynq" > set +e > stty -F ${SERIAL_DEV} 115200 > -timeout -k 1 120 nohup sh -c "cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'" > > -# stop the board > -cd /scratch/gitlab-runner > -bash zcu102.sh 2 > -cd $START > +# Capture test result and power off board before exiting. > +export PASSED="${passed}" > +export LOG_MSG="Welcome to Alpine Linux" > +export TEST_CMD="cat ${SERIAL_DEV}" > +export TEST_LOG="smoke.serial" > > -set -e > -(grep -q "^Welcome to Alpine Linux" smoke.serial && grep -q "${passed}" smoke.serial) || exit 1 > -exit 0 > +./automation/scripts/console.exp | sed 's/\r\+$//' > +TEST_RESULT=$? > +sh "/scratch/gitlab-runner/zcu102.sh" 2 > +exit ${TEST_RESULT} > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] automation: use expect utility in xilinx tests 2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira 2024-08-30 0:48 ` Stefano Stabellini @ 2024-08-30 12:35 ` Andrew Cooper 1 sibling, 0 replies; 6+ messages in thread From: Andrew Cooper @ 2024-08-30 12:35 UTC (permalink / raw) To: victorm.lira, xen-devel Cc: Jan Beulich, Julien Grall, Stefano Stabellini, Michal Orzel On 29/08/2024 11:34 pm, victorm.lira@amd.com wrote: > From: Victor Lira <victorm.lira@amd.com> > > Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty)) > This patch introduced a CI failure due to a timeout in xilinx-x86_64 test. > > Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility > to determine test result and allow early exit from tests. > Add "expect" to xilinx container environment (dockerfile). > Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be > used by both QEMU and hardware tests. This is committed now, but just a quick note for the future. "Fixes:" is a tag, and it normally lives here just above the SoB. If you need to refer to it in the commit message, as you do here, then the normal way is "Commit $sha ("$subject") introduced ..." in a regular sentence. > Signed-off-by: Victor Lira <victorm.lira@amd.com> ~Andrew ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-08-30 12:35 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-08-29 22:34 [XEN PATCH v2 0/2] automation: test script fixes (pipefail) victorm.lira 2024-08-29 22:34 ` [PATCH v2 1/2] automation: fix false success in qemu tests victorm.lira 2024-08-30 0:48 ` Stefano Stabellini 2024-08-29 22:34 ` [PATCH v2 2/2] automation: use expect utility in xilinx tests victorm.lira 2024-08-30 0:48 ` Stefano Stabellini 2024-08-30 12:35 ` Andrew Cooper
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.