From: "Alex Bennée" <alex.bennee@linaro.org>
To: qemu-devel@nongnu.org
Cc: fam@euphon.net, berrange@redhat.com, f4bug@amsat.org,
aurelien@aurel32.net, pbonzini@redhat.com, stefanha@redhat.com,
crosa@redhat.com, "Alex Bennée" <alex.bennee@linaro.org>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Thomas Huth" <thuth@redhat.com>,
"Yoshinori Sato" <ysato@users.sourceforge.jp>,
"Magnus Damm" <magnus.damm@gmail.com>,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Beraldo Leal" <bleal@redhat.com>
Subject: [PATCH v4 20/30] tests/avocado: disable sh4 rd2 tests on Gitlab
Date: Thu, 27 Oct 2022 19:36:26 +0100 [thread overview]
Message-ID: <20221027183637.2772968-21-alex.bennee@linaro.org> (raw)
In-Reply-To: <20221027183637.2772968-1-alex.bennee@linaro.org>
Running repeated invocations on a number of test boxes show a fairly
high error rate:
$ retry.py -n 100 -c -- ./tests/venv/bin/avocado run tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d
retry.py called with ['./tests/venv/bin/avocado', 'run', 'tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d']
Results:
Run, Ret, Pass/Fail, Time, Total Pass, Total Run
...
Results summary:
0: 94 times (94.00%), avg time 2.254 (0.00 varience/0.04 deviation)
1: 3 times (3.00%), avg time 1.837 (0.02 varience/0.14 deviation)
8: 3 times (3.00%), avg time 91.288 (0.02 varience/0.15 deviation)
Examining the logs they fall into various categories of un-handled
unaligned access by user space and unexpected FPU usage by the kernel
which ultimately lead to the failure to reach the login prompt. This
could be bugs in the translator that only get hit occasionally or just
a flaky kernel - its hard to tell. To avoid these failures gating CI
lets skip on GitLab.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Magnus Damm <magnus.damm@gmail.com>
---
v2
- add import for skipIf
---
tests/avocado/boot_linux_console.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index eed4b49e6e..4c9d551f47 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -15,6 +15,7 @@
from avocado import skip
from avocado import skipUnless
+from avocado import skipIf
from avocado_qemu import QemuSystemTest
from avocado_qemu import exec_command
from avocado_qemu import exec_command_and_wait_for_pattern
@@ -1175,6 +1176,10 @@ def test_ppc_mac99(self):
self.vm.add_args('-M', 'graphics=off')
self.do_test_advcal_2018('15', tar_hash, 'invaders.elf')
+ # This test has a 6-10% failure rate on various hosts that look
+ # like issues with a buggy kernel. As a result we don't want it
+ # gating releases on Gitlab.
+ @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
def test_sh4_r2d(self):
"""
:avocado: tags=arch:sh4
--
2.34.1
next prev parent reply other threads:[~2022-10-27 19:17 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-27 18:36 [PATCH v4 00/30] testing/next pre-PR Alex Bennée
2022-10-27 18:36 ` [PATCH v4 01/30] tests/docker: update fedora-win[32|64]-cross with lcitool Alex Bennée
2022-10-27 21:24 ` Richard Henderson
2022-10-27 18:36 ` [PATCH v4 02/30] tests/docker: update test-mingw to run single build Alex Bennée
2022-10-27 18:36 ` [PATCH v4 03/30] tests/docker: Add flex/bison to `debian-all-test` Alex Bennée
2022-10-27 18:36 ` [PATCH v4 04/30] configure: don't enable cross compilers unless in target_list Alex Bennée
2022-10-27 18:36 ` [PATCH v4 05/30] configure: fix the --enable-static --disable-pie case Alex Bennée
2022-10-27 18:36 ` [PATCH v4 06/30] tests/avocado: extend the timeout for x86_64 tcg tests Alex Bennée
2022-10-27 18:36 ` [PATCH v4 07/30] tests/docker: Add flex/bison to `debian-hexagon-cross` Alex Bennée
2022-10-27 18:36 ` [PATCH v4 08/30] tests/tcg: use regular semihosting for nios2-softmmu Alex Bennée
2022-10-27 18:36 ` [PATCH v4 09/30] tests/tcg/nios2: Tweak 10m50-ghrd.ld Alex Bennée
2022-10-27 18:36 ` [PATCH v4 10/30] semihosting/arm-compat-semi: Avoid using hardcoded /tmp Alex Bennée
2022-10-27 18:36 ` [PATCH v4 11/30] tcg: " Alex Bennée
2022-10-27 18:36 ` [PATCH v4 12/30] util/qemu-sockets: Use g_get_tmp_dir() to get the directory for temporary files Alex Bennée
2022-10-27 18:36 ` [PATCH v4 13/30] block/vvfat: Unify the mkdir() call Alex Bennée
2022-10-27 18:36 ` [PATCH v4 14/30] hw/usb: dev-mtp: Use g_mkdir() Alex Bennée
2022-10-27 18:36 ` [PATCH v4 15/30] MAINTAINERS: add entries for the key build bits Alex Bennée
2022-10-27 18:36 ` [PATCH v4 16/30] MAINTAINERS: add features_to_c.sh to gdbstub files Alex Bennée
2022-10-27 18:36 ` [PATCH v4 17/30] MAINTAINERS: fix-up for check-tcg Makefile changes Alex Bennée
2022-10-27 18:36 ` [PATCH v4 18/30] tests/avocado: set -machine none for userfwd and vnc tests Alex Bennée
2022-10-27 18:36 ` [PATCH v4 19/30] tests/avocado: raspi2_initrd: Wait for guest shutdown message before stopping Alex Bennée
2022-10-27 18:36 ` Alex Bennée [this message]
2022-10-27 18:36 ` [PATCH v4 21/30] tests/tcg: re-enable linux-test for sh4 Alex Bennée
2022-10-27 18:36 ` [PATCH v4 22/30] tests/tcg: re-enable threadcount " Alex Bennée
2022-10-27 18:36 ` [PATCH v4 23/30] target/s390x: don't use ld_code2 to probe next pc Alex Bennée
2022-10-27 18:36 ` [PATCH v4 24/30] target/s390x: don't probe next pc for EXecuted insns Alex Bennée
2022-10-27 18:36 ` [PATCH v4 25/30] target/s390x: fake instruction loading when handling 'ex' Alex Bennée
2022-10-27 18:36 ` [PATCH v4 26/30] tests/tcg: include CONFIG_PLUGIN in config-host.mak Alex Bennée
2022-10-27 18:36 ` [PATCH v4 27/30] contrib/plugins: enable debug on CONFIG_DEBUG_TCG Alex Bennée
2022-10-27 18:36 ` [PATCH v4 28/30] contrib/plugins: protect execlog's last_exec expansion Alex Bennée
2022-10-27 21:21 ` Richard Henderson
2022-10-27 18:36 ` [PATCH v4 29/30] tests/unit: cleanups for test-io-channel-command Alex Bennée
2022-10-31 10:24 ` Daniel P. Berrangé
2022-10-27 18:36 ` [PATCH v4 30/30] tests/vm: use -o IdentitiesOnly=yes for ssh Alex Bennée
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221027183637.2772968-21-alex.bennee@linaro.org \
--to=alex.bennee@linaro.org \
--cc=aurelien@aurel32.net \
--cc=berrange@redhat.com \
--cc=bleal@redhat.com \
--cc=crosa@redhat.com \
--cc=f4bug@amsat.org \
--cc=fam@euphon.net \
--cc=magnus.damm@gmail.com \
--cc=pbonzini@redhat.com \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
--cc=ysato@users.sourceforge.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).