From: "Alex Bennée" <alex.bennee@linaro.org>
To: qemu-devel@nongnu.org
Cc: fam@euphon.net, berrange@redhat.com, f4bug@amsat.org,
aurelien@aurel32.net, pbonzini@redhat.com, stefanha@redhat.com,
crosa@redhat.com, "Alex Bennée" <alex.bennee@linaro.org>,
"Thomas Huth" <thuth@redhat.com>,
"Yoshinori Sato" <ysato@users.sourceforge.jp>,
"Magnus Damm" <magnus.damm@gmail.com>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Beraldo Leal" <bleal@redhat.com>
Subject: [PATCH v3 20/26] tests/avocado: disable sh4 rd2 tests on Gitlab
Date: Thu, 20 Oct 2022 12:52:03 +0100 [thread overview]
Message-ID: <20221020115209.1761864-21-alex.bennee@linaro.org> (raw)
In-Reply-To: <20221020115209.1761864-1-alex.bennee@linaro.org>
Running repeated invocations on a number of test boxes show a fairly
high error rate:
$ retry.py -n 100 -c -- ./tests/venv/bin/avocado run tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d
retry.py called with ['./tests/venv/bin/avocado', 'run', 'tests/avocado/boot_linux_console.py:BootLinuxConsole.test_sh4_r2d']
Results:
Run, Ret, Pass/Fail, Time, Total Pass, Total Run
...
Results summary:
0: 94 times (94.00%), avg time 2.254 (0.00 varience/0.04 deviation)
1: 3 times (3.00%), avg time 1.837 (0.02 varience/0.14 deviation)
8: 3 times (3.00%), avg time 91.288 (0.02 varience/0.15 deviation)
Examining the logs they fall into various categories of un-handled
unaligned access by user space and unexpected FPU usage by the kernel
which ultimately lead to the failure to reach the login prompt. This
could be bugs in the translator that only get hit occasionally or just
a flaky kernel - its hard to tell. To avoid these failures gating CI
lets skip on GitLab.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Magnus Damm <magnus.damm@gmail.com>
---
tests/avocado/boot_linux_console.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py
index eed4b49e6e..26eaf1a119 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -1175,6 +1175,10 @@ def test_ppc_mac99(self):
self.vm.add_args('-M', 'graphics=off')
self.do_test_advcal_2018('15', tar_hash, 'invaders.elf')
+ # This test has a 6-10% failure rate on various hosts that look
+ # like issues with a buggy kernel. As a result we don't want it
+ # gating releases on Gitlab.
+ @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
def test_sh4_r2d(self):
"""
:avocado: tags=arch:sh4
--
2.34.1
next prev parent reply other threads:[~2022-10-20 13:03 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-20 11:51 [PATCH v3 00/26] testing/next (docker, avocado, s390x, MAINTAINERS) Alex Bennée
2022-10-20 11:51 ` [PATCH v3 01/26] tests/docker: update fedora-win[32|64]-cross with lcitool Alex Bennée
2022-10-20 22:07 ` Richard Henderson
2022-10-20 11:51 ` [PATCH v3 02/26] tests/docker: update test-mingw to run single build Alex Bennée
2022-10-20 11:51 ` [PATCH v3 03/26] tests/docker: Add flex/bison to `debian-all-test` Alex Bennée
2022-10-20 11:54 ` Thomas Huth
2022-10-20 11:51 ` [PATCH v3 04/26] configure: don't enable cross compilers unless in target_list Alex Bennée
2022-10-20 22:10 ` Richard Henderson
2023-01-02 13:24 ` Stefan Weil via
2023-01-02 20:50 ` Alex Bennée
2022-10-20 11:51 ` [PATCH v3 05/26] configure: fix the --enable-static --disable-pie case Alex Bennée
2022-10-20 22:09 ` Richard Henderson
2022-10-20 11:51 ` [PATCH v3 06/26] tests/avocado: extend the timeout for x86_64 tcg tests Alex Bennée
2022-10-20 13:05 ` Thomas Huth
2022-10-20 22:11 ` Richard Henderson
2022-10-20 11:51 ` [PATCH v3 07/26] tests/docker: Add flex/bison to `debian-hexagon-cross` Alex Bennée
2022-10-20 11:55 ` Thomas Huth
2022-10-20 11:51 ` [PATCH v3 08/26] tests/tcg: use regular semihosting for nios2-softmmu Alex Bennée
2022-10-20 22:12 ` Richard Henderson
2022-10-20 11:51 ` [PATCH v3 09/26] semihosting/arm-compat-semi: Avoid using hardcoded /tmp Alex Bennée
2022-10-20 15:53 ` Philippe Mathieu-Daudé
2022-10-20 22:13 ` Richard Henderson
2022-10-20 11:51 ` [PATCH v3 10/26] tcg: " Alex Bennée
2022-10-20 15:54 ` Philippe Mathieu-Daudé
2022-10-20 11:51 ` [PATCH v3 11/26] util/qemu-sockets: Use g_get_tmp_dir() to get the directory for temporary files Alex Bennée
2022-10-20 15:58 ` Philippe Mathieu-Daudé
2022-10-20 11:51 ` [PATCH v3 12/26] block/vvfat: Unify the mkdir() call Alex Bennée
2022-10-20 15:59 ` Philippe Mathieu-Daudé
2022-10-21 8:10 ` Kevin Wolf
2022-10-20 11:51 ` [PATCH v3 13/26] fsdev/virtfs-proxy-helper: Use g_mkdir() Alex Bennée
2022-10-20 15:59 ` Philippe Mathieu-Daudé
2022-10-25 13:37 ` Christian Schoenebeck
2022-10-20 11:51 ` [PATCH v3 14/26] hw/usb: dev-mtp: " Alex Bennée
2022-10-20 16:00 ` Philippe Mathieu-Daudé
2022-10-20 11:51 ` [PATCH v3 15/26] MAINTAINERS: add entries for the key build bits Alex Bennée
2022-10-20 16:01 ` Philippe Mathieu-Daudé
2022-10-27 16:02 ` Thomas Huth
2022-10-20 11:51 ` [PATCH v3 16/26] MAINTAINERS: add features_to_c.sh to gdbstub files Alex Bennée
2022-10-20 22:16 ` Richard Henderson
2022-10-24 13:06 ` Philippe Mathieu-Daudé
2022-10-20 11:52 ` [PATCH v3 17/26] MAINTAINERS: fix-up for check-tcg Makefile changes Alex Bennée
2022-10-20 16:02 ` Philippe Mathieu-Daudé
2022-10-20 22:16 ` Richard Henderson
2022-10-20 11:52 ` [PATCH v3 18/26] tests/avocado: set -machine none for userfwd and vnc tests Alex Bennée
2022-10-20 16:03 ` Philippe Mathieu-Daudé
2022-10-20 11:52 ` [PATCH v3 19/26] tests/avocado: raspi2_initrd: Wait for guest shutdown message before stopping Alex Bennée
2022-10-20 15:48 ` Philippe Mathieu-Daudé
2022-10-27 16:03 ` Thomas Huth
2022-10-20 11:52 ` Alex Bennée [this message]
2022-10-20 15:49 ` [PATCH v3 20/26] tests/avocado: disable sh4 rd2 tests on Gitlab Philippe Mathieu-Daudé
2022-10-20 15:56 ` Alex Bennée
2022-10-21 11:59 ` Alex Bennée
2022-10-20 11:52 ` [PATCH v3 21/26] tests/tcg: re-enable linux-test for sh4 Alex Bennée
2022-10-20 15:50 ` Philippe Mathieu-Daudé
2022-10-20 11:52 ` [PATCH v3 22/26] tests/tcg: re-enable threadcount " Alex Bennée
2022-10-20 16:03 ` Philippe Mathieu-Daudé
2022-10-20 11:52 ` [PATCH v3 23/26] target/s390x: don't use ld_code2 to probe next pc Alex Bennée
2022-10-20 12:06 ` Richard Henderson
2022-10-20 13:22 ` David Hildenbrand
2022-10-21 10:35 ` Ilya Leoshkevich
2022-10-20 11:52 ` [PATCH v3 24/26] target/s390x: don't probe next pc for EXecuted insns Alex Bennée
2022-10-20 12:06 ` Richard Henderson
2022-10-20 13:22 ` David Hildenbrand
2022-10-21 10:37 ` Ilya Leoshkevich
2022-10-20 11:52 ` [PATCH v3 25/26] target/s390x: fake instruction loading when handling 'ex' Alex Bennée
2022-10-20 12:06 ` Richard Henderson
2022-10-20 13:24 ` David Hildenbrand
2022-10-21 10:56 ` Ilya Leoshkevich
2022-10-20 11:52 ` [PATCH v3 26/26] tests/tcg: include CONFIG_PLUGIN in config-host.mak Alex Bennée
2022-10-27 16:11 ` Thomas Huth
2022-10-27 15:06 ` [PATCH v3 00/26] testing/next (docker, avocado, s390x, MAINTAINERS) Bin Meng
2022-10-27 17:16 ` Alex Bennée
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221020115209.1761864-21-alex.bennee@linaro.org \
--to=alex.bennee@linaro.org \
--cc=aurelien@aurel32.net \
--cc=berrange@redhat.com \
--cc=bleal@redhat.com \
--cc=crosa@redhat.com \
--cc=f4bug@amsat.org \
--cc=fam@euphon.net \
--cc=magnus.damm@gmail.com \
--cc=pbonzini@redhat.com \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
--cc=ysato@users.sourceforge.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).