qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Stefan Hajnoczi <stefanha@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@linaro.org>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Stefan Hajnoczi" <stefanha@redhat.com>
Subject: [PATCH] gitlab: remove unreliable avocado CI jobs
Date: Tue, 12 Sep 2023 11:06:11 -0400	[thread overview]
Message-ID: <20230912150611.70676-1-stefanha@redhat.com> (raw)

The avocado-system-alpine, avocado-system-fedora, and
avocado-system-ubuntu jobs are unreliable. I identified them while
looking over CI failures from the past week:
https://gitlab.com/qemu-project/qemu/-/jobs/5058610614
https://gitlab.com/qemu-project/qemu/-/jobs/5058610654
https://gitlab.com/qemu-project/qemu/-/jobs/5030428571

Thomas Huth suggest on IRC today that there may be a legitimate failure
in there:

  th_huth: f4bug, yes, seems like it does not start at all correctly on
  alpine anymore ... and it's broken since ~ 2 weeks already, so if nobody
  noticed this by now, this is worrying

It crept in because the jobs were already unreliable.

I don't know how to interpret the job output, so all I can do is to
propose removing these jobs. A useful CI job has two outcomes: pass or
fail. Timeouts and other in-between states are not useful because they
require constant triaging by someone who understands the details of the
tests and they can occur when run against pull requests that have
nothing to do with the area covered by the test.

Hopefully test owners will be able to identify the root causes and solve
them so that these jobs can stay. In their current state the jobs are
not useful since I cannot cannot tell whether job failures are real or
just intermittent when merging qemu.git pull requests.

If you are a test owner, please take a look.

It is likely that other avocado-system-* CI jobs have similar failures
from time to time, but I'll leave them as long as they are passing.

Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1884
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 .gitlab-ci.d/buildtest.yml | 27 ---------------------------
 1 file changed, 27 deletions(-)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index aee9101507..83ce448c4d 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -22,15 +22,6 @@ check-system-alpine:
     IMAGE: alpine
     MAKE_CHECK_ARGS: check-unit check-qtest
 
-avocado-system-alpine:
-  extends: .avocado_test_job_template
-  needs:
-    - job: build-system-alpine
-      artifacts: true
-  variables:
-    IMAGE: alpine
-    MAKE_CHECK_ARGS: check-avocado
-
 build-system-ubuntu:
   extends:
     - .native_build_job_template
@@ -53,15 +44,6 @@ check-system-ubuntu:
     IMAGE: ubuntu2204
     MAKE_CHECK_ARGS: check
 
-avocado-system-ubuntu:
-  extends: .avocado_test_job_template
-  needs:
-    - job: build-system-ubuntu
-      artifacts: true
-  variables:
-    IMAGE: ubuntu2204
-    MAKE_CHECK_ARGS: check-avocado
-
 build-system-debian:
   extends:
     - .native_build_job_template
@@ -127,15 +109,6 @@ check-system-fedora:
     IMAGE: fedora
     MAKE_CHECK_ARGS: check
 
-avocado-system-fedora:
-  extends: .avocado_test_job_template
-  needs:
-    - job: build-system-fedora
-      artifacts: true
-  variables:
-    IMAGE: fedora
-    MAKE_CHECK_ARGS: check-avocado
-
 crash-test-fedora:
   extends: .native_test_job_template
   needs:
-- 
2.41.0



             reply	other threads:[~2023-09-12 15:07 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-12 15:06 Stefan Hajnoczi [this message]
2023-09-12 15:20 ` [PATCH] gitlab: remove unreliable avocado CI jobs Daniel P. Berrangé
2023-09-12 16:01   ` Alex Bennée
2023-09-12 16:14     ` Daniel P. Berrangé
2023-09-12 16:19       ` Stefan Hajnoczi
2023-09-12 17:39         ` Alex Bennée
2023-09-12 18:52           ` Stefan Hajnoczi
2023-09-12 19:58 ` Thomas Huth
2023-09-13  6:43   ` Philippe Mathieu-Daudé
2023-09-13  9:18   ` Peter Maydell
2023-09-13  9:45     ` Philippe Mathieu-Daudé
2023-09-13 10:35       ` Alex Bennée
2023-09-13 10:39         ` Daniel P. Berrangé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230912150611.70676-1-stefanha@redhat.com \
    --to=stefanha@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=bleal@redhat.com \
    --cc=philmd@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).