dri-devel Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Maxime Ripard <mripard@kernel.org>
To: Daniel Vetter <daniel.vetter@intel.com>, David Airlie <airlied@linux.ie>
Cc: Maxime Ripard <mripard@kernel.org>,
	dri-devel@lists.freedesktop.org,
	Helen Koike <helen.koike@collabora.com>,
	Thomas Zimmermann <tzimmermann@suse.de>
Subject: [PATCH] drm/doc: ci: Require more context for flaky tests
Date: Thu, 19 Oct 2023 11:46:09 +0200	[thread overview]
Message-ID: <20231019094609.251787-1-mripard@kernel.org> (raw)

Flaky tests can be very difficult to reproduce after the facts, which
will make it even harder to ever fix.

Let's document the metadata we agreed on to provide more context to
anyone trying to address these fixes.

Link: https://lore.kernel.org/dri-devel/CAPj87rPbJ1V1-R7WMTHkDat2A4nwSd61Df9mdGH2PR=ZzxaU=Q@mail.gmail.com/
Signed-off-by: Maxime Ripard <mripard@kernel.org>
---
 Documentation/gpu/automated_testing.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
index 469b6fb65c30..2dd0e221c2c3 100644
--- a/Documentation/gpu/automated_testing.rst
+++ b/Documentation/gpu/automated_testing.rst
@@ -67,6 +67,19 @@ Lists the tests that for a given driver on a specific hardware revision are
 known to behave unreliably. These tests won't cause a job to fail regardless of
 the result. They will still be run.
 
+Each new flake entry must be associated with a link to a bug report to
+the author of the affected driver, the board name or Device Tree name of
+the board, the first kernel version affected, and an approximation of
+the failure rate.
+
+They should be provided under the following format::
+
+  # Bug Report: $LORE_OR_PATCHWORK_URL
+  # Board Name: broken-board.dtb
+  # Version: 6.6-rc1
+  # Failure Rate: 100
+  flaky-test
+
 drivers/gpu/drm/ci/${DRIVER_NAME}-${HW_REVISION}-skips.txt
 -----------------------------------------------------------
 
-- 
2.41.0


             reply	other threads:[~2023-10-19  9:46 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-19  9:46 Maxime Ripard [this message]
2023-10-19 10:46 ` [PATCH] drm/doc: ci: Require more context for flaky tests Daniel Vetter
2023-10-19 16:51 ` Helen Koike
2023-10-20  4:33   ` Helen Koike
2023-10-23 15:09     ` Maxime Ripard
2023-10-25 12:47       ` Helen Koike
2023-10-25 14:19         ` Maxime Ripard
2023-10-23 15:05   ` Maxime Ripard
2023-10-26 10:58 ` Maxime Ripard
2023-10-26 11:02   ` Maxime Ripard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231019094609.251787-1-mripard@kernel.org \
    --to=mripard@kernel.org \
    --cc=airlied@linux.ie \
    --cc=daniel.vetter@intel.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=helen.koike@collabora.com \
    --cc=tzimmermann@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox