public inbox for igt-dev@lists.freedesktop.org
 help / color / mirror / Atom feed
* [igt-dev] [PATCH i-g-t 1/2] scripts/verify-blacklist: Script for checking blacklist files
@ 2020-06-12  8:51 Petri Latvala
  2020-06-12  8:51 ` [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh Petri Latvala
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Petri Latvala @ 2020-06-12  8:51 UTC (permalink / raw)
  To: igt-dev; +Cc: Petri Latvala

tests/intel-ci/blacklist*.txt files can collect bitrot unless there's
an easy way to check for lines that are no longer needed due to the
tests being renamed or removed. Therefore, a script just for that.

v2: Use long options for readability, exit with 1 if something found

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> #v1
---
 scripts/verify-blacklist.sh | 52 +++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100755 scripts/verify-blacklist.sh

diff --git a/scripts/verify-blacklist.sh b/scripts/verify-blacklist.sh
new file mode 100755
index 00000000..599711ff
--- /dev/null
+++ b/scripts/verify-blacklist.sh
@@ -0,0 +1,52 @@
+#!/bin/sh
+
+# Verify that all entries in a blacklist file are still valid
+
+usage() {
+    echo "Usage: $0 <path-to-igt-runner> <test-binary-directory> <blacklist-file>"
+    echo
+    echo " path-to-igt-runner: For example build/runner/igt_runner"
+    echo " test-binary-directory: For example build/tests"
+    echo " blacklist-file: For example tests/intel-ci/blacklist.txt"
+    exit 2
+}
+
+if [ $# -ne 3 ]; then
+    usage
+fi
+
+RUNNER="$1"
+BINDIR="$2"
+BLFILE="$3"
+
+if [ ! -x "$RUNNER" ]; then
+    echo "$RUNNER not found"
+    echo
+    usage
+fi
+
+if [ ! -f "$BINDIR/test-list.txt" ]; then
+    echo "$BINDIR doesn't look like a test-binary directory"
+    echo
+    usage
+fi
+
+if [ ! -f "$BLFILE" ]; then
+    echo "$BLFILE not found"
+    echo
+    usage
+fi
+
+STATUS=0
+
+cat "$BLFILE" | while read line; do
+    test=$(echo "$line" | sed 's/#.*//' | tr -d '[:space:]')
+    if [ "$test" = "" ]; then continue; fi
+
+    if ! "$RUNNER" --list-all --include-tests "$test" "$BINDIR" >/dev/null 2>/dev/null; then
+        echo Useless blacklist entry: "$test"
+	STATUS=1
+    fi
+done
+
+exit $STATUS
-- 
2.20.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply related	[flat|nested] 9+ messages in thread
* [igt-dev] [PATCH i-g-t v3 1/2] scripts/verify-blacklist: Script for checking blacklist files
@ 2020-06-15 10:25 Petri Latvala
  2020-06-15 10:25 ` [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh Petri Latvala
  0 siblings, 1 reply; 9+ messages in thread
From: Petri Latvala @ 2020-06-15 10:25 UTC (permalink / raw)
  To: igt-dev; +Cc: Petri Latvala

tests/intel-ci/blacklist*.txt files can collect bitrot unless there's
an easy way to check for lines that are no longer needed due to the
tests being renamed or removed. Therefore, a script just for that.

v2: Use long options for readability, exit with 1 if something found

v3: Verify manually against the list of all tests instead of trying
    with igt_runner's --include-tests

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> #v1
---
 scripts/verify-blacklist.sh | 54 +++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
 create mode 100755 scripts/verify-blacklist.sh

diff --git a/scripts/verify-blacklist.sh b/scripts/verify-blacklist.sh
new file mode 100755
index 00000000..93dca495
--- /dev/null
+++ b/scripts/verify-blacklist.sh
@@ -0,0 +1,54 @@
+#!/bin/sh
+
+# Verify that all entries in a blacklist file are still valid
+
+usage() {
+    echo "Usage: $0 <path-to-igt-runner> <test-binary-directory> <blacklist-file>"
+    echo
+    echo " path-to-igt-runner: For example build/runner/igt_runner"
+    echo " test-binary-directory: For example build/tests"
+    echo " blacklist-file: For example tests/intel-ci/blacklist.txt"
+    exit 2
+}
+
+if [ $# -ne 3 ]; then
+    usage
+fi
+
+RUNNER="$1"
+BINDIR="$2"
+BLFILE="$3"
+
+if [ ! -x "$RUNNER" ]; then
+    echo "$RUNNER not found"
+    echo
+    usage
+fi
+
+if [ ! -f "$BINDIR/test-list.txt" ]; then
+    echo "$BINDIR doesn't look like a test-binary directory"
+    echo
+    usage
+fi
+
+if [ ! -f "$BLFILE" ]; then
+    echo "$BLFILE not found"
+    echo
+    usage
+fi
+
+STATUS=0
+
+TESTLIST="$("$RUNNER" --list-all "$BINDIR")"
+
+cat "$BLFILE" | while read line; do
+    blentry=$(echo "$line" | sed 's/#.*//' | tr -d '[:space:]')
+    if [ "$blentry" = "" ]; then continue; fi
+
+    if ! (echo "$TESTLIST" | grep -Pq "$blentry") >/dev/null 2>/dev/null; then
+	echo Useless blacklist entry: "$blentry"
+	STATUS=1
+    fi
+done
+
+exit $STATUS
-- 
2.20.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply related	[flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-06-15 10:30 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-06-12  8:51 [igt-dev] [PATCH i-g-t 1/2] scripts/verify-blacklist: Script for checking blacklist files Petri Latvala
2020-06-12  8:51 ` [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh Petri Latvala
2020-06-12  8:57   ` Petri Latvala
2020-06-12 12:52     ` Arkadiusz Hiler
2020-06-15  9:44       ` Petri Latvala
2020-06-12  9:33 ` [igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/2] scripts/verify-blacklist: Script for checking blacklist files Patchwork
2020-06-12 11:21 ` [igt-dev] ✗ Fi.CI.IGT: failure " Patchwork
  -- strict thread matches above, loose matches on Subject: below --
2020-06-15 10:25 [igt-dev] [PATCH i-g-t v3 1/2] " Petri Latvala
2020-06-15 10:25 ` [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh Petri Latvala
2020-06-15 10:30   ` Arkadiusz Hiler

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox