From: Adam Spiers <git@adamspiers.org>
To: git list <git@vger.kernel.org>
Subject: [PATCH] Make test output coloring more intuitive
Date: Mon, 17 Sep 2012 12:50:37 +0100 [thread overview]
Message-ID: <1347882637-21766-1-git-send-email-git@adamspiers.org> (raw)
1. Change the color of individual known breakages from bold green to
bold yellow. This seems more appropriate when considering the
universal traffic lights coloring scheme, where green conveys the
impression that everything's OK, and amber that something's not
quite right.
2. Likewise, change the color of the summarized total number of known
breakages from bold red to bold yellow to be less alarmist and more
consistent with the above.
3. Change color of unexpectedly fixed known breakages to bold red. An
unexpectedly passing test indicates that the test is wrong or the
semantics of the code being tested have changed. Either way this
is an error which is arguably as bad as a failing test, and as such
is now counted in the totals too.
The end result of these changes is that:
- red is _only_ used for things which have gone unexpectedly wrong:
test failures, unexpected test passes, and failures with the
framework,
- yellow is _only_ used for known breakages, and
- green is _only_ used for things which have gone to plan and
require no further work to be done.
Signed-off-by: Adam Spiers <git@adamspiers.org>
---
t/t0000-basic.sh | 7 ++++---
t/test-lib.sh | 13 ++++++++-----
2 files changed, 12 insertions(+), 8 deletions(-)
mode change 100644 => 100755 t/test-lib.sh
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index ae6a3f0..4e111b4 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -81,9 +81,10 @@ test_expect_success 'pretend we have fixed a known breakage (run in sub test-lib
./passing-todo.sh >out 2>err &&
! test -s err &&
sed -e 's/^> //' >expect <<-\\EOF &&
- > ok 1 - pretend we have fixed a known breakage # TODO known breakage
- > # fixed 1 known breakage(s)
- > # passed all 1 test(s)
+ > ok 1 - pretend we have fixed a known breakage # TODO known breakage vanished
+ > # fixed 1 known breakage(s); please update test(s)
+ > # still have 1 known breakage(s)
+ > # passed all remaining 0 test(s)
> 1..1
EOF
test_cmp expect out)
diff --git a/t/test-lib.sh b/t/test-lib.sh
old mode 100644
new mode 100755
index f8e3733..9907035
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -183,10 +183,12 @@ then
tput bold; tput setaf 1;; # bold red
skip)
tput bold; tput setaf 2;; # bold green
+ warn)
+ tput bold; tput setaf 3;; # bold yellow
pass)
tput setaf 2;; # green
info)
- tput setaf 3;; # brown
+ tput setaf 3;; # yellow/brown
*)
test -n "$quiet" && return;;
esac
@@ -276,12 +278,13 @@ test_failure_ () {
test_known_broken_ok_ () {
test_fixed=$(($test_fixed+1))
- say_color "" "ok $test_count - $@ # TODO known breakage"
+ test_broken=$(($test_broken+1))
+ say_color error "ok $test_count - $@ # TODO known breakage vanished"
}
test_known_broken_failure_ () {
test_broken=$(($test_broken+1))
- say_color skip "not ok $test_count - $@ # TODO known breakage"
+ say_color warn "not ok $test_count - $@ # TODO known breakage"
}
test_debug () {
@@ -371,11 +374,11 @@ test_done () {
if test "$test_fixed" != 0
then
- say_color pass "# fixed $test_fixed known breakage(s)"
+ say_color error "# fixed $test_fixed known breakage(s); please update test(s)"
fi
if test "$test_broken" != 0
then
- say_color error "# still have $test_broken known breakage(s)"
+ say_color warn "# still have $test_broken known breakage(s)"
msg="remaining $(($test_count-$test_broken)) test(s)"
else
msg="$test_count test(s)"
--
1.7.12.147.g6d168f4
next reply other threads:[~2012-09-17 11:50 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-17 11:50 Adam Spiers [this message]
2012-09-17 20:11 ` [PATCH] Make test output coloring more intuitive Jeff King
2012-09-18 21:21 ` Adam Spiers
2012-09-19 20:02 ` Stefano Lattarini
2012-09-19 20:12 ` Adam Spiers
2012-09-19 20:13 ` Jeff King
2012-09-19 20:24 ` [PATCH v4 3/6] Color skipped tests blue Adam Spiers
2012-09-20 5:48 ` Johannes Sixt
2012-09-20 9:04 ` Adam Spiers
2012-09-20 9:08 ` [PATCH v5 3/3] Color skipped tests bold blue Adam Spiers
2012-09-20 10:08 ` Stefano Lattarini
2012-09-20 16:20 ` Junio C Hamano
2012-09-21 6:13 ` [PATCH v4 3/6] Color skipped tests blue Jeff King
2012-11-11 2:04 ` Adam Spiers
2012-09-17 20:50 ` [PATCH] Make test output coloring more intuitive Junio C Hamano
2012-09-18 21:36 ` Adam Spiers
2012-09-18 21:59 ` Jeff King
2012-09-18 22:14 ` Adam Spiers
2012-09-19 17:15 ` [PATCH v2 0/6] make " Adam Spiers
2012-09-19 17:15 ` [PATCH v2 1/6] Change the color of individual known breakages Adam Spiers
2012-09-19 17:15 ` [PATCH v2 2/6] Make 'not ok $count - $message' consistent with 'ok $count - $message' Adam Spiers
2012-09-19 17:50 ` Jeff King
2012-09-19 23:39 ` Junio C Hamano
2012-09-19 23:45 ` Adam Spiers
2012-09-19 17:15 ` [PATCH v2 3/6] Color skipped tests the same as informational messages Adam Spiers
2012-09-19 17:15 ` [PATCH v2 4/6] Refactor mechanics of testing in a sub test-lib Adam Spiers
2012-09-19 17:56 ` Jeff King
2012-09-19 18:44 ` Adam Spiers
2012-09-19 18:49 ` [PATCH v3 " Adam Spiers
2012-09-19 18:49 ` [PATCH v3 5/6] Test the test framework more thoroughly Adam Spiers
2012-09-19 18:49 ` [PATCH v3 6/6] Treat unexpectedly fixed known breakages more seriously Adam Spiers
2012-09-20 21:13 ` [PATCH v3 4/6] Refactor mechanics of testing in a sub test-lib Junio C Hamano
2012-09-19 19:37 ` [PATCH v2 " Jeff King
2012-09-19 20:15 ` Adam Spiers
2012-09-19 17:15 ` [PATCH v2 5/6] Test the test framework more thoroughly Adam Spiers
2012-09-19 17:15 ` [PATCH v2 6/6] Treat unexpectedly fixed known breakages more seriously Adam Spiers
2012-09-19 18:05 ` [PATCH v2 0/6] make test output coloring more intuitive Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1347882637-21766-1-git-send-email-git@adamspiers.org \
--to=git@adamspiers.org \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).