From: "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Johannes Schindelin <johannes.schindelin@gmx.de>,
Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: [PATCH 7/9] ci: optionally mark up output in the GitHub workflow
Date: Mon, 24 Jan 2022 18:56:12 +0000 [thread overview]
Message-ID: <98b32630fcd76d591ce64f19314f3081134f541e.1643050574.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1117.git.1643050574.gitgitgadget@gmail.com>
From: Johannes Schindelin <johannes.schindelin@gmx.de>
A couple of commands exist to spruce up the output in GitHub workflows:
https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions
In addition to the `::group::<label>`/`::endgroup::` commands (which we
already use to structure the output of the build step better), we also
use `::error::`/`::notice::` to draw the attention to test failures and
to test cases that were expected to fail but didn't.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
t/test-lib-functions.sh | 4 +--
t/test-lib-github-workflow-markup.sh | 50 ++++++++++++++++++++++++++++
t/test-lib.sh | 5 ++-
3 files changed, 56 insertions(+), 3 deletions(-)
create mode 100644 t/test-lib-github-workflow-markup.sh
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index c3d38aaccbd..b5fe5f66085 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -719,7 +719,7 @@ test_verify_prereq () {
}
test_expect_failure () {
- test_start_
+ test_start_ "$@"
test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
test "$#" = 2 ||
BUG "not 2 or 3 parameters to test-expect-failure"
@@ -739,7 +739,7 @@ test_expect_failure () {
}
test_expect_success () {
- test_start_
+ test_start_ "$@"
test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
test "$#" = 2 ||
BUG "not 2 or 3 parameters to test-expect-success"
diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
new file mode 100644
index 00000000000..d8dc969df4a
--- /dev/null
+++ b/t/test-lib-github-workflow-markup.sh
@@ -0,0 +1,50 @@
+# Library of functions to mark up test scripts' output suitable for
+# pretty-printing it in GitHub workflows.
+#
+# Copyright (c) 2022 Johannes Schindelin
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see http://www.gnu.org/licenses/ .
+#
+# The idea is for `test-lib.sh` to source this file when run in GitHub
+# workflows; these functions will then override (empty) functions
+# that are are called at the appropriate times during the test runs.
+
+start_test_output () {
+ test -n "$GIT_TEST_TEE_OUTPUT_FILE" ||
+ die "--github-workflow-markup requires --verbose-log"
+ github_markup_output="${GIT_TEST_TEE_OUTPUT_FILE%.out}.markup"
+ >$github_markup_output
+ GIT_TEST_TEE_OFFSET=0
+}
+
+# No need to override start_test_case_output
+
+finalize_test_case_output () {
+ test_case_result=$1
+ shift
+ case "$test_case_result" in
+ failure)
+ echo >>$github_markup_output "::error::failed: $this_test.$test_count $1"
+ ;;
+ fixed)
+ echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
+ ;;
+ esac
+ echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
+ test-tool >>$github_markup_output path-utils skip-n-bytes \
+ "$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
+ echo >>$github_markup_output "::endgroup::"
+}
+
+# No need to override finalize_test_output
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e13e1cb9124..076bee58c19 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -174,6 +174,9 @@ parse_option () {
--write-junit-xml)
. "$TEST_DIRECTORY/test-lib-junit.sh"
;;
+ --github-workflow-markup)
+ . "$TEST_DIRECTORY/test-lib-github-workflow-markup.sh"
+ ;;
--stress)
stress=t ;;
--stress=*)
@@ -1027,7 +1030,7 @@ test_start_ () {
test_count=$(($test_count+1))
maybe_setup_verbose
maybe_setup_valgrind
- start_test_case_output
+ start_test_case_output "$@"
}
test_finish_ () {
--
gitgitgadget
next prev parent reply other threads:[~2022-01-24 19:02 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-24 18:56 [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-01-24 23:22 ` Eric Sunshine
2022-01-25 14:34 ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-25 23:48 ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-02-23 12:13 ` Phillip Wood
2022-02-25 13:40 ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-01-26 0:10 ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` Johannes Schindelin via GitGitGadget [this message]
2022-01-24 18:56 ` [PATCH 8/9] ci: use `--github-workflow-markup` in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-01-26 0:25 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Ævar Arnfjörð Bjarmason
2022-01-27 16:31 ` CI "grouping" within jobs v.s. lighter split-out jobs (was: [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful) Ævar Arnfjörð Bjarmason
2022-02-19 23:46 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin
2022-02-20 2:44 ` Junio C Hamano
2022-02-20 15:25 ` Johannes Schindelin
2022-02-21 8:09 ` Ævar Arnfjörð Bjarmason
2022-02-22 10:26 ` Johannes Schindelin
2022-02-20 12:47 ` Ævar Arnfjörð Bjarmason
2022-02-22 10:30 ` Johannes Schindelin
2022-02-22 13:31 ` Ævar Arnfjörð Bjarmason
2022-02-23 12:07 ` Phillip Wood
2022-02-25 12:39 ` Ævar Arnfjörð Bjarmason
2022-02-25 14:10 ` Johannes Schindelin
2022-02-25 18:16 ` Junio C Hamano
2022-02-26 18:43 ` Junio C Hamano
2022-03-01 2:59 ` Junio C Hamano
2022-03-01 6:35 ` Junio C Hamano
2022-03-01 10:18 ` Johannes Schindelin
2022-03-01 16:52 ` Junio C Hamano
2022-03-01 10:10 ` Johannes Schindelin
2022-03-01 16:57 ` Junio C Hamano
2022-03-01 10:20 ` Johannes Schindelin
2022-03-04 7:38 ` win+VS environment has "cut" but not "paste"? Junio C Hamano
2022-03-04 9:04 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:51 ` Johannes Schindelin
2022-03-07 17:05 ` Junio C Hamano
2022-03-09 13:02 ` Johannes Schindelin
2022-03-10 15:23 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:48 ` Johannes Schindelin
2022-03-07 16:58 ` Junio C Hamano
2022-03-02 10:58 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Phillip Wood
2022-03-07 16:07 ` Johannes Schindelin
2022-03-07 17:11 ` Junio C Hamano
2022-03-09 11:44 ` Ævar Arnfjörð Bjarmason
2022-03-07 17:12 ` Phillip Wood
2022-03-01 10:24 ` [PATCH v2 " Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-03-01 19:07 ` [PATCH v2 0/9] ci: make Git's GitHub workflow output much more helpful Junio C Hamano
2022-03-02 12:22 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:57 ` Johannes Schindelin
2022-03-07 16:05 ` Ævar Arnfjörð Bjarmason
2022-03-07 17:36 ` Junio C Hamano
2022-03-09 10:56 ` Ævar Arnfjörð Bjarmason
2022-03-09 13:20 ` Johannes Schindelin
2022-03-09 19:39 ` Junio C Hamano
2022-03-09 19:47 ` Ævar Arnfjörð Bjarmason
2022-03-25 0:48 ` Victoria Dye
2022-03-25 9:02 ` Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` Victoria Dye
2022-05-21 21:42 ` Johannes Schindelin
2022-05-21 23:05 ` Junio C Hamano
2022-05-22 18:48 ` Johannes Schindelin
2022-05-22 19:10 ` Junio C Hamano
2022-05-23 12:58 ` Johannes Schindelin
2022-05-22 23:27 ` Junio C Hamano
2022-05-23 18:55 ` Junio C Hamano
2022-05-23 19:21 ` Johannes Schindelin
2022-05-23 9:05 ` Ævar Arnfjörð Bjarmason
2022-05-23 18:41 ` Johannes Schindelin
2022-05-24 8:40 ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18 ` [PATCH v3 00/12] " Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 01/12] ci: fix code style Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 02/12] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 03/12] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 04/12] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 05/12] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 06/12] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 07/12] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 08/12] ci(github): skip the logs of the successful test cases Johannes Schindelin via GitGitGadget
2022-05-24 10:47 ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18 ` [PATCH v3 09/12] ci(github): avoid printing test case preamble twice Victoria Dye via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 10/12] ci: use `--github-workflow-markup` in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 11/12] ci(github): mention where the full logs can be found Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 12/12] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=98b32630fcd76d591ce64f19314f3081134f541e.1643050574.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=git@vger.kernel.org \
--cc=johannes.schindelin@gmx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.