git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>,
	Michael Haggerty <mhagger@alum.mit.edu>
Subject: [PATCH 3/3] test-lib.sh: support -x option for shell-tracing
Date: Fri, 10 Oct 2014 02:13:56 -0400	[thread overview]
Message-ID: <20141010061355.GC15277@peff.net> (raw)
In-Reply-To: <20141010060636.GA15057@peff.net>

Usually running a test under "-v" makes it clear which
command is failing. However, sometimes it can be useful to
also see a complete trace of the shell commands being run in
the test. You can do so without any support from the test
suite by running "sh -x tXXXX-foo.sh". However, this
produces quite a large bit of output, as we see a trace of
the entire test suite.

This patch instead introduces a "-x" option to the test
scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this
turns on "set -x" only for the tests themselves. This can
still be a bit verbose, but should keep things to a more
manageable level. You can even use "--verbose-only" to see
the trace only for a specific test.

The implementation is a little invasive. We turn on the "set
-x" inside the "eval" of the test code. This lets the eval
itself avoid being reported in the trace (which would be
long, and redundant with the verbose listing we already
showed). And then after the eval runs, we do some trickery
with stderr to avoid showing the "set +x" to the user.

We also show traces for test_cleanup functions (since they
can impact the test outcome, too). However, we do avoid
running the noop ":" cleanup (the default if the test does
not use test_cleanup at all), as it creates unnecessary
noise in the "set -x" output.

Signed-off-by: Jeff King <peff@peff.net>
---
Having finally figured out how to drop the "set +x" from the output, I
have noticed that I kind of liked the "test_eval_ret=$?" part of the
trace (which is now gone, too), because it pretty explicitly tells you
that the last traced command failed. But now that it has been silenced,
there's no reason we couldn't add back in our own output to make it more
clear.

 t/README      |  4 ++++
 t/test-lib.sh | 34 ++++++++++++++++++++++++++++++----
 2 files changed, 34 insertions(+), 4 deletions(-)

diff --git a/t/README b/t/README
index 52c77ae..38cb078 100644
--- a/t/README
+++ b/t/README
@@ -82,6 +82,10 @@ appropriately before running "make".
 	numbers matching <pattern>.  The number matched against is
 	simply the running count of the test within the file.
 
+-x::
+	Turn on shell tracing (i.e., `set -x`) during the tests
+	themselves. Implies `--verbose`.
+
 -d::
 --debug::
 	This may help the person who is developing a new test.
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 82095e3..a60ec75 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -187,6 +187,8 @@ export _x05 _x40 _z40 LF
 	) &&
 	color=t
 
+test_eval_start_=
+test_eval_end_=
 while test "$#" -ne 0
 do
 	case "$1" in
@@ -233,6 +235,11 @@ do
 	--root=*)
 		root=$(expr "z$1" : 'z[^=]*=\(.*\)')
 		shift ;;
+	-x)
+		test_eval_start_='set -x'
+		test_eval_end_='set +x'
+		verbose=t
+		shift ;;
 	*)
 		echo "error: unknown test option '$1'" >&2; exit 1 ;;
 	esac
@@ -517,10 +524,28 @@ maybe_setup_valgrind () {
 	fi
 }
 
+# This is a separate function because some tests use
+# "return" to end a test_expect_success block early
+# (and we want to make sure we run any $test_eval_end_).
+test_eval_inner_ () {
+	eval "$test_eval_start_ $*"
+}
+
 test_eval_ () {
-	# This is a separate function because some tests use
-	# "return" to end a test_expect_success block early.
-	eval </dev/null >&3 2>&4 "$*"
+	# We run this block with stderr redirected to avoid extra cruft
+	# during a "-x" trace. Once in "set -x" mode, we cannot prevent
+	# the shell from printing the "set +x" to turn it off (nor the saving
+	# of $? before that). But we can make sure that the output goes to
+	# /dev/null.
+	#
+	# The test itself is run with stderr put back to &4 (so either to
+	# /dev/null, or to the original stderr if --verbose was used).
+	{
+		test_eval_inner_ "$@" </dev/null >&3 2>&4
+		test_eval_ret_=$?
+		$test_eval_end_
+	} 2>/dev/null
+	return $test_eval_ret_
 }
 
 test_run_ () {
@@ -531,7 +556,8 @@ test_run_ () {
 	eval_ret=$?
 	teardown_malloc_check
 
-	if test -z "$immediate" || test $eval_ret = 0 || test -n "$expecting_failure"
+	if test -z "$immediate" || test $eval_ret = 0 ||
+	   test -n "$expecting_failure" && test "$test_cleanup" != ":"
 	then
 		setup_malloc_check
 		test_eval_ "$test_cleanup"
-- 
2.1.2.596.g7379948

  parent reply	other threads:[~2014-10-10  6:14 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-10-10  6:06 [PATCH 0/3] "-x" tracing option for tests Jeff King
2014-10-10  6:07 ` [PATCH 1/3] t5304: use test_path_is_* instead of "test -f" Jeff King
2014-10-10  6:11 ` [PATCH 2/3] t5304: use helper to report failure of "test foo = bar" Jeff King
2014-10-13 16:10   ` Jonathan Nieder
2014-10-13 21:15     ` Jeff King
2014-10-13 21:31       ` Jonathan Nieder
2014-10-13 21:33         ` Junio C Hamano
2014-10-13 21:38           ` Jonathan Nieder
2014-10-13 21:56             ` Junio C Hamano
2014-10-13 21:36         ` Jeff King
2014-10-10  6:13 ` Jeff King [this message]
2014-10-10  6:21   ` [PATCH 3/3] test-lib.sh: support -x option for shell-tracing Jeff King
2014-10-10  6:47     ` [PATCH v2 " Jeff King
2014-10-13 18:43       ` Junio C Hamano
2014-10-13 22:22       ` Junio C Hamano
2014-10-13 22:31         ` Junio C Hamano
2014-10-13 22:33         ` Jeff King
2014-10-13 22:38           ` Junio C Hamano
2014-10-13 22:43           ` Jeff King
2014-10-13 23:14             ` Junio C Hamano
2014-10-14  0:46               ` Jeff King
2014-10-10  6:27   ` [PATCH " Jeff King
2014-10-13 18:24 ` [PATCH 0/3] "-x" tracing option for tests Junio C Hamano
2014-10-13 21:07   ` Jeff King
2014-10-14  8:52     ` Michael Haggerty
2014-10-14 13:44       ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141010061355.GC15277@peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=mhagger@alum.mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).