[PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

[PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

[Lea Wiemann]

This is for running external test scripts in other programming
languages that provide continuous output about their tests.  Using
test_expect_success (like "test_expect_success 'description' 'perl
test-script.pl'") doesn't suffice here because test_expect_success
eats stdout in non-verbose mode, which is not fixable without major
file descriptor trickery.

Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>
---
 t/test-lib.sh |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 58 insertions(+), 0 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 7a8bd27..99b63da 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -302,6 +302,64 @@ test_expect_code () {
 	echo >&3 ""
 }
 
+# test_external runs external test scripts that provide continuous
+# test output about their progress, and succeeds/fails on
+# zero/non-zero exit code.  It outputs the test output on stdout even
+# in non-verbose mode, and announces the external script with "* run
+# <n>: ..." before running it.  When providing relative paths, keep in
+# mind that all scripts run in "trash directory".
+# Usage: test_external description command arguments...
+# Example: test_external 'Perl API' perl ../path/to/test.pl
+test_external () {
+	test "$#" -eq 3 ||
+	error >&5 "bug in the test script: not 3 parameters to test_external"
+	descr="$1"
+	shift
+	if ! test_skip "$descr" "$@"
+	then
+		# Announce the script to reduce confusion about the
+		# test output that follows.
+		say_color "" " run $(expr "$test_count" + 1): $descr ($*)"
+		# Run command; redirect its stderr to &4 as in
+		# test_run_, but keep its stdout on our stdout even in
+		# non-verbose mode.
+		"$@" 2>&4
+		if [ "$?" = 0 ]
+		then
+			test_ok_ "$descr"
+		else
+			test_failure_ "$descr" "$@"
+	  	fi
+	fi
+}
+
+# Like test_external, but in addition tests that the command generated
+# no output on stderr.
+test_external_without_stderr () {
+	# The temporary file has no (and must have no) security
+	# implications.
+	tmp="$TMPDIR"; if [ -z "$tmp" ]; then tmp=/tmp; fi
+	stderr="$tmp/git-external-stderr.$$.tmp"
+	test_external "$@" 4> "$stderr"
+	[ -f "$stderr" ] || eror "Internal error: $stderr disappeared."
+	descr="no stderr: $1"
+	shift
+	say >&3 "expecting no stderr from previous command"
+	if [ ! -s "$stderr" ]; then
+		rm "$stderr"
+		test_ok_ "$descr"
+	else
+		if [ "$verbose" = t ]; then	    	
+		    output=`echo; echo Stderr is:; cat "$stderr"`
+		else
+		    output=
+		fi
+		# rm first in case test_failure exits.
+		rm "$stderr"
+		test_failure_ "$descr" "$@" "$output"
+        fi
+}
+
 # This is not among top-level (test_expect_success | test_expect_failure)
 # but is a prefix that can be used in the test script, like:
 #
-- 
1.5.5.GIT

Re: [PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

[Lea Wiemann]

Lea Wiemann wrote:
> Subject: [PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

Yikes, hit send-email too early again. :(  Anyways, I just wanted to say 
that I've tested both functions for both the success and failure case 
(with my first version of a Git.pm test suite, written with Test::More, 
which I'll post later when it's finished).

-- Lea

Re: [PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

[Lea Wiemann]

Lea Wiemann wrote:
> [PATCH] t/test-lib.sh: add test_external and test_external_without_stderr

Please don't merge this patch or the "test suite for Git.pm" patch -- 
I'll rather stay on my branch for now so I can easily rebase and change 
things.  Comments are still welcome, of course.

-- Lea