[PATCH 0/3] moving test trash directories

[PATCH 0/3] moving test trash directories

[Jeff King]

This series adds support for making trash directories for the test
scripts outside of the usual hierarchy. The main benefit is improved
performance. By putting the trash directories onto a tmpfs (rather than
SATA disks), my "make -j32 test" time went from around 1m51s to around
1m01s. I can only imagine what improvement it would have for people on
NFS.  :)

It also has the nice side effect that running the test suite doesn't
thrash your disk as much, in case you wanted to do something else while
it was running.

  [1/3] tests: use "$TEST_DIRECTORY" instead of ".."
  [2/3] tests: provide $TRASH_DIRECTORY variable
  [3/3] tests: allow user to specify trash directory location

-Peff

[PATCH 1/3] tests: use "$TEST_DIRECTORY" instead of ".."

[Jeff King]

The $TEST_DIRECTORY variable allows tests to find the
top-level test directory regardless of the current working
directory.

In the past, this has been used to accomodate tests which
change directories, but it is also the first step to being
able to move trash directories outside of the
$TEST_DIRECTORY hierarchy.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t3404-rebase-interactive.sh            |    2 +-
 t/t3411-rebase-preserve-around-merges.sh |    2 +-
 t/t3414-rebase-preserve-onto.sh          |    2 +-
 t/t4020-diff-external.sh                 |    2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index a973628..4cae019 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -10,7 +10,7 @@ that the result still makes sense.
 '
 . ./test-lib.sh
 
-. ../lib-rebase.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
 
 set_fake_editor
 
diff --git a/t/t3411-rebase-preserve-around-merges.sh b/t/t3411-rebase-preserve-around-merges.sh
index 6533505..14a23cd 100755
--- a/t/t3411-rebase-preserve-around-merges.sh
+++ b/t/t3411-rebase-preserve-around-merges.sh
@@ -10,7 +10,7 @@ a merge to before the merge.
 '
 . ./test-lib.sh
 
-. ../lib-rebase.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
 
 set_fake_editor
 
diff --git a/t/t3414-rebase-preserve-onto.sh b/t/t3414-rebase-preserve-onto.sh
index 80019ee..ee0a6cc 100755
--- a/t/t3414-rebase-preserve-onto.sh
+++ b/t/t3414-rebase-preserve-onto.sh
@@ -10,7 +10,7 @@ aren'"'"'t on top of $ONTO, even if they are on top of $UPSTREAM.
 '
 . ./test-lib.sh
 
-. ../lib-rebase.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
 
 # Set up branches like this:
 # A1---B1---E1---F1---G1
diff --git a/t/t4020-diff-external.sh b/t/t4020-diff-external.sh
index 4ea42e0..a7602cf 100755
--- a/t/t4020-diff-external.sh
+++ b/t/t4020-diff-external.sh
@@ -166,7 +166,7 @@ test_expect_success 'diff --cached' '
 	git update-index --assume-unchanged file &&
 	echo second >file &&
 	git diff --cached >actual &&
-	test_cmp ../t4020/diff.NUL actual
+	test_cmp "$TEST_DIRECTORY"/t4020/diff.NUL actual
 '
 
 test_done
-- 
1.6.4.178.g7a987

[PATCH 2/3] tests: provide $TRASH_DIRECTORY variable

[Jeff King]

Most scripts don't care about the absolute path to the trash
directory. The one exception was t4014 script, which pieced
together $TEST_DIRECTORY and $test itself to get an absolute
directory.

Instead, let's provide a $TRASH_DIRECTORY which specifies
the same thing. This keeps the $test variable internal to
test-lib.sh and paves the way for trash directories in other
locations.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t4014-format-patch.sh |    3 +--
 t/test-lib.sh           |    3 ++-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 922a894..531f5b7 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -493,13 +493,12 @@ test_expect_success 'format-patch from a subdirectory (2)' '
 '
 
 test_expect_success 'format-patch from a subdirectory (3)' '
-	here="$TEST_DIRECTORY/$test" &&
 	rm -f 0* &&
 	filename=$(
 		rm -rf sub &&
 		mkdir -p sub/dir &&
 		cd sub/dir &&
-		git format-patch -1 -o "$here"
+		git format-patch -1 -o "$TRASH_DIRECTORY"
 	) &&
 	basename=$(expr "$filename" : ".*/\(.*\)") &&
 	test -f "$basename"
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 5fdc5d9..e6a7bb1 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -645,7 +645,8 @@ fi
 
 # Test repository
 test="trash directory.$(basename "$0" .sh)"
-test ! -z "$debug" || remove_trash="$TEST_DIRECTORY/$test"
+TRASH_DIRECTORY="$TEST_DIRECTORY/$test"
+test ! -z "$debug" || remove_trash=$TRASH_DIRECTORY
 rm -fr "$test" || {
 	GIT_EXIT_OK=t
 	echo >&5 "FATAL: Cannot prepare test area"
-- 
1.6.4.178.g7a987

[PATCH 3/3] tests: allow user to specify trash directory location

[Jeff King]

The tests generate a large amount of I/O activity creating
and destroying repositories and files. We can improve the
time it takes to run the test suite by creating trash
directories on filesystems with better performance
characteristic, even though we may not want the rest of the
git repository on those filesystems (e.g., because they are
not network connected, or because they are temporary
ramdisks).

For example, on a dual processor system:

  $ cd t && time make -j32
  real    1m51.562s
  user    0m59.260s
  sys     1m20.933s

  # /dev/shm is tmpfs
  $ cd t && time make -j32 GIT_TEST_OPTS="--root=/dev/shm"
  real    1m1.484s
  user    0m53.555s
  sys     1m5.264s

We almost halve the wall clock time, and we utilize the
dual processors much better.

Signed-off-by: Jeff King <peff@peff.net>
---
There is a test below for absolute versus relative path in the root
provided.  Do we need some extra magic to make it work on non-Unix
platforms?

 t/test-lib.sh |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index e6a7bb1..a5b8d03 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -114,6 +114,9 @@ do
 		valgrind=t; verbose=t; shift ;;
 	--tee)
 		shift ;; # was handled already
+	--root=*)
+		root=$(expr "z$1" : 'z[^=]*=\(.*\)')
+		shift ;;
 	*)
 		echo "error: unknown test option '$1'" >&2; exit 1 ;;
 	esac
@@ -645,7 +648,11 @@ fi
 
 # Test repository
 test="trash directory.$(basename "$0" .sh)"
-TRASH_DIRECTORY="$TEST_DIRECTORY/$test"
+test -n "$root" && test="$root/$test"
+case "$test" in
+/*) TRASH_DIRECTORY="$test" ;;
+ *) TRASH_DIRECTORY="$TEST_DIRECTORY/$test" ;;
+esac
 test ! -z "$debug" || remove_trash=$TRASH_DIRECTORY
 rm -fr "$test" || {
 	GIT_EXIT_OK=t
-- 
1.6.4.178.g7a987

Re: [PATCH 3/3] tests: allow user to specify trash directory location

[Johannes Sixt]

On Sonntag, 9. August 2009, Jeff King wrote:
> There is a test below for absolute versus relative path in the root
> provided.  Do we need some extra magic to make it work on non-Unix
> platforms?

Not for Windows. Those devolpers who want to use an absolute path can use the 
POSIX path notation:

   GIT_TEST_OPTS=--root=/c/temp/gittests

This works in t4014.

-- Hannes

Re: [PATCH 3/3] tests: allow user to specify trash directory location

[Johannes Schindelin]

Hi,

On Sun, 9 Aug 2009, Johannes Sixt wrote:

> On Sonntag, 9. August 2009, Jeff King wrote:
> > There is a test below for absolute versus relative path in the root
> > provided.  Do we need some extra magic to make it work on non-Unix
> > platforms?
> 
> Not for Windows. Those devolpers who want to use an absolute path can use the 
> POSIX path notation:
> 
>    GIT_TEST_OPTS=--root=/c/temp/gittests
> 
> This works in t4014.

But I am not aware of any reasonable tmpfs equivalent on Windows.  Are 
you?  I would be _very_ interested.

Ciao,
Dscho

[PATCH 4/3] t/Makefile: include config.mak

[Jeff King]

This is useful if you want to specify GIT_TEST_OPTS that you
always use.

Signed-off-by: Jeff King <peff@peff.net>
---
On Sun, Aug 09, 2009 at 04:35:19AM -0400, Jeff King wrote:

>   [1/3] tests: use "$TEST_DIRECTORY" instead of ".."
>   [2/3] tests: provide $TRASH_DIRECTORY variable
>   [3/3] tests: allow user to specify trash directory location

I think a 4/3 like this is nice to let you use --root without extra
typing each time.  And when you do "cd t && make t$WHATEVER.sh
GIT_TEST_OPTS='-v -i'", it will automatically be overridden, which is
probably what you want, so you can inspect the failed tests in t/trash*.

 t/Makefile |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/t/Makefile b/t/Makefile
index bf816fc..bd09390 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -3,6 +3,8 @@
 # Copyright (c) 2005 Junio C Hamano
 #
 
+-include ../config.mak
+
 #GIT_TEST_OPTS=--verbose --debug
 SHELL_PATH ?= $(SHELL)
 TAR ?= $(TAR)
-- 
1.6.4.178.g7a987